80
`
`10 October 1970
`Computer-held Clinical Record System-II, Assessment
`L. J. OPIT,* B.SC., F.R.C.S., F.R.A.C.S.; F. J. WOODROFFE,t M.B., B.S., M.R.C.P.
`
`B* ITISH
`MEDICAL JOURNAL
`
`the number of times users logged in to the system. Each "conver-
`sation" can be patient registration or deletions, messages input
`about current patients, or interrogation. A "message" as explained
`in the previous paper is a syntactically complete string of numbers
`and letters (characters) and can vary from something like "PULSE
`76/MIN" to a full screen of descriptive text.
`"V.D.U. responses" represents the number of times displays
`were sent to the visual display unit screen. It is possible by using
`this information to estimate the time involved in message construc-
`tion.
`"Total V.D.U. time" is the sum of the time periods that the
`four visual display units were being used to record or receive
`messages.
`"Narrative messages" are messages recorded by using the visual
`display unit keyboard essentially as an ordinary typewriter rather
`than using the display system.
`"Interrogation entries" signifies the number of times that the
`visual display units were used to inspect a patient's record in the
`computer.
`total number of
`interrogation screens"
`is
`the
`"Number of
`messages looked at during the interrogation entries.
`
`One could interpret the data in Table II as follows:
`three
`of
`recorded
`staff
`nursing
`The
`average
`an
`messages/patient admitted. The medical staff recorded an average
`of about five messages/patient admitted. Of all medical messages,
`except week 1, 30-400, were written using the "narrative" facility
`and bypassing the display system. Each "message" required about
`12-13 visual display unit responses. If one allows 5 seconds per
`response this means that a message takes about 1 minute to
`record, though the response time, of course, varies considerably.
`Many will take longer than 5 seconds, for this includes the time
`taken to read, decide, press the keyboard, and allow the system to
`produce the next display. Clearly, narrative messages will take
`even longer.
`We have observed that an "acceptable" medical history and
`examination requires some 25-40 messages, entailing about 400 vis-
`ual display unit responses. In addition, an allowance for 10 "follow
`up" messages per patient per week would not represent excessive
`progress notes for a teaching hospital medical record. Assuming
`that these figures indicate proper but by no means extensive usage
`of the computer system to record clinical notes, one could expect
`on the basis of 38 patients and about 20 new admissions.per week
`to record 900-1,000 medical messages per week. This would corre-
`spond to a steady state level of about 1,500 medical messages at
`any one time.
`The figures in Table II indicate how far short of this expecta-
`tion the usage of the system has been. This is reinforced by Table
`III, which is an analysis of the records of patients discharged dur-
`ing a one-week period and where each row shows the messages
`contained in a patient's record classified by type of "author."
`TABLE I
`
`Weeks since start
`
`No. of messages current
`
`..
`
`0
`
`0
`
`3
`
`622
`
`6
`
`923
`
`9
`
`617
`
`12
`
`434
`
`426*
`
`*Detailed usage shows 16,000 words stored, an average of 1,600 characters per
`patient.
`TABLE II
`
`00
`
`z U
`
`80
`78
`83
`57
`
`U
`
`98
`180
`202
`170
`
`U
`
`:
`
`24
`111
`152
`126
`
`co~I U
`
`.1
`
`Z~
`
`Z:~ z<
`
`711
`69
`
`50
`44
`
`1
`
`42
`46
`77
`
`28
`21
`21
`18
`
`H .
`
`_
`
`i;s
`
`Week
`
`o
`
`00m c)
`OX(1
`
`1,514
`2,214
`2,228
`2,209
`
`4 hr. 9 min. (25" )
`6 hr. 16 min. (3-0o0)
`8 hr. 32 min. (4",,)
`7 hr. 2 min. (3 5 %)
`
`..
`
`I
`
`234
`
`*Percentage of time available in parentheses.
`
`British Medical Journal, 1970, 4, 80-92
`
`Summary: A real-time medical recording system installed-
`in two general medical wards has proved to be
`unacceptable to many of the ward staff. Reasons for this
`include operational problems, such as the impossibility of
`providing a 24-hour service, and conceptual problems,
`such as the difficulty of adapting the method for record-
`is suggested that an outpatient
`It
`ing case histories.
`department might have been a better site for this trial,
`in medical recording
`instruction
`deliberate
`and that
`should be given to students as a prerequisite to successful
`computer record keeping.
`
`Introduction
`In the previous paper we presented some details of phase 1 of
`King's College Hospital real-time medical recording project.
`Essentially this project provides a method for ward staff to
`record clinical notes in a computer medium using visual
`display units and allows interrogation of the computer-stored
`record via this same terminal. In addition, daily printed notes
`are produced for each patient and a ward nursing sheet is
`prepared by the computer from nursing treatment details
`entered in the patient's computer record. This system has
`been in operation in two medical wards for four months, and
`this paper attempts to assess some aspects of the project and
`its implementation to date.
`
`Assessment
`The two tests by which one might measure the performance
`of this computer project are (1) acceptability to medical and
`nursing staff, and (2) the attainment of objectives defined at
`the outset of the project. Obviously failure in (1) will almost
`certainly imply failure to meet any objective defined in (2),
`first. We must
`and we propose to consider this
`aspect
`conclude that, to date, the computer has not been accepted as
`a method of clinical recording. This is shown objectively by
`Table I. This shows the steady-state number of "messages"
`held in the computer about the current 38 inpatients. This
`count of held "messages" is a direct measure of the use of the
`system by the ward staff. Even with two specially appointed
`senior house officers to provide advice, example, and en-
`couragement there has been a decline in the number of
`"messages" held in the computer. The nursing "messages" are
`recorded by ward nursing staff and patient registration is
`a ward clerk. These two
`responsibility
`of
`usually
`the
`categories of message account for about 150-250 of the total
`numbers of messages.
`A more detailed analysis of the use of the computer system
`is possible by use of specially allocated files and programmes
`which monitor the number and nature of user responses as
`well as recording the amount of "traffic." Some of these data
`from a period starting about 10 weeks after the ward installation
`had taken place are presented in tabular form. Before making
`any deductions about these data it is necessary to define the
`terms used in the table headings.
`essentially a variable set of "messages"
`A "conversation" is
`which have been recorded or interrogated at one sitting by one
`computer user and may refer to one or several patients. It signifies
`* Senior Lecturer, King's College Hospital Computer Unit, London S.E.5.
`t Consultant Physician to the Forest Group of Hospitals. Present Appoint-
`Medicine,
`ment: Senior Research Fellow, Department of Social
`University of Birmingham.
`
`SKYHAWKE Ex. 1024, page 1
`
`

`

`10 October 1970 Computer Clinical Record System: Assessment-Opit and Woodroffe MEDItAL JOURMAH
`81
`recorded stepwise, with little need for translation by the
`doctor to make it suitable for the display system.
`
`TABLE I I I
`
`Days of Stay Doctor Messages Nurse Messages
`I-
`I-
`I-
`18
`46
`62
`17
`23
`49
`
`633625 0
`
`i 1
`
`3 048
`
`3 0
`
`31
`
`Case No.
`
`23456789
`
`10
`11
`12
`13
`
`1234
`
`Week
`
`History Display System
`Less satisfaction was encountered with the technique when
`it was used to record the history. Three main difficulties
`were evident.
`(1) The history record has a more ill-defined structure than the
`examination. The experienced clinician tends to take most of the
`history in one continuous interview, then condenses and rearranges
`the information before recording it. The more inexperienced house
`staff usually take and record the history stepwise. Thus a typical
`segment of history (taken from hand-written notes) might read: "3
`a.m. in bed at home, awoke and couldn't get his breath, sat up,
`wife called doctor who called ambulance. Brought into casualty."
`To record the essential information in
`this history using our
`display tree could require the following steps: history-dyspnoea-
`sudden onset-"X" hours ago. Though the transposition of words
`in this example is not extensive this type of condensation is not
`liked by the house staff.
`(2) The second difficulty arose from the "pointer" function of
`some text in the displays. As explained in our earlier paper the
`text directs the user to the next logical part of the structure.
`Because the history does not have a well-defined tree structure the
`doctor may need several attempts to find the symptom to be
`described. This is very aggravating and the user will commonly
`bypass the display tree and type details by hand, using the narra-
`tive facility.
`(3) The third problem encountered in the use of history displays
`a semantic one. Fixed text provides constraints
`is
`in use of
`descriptive terms. In some cases these terms were not the expres-
`sions which the doctor wanted to use. This problem occurs partly
`because traditionally medical students
`are taught to use the
`patient's own words in describing some episode of the history.
`text immediately produces conflict
`The fixed
`this aim is
`if
`attempted.
`
`Follow-up Displays
`These displays proved to be unacceptable to the house
`staff. Basically this arose from a failure to define the scope ot
`progress notes required for phase 1 system. Initially, a display
`system was designed to record only the physical examination.
`Before proper trials were done this was extended to include
`the history and finally a follow-up section was added. Though
`provision for investigation and treatment displays was made,
`these had not been designed at the time of installation in the
`ward, so the follow-up displays contained no reference to
`treatment or investigations. Because the project policy makers
`were isolated from the display design team the rather unfor-
`tunate decision was made to use the system as the primary
`clinical recording system rather than as a limited experimen-
`tal technique. As a result the house staff felt they needed a
`display system for follow-up notes with the capacity to record
`orders for tests, test results, transfer notices, treatment orders,
`diagnosis (even though a separate complete diagnosis display
`system was in existence), response to treatment, and other
`features. For reasons outlined above, these displays did not
`exist and the house staff were obliged to type much of this
`information into the computer-held record using the narrative
`facility.
`Though a new display system to include these features has
`been designed and implemented many of the problems remain
`because of the nature of traditional follow-up record. Much
`more thought needs to be given to the purposes of this
`progress record (Woodroffe, 1970). As in the case of the his-
`tory, the traditional type of hand-written progress record
`seems to us to contain much that is redundant or valueless as
`information for long-term storage.
`
`3 0 0
`
`29
`
`0
`
`23
`
`0 0 0 0
`
`4 3
`
`14
`25
`1
`15
`12
`
`TABLE IV
`
`No. of Entries to
`Interrogation
`7
`12
`6
`22
`
`ITotal No. of Screens
`Inspected
`65
`196
`53
`223
`
`Interrogation
`One of the projected advantages of a real-time system is
`that it can allow instant inspection of the patient's record
`without any possibility of losing case notes. The results show-
`ing the usage of interrogation presented in Table IV, how-
`ever, indicate that this feature held little attraction for the
`clinical staff. It is quite clear that the value of interrogation is
`directly proportional to the amount of information put into
`the system; hence the very limited usage of this facility merely
`reinforces the lack of acceptability of the recording tech-
`niques. In the next section we attempt to analyse some causes
`of this failure of the system.
`
`Operational Problems
`One major problem of the system at present is that far from
`reducing the time spent in case recording it has increased it.
`As indicated earlier the record-keeping system was installed in
`two general medical wards and though conceived as an
`experiment it has been implemented in one ward as the
`primary method of clinical recording for all patients admitted
`to that ward. This has led to duplication of clinical recording
`for two main reasons : (1) Only two shifts of computer onera-
`tors have been supplied, so that the system is available in
`the wards for only about eight hours a day. This ward, however,
`provides a 24-hour-a-day clinical service. Thus the notes of
`new patients or of clinical findings in patients already in the
`ward must be hand-written when the computer recording sys-
`tem is not "live." Later these notes must be transferred via a
`visual display unit to the computer. (2) A second problem of
`this sort arises when patients are admitted indirectly or trans-
`ferred from other wards where no computer recording facility
`-exists. These objections may seem trivial yet they procure
`considerable ill-will for the system by increasing the clerical
`load on the junior staff.
`
`Display System
`The technique of using visual displays and a multiple
`branching questionnaire has been reported by others (Kiely et
`al., 1968; Uber et al., 1968; Greenes et al., 1970). Our experi-
`ence in using this technique to cover a wide spectrum of
`medical case recording suggests that the following lessons can
`be learned.
`
`Physical Examination Display System
`The technique seemed acceptable to the house staff for
`recording the physical examination. The information acquired
`during examination has a natural structure enabling it to be
`
`SKYHAWKE Ex. 1024, page 2
`
`

`

`82 10 October 1970 Computer Clinical Record System: Assessment-Opit and Woodroffe
`MEDWIL JOURINAL
`Diagnosis Displays
`programmes themselves have proved entirely satisfactory and
`no dislocation of recording has occurred through a major
`A branching questionnaire technique was adopted, the
`programme breakdown.
`Nomenclature of Disease (Joint Committee of the Royal
`College of Physicians of London, 1961) being used as the
`model. This display system, too, was disliked and poorly used
`by the medical staff. They felt
`it was cumbersome, and
`frequently experienced difficulty in isolating
`particular
`a
`disease on the display tree.
`
`Conclusions
`Any attempt to use a large hospital-based computer as a
`means of record storage and retrieval must be conceived as
`an experiment. There is an unfortunate tendency on the part
`of the enthusiasts
`to assume a successful outcome, even
`though experience elsewhere with these systems is not en-
`couraging.
`As with any scientific experiment the objectives must be
`defined explicitly, and though an hypothesis can be enter-
`tained it is not wise to assume the result. Though phase 1 of
`the King's College Hospital project was conceived initially as
`an experiment it was not in fact implemented as one, at least
`not in our opinion. The objectives were described variously
`as, "to facilitate clinical research," "to improve patient care,"
`"to abolish paper records," "to provide a communication
`network," "to provide statistical and management informa-
`tion." These all seem working objectives, but as such are not
`sufficiently defined to enable any measure of attainment. As
`we indicated earlier, on a test of acceptability the real-time
`recording system has so far failed. Some of the problems are
`technical and can be overcome; others are logical, educa-
`tional, political, and social, and these present much greater
`problems.
`In a commercial application of the computer there is
`usually a well-defined "customer," and in medical areas such
`as laboratories where computer installations have succeeded
`there is almost always one person who sets the goals and
`compels action towards these goals. Unfortunately this situa-
`tion does not exist in the clinical departments of many
`teaching hospitals. We therefore believe that it is important
`to implement any experimental recording system in such a
`way that the technical problems can be solved, administrative
`procedures
`modified,
`hospital
`social
`and
`the
`structures
`allowed to adapt without generating the user's antagonism.
`When this antagonism develops it can prevent even a proper
`determination of the outcome of the experiment.
`In our opinion a busy medical ward is not the ideal site in
`which to install an experimental computer svstem. We believe
`that a trial installation would have been better in an out-
`patient clinic where "on-line" times would correspond more
`sensibly to clinic working times and where a gradual exten-
`sion of the clinicians' involvement could be fostered by an
`awareness of the help a computer can offer. Finally, we
`believe that the experience at King's College Hospital has
`shown that deliberate instruction in medical recording to
`students might well be a necessary prerequisite to the suc-
`cessful installation of computer record keeping. Though three
`years are spent teaching students how to observe symptoms
`and signs and to make deductions from them, usually no
`effort is made to teach them how to record their findings. To
`do this a determined effort must be made to understand the
`structure and purpose of clinical records, and it may well be
`that this is the right starting-point for a project to store the
`clinical record on a computer.
`
`This project is one of a number financed by the Department of
`Health and is under the medical direction of Professor J. Anderson.
`
`Remarks on Branching Questionnaire Technique
`Some general observations about the problems of using
`branching questionnaire techniques seem relevant.
`(1) One justification for using this approach for clinical record-
`ing at King's College Hospital is that it produces a record with
`easily
`identifiable
`text which can be searched by computer
`programmes for retrospective analysis of these records. Unfortu-
`nately it is immediately apparent that the objections to retrospec-
`tive studies using traditional hand-written notes still exist, so that
`though the speed and accuracy of data search are vastly improved
`by this technique there is nothing to suggest that the data are any
`more reliable.
`(2) As we pointed out in the previous paper messages must ter-
`minate automatically at certain points. The maximum possible
`length is determined partly by visual display unit screen size, and
`at King's College Hospital this is 400 characters. If a message
`which occupies more than one screen is constructed it cannot be
`verified and hence gives rise to an impossible situation for the per-
`son trying to record. On the other hand, short messages produce a
`disjointed, confusing text. In addition, interrogation of a patient's
`file using the visual display unit becomes a tedious procedure and
`obtaining clear association between several related messages is dif-
`ficult.
`(3) When designing the branching questionnaire another conflict
`arises between these two philosophies of design. The decision trees
`can be made obsessively complete, grouping together only ques-
`tions which are related, producing "long" trees with many displays
`to be viewed (and allowing few choices for each display). Alter-
`natively, "shorter" trees can be designed, grouping together ques-
`tions which commonly occur together, using many questions on
`each display screen. This might be called the intuitive approach.
`Both techniques have advantages and disadvantages. The display
`file designed at King's College Hospital has examples of both
`types of decision tree. Inquiry has shown that house staff prefer
`the "intuitive" displays, though they have not used either type
`extensively.
`
`Software and Hardware Problems
`"Hardware" is computer jargon for the electromechanical
`equipment, and "software" is a term to denote the specifically
`designed and coded programmes which control the operation
`of the computer. As indicated previously the visual display
`unit was originally conceived as being a mobile recording unit
`and trolleys were built specially to achieve this. Unfortunately
`these trolleys were badly designed, proving heavy, cumber-
`some, and far too large to enable the visual display unit to be
`used as a bedside recording device. In addition there are two
`cables to be connected-one a simple three-pin 13-amp. plug,
`the other a multipin plug, which is difficult to manipulate.
`Since only a limited number of sockets are available the
`cables trail about the ward. After about a month of trial it
`was necessary to retreat from this concept and leave the
`visual display unit as a stationary device.
`
`Reliability
`It is difficult to measure precisely the failure rate of the
`visual display units since a spare was always available to
`replace a faulty unit. Certainly no real difficulties have yet
`been experienced with the care and maintenance of this unit.
`The computer itself has proved reliable to date, the time lost
`from breakdowns (as a percentage of total on-line available
`time) over the first three months being about 2.5 %/0. The
`
`REFERENCES
`Greenes, R. A., Barnett, G. O., Klein, S. W., Robbins, A., and Prior, R. E.
`(1970). New England Journal of Medicine, 282, 307.
`Kiely, J. M., Juergens, J. L., Hisey, B. L., and Williams, P. E. (1968).
`Journal of the American Medical Association, 205, 571.
`Uber, G. T., Williams, P. E., Hisey, B. L., and Siekert, R. G. (1968).
`Proceedings of the Conference of the American Federation of Informnation
`Processing Societies, 33, 387;
`Woodroffe, F. (1970).
`Proceedings of the Working Conference of the Inter-
`national Federation of Information Processing. North Holland Publishing
`- in Press.
`
`SKYHAWKE Ex. 1024, page 3
`
`