68
`AN ADAPTABLE USER INTERFACE TO A
`MULTIMEDIA TELECOMMUNICATIONS
`CONVERSATION SERVICE FOR PEOPLE
`WITH DISABILITIES
`
`Hine N.A, Wilkinson D, Gordon I. A. S., Arnott J.L.
`MicroCentre, University of Dundee, Dundee, UK.
`e-mail: nhine@mic.dundee.ac.uk
`
`KEYWORDS: Adaptable User Interface, Multimedia, Telecommunications, People with
`Disabilities.
`
`ABSTRACT: The emergence of multimedia telecommunications services is potentially
`particularly attractive for people who have reduced mobility and other disabilities. It is
`often the case, however, that a user will have a complex range of impairments that require
`specific and often individual attention. By using a computer as the telecommunications
`terminal additional media can be made available and special intelligent assistive techniques
`can be added. These changes have implications for the design and customisation of the
`user interface. Some initial experiences gained in conducting experiments with a group of
`users with disabilities employing an adaptable emulated multimedia conversation
`environment are reported.
`
`BACKGROUND
`
`In recent years, a number of advances have been
`made in telecommunications and computing
`technology that have resulted in videotelephone
`equipment and services becoming available for
`corporate and domestic customers. It is now
`possible to choose either dedicated videophone
`terminals or to add videophone functionality to
`suitably powerful desktop computer systems. These
`terminals can operate over local area networks,
`conventional analogue telephone systems or ISDN
`digital telecommunications networks. In general,
`however, all these networks do not have sufficient
`bandwidth to handle full motion, real time video
`signals. Recent advances in broadband
`telecommunications technology, however, promise
`bandwidth that could readily accommodate full
`motion, real time, high quality video. In addition,
`a host of other multimedia services should be
`available to be added to basic videophone services,
`allowing flexible and versatile services to be
`constructed to suit specific applications and user
`requirements.
`
`The work reported in this paper involved the
`construction of an emulated videophone service that
`can be employed in the study of how such a service
`can be used by people with disabilities. The
`emulated service was operated through a flexible
`user interface whose layout could be changed to
`reflect the information media being exchanged and
`the specific requirements of the user to compensate
`for their disability. This interface was designed to
`
`follow standard best practice in multimedia interface
`design. [1,2,3]
`
`The work was undertaken within the framework of
`the EC funded RACE project R2009 IPSNI II,
`Access to B-ISDN Services and Applications by
`People with Special Needs.
`
`DESCRIPTION OF THE SERVICE
`
`The emulated service allows the simultaneous
`exchange of video and audio information between
`two Apple Macintosh computers with video overlay
`and audio play-through functionality. The
`functionality to pass text between the two machines
`and the functionality to simultaneously show a
`picture and to draw or type on it were added. These
`functions were added to allow the service to be used
`to study the accessibility issues for people with
`disabilities. In the case that users are unable to
`speak, they may be able to communicate their ideas
`via a text telephone. If they have poor language or
`literacy skills, they may be able to highlight aspects
`of a displayed picture.
`
`The emulated service was constructed in the
`SuperCard rapid prototyping environment. The user
`interface allows parts of the computer display to be
`allocated for the display of video or the exchange of
`text, or the display and annotation of pictures as
`required. A possible layout where these three
`functions are available are shown in figure 1 below.
`
`K. Nordby et al. (eds.), Human-Computer Interaction
`© IFIP International Federation for Information Processing 1995
`
`Epic Games Ex. 1021
`Page 1
`
`

`

`An adaptable user interface
`
`395
`
`Figure 1 - Emulated Videophone with additional text exchange and picture display/annotation
`functionality.
`
`In order to study the relative contribution that
`different media make to the successful exchange of
`information, and to study the accessibility of
`different media for people with disabilities, a
`number of goals were set.
`
`1) To be able to construct interfaces where the
`users, particularly users with disabilities, would see
`one single “application” on the screen of the
`computer. Although each function (i.e. video
`exchange, text exchange and picture display and
`annotation) was handled through a separate window,
`this was not supposed to be apparent to the user.
`This was designed to avoid confusion arising from
`information being handled by one window being
`obscured by another window.
`
`2) To develop a system that could be readily altered
`to move, resize or hide the part of the interface that
`handled a particular medium.
`
`3) To develop an interface where parts of the
`display area could be allocated to additional
`software elements that would provide a user with
`disabilities with additional assistance, for example
`an on-screen keyboard.
`
`The flexibility of the resulting system allows the
`layout of the screen to be altered to reflect the needs
`of the user and the information to be exchanged. It
`is possible to restore to each window its
`conventional boundary features. This allows each
`window to be resized and repositioned. Once a new
`layout is approved, it can be frozen and the window
`boundary features removed. The parameters
`governing each layout can be stored for later re-use,
`
`so task or user specific layouts could be recalled,
`and the same system tailored to the needs of a
`variety of different users.
`
`This became particularly important when the needs
`of users with disabilities were considered. Many
`people have a complex mix of physical and sensory
`impairments that not only result in difficult in
`speaking, but also affect their ability to use a
`conventional computer keyboard or pointing device.
`For this reason, a set of devices and software were
`assembled that could be added to the basic system
`to adapt the service interface to allow it to be made
`useable.
`
`Because information could be exchanged in a
`number of media, and because each mix of
`impairments and the resulting disabilities are unique
`to each user, it is impossible to predict in advance
`exactly how the system should be adapted. For this
`reason, a structured approach to testing the
`possibility of providing access to the emulated
`service was devised. This is described below.
`PILOT STUDY
`
`Method
`Six attenders at a rehabilitation day care facility
`agreed to take part in the pilot study to test the
`service and to adapt the interface to make it
`accessible to them. In each case, the following
`procedure was followed:
`
`1) A conversation scenario was constructed that
`involved the user in a discussion about a visit to
`New York State in the USA. The scenario was
`
`Epic Games Ex. 1021
`Page 2
`
`

`

`396
`
`Part Five Case Studies: Application and Practice
`
`loosely scripted, in that the data to be exchanged
`and the media to be employed were given, but the
`users were free to “package” their questions and
`responses as they wished. A member of the research
`team took the role of the “travel agent” throughout
`the study.
`
`2) Each user in the study had previous experience
`of text processing and drawing and painting
`applications on the Macintosh (the preferred
`machine at the rehabilitation centre). A number of
`the users had taken part in earlier studies involving
`the use of the text telephone. None had any
`experience of using a videophone. They were all
`given hands on training in the use of the various
`functions of the system, including a practice run
`through the scenario script. Then an attempt was
`made to run through the script as a “real”
`conversation. Where a user encountered a problem,
`a “helper” (another member of the research team)
`handled that part of the conversation. The problems
`were logged.
`
`3) The experience of using the service was
`analysed. Each problem encountered was
`considered, and an adaptation to the hardware or an
`additional software assistive function was proposed.
`The user interface layout was re-designed to
`accommodate any changes that were necessary.
`
`4) The system, complete with adaptations was
`tested with the users. If problems remained, the last
`step was repeated once more.
`
`SUBJECTS
`The subjects in this study had the following
`disabilities that affected their ability to use the
`emulated service.
`
`1) Subject R: Cerebral Palsy, resulting in severe
`loss of speaking abilities and no useable control of
`hand movements. R is able to use head movements
`to press a switch.
`
`2) Subject E: Cerebral Palsy resulting in major loss
`of speaking ability and little useable control of
`hands. E uses her chin to control a motorised
`wheelchair through a set of switches.
`
`3) Subject I: Advanced muscular dystrophy,
`resulting in major loss of strength and severe loss of
`speaking abilities. I is able to write, but with
`difficulty.
`
`4) Subject G: Cerebral Palsy, resulting in major
`loss of speaking abilities and reduced ability to use
`hands. G employs an assistive speaking device to
`generate speech and can slowly use a keyboard and
`mouse.
`
`5) Subject J: Impairment unknown, but has speech
`that is difficult to understand and some reduction in
`
`the ability to use hands.
`
`6) Subject D: Pre-lingually profoundly deaf with
`the consequence that speech is very difficult to
`understand.
`
`Results
`In all cases it was found that the user interface could
`be adapted so that the users could handle every part
`of the conversation script. This was achieved after a
`maximum of two attempts to adapt the interface.
`
`The problems encountered and the adaptations made
`for each subject are listed below.
`
`1) Subject R: Without adaptation, this user was
`unable to take part in a conversation using any
`functional part of the service. Adaptation of the
`service involved adding a switch interface that
`triggered an on-screen scanning array. The on-screen
`scanning array scans through objects in the scan
`window. The scan can be started by a switch press
`and stopped when the scan marker is over the object
`that is desired. In this case, a full set of
`alphanumeric and punctuation characters were
`provided in addition to a set of mouse emulation
`functions. This allowed the user to type and to
`perform mouse functions on the screen objects,
`simply by pressing a switch with a movement of
`the head. This was so slow that text prediction
`software was also added to the service interface.
`
`The text prediction software presented the user with
`a list of the most predictable words that matched a
`character typed by the user. If the required word
`was in the list, it was selected. If not the next letter
`was typed. Considerable keystroke savings, and
`hence time savings can be made using this
`technique.
`
`The addition of on-screen assistive techniques meant
`that the layout of the screen had to be altered to
`accommodate the additional elements. This was
`accompanied by a reduction in the size of elements
`such as the video window and the text telephone
`window. The effect of this on conversation fluency
`and effectiveness is a matter for further study.
`
`2) Subject E: This user was unable to type or use a
`mouse, but could use a rollerball mounted so that it
`could be operated with movements of the chin. The
`roller ball was equipped with two buttons. The left
`hand one was configured to act as a conventional
`button and the right hand one was configured to
`stay “down” when it was used, allowing screen
`elements to be dragged. Typing was provided by
`adding an on-screen keyboard. Items on the
`keyboard were selected by clicking the pointer on
`them. In order to improve speed and typing
`accuracy, text prediction was added. The layout of
`the interface for this user is shown in figure 2
`below.
`
`Epic Games Ex. 1021
`Page 3
`
`

`

`An adaptable user interface
`
`397
`
`Figure 2 - Emulated Videophone with adaptations for subject E
`
`3) Subject I: The adaptation employed for this user
`consisted of an on-screen keyboard and text
`prediction.
`
`4) Subject G: This user did not need any adaptation
`to the interface. The siting of the keyboard, mouse
`and the assistive speaking device was critical.
`Because of reduced hand movement control, a
`rollerball was tried, but this proved to require too
`fine motor control for this user.
`
`5) Subject J: The only adaptation made to the
`system was the addition of the text prediction,
`added in the space below the picture window.
`
`6) Subject D: Again, the only adaptation made to
`the system was the addition of the text prediction.
`
`As a result of these adaptations, all the users were
`able to complete all parts of the conversation
`without any assistance. It was found that the
`interface could be configured to suit each user in a
`matter of minutes by recalling saved layouts.
`
`CONCLUSIONS
`
`This exercise demonstrated the need for an adaptable
`interface that could be tailored to the needs of
`different users, and it showed that it is possible for
`people with quite severe disabilities to participate
`independently in a multimedia conversation.
`
`Having verified that the basic emulated service tool
`operated as required and that the interfaces can be
`adjusted, stored and retrieved when required, a
`number of aspects of the interface to multimedia
`conversation services can now be explored using this
`tool. These include:
`
`1) The usefulness of different media to a
`conversation or collaborative task, and how this
`usefulness can be affected by the absolute and
`relative positions or sizes of the “media spaces”
`within the interface.
`
`2) The value of being able to customise the
`appearance of the interface and its elements
`according to task or personal preference, and being
`able to recall task or user dependent layouts.
`
`3) Interaction between people in a conversation or
`collaborative task, particularly when one or more
`participants have difficulty handling an information
`medium. Issues such as those that govern the
`suitability of another medium to convey the same
`information, the effect that this difficulty has on the
`richness of the information being conveyed, and the
`types of assistance that the remote partner can give
`the person with a disability can all be explored.
`
`It is the intention of the authors to utilise this tool
`to explore some of these issues.
`
`REFERENCES
`
`[1] Blattner M. M. and Dannenberg R. B., (1992)
`“Multimedia Interface Design”, Addison-Wesley
`Publishing Company
`
`[2] Schneiderman B., (1992) “Designing the User
`Interface”, Addison-Wesley Publishing Company
`
`[3] Weisbecker A., Machate J. and Koller F., (1993)
`“Guidelines & Rules for Development of MADE
`Multimedia Applications”, FhG-IAO, Deliverable of
`Esprit Project 6307 (MADE1)
`
`Epic Games Ex. 1021
`Page 4
`
`