United States Patent [19J
`Nielsen
`
`I IIIII IIIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111
`US005991781A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,991,781
`Nov. 23, 1999
`
`[54] METHOD AND APPARATUS FOR
`DETECTING AND PRESENTING CLIENT
`SIDE IMAGE MAP ATTRIBUTES
`INCLUDING SOUND ATTRIBUTES USING
`PAGE LAYOUT DATA STRINGS
`
`[75]
`
`Inventor: Jakob Nielsen, Atherton, Calif.
`
`[73] Assignee: Sun Microsystems, Inc., Palo Alto,
`Calif.
`
`[21] Appl. No.: 08/722,757
`
`[22]
`
`Filed:
`
`Sep. 27, 1996
`
`[51]
`[52]
`[58]
`
`[56]
`
`Int. Cl. 6
`•••••••••••••••••••••••••••••••••••••••••••••••••••••• G06F 17/21
`U.S. Cl. .............................................................. 707/513
`Field of Search ............................. 707/513; 434/114,
`434/450; 340/825.19; 345/349
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,186,629
`5,223,828
`5,287,102
`5,721,851
`5,727,950
`
`2/1993 Rohen ..................................... 434/114
`6/1993 McKiel, Jr ......................... 340/825.19
`2/1994 McKiel, Jr ......................... 340/825.19
`2/1998 Cline et al. ............................. 345/349
`3/1998 Cook et al. ............................. 434/450
`
`FOREIGN PATENT DOCUMENTS
`
`0604340 Al
`2317070
`
`6/1994 European Pat. Off ......... G06F 17/21
`3/1988 United Kingdom ............ G06F 17/21
`
`OTHER PUBLICATIONS
`
`"Inline Images: External Images, Sounds and Animations."
`A Beginner's Guide to HTML. Document on-line. Available
`from http://www.physics.iastate.edu//numaps/96/references/
`htmlprimer.html; Internet; accessed Aug. 10, 1997. pp.
`16-20, Apr. 1996.
`Tittel, Ed and Steve James. "HTML for Dummies" 2d ed.
`(Foster City: IDG Books Worldwide, Inc. 1996) pp. 3, 28,
`31, 32, 100, 101, 157, 187-193, 213-215, 231, Mar. 11,
`1996.
`T. V. Raman, "Emacspeak - A Speech Interface", Apr. 13,
`1996, Common Ground, Chicago '96 Conference Proceed(cid:173)
`ings, Conference on Human Factors in Computing Systems,
`Vancouver, Apr. 13-18, 1996, pp. 66-71.
`
`301 Start
`
`M. Krell et al., "V-Lynx: Bringing the World Wide Web to
`Sight Impaired Users", Apr. 11, 1996, Assets '96, 2nd
`Annual Conference on Assistive Technologies, Vancouver,
`Apr. 11-12, 1996, Nr. Conf. 2, pp. 23-26.
`A R. Kennel, "Audiograf: A Diagram-Reader for the
`Blind", Apr. 11, 1996, Assets '96, 2nd Annual Conference
`onAssistive Technologies, Vancouver,Apr. 11-12, 1996, Nr.
`Conf. 2, pp. 51-56.
`Database WPI: Section EI, Weed 9636, Derwent Publica(cid:173)
`tions Ltd., London, GB; Class TOl, AN 96359991,
`XP002074768; & JP 08 171 647 A (Hitachi Ltd), Jul. 2,
`1996; & US 5,715,412 (Abstract).
`Instant HTML Web Pages, by Wayne Ause, Ziff-Davis
`Press, ISBN 1-56276-363-6, 1995, pp. 1-15.
`
`Uniform Resource Locator (URL), Berners-Lee, Masinter
`& McCahill, Dec. 1994, pp. 1-25, (http://www.cis.o(cid:173)
`hio-state.edu/htbin/rfc/rfcl 738.html).
`"Extensions to HTML 3.0", Netscape Communication Cor(cid:173)
`poration, 1996, pp. 1-4, (http://home.netscape.com/assist/
`net_sites/html_extensions_3.html).
`"A Proposed Extension to HTML: Client-Side Image
`Maps", J. Seidman, Spyglass, Inc., Aug. 1996, pp. 1-7
`( ftp ://ds.internic.net/rfc/rfc 1980. txt ).
`"HTML 3.2 Features at a Glance", 7 pages, (http://
`www.w3.org/pub/WWW/MarkUp/Wilber/features.html).
`"Hypertext Markup Language -
`2.0", rfc1866, pp. 1-77,
`(http://www.cis.ohio-state.edu/htbin/rfc/rfc1866.html).
`
`Primary Examiner-James P. Trammell
`Assistant Examiner-John Leonard Young
`Attorney, Agent, or Firm-McDermott, Will & Emery
`
`[57]
`
`ABSTRACT
`
`Apparatus, methods, systems and computer program prod(cid:173)
`ucts are disclosed that provide enhancements to client-side
`image maps in HTML based documents. These enhance(cid:173)
`ments provide audio feedback to a user of a HTML capable
`application as to the hyperlinks associated with an area in an
`image. These enhancements also provide a visual represen(cid:173)
`tation of the areas defined by a client-side image map when
`the image is not available.
`
`60 Claims, 13 Drawing Sheets
`
`711
`
`713
`
`Ralph Lauren Corp., Exhibit 1006 Page 1
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 1 of 13
`
`5,991,781
`
`.
`
`~105
`
`WWW ServerJ ~
`
`Internet
`
`Figure 1a
`(Prior Art)
`
`WWW Browser 101
`
`Ralph Lauren Corp., Exhibit 1006 Page 2
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 2 of 13
`
`5,991,781
`
`Figure 1b
`(Prior Art)
`
`Ralph Lauren Corp., Exhibit 1006 Page 3
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 3 of 13
`
`5,991,781
`
`Network
`
`215
`
`223
`
`E) 12g7 I
`
`205
`
`1/0
`
`=== Im 211
`
`203{
`
`CPU
`207
`
`Memory
`
`2_Q9
`
`201/
`
`22
`
`~222
`
`Figure2
`
`Ralph Lauren Corp., Exhibit 1006 Page 4
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 4 of 13
`
`5,991,781
`
`no
`
`311
`
`313
`
`Get the ALT String
`
`Pass ALT String to Speech .,__ ____ ....
`
`Synthesis Facility
`
`Figure 3
`
`Ralph Lauren Corp., Exhibit 1006 Page 5
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 5 of 13
`
`5,991,781
`
`400
`/
`
`401
`
`40 Start Loa ing Sound
`Data
`
`410',,.
`
`411
`
`405
`
`41
`
`yes
`
`419
`
`Soun Data
`Processing
`
`Figure 4
`
`Ralph Lauren Corp., Exhibit 1006 Page 6
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 6 of 13
`
`5,991,781
`
`Figures
`
`Start Play of Error
`519 Message Data, Set
`Err_Stat to err_inprogress
`
`Ralph Lauren Corp., Exhibit 1006 Page 7
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 7 of 13
`
`5,991,781
`
`600
`/
`
`601
`
`rror Message Oat
`Play Completes
`
`610
`/
`
`603
`
`Set Err_Stat to no_err
`
`605
`
`Return
`
`611
`
`Sound Data Load
`Completes
`
`617
`
`Stop Play of Error
`Message Data
`
`621
`
`Set Err_Stat to no_err
`
`623
`
`619
`
`Play Beep
`Play ound
`Data ~----._ ____ ......,1 625
`
`Return
`
`615
`
`Figure 6a
`
`Ralph Lauren Corp., Exhibit 1006 Page 8
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 8 of 13
`
`5,991,781
`
`Set ATP to 0
`
`643
`
`yes
`
`Play Audio Interval in the Sound Data from ATP to
`minimum(length of sound data, ATP+lnterval_length)
`
`651
`
`ATP += lnterval_length
`
`653
`
`Figure 6b
`
`Ralph Lauren Corp., Exhibit 1006 Page 9
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 9 of 13
`
`5,991,781
`
`70
`Prior Art
`Processing
`
`711
`
`713
`
`Draw Rectangle
`of Hand W
`
`Draw Areas in
`Rectangle
`
`no
`
`Figure 7
`
`717
`Label Default Area
`
`Ralph Lauren Corp., Exhibit 1006 Page 10
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 10 of 13
`
`5,991,781
`
`es
`
`no
`
`809
`Set Unvisited
`..... ~ Color
`
`yes
`
`Set Visited
`Color ~ -...
`
`811
`
`813
`
`Draw
`Rectangle
`
`Figure B
`
`Ralph Lauren Corp., Exhibit 1006 Page 11
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 11 of 13
`
`5,991,781
`
`For Each Area in
`Reverse Order
`
`Se ect
`Shape Color 907
`
`911
`
`Figure 9
`
`Ralph Lauren Corp., Exhibit 1006 Page 12
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 12 of 13
`
`5,991,781
`
`1005
`
`1007
`
`Find Largest
`Uncovered Region
`
`yes
`
`1011
`
`Choose Rectangle
`Color
`
`1013
`
`Label Default Area
`
`Figure 10
`
`Ralph Lauren Corp., Exhibit 1006 Page 13
`
`

`

`U.S. Patent
`
`Nov. 23, 1999
`
`Sheet 13 of 13
`
`5,991,781
`
`Default Area
`1103 1
`
`http://www.sunmicros
`ystem.com/eng/demo/
`sample.html
`1107/
`
`Area2 I
`1111
`
`1109
`
`Area 1
`1115/
`
`1101
`
`Figure 11
`
`Ralph Lauren Corp., Exhibit 1006 Page 14
`
`

`

`5,991,781
`
`1
`METHOD AND APPARATUS FOR
`DETECTING AND PRESENTING CLIENT
`SIDE IMAGE MAP ATTRIBUTES
`INCLUDING SOUND ATTRIBUTES USING
`PAGE LAYOUT DATA STRINGS
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`This invention relates to the field of presenting data
`streams constructed using a hypertext markup language
`(HTML). Specifically, this invention is a new and useful
`method, apparatus, system and computer program product
`for enhancing the operation of client-side image maps for
`those who cannot see the displayed image or for those who
`do not desire to download the image.
`2. Background
`Because most of the information provided to sighted
`computer users consists of visual information provided on a
`computer monitor, the visually impaired or blind computer
`user works under a significant disadvantage. This same
`disadvantage applies to users who are remotely operating a
`computer using a keypad but without a visual display, such
`as a telephone. For the visually impaired the computer
`monitor is a featureless expanse of glass or plastic that is
`unlike paper media that can provide a change in texture so
`that the blind can tactically read the change in texture (such
`as Braille). Screen-readers enable computer access to visu(cid:173)
`ally impaired users. Screen-readers are well known in the art
`and convert the text displayed on a computer screen into a
`sound that mimics spoken language. In other words, the
`computer reads the displayed text to the user. However, the
`information presented to the user must be in textual form.
`These devices work well when textual information is pre(cid:173)
`sented on the computer. However, these devices are very
`limited when graphical information is presented. The tech(cid:173)
`nology does not yet exist that allows a computer to describe
`an image displayed on a monitor. Traditionally this limita(cid:173)
`tion has not significantly affected the visually impaired
`because, until recently, most information presented on com(cid:173)
`puter monitors has been textual based. However, with the
`explosion of the World Wide Web (WWW), graphical infor(cid:173)
`mation is being widely used to carry information content.
`Thus, visually impaired computer users find it difficult to
`navigate the WWW.
`World Wide Web
`The WWW is a massive hypertext system that a user
`accesses using a WWW browser application executing on a
`computer-an information access apparatus. The WWW
`browser apparatus communicates with, and is a client of,
`information provider apparatus such as server computers
`each executing server applications capable of communicat(cid:173)
`ing with the client browser application. These clients obtain
`information and services, in the form of web pages, from the
`server. These web pages are identified by unique universal
`resource locators (URL) and are usually specified using a
`markup language-generally a version of the hypertext
`markup language (HTML). HTML 2.0, is described in
`RFC1866 and can be found on the WWW at: "http://
`www.cis.ohio-state.edu/htbin/rfc/rfc1866.html" and is
`hereby incorporated by reference as illustrative of the prior 60
`art. However, HTML is evolving and variants exist. For
`example, the HTML 3.2 proposal is found at "http://
`www.w3.org/pub/WWW/MarkUP/Wilbur/features.html",
`and a proposed HTML extension that describes client-side
`image maps is found at "http://home.netscape.com/assist/
`net_sites/html_extensions_3.html" and by RFC1980
`found at "ftp://ds.internic.net/rfc/rfc1980.txt".
`
`2
`The background of the WWW, WWW browser
`applications, and URLs are well described by reference to
`the first chapter of Instant HTML Web Pages, by Wayne
`Ause, Ziff-Davis Press, ISBN 1-56276-363-6, copyright
`5 1995, pages 1-15, hereby incorporated by reference as
`illustrative of the prior art. The URL specification, also
`incorporated by reference, is described in RF Cl 738 and can
`be found on the WWW at: "http://www.cis.ohio-state.edu/
`htbin/rfc/rfcl 738/html". Briefly, the URL contains a proto-
`10 col specification and a path specification. The protocol
`specification notifies the browser of what protocol to use
`when accessing the remote server. The path specification is
`generally a hierarchical path that specifies a data server
`followed by a hypernode (such as a web page) that actually
`15 provides the information for the browser.
`FIG. la illustrates an example client-server information
`access system implemented using a plurality of computers.
`An information client system 101 communicates over a
`network 103 such as the internet 103 to a plurality of
`20 information server systems 105. The client system 101
`encapsulates requests for services and information within an
`applicable internet protocol and passes the encapsulated
`requests to the internet 103 as indicated by an arrow 107.
`The internet 103 routes these requests to each of the plurality
`25 of information server systems 105 addressed within the
`request as indicated by a plurality of arrows 109. Each of the
`plurality of addressed information server systems 105
`respond to the client system 101 with responses appropriate
`to the service or information requested by the client system
`30 101. Once the client system 101 receives this information, it
`is presented to the user by using an application program (for
`example, a WWW browser) executing on the client com(cid:173)
`puter.
`Hypertext documents, composed of HTML data, can be
`35 presented to a user by an application. The application
`processes the HTML data contained within the document to
`generate an image that can be displayed to a user on a
`computer display or tangible page. Unlike page description
`languages, such as PostScript, the "page" layout of HTML
`40 documents is dependent on the drawing area used to display
`the image described by the HTML data. Thus, HTML is used
`to describe hypertext documents that are portable from one
`computing platform to another and that do not require
`WYSIWYG functionality. The HTML concept is that of a
`45 scrolling page that can be resized as desired by the user.
`FIG lb illustrates a web page 151 that uses a mapped
`image to indicate hyperlinks to other information. The web
`page 151 has a background 153, a first selectable control
`area (SCA) 155, a second SCA 157 and a third SCA 159.
`50 One skilled in the art will understand that the first SCA 155,
`the second SCA 157 and the third SCA, need not be
`distinguishable from the background 153, although gener(cid:173)
`ally the SCAs 155, 157, 159 are visually distinct.
`Sighted users navigate this web page by positioning a
`55 cursor over an SCA by manipulating a pointing device and
`activating the pointing device to invoke the associated web
`page. This approach to selecting information is difficult for
`visually impaired users because they cannot see where to
`position the cursor on the image.
`The visually impaired user accesses a web page, such as
`that in FIG. lb, by using leap keys to move a cursor. Leap
`keys are often assigned as the up, down, left and right arrow
`keys. Often these leap keys have the following functions: the
`up key causes the parent page of the currently displayed
`65 page to be displayed; the down key causes activation of the
`currently selected URL and typically results in a child page
`of the current page to be displayed; the left key causes the
`
`Ralph Lauren Corp., Exhibit 1006 Page 15
`
`

`

`5,991,781
`
`4
`art. Further, sound reproduction facilities are also well
`known in the art and are used to play reproduce information
`stored in digital form into sound.
`Thus, there is a need to extend image map capabilities to
`5 better operate when the images associated with the image
`map are not available for viewing.
`
`20
`
`3
`cursor to backup to the previous SCA on the current page;
`and the right key causes the cursor to advance to the next
`SCA on the current page. Each of these leap keys cause an
`auditory message to be played to the user that provides
`information as to where the cursor is in the Web Page. The
`auditory message is often the text displayed by the hyper(cid:173)
`link's URL. Thus, if URL was "http://www.sun.com/", the
`auditory message would be the following spoken words: "H
`T T P colon slash slash W W W dot Sun dot Com slash".
`Another aspect of WWW browser applications is that they
`often maintain a database of recently accessed hyperlinks.
`These recently accessed links are displayed in a different
`color from links that have not been accessed.
`Server-side image maps operate by sending the selected
`coordinates with respect to the displayed image to a process 15
`identified by a URL. This process then determines what
`action is to be associated with the selected coordinate and
`performs that action. Thus, although a particular image may
`have multiple selectable areas, the client does not have the
`information needed to directly invoke an action and does not
`receive any feedback about the selectivity of an area.
`Client-side image maps operate by associating a map of
`areas to the image map. The map provides facilities for
`specifying a shape at a given coordinate of the image. This
`area is also associated with a reference (a URL) that allows
`the client to directly invoke the reference when the user
`selects an area. The area also may contain an "ALT"
`attribute that contains text that describes the containing area.
`This text is used with text-only WWW browsers (or brows(cid:173)
`ers that do not automatically download images) as a substi(cid:173)
`tute for the areas of the image. The ALT attribute is also used
`when the prior art cannot display the image. Among others,
`this condition occurs when the image is not available to the
`WWW browser, such as when the image-loading facility of
`the browser is turned off, when the image is unable to be 35
`retrieved from the server, or when the user has stopped the
`image download by invoking the stop download command
`(usually by activating the browser's stop download SCA).
`Users of WWW browser facilities who are connected to
`the internet through analog phone lines often turn off the 40
`image capability of the browser so that the time required to
`display web pages is reduced (because the data intensive
`images are not transmitted to the application). This operating
`mode allows those who do not desire to view the images to
`more quickly navigate the WWW. A problem with this mode 45
`of operation is that the structure of client-side image maps
`is also not displayed. Thus, a client side image map only
`displays the ALT attribute in the <AREA> element (if
`provided) without any spatial information as to the shape,
`size, and arrangement of the <AREA> elements.
`Another problem with client-side image maps is that
`many users disable the presentation of graphical information
`when browsing the WWW because these images take sig(cid:173)
`nificant amounts of time to download. This problem is
`exacerbated by low speed data links or if too many users ss
`share a high-speed digital connection. Thus, users often
`browse the WWW with image loading disabled until they
`reach an interesting web page. The problem with this
`approach to browsing the WWW is that client-side image
`maps have user selectable control areas (SCAs). Without the 60
`image, the position, shape, and structure of the SCAs
`associated with the image are not apparent even if the
`hyperlink is displayed
`Speech Synthesis and Audio Playback
`Speech synthesis systems such as the DECtalk® and 65
`Macintalk® facilities convert textual strings into sounds that
`mimic human speech. These facilities are well known in the
`
`10
`
`SUMMARY OF THE INVENTION
`The present invention provides an economical, apparatus,
`method, system and computer program product for provid(cid:173)
`ing enhanced facilities to users of client-side image maps.
`One aspect of the invention is a computer controlled method
`for human-computer interactions. The method detects an
`image map that is associated with an image. This image map
`has an audio attribute associated with a selectable area of the
`image. The method also detects a preselection condition on
`the selectable area of the image. The method then presents
`a sound associated with the audio attribute through a sound
`production facility.
`In another aspect of the invention, an information display
`system is disclosed for enhancing human-computer interac(cid:173)
`tions. This system includes an image map detection mecha(cid:173)
`nism. The image map detection mechanism detects an image
`map that is associated with an image. The image map
`25 detection mechanism has an audio attribute associated with
`a selectable area of the image. The system also includes a
`preselection condition detection mechanism that is config(cid:173)
`ured to detect a preselection condition on the selectable area
`in the image. Additionally, the system includes a sound
`30 presentation mechanism configured to present a sound
`through a sound production facility upon detection of the
`preselection condition. The sound is associated with the
`audio attribute of the selectable area.
`In yet another aspect of the invention an apparatus for
`enhancing human-computer interactions is disclosed. The
`apparatus has a CPU and a memory. The apparatus also has
`an image map detection mechanism, a preselection condi(cid:173)
`tion detection mechanism, and a sound presentation mecha(cid:173)
`nism having the same functions as the system described
`above.
`Another aspect of the invention is a computer program
`product on a computer usable medium for causing a com(cid:173)
`puter to provide enhanced human-computer interactions.
`When executed on the computer, the computer readable
`code causes a computer to effect an image map detection
`mechanism, a preselection condition detection mechanism,
`and a sound presentation mechanism having the same func(cid:173)
`tions as the system described above.
`Additionally, another aspect of the invention is a com-
`so puter controlled method of enhancing human-computer
`interactions by first detecting an image map associated with
`an image area. The method also detects a height attribute and
`a width attribute for the image area. The method then
`displays a border around the image area. This border has a
`status representation, and the image map specifies at least
`one selectable area in the image area.
`The description of the invention also discloses an infor-
`mation display system for enhancing human-computer inter(cid:173)
`actions. This system includes an image map detection
`mechanism that is configured to detect an image map
`associated with an image area. The system also includes an
`image size detection mechanism that is configured to detect
`a height attribute and a width attribute for the image area.
`Additionally, the system includes a border displaying
`mechanism configured to display a border around the image
`area, the border having a status representation and the image
`map specifying at least one selectable area in the image area.
`
`Ralph Lauren Corp., Exhibit 1006 Page 16
`
`

`

`5,991,781
`
`6
`FIG. 10 illustrates the process used to label the default
`area in the image map in accordance with a preferred
`embodiment; and
`FIG. 11 illustrates the resultant display of the areas in the
`rectangle in accordance with a preferred embodiment.
`
`5
`Another aspect of the invention is an information display
`apparatus for enhancing human-computer interactions hav(cid:173)
`ing a central processing unit (CPU) and a memory. This
`aspect of the invention includes an image map detection
`mechanism that is configured to detect an image map s
`associated with an image area. The apparatus also includes
`an image size detection mechanism that is configured to
`detect a height attribute and a width attribute for the image
`area. Additionally, the apparatus includes a border display(cid:173)
`ing mechanism configured to display a border around the 10
`image area, the border having a status representation and the
`image map specifying at least one selectable area in the
`image area.
`Yet a further aspect of the invention is a computer
`program product embedded on a computer usable medium
`for causing a computer to provide enhanced human(cid:173)
`computer interactions. When executed on a computer, the
`computer readable code causes a computer to effect an
`image map detection mechanism that is configured to detect
`an image map associated with an image area. The computer 20
`readable code also causes a computer to effect an image size
`detection mechanism that is configured to detect a height
`attribute and a width attribute for the image area.
`Additionally, the computer readable code causes a computer
`to effect a border displaying mechanism configured to
`display a border around the image area, the border having a
`status representation, and the image map specifying at least
`one selectable area in the image area.
`The foregoing and many other objects and advantages of
`the present invention will no doubt become obvious to those 30
`of ordinary skill in the art after having read the following
`detailed description of the preferred embodiments which are
`illustrated in the various drawing figures.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`Notations and Nomenclature
`The following "notations and nomenclature" are provided
`to assist in the understanding of the present invention and the
`preferred embodiments thereof.
`Cursor-A device on a computer display that indicates
`what portion of the computer display is of interest to a user.
`15 A cursor can be positioned by a pointing device, by opera(cid:173)
`tion of leap keys or other methods well understood in the art.
`The cursor indicates where an activation will be applied.
`Relevant Active Area-An area defined by an image map,
`within the area of an image, that includes an audio attribute.
`Preselection condition-A condition where a cursor has
`continuously been in an area defined by an image map for a
`specified period of time.
`Graphical User Interface (GUI)-A user interface that
`allows a user to interact with a computer display by pointing
`25 at selectable control areas on the display and activating a
`command or computer operation associated with the select(cid:173)
`able control area. GUis are well known in the art.
`Pointing device-A device that is responsive to a com(cid:173)
`puter user's input that moves an indicator (a form of cursor)
`on a computer display screen. Such an indicator has an
`active point such that if the pointing device is activated (for
`example, by a button push for a mouse device) a command
`associated with the selectable control area covered by the
`active point is invoked. Pointing devices are generally used
`35 with graphical user interfaces.
`Selectable control area (SCA)-An area on a computer
`display that is sensitive to activation of a pointing device. On
`activation of the pointing device over the selectable control
`area, a command or computer operation associated with the
`40 selectable control area is invoked. Most computer systems
`that provide a Graphical User Interface (GUI) also provide
`other methods for invoking these commands or computer
`operations such as keyboard function keys or command
`lines.
`Procedure-A self-consistent sequence of steps leading to
`a desired result. These steps are those requiring physical
`manipulation of physical quantities. Usually these quantities
`take the form of electrical or magnetic signals capable of
`being stored, transferred, combined, compared, and other-
`so wise manipulated. These signals are referred to as bits,
`values, elements, symbols, characters, terms, numbers, or
`the like. It will be understood by those skilled in the art that
`all of these and similar terms are associated with the
`appropriate physical quantities and are merely convenient
`ss labels applied to these quantities.
`Overview
`The manipulations performed by a computer in executing
`opcodes are often referred to in terms, such as adding or
`comparing, that are commonly associated with mental
`60 operations performed by a human operator. In the present
`invention no such capability of a human operator is neces(cid:173)
`sary in any of the operations described herein. The opera(cid:173)
`tions are machine operations. Useful machines for perform(cid:173)
`ing the operations of the invention include programmed
`65 general purpose digital computers or similar devices. In all
`cases the method of computation is distinguished from the
`method of operation in operating a computer. The present
`
`45
`
`DESCRIPTION OF THE DRAWINGS
`FIG. la illustrates an example client-server information
`access system implemented using a plurality of computers;
`FIG. lb illustrates a web page display using a mapped
`image to indicate hyperlinks to other information;
`FIG. 2 illustrates a computer configured to support the
`invention in accordance with a preferred embodiment of the
`invention;
`FIG. 3 illustrates the process used to practice the inven(cid:173)
`tion using the ALT attribute in accordance with a preferred
`embodiment;
`FIG. 4 illustrates process used to practice the invention
`using a SOUND attribute in accordance with a preferred
`embodiment;
`FIG. 5 illustrates the process used by the "Sound Data
`Processing" procedure in accordance with a preferred
`embodiment;
`FIG. 6a illustrates asynchronous processes used to sup(cid:173)
`port presenting audio messages in accordance with a pre(cid:173)
`ferred embodiment;
`FIG. 6b illustrates the process used to stop presentation of
`audio messages after moving the cursor off of an area in
`accordance with a preferred embodiment;
`FIG. 7 illustrates the process used to display the active
`areas defined by an image map in accordance with a pre(cid:173)
`ferred embodiment;
`FIG. 8 illustrates the process used to draw a rectangle
`indicating the extent of an unavailable image in accordance
`with a preferred embodiment;
`FIG. 9 illustrates the process for drawing and labeling the
`areas within the rectangle in accordance with a preferred
`embodiment;
`
`Ralph Lauren Corp., Exhibit 1006 Page 17
`
`

`

`5,991,781
`
`7
`invention relates to method steps for operating a computer in
`processing electrical or other (for example, mechanical,
`chemical) physical signals to generate other desired physical
`signals.
`The invention also relates to an apparatus for performing 5
`these operations. This apparatus may be specially con(cid:173)
`structed for the required purposes or it may comprise a
`general purpose computer as selectively activated or recon(cid:173)
`figured by a computer program stored in the memory of a 10
`computer. The procedures presented herein are not inher(cid:173)
`ently related to a particular computer or other apparatus. In
`particular, various general purpose machines may be used
`with programs written in accordance with the teachings
`herein, or it may prove more convenient to construct more 15
`specialized apparatus to perform the required method steps.
`The required structure for a variety of these machines will
`appear from the following description. Also, the invention
`may be embodied in a computer readable storage medium
`encoded with a program that causes a computer to perform 20
`the programmed logic.
`Operating Environment
`Some of the elements of a computer system, referred to by
`a general reference character 201, configured to support the
`invention are shown in FIG. 2 wherein a processor 203 is
`shown, having an Input/Output ("1/0") section 205, a central
`processing unit ("CPU") 207 and a memory section 209. The
`1/0 section 205 is connected to a keyboard 211, a disk
`storage unit 213, a network interface 215 to provide access
`to a network 217, a display unit 219, a pointing device 221,
`a sound production facility 222, and a CD-ROM drive unit
`223. The CD-ROM unit 223 can read a CD-ROM medium
`225 that typically contains programs and data 227. The
`CD-ROM unit 223 (using the CD-ROM medium 225) and
`the disk storage unit 213 comprise a filestorage mechanism.
`Such a computer system is capable of executing information
`display applications that embody the invention.
`One skilled in the art will understand that the invention
`described herein does not depend on the existence of a
`client-server information access mechanism, similar to the
`one illustrated in FIG. la, because information to be dis(cid:173)
`played to a user often resides on the same computer that
`accesses the information. For the same reason, the invention
`does not require the network interface 215 or access to the
`network 217. Rather, the invention is directed toward pre(cid:173)
`senting HTML information to a user of a computer con(cid:173)
`trolled display device independent of the source of the
`information. The computer system illustrated by FIG. 2 is
`capable of executing WWW browser applications that
`embody the invention.
`Image Map Specification
`The client-side image map facility allows the web page
`author to provide a definition of the SCAs on the image as
`well as what action the browser should take when the area
`is selected. Table 1 illustrates the HTML markup used to
`describe such an image map according to the RFC1980
`specification referred to above.
`
`TABLE 1
`
`<MAP NAME="navlbar">
`<AREA SHAPE="RECT" COORDS="0,0,60,30"
`HREF="http://www.sun.com/"
`ALT="Sun Microsystem's Home Page">
`<AREA SHAPE="RECT" COORDS="61,0,500,30"
`HREF="/current/columns/alertbox''
`ALT="Jakob's Alert Box column">
`
`8
`
`TABLE 1-continued
`
`<AREA SHAPE="RECT" COORDS="501,0,579,30"
`HREF="search.html''
`ALT ="search"