United States Patent
`US 6,677,965 Bl
`(10) Patent No.:
`12
`9
`9
`Ullmannet al.
`(45) Date of Patent:
`Jan. 13, 2004
`
`
`US006677965B1
`
`(54) RUBBER BAND GRAPHICAL USER
`INTERFACE CONTROL
`
`(75)
`
`73)
`
`Assi
`
`:
`
`Inventors: Cristi N. Ullmann, Austin, TX (US);
`.
`Nice c won Round Rock. TxOS):
`(Us) ael
`W.
`Brown,
`lcorgelown,
`Int
`tional Busi
`Machi
`sien’ Corporation ArmonkNY.(US).
`(73)
`.
`(*) Notice:
`Subject to any disclaimer,the term ofthis
`patent is extended or adjusted under 35
`USS.C. 154(b) by 447 days.
`
`1/1999 Dunmetal. oe 725/87
`5,861,906 A *
`5,963,191 A * 10/1999 Jaaskelainen, Jr.
`.......... 345/856
`ve 345/788
`6,075,531 A *
`6/2000 DeStefano .......
`
`CORDBL SO00L HanSt ak en SES
`
`beve
`2.0.00...
`5282.5
`*
`p
`isch
`y
`
`........
`| 345/857
`6,480,211 Bl
`* 11/2002 Masri
`
`6,483,509 BL * 11/2002 Rabenhorst....
`. 345/442
`OTHER PUBLICATIONS
`Screen Dumps of Microsoft Publisher (pp. 1-4, 1999).*
`* cited by
`examiner
`.
`¥
`.
`a ,
`Primary Examiner—Knistine Kincaid
`Assistant Examiner—Le Nguyen
`(74) Attorney, Agent, or Firm—Robert H. Frantz; David A.
`Mims,Jr.
`ABSTRACT
`67)
`A rubberbandvariable-rate GUI control for use in conjunc-
`tion with GUI controls, such as sliders, scroll bars, virtual
`joysticks, and oppositional buttons (up/down,
`left/right,
`plus/minus, next/previous, etc.). A user mayplace a pointer
`or cursor over a GUI control, select (e.g. click and hold a
`mouse button) and drag away from the control, and a virtual
`rubber band extending between the pointer and the GUI
`control is displayed. The further the user moves or “pulls”
`the pointer from the GUI control,
`the thinner and more
`elyptical the rubber band is displayed, and the faster the
`selections scrolls or the control operation is repeated. When
`the user movesthe pointer closer to GUI control, the rubber
`bandis displayed fatter and more circular in shape, and the
`scrolling or control operation repetition slows.
`
`30 Claims, 6 Drawing Sheets
`
`(21) Appl. No.: 09/615,771
`(22)
`Filed:
`Jul. 13, 2000
`7
`” ie “a eesagesasnet
`(52) Onsr -;
`(58) Field of Search oo. ceeeeee 345/684, 688,
`345/784-7187, 830, 973
`References Cited
`U.S. PATENT DOCUMENTS
`
`(56)
`
`5,495,566 A *
`2/1906 Kwatinetz 000. 345/785
`5,508,717 A *
`vec 345/858
`4/1996 Miller
`
`5,615,367 A *
`3/1997 Bennett et al.
`............. JO7/102
`5,634,095 A *
`5/1997 Wang et al. oo... 345/763
`5,664,128 A *
`.. 345/708
`9/1997 Bauer.......
`
`5,721,847 A
`.. 395/333
`2/1998 Johnson ....
`
`5,790,115 A
`_ 345/327
`8/1998 Pleyeretal.
`5,838,326 A * 11/1998 Card et al. oo. cccccee. 345/775
`
`201
`
`Booking, Exh. 1052, Page 1
`
`Booking, Exh. 1052, Page 1
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 1 of 6
`
`US 6,677,965 Bl
`
`OL
`
`MyTOI
`
`Deeeeeeeeeneneneeeeenee’=é
`
`
`QSTAOC]Poseg
`EOL>ZOT
`-Jainduto,)Joayndwog
`
`ee eeeeeeeeeeeeee ene seenee4
`
`orempley]
`
`QEMYOS
`
`suneladg
`
`WId1sAg
`
`_9l
`
`JOUINU]
`
`Jo
`
`oueyUyT
`
`[santig
`
`Booking, Exh. 1052, Page 2
`
`Booking, Exh. 1052, Page 2
`
`
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 2 of6
`
`US 6,677,965 B1
`
`
`
`
`Example.exe
`
`PriorArt
`
`Help
`Special
`
`View§[|Program:
`FileEdit
`
`
`
`Figure2
`
`26
`
`28201
`
`Booking, Exh. 1052, Page 3
`
`Booking, Exh. 1052, Page 3
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 3 of 6
`
`US 6,677,965 Bl
`
`
`
`WALIGALOATASATLNAYEND
`
`‘e010yDyuan
`
`HVOlid
`
`
`
`
`
`gsantig
`
`Vytold
`
`Booking, Exh. 1052, Page 4
`
`Booking, Exh. 1052, Page 4
`
`

`

`ty
`
`
`
`Ppainsi
`
`
`
`[i|Aj|wantATLNaeuNd|:20!0N9aaioatas
`
`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 4 of 6
`
`US 6,677,965 Bl
`
`a7
`
`IpauNsI4
`
`Booking, Exh. 1052, Page 5
`
`Booking, Exh. 1052, Page 5
`
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 5 of 6
`
`US 6,677,965 B1
`
`201
`
`
`Booking, Exh. 1052, Page 6
`
`Booking, Exh. 1052, Page 6
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 6 of 6
`
`US 6,677,965 Bl
`
`henencatemnnennncannnnannnannanennenOT
`
`
`
`poyeos)10j0e.]
`
`JeUI]-UON
`
`(Ayrun0}
`
`
`puegJoqqny
`josuey
`
`xeur70
`
`9snbIE
`
`Booking, Exh. 1052, Page 7
`
`Booking, Exh. 1052, Page 7
`
`
`

`

`US 6,677,965 B1
`
`1
`RUBBER BAND GRAPHICAL USER
`INTERFACE CONTROL
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is related to U.S. application Ser. No.
`09/578,748, filed on May25, 2000, by Cristi Ullmann,et al.;
`and to U.S. application Ser. No. 09/579,257, filed on May
`25, 2000, by Allen Wynn,et al.; and to U.S. application Ser.
`No. 09/578,749, filed on May 25, 2000, by Allen Wynn,et
`al; all of which arc commonly assigned.
`
`FEDERALLY SPONSORED RESEARCIT AND
`DEVELOPMENT STATEMENT
`
`This invention was not developed in conjunction with any
`Fedcrally-sponsored contract.
`
`MICROPICIIE APPENDIX
`
`Not applicable.
`
`INCORPORATION BY REFERENCE
`
`The related applications, all filed on May 25, 2000, now
`Ser. Nos. 09/578,748; 09/578,749; 09/579,257 by Cristi
`Ullmann,et al.; Allen Wynn,et al; and Allen Wynn,et al.,
`respectively, are incorporated herein by reference in their
`entirety, including drawings, and hereby are made a part of
`this application.
`BACKGROUNDOF THE INVENTION
`
`1. Field of the Invention
`
`This invention relates to the arts of graphical user inter-
`face controls provided on computer displays, and to related
`displays such as personal digital assistanis (“PDA”) and
`wireless communications devices.
`
`40
`
`2
`(“NIC”) (17) or wireless communications interface. Com-
`monly used NICs include EtherNet
`local area network
`(“LAN”) cards, dial-up modem cards, and wireless commu-
`nications circuits. Some of these circuits may be provided
`integral to the device (10), such as the case with web-
`enabled cell phones, or may be available as add-in circuils
`such as PCI cards or PCMCIA cards for personal computers.
`Additional typical hardware provided in the system (10)
`includes a group of user input/output (“I/O”) (16) devices
`(13), such as a display, keyboard, and/or pointing device,
`accompanied by commonuserI/O interface circuits to allow
`the CPU and system software to access and use the user I/O
`devices (13). Common display devices include VGA and
`LCD monitors for personal computers, and LCD panels for
`PDA and cell phones. Common pointing devices include
`“arrowkeys”, “tab” keys, mouse and scroll-mouse, joystick,
`track balls, glide point
`touch sensitive pads, and touch
`sensitive displays with stylus pens. Some devices, such as
`certain PDA’s, do notinclude an actual keyboard, but rather
`rely upon a touch-sensitive LCD display with virtual point-
`ing device and stylus.
`The software functionality of computer-based systems
`(10) can be divided into low-level device drivers and BIOS
`(18), an operating system (103), and applications programs.
`Low-level device drivers provide hardware-specific inter-
`face functions which allow the operating system and appli-
`calion programs to access ihe system hardware through
`commonor generic application program interfaces (“API”).
`Some device drivers include necessary communications
`protocols and methods associated with a particular hardware
`device, such as a modem or wireless communicationsinter-
`ace. Application programs can be further divided into
`portable applications, or applets (101), a portable application
`program interpreter (19), and non-portable, system-
`dependentapplications (102). Such division of hardware and
`software functionality is well-understood in the art, and can
`be applied to large computer systems such as an IBM
`2. Description of the Related Art
`AS/400 equally as well to small computer systems such as
`PDA’s running Windows CE.
`Graphical user displays on electronic devices such as
`computers, personal digital assistants, and wireless commu-
`On the system display is typically shown a conventional
`nications devices are commonin the art. As these devices
`rame or “window” of information related to a specific
`become more and more integral to activities in the daily lives
`system function or program, such as a word processor,
`of people such as web browsing, word processing, and even
`spreadsheet, to-do list, address book, e-mail client, or web
`making telephone calls, the efficiency and ergonomics of
`browser. FIG. 2 showsa typical arrangementof a frame (20)
`these interfaces are extremely important. A user will quickly
`s includingatitle bar (21), tool bar (24), display area (25), and
`notice whether one uscr interface is casicr and quicker to usc
`a pointer (201). The pointer (201) movesin response to the
`than another based upon the number of operations required
`system pointing device and/or keyboard, such as typing on
`to perform a certain task. Further, the level of “intuitiveness”
`he arrow keys, movementof a mouse,track ball, or joystick.
`implemented in the user interface can greatly increase the
`The title bar may include “buttons” for minimizing or
`usefulness and speed with which the user can operate the
`restoring the frame (23), closing the program (22), as well as
`device.
`a display of the name of the function or program. The tool
`bar (24) typically includes a number of text and/or iconic
`options, such as “file” and “edit” drop-downlists.
`The display arca may be further navigable using vertical
`scroll (27) and horizontal scroll (26) bars, which are espe-
`cially useful
`for system displays of information which
`cannot be shown completely within one frame on the
`display. “Clicking” on the right button (29) or left button
`(28), or “dragging” the horizontal position indicator (201)
`causes the information in the display area (25) to be panned
`or scrolled left or right. Likewise, using the up button (29),
`down button (28), or vertical position indicator (200) can
`cause panningor scrolling in a vertical motion. These types
`of scroll bars arc common on word processors, sprcadshcets,
`graphical design packages,and web browsers.
`In typical operation of these types of GUI displays, the
`user moves the pointer (201) to a button ortext option, and
`
`Manygraphical user interfaces follow the same conven-
`tions for navigation of and selection of controls on the user
`interface. Such commongraphical user interfaces (“GUI’)
`include windows or frames-type displays in Microsoft’s
`Windowsoperating system, UNIX operating system, IBM’s
`OS/2 and AIX operating systems, Microsoft’s Windows CE
`operating system, Apple Computer’s operating system, and
`many “proprietary” windows-like GUI’s on various devices
`such as PDA’s, cell phones, electronic organizers, etc.
`FIG. 1 shows the basic organization of manyprior art
`computer systems and computer-based devices. The system
`(10) typically includes a microprocessoror central processor
`unit (“CPU”) including computer memory, and may also
`include interfaces (15) to hard disks and removable disk
`media (14), and possibly interfaces to computer or commu-
`nications networks (11) such as a network interface card
`
`
`
`
`
`60
`
`Booking, Exh. 1052, Page 8
`
`Booking, Exh. 1052, Page 8
`
`

`

`US 6,677,965 B1
`
`4
`viewfinder of the camera, the user would repeatedly operate
`the up/down and left/right controls until
`the subject
`is
`centered, using a few long duration movesto find the general
`area of the subject, and manyshort “bursts” to fine position
`the subject. If the subject is initially far from the current
`view of the camera (or frame), this can take very long to pan
`the subject
`into the frame. To compensate for this,
`the
`constant panning speed may be increased, which makesit
`more difficult
`to achieve the fine positioning within the
`frame. In the real world, this kind of problem, such as for
`remotely controlled security cameras,
`is solved by using
`variablc-ratc panning controls, such as potentiometers, but
`the problem is prevalent
`in computer frame-based GUI
`environments.
`
`Therefor, there is a need in the art for a virtual control
`method which allows for intuitive variable-rate control of
`common GUI control types. This new control method is
`preferably compatible with a wide range of existing GUI
`controls and the methodof rate variation indicationis easily
`recognizable in order to enhance the efficiency and intu-
`itiveness of the new method.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`3
`selects that option by clicking on it and/or dragging it. In
`some cases, keyboard-only navigation or selection is pos-
`sible by using special combinations of keys, such as ALT-I'
`to select the “file” drop-down menu. FIGS. 3a through 3c
`show a typical slider or scroll control (30),
`typical
`list
`control (34) with up/down choice buttons, and a virtualjoy-
`stick (38) which is described in the related application. The
`slider control (30) typically includes a slider control button
`(31), and two oppositional movement buttons (32 and 33).
`Similarvertical slider controls are also well knownin theart.
`The typical
`list control
`(34) includes a display of the
`currently selected list item (35), up/down (next/previous,
`etc.) choice buttons (36, 37), and may include a drop-down
`list of possible choices. The virtual joystick (38) includes a
`stick portion (39) which can be movedbyselecting it with
`the pointer and moving the pointer control device, such as a
`mouse, left, right, up or down.
`The article “Multistream input: An experimental study of
`document scrolling methods” in the IBM Systems Journal
`Vol. 38, No. 4, 1999, “investigates alternative methods to 4,
`improve user performance for browsing World Wide Web
`and other documents.” This investigation involved a task
`requiring both scrolling and pointing, and compared the
`following input devices:
`(a) standard mouse, (b) wheel
`‘The following detailed description when taken in con-
`mouse, (c) joystick mouse (combination Standard Mouse
`junction with the figures presented herein present a complete
`with a joystick), and (d) trackpoint(such as found on an IBM
`description of the present invention.
`ThinkPad computer) with supplemental Standard Mouse.
`FIG. 1 showsthe prior art arrangement of computers and
`The results of the experimental study showed that a
`computer-based devices with graphical user interfaces.
`mouse with a finger wheel did not improve a user’s perfor-
`mance or subjective rating over a standard mouse. Indeed,
`FIG. 2 presents an example GUI frame.
`the participants commentedthat it was tedious andtiring to
`FIGS. 3a through 3c illustrates several GUI controls,
`repeatedly roll the wheel, although this mode wasintuitive.
`includingaslider bar, a 2-dimensional virtual joystick, and
`The researchers believed that the lack of “self-centering”
`list up/downbuttons.
`placed the wheel mouseat a fundamental disadvantage to do
`FIGS. 4a through 4e show a rubber band control applied
`effective rate control. The results also indicated that
`the
`to several GUI controls,
`including a slider bar,
`a
`Joystick Mouse and the Trackpoint with supplemental Stan-
`2-dimensional virtual joystick, and list up/down buttons.
`dard Mouse performed better than the Standard Mouse
`FIG. 5 illustrates a new type of GUI control, a “fixed
`alone, both in terms of performance and user perception.
`slider’, usable with the new rubber band control.
`“Self-centering” is a very useful attribute of an input device
`TIG. 6 shows an sample nonlinear function for use in
`when scrolling through documents, maps, etc. In addition,
`determining the nonlinear response to the position of the
`there are some instances where the user may wish to scroll
`joystick.
`in a direction other than directly up, down,left or right. For
`example, a user may wantto scroll a map at an angle towards
`a destinationcity.
`In the related patent applications, a virtual joystick, an
`infinite spinlist and a spinlist with preview and postviewlists
`were disclosed. All of these new GUI controls enhance the
`efficiency and the intuitiveness of graphical user interfaces.
`However, a intuitive control for rate of change is needed
`beyond that currently found in the art. For example, when a
`user is panning around a large documentin framed display,
`only a portion of the whole document may be displayed and
`viewed at a time. Repeated clicking on up/down andleft/
`right icons, grabbing a sliding of slider buttons, and grab-
`bing a tilting of virtual joysticks do not provide intuitive
`interaction as one may find in the “real world” through
`similar viewing arrangements. An example of such a real
`world viewing arrangementis viewing part of a total scene
`through the viewfinder of a camera. When manually moved
`to find a particular subject in the scene, a human user will
`movethe camera quickly until the general area of the subject
`is found, and will slow movement until
`the subject
`is
`properly centered in the view finder. To draw an analogous
`comparison to present day GUI controls and cameras, one
`can imagine a camera mounted to a motorized tripod, with
`the motor controls only having a single speed of up/down
`control and left/right control. To center a subject in the
`
`conjunction with existing GUI controls, such assliders,
`scroll bars, virtual joysticks, and oppositional buttons (up/
`down,left/right, plus/minus, next/previous, etc.). The rubber
`band method allows for a user to place a pointer or cursor
`over a GUIcontrol, and select (e.g. click and hold a mouse
`button) and drag away from the control, at which time a
`virtual rubber band extending between the pointer and the
`GUI control is displayed. The further the user moves or
`“pulls” the pointer from the GUI control, the thinner and
`more elyptical the rubber band will be displayed, and the
`faster the selections will scroll or the control operation will
`be repeated. When the user movesthe pointer closer to GUI
`control, the rubber band will be displayed fatter and more
`circular in shape, and the scrolling or control operation
`repetition will slow. If the pointer is moved past the point of
`origin for the GUI control, the rubber bandis displayed from
`the other side of where the pointer started and scrolling will
`reverse. De-sclection of the control, such as by releasing a
`mouse button, causes the rubber band displayto be removed,
`and operation of the GUI control is ceased. The rubber band
`control operation may be vertical, horizontal, or a combi-
`nation of both, on the display. A new variation of a slider
`
`i}wn
`
`3
`
`ey}wn
`
`40
`
`60
`
`SUMMARYOF THE INVENTION
`
`The rubber band variable-rate GUI control is used in
`
`Booking, Exh. 1052, Page 9
`
`Booking, Exh. 1052, Page 9
`
`

`

`US 6,677,965 B1
`
`5
`control, called a “fixed slider”, which is enabled by the use
`and existence of the rubber band control is also disclosed.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The foregoing and other objects, features and advantages
`of the invention will be apparent from the following more
`particular description of a preferred embodiment of the
`invention, as illustrated in the accompanying drawings
`wherein like reference numbersrepresent like parts of the
`invention.
`
`The invention is preferably realized as a user graphical
`interface control or dialog object integrated into a function
`or application program on a compuler or computer-based
`system. The computer-based system is preferably provided
`with an operating system which provides common GUI
`control functions, such as pointer location and movement
`detection, event handling based uponareas onthe display in
`which the pointer is located, and event handling of pointer
`selection such as mouse clicks. As such, the invention can be
`realized on personal computers, palm-top computers and
`personaldigital assistants, as well as other devices incorpo-
`rating graphical user displays such as cell phones and
`wireless web browsers.
`
`6
`pulling the rubber band further from either the up or down
`buttons would cause faster sequencing through the list
`choices. FIG. 4e showsuse of the rubber band on a virtual
`joystick (44, 45), which was described in the related
`application,to allowrate control of the joystick in one or two
`variables.
`
`5
`
`In FIG. 5, a newtype of control not previously possible
`but cnabled by the rubber band control is shown. Whilc this
`control (50) has a basic appearance of a scroll or slider
`control, the center button (51) is “pinned” or fixed at the
`center of the control. Clicking onit (or otherwise selecting
`it) and “stretching” the rubber band (42) causesit to operate
`in either direction at varying rates depending onthe distance
`drawn between the pointer and the center button (51). The
`userscrolls by “tugging”the scroll button. The remainder of
`the control (up/down arrows at end, and empty portions of
`scroll bar) function exactly like a normal scroll-bar. This can
`of course be applied to a vertical scroll bar or control, as
`well. As disclosed previously, de-selection of the control
`causes the rubber band display to be removed and operation
`of the control to stop.
`
`Rubber Band Software Control
`
`i}wn
`
`3
`
`ey}wn
`
`40
`
`‘The following disclosure sets forth the invention as a
`Theresult of the stretching of the rubber band between the
`modification or enhancementto a pointer display and control
`pointer and the control can bealinear or nonlinear relation-
`in a graphical user interface, which allows it to be used and
`ship to the distance between the pointer and the control. For
`compatible with existing GUI controls. However, it will be
`example, for each unit of distance increase between the
`readily recognized by those skilled in the art that the method
`pointer and the control (the length of the rubber band), a unit
`of the invention can be directly incorporated in to each GUI
`of rate change can be increased, for a lincarrelationship. For
`control as well. The rubber band control operation maybe
`a nonlinear relationship, each unit increase in the distance
`vertical, horizontal, or a combination of both, on the display,
`between the pointer and the control (the length of the rubber
`and mayinclude 1, 2 and 3 dimensional or variable opera-
`band) and cause increasingly more rate change. FIG. 6
`tion.
`shows an example nonlinear function for possible use in
`The rubber band variable-rate GUI control is used in
`implementing a nonlinear rubber band control.
`conjunction with existing GUI controls, such as sliders,
`In the preferred embodiment, the rubber band control is
`scroll bars, virtual joysticks, and oppositional buttons (up/
`implemented as a “control of a control”. In other words,
`down, left/right, plus/minus, next/previous, etc.), and with
`instead of the pointer directly operating a GUI control, such
`new types of GUI controls such as a “fixed scroll button” or
`as a slider control button,
`the rubber band control will
`“fixed slider”, which are made possible by the rubber band
`operate the control. So, the rubber band control software is
`control.
`activated when the pointer is moved in the vicinity of the
`GUI control and when the GUI control button has been
`selected. As the user moves the pointer, the rubber band
`control software receives pointer control status, such as
`mouse location, movement and mouse button states. The
`rubber band control software then processes the pointer
`control device data, and outputs data directly to the GUI
`control software to emulate normal inputs from the mouse.
`TABLE1 showsthe pseudocodeto realize this control flow,
`which can be implemented by those skilled in the art using
`any appropriate programming language and methodology
`suitable for the computing platform target, such as “C’,
`“C++”, Microsoft Windows CE,or Java.
`
`Operation of the Rubber Band Control
`The rubber band control allows a user to place a pointer
`or cursor over a GUI control, and select and drag away from
`the control, at which time a virtual rubber band extending
`between the pointer and the GUI control is displayed, as
`shown in FIGS.
`4a@ through 4f. FIG. 4a showsa typical
`horizontal slider or scroll control (40) with the rubber band
`display (42) between the pointer (201) and the scroll button 5
`(41). The further the user movesor “pulls”the pointer (201)
`from the GUI control,
`the thinner and more elyptical in
`shape the rubber band will be displayed, and the faster the
`selections will scroll or
`the control operation will be
`repeated, as shown in FIG. 4b. When the user moves the
`pointer closer to GUI control,
`the rubber band will be
`displayed fatter and more round in shape, and the scrolling
`or control operation repetition will slow. If the pointer is
`moved pass the point of origin for the GUI control,
`the
`rubber band is displayed from the other side of where the
`pointer started and scrolling will reverse, as shown in FIG.
`4c. The the control is de-selected, such as by releasing the
`mouse button or lifting a touch-screen stylus, the rubber
`band display is removed and opcration of the control is
`ceased.
`FIG. 4d illustrates how the rubber band control can be
`
`60
`
`applied to a list control with up/down buttons (43), which
`
`TABLE 1
`
`Rubber Band Control Pseudocode
`
`While pointer has selected control
`get position of pointer “ptr_pos”
`get position of control “ctl_pos”
`calculate distance D between pointer and control
`wait_time = function of distance D
`band_thick = function of distance D
`band shape = function of distance D
`draw rubber band with thickness and shape from plt_poslo cll_pos
`operate the targeted control one time (i.e. emulate a click on the
`control)
`
`Booking, Exh. 1052, Page 10
`
`Booking, Exh. 1052, Page 10
`
`

`

`7
`
`TABLE1-continued
`
`Rubber Band Control Pseudocode
`
`wait for wait_time
`End While
`
`US 6,677,965 B1
`
`8
`
`TABLE 3
`
`Wait Time
`
`the control software
`In the pseudocode of TABLE 1,
`retrieves the current pointer position and control button
`positions, cithcr through a function call to the opcrating
`system, passing in of variables, or accessing global vari-
`ables. Many systems will represent these positions as x-y
`coordinates, often in units of pixels or PELS, either in
`absolute terms (from a fixed point on the display) or frame-
`relative terms (from a fixed point in the frame).
`
`The distance between the pointer and the control button
`can be found simply by applying the Pythagorean theorem
`between the x-y coordinates of the pointer and the x-y
`coordinates of the control button, by calculating the square
`root of the sum of the squared difference in the x positions
`and the squared difference in the y positions, otherwise
`known as the Pythagorean theorem:
`
`D=SQRT {(Xpornter—Xcontrol_buson)+0pointercontrolbutton)$
`
`Then, this distance value can be used to determine the
`display thickness and amount of elyptical shape of the
`rubberband,and the time to wait for the next position update
`and control operation. TABLE 2 shows an example to
`determine the rubber band thickness, which can be easily
`implemented as a look up table or code function. It allows
`a numberofdistance ranges, delimited by C, constants, to set
`several discrete values of the thickness T; of the rubber band
`displayed. “Roundness”of the shape of the rubber band can
`be indexed to T, or determined byan altcrnate function or
`table.
`
`TABLE 2
`
`Rubber Band Thickness
`
`Distance (pel)
`o<DSC,
`O©<DECG,
`G<DEG,
`
`Thickness (pel)
`T,
`T,
`T;
`
`Cava <D S Cras
`
`Trax
`
`TABLE3 showsan example function with both linear and
`nonlinear (in this case accelerating) response to the wait
`time based upondistance, which can also be implemented in
`a variety of manners, such as a look up table or software
`function.In the example of TABLE3, W,,,, is a standard wait
`value. For a linear relationship, the time waited between
`each successive loop through the code and operation of the
`control is inversely proportional to the distance between the
`pointer and the control. Thus, the greater the distance, the
`faster the operation of the control. For a nonlinear response,
`any appropriate function or table of data can be used, such
`as the inverse hyperbolic relationship shown in TABLE3.
`
`10
`
`40
`
`60
`
`Distance (pel)
`0<DECG,
`G<DSC,
`C;<DEqQ
`
`Linear
`Wail (msec)
`Wora/Cy
`WaralCr
`Wora/Cs
`
`Nonlinear
`Wail (msec)
`Woa/D
`Wa/D
`Wea/D)
`
`Coax 1 < DS Cnax
`
`Wora/Cmax
`
`Wara/D)
`
`Finally, in the code of TABLE 1, the actual control is
`operated once, such as emulating a click on the up, down,
`left or right button, by the rubber band control code, thus
`causing one step of action on the targeted GUI control. Then,
`the wait is executed, and the loop is repeated if the control
`is still selected.
`
`New Types of Controls Enabled by the Rubber
`Band Control
`
`Whereasprior art GUI controls are defined based uponthe
`need to display movementof the control, the advent of the
`rubber band display and control allows definition of new
`controls which do not themselves move. Suchas control is
`a “fixed scroll bar’, as shown in FIG. 5. This control
`operates and appears muchthe sameas a normalscroll bar,
`with operable left and right buttons. But, the center button
`(51) does not move and is “pinned” in place. Pulling or
`tugging on the center button with the rubber band control
`would operate the scrolling action as if the button were
`actually moving. This allows the scroll control to be dis-
`played in muchless display space, while it
`is intuitively
`obvious to a user how to operate it. Conservation of display
`space is increasing important as the display space available
`on miniaturized computer-based devices such as PDAs and
`cell phones is very limited, and as full-size GUI computer
`displays become more crowded with control and short cut
`icons.
`
`Summary
`
`The rubber band GUI control element as disclosed pro-
`vides enhanced control of GUI-based computer applications.
`Oneprimary advantageis that it can be used in conjunction
`with existing GUI controls to give improved, variable rate
`operation of those controls. Another advantage is that
`it
`allows definition of new types of immovable GUI controls
`which conserve displayspace. It will be understood from the
`foregoing description that various modifications and
`changes may be made in the preferred embodiment of the
`present invention without departing from its true spirit, such
`as the use of alternate programming methodologies or
`languages, alternate computer platforms and software, oper-
`ating systemsanduser control objects. It is intendedthat this
`description is for purposesof illustration only and should not
`be construed in a limiting sense. The scope of this invention
`should be limited only by the language of the following
`claims.
`Whatis claimedis:
`1. A method for interfacing with a user of a computer
`system having a graphical user display, the method com-
`prising the steps of:
`(a) controlling a pointer element on the graphical user
`display with a user pointer controller, the pointer con-
`troller having position and selection status responsive
`to operation by a user;
`
`Booking, Exh. 1052, Page 11
`
`Booking, Exh. 1052, Page 11
`
`

`

`US 6,677,965 B1
`
`9
`(b) displaying on said graphical user display a control
`icon having one or more action areas, said action areas
`comprising at least one forward action area or one
`backward action area;
`(c) monitoring said position and selection status of said
`pointer controller to detect if one of said action areas
`has been selected by said user;
`(d) altering the appearance of said control icon and said
`pointer to visually indicate the sclection of said action
`area by displaying a virtual rubber band onsaid graphi-
`cal user display, said virtual rubber bandencircling said
`action area and said pointer element; and
`(e) allowing repetitive operation of said action area while
`said selection is detected by autonomously repeatedly
`periodically generating a signal to said computer sys-
`tem indicative of a selection made bysaid user of said
`control icon action area.
`2. The method of claim 1, wherein said forward action
`area is presented as a button.
`3. The method of claim 1, wherein said backward action
`area is presented as a button.
`4. The method of claim 1, wherein said action areas
`comprise a slider action area.
`5. The method of claim 4, wherein said backward action
`area is presented as a scroll bar control button.
`6. The method of claim 1, wherein said action areas
`comprise a virtual joystick action area.
`7. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the rate of periodic generation ofa signal relative
`to a determined distance between the pointer element and
`the control icon.
`8. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the apparent thickness of the virtual rubber band
`relative to a determined distance between the pointer ele-
`ment and the control icon.
`9. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the apparent oval shape of the virtual rubber band
`relative to a determined distance between the pointer ele-
`ment and the control icon.
`10. The method of claim 1, wherein said step of display-
`ing on said graphical user display a control
`icon further
`comprises displaying a slider bar control icon having two
`oppositional action areas and a virtual slider button, said
`virtual slider button being fixed in place and operable bysaid
`virtual rubber band.
`11. A computer-readable storage medium having stored
`therein computer-readable program codefor interfacing with
`a user of a computer system, the computer system having a
`graphical user display and a processor, wherein the progr