United States Patent (19)
`Bryan, Jr. et al.
`
`|||||||||||
`USOO5559301A
`5,559,301
`Patent Number:
`Sep. 24, 1996
`Date of Patent:
`
`11
`45
`
`54) TOUCHSCREEN INTERFACE HAVING
`POP-UP VARIABLE ADJUSTMENT
`DISPLAYS FOR CONTROLLERS AND
`AUDIO PROCESSING SYSTEMS
`
`75)
`
`Inventors: Marcus K. Bryan, Jr.; Alexander J.
`Linberis; John S. Bowen, all of San
`Jose; Daniel A. Phillips, Palo Alto, all
`of Calif.
`
`73 Assignee: Korg, Inc., Tokyo, Japan
`
`21)
`22)
`(51)
`
`52
`
`58)
`
`56
`
`Appl. No.: 306,365
`Filed:
`Sep. 15, 1994
`Int. Cl. ........................ G10H 1/057; G10H 1/06;
`G10H 1708; G1OH 1/46
`U.S. Cl. ................................. 84/653; 84/659; 84/660;
`84/663; 84/665; 34.5/173; 395/159
`Field of Search .............................. 84/600, 615-620,
`84/622-633, 653–665, 477 R, 478,462;
`34.5/173-178; 395/159
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,027,689
`5,146,833
`5,153,829
`
`7/1991 Fujimori ............................... 84f659 X
`9/1992 Lui .................
`... 84,477 RX
`10/1992 Furuya et al. .................... 84,477 RX
`OTHER PUBLICATIONS
`Rubine, et al., "Programmable Finger Tracking Instrument
`Controllers'; Computer Music Journal, vol. 14, No. 1,
`Spring, 1990.
`
`Primary Examiner-Stanley J. Witkowski
`Attorney, Agent, or Firm-Haynes & Davis
`
`(57)
`ABSTRACT
`A touchscreen interface for a sound processing system, such
`as music synthesizers, which has a display panel and a touch
`Sensitive panel overlying the display panel, includes an icon
`which represents an adjustable parameter used by the pro
`cessing system. The processing resources supply a variable
`adjustment display to the display panel in response to a
`touch on the position of the icon, using pop-up slider or
`pop-up knob motif. The variable adjustment display overlies
`the interface display and has a size on the touch sensitive
`panel larger than the size of the icon to facilitate manipu
`lation of the variable using a finger over a significant range
`of values. The variable adjustment display pops up when
`touched to obscure a portion of the graphical display used
`for the interface. When the variable is adjusted using the
`touch sequence, the variable adjustment display is removed,
`and the interface display is left unobscured. This allows the
`user to manipulate a particular variable while maintaining
`the window which shows the values of related variables on
`the screen. By maintaining the current window on the
`screen, the user is less likely to get lost in a hierarchy of
`windows used for setting variables.
`
`44 Claims, 6 Drawing Sheets
`
`
`
`
`
`
`
`
`
`Per For more
`Banki> O Basic Sounds
`
`(2) 13 Jack Chorus
`T Layered bells and choir pad,
`Wheel 1 lorings in Fernale vocals,
`Delay time is affected loy velocity,
`
`
`
`50
`
`51
`
`
`
`S3
`
`DN-SCREEN
`SIDERS
`52
`
`
`
`
`
`
`
`6
`5
`
`Group A O Group B O Group C O Group D
`Sliders
`Voice Wolce Bell
`Bell
`Voice
`Rverko Chorus
`Bell
`Attack Attack Bright Tirore Volume Release Depth Depth
`+O1
`-72
`-68
`$5
`+32
`-64
`+30
`--71
`
`<---'
`4
`
`5
`
`6
`
`l th t
`
`7
`
`8
`
`Petitioner Exhibit 1007, Page 1
`
`

`

`Sheet 1 of 6
`
`5,559,301
`5,959,301
`
`coOld
`
`.OvI“
`
`SESIG
`
`SAIN
`
`awoOSS3Io0dd
`
`oanv
`
`ve
`
`ce
`
`‘ALSNI
`
`AYOWSW
`
`dNdOd-
`
`a4aqrS
`
`-3NILnOa
`
`2€
`
`ONINAOM
`
`AYOWAW
`
`BE
`
`
`O/LNSHLOAW14810
`LE9
`
`qaNVd
`
`G¥dHONOL
`
`Petitioner Exhibit 1007, Page 2
`
`U.S. Patent
`U.S. Patent
`
`Sep. 24, 1996
`
`
`
`
`
`
`
`
`
`
`
`
`
`Petitioner Exhibit 1007, Page 2
`
`
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 2 of 6
`
`5,559,301
`
`0S
`
`TS
`
`oc/*valvddn.
`
`abDldtejk=lFR
`qdnougQd)JQdnowgd
`
`SNUOYUAUarg
`Ce99
`"9|L|
`
`STWAL89.-grwva-OVld"?
`salonnedned
`E9;69
`‘AqinoyanAqpazraygoSiawyyAvjag
`'S}ODOAayoWwayulSGujuqJjeauA!
`
`SNuoyug4207€TEl
`
`
`‘podujoyudpuos)aqpauako7[1}:
`spunoszIsvgo[<]pug|QADUDWUO
`
`ad0Aadlon
`4FUad
`
`qvOldaeaeapOldeeVPOI
`
`gqdnouyOC)
`y¥dnouy©
`SAPS
`
`&Old
`
`€S
`
`N33Y49S-NO
`
`SaaaTis
`
`2S
`
`#HINOLs
`
`Petitioner Exhibit 1007, Page 3
`
`Petitioner Exhibit 1007, Page 3
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 3 of 6
`
`5,559,301
`
`PPUP SLIDER FLOWCHARTS
`
`OC
`
`SET CURRENT EVENT
`CFIG6)
`
`EVENT A
`SPOPUP SLIDER
`EVENT
`
`
`
`O2
`
`SET TIMER T A
`VALUE X=0
`
`
`
`SET CURRENT EVENT
`(FIG,6)
`
`O1
`
`O3
`
`O4
`
`105
`
`S
`CURRENT
`EVENT A
`TIMEDUT
`EVENT
`
`
`
`
`
`EVENT A
`TUCH UP
`EVENT
`
`N
`
`
`
`106
`
`SHDW POPUP SLIDER
`
`107
`
`SET CURRENT EVENT
`CFIG.S.)
`
`IS
`CURRENT
`EVENT A
`"TOUCH UP!
`EVENT
`
`108
`
`
`
`
`
`O9
`
`CLSE POPUP
`
`
`
`- REQUEST PARAMETER
`VALUE UPDATES
`
`FIG5
`
`Petitioner Exhibit 1007, Page 4
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 4 of 6
`
`5,559,301
`
`CURRENT EVENT FLWCHART
`
`
`
`START
`
`
`
`120
`
`WAIT FR BUFFER
`DR TIMEUT
`
`122
`
`
`
`125
`
`IS
`IT A
`BUFFER
`
`ND
`
`SET CURRENT EVENT
`D
`IME OUT
`
`FIND TOUCHED OBJECT
`IN THE DISPLAY
`
`
`
`129
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PARAMETER
`VALVE
`BUFFER
`
`IS N
`CURRENT
`PARAMETER
`A SLIDER
`
`SHW CURRENT
`PARAMETER VALUE
`FOR SLIDER
`CFIG.9)
`
`
`
`
`
`IS
`SLIDER
`TUCHED
`
`IS
`ND-1DUCH ZDNSND
`
`IS
`TUCH
`STATUS
`Up
`
`
`
`
`
`
`
`SET CURRENT
`EVENT TO
`"POPUP SLIDER
`
`SET CURRENT
`EVENT TO
`"TOUCH UP
`
`PROCESS TOUCH
`FR SLIDER
`(FIG,8)
`
`
`
`FIG6
`
`Petitioner Exhibit 1007, Page 5
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet S of 6
`
`5,559,301
`
`SLIDER FLWCHARTS
`
`SHW CURRENT
`(B) PARAMETERVALUE
`FDR SLIDER
`160
`DRAW SLIDER
`BACKGROUND T
`ERASE HANDLE
`
`CALCULATE NEW HANDLE1
`POSITION USING
`PARAMETER VALUE
`AND IT MAXIMUM
`AND MINIMUM RANGE
`
`DRAW ANDLE
`
`162
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`163
`
`FIG.9
`
`GA) SLIDER TUCH
`PROCESSING
`140
`
`141
`
`N
`
`IS
`TOUCH STATUS
`DOWN
`1.
`
`
`
`YES
`
`DONE
`GDDNE)
`
`142
`
`CURRENT PARAMETER = SLIDER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS
`SLIDER
`HANDLE
`TUCHED
`
`
`
`
`
`YES
`
`143
`
`N
`
`144
`ONE
`GDONE
`
`145
`
`146
`
`SEERNNO save Touch
`
`WAS
`
`PREVIOUSLY
`1.
`YES
`
`LCATION
`
`DNE
`
`CALCULATE DISTANCE
`HANDLE MOVED FRM
`INITIAL TOUCH DCATION
`
`147
`148
`
`CAL CULATE NEW PARAMETER
`VALUE USING MINIMUM
`AND MAXIMUM RANGE. F.
`PARAMETER AND THE
`DISTANCE THE HANDLE
`MOVED
`
`SEND PARAMETER CHANGE
`
`49
`
`150
`
`ONE
`
`151
`
`FIG.8
`
`Petitioner Exhibit 1007, Page 6
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 6 of 6
`
`5,559,301
`
`
`
`
`
`2OO
`
`FIG 1 OA
`
`2OO
`
`TUCH
`
`FIG 1 OB
`
`"DRAG
`
`tLIFT
`
`FIG 1 OE
`
`204
`
`2O3
`
`200
`
`R 205
`"PP-UP"
`
`208
`
`UPDATE
`
`FIG 1 OC
`
`FIG 1 OF
`
`Petitioner Exhibit 1007, Page 7
`
`

`

`5,559,301
`
`1.
`TOUCHSCREEN INTERFACE HAVING
`POP-UPVARIABLE ADJUSTMENT
`DISPLAYS FOR CONTROLLERS AND
`AUDIO PROCESSING SYSTEMS
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention relates to graphical user interface
`systems; and more particularly to touchscreen interfaces for
`use with controllers and processing systems such as music
`synthesizers, or other sound processors.
`2. Description of Related Art
`Classical musical instrument control devices have been
`mechanical, sometimes having electrical couplings. For
`instance, sliders and knobs used on traditional mixers and
`music synthesizers are often connected to potentiometers for
`setting variables. Musicians have learned to utilize these
`instruments with the familiar sliders and knobs and other
`mechanical switches for the purposes of setting parameters
`of a performance to be synthesized by the device. For
`background concerning controls on musical instruments, see
`Rubine, et al., “Programmable Finger Tracking Instrument
`Controllers, COMPUTER MUSIC JOURNAL, Vol. 14,
`No. 1, Spring, 1990.
`As sound synthesis techniques become more computer
`ized, the flexibility of the traditional mechanical user inter
`face on a music synthesizer or other sound processor is
`strained. Accordingly, it is desirable to use graphical user
`interfaces with such devices, incorporating their greater ease
`of use for complicated data processing interfaces.
`However, prior art graphical interfaces typically require
`either a mouse or other computerized pointing device.
`Typically, these techniques require a relatively large screen,
`which is impractical for music synthesizers which are used
`during a performance. Thus, the use of a relatively small, flat
`panel touchscreen is desirable for these applications.
`However, the small display which is used for displaying
`the user interface limits the range of motion that can be used
`to set parameters, and limits the number of parameters that
`might be adjusted based on a single display. This requires a
`rather deep hierarchy of menus and windows for the pur
`poses of setting parameters associated with a particular
`performance to be synthesized.
`Accordingly, it is desirable to provide a graphical user
`interface method and apparatus which allows use of rela
`tively small touchscreen displays with music synthesizers or
`other sound processing systems, yet provides improved
`flexibility in the range of values which may be set using the
`interface, and the number of variables which may be
`manipulated with a single interface screen.
`
`SUMMARY OF THE INVENTION
`The present invention provides a touchscreen interface for
`a controller which comprises a display panel and a touch
`sensitive panel overlying the display panel. Processing
`resources are coupled with the display panel and the touch
`sensitive panel which supply an interface display to the
`display panel. The interface display includes an icon which
`represents an adjustable parameter used by the processing
`system. In addition, the processing resources supply a vari
`able adjustment display to the display panel in response to
`a touch on the position of the icon. The variable adjustment
`display "pops up' to overlie the interface display, and has a
`size on the touch sensitive panel larger than the size of the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`65
`
`2
`icon to facilitate manipulation of the variable using a finger
`over a significant range of values. The processing resources
`adjust the variable parameter and remove the variable
`adjustment display from the display panel in response to a
`touch sequence on the touch sensitive panel over the vari
`able adjustment display.
`Thus, when the variable is adjusted using the touch
`sequence, the variable adjustment display is removed, and
`the interface display is left unobscured. This allows the user
`to manipulate a particular variable while keeping the win
`dow which shows the values of related variables on the
`screen. By keeping the current window on the screen, the
`user is less likely to get lost in a hierarchy of windows used
`for setting variables. This feature is particularly important
`for music synthesizers or other systems which synthesize a
`performance requiring manipulation of a number of vari
`ables with the skill of a musician.
`According to one aspect of the invention, the icon on the
`interface display may comprise a symbol suggesting a slider,
`and display a current value for the variable adjacent the
`symbol. When the user touches the icon, the variable adjust
`ment display pops up. This display will resemble a slider,
`having a slider bar and a slider background. The slider
`background will include a scale, indicating a range of
`possible values for the variable parameter. The processing
`resources adjust the variable parameter and remove the
`variable adjustment display in response to a touch sequence,
`including detection of a touch over the slider bar on the
`touch sensitive panel, detection of a drag of the touch on the
`touch sensitive panel to a new position on the slider back
`ground representing a selected parameter value, and detec
`tion of a lift of the touch at the new position.
`Another representative variable adjustment display,
`according to the present invention, will resemble a knob,
`having a knob handle and a knob background. The user
`manipulates the knob handle on the variable adjustment
`display so that it points to a selected variable parameter. The
`touch sequence used for manipulating the knob may be
`similar to that described above for the slider.
`According to one aspect of the invention, a user interface
`based on the touch sensitive panel described above is
`applied to a sound synthesizer which adjusts variable param
`eters associated with the synthesized sounds. In this aspect,
`the interface display will include a plurality of icons repre
`sentative of respective adjustable parameters used by the
`sound synthesizer. The processing resources supply a vari
`able adjustment display in response to a touch on the
`position of a particular icon, where the variable adjustment
`display provides the tools for selecting a new value for the
`corresponding variable.
`The present invention can also be characterized as a
`method for setting a variable used by a processing system
`having a touchscreen. The method is based on the use of
`pop-up variable adjustment displays as described above.
`According to yet another aspect of the invention, the
`invention can be characterized as a method for adjusting
`parameters associated with a synthesized performance in a
`sound processing system having a touchscreen. The method,
`according to this aspect, includes the steps of:
`supplying an interface display to the touchscreen having
`a plurality of variable icons representative of respective
`adjustable parameters used by the processing system in
`producing the synthesized performance, said icons hav
`ing respective sizes and positions on the touchscreen;
`supplying a variable adjustment display to the touch
`screen in response to a touch on the position of a
`
`Petitioner Exhibit 1007, Page 8
`
`

`

`3
`particular icon in the plurality of icons, the variable
`adjustment display overlying said interface display and
`having a size on the touchscreen larger than the size of
`said particular icon; and
`adjusting the adjustable parameter represented by the
`particular icon and removing the variable adjustment
`display from the touchscreen, in response to a touch
`sequence on the touchscreen over the variable adjust
`ment display.
`The variable icons for the sound synthesizing system,
`according to the present invention, represent audio param
`eters associated with the synthesized performance, including
`combinations of attack, brightness, release, timbre, volume,
`and depth, as are appropriate to the particular sounds being
`included in the performance.
`Accordingly, the present invention provides a flexible
`user interface technique which may be applied to controllers,
`music synthesizers, recorders, mixers, and other sound pro
`cessing systems that allows for precise adjustment of param
`eters associated with a performance using a relatively small
`touchscreen on the front panel of the synthesizing device.
`This technique greatly increases the ease of use of a com
`plicated sound processing device by allowing adjustment of
`a relatively large number of parameters based on a single
`interface display, while preventing the user from becoming
`lost among a hierarchy of windows.
`This process is centered around the musician, not the
`technology. Since no two musicians do things in exactly the
`same way, this interface provides a musician maximum
`flexibility, making few assumptions about the way that a
`musician actually works. This system facilitates a musi
`cian's real-time needs which can be changed from session to
`session, from gig to gig, and from minute to minute.
`In addition to audio processor systems, the controller of
`the present invention can be applied to thermostats, volume
`and picture quality controllers for video systems, signal
`strength controllers, attenuators, speed controllers such as
`for toy trains, or other uses which benefit from a graphical
`user interface on a touchscreen.
`Other aspects and advantages of the present invention can
`be seen upon review of the figures, the detailed description,
`and the claims which follow.
`
`O
`
`15
`
`20
`
`25
`
`30
`
`35
`
`5,559,301
`
`4
`FIG. 9 is a flow chart of the "Show Current Parameter
`Value” process which is executed in the process of FIG. 6.
`FIGS. 10A-10F illustrate a pop-up adjustment knob
`sequence according to the present invention.
`
`DETALED DESCRIPTION
`A detailed description of preferred embodiments of the
`present invention is provided with respect to FIGS. 1-10, in
`which FIGS. 1 and 2 illustrate a representative hardware
`environment for the graphical user interface of the present
`invention, and FIGS. 3-10 illustrate representative interface
`techniques.
`In FIG. 1, a music synthesizer front panel 10 is illustrated.
`The front panel includes a keyboard, generally 11, and a
`number of keypads, such as a numerical keypad 12, a
`function keypad 13, function keypad 14, and display panel
`button sets 15 and 16. Also on the front panel are a ribbon
`controller 17, and a number of physical sliders 18, 19, and
`20. The front panel also includes other physical switches,
`such as the wheel 21 (or "knob"). These physical switches
`are either hardwired to specific functions and are so labelled
`on the front panel, or are programmed for a given synthesis
`performance to be used for a performance specific function.
`The front panel, according to the present invention, also
`includes a touchscreen 22 which is about 4 inches by about
`6 inches in size. The touchscreen includes a display panel,
`and a touch sensitive pad overlying the display panel. As
`schematically illustrated in FIG. 1, the display panel dis
`plays an interface display including a plurality of icons, such
`as icon 23, which is representative of a particular variable
`which might be adjusted by a slider, and icon 24 which is
`representative of a particular variable which might be
`adjusted by a knob.
`Associated with the front panel shown in FIG. 1, are
`processing resources, such as those illustrated in FIG. 2.
`These processing resources include a central processing unit
`CPU 30 coupled to a bus 31. Working memory 32 and
`instruction memory 33 are also coupled to the bus 31. The
`instruction memory 33, according to the present invention,
`stores routines for controlling the user interface and the
`touchscreen, such as a pop-up slider routine described
`below. The bus 31 may also be coupled to a disk drive 35 or
`other non-volatile memory system, as known in the art.
`Also coupled to the bus 31 is an audio processor 34 which
`generates audio data for a particular performance being
`synthesized. The audio processor 34 drives a speaker system
`suitable for the particular performance being executed, rep
`resented by the symbol 40. The audio processor may be
`replaced by other resources such as software executed by the
`CPU 30 itself.
`The bus31 is also coupled to the touchpad 36 and display
`panel 37 which, together, provide a touchscreen 22 for the
`front panel of the device. Also coupled to the bus 31 are
`other I/O devices, including the physical switches on the
`front panel, and potentially other devices as known in the art.
`These other I/O devices are collectively represented by
`block 38.
`The block diagram illustrated in FIG. 2 is representative
`of basic data processing resources which may be used with
`a mixer, recorder, or other audio processing system, like the
`music synthesizer illustrated in FIG. 1. A preferred system
`may include resources such as those described in co-pending
`U.S. patent application entitled OPEN ARCHITECTURE
`MUSIC SYNTHESIZER WITH DYNAMIC VOICE
`ALLOCATION, application No. 08/016,865, filed Feb. 10,
`
`BRIEF DESCRIPTION OF THE FIGURES
`FIG. 1 is a drawing of a front panel of a music synthesizer
`with a touchscreen interface according to the present inven
`tion.
`FIG. 2 is a block diagram of processing resources coupled
`with the touchscreen interface of the present invention.
`FIG. 3 is a representative window showing a plurality of
`variable icons according to the present invention.
`FIGS. 4A-4E illustrate the touch sequence used for
`manipulating a variable using the pop-up slider motif of the
`present invention.
`FIG. 5 is a flow chart of the algorithm executed by the
`processing resources for a pop-up slider touch sequence.
`FIG. 6 is a flow chart of the "Set Current Event' process
`used in the algorithm of FIG. 5.
`FIGS. 7A-7C illustrate slider background, a slider bar,
`and the combination of the slider background and the slider
`bar for the graphical variable adjustment display of the
`present invention.
`FIG. 8 is a flow chart of the "Slider Touch Processing
`initiated by the process of FIG. 6.
`
`45
`
`50
`
`55
`
`65
`
`Petitioner Exhibit 1007, Page 9
`
`

`

`5,559,301
`
`5
`1993, which is incorporated by reference herein for the
`purposes of illustrating one particular system which might
`be represented generically by FIG. 2 in the present appli
`cation. Alternatively, for controllers in simpler applications,
`such as thermostats and the like, the processing resources
`may be implemented with application-specific circuits.
`FIG.3 illustrates a representative interface display which
`will be provided to the touchscreen by the processing
`resources of FIG. 2 according to the present invention. The
`interface includes a label 50 identifying the performance to
`be synthesized which, in this example, is called "Jack
`Chorus.' Also, text 51 is included on the interface describing
`characteristics of the "Jack Chorus.” In this example, the
`"Jack Chorus' consists of a layered bells and choir perfor
`mance. Manipulation by the user of a knob called "wheel 1"
`on the front panel brings in female vocals. The delay time
`associated with bringing in the female vocals is affected by
`the velocity that the wheel is turned.
`Also shown on the interface is a window 52 which
`illustrates a bank of sliders 1-8. The window 52 also
`includes a group selection bar 53. By touching one of the
`buttons, such as button 54, a group of onscreen sliders is
`selected for display in the window 52.
`As can be seen, the onscreen sliders are icons represen
`tative of variables which can be adjusted using the user
`interface. The icons include a symbol, such as symbol
`enclosed by circle 55 and a current value field such as the
`field encircled by circle 56. Furthermore, in this embodi
`ment, the name of the variable that is represented by the icon
`55 is shown onscreen.
`The variables that are adjusted using this interface include
`such parameters associated with synthesis of music as
`attack, brightness, timbre, volume, release, and depth. The
`combinations of parameters that would be adjusted for
`particular basic sound will differ depending on the particular
`composition.
`As can be seen, a large number of icons 55 is displayed
`on a relatively small touchscreen. Thus, the value of these
`parameters could not be readily adjusted using a finger as the
`touch device over a significant range, or with significant
`accuracy. Accordingly, the present invention has provided
`the pop-up slider technique which is schematically illus
`trated in FIGS. 4A-4E.
`FIG. 4A illustrates an icon, such as icon 55 from FIG. 3.
`A fingertip 60 is shown schematically touching the icon 55.
`Processing resources are responsive to a touch on the icon
`55, to cause a variable adjustment display, generally 61, to
`pop-up on the screen as shown in FIG. 4B. This display
`overlies the interface display shown in FIG.3 and obscures
`the icon being adjusted, and possibly other regions of the
`screen. The variable adjustment display 61 is significantly
`larger than the icon 55 as illustrated by the relative sizes of
`the finger 60 in the display.
`As shown in FIG. 4B, the variable adjustment display
`resembles a slider which pops up on the screen. The user's
`finger 60 will lie at the position of the icon prior to popping
`up of the graphical slider. The graphical slider will include
`a slider bar 63 and a background, generally 64. The back
`ground 64 will indicate a range of values over which the
`slider can be used to adjust the parameter, and a current
`value, generally 65, of the parameter.
`When the pop-up slider appears on the screen, the pro
`cessing resources will monitor the touchscreen for a par
`ticular touch sequence. In particular, as illustrated in FIG.
`4C, the resources wait for a touch on the slider bar 63 of the
`pop-up slider by which the user "grabs' the slider bar. The
`
`45
`
`50
`
`55
`
`60
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`6
`user may be required to drag his or her finger 60 without
`lifting it from the touchscreen as indicated at arrow 66 to the
`current position of the slider bar 63, or the touchscreen may
`simply wait until a touch on the slider bar 63 is detected,
`independent of whether the touch is lifted or not in between.
`Upon "grabbing” the slider bar 63, the touchscreen monitors
`for a drag of the touch of the slider bar to a new position as
`illustrated in FIG. 4D. As the slider bar 63 is being dragged,
`as represented by arrow 67, to the new position, the value
`field 65 will be updated to display the value represented by
`the position of the slider bar 63. When the user moves the
`slider bar 63 to a position which corresponds to the value
`desired, then the user lifts the finger 60. When the user lifts
`the finger, the new value of the parameter is sent to the
`appropriate data location, so it may be used by the process
`ing resources, and the variable adjustment display composed
`of the background 64 and the slider bar 63 are removed from
`the screen, leaving the original icon 55. When the icon 55 is
`associated with a current value field 56, such as shown in
`FIG. 3, then the current value field 56 is updated when the
`pop-up slider is removed.
`In summary, the process begins with a slider icon dis
`played on the user interface showing a current value. The
`user puts the finger down on the slider icon, the finger being
`generally larger than the slider handle on the icon. After, for
`example, 200 milliseconds of holding down the touch, the
`pop-up slider appears. If the finger is lifted, the pop-up slider
`disappears and uncovers the interface display without
`changing the value. The user slides the finger up to the
`handle to "grab' the handle. Then, continuing to slide the
`finger, the value of the parameter is adjusted. After adjusting
`the parameter to the desired value, the finger is lifted up, the
`pop-up slider disappears, and the original interface is uncov
`ered with a new value shown adjacent the icon. The pro
`cessing resources may also be programmed to detect the
`velocity of the "drag," and provide parameters to the appro
`priate routine based on the velocity.
`More details concerning the routine executing by the
`processing resources to perform this sequence are provided
`with respect to FIGS. 5-9.
`FIG. 5 is a flow chart of the main algorithm for the pop-up
`slider routine. The algorithm begins at block 100. Next, it
`executes a "Set Current Event' routine 101, which is illus
`trated with respect to FIG. 6. After setting the current event,
`the algorithm determines whether the current event is a
`"pop-up slider” eventin block 102. If not, then the algorithm
`loops back to block 101. If it is a pop-up slider event, then
`the timer is set to a preselected value greater than Zero, e.g.,
`200 milliseconds (block 103). After setting the timer, the Set
`Current Event routine is executed again (block 104). After
`setting the current event, the algorithm determines whether
`the current event is a timeout event at block 105. If it is not
`a timeout event, then it is determined whether the current
`event is a touchup event 106, corresponding to lifting the
`finger. If it is not a touchup event, then the algorithm loops
`back to block 104 and continues monitoring for a timeout
`event. If it is a touchup event, then the algorithm loops back
`to block 101 to begin monitoring for a pop-up slider event.
`If, at block 105, a timeout is detected, then the pop-up
`slider graphic is sent to the display (block 106). After
`showing the pop-up slider, the Set Current Event routine is
`executed (block 107). The pop-up slider may be positioned
`near the touch on the icon to facilitate a touch sequence
`which requires the user to keep the touch down until the
`adjustment is complete. In alternative systems, the pop-up
`slider can be positioned elsewhere on the screen, such as in
`a consistent position for all variables.
`
`Petitioner Exhibit 1007, Page 10
`
`

`

`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`7
`After executing the Set Current Event routine at block
`107, the algorithm determines whether a touchup event has
`occurred at block 108. If not, then the algorithm loops to
`block 107 to monitor for a touchup event. If a touchup event
`is detected at block 108, then the pop-up slider is closed at
`block 109, and the algorithm requests parameter value
`updates at block 110 for other routines executed by the
`processing resources using the parameter. After requesting
`updates at block 110, the algorithm loops back to block 101
`to continue monitoring for pop-up slider events.
`The "Set Current Event' flow chart begins with block 120
`in FIG. 6. The routine waits for a message buffer event or a
`timeout, as indicated at block 121. At block 122, it deter
`mines whether the detected event is a message buffer. If not,
`then it is a timeout event and the current event parameter is
`set to timeout at block 123, and the process is done, as
`indicated at block 124 returning to the corresponding loca
`tion in the process of FIG. 5.
`If a message buffer was detected at block 122, then it is
`determined whether the message comes from a touchscreen
`manager, indicating a touch on the touchscreen. If it is not,
`then it is determined whether it is a parameter value buffer
`sent from a parameter editor routine in the processing
`resources. If not, then the algorithm returns to wait for a
`message buffer at block 121. If it is a parameter value buffer,
`then it is determined whether the parameter corresponds to
`a particular slider icon or pop-up slider on the screen. If it
`does not, then the algorithm loops back to block 121. If it
`does relate to a slider, then the "Show Current Parameter
`Value for Slider' routine illustrated in FIG. 9 is executed
`(block 128). After updating the parameter value on the
`display icon or the displayed pop-up slider, the algorithm
`loops back to block 121.
`If, at block 125, a touch buffer is detected, then the
`touched object on the screen is determined at block 129. If
`the touched object is a slidericon, as indicated at block 130,
`then the "Process Touch for Slider' routine of FIG. 8 is
`executed (block 131). If the object touched at block 130 is
`not a slider, then it is determined whether a touch Zone is
`touched, as indicated at block 132. A touch zone is a region
`on the screen corresponding to a pop-up slider. If it is a touch
`zone, then the current event is set to pop-up slider, as
`indicated at block 133, and the algorithm returns to the
`process of FIG. 5, as indicated at block 134. If the object
`touched is not a touch zone (touch dragged outside of touch
`zone), then the algorithm determines whether the touch
`status is up, corresponding to a lifted touch (block 135). If
`the touch has not been lifted, then the algorithm loops to
`block 121 to wait for a buffer message. If the touch is lifted,
`then the current event is set to touchup, as indicated at block
`136, and the algorithm is done at block 137 returning to the
`process of FIG. 5.
`FIGS. 7A-7C are used to illustrated slider terminology
`for the flow chart of FIG.8. Thus, the slidericon will consist
`of a slider background, as shown in FIG. 7A, which includes
`a current value field 80, a slide symbol 81, and an indication
`of the range of values which can be achieved by this slider,
`such as a top maximum value X, and a minimum value Y.
`As illustrated in FIG. 7B, a slider bar 82 is a second
`portion of the pop-up slider graphic. FIG.7C illustrates the
`combination of the background including the current value
`field 80, the slide symbol 81, and the slider bar 82.
`In one embodiment of the present invention, the icons on
`the screen use the same graphic as the pop-up slider. The
`pop-up slider is graphically an expanded version of the icon.
`Thus, the slider background and slider bar are drawn on the
`
`45
`
`50
`
`55
`
`65
`
`5,559,301
`
`8
`screen using the same basic routine whether or not it is
`displayed as an icon, or as a pop-up slider.
`FIG. 8 illustrates the "Process Touch for Slider' routine
`which is entered at block 131 of FIG. 6. The algorithm
`begins by determining whether the touch status is down at
`block 140. If not, the algorithm is done at block 141
`returning to the process of FIG. 6. If the touch status remains
`down, then the current parameter is set equal to a slider at
`block 142. Next, the algorithm determines whether the slider
`handle is being touched at block 143.