United States Patent 1»
`5,559,301
`(11] Patent Number:
`Bryan,Jr. et al.
`
`[45] Date of Patent: Sep. 24, 1996
`
`AACTRAAA
`
`Primary Examiner—Stanley J. Witkowski
`Attorney, Agent, or Firm—Haynes & Davis
`
`(57]
`
`ABSTRACT
`
`[54] TOUCHSCREEN INTERFACE HAVING
`POP-UP VARIABLE ADJUSTMENT
`DISPLAYS FOR CONTROLLERS AND
`AUDIO PROCESSING SYSTEMS
`
`[75]
`
`Inventors: Marcus K. Bryan, Jr.; Alexander J.
`Limberis; John S. Bowen,all of San
`Jose; Daniel A. Phillips, Palo Aito, all
`of Calif.
`
`[73] Assignee: Korg, Inc., Tokyo, Japan
`
`[21] Appl. No.: 306,365
`
`(22]
`
`Filed:
`
`Sep. 15, 1994
`
`[51] mt GLe scsccesscssscsesecesvcesven G10H 1/057; G10H 1/06;
`GI0H 1/08; G10H 1/46
`52} US. Che ceeeeecseescesetcenneseceeeeees 84/653, 84/659; 84/660;
`84/663; 84/665; 345/173; 395/159
`[58] Field of Search 0...cee 84/600, 615-620,
`84/622-633, 653-665, 477 R, 478, 462,
`345/173-178; 395/159
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`A touchscreen interface for a sound processing system, such
`as music synthesizers, which has a display panel and a touch
`sensitive panel overlying the display panel, includes an icon
`which represents an adjustable parameter used by the pro-
`cessing system. The processing resources supply a variable
`adjustment display to the display panel in response to a
`touch on the position of the icon, using pop-up slider or
`pop-up knob motif. The variable adjustment display overlies
`the interface display and has a size on the touch sensitive
`pane! larger than the size of the icon to facilitate manipu-
`lation of the variable using a finger over a significant range
`of values. The variable adjustment display pops up when
`touched to obscure a portion of the graphical display used
`for the interface. When the variable is adjusted using the
`touch sequence, the variable adjustment display is removed,
`and the interface display is left unobscured. This allows the
`user to manipulate a particular variable while maintaining
`the window which showsthe values of related variables on
`5,027,689=7/1991 Fujimori oeeeeseeeeeeeeee 84/659 X
`the screen. By maintaining the current window on the
`5,146,833
`9/1992 LUE «esses
`. 84/477 RX
`
`screen, the user is less likely to get lost in a hierarchy of
`
`5,153,829
`10/1992 Furuyaet al. oe 84/477 RX
`windowsused for setting variables.
`
`OTHER PUBLICATIONS
`
`Rubine,et al., “Programmable Finger Tracking Instrument
`Controllers”; Computer Music Journal, vol. 14, No. 1,
`Spring, 1990.
`
`44 Claims, 6 Drawing Sheets
`
` MENU 7
`
`30
`
`
`
`
`i ‘
`
`Performance
`Bank [B] 0 Basic Sounds
`
`
`
`
`13 Jack Chorus
`[T] Layered bells and choir pad.
`ol
`
`Wheel
`1 brings in Female vocals,
`by velocity,
`Delay time is affected
`
`
`
`
`
`
`
`
`CO Group B OGroup C O Group D
`Sliders
`Group A
`
`
`
`Bell
`Voice
`Rverb Chorus
`
`
`Bell
`Vaice Voice Bell
`Attack Attack Bright Timbre Volume Release Depth Depth
`
`
`+01
`=e
`+68
`«51>
`+32
`-64
`+30
`+71
`
`1 i
`
`t
`
`1J|
`
`33
`
`DN-SCREEN
`SLIDERS
`sé
`
`36
`
`1
`
`CYPRESS 1007
`CYPRESS 1007
`
`1
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 1 of 6
`
`5,559,301
`
`BEartwauo|AW'TASIO|g
`
`
`
`GOld
`
`.OrWI
`
`SEASIC
`
`AAING
`
`aOSS330dd
`
`o1gnv
`
`PE
`
`cf
`
`‘ALSNI
`
`AXOWSW
`
`dndOd-
`
`Aaqris
`
`-3NILNOa
`
`2€
`
`ONTNYOM
`
`AYOWAW
`
`ia‘=
`
`
`
`
`OOOO
`
`O/1NSHLO
`
`TSNVd
`
`
`
`aV¥dHONOL
`
`Nd3
`
`TLOld
`
`2
`
`
`
`
`
`

`

`
`
`AVYAtusas09-r——=]|‘OSHONOLs
`
`og|abvddneANIAie“ANWAHSocot09
`dvoldiTVavOPOaeeeVPOL
`0S
`.ES[ss
`
`gqdnouwyC)SNUOUQuedaarIOA
`
`E9rLy19v9WEEog._OFDdaEo
`
`qdnougQd32dnowggd
`‘AqinayanAqpazragsvsiawAvojagq
`
`'S)ODOAayowayulsBujuq,jeaum
`
`
`Snuoyug4>20¢€TEll
`
`‘podujoudpuvsyaqpauato7[1]!
`
`Salo”nedned
`aDIO/Aydnoug©
`SASPHS$Xs
`
`
`
`
`
`spunosIsego[<q]ryUvg
`
`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 2 of 6
`
`5,559,301
`
`1S
`
`TZ+
`
`0E+
`
`¥9-oe+
`
`6OLA
`
`N3349S-NO
`
`Sa3aaTis
`
`2S
`
`
`
`QDUVWUOJUSq
`
`3
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 3 of 6
`
`5,559,301
`
`POPUP SLIDER FLOWCHARTS
`
`100
`
`BEGIN
`
`SET CURRENT EVENT
`(F1G.6)
`
`
`
`SET TIMER TO A
`VALUE >=0
`
`SET CURRENT EVENT
`(F1G.6)
`
`
`
`
`
`“TIMEQUT”
`“TOUCH UP’
`EVENT
`EVENT
`
`
`4
`
`SHOW POPUP SLIDER
`
`SET CURRENT EVENT
`(F1G.6)
`
`
`
`
`
`
`
`102
`
`105
`
`101
`
`103
`
`104
`
`106
`
`107
`
`108
`
`109
`
`1p
`
`
`“TOUCH UP’
`EVENT
`?
`
`CLOSE POPUP
`
`{REQUEST PARAMETER
`VALUE UPDATES
`
`FIG.S
`
`4
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 4 of 6
`
`CURRENT EVENT FLOWCHART
`
`START
`
`120
`
`WAIT FOR BUFFER
`DR TIMEDUT
`
`IS
`IT A
`
`122
`
`125
`
`NO
`
`ISET CURRENT EVENT
`TO *TIMEQUT”
`
`
`
`
`
`
`
`
`
`
`
`
`i
`PARAMETER
`? YES
`VALVE
`FIND TOUCHED OBJECT
`BUFFER
`IN THE DISPLAY
`
`129
`
`
`
`™\
`Is
`CURRENT
`PARAMETER
`A SLIDER
`
` SHOW CURRENT
`
`
`PARAMETER VALUE
`FOR SLIDER
`
`
`CF 1G.9)
`
`
`
`I$
`
`SLIDER
`TOUCHED
`a
`
`
`
`SET CURRENT
`EVENT TO
`“TOUCH UP”
`
`
`
`
`SET CURRENT
`EVENT TO
`
`
`
`
`
`
`“POPUP SLIDER*
`
`PROCESS TOUCH
`FOR SLIDER
`
`(F1G.8)
`
`FIG.6
`
`5,559,301
`
`
`
`5
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 5 of 6
`
`5,959,301
`
`SLIDER FLOWCHARTS
`
`141
`
`160
`
`SHOW CURRENT
`
`(B) PARAMETER VALUE
`(A) SLIDER TOUCH
`PROCESSING
`
`FOR SLIDER
`140
`
`DRAW SLIDER
`
`
`BACKGROUND TO
`
`
`1S
`
`NO
`FRASE HANDLE
`TOUCH STATUS>“(pENE)
`DOWN
`
`
`
`7
`
`YES
`
`CURRENT PARAMETER = SLIDER
`
`142
`
`6!
`
`
`CALCULATE NEW HANDLED
`
`
`
`
`POSITION USING
`PARAMETER VALUE
`AND IT MAXIMUM
`
`
`
`
`AND MINIMUM RANGE
`
`
`lee
`
`
`
`DRAW HANDLE
`
`163
`
`FIG.9
`
`143
` SLIDER
`Rica
`TOUCHED
`HANDLE
`
`
`
`
`
`IS
`
`2
`
`ae
`DONE)
`DONE
`
`
`
`145
`
`146
`
`NO|SAVE TOUCH
`LOCATION
`
`DONE
`INITIAL TOUCH LOCATION
`
`
`CALCULATE DISTANCE
`HANDLE MOVED FROM
`
`CALCULATE NEW PARAMETER
`VALUE USING MINIMUM
`AND MAXIMUM RANGE OF
`PARAMETER AND THE
`DISTANCE THE HANDLE
`MOVED
`
`147
`
`148
`
`149
`
`SEND PARAMETER CHANGE
`
`150
`
`DONE
`
`151
`
`FIG.8
`
`6
`
`

`

`U.S. Patent
`
`Sep. 24, 1996
`
`Sheet 6 of 6
`
`5,559,301
`
`a F
`
`IG.10D
`
`“DRAG’
`
`“LIF T*
`
`PIG.10E
`
`208
`
`“UPDATE”
`
`FIG.10F
`
`200
`
`FIG.10A
`
`202
`
`200
`
`“TOUCH”
`
`FIG.10B
`
`203
`
`7
`
`

`

`The present invention relates to graphical user interface
`systems; and more particularly to touchscreen interfaces for
`use with controllers and processing systems such as music
`synthesizers, or other sound processors.
`2. Description of Related Art
`Classical musical instrument contro! devices have been
`mechanical, sometimes having electrical couplings. For
`instance, sliders and knobs used ontraditional mixers and
`music synthesizers are often connectedto potentiometers for
`According to one aspect of the invention,the icon on the
`setting variables. Musicians have learned to utilize these
`interface display may comprise a symbol suggestingaslider,
`instruments with the familiar sliders and knobs and other
`20
`and display a current value for the variable adjacent the
`mechanical switches for the purposes of setting parameters
`symbol. When the user touches the icon, the variable adjust-
`of a performance to be synthesized by the device. For
`ment display pops up. This display will resemble a slider,
`background concerning controls on musical instruments, see
`having a slider bar and a slider background. The slider
`Rubine,et al., “Programmable Finger Tracking Instrument
`background will
`include a scale,
`indicating a range of
`Controllers,” COMPUTER MUSIC JOURNAL, Vol. 14,
`possible values for the variable parameter. The processing
`No. 1, Spring, 1990.
`resources adjust
`the variable parameter and remove the
`Assound synthesis techniques become more computer-
`variable adjustment display in response to a touch sequence,
`ized, the flexibility of the traditional mechanical user inter-
`including detection of a touch over the slider bar on the
`face on a music synthesizer or other sound processor is
`touch sensitive panel, detection of a drag ofthe touch on the
`sirained. Accordingly, it is desirable to use graphical user
`touch sensitive panel io a new position on the slider back-
`interfaces with such devices, incorporating their greater ease
`ground representing a selected parameter value, and detec-
`of use for complicated data processing interfaces.
`tion of a lift of the touch at the new position.
`However, prior art graphical interfaces typically require
`Another
`representative variable adjustment display,
`either a mouse or other computerized pointing device.
`according to the present invention, will resemble a knob,
`Typically, these techniques require a relatively large screen,
`having a knob handle and a knob background. The user
`which is impractical for music synthesizers which are used
`manipulates the knob handle on the variable adjustment
`during a performance. Thus, the use ofa relatively small, flat
`display so thatit points to a selected variable parameter. The
`panel touchscreen is desirable for these applications.
`touch sequence used for manipulating the knob may be
`similar to that described above for theslider.
`However, the small display which is used for displaying
`the user interface limits the range of motion that can be used
`to set parameters, and limits the number of parameters that
`might be adjusted based on a single display. This requires a
`rather deep hicrarchy of menus and windowsfor the pur-
`poses of setting parameters associated with a particular
`performance to be synthesized.
`Accordingly, it is desirable to provide a graphical user
`interface method and apparatus which allows use of rela-
`tively small touchscreen displays with music synthesizers or
`other sound processing systems, yet provides improved
`flexibility in the range of values which maybeset using the
`interface, and the number of variables which may bc
`manipulated with a single interface screen.
`
`2
`icon to facilitate manipulation of the variable using a finger
`over a significant range of values. The processing resources
`adjust
`the variable parameter and remove the variable
`adjustment display from the display panel in response to a
`touch sequence on the touch sensitive panel over the vari-
`able adjustment display.
`Thus, when the variable is adjusted using the touch
`sequence, the variable adjustment display is removed, and
`the interface display is left unobscured. This allowsthe user
`to manipulate a particular variable while keeping the win-
`dow which shows the values of related variables on the
`screen. By keeping the current window on the screen, the
`useris less likely to get lost in a hierarchy of windows used
`for setting variables, This feature is particularly important
`for music synthesizers or other systems which synthesize a
`performance requiring manipulation of a number of vari-
`ables with the skill of a musician.
`
`According to one aspectof the invention, a user interface
`based on the touch sensitive panel described above is
`applied to a sound synthesizer which adjusts variable param-
`eters associated with the synthesized sounds.In this aspect,
`the interface display will include a plurality of icons repre-
`sentative of respective adjustable paramctcrs used by the
`sound synthesizer. The processing resources supply a vari-
`able adjustment display in response to a touch on the
`position of a particular icon, where the variable adjustment
`display provides the tools for selecting a new value for the
`corresponding variable.
`The present invention can also be characterized as a
`method for setting a variable used by a processing system
`having a touchscreen. The method is based on the use of
`pop-up variable adjustment displays as described above.
`According to yet another aspect of the invention, the
`invention can be characterized as a method for adjusting
`parameters associated with a synthesized performance in a
`sound processing system having a touchscreen. The method,
`according to this aspect, includesthe stepsof:
`supplying an interface display to the touchscreen having
`a plurality of variable icons representative of respective
`adjustable parameters used by the processing system in
`producing the synthesized performance,said icons hav-
`ing respective sizes and positions on the touchscreen;
`supplying a variable adjustment display to the touch-
`screen in response to a touch on the position of a
`
`5,599;30].
`
`1
`TOUCHSCREEN INTERFACE HAVING
`POP-UP VARIABLE ADJUSTMENT
`DISPLAYS FOR CONTROLLERS AND
`AUDIO PROCESSING SYSTEMS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`10
`
`15
`
`25
`
`30
`
`35)
`
`40
`
`45
`
`30
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a touchscreen interface for
`a controller which comprises a display panel and a touch
`sensitive panel overlying the display panel. Processing
`resources are coupled with the display panel and the touch
`sensitive panel which supply an interface display to the
`display panel. The interface display includes an icon which
`represents an adjustable parameter used by the processing
`system. In addition, the processing resources supply a vari-
`able adjustment display to the display panel in response to
`a touch ontheposition of the icon. The variable adjustment
`display “pops up”to overlie the interface display, and has a
`size on the touch sensitive panel larger than the size of the
`
`55
`
`60
`
`65
`
`8
`
`

`

`5,559,301
`
`4
`FIG. 9 is a flow chart of the “Show Current Parameter
`Value” process which is executed in the process of FIG.6.
`FIGS. 10A-10F illustrate a pop-up adjustment knob
`sequence according to the present invention.
`
`DETAILED DESCRIPTION
`
`15
`
`3
`particular icon in the ‘plurality of icons, the variable
`adjustment display overlying said interface display and
`having a size on the touchscreen larger than the size of
`said particular icon; and
`adjusting the adjustable parameter represented by the
`particular icon and removing the variable adjustment
`display from the touchscreen, in response to a touch
`A detailed description of preferred embodiments of the
`sequence on the touchscreen over the variable adjust-
`present invention is provided with respect to FIGS. 1-10, in
`ment display.
`which FIGS. 1 and 2 illustrate a representative hardware
`The variable icons for the sound synthesizing system,
`environment for the graphical user interface of the present
`according to the present invention, represent audio param-
`invention, and FIGS. 3-10 illustrate representative interface
`eters associated with the synthesized performance, including
`techniques.
`combinations of attack, brightness, release, timbre, volume,
`In FIG.1, a music synthesizerfront panel 10 is illustrated.
`and depth, as are appropriate to the particular sounds being
`The front panel includes a keyboard, generally 11, and a
`included in the performance.
`number of keypads, such as a numerical keypad 12, a
`Accordingly, the present invention provides a flexible
`function keypad 13, function keypad 14, and display panel
`userinterface technique which maybe applied to controllers,
`button sets 15 and 16. Also on the front panel are a ribbon
`music synthesizers, recorders, mixers, and other sound pro-
`controller 17, and a numberof physical sliders 18, 19, and
`cessing systemsthat allowsfor precise adjustment of param-
`20. The front panel also includes other physical switches,
`eters associated with a performance using a relatively small
`such as the wheel 21 (or “knob”). These physical switches
`touchscreen on the front panel of the synthesizing device.
`are either hardwiredto specific functions and are so labelled
`This technique greatly increases the ease of use of a com-
`on the front panel, or are programmedfor a given synthesis
`plicated sound processing device by allowing adjustment of
`performanceto be used for a performancespecific function.
`a relatively large number of parameters based onasingle
`25
`The front panel, according to the present invention, also
`interface display, while preventing the user from becoming
`includes a touchscreen 22 which is about 4 inches by about
`lost among a hierarchy of windows.
`6 inches in size. The touchscreen includes a display panel,
`This process is centered around the musician, not the
`and a touch sensitive pad overlying the display panel. As
`technology. Since no two musicians do things in exactly the
`schematically illustrated in FIG. 1, the display panel dis-
`same way, this interface provides a musician maximum
`plays an interface display including a plurality of icons, such
`flexibility, making few assumptions about the way that a
`as icon 23, which is representative of a particular variable
`musician actually works. This system facilitates a musi-
`which might be adjusted by a slider, and icon 24 which is
`cian’s real-time needs which can be changed from session to
`representative of a particular variable which might be
`session, from gig to gig, and from minute to minute.
`adjusted by a knob.
`In addition to audio processor systems, the controller of
`the present invention can be applied to thermostats, volume
`Associated with the front panel shown in FIG. 1, are
`and picture quality controllers for video systems, signal
`processing resources, such as those illustrated in FIG. 2.
`strength controllers, attenuators, speed controllers such as
`These processing resources include a central processing unit
`for toy trains, or other uses which benefit from a graphical
`CPU 30 coupled to a bus 31. Working memory 32 and
`user interface on a touchscreen.
`instruction memory 33 are also coupled to the bus 31. The
`Other aspects and advantagesof the present invention can
`instruction memory 33, according to the present invention,
`be seen upon review of the figures, the detailed description,
`stores routines for controlling the user interface and the
`and the claims which follow.
`touchscreen, such as a pop-up slider routine described
`below. The bus 31 may also be coupled to a disk drive 35 or
`other non-volatile memory system, as knownin theart.
`Also coupledto the bus 31 is an audio processor 34 which
`generates audio data for a particular performance being
`synthesized. The audio processor 34 drives a speaker system
`suitable for the particular performance being executed, rep-
`resented by the symbol 40. The audio processor may be
`replaced by other resources such as software executed by the
`CPU 30 itself.
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`45
`
`FIG. 1 is a drawing of a front panel of a music synthesizer
`with a touchscreen interface according to the present inven-
`tion.
`
`FIG,2 is a block diagram of processing resources coupled
`with the touchscreen interface of the present invention.
`FIG. 3 is a representative window showing a plurality of
`variable icons according to the present invention.
`FIGS. 4A-4E illustrate the touch sequence used for
`manipulating a variable using the pop-up slider motif of the
`present invention.
`FIG. 5 is a flow chart of the algorithm executed by the
`processing resources for a pop-up slider touch sequence.
`FIG.6 is a flow chart of the “Set Current Event”process
`used in the algorithm of FIG.5.
`FIGS. 7A-7C illustrate slider background, a slider bar,
`and the combination of the slider backgroundandthe slider
`bar for the graphical variable adjustment display of the
`present invention.
`FIG. 8 is a flow chart of the “Slider Touch Processing”
`initiated by the process of FIG.6.
`
`Thebus 31is also coupled to the touch pad 36 anddisplay
`panel 37 which, together, provide a touchscreen 22 for the
`front panel of the device. Also coupled to the bus 31 are
`other /O devices, including the physical switches on the
`front panel, and potentially other devices as knownintheart.
`These other I/O devices are collectively represented by
`block 38.
`
`The block diagram illustrated in FIG. 2 is representative
`of basic data processing resources which may be used with
`a mixer, recorder, or other audio processing system, like the
`music synthesizer illustrated in FIG. 1. A preferred system
`mayinclude resourcessuchas those described in co-pending
`U.S. patent application entitled OPEN ARCHITECTURE
`MUSIC SYNTHESIZER WITH DYNAMIC VOICE
`ALLOCATION,application No. 08/016,865, filed Feb. 10,
`
`65
`
`9
`
`

`

`5,559,301
`
`5
`1993, which is incorporated by reference herein for the
`purposes of illustrating one particular system which might
`be represented generically by FIG. 2 in the present appli-
`cation. Alternatively, for controllers in simpler applications,
`such as thermostats and the like, the processing resources
`may be implemented with application-specific circuits.
`FIG.3 illustrates a representative interface display which
`will be provided to the touchscreen by the processing
`resources of FIG. 2 according to the present invention. The
`interface includesa label 50 identifying the performanceto
`be synthesized which,
`in this example,
`is called “Jack
`Chorus.” Also,text 51 is included onthe interface describing
`characteristics of the “Jack Chorus.” In this example, the
`“Jack Chorus” consists of a layered bells and choir perfor-
`mance. Manipulation by the user of a knob called “wheel 1”
`on the front panel brings in female vocals. The delay time
`associated with bringing in the female vocals is affected by
`the velocity that the wheel is tumed.
`Also shown on the interface is a window 52 which
`illustrates a bank of sliders 1-8. The window 52 also
`includes a group selection bar 53. By touching one of the
`buttons, such as button 54, a group of onscreen sliders is
`selected for display in the window 52.
`Ascan be seen, the onscreen sliders are icons represen-
`tative of variables which can be adjusted using the user
`interface. The icons include a symbol, such as symbol
`enclosed by circle 55 and a current value field such as the
`field encircled by circle 56. Furthermore, in this embodi-
`ment, the nameof the variable that is represented by the icon
`55 is shown onscreen.
`
`The variablesthat are adjusted using this interface include
`such parameters associated with synthesis of music as
`attack, brightness, timbre, volume, .release, and depth. The
`combinations of parameters that would be adjusted for
`particular basic sound will differ depending on the particular
`composition.
`As can be seen, a large number of icons 55 is displayed
`on a relatively small touchscreen, Thus, the value of these
`parameters could notbe readily adjusted using a finger as the
`touch device over a significant range, or with significant
`accuracy. Accordingly, the present invention has provided
`the pop-up slider technique which is schematically illus-
`trated in FIGS. 4A-4E.
`
`FIG. 4A illustrates an icon, such as icon 55 from FIG.3.
`A fingertip 60 is shown schematically touching the icon 55.
`Processing resources are responsive to a touch on the icon
`55, to cause a variable adjustment display, generally 61, to
`pop-up on the screen as shown in FIG. 4B. This display
`overlies the interface display shown in FIG. 3 and obscures
`the icon being adjusted, and possibly other regions of the
`screen. The variable adjustment display 61 is significantly
`larger than the icon 55 as illustrated by the relative sizes of
`the finger 66 in the display.
`As shown in FIG, 4B, the variable adjustment display
`resembles a slider which pops up on the screen. The user’s
`finger 60 will lic at the position of the icon prior to popping
`up of the graphical slider. The graphical slider will include
`a slider bar 63 and a background, generally 64. The back-
`ground 64 will indicate a range of values over which the
`slider can be used to adjust the parameter, and a current
`value, generally 65, of the parameter.
`When the pop-up slider appears on the screen, the pro-
`cessing resources will monitor the touchscreen for a par-
`ticular touch sequence. In particular, as illustrated in FIG.
`4C, the resources wait for a touch onthe slider bar 63 ofthe
`pop-up slider by which the user “grabs” the slider bar. The
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`10
`
`6
`user may be required to drag his or her finger 60 without
`lifting it from the touchscreen asindicated at arrow 66to the
`current position of the slider bar 63, or the touchscreen may
`simply wait until a touch on the slider bar 63 is detected,
`independent of whether the touchis lifted or not in between,
`Upon “grabbing” the slider bar 63, the touchscreen monitors
`for a drag of the touchoftheslider bar to a new position as
`illustrated in FIG. 4D. Astheslider bar 63 is being dragged,
`as represented by arrow 67, to the new position, the value
`field 65 will be updated to display the value represented by
`the position of the slider bar 63. When the user moves the
`slider bar 63 to a position which corresponds to the value
`desired, then the user lifts the finger 60. When theuserlifts
`the finger, the new value of the parameter is sent to the
`appropriate data location, so it may be used by the process-
`ing resources,andthe variable adjustment display composed
`of the background 64 andtheslider bar 63 are removed from
`the screen, leaving the original icon 55. When the icon 55 is
`associated with a current value field 56, such as shown in
`FIG. 3, then the current valuc ficld 56 is updated when the
`pop-up slider is removed.
`In summary, the process begins with a slider icon dis-
`played on the user interface showing a current value. The
`user puts the finger downontheslidericon,the finger being
`generally larger than the slider handlc on the icon. After, for
`example, 200 milliseconds of holding down the touch, the
`pop-up slider appears.If the fingeris lifted, the pop-up slider
`disappears and uncovers the interface display without
`changing the value. The user slides the finger up to the
`handle to “grab” the handle. Then, continuing to slide the
`finger, the value of the parameter is adjusted. After adjusting
`the parameter to the desired value, the fingeris lifted up, the
`pop-up slider disappears, and the original interface is uncov-
`ered with a new value shown adjacent the icon. The pro-
`cessing resources may also be programmed to detect the
`velocity of the “drag,” and provide parameters to the appro-
`priate routine based on the velocity.
`More details concerning the routine executing by the
`processing resources to perform this sequence are provided
`with respect to FIGS. 5-9.
`FIG. 3 is a flow chart of the main algorithm for the pop-up
`slider routine. The algorithm begins at block 100. Next, it
`executes a “Set Current Event” routine 101, whichis illus-
`trated with respect to FIG.6. After setting the current event,
`the algorithm determines whether the current event is a
`“pop-up slider” event in block 102. If not, then the algorithm
`loops back to block 101.If it is a pop-up slider event, then
`the timeris set to a preselected value greater than zero,e.g.,
`200 milliseconds (block 103). After setting the timer, the Set
`Current Event routine is executed again (block 104). After
`setting the current event, the algorithm determines whether
`the current event is a timeout event at block 105. If it is not
`a timeout event, then it is determined whether the current
`event is a touchup event 106, corresponding to lifting the
`finger. If it is not a touchup event, then the algorithm loops
`back to block 104 and continues monitoring for a timeout
`event. If it is a touchup event, then the algorithm loops back
`to block 101 to begin monitoring for a pop-up slider event.
`If, at block 105, a timeout is detected, then the pop-up
`slider graphic is sent
`to the display (block 106). After
`showing the pop-up slider, the Set Current Event routineis
`executed (block 107). The pop-up slider may be positioned
`near the touch on the icon to facilitate a touch sequence
`which requires the user to keep the touch down until the
`adjustment is complete. In alternative systems, the pop-up
`slider can be positioned elsewhere on the screen, such as in
`a consistent position for all variables.
`
`10
`
`

`

`5,559,301
`
`10
`
`20
`
`25
`
`30
`
`8
`7
`screen using the same basic routine whether or notit is
`After executing the Set Current Event routine at block
`displayed as an icon, or as a pop-upslider.
`107, the algorithm determines whether a touchup event has
`FIG.§illustrates the “Process Touch for Slider” routine
`occurred at block 108. If not, then the algorithm loops to
`block 107 to monitor for a touchup event. If a touchup event
`which is entered at block 131 of FIG. 6. The algorithm
`is detected at block 108, then the pop-upslider is closed at
`begins by determining whether the touch status is down at
`block 109, and the algorithm requests parameter value
`block 140. If not,
`the algorithm is done at block 141
`updates at block 110 for other routines executed by the
`returning to the process of FIG.6.If the touch status remains
`down, then the current parameter is set equal to a slider at
`processing resources using the parameter. After requesting
`block 142. Next, the algorithm determines whether theslider
`updatesat block 110, the algorithm loops back to block 101
`handle is being touched at block 143. If not, the algorithm
`to continue monitoring for pop-up slider events.
`is done, as indicated at block 144, returning to the process
`The “Set Current Event”flow chart begins with block 120
`of FIG. 6. If the slider handle is touched, the algorithm
`in FIG. 6. The routine waits for a message buffer event or a
`determines whether the slider handie had been previously
`timeout, as indicated at block 121. At block 122, it deter-
`touched at block 145. If not, then this is the first time the
`mincs whether the detected event is a messagebuffer. If not,
`slider handle has been touched during this touch sequence,
`then it is a timeout event and the current event parameter is
`and the touch location is saved at block 146. Then the
`set to timeout at block 123, and the process is done, as
`algorithm returns to the process of FIG. 6, as indicated at
`indicated at block 124 returning to the corresponding loca-
`block 147. If it had been touched previously,
`then the
`tion in the process of FIG.5.
`distance the handle has moved from theinitial touch location
`If a message buffer was detected at block 122, then it is
`to the current touch locationis calculated at block 148. Next,
`determined whether the message comes from a touchscreen
`the new parameter value is calculated using the minimum
`manager, indicating a touch on the touchscreen.Ifit is not,
`and maximum range of the parameter, and the distance the
`then it is determined whetherit is a parameter value buffer
`handle had movedat block 149.
`sent from a parameter editor routine in the processing
`to the parameter
`the parameter change is sent
`Next,
`resources. If not, then the algorithm returns to wait for a
`management software, as indicated at block 150, and the
`message buffer at block 121. If it is a parameter valuebuffer,
`process is done at block 151,returning to the algorithm of
`then it is determined whether the parameter corresponds to
`FIG.6.
`a particular slider icon or pop-up slider on thescreen.If it
`FIG. 8 illustrates the “Show Current Parameter Value for
`does not, then the algorithm loops back to block 121. If it
`Slider” routine which is entered at block 128 of FIG. 6. This
`does relate to a slider, then the “Show Current Parameter
`Value for Slider” routine illustrated in FIG. 9 is executed
`algorithm begins at block 160 by drawing the slider back-
`groundto erase the handleofthe slider on the screen. Next,
`(block 128). After updating the parameter value on the
`a new handle position using the parameter value and its
`display icon or the displayed pop-up slider, the algorithm
`maximum and minimum range is determined at block 161.
`loops back to block 121.
`Finally, the handle is drawn at the new position (block 162)
`If, at block 125, a touch buffer is detected, then the
`over the backgroundto provide the new slider on the screen.
`touched object on the screen is determined at block 129.If
`Then, the process is done, as indicated at block 163 returning
`the touched objectis a slider icon, as indicated at block 130,
`to the algorithm of FIG. 6. This process applies to both the
`then the “Process Touch for Slider” routine of FIG. 8 is
`icon and the pop-upslider in the embodiment wherethe icon
`executed (block 131). If the object touched at block 130 is
`and the slider use the same graphic, just of different sizes.
`notaslider, then it is determined whether a touch zoneis
`40
`Basically, the routines of FIGS. 5-9 can be summarized
`touched, as indicated at block 132. A touch zoneis a region
`for pop-up slider event as follows. If at block. 101 of FIG.5,
`onthe screen corresponding to a pop-upslider. If itis a touch
`a pop-up slider event, such asset at block 133 of FIG.6,is
`zone,
`then the current event is set to pop-up slider, as
`detected, then the timer is set to ensure that the slider icon
`indicated at block 133, and the algorithm returns to the
`is touched for 200 milliseconds. Once it has been touched
`process of FIG. 5, as indicated at block 134. If the object
`for the timeout event, without having lifted the finger, the
`touchedis not a touch zone (touch dragged outside of touch
`pop-up slider appears on the screen. The current event flow
`zone), then the algorithm determines whether the touch
`chart will process the touch sequencefor the slider executing
`status is up, corresponding to a lifted touch (block 135). If
`the routine of FIG. 8 which results in sending a new
`the touch has not been lifted, then the algorithm loops to
`parameter value to the process. This new parameter results
`block 121 to wait for a buffer message. If the touch is lifted,
`in redrawing the slider on the screen, using the process of
`then the current eventis set to touchup,as indicated at block
`FIG. 9, and continuing to wait fo