(12) United States Patent
`US 6,850,256 B2
`(10) Patent No.:
`Crow et al.
`(45) Date of Patent:
`Feb. 1, 2005
`
`US006850256B2
`
`(54) USER INTERFACE FOR PRESENTING
`MEDIA INFORMATION
`
`(75)
`
`Inventors: Daniel N. Crow, San Francisco, CA
`(US); Cary Dean, Sunnyvale, CA (US);
`Elizabeth Dykstra-Erickson, San
`Francisco, CA (US); J. Peter Hoddie,
`Mountain View, CA (US); Steven P.
`Jobs, Palo Alto, CA (US); Timothy E.
`Wasko, High River (CA)
`
`(73) Assignee: Apple Computer, Inc., Cupertino, CA
`(US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 136 days.
`
`(21) Appl. No.: 10/374,831
`
`(22)
`
`Filed:
`
`Feb. 24, 2003
`
`(65)
`
`Prior Publication Data
`US 2003/0128227 A1 Jul. 10, 2003
`
`Related US. Application Data
`
`(63) Continuation of application No. 09/293,507, filed on Apr.
`15, 1999, now Pat. No. 6,538,665.
`
`(51)
`Int. Cl.7 .................................................. G09G 5/00
`
`(52) US. Cl. ....................... 345/798; 345/801
`(58) Field of Search ................................. 345/798, 788,
`345/799—801, 790—793, 804
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4/1995 Klingler et a1.
`............. 345/328
`5,404,316 A *
`2/1996 Kwantinetz .............. 345/341
`5,495,566 A *
`
`4/1996 Leong et a1.
`345/798
`5,513,342 A *
`
`5/1996 Rayner ................. 345/326
`5,519,828 A *
`..
`3/1997 Belfiore et a1.
`345/819
`5,611,060 A *
`
`9/1997 Bauer ...................... 345/334
`5,664,128 A
`............. 345/328
`5,682,326 A * 10/1997 Klingler et a1.
`
`OTHER PUBLICATIONS
`
`Quick Time Movie Player Screendumps illustrating Current
`Time Indicator Dragging Operation (Ver. 2.1.2.59, 1996, p.
`1).*
`QuickTime Movie Player Ver. 2.1.2.59 ScreenDump (Cur-
`rent Time Indicator Dragging Operation; p. 1; 1996).*
`RealOne Player Version 2.0 ScreenDumps (pp. 4; 2002).*
`Windows Media Player for Windows XP version 8.0 (pp. 2;
`2001).*
`PCT International Search Report, mailed Jul. 11, 2000,
`PCT/USOO/10441, 5 pages.
`
`Primary Examiner—8y D. Luu
`(74) Attorney, Agent, or Firm—Blakely, Sokoloff, Taylor &
`Zafman LLP
`
`(57)
`
`ABSTRACT
`
`A user interface and methods for using a user interface for
`controlling processing of time-based media files. In one
`exemplary method, a graphical representation of a time line
`for a time-based media is displayed along with a graphical
`representation of a current time along the graphical repre-
`sentation of the time line. A start graphical indicator and a
`stop graphical indicator is also displayed along the graphical
`representation of the time line. A portion of the time-based
`media may be selected for presentation by dragging or
`positioning at least one of the start graphical indicator and
`the stop graphical indicator along the graphical representa-
`tion of the time line. In another aspect of the invention, an
`exemplary method allows for the adaptive control of a
`portion of the interface which indicates time relating to a
`time-based media. An input speed is determined where this
`input is to change the portion and the rate at which the
`change to this portion occurs is dependent upon the input
`speed. Other aspects of the present invention relating to the
`interface for controlling the processing of time-based media
`files are also described.
`
`(List continued on next page.)
`
`44 Claims, 25 Drawing Sheets
`
`MEDIA PLAYER
`DISPLAY WlNDOW
`200
`202
`[
`
`
`
`oiFmLMvING razmzwMsum wmwsnrm
`ALLAHDIEMBS
`avmammmannaAsmlmw orwarm ‘
`
`
`
`00000111
`
`
`
`
`
`
`
`KESKZE
`CONTROL
`205
`
`Apple Exhibit 4427
`
`Apple v. SightSound Technologies
`CBM2013-00023
`
`Page 00001
`
`Apple Exhibit 4427
`Apple v. SightSound Technologies
`CBM2013-00023
`Page 00001
`
`

`

`US 6,850,256 132
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`3/1998 Belfiore et a1.
`............. 345/684
`5,726,687 A *
`3/1998 Chao et a1.
`.....
`345/328
`5,732,184 A *
`
`............ 345/340
`4/1998 Ludolph et a1.
`5,745,096 A
`5/1998 Duffy et a1.
`................ 710/122
`5,758,180 A *
`6/1998 Shore et a1.
`................ 345/328
`5,760,767 A *
`6/1998 Austin ........................ 345/798
`5,760,772 A *
`
`2/1999 Ludolph .....
`345/339
`5,874,958 A
`3/1999 Southgate ................... 345/790
`5,880,725 A *
`5,999,173 A * 12/1999 Ubillos ....................... 345/328
`6,026,389 A *
`2/2000 Nakajima et a1.
`.............. 707/1
`
`............... 345/783
`2/2000 Migos et a1.
`6,031,529 A *
`5/2000 Venolia ...................... 345/856
`6,061,062 A *
`6/2000 Tani et a1.
`.................. 345/473
`6,072,503 A *
`6,166,736 A * 12/2000 Hugh .......
`345/798
`
`3/2001 Petelycky ................ 707/500.1
`6,204,840 B1 *
`6,332,147 B1 * 12/2001 Moran et a1.
`............ 707/500.1
`
`6366296 El *
`4/2002 Boreczky 6t a1~
`345/716
`4/2002 Lin ............................ 345/726
`6,369,835 B1 *
`6,456,305 B1 *
`9/2002 Qureshi et a1.
`............. 345/800
`2002/0191028 A1 * 12/2002 Senechalle et a1.
`......... 345/800
`
`* cited by examiner
`
`Page 00002
`
`Page 00002
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 1 0f 25
`
`US 6,850,256 B2
`
`
`
`Client
`Computer
`System Q2
`
`
`
`
`
`
`
`Client
`Computer
`Syetem 1% Internet fl
`
`
`
`Gateway
`Syetem
`
`]_1_Q
`
`
`
`
`
`
`Network
`Network
`Interface
`Interface
`
`
`
`11
`
`M
`
`
`
`
`
`
`Client
`Client
`Computer
`Computer
`
`
`Syetem
`
`
`
`m
`12
`
`Syetem
`
`
`
`b¥1OO
`
`Fig.1
`
`Page 00003
`
`Page 00003
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 2 0f 25
`
`US 6,850,256 B2
`
`
`Prooeaaor
`
`152
`.
`154
`
`
`
`
`
`[/0
`Display
`
`Controller
`Controller
`165%
`fié
`
`
` l/O
`
`Dovicoe
`
`13262
`
`
`
`
`
`
`
`Digital F’roooaaing
`Syatem
`1.512
`
`Modern
`0r
`
`Network
`Interface
`
`
`1.6_5
`
`
`Fig. 2
`
`Page 00004
`
`Page 00004
`
`

`

`U
`
`P
`
`wF
`
`S
`
`m6
`
`._._<M«GE—)2no292682$5552952ME5m.mmozuai
`
`
`
`1.,m885mg?zmmmm<zBEBE02.2638E:
`tJwNew08m2,8225965MES“.
`.NNNS.xommood
`
`
`5ENE239,n48.508?de
`0MMEEUEEEEmm209m992
`
`$5
`
`m8
`
`
`
`
`5.9m29:6New295mimSE28M,(m@Edwmmoo2925.82mmaéMmaéo9N295m><ESMWLB
`6gm:m9239,
`
`04m,£65.2665
`
`20:8
`
`SE288N
`
`Page 00005
`
`Page 00005
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 4 0f 25
`
`US 6,850,256 B2
`
`000200246
`
`55552555
`
`204
`
`Controls
`
`A
`
`
`
`m LIE!)
`
`t
`
`J
`
`.
`
`J
`
`_# rw—nud’uuuu
`\.—..____—
`-----
`
`MEI]
`—-——
`UUUUDDUUU
`
`Controle
`
`;
`
`Tray
`
`Faat
`. Rewind
`
`Fast
`Forward
`
`Step
`535k
`
`Step
`Forward
`
`(30 To Start
`
`60 To End
`
`Fig. 55
`
`Page 00006
`
`Page 00006
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 5 0f 25
`
`US 6,850,256 B2
`
`MEDIA PLAYER
`200
`
`V/
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`MEDIA SOURCE
`
`WELL
`ICON 245
`WELL 256
`254
`
`214
`
`MEDIA SOURCE
`ICON 250
`
`DRAWER
`2:50
`
`WELL
`240 FIG. 4
`
`Page 00007
`
`Page 00007
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 6 0f 25
`
`US 6,850,256 B2
`
`
`
`MEDIA
`PLAYER
`200
`
`J
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`
`
`
`
`DRAWER
`250
`
`Page 00008
`
`Page 00008
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 7 0f 25
`
`US 6,850,256 B2
`
`LOON
`
`mm.5
`
`3DE
`
`9:9,
`
`Page 00009
`
`Page 00009
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 8 0f 25
`
`US 6,850,256 B2
`
`INFORMATION
`BUTTON
`210
`
`
`
`
`
`
`
`%
`,. Cg]
`
`Compressed with Media CleanerTM
`
`INFORMATION
`
`DRQgCVDER
`
`Fig. 6A
`
`
`
` CONTROL
`
`BUTTON
`206
`
`CONTROL
`DRAWER
`252
`
`@ nunmfimnun
`BALANCE
`
`Fig. 65
`
`Page 00010
`
`Page 00010
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 9 0f 25
`
`US 6,850,256 B2
`
`TIME/CHAPTER DI5F’LAY/CONTROL 204
`
`3
`
`Swum em...
`
`TIME DI
`
`I7 AY 262
`5 L
`
`ADAPTIVE CHAPTER
`DISPLAY/CONTROL 260
`
`Fig. 7A
`
`TIME/CHAPTER DISPLAY/CONTKOL 204
`
`
`
`
`1- w Swim‘excitedly
`
`
`
`
`TIME DISPLAY 262
`ADAPTIVE CHAPTER
`DISPLAY/CONTKOL 260
`
`Fig. 75
`
`Page 00011
`
`Page 00011
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 10 0f 25
`
`US 6,850,256 B2
`
`
`
`260
`
`260
`
`
`
`Page 00012
`
`Page 00012
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 11 0f 25
`
`US 6,850,256 B2
`
`TIME/CHARTER DISPLAY/CONTROL 20Ar
`
`ADAPTIVE CHAPTER
`
`CURRENT TIME MARKER 274
`
`TIME BAR 271
`
`DISPLAY/CONTROL 260
`
`
`
`SELECT START MARKER 270
`
`TIME DISPLAY 262
`
`SELECT END MARKER 272
`
`Fig. 5A
`
`TIME/CHAPTER DISPLAY/CONTROL 204
`
`ADAPTIVE CHAPTER
`
`CURRENT TIME MARKER 274
`
`TIME BAR 271
`
`DISPLAY/CONTROL 260
`
`
`
`
`
`.0:00:11
`
`,.
`
`SELECT START MARKER 270
`
`“ME DISPLAY 262
`
`SELECT END MARKER 272
`
`Fig. 85
`
`Page 00013
`
`Page 00013
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 12 0f 25
`
`US 6,850,256 B2
`
`Display time bar with current time indicator
`
`275A
`
`2755
`
`Position (cg. drag) a start marker
`
`on a location along the time bar
`
`275C
`
`
`Update time display (e.g. update as drag start marker);
`display frame dragged to; update time bar(e.g. highlight/shade
`
`
`selected portion of time based media); Optionally move current
`
`
`time marker to coincide with start marker; update chapter display
`to show chapter corresponding to time selected by start marker’s
`
`
`position along time bar.
`
`
`
`Position (e.g. drag or click) an end marker
`
`on a location along the time bar 275E
`
`2750
`
`Update time display (e.g. update as drag end marker);
`display frame selected as end Frame; update time bar
`(e.g. highlight/shade selected portion of time based media);
`update chapter display to show chapter corresponding to
`
`time selected by end marker’s position along time bar
`
`Present selected range/portion of time based media
`
`(e.g. playback selected portion of Quicktime TM movie)
`
`275F
`
`Fig. 55C
`
`Page 00014
`
`Page 00014
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 13 0f 25
`
`US 6,850,256 B2
`
`204
`
`/ 2
`
`04
`
`262
`
`1/
`
` / P
`
`age 00015
`
`Page 00015
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 14 0f 25
`
`US 6,850,256 B2
`
`TIME/CHAPTER DISPLAY/CONTROL 204
`-
`
`“-‘
`
`_--__-_---__---__--_-_____
`
`ADAPTIVE STREAMING
`MESSAGE DISPLAY 280
`
`TIME/CHAPTER DISPLAY/CONTROL 204
`
`’1’ ,'
`
`
`
`
`.......
`
`ADAPTIVE STREAMING
`MESSAGE DISPLAY 280
`---‘——______________________________
`
`Fig. 95
`
`Page 00016
`
`Page 00016
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 15 0f 25
`
`US 6,850,256 B2
`
`/ 204
`
`r204
`
`Negotiating
`Connecting
`
`Fl@. 90
`
`[— 204-
`
`[“204
`
`Buffering
`
`Fig 9F
`
`
`
`270
`
`272
`
`Page 00017
`
`Page 00017
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 16 0f 25
`
`US 6,850,256 B2
`
`MEDIQOg-AYER
`(
`
`DISPLAY WINDOW
`202
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`RESIZE
`
`CONTROL
`208
`
`Fig.10A
`
`Page 00018
`
`Page 00018
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 17 0f 25
`
`US 6,850,256 B2
`
`IA P A
`
`MEDzooL YER
`'/
`
`DISPLAY WINDOW
`202
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`' ALL AUDIENCES
`
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`RESIZE
`
`CONTROL
`
`208
`
`Fig.105
`
`Page 00019
`
`Page 00019
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 18 0f 25
`
`US 6,850,256 B2
`
`53055
`
`>>OQz_>>
`
`NON
`
`(REE
`
`Mmkxi
`
`OQN
`
`49:280uunnF38.MNQMMWWImnWW.
`
` 8N_
`
`
`
`\vmomQm>omma<2mmmm<I§m_>mmn_02:30.30”.mIp
`
`
`<o_mm_>_<u_OzO:.<_oomm<mmEbEzo_._.o_>_m=.:.>m
`
`
`
`mmozm53<._._<
`
`Page 00020
`
`Page 00020
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 19 0f 25
`
`US 6,850,256 B2
`
`-
`.
`Detect user input to change size of a window
`In time based media player
`
`
`
`290A
`
`2905
`
`
`
`
`size of an outer
`window being changed below a
`
`
`predetermined size
`
`(e.g. a predetermined
`
`area?)
`
`
`
`
`animate both outer and inner windows
`
`while user drags resizing control
`
`window (expanding or
`
`
`shrinking outline)
`
`while dragging resizing
`
`
`control (no shrinking
`
`
`Is the
`or expanding of outer
`size of the outer window
`
`
`
`window if size of outer
`less than the
`
`
`window is at minimum size)
`predetermined
`
`
`
`size?
`
`animate changing
`outline of inner
`
`
`
`
`Is the
`
`size of the
`
`
`end of dragging?
`outer window less than
`
`
`
`
`
`
`
`Yes
`
`290F
`
`set inner and outer windows
`
`to new selected sizes
`
`the predetermined
`size?
`
`’Yes
`
`end of dragging?
`
`290i
`
`NO
`
`
`Yea
`
`290J
`
`shrink (or expand)
`inner window to final
`
`selected size of inner window
`
`Fig. 109
`
`Page 00021
`
`Page 00021
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 20 0f 25
`
`US 6,850,256 B2
`
`291F
`
`mm92
`
`
`
`
`(inner window
`291E
`periphery at end of drag)
`
`291D (inner window
`periphery at start; of drag)
`
`291C (outer window
`periphery)
`
`2915 (predetermined size)
`
`291A
`(cursor)
`
`Fig.1OE
`
`m92
`
`Fig.1OF
`
`Page 00022
`
`Page 00022
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 21 0f 25
`
`US 6,850,256 B2
`
`502
`
`504
`
`
` Detect input to
`open channel well
`
`
`drawer window
`
`
`
`Dieplay channel-well
`drawer window
`
`506
`
`
`
`
`
`508
` Detect
`
`input to eelect or alter
`channel well
`
`icon?
`
`
`
`Detect
`Yes
`
`
`input to close
`
`channel well drawer
`
`window?
`
`
`is
`
`input to
`
`play back
`
`media?
`
`
`
`
`
`C1056 channel well
`
`
`
`drawer window
`
`516
`
`Alter channel
`
`well drawer
`
`518
`
`
`
`Play back media
`
`Fig. 11
`
`Page 00023
`
`Page 00023
`
`

`

`based on a characteristic of the input
`
`Detect input to
`alter time and/or
`
`chapter mechanlema
`
`Adaptively vary scroll speed
`of time and/or chapter display
`
`US. Patent
`
`Feb. 1, 2005
`
`Sheet 22 0f 25
`
`US 6,850,256 B2
`
`550
`
`Display tim‘e—baaed
`media player window
`
`Display media time and/or
`chapter mechaniema
`
`555
`
`Fig. 12A
`
`Page 00024
`
`Page 00024
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 23 0f 25
`
`US 6,850,256 B2
`
`/ 559A
`- l)
`
`559E
`
`559‘:
`
`8
`559C
`
`5590
`
`5595
`
`Fig. 125
`
`Page 00025
`
`Page 00025
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 24 0f 25
`
`US 6,850,256 B2
`
`540
`
`Display time-based
`mealia player window
`
`window
`
`Detect input to “open"
`one or more media player
`auxiliary drawer/panel windows
`
`Display auxiliary panel
`
`Fig.15
`
`Page 00026
`
`Page 00026
`
`

`

`US. Patent
`
`Feb. 1, 2005
`
`Sheet 25 0f 25
`
`US 6,850,256 B2
`
`Channel Well
`Drawer Routine
`
`5%
`
`Ada tive
`Time/ hapter
`Selection Routine
`556
`
`Time Bar
`Selection Marker
`Routine
`
`558
`
`Streaming
`Status Message
`Routine
`
`560
`
`Player/display
`Wineiow Reeizing
`Routine
`
`39.62.
`
`Auxiliary
`Drawer/Panel
`Routine
`7 @
`
`Time—baeed Media
`
`Player Routine
`:52
`
`5_5Q
`
`Data
`Communication
`Routine
`
`570
`
`Stored! Buffered
`Time-baeed
`Media 565
`1——
`
`Machine Readable
`6torage Medium V
`
`Fig.14
`
`Page 00027
`
`Page 00027
`
`

`

`US 6,850,256 B2
`
`1
`USER INTERFACE FOR PRESENTING
`MEDIA INFORMATION
`
`This application is a continuation application of US.
`patent application Ser. No. 09/293,507, filed Apr. 15, 1999
`now US. Pat. No. 6,538,665.
`
`FIELD OF THE INVENTION
`
`The present invention relates to user interfaces for pro-
`cessing (e.g., generating, storing,
`transmitting and/or
`receiving, playing back/displaying, editing,
`referencing,
`etc.) media information, such as time-based media data
`representing video and/or audio. In particular, the present
`invention provides an interactive digital processing system-
`controlled graphical user interface that provides functional-
`ity for play back or other processing of time-based and still
`media data.
`
`INTRODUCTION AND BACKGROUND
`
`There are a number of file structures used today to store
`time-based media: audio formats such as AIFF, video for-
`mats such as AVI, and streaming formats such as RealMedia.
`They are different at least in part because of their different
`focus and applicability. Some of these formats are suffi-
`ciently widely accepted, broad in their application, and
`relatively easy to implement, that they are used not only for
`content delivery but also as interchange formats such as the
`QuickTimeTM file format. The QuickTime format is used
`today by many web sites serving time-based data; in many
`authoring environments, including professional ones; and on
`many multimedia CD ROM (e.g., DVD or CD-I) titles.
`The QuickTime media layer supports the relatively effi-
`cient display and management of general multimedia data,
`with an emphasis on time-based material (video, audio,
`video and audio, motion graphics/animation, etc.). The
`media layer uses the QuickTime file format as the storage
`and interchange format for media information. The archi-
`tectural capabilities of the layer are generally broader than
`the existing implementations, and the file format is capable
`of representing more information than is currently
`demanded by the existing QuickTime implementations.
`Furthermore, the QuickTime file format has structures to
`represent
`the temporal behavior of general
`time-based
`streams, a concept which covers the time-based emission of
`network packets, as well as the time-based local presentation
`of multimedia data.
`
`Given the capabilities and flexibility provided by time-
`based media formats,
`it
`is desirable to provide a user
`interface that provides suitable functionality and flexibility
`for playback and/or other processing of time-based media in
`such formats.
`
`Prior user interfaces for controlling the presentation of
`time-based media include user interfaces for the RealPlayers
`from RealNetworks of Seattle, Wash., user interfaces for the
`QuickTime MoviePlayers from Apple Computer, Inc. of
`Cupertino, Calif., and user interfaces for the Windows
`Media Players from Microsoft Corporation of Redmond,
`Wash. Also, there are a number of time-based media author-
`ing systems which allow the media to be created and edited,
`such as Premiere from Adobe Systems of San Jose, Calif.
`These prior user interfaces typically use “pop-up” or
`pull-down menus to display controls (e.g. controls for con-
`trolling playback) or to display a list of “favorites” or
`“channels” which are typically predetermined, selected
`media (e.g. CNN or another broadcast source which is
`remotely located or a locally stored media source). While
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`these lists or menus may be an acceptable way of presenting
`this information,
`the lists or menus may not be easily
`alterable and the alteration operations are not
`intuitive.
`Further, these lists or menus are separate from any window
`presenting the media and thus do not appear to be part of
`such window.
`
`the various controls are
`In some prior user interfaces,
`displayed on a border of the same window which presents
`the media. For example, a time bar may be displayed on a
`window with controls for playback on the same window.
`While these controls are readily visible and available to a
`user, a large number of controls on a window causes the
`window to appear complex and tends to intimidate a novice
`user.
`
`Some prior user interfaces include the ability to select, for
`presentation, certain chapters or sections of a media. Laser-
`Disc players typically include this capability which may be
`used when the media is segmented into chapters or sections.
`A user may be presented with a list of chapters or sections
`and may select a chapter or section from the list. When this
`list contains a large number of chapters or sections, the user
`may scroll through the list but the speed of scrolling is fixed
`at a single, predetermined rate. Thus, the user’s ability to
`scroll through a list of chapters is limited in these prior user
`interfaces.
`
`Certain prior user interfaces which allow for editing of
`time-based media may include the ability of selecting a
`range of time of the media for editing in that selected range.
`However, these interfaces do not allow for the playback,
`through a single playback button, of a selected range of a
`media which has been selected by at least one dragging
`action of an indicator.
`
`SUMMARY OF THE INVENTION
`
`The present invention relates to various aspects for con-
`trolling and/or presenting information concerning time-
`based media, such as a movie.
`In one aspect of the invention, an exemplary method of
`the invention allows for the selection of a presentation of
`time-based media. In this method, a graphical representation
`of a time line for the time-based media is displayed, and a
`graphical representation of a current time is also displayed
`along the graphical representation of the time line. A start
`graphical indicator and a stop graphical indicator is also
`displayed along the graphical representation of the time line.
`The start graphical indicator or the stop graphical indicator
`may be positioned, for example, by dragging, either of the
`indicator along the graphical representation of the time line
`to select a portion of the time-based media for presentation.
`In one typical example of this method, both the start
`graphical indicator and the stop graphical indicator may be
`dragged along the time line to select a portion of a movie for
`presentation.
`According to another aspect of the present invention, a
`method is provided for generating an interface for display
`for controlling the processing of time-based media data. A
`first set of data representing a graphical user interface as a
`primary window is generated and displayed. A second set of
`data representing a drawer window is generated and contains
`objects such as representations (e.g. icons) of “favorites” or
`“channels” media. In a first display state, the drawer window
`is at least partially overlapped by the primary window, and
`in a second display state is at least partially viewable. In one
`embodiment, the drawer window appears to be part of the
`primary window and slides under the primary window when
`the drawer window is closed. In one embodiment, when the
`
`Page 00028
`
`Page 00028
`
`

`

`US 6,850,256 B2
`
`3
`primary window is moved, the drawer window moves with
`the primary window as part of the primary window. The
`drawer window, in one embodiment, provides a plurality of
`locations which are fixed and predetermined, each for dis-
`playing a time-based media icon (e.g. a thumbnail image)
`associated with a time-based media file. In one embodiment,
`the thumbnail image represents a favorite or channel time-
`based media, and the user may drag an icon into the drawer
`window or out of the drawer window in order to alter the
`contents of the drawer window.
`
`According to another aspect of the present invention, a
`method is provided for generating a graphical user interface
`for control of processing of time-based media data. In an
`exemplary method according to this aspect, a first set of data
`representing the graphical user interface is generated and
`displayed as a primary window. A second set of data
`representing an auxiliary drawer window is generated for
`display. In a first display state, the auxiliary drawer window
`is at least partially overlapped by the primary window and
`in a second display state it is at least partially viewable. In
`one embodiment, the auxiliary drawer window appears to be
`part of the primary window and slides under the primary
`window when the auxiliary drawer is closed. When the
`primary window is moved in one embodiment, the auxiliary
`drawer window moves with the primary window as part of
`the primary window.
`In one embodiment,
`the auxiliary
`drawer window provides information relating to a time-
`based media file or controls for the control of a time-based
`media file.
`
`According to another aspect of the present invention, a
`method is provided for controlling a graphical user interface
`which processes time-based media information.
`In one
`exemplary method of this aspect, input to change a portion
`of the graphical user interface is detected. This portion
`indicates time relating to the time-based media. An input
`speed of the input is determined and a rate at which to
`change the portion is also determined, where the rate is
`dependent on the input speed. The information displayed in
`the portion is changed according to the rate which is
`dependent upon the input speed. In one particular example
`according to this aspect of the invention, a chapter selection
`mechanism allows the user to select different chapters within
`a movie by causing a chapter display area to have chapters
`scroll within the area. The speed of scrolling is controlled by
`the movement of a cursor or other input.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram of a network of computer systems in
`which media data may be processed, according to one
`embodiment of the present invention.
`FIG. 2 is a block diagram of a digital processing system
`which may be used to process time-based media data, in
`accordance with one embodiment of the present invention.
`FIG. 3A shows a GUI (graphical user interface), and in
`particular, a time-based media player display window for
`displaying, controlling, and/or otherwise processing time-
`based media data, according to one embodiment of the
`invention.
`
`FIG. 3B shows an example of an audio-only window for
`a time-based media player.
`FIG. 4 illustrate the time-based media player of FIG. 3A
`depicting a favorite/channel drawer that is partially open
`(i.e., partially displayed), according to one embodiment of
`the invention.
`
`FIG. 5A is a diagram of a time-based media player
`providing a favorite/channel drawer generated as a GUI,
`according to one embodiment of the invention.
`
`4
`FIG. 5B shows an example of a user interface for altering
`the icons in a drawer of a player window according to one
`embodiment of the invention.
`
`FIG. 6A is a diagram of time-based media player provid-
`ing an information panel (or drawer) window generated as
`GUI, according to one embodiment of the invention.
`FIG. 6B is a diagram of a time-based media player
`providing a control panel (or drawer) window generated as
`a GUI, according to one embodiment of the invention.
`FIG. 7A illustrates a time/chapter display control of a
`time-based media player generated as a GUI, and in
`particular, FIG. 7A illustrates an adaptive chapter display/
`control, according to one embodiment of the invention.
`FIG. 7B illustrates a time/chapter display control of a
`time-based media player generated as a GUI, and in
`particular, FIG. 7B illustrates an adaptive chapter display/
`control, according to one embodiment of the invention.
`FIGS. 7C—7J show the scrolling capabilities of the chapter
`display area 260 according to one embodiment of the
`invention.
`FIG. 8A illustrates a GUI mechanism of a time-based
`
`media player for selecting a range of time within a time-
`based media file, according to one embodiment of the
`invention.
`FIG. 8B illustrates a GUI mechanism of a time-based
`
`media player for selecting a range of time within a time-
`based media file, according to one embodiment of the
`invention.
`FIG. 8C illustrates a flowchart which shows a method for
`
`selecting a range of time for playback of a time-based media
`file.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`FIGS. 8D—8F show an example of a timecode mode
`selector interface which may be used with the current time,
`start and end markers.
`
`35
`
`FIG. 9A illustrates an adaptive streaming status message
`display of a time-based media player generated as a GUI,
`according to one embodiment of the invention.
`FIG. 9B illustrates an adaptive streaming status message
`display of a time-based media player generated as a GUI,
`according to one embodiment of the invention.
`FIGS. 9C—9H show another example of a status message
`display for a media player according to the present inven-
`tion.
`
`FIGS. 10A—10C illustrate resizing a time-based media
`player generated as a GUI and/or a display window for
`displaying time-based images, according to one embodiment
`of the invention. FIG. 10D is a flowchart which shows one
`
`method for controlling the display of a time-based media
`player during a resizing operation. FIGS. 10E and 10F show
`examples of resizing operations.
`FIG. 11 is a flow diagram illustrating a method for
`providing a favorite/channel drawer window for a time-
`based media player generated as a GUI, according to one
`embodiment of the invention.
`
`FIG. 12A is a flow diagram illustrating a method for
`providing adaptive chapter/time selection for play back or
`other processing of time-based media in a GUI, according to
`one embodiment of the invention. FIG. 12B shows an
`
`example of a chapter selection interface which adaptively
`scrolls a chapter indicator (e.g. a chapter number or name)
`in response to user input (e.g. speed of cursor movement).
`FIG. 13 is a flow diagram illustrating a method for
`providing auxiliary drawers/panels for a time-based media
`player generated as a GUI, according to one embodiment of
`the invention.
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Page 00029
`
`Page 00029
`
`

`

`US 6,850,256 B2
`
`5
`FIG. 14 is a block diagram of a machine-readable medium
`storing executable code and/or other data to provide one or
`a combination of mechanisms for play back or other pro-
`cessing of time-based media in a GUI, according to one
`embodiment of the invention.
`
`DETAILED DESCRIPTION
`
`The present invention provides methods and apparatuses
`for processing media information. In one embodiment of the
`present invention, a graphical user interface (GUI) is pro-
`vided that includes a number of features for referencing,
`playing back, and/or otherwise processing time-based media
`information (e.g., video, animated graphics, and/or audio,
`etc.). In one embodiment, the GUI, also sometimes referred
`to as a media player, provides functionality for processing
`time-based media in the QuickTime format, such as Quick-
`Time “movies,” which typically include audio and motion
`video information. Although the present
`invention is
`described with reference to the QuickTime media data
`format, it will be appreciated that the invention may also be
`used in a variety of environments, in conjunction with other
`media data formats, and with various types of data process-
`ing systems having a number of different types of architec-
`tures. Thus,
`the invention should not be limited to the
`systems, media data formats, or architectures disclosed
`herein, which are meant only to provide an understanding of
`the invention, whose scope is defined by the claims which
`follow.
`
`Hardware Overview
`
`FIG. 1 is a diagram of a network of computer systems in
`which media data may be processed, according to one
`embodiment of the present invention. As shown in FIG. 1,
`a network 100 includes a number of client computer systems
`that are coupled together through an Internet 122. It will be
`appreciated that the term “Internet” refers to a network of
`networks. Such networks may use a variety of protocols for
`exchange of information, such as TCP/IP, ATM, SNA, SDI,
`etc. The physical connections of the Internet and the proto-
`cols and communication procedures of the Internet are well
`known to those in the art.
`
`Access to the Internet 122 is typically provided by Inter-
`net service providers (ISPs), such as the ISP 124 and the ISP
`126. Users on client systems, such as the client computer
`systems 102, 104, 118, and 120, generally obtain access to
`the Internet through Internet service providers, such as ISPs
`124 and 126. Access to the Internet may facilitate transfer of
`information (e.g., email, text files, media files, etc.) between
`two or more digital processing systems, such as the client
`computer systems 102, 104, 118, and 120 and/or a Web
`server system 128. For example, one or more of the client
`computer systems 102, 104, 118, and 120 and/or the Web
`server 128 may provide media data (e.g., video and audio, or
`video, or audio)
`to another one or more of the client
`computer systems 102, 104, 118, and 120 and/or the Web
`server 128. For example,
`in one embodiment of the
`invention, one or more of the client computer systems 102,
`104, 118 and 120 may request play-back of time-based
`media data that may be stored at a remote location, such as
`the Web server 128. In the case of remote storage, the data
`may be transferred as a file (e.g. downloaded) and then
`played back after transferring the file or,
`if a streaming
`media system is available, the data may be played back at the
`client while the transfer occurs. In another embodiment, the
`requested time-based media data may be stored locally at the
`client computer system 102, 104, 118, and/or 120. In the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`the client system may play back
`case of local storage,
`time-based media using a playback system (e.g. a Quick-
`Time Player) without requiring a network connection.
`The Web server 128 typically includes at least one com-
`puter system to operate with one or more data communica-
`tion protocols, such as the protocols of the World Wide Web,
`and as such,
`is typically coupled to the Internet 122.
`Optionally, the Web server 128 may be part of an ISP which
`may provide access to the Internet and/or other network(s)
`for client computer systems. The client computer systems
`102, 104, 118, and 120 may each, with appropriate web
`browsing software, access data, such as HTML documents
`(e.g., Web pages), which may be provided by the Web server
`128. Such data may provide media, such as QuickTime
`movies, which may be played back/presented by the client
`computer systems 102, 104, 118, and 120.
`The ISP 124 provides Internet connectivity to the client
`computer system 102 via a modem interface 106, which may
`be considered as part of the client computer system 102. The
`client computer systems 102, 104, 118, and 620 may be a
`conventional data processing system, such as a Macintosh
`computer, a “network” computer, a handheld/portable
`computer, a Web TV system, or other types of digital
`processing systems (e.g., a cellular telephone having digital
`processing capabilities).
`Similarly, the ISP 126 provides Internet connectivity for
`the client computer systems 104, 118 and 120. However, as
`depicted in FIG. 1, such connectivity may vary between
`various client computer systems, such as the client computer
`systems 102, 104, 118, and 120. For example, as shown in
`FIG. 1, the client computer system 104 is coupled to the ISP
`126 through a modem interface 108, whi