112) United States Patent
`(16; Patent N0.:
`US 6,538,665 32
`
`Crow et a].
`(45) Date of Patent:
`Mar. 25, 2003
`
`USUU6538665B2
`
`(54) USER INTERFACE FOR PRESENTING
`MEDIA INFORMATION
`
`(75)
`
`Inventors: Daniel N. Crow, San Francisco, CA
`(RS); Gary Dean, SE'DPYWICo CANS);
`Elizabeth DykstriI-hrlcksun. San
`Francisco, CA (US); ,I. Peter Huddie,
`Mountain View, CA (US); Steven R
`Jobs, Palo Alto, CA (US); Timothy E.
`Wasku, High River (CA)
`
`.
`A
`.
`A
`(73) A‘i‘ilgncm Apple Computer, Inc-a (-Upcrllnoa (-A
`(US)
`_
`.
`y
`.
`.
`SUbJCCI to any disclaimer, the term ot thS
`patent is extended or adjusted under 35
`U.S.(I. 154(1)) 15in days.
`
`y
`( *) Notice:
`
`_
`(2]) Appl. No" 09"293’507
`(22)
`Filed:
`Apr. 15, 1999
`
`3451328
`5,999,123 A * 1211999 UhiIlos
`2071]
`6,026,389 A *
`212000 Nakajima el al.
`.. 3451340
`6,031,529 A
`212000 Migos
`
`E’ggé‘ggg : ,, 232$ Extfdim
`' 232:???
`
`.. 20215001
`0:204:84“ B] 5
`312001 Petelycky .....
`. 20215001
`6,332,142 Bl * 1212001 Moran el al.
`
`4120112 Bureaky el al.
`15,306,296 Bl *
`3451211:
`6,369,835 Bl *
`412002 Lin
`3451726
`
`OTHER PUBLICATIONS
`Quick Time Movie Player Screendumps illustrating Current
`Time Indicator Dragging Operation (Ver. 2.1.2.59, 1996, p.
`1).*
`PCT International Search Report, mailed Jul. 1], 2000,
`pcrmsooflmz”, 5 pages.
`_
`A
`* filled by cxamlncr
`
`Primary Exmiiirier—Sy D. Luv
`(74) Ai’i‘ormy, Agem, or Hrfli—Blakely, SokololT, 'I'aylor &
`Zafman UP
`
`(65)
`
`Prior Publication Data
`
`{57)
`
`ABSTRACT
`
`[is 2111121005728'1Al May 10, 21102
`7
`
`-------------------------------------------------- 009G 5100
`Int- Cl-
`(51)
`........................................ 3451723; 3451786
`(52) U.S. Cl.
`(58)
`Field Of Search
`3451328, 320',
`3451339’ 348, 723, 754’ 725, 725’ 735’
`9'23; 71171500.]; 386152
`
`(56)
`
`References Cited
`
`U.S. PA'I'IiN'I' DOCUMENTS
`3451328
`5,404,316 A ,,
`4,1095 Klingler et a1.
`3451341
`5495 566 A *
`211996 Kwanlinelz
`3451326
`5’5“)"st A s
`531995 Rayner __________
`3451,1319
`Sjélt:tlbfl A =2
`3,1199? Bclfiorc el al.
`3451334
`5,664,128 A
`911992 Bauer ............
`3451328
`5,682,326 A * 1011992 Klinglcr ei a1.
`3451684
`5,226,687 A *
`311998 Belfiorc et at.
`gig-:33
`Digigag :
`21:32:
`tholehal- I
`
`5,
`:3,
`.
`.uop eta.
`2101122
`5,258,180 A *
`511993 Duffy ct al.
`
`345/328
`5,760,716? A *
`611998 Shore Ct at.
`211999 Ludolph ..................... 3451339
`5,824,958 A
`
`
`
`Auser interface and methods for using a user interface for
`controlling processing of time-based media files. In one
`exemplary method, a graphical representation of a time line
`for a time—based media is displayed along with a graphical
`representation of a current time along the graphical repre—
`sentation of the time line. A start graphical indicator and a
`stop graphical indicator is also displayed along the graphical
`representation of the time line. A portion of the lime—based
`media may be selected for presentation by dragging or
`positioning at least one of the start graphical indicator and
`the stop graphical indicator along the graphical representa-
`tion otlthe timehlipie. Ijn anotfher aspectlot the inventiion,fan
`““913 “X “1“.“
`a 0"“ .0‘ .‘
`°. “apt!“ ““1”." 0 a
`portion 01 the interface which indicates lime relating to a
`time-based media. An input speed is determined where this
`input
`is to change the portion and the rate at which the
`change to this portion occurs is dependent upon the input
`speed. Other aspects of the present invention relating to the
`interface for controlling the processing oftime-based media
`-
`.
`1.
`I
`hles are also dCMIbm'
`
`27 Claims, 25 Drawing Sheets
`
`LlUlf'J'tTlK =-
`222 » ..
`
`HM.“
`£1.30
`\
`
`[El
`
`
`
`_-——
`
`DISPLAY1011100111
`
`k _ //
`
`THE FOLLOWING PREVEW HAS BEEN JIFPROVED FOR
`ALLAllDIENCES
`
`
`BY THE motion PICTURE assocmm Cf mu
`
`f2”:-
`
`
`
`IIIvCHCHN’II N
`D‘if'JIA’I 1111: 119411401
`
`-10
`§
`LINN-KO!
`eiiiiiaii
`"
`206
`
`htJDID
`11:1)"
`2"!
`“511:9
`920
`k.___
`I'UIJJMI:
`11m _--
`'
`
`_
`.
`Liliifi’ln?‘l’f
`
`{,Url'IIf-JL 2|4
`‘ ll
`I'LA‘I I‘UII‘JN Elia
`L'JI‘IA'If-JI
`I’FfAWI'K \
`
`
`
`
`__
`
`-"
`
`__ _
`
`.
`
`—,
`
`_._
`fliJIIUIZIJ
`“£115.12
`
`'
`.
`
`ifIiJ
`It‘IJIIIJ-‘I
`IJIFIJIEMliIIHI‘I
`
`Apple Exhibit 4423
`
`Apple v. SightSound Technologies
`CBM2013-00023
`
`Page 00001
`
`Apple Exhibit 4423
`Apple v. SightSound Technologies
`CBM2013-00023
`Page 00001
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 1 0f 25
`
`US 6,538,665 B2
`
`
`
`Client
`Computer
`
`Syetem 1 2
`
`
`Ciient
`Computer
`
`55/538211 104 Internet 12_2
`
`
`
`
`
`Network
`Network
`Inter-Face
`Interface
`
`11
`m
`
`
`
`
`
`Client
`Computer
`
`System
`
`@
`
`X100
`
`Fig.1
`
`Page 00002
`
`Page 00002
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 2 0f 25
`
`US 6,538,665 B2
`
`Froooooor
`
`Q
`
`Memory
`
`Q4
`
`
`
`
`
`
`
`
`[/O
`Dieplay
`
`Controilcr
`Controller
`
`@
`£62
`
`Diaplay
`
`16—0
`
`
`
`
`
`
`Digital Processing
`Syotem
`1§Q
`
`
`
`
`Modem
`or
`
`Network
`Interface
`
`@
`
`Fig. 2
`
`Page 00003
`
`Page 00003
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 3 of 25
`
`US 6,538,665 B2
`
`:82?EdgeMESL
`
`New08
`
`
`
`xommmod
`
`NNN
`
`
`
`
`
`mo“.om>omni<zmmmm<_._BER—mn—ozéojo“.mE.
`
`
`
`
`
`.fi<
`
`
`
`
`
`«cams:to20.568?#55520.5.2m53mmozm_n=<
`
`20:3
`
`.98
`
`mmiéumg:
`
`SEZSEEQQ
`
`
`
`49:288m
`
`wowNEE?
`
`ago
`
`
`
`x:x4m>m4
`
`£239,
`
`992
`
`209
`
`
`
`(m.®_n_wowmi2958Na2932EN49:28@N.SE18zoz<2mou§tIMmaég@N20:3E:
`
`
`
`$8Mifimma
`
`48:28
`
`Page 00004
`
`Page 00004
`
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 4 0f 25
`
`US 6,538,665 B2
`
`204
`
`
`
`Controle
`
`Controle
`
`Tray
`
`
`Go To Start
`
`Fagt
`Rewind
`
`F354;
`Forward
`
`Step
`Back
`
`Step
`Forward
`
`(30 To End
`
`Fig. 55
`
`Page 00005
`
`Page 00005
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 5 of 25
`
`US 6,538,665 B2
`
`MEDIA PLAYER
`200
`
`(
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`
`
`
`
`
`
` 000:00:08A
`
`WELL
`
`DRAWEEEPNTROL
`252 MEDIA SOURCE
`ICON 24a WELL WEL6L
`254
`
`25
`
`MEDIA SOURCE
`ECON 250
`
`WELL
`
`DRAWER
`IEEEL
`250
`WELL
`240 FIG. 4
`
`Page 00006
`
`Page 00006
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 6 of 25
`
`US 6,538,665 B2
`
`
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`“3090:0849
`
`
`
`
`
`
`
`MEDIA
`PLAYER
`ZOO
`
`J
`
`DRAWER
`250
`
`Page 00007
`
`Page 00007
`
`

`

`2i,2
`
`3
`
`0H...m..flS
`
`5
`
`US 6,538,665 132
`
`M 5
`
`mm.®Ewow98ENmfi
`
`8ON
`
`JlJ-Inn-I-lnn-III
`
`Page00008
`
`6fl
`
`8%
`
`memu
`
`E
`
`%\rCON
`
`US. Patent
`
`nQQN-k\
`
`
`
`«CON:QQN
`
`
`
`252:36.:
`
`
`
`gcam”.2E
`
`Page 00008
`
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 8 of 25
`
`US 6,538,665 B2
`
`________________________________
`
`
`
`
`
`
`
`INFORMARGN
`BUTfifii
`ZO
`
`
`
`
`
`
`,_
`
`lNFORMATION
`DRAWER
`250
`
`Fig. 6A
`
`
`
`
`
`CONTROL
`
`BUTTON
`206
`
`CONTROL
`DRAWER
`
`252
`
`Fig. 65
`
`Page 00009
`
`
`
`
`
`
`
`
`
`____________________________________________a
`
`
`
`
`
`
`numfinnnn
`unnnnuhnuun®
`nunn°nnnn
`®BALANCE. @ BASS
`eTREBLE
`',I
`
`#
`
`
`
`Page 00009
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 9 of 25
`
`US 6,538,665 B2
`
`TIMEECHAPTER DISPLAYKCONTROL 204
`
`g
`
`Swim exci...
`
`TIME DISPLAY 262
`
`ADAPTIVE CHAPTER
`
`DISPLAYKCONTROL 260
`
`Fig. 7A
`
`TIMEICHAF‘TER DLSF’LAWCONTKOL 204
`
`
`
`
`TIME DISPLAY 262
`ADAPTIVE CHAPTER
`DISPLAYXCONTROL 260
`
`Swim excitedly
`
`_
`
`Fig. 75
`
`Page 00010
`
`Page 00010
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 10 0f 25
`
`US 6,538,665 B2
`
`
`
`|
`J
`
`2605
`2600
`
` Chapter 2
`
`260'“ 0
`
`Fig. 7C
`
`2600
`
`260
`i
`Cha tar Laet:
`
`
`2605
`2,0,
`
`260’“
`
`Fig. 7E
`
`260
`J
`261A
` 2605
`
`__x_P_J
`Cha tcr4
`260C
`
`260A
`
`260D
`
`Fig. 7F
`
`2600
`
`260
`
`Chapter 2 ‘
`
`2605
`
`2606 260A
`
`
`Fig. 7H
`
`260A 215 260%,?» 7|
`
`260A
`
`Fig. 7J
`
`Page 00011
`
`Page 00011
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 11 of 25
`
`US 6,538,665 B2
`
`'/ TIMEJ’CHAFTER DESF’LAWCONTROL 204
`
`CURRENT TIME MARKER 274
`
`TIME BAR 271
`
`
`
`ADAPTIVE CHAPTER
`DEPLAWCONTEOL 260
`
`SELECT START MARKER 270
`TIME DISPLAY 262
`
`SELECT END MARKER 272
`
`Fig. 8A
`
`TIME/CHAPTER DiSFLAYfCONTROL 204
`
`CURRENT TIME MARKER 274
`
`TIME BAR 271
`
`
`
`ADAPTIVE CHARTER
`DISPLAWCONTROL 260
`
`SELECT START MARKER 270
`
`TIME 915“” 262
`
`SELECT END MARKER 272
`
`Fig. 55
`
`Page 00012
`
`Page 00012
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 12 0f 25
`
`US 6,538,665 B2
`
`275A
`
` Display time bar with current time indicator
`
`
`
` 275C
`
`Position (eg. drag) a start marker
`on a location along the time bar
`
`275E
`
`Update time display (eg. update as drag start marker);
`display frame dragged to; update time bar(e.g. highlight/shade
`selected portion of time based media): Optionally move current
`time marker to coincide with start marker; update chapter display
`to show chapter corresponding to time selected by start marker’s
`position along time bar.
`
`Position [e.g. drag or click) an end marker
`
`on a location along the time bar 275E
`
`275D
`
`Update time display (eg. update as drag end marker);
`display frame selected as end Frame: update time bar
`(eg. highlight/shade selected portion of time based media);
`update chapter display to show chapter corresponding to
`
`time selected by end marker's position along time bar
`
`Present selected rangei’portion of time based media
`
`[eg playback selected portion of Quicktime T“ movie)
`
`275F
`
`Fig. 8C
`
`Page 00013
`
`Page 00013
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 13 0f 25
`
`US 6,538,665 B2
`
`204
`
`04-
`
`262
`
`/ 2
`
`262
`
`/ P
`
`age 00014
`
`Page 00014
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 14 0f 25
`
`US 6,538,665 B2
`
`TIME/CHAPTER DISPLAYICONTROL 204
`
`ADAPTIVE STREAMING
`MESSAGE DISPLAY 250
`
`__________________________________
`
`TIMEECHAFTER DISPLAYICONTROL 204-
`
`ADAF’TIVE STREAMING
`MESSAGE DISPLAY 250
`
`
`
`
`
`Fig. 95
`
`Page 00015
`
`Page 00015
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 15 0f 25
`
`US 6,538,665 B2
`
`E
`
`Connecting
`
`‘f‘ 204
`
`g
`
`Fig. 9C
`
`
`
`Page 00016
`
`Page 00016
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 16 0f 25
`
`US 6,538,665 B2
`
`ME IA P AYER
`
`D zooL
`(
`
`DISPLAY WINDOW
`202
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
`
`
`RESIZE
`
`CONTROL
`
`205
`
`Fig.10A
`
`Page 00017
`
`Page 00017
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 17 0f 25
`
`US 6,538,665 B2
`
`MEDIA PLAY R
`
`E
`
`200
`(
`
`DISPLAY WINDOW
`202
`
`THE FOLLOWING PREVIEW HAS BEEN APPROVED FOR
`ALL AUDIENCES
`
`BY THE MOTION PICTURE ASSOCIATION OF AMERICA
`
` 0 00:00:11
`
`RESIZE
`
`CONTROL
`
`205
`
`Fig.105
`
`Page 00018
`
`Page 00018
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 18 0f 25
`
`US 6,538,665 B2
`
`\CELQQ
`
`309z:5
`
`NON
`
`Sam—2
`
`CONMP?.E
`
`\vmo“.om>omam<2mmmw<_._3m_>mmn_02.30.30“.m__._._.
`
`
`
`
`
`
`
`
`
`
`
`<o_m_m__>_<“—0zO_._.<_oOmm<mmskoi20:.0—2m=._._.>m
`
`mmozm52<44¢
`
`4OMFZOU
`
`09.5
`
`MNENM$69990
` momg?m
`
`Page 00019
`
`Page 00019
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 19 0f 25
`
`US 6,538,665 B2
`
`Detect user input to change size of a window
`in time baeed media player
`
`290A
`
`
`eize of an outer
`window being changed below a
`predetermined size
`(eg. a predetermined
`
`area?)
`
`2905
`
`No
`
`
`
`
`
`
`
`animate changing
`
`
`outline of inner
`
`
`
`
`window (expanding or
`9hrinking outline)
`while dragging resizing
`control (no ehrinking
`or expanding of outer
`window if eize of outer
`
`
`
`
`
`
`
`
`Yea
`
`
`290i
`
`No
`
`end of dragging?
`
`eet inner and outer windowe
`
`to new eeiected eizee
`
`ZQOF
`
`
`Yea
`
`290-.)
`
`ehrink (0r expand)
`inner window to final
`
`eelected eize of inner windOw
`
`Fig.1OD
`
`Page 00020
`
`animate both outer and inner windows
`
`whiie ueer drags resizing control
`
`290D
`
`290E
`
`No
`
`
`
`N0
`
`
`
`
`
` Ie the
`eize of the outer window
`
`
`
`less than the
`
`
`window is at minimum size)
`
`
`predetermined
`
`eize?
`
`
`No
`
`
`la the
`
`.
`size of the
`
`
`
`outer window less than
`end 01c dragemg?
`
`
`
`the predetermined
`Yee
`5'26?
`
`Page 00020
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 20 0f 25
`
`US 6,538,665 B2
`
`291i:
`
`MM92
`
`
`
`
`(inner window
`291E
`periphery at end of drag)
`
`291D
`(inner windOw
`periphery at start of drag)
`
`291C (outer window
`periphery)
`
`291B (predetermined 5ize)
`
`291A
`(cursor)
`
`Fig. 10E
`
`m92
`
`Fig.1OF
`
`Page 00021
`
`Page 00021
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 21 0f 25
`
`US 6,538,665 B2
`
`START
`
`502
`
`504
`
`Detect input to
`open channel well
`drawer window
`
`
`
`Dieplay channel-well
`drawer window
`
`
`Detect
`
`
`506
`
`508‘
`
`input to select or alter
`channel well
`
`icon?
`
`Yea
`
`
`
`Detect
`input to close
`
`
`
`
`channel well drawer
`
`
`
`window?
`
`
`
`512
`
`514
`
`Close channel well
`
`drawer window
`
`
`
`Play back media
`
`516
`
`Alter channel
`
`well drawer
`
`515
`
`
`
`Fig.11
`
`Page 00022
`
`Page 00022
`
`

`

`based on a characteriatic of the input
`
`Detect input to
`alter time and/or
`
`chapter mechaniama
`
`Adaptiveiy vary acroll apeed
`of time. and/or chapter display
`
`US. Patent
`
`Mar. 25, 2003
`
`Sheet 22 0f 25
`
`US 6,538,665 B2
`
`550
`
`START
`
`Diaplay time—based
`media player window
`
`Diapiay media time and/or
`chapter mechaniama
`
`555
`
`Fig. 12A
`
`Page 00023
`
`Page 00023
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 23 0f 25
`
`US 6,538,665 B2
`
`[//»-359A
`
`5595
`
`559C
`
`559D
`
`559E
`
`l)
`
`559}:
`
`Fig. 125
`
`Page00024
`
`Page 00024
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 24 0f 25
`
`US 6,538,665 B2
`
`540
`
`START
`
`542
`
`Display time-based
`media player window
`
`Detect input to “open"
`one or more media player
`
`auxiliary drawer/panel windows
`
`Display auxiliary panel
`windOW
`
`546
`
`Fig. 15
`
`Page 00025
`
`Page 00025
`
`

`

`US. Patent
`
`Mar. 25, 2003
`
`Sheet 25 0f 25
`
`US 6,538,665 B2
`
`Channel Well
`Drawer Routine
`
`fig
`
`Ada tive
`Time! hapter
`Seieetion Routine
`356
`
`Time Bar
`5e|ection Marker
`Routine
`
`3:55
`
`Streaming
`Status Message
`Routine
`
`EGQ
`
`Playerloiiepiay
`Window Resizing
`Routine
`
`Auxiliary
`Drawer!Panel
`Routine
`
`562
`
`564
`
`@
`
`Data
`Communication
`Routine
`
`Storedi’fiuFFei-ed
`Time-besed
`Media
`
`Time-based Media
`
`Player Routine
`fl
`
`Machine Readabie
`Etorage Medium
`
`Fig. 14
`
`Page 00026
`
`Page 00026
`
`

`

`US 6,538,665 BZ
`
`1
`USER INTERFACE FOR PRESENTING
`MEDIA INFORMATION
`
`FIELD ()l“ 'I‘IIL“. INVENTION
`
`The present invention relates to user interfaces for pro-
`cessing {e.g., generating, storing,
`transmitting andi’or
`receiving, playing backidisplaying, editing,
`referencing,
`etc.) media information, such as time—based media data
`representing video andt'or audio. In particular, the present
`invention provides an interactive digital processing system-
`controlled graphical user interface that provides functional-
`ity for play back or other processing of time—based and still
`media data.
`
`INTRODUCTION AND BACKGROUND
`
`There are a number of file stmctures used today to store
`time—based media: audio formats such as AIFF, video for—
`mats such as AVI, and streaming formats such as RealMedia.
`They are dilferent at least in part because of their dilTerent
`focus and applicability. Some of these formats are suffi-
`ciently widely accepted, broad in their application, and
`relatively easy to implement, that they are used not only for
`content delivery but also as interchange formats such as the
`QuickTimeTM file fonTiat. The QuickTime format
`is used
`today by many Web sites serving time-based data; in many
`authoring environments, including professional ones; and on
`many multimedia CD ROM (e.g., DVD or CD—I) titles.
`The QuickTime media layer suppons the relatively effi—
`cient display and management of general multimedia data,
`with an emphasis on time-based material (video, audio,
`video and audio, motion graphicsi'animation, etc.). The
`media layer uses the QuickTime file format as the storage
`and interchange format for media information. The archi—
`tectural capabilities of the layer are generally broader than
`the existing implementations, and the file format is capable
`of representing more information than is currently
`demanded by the existing QuickTime implementations.
`Furthermore,
`the QuickTime file format has structures to
`represent
`the temporal behavior of general
`time-based
`streams, a concept which covers the time-based emission of
`network packets, as well as the time—based local presentation
`of multimedia data.
`
`Given the capabilities and flexibility provided by time—
`based media formats,
`it
`is desirable to provide a user
`interface that provides suitable functionality and flexibility
`for playback andfor other processing of time—based media in
`such formats.
`
`Prior user interfaces for controlling the presentation of
`time-based media include user interfaces for the Reall’layers
`from RealNetworks of Seattle, Wash, user interfaces for the
`QuickTime MoviePlayers from Apple Computer, Inc. of
`Cupertino, Calif, and user interfaces for
`the Windows
`Media Players from Microsoft Corporation of Redmond,
`Wash. Also, there are a number of time-based. media author-
`ing systems which allow the media to be created and edited,
`such as Premiere from Adobe Systems of San Jose, Calif.
`These prior user interfaces typically use “pop—up" or
`pull—down menus to display controls (e .g. controls for con—
`trolling playback) or to display a list of “favorites" or
`“channels” which are typically predetermined, selected
`media (e.g. CNN or another broadcast source which—is
`remotely located or a locally stored media source). While
`these lists or menus may be an acceptable way of presenting
`this information,
`the lists or menus may not be easily
`alterable and the alteration operations are not
`intuitive.
`
`1t]
`
`15
`
`1t]
`
`40
`
`60
`
`2
`
`Further, these lists or menus are separate from any window
`presenting the media and thus do not appear to be part of
`such window.
`
`the various controls are
`In some prior user interfaces,
`displayed on a border of the same window which presents
`the media. For example, a time bar may be displayed on a
`window with controls for playback on the same window.
`While these controls are readily visible and available to a
`user, a large number of controls on a window causes the
`window to appear complex and tends to intimidate a novice
`LISBL
`
`Some prior user interfaces include the ability to select, for
`presentation, certain chapters or sections of a media. Laser—
`Disc players typically include this capability which may be
`used when the media is segmented into chapters or sections.
`A user may be presented with a list of chapters or sections
`and may select a chapter or section from the list. When this
`list contains a large number of chapters or sections, the user
`may scroll through the list but the speed of scrolling is fixed
`at a single, predetermined rate. Thus, the user’s ability to
`scroll through a list of chapters is limited in these prior user
`interfaces.
`
`Certain prior user interfaces which allow for editing of
`time—based media may include the ability of selecting a
`range of time of the media for editing in that selected range.
`However, these interfaces do not allow for the playback,
`through a single playback button, of a selected range of a
`media which has been selected by at
`least one dragging
`action of an indicator.
`
`SUMMARY 01" THE INVENTION
`
`The present invention relates to various aspecLs for con-
`trolling anclior presenting information concerning time—
`based media, such as a movie.
`In one aspect of the invention, an exemplary method of
`the invention allows for the selection of a presentation of
`time-based media. In this method, a graphical representation
`of a time Line for the time—based media is displayed, and a
`graphical representation of a current time is also displayed
`along the graphical representation of the time line. A start
`graphical indicator and a stop graphical indicator is also
`displayed along the graphical representation of the time line.
`The start graphical indicator or the stop graphical indicator
`may be positioned, for example, by dragging, either of the
`indicator along the graphical representation of the time line
`to select a portion of the time-based media for presentation.
`In one typical example of this method, both the start
`graphical indicator and the stop graphical indicator may be
`dragged along the time line to select a portion of a movie for
`presentation.
`According to another aspect of the present invention, a
`method is provided for generating an interface for display
`for controlling the processing of time—based media data. A
`first set of data representing a graphical user interface as a
`primary window is generated and displayed. A second set of
`data representing a drawer window is generated and contains
`objects such as representations (e.g. icons) of “favorites” or
`“channels” media. In a first display state, the drawer window
`is at least partially overlapped by the primary window, and
`in a second display state is at least partially viewable. In one
`embodiment, the drawer window appears to be part of the
`primary window and slides under the primary window when
`the drawer window is closed. In one embodiment, when the
`primary window is moved, the drawer window moves with
`the primary window as part Of the primary window. The
`drawer window, in one embodiment, provides a plurality of
`
`Page 00027
`
`Page 00027
`
`

`

`3
`
`4
`
`US 6,538,665 BZ
`
`locations which are fixed and predetermined, each for dis-
`playing a time—based media icon (e.g. a thumbnail image)
`associated with a time—based media file. In one embodiment,
`the thumbnail image represents a favorite or channel time—
`based media, and the user may drag an icon into the drawer
`window or out of the drawer window in order to alter the
`contents of the drawer window.
`
`According to another aspect of the present invention, a
`method is provided for generating a graphical user interface
`for control of processing of time-based media data. In an
`exemplary method according to this aspect, a first set of data
`representing the graphical user interface is generated and
`displayed as a primary window. A second set of data
`representing an auxiliary drawer window is generated for
`display. In a first display state, the auxiliary drawer window
`is at least partially overlapped by the primary window and
`in a second display state it is at least partially viewable. In
`one embodiment, the auxiliary drawer window appears to be
`part of the primary window and slides under the primary
`window when the auxiliary drawer is cIOSed. When the
`primary window is moved in one embodiment, the auxiliary
`drawer window moves with the primary window as part of
`the primary window.
`In one embodiment,
`the auxiliary
`drawer window provides information relating to a time-
`based media file or controls for the control of a time-based
`media file.
`
`According to another aspect of the present invention, a
`method is provided for controlling a graphical user interface
`which processes time-based media information.
`In one
`exemplary method of this aspect, input to change a portion
`of the graphical user interface is detected. This portion
`indicates time relating to the time—based media. An input
`speed of the input
`is determined and a rate at which to
`change the portion is also determined, where the rate is
`dependent on the input speed. The information displayed in
`the portion is changed according to the rate which is
`dependent upon the input speed. In one particular example
`according to this aspect of the invention, a chapter selection
`mechanism allows the user to select different chapters within
`a movie by causing a chapter display area to have chapters
`scroll within the area. The speed ofscrolling is controlled by
`the movement of a cursor or other input.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. I is a diagram of a network of computer systems in
`which media data may be processed, according to one
`embodiment of the present invention.
`FIG. 2 is a block diagram of a digital processing system
`which may be used to process time-based media data, in
`accordance with one embodiment of the present invention.
`FIG. 3A shows a GUI and in particular, a time—based
`media player display window for displaying, controlling,
`andfor otherwise processing time—based media data, accord—
`ing to one embodiment of the invention.
`FIG. 3]} shows an example of an audio-only window for
`a time-based media player.
`FIG. 4 illustrate the time—based media player of FIG. 3A
`depicting a favoritefchannel drawer that
`is partially open
`(i.e., partially displayed), according to one embodiment of
`the invention.
`
`FIG. 5A is a diagram of a time-based media player
`providing a favoriter‘ehannel drawer generated as a GUI,
`according to one embodiment of the invention.
`FIG. 5B shows an example of a user interface for altering
`the icons in a drawer of a player window according to one
`embodiment of the invention.
`
`It]
`
`15
`
`1E]
`
`FIG. 6A is a diagram of time-based media player provid-
`ing an information panel (or drawer) window generated as
`GUI, according to one embodiment of the invention.
`FIG. 63 is a diagram of a time—based media player
`providing a control panel (or drawer) window generated as
`a GUI, according to one embodiment of the invention.
`FIG. 7A illustrates a timefchapter display control of a
`time—based media player generated as a GUI, and in
`particular, FIG. 7A illustrates an adaptive chapter displayH
`control, according to one embodiment of the invention.
`FIG. 7B illustrates a timefchapter display control of a
`time-based media player generated as a GUI, and in
`particular, FIG. 7B illustrates an adaptive chapter display;r
`control, according to one embodiment of the invention.
`FIGS. 7C—7J show the scrolling capabilities of the chapter
`display area 260 according to one embodiment of the
`invention.
`FIG. 8A illustrates a GUI mechanism of a time-based
`media player for selecting a range of time within a time-
`based media file, according to one embodiment of the
`invention.
`FIG. SB illustrates a GUI mechanism of a time—based
`media player for selecting a range of time within a time-
`based media file, according to one embodiment of the
`invention.
`FIG. 8C illustrates a flowchart which shows a method for
`
`selecting a range of time for playback of a time—based media
`file.
`
`FIGS. SD~8F show an example of a timecode mode
`selector interface which may be used with the current time,
`start and end markers.
`
`FIG. 9A illustrates an adaptive streaming status message
`display of a time—based media player generated as a GUI,
`according to one embodiment of the invention.
`FIG. 9B illustrates an adaptive streaming status message
`display of a time-based media player generated as a GUI,
`according to one embodiment of the iHVention.
`FIGS. 9(T—9ll show another example of a status message
`display for a media player according to the present inven—
`tion.
`
`40
`
`FIGS. 10—10C illustrate resizing a time—based media
`player generated as a GUI andfor a display window for
`displaying time-based images, according to one embodiment
`of the invention. FIG. 10D is a flowchart which shows one
`
`method for controlling the display of a time—based media
`player during a resizing operation. FIGS. 1015. and 10F show
`examples of resizing operations.
`FIG. 11 is a [low diagram illustrating a method for
`providing a favoritefchannel drawer window for a time—
`based media player generated as a GUI, according to one
`embodiment of the invention.
`
`FIG. 12A is a flow diagram illustrating a method for
`providing adaptive chapterftime selection for play back or
`other processing oftime-based media in a GUI, according to
`one embodiment of the invention. FIG. 12B shows an
`
`60
`
`example of a chapter selection interface which adaptively
`scrolls a chapter indicator (e.g. a chapter number or name)
`in response to user input (e.g. speed of cursor movement).
`FIG. 13 is a flow diagram illustrating a method for
`providing auxiliary drawersfpanels for a time—based media
`player generated as a GUI, according to one embodiment of
`the invention.
`
`FIG. 14 is a bloek diagram of a machine-readable medium
`storing executable code andfor other data to provide one or
`
`Page 00028
`
`Page 00028
`
`

`

`US 6,538,665 BZ
`
`5
`a combination of mechanisms for play back or other pro-
`cessing of time—based media in a GUI, according to one
`embodiment of the invention.
`
`DETAILED DESCRIPTION
`
`The present iHVention provides methods and apparatuses
`for processing media information. In one embodiment of the
`present invention, a graphical user interface (GUI) is pro—
`vided that includes a number of features for referencing,
`playing back, andi’or otherwise processing time-based media
`information (e.g., video, animated graphics, andfor audio,
`etc.). In one embodiment, the GUI, also sometimes referred
`to as a media player, provides functionality for processing
`time-based media in the Quick’l'ime fon'nal, such as Quick-
`Time “movies," which typically include audio and motion
`video information. Although the present
`invention is
`described with reference to the QuickTime media data
`format, it will be appreciated that the invention may also be
`used in a variety of environments, in conjunction with other
`media data formats, and with various types of data process-
`ing systems having a number of different types of architec—
`tu res. Thus,
`the invention should not be limited to the
`systems, media data formats, or architectures disclosed
`herein, which are meant only to provide an understanding of
`the invention, whose scope is defined by the claims which
`follow.
`
`HARDWARE OVERVIEW
`
`FIG. I is a diagram of a network of computer systems in
`which media data may be processed, according to one
`embodiment of the present invention. As shown in FIG. 1,
`a network 100 includes a number of client computer systems
`that are coupled together through an Internet 122. It will be
`appreciated that the term “Internet” refers to a network of
`networks. Such networks may use a variety of protocols for
`exchange of information, such as TCPill“, ATM, SNA, SDI,
`etc. The physical connections of the Internet and the proto-
`cols and communication procedures of the Internet are well
`known to those in the art.
`
`Access to the Internet 122 is typically provided by Inter—
`net service providers {ISPs), such as the ISP 124 and the [SP
`126. Users on client systems, such as the client computer
`systems 102, 104, 118, and 120, generally obtain access to
`the Internet through Internet service providers, such as ISPs
`124 and 126. Access to the Internet may facilitate transfer of
`information (e.g., email, text files, media files, etc.) between
`two or more digital processing systems, such as the client
`computer systems 102, 104, 118, and 120 andfor a Web
`server system 128. For example, one or more of the client
`computer systems 102, 104, 118, and 120 andfor the Web
`server 128 may provide media data (e.g., video and audio, or
`video, or audio)
`to another one or more of the client
`computer systems 102, 104, 118, and 120 £111th the Web
`server 128. For example,
`in one embodiment of the
`invention, one or more of the client computer systems 102,
`I04, 118 and 120 may request play-back of time-based
`media data that may be stored at a remote location, such as
`the Web server 128. In the case of remote storage, the data
`may be transferred as a file {e.g. downloaded) and then
`played back after transferring the file or,
`if a streaming
`media system is available, the data may be played back at the
`client while the transfer occurs. In another embodiment, the
`requested time—based media data may be stored locally at the
`client computer system 102, 104, 118, andfor 120. In the
`case of local storage,
`the client system may play back
`time-based media using a playback system (e.g. a Quick-
`Time Player) without requiring a network connection.
`
`It]
`
`‘15
`
`1E]
`
`40
`
`6
`The Web server 128 typically includes at least one com-
`puter system to operate with one or more data communica—
`tion protocols, such as the protocols of the World Wide Web,
`and as such,
`is typically coupled to the Internet 122.
`Optionally, the Web server 128 may be part of an ISP which
`may provide access to the Internet andfor other network(s)
`for client computer systems. The client computer systems
`102, 104, 118, and 120 may each, with appropriate web
`browsing software, access data, such as HTML documents
`(e.g., Web pages), which may be provided by the Web server
`128. Such data may provide media, such as QuickTime
`movies, which may be played backfpresented by the client
`computer systems 102, I04, 118, and 120.
`The ISP 124 provides Internet connectivity to the client
`computer system 102 via a modern interface 106, which may
`be considered as part of the client computer system 102. The
`client computer systems 102, 104, 118, and 620 may be a
`conventional data processing system, such as a Macintosh
`computer, a “network” computer, a handheldfportable
`computer, a Web TV system, or other types of digital
`processing systems (e.g., a cellular telephone having digital
`processing capabilities).
`Similarly, the ISP 126 provides lntemet connectivity for
`the client computer systems 104, 118 and 120. However, as
`depicted in FIG. 1, such connectivity may vary between
`various client computer systems, such as the client computer
`systems 102, 104, 118, and 120. For example, as shown in
`FIG. 1, the client computer system 104 is coupled to the ISP
`126 through a modem interface 108, while the client com-
`puter systems 118 and 120 are part of a Local Area Network
`(LAN). The interfaces 106 and 108, shown as modems 106
`and 108, respectively, may represent an analog modem, an
`ISI)N modem, 3 cable modem, a satellite transmission
`interface (e.g., “Direct PC”), a wireless interface, or other
`interface for coupling a digital processing system, such as a
`client computer system, to another digital processing sys—
`tem. The client computer systems 118 and 120 are coupled
`to a LAN bus 112 through network interfaces 114 and 116,
`respectively. The network interfaces 114 and 116 may be an
`Ethernet—ty