(12) United States Patent
`Abecassis
`
`USOO6408.128B1
`(10) Patent No.:
`US 6,408,128 B1
`(45) Date of Patent:
`Jun. 18, 2002
`
`(54) REPLAYING WITH SUPPLEMENTARY
`INFORMATION A SEGMENT OF A VIDEO
`
`(76) Inventor: Max Abecassis, 1902ONE. 20 Ave.,
`Miami, FL (US) 33179
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/190,773
`(22) Filed:
`Nov. 12, 1998
`(51) Int. Cl." ............................ H04N 5/91; H04N 7/04;
`H04N 7/06; H04N 7/08; H04N 7/52
`(52) U.S. Cl. ............................. 386/68; 386/81; 386/95;
`386/125
`(58) Field of Search ................................ 386/1, 45, 46,
`386/95, 97, 125-126, 69-70; 725/88-89,
`102. 136
`s
`
`6,058.239 A * 5/2000 Doyle ......................... 386/46
`6,289,165 B1
`9/2001 Abecassis .................... 386/46
`6,311,155 B1 * 10/2001 Vaudrey et al. ............. 704/225
`OTHER PUBLICATIONS
`RCA Digital Video Disc Player RC520OP User's Manual,
`1996.
`* cited b
`cited by examiner
`Primary Examiner Andrew B. Christensen
`ASSistant Examiner Polin Chieu
`(57)
`ABSTRACT
`A method of, and a System capable of, replaying a portion of
`a Video comprising the Steps of, and the means for receiving,
`during a playing of a Video, a replay request to replay a
`portion of a Video, the replay request comprising a voice
`command; Storing a request position of the playing of the
`Video responsive to the replay request; Skipping, responsive
`to the replay request, the Video to a replay position that is
`responsive to a preestablished replay preference amount
`adjusted by Segment information Subject to an override
`range; enabling a playing of Supplementary information;
`playing the Video and the Supplementary information from
`U.S. PATENT DOCUMENTS
`the replay position; and discontinuing the playing of Supple
`4,305,131. A * 12/1981 Best
`mentary information responsive to the request position
`537.532 A 12/1994 Gelman et al. ... 3487
`responsive to a preestablished replay preference amount
`5,371,551 A * 12/1994 Logan et al. .....
`... 348/571
`adjusted by Segment information Subject to an override
`5.434678 A 7/1995 Abecassis .........
`... 358/342
`range. The Supplementary information including, for
`5,598.276 A
`1/1997 Coouson et al. .............. 386/46
`example, Subtitles, Substitution of the audio language, dif
`5,610,653 A 3/1997 Abecassis .........
`... 348/110
`ferent camera angles, other textual, Video, and/or audio
`5,630,006 A 5/1997 Hirayama et al. .........
`386/92
`information that may Supplement, complement, the infor
`5,636.200 A 6/1997 Taira et al. .............. 369/275.3
`SE A : 2. R et al.
`- - - 33. mation being played, and/or Substitute, the audio dialogue
`5,696 s.69 A 12/1997 S.S. s,
`and/or segment of a video; and/or any information which
`5.737470 A 4/1998 Fujinami ..................... 386/95
`may be utilized to elucidate, illuminate, illustrate, clarify,
`5,745,643 A 4/1998 Mishina ........
`... 386/106
`and/or explain, during a replay, a portion of a Video.
`5,828,370 A * 10/1998 Moeller et al. ...
`... 345/328
`6,018,612 A
`1/2000 Thomason et al. ........... 386/82
`
`(56)
`
`References Cited
`
`36 Claims, 27 Drawing Sheets
`
`
`
`ReCEIVE)
`RePEAY REQUES:
`
`STORE
`REEST Y
`POSITION2O2
`
`WIEWERSPECIFIC
`SKF
`:NO
`1211 sess
`y
`"EF 212
`
`TOSYSTEM
`AMOUNT
`
`23
`
`YES
`SET SkiP
`
`RECEIVED
`comiTMPORANOUS
`REQUEST
`
`NC
`
`AUSMENT
`SKIP
`AMOUNT
`25
`
`SEGENT
`INFORMATION 1Yes
`ACJST
`126 SKIP
`AMCUN
`
`NC
`
`27
`
`IEWERSPECIFIC
`NFC OFF
`
`ST MFC OFF
`ATEWAY
`AMOUNT 1222
`
`SET INFO OFF
`ATSYSTEM
`AMOUNT
`223
`
`SEGMENT
`22s
`INFORMATION 1S
`ADJUST
`1224 SSSF
`POSITION
`
`NC
`
`-
`SKEP
`ORPLAY Y
`POSITON23
`
`SUPPLEMENARY
`
`NESES) 24
`
`YES
`
`WIEWER-SPECIFIC
`INFORMATION-No
`242
`
`YES
`
`SYSTEM
`INFORMATION
`1244
`TURN ON EE Y
`NFORMATION243
`
`REPLAY
`VDEO
`
`-
`125
`
`SUPPLMENTARY
`INFORMATON
`PAYED
`
`
`
`YES
`
`INFORMATON
`OFF-POSITION--
`262
`
`YES
`
`ANY
`INFORMATION1263
`
`1
`
`Comcast, Ex. 1239
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 1 of 27
`
`US 6,408,128 B1
`
`Services
`provider
`
`102 communications module
`106 video Sound
`MULTIMEDIA PLAYER
`O7 multi-user
`108 multi-services
`111 system Cpu
`
`113
`dSp
`114
`video/
`data
`
`by
`
`O-
`
`LZ
`
`109
`power
`Supply
`
`103
`fixed
`Storage
`
`USer
`2CCeSS
`
`105
`compact
`Storage
`
`
`
`131
`USe
`interface
`
`132
`system status
`display
`
`101
`121
`video/data
`Control
`programs disc module
`
`104
`removable
`storage
`
`142
`141
`remote keyboard
`Control
`
`143
`VOICe
`recognition
`Viewer ACCeSS
`
`FIG. 1
`
`2
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 2 of 27
`
`US 6,408,128 B1
`
`
`
`204
`
`201
`
`2O2
`2O3
`
`251
`
`272
`
`289
`
`2O5
`
`211
`
`TIME 10:30AM 04-20-92
`RMDR 1:50PM 04-20-92
`
`DAD MOM TEEN CHILD
`
`MENU HELP SELECTION
`VA 2 CVA
`
`221
`
`AM FM UP DWN SET
`PREV NEXT DEL DIAL
`RMD 437 O SE. 219
`8 (B SS is
`i
`i
`v
`v.
`
`231
`
`241
`242
`
`.261
`262
`271
`
`281
`
`VOLUME
`
`CHANNEL 4, A
`of Vi Ao
`MARK CLEAR DNG
`PS/PLAY STOP FREW FFWD
`O
`REV FWD SFWD
`
`SREW
`
`WHAT ANGLE
`92 re
`
`SPEED
`
`FIG. 2
`
`3
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 3 of 27
`
`US 6,408,128 B1
`
`g &
`
`ESC
`
`301
`WHAT ANGLE SLOW FAST-302
`
`F1
`
`F2
`
`F3
`
`F4 N-304
`
`F5
`
`F6
`
`F7
`
`F8
`
`ESES
`KK|| <k. DDDDD
`ASEYE".
`DOKDD A
`
`F9
`
`F10
`
`F1
`
`F12
`
`TIME
`TL CH
`OO OO OOOOOOOO
`
`FIG. 3
`
`4
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 4 of 27
`
`US 6,408,128 B1
`
`HR 433
`
`
`
`| Ó-405 W. 41
`401 N464
`| |
`2NAY
`
`424
`
`434
`
`421 422 423
`
`5
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 5 of 27
`
`US 6,408,128 B1
`
`500
`
`REVIEW
`SCREENPLAY
`CONTENT
`
`O
`
`ADD CONTENT
`droNS AND 502
`TRANSiTIONS
`
`FILM
`SCREENPLAY 503
`AS EXPANDED
`
`MULTI OPTION
`CONTENT
`EDITING
`
`04
`
`CONTENT
`CODENG
`
`O5
`
`CONTENT
`MAPPING
`
`O6
`
`FIG. 5A
`
`6
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 6 of 27
`
`US 6,408,128 B1
`
`Segment Category Descriptive Structure
`51O
`512
`Code Description
`110 Profanity
`130 Violence
`511-135 Bloodshed
`150 Monsters
`17O Nudi
`175 Sex
`
`None
`1
`
`
`
`52
`
`522
`523,420 Expertise
`
`
`
`531
`
`
`
`Code Description
`610 inclusion
`
`Highlight summary condensed Detailed
`4.
`
`FIG. 5D
`
`Segment Generalized Descriptive Structure
`G
`PG PG-13
`R
`NC-17 N540
`41
`Symbols MPAA Trademark
`5
`FIG. 5E
`
`7
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 7 of 27
`
`US 6,408,128 B1
`
`LINEAR VIDEO He - boi
`1
`||
`2
`||
`3
`|| 4 ||
`5
`SCENES
`N602
`
`r
`
`SEGMENTS-603
`IL FRAMES-604
`4112 6-1 6026 6O27
`Code Description None implied Explicit araphic
`135 Bloodshed
`3
`619
`
`FIG. 6A
`
`4112,5205 35205-35350 6027
`Code Description None implied Explicit graphic
`135 Bloodshed
`1 -
`629
`
`FIG. 6B
`
`38975 5175 J 6026 6027
`412 L 3535
`632
`631
`633
`Code Description None implied ExplicitGraphic
`135 Bloodshed
`4
`639
`
`FIG. 6C
`
`VARIABLE CONTENT WIDEO N641
`
`631 632
`611
`4112-5109
`5175-6026
`
`FIG. 6D
`
`622 633
`
`35351-38975
`
`8
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 8 of 27
`
`US 6,408,128 B1
`
`
`
`NONLINEAR EDITOR - CONTENTOO1
`Edit New Bin
`Codes Map Link
`Utilities Print
`File
`Segments
`Scene 3
`3i
`
`II
`
`3V
`Scene 4
`Category wa
`Bloodshed
`Nudity
`Profanity
`
`|
`
`|
`
`|
`
`4112-6026
`4112-5205.5908-6026
`4112-520535205-35350
`
`|
`
`135-3 6027
`135-2 6027.
`135-
`6027
`
`FIG. 6E
`
`9
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 9 of 27
`
`US 6,408,128 B1
`
`
`
`702
`
`704
`
`Please enter the code for the category to modify:
`Please enter the level for this category:
`EXIT
`HELP
`PREV
`NEXT
`
`10
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 10 Of 27
`
`US 6,408,128 B1
`
`Violence
`
`716
`
`717
`
`
`
`
`
`
`
`
`
`713 715
`
`74
`
`723
`
`721
`
`Content-On-Demand
`
`Technology News
`
`
`
`I DI
`OI DI
`
`Database program for library
`1. of digitized programming announced, essee
`Software patents could turn information
`highway into toll road
`interactive programming networ
`success exceeds expectations.
`
`eroe
`sm
`
`1N
`
`2 72
`FIG. 7C
`
`11
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 11 of 27
`
`US 6,408,128 B1
`
`741
`Viewer Access
`
`745
`
`744 736 735
`
`SECT
`
`VIEWER CONTENT EXIT HP
`
`733
`
`
`
`Content Preferences
`
`SELECT
`
`SEL
`
`PASSWORD
`
`743
`
`EXIT HP
`
`9
`
`o
`
`752
`
`Chan+ 1stoodheads," "," "is
`Nudity "F")
`so
`"F," "
`
`NONE IMPLIED EXPLCT GRAPHIC
`
`753
`
`12
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 12 of 27
`
`US 6,408,128 B1
`
`761
`
`764 763
`
`XT HLP
`
`9
`
`o
`
`SELECT
`
`SEL)
`
`BROADCAST CAEGORY
`
`5
`
`6
`
`762
`
`Shopping
`
`FIG. 7F
`
`771
`Program Selection
`
`SELECT
`
`SEL)
`Chan
`
`
`
`
`
`
`
`
`
`
`
`
`
`772
`
`CONTENT EXT HELP
`
`8
`
`9
`
`o
`
`
`
`773
`
`FIG. 7G
`
`13
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 13 of 27
`
`US 6,408,128 B1
`
`
`
`-842
`> 30 FRAMES PER SECOND 841
`>60 FRAMES PER SECOND-843
`D120 FRAMES PER SECOND
`FIG. 8A
`
`-
`
`READ STREAM
`351-A 351-C 352-A 352-C 353-A 353-C
`N-851
`
`351-B 351-D 352-B 352-D 353-B 353-D
`
`351-A
`
`352-A
`
`353-A
`
`TRANSMISSION STREAM
`N852
`
`FIG. 8B
`
`14
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 14 Of 27
`
`US 6,408,128 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`VIDEO
`
`PLAY
`
`FUNicon 902
`
`
`
`VIEWER
`CONTROL
`ENABLED
`
`
`
`OBTAIN ID
`AND/OR
`PASSWORD
`
`INVALID ID
`AND/OR
`PASSWORD
`
`ID/PASSWORD
`ACCEPTED
`
`NO
`
`RESTRICTED
`USE
`909
`
`OTHER
`RESTRICTIONS
`
`
`
`
`
`
`
`EXIT
`MESSAGE
`917
`
`VIDEO
`PREFERENCE
`STRUCTURE
`
`CATEGORY
`PREFERENCE
`MATCH
`
`EDI
`PRIVLIGES
`
`EDIT
`PREFERENCE
`STRUCTURE
`
`ADECUATE
`PREFERENCE
`STRUCTURE
`
`EDT
`PRIVILIGES
`
`PREFERENCE
`SRUCTURE
`
`PLAY
`Rolf Es 921
`
`READ
`VIDEO
`MAP
`
`922
`
`APPLY
`PREFERENCE 923
`STRUCTURE
`
`SEGMENT
`SEGENCE 924
`
`PLAY
`SEGMENTS
`
`925
`
`FG. 9A
`
`15
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 15 of 27
`
`US 6,408,128 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`READ
`VIDEO MAP
`INFORMATION
`951
`
`952
`
`MULTIPLE
`PLAY
`FUNCTION
`
`YES
`
`PREVIOUS
`PLAY
`TRACKED
`
`YES
`
`OBTAIN PLAY
`INSTANCE
`
`
`
`PLAY INSTANCE
`SPECIFIC
`
`
`
`READ
`SEGMENT
`CODE
`
`FIRST PLAY
`NSTANCE
`955
`
`
`
`NO
`971
`
`RANDOM
`PLAY
`
`YES
`
`READ
`SEGMENT
`CODE
`
`OBTAIN
`RANDOM
`CODE
`
`ALTERNATE
`PLAYING
`METHOD
`
`SEGMENT CODE
`PLAY INSTANCE
`MATCH
`
`NO
`
`
`
`
`
`SEGMENT CODE
`RANDOM CODE
`MATCH
`
`YES
`SEGMENT
`PLAYABLE
`
`SEGMENT
`NOT PLAYABLE
`966
`
`
`
`YES
`SEGMENT
`PLAYABLE
`
`SEGMENT
`NOT PLAYABLE
`976
`
`991
`
`16
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 16 0f 27
`
`US 6,408,128 B1
`
`1001
`ACCEPT STRUCTURE 1002
`MODIFY STRUCTURE 1003
`1004
`
`ENTER NEW 1009
`VIEWER
`
`
`
`
`
`
`
`CREATE
`PREFERENCE
`STRUCTURE
`
`UPDATE
`PREFERENCE 1012
`STRUCTURE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`CATEGORY
`PREFERENCE
`MATCH
`
`1022
`
`BEARE
`SEGMENT
`TABLE/LIST
`
`023
`
`SHOW NEXT
`SEGMENT 1031
`
`1032
`
`SEGMENT
`TABLE
`1033
`
`1034
`
`O
`SAVE
`sENENT 1041
`TABLE
`
`FIG. O
`
`17
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 17 of 27
`
`US 6,408,128 B1
`
`USER
`COMMAND
`
`ACKNOWLEDGE
`USER
`COMMAND
`
`
`
`ACCESS
`PERMITTED
`
`YES
`ENABLE
`RETRIEVAL
`ROUTINES
`
`ACCESS
`DENED
`NO MESSAGE
`1103
`1104
`
`1105
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MULTI-STAGE
`REREVAL
`
`PROVIDER
`SELECTION
`
`
`
`
`
`
`
`
`
`
`
`VIDEO
`SELECTION
`
`PROVIDER
`LINKAGE
`
`TRANSFER
`VIDEO
`SELECTION
`
`LINKAGE TO Ca
`
`
`
`RETRIEVE
`VIDEO &
`VIDEO MAP
`1134
`
`APPLY
`PREFERENCES
`TO WIDEO MAP
`1135
`
`TRANSFER
`CONTENT
`PREFERENCES
`
`PREFERENCES
`TO VIDEO MAP
`APPLIED
`
`
`
`PROVIDE
`cSMZED 1139
`VIDEO
`
`NETWORK
`1111
`
`PROVIDER
`SELECTION
`1112
`
`VIDEO
`SELECTION
`1120
`
`FIG. 11
`
`18
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 18 of 27
`
`US 6,408,128 B1
`
`
`
`
`
`
`
`
`
`CSA)-1200
`
`RECEIVED
`REPLAY RECRUEST
`NO
`12O1
`
`YES
`STORE
`REGUEST
`POSITION 1202
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`VIEWER-SPECIFIC
`SKIP
`
`
`
`
`
`SET SKIP
`TO VIEWER
`AMOUNT
`
`RECEIVED
`CONTEMPORANEOUS
`RECQUEST
`
`SEGMENT
`INFORMATION
`
`VIEWER-SPECIFIC
`INFO OFF
`AMOUNT
`
`SET INFO OFF
`AT VIEWER
`AMOUNT
`
`SET SKIP
`TO SYSTEM
`AMOUNT
`1213
`
`AUGMENT
`SKIP
`AMOUNT
`1215
`
`ADJUST
`SKIP
`AMOUNT
`1217
`
`SET INFO OFF
`AT SYSTEM
`AMOUNT
`1223
`
`SEGMENT
`INFORMATION
`
`NO
`
`1225
`ADJUST
`INFO OFF
`POSITION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SKIP
`TO REPLAY
`POSITION 1231
`
`
`
`
`
`
`
`
`
`SUPPLEMENTARY
`INFORMATIQN 1so
`RECURED 1241
`YES
`
`VIEWER-SPECIFIC
`INFORMATION
`NO
`1242
`
`
`
`YES
`
`TURN ON
`SYSTEM
`INFORMATION
`1244
`
`TURN ON
`VIEWER
`INFORMATION1243
`
`VIDEO
`
`1251
`
`
`
`SUPPLEMENTARY
`NFORMATION
`PLAYED
`
`
`
`
`
`YES
`
`INFORMATION
`OFF-POSITION 1so
`1262
`
`
`
`YES
`
`TURN OFF
`INFORMATION
`
`FIG.
`
`12
`
`19
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 19 of 27
`
`US 6,408,128 B1
`
`1361
`
`1362
`
`1363
`
`1364 137
`
`hum-www.uyv-wi-rrar
`AUDIOy?tworror rr
`(AUDIO2- - -
`Vir-1373
`- - -
`
`1372
`
`AUDIO3 - re- - 1374
`AUDIO2) sea---v-e-r-sear'
`1381
`
`
`
`Mr.
`AUDIO2- - -nm-w
`(AUDIO2 F
`
`AUDIO3-a-Asea-------- Fr P --
`Y w
`\r
`--
`
`New-r-wv- ' "
`
`st-r
`
`7 r" tr.
`
`20
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 20 Of 27
`
`US 6,408,128 B1
`
`/N -1N 1Y 1412
`/N
`/N
`/N
`/N
`BBPBBPBBPBBPBBBBPBBP-1411
`
`1413
`
`FIG. 14A
`
`F.G. 14B
`
`1
`
`
`
`||
`
`1N ON 1422
`2 | 843434 5
`6
`N142
`
`21
`
`

`

`U.S. Patent
`US. Patent
`
`Jun. 18, 2002
`Jun. 18, 2002
`
`Sheet 21 of 27
`Sheet 21 0f 27
`
`US 6,408,128 B1
`US 6,408,128 B1
`
`
`
`1511
`1511
`
`
`
`1523 1522 1521
`1523
`1522 1521
`1524
`1524
`
`
`
`22
`
`22
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 22 of 27
`
`US 6,408,128 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TARGET
`SELECTION
`ENABLED
`
`
`
`YES
`
`1600
`
`S5
`
`TRANSMISSION
`
`601
`
`IMAGE
`NO
`PREDEFINED
`1602
`
`YES
`
`DISPLAY
`TARGE
`OPTIONS
`
`| 1603
`
`1604
`
`
`
`DISPLAY
`TARGET
`POINTER
`PROCESS
`POSITONING 1605
`DATA
`REPOSITION
`
`TARGET
`POINTER
`
`
`
`TARGET
`SELECTED
`
`NO
`1607
`YES
`
`OBTAIN
`A6E
`DEFINITION
`
`1608
`
`
`
`
`
`
`
`
`
`ZOOMING
`ENABLED
`
`NO
`
`YES 1610
`
`TARGET
`CENTERING
`
`
`
`NO
`
`YES 1611
`
`DISPLAY
`WINDOW
`FRAME
`
`1612
`
`ANALYSE
`FRAMET 1613
`POSITIONING
`region
`REPOSITION 1614
`FRAME
`PROCESS
`ZOOM
`DATA
`
`1615
`
`SIZE
`wiw 1616
`
`FIG. 16
`
`23
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 23 of 27
`
`US 6,408,128 B1
`
`-- N - N --- 1711
`1A 2A 3A 4A 1B 2B 3B 4B
`-r
`1701
`- 1 N-- N -1N
`- 1712
`FIG. 17A
`
`An ANGLE CHANGE
`
`FIG. 17B
`
`
`
`A ANGLE CHANGE
`
`FIG. 17C
`
`24
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 24 of 27
`
`US 6,408,128 B1
`
`D
`
`- O - 1741
`1A 2A 3A4A
`2B
`4B
`- 1701
`g-NOS-S
`1742
`2S
`
`FIG. 17D
`
`- 1751
`1A 2A 3A4A1B2B 3B 4B
`1701
`SR 1752
`
`S is
`O- Os —- 1753
`s-S
`
`S FIG. 17E
`
`25
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 25 of 27
`
`US 6,408,128 B1
`
`
`
`1761
`
`1762
`
`1764
`
`1761
`
`1762
`
`1773
`
`F.G. 17F
`
`F.G. 17G
`
`1784
`-1N- 1782
`
`1783
`
`1781
`
`S 2" ("?"-"
`S - S -
`1792
`
`1A 2A 3A 4A B2B
`
`1701
`
`An ANGLE CHANGE
`
`FIG. 17H
`
`26
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 26 of 27
`
`US 6,408,128 B1
`
`C 1797
`
`A END LOOPING
`FIG. 17
`
`27
`
`

`

`U.S. Patent
`
`Jun. 18, 2002
`
`Sheet 27 Of 27
`
`US 6,408,128 B1
`
`
`
`WIOEO
`TRANSMSSION
`
`801
`
`COMMUNICATION
`RECEIVED
`
`1802
`
`
`
`
`
`
`
`
`
`NTENDED
`RECIPIENT
`DENTIFIED
`
`
`
`
`
`NTENDED
`RECIPIENT
`DETERMINED
`
`ROUTE TO
`DEFAULT
`DEVCE
`
`ROUTE TO
`APPROPRIATE
`DEVICE
`
`COMMUNICATIONS
`SIGNAL
`
`COMMUNICATIONS
`ACCEPTED
`
`WDEO SERVER
`TRANSMISSION
`
`RETRIEWAL
`PAUSED
`
`VIDEO SERVER
`RANSNSSoN1822
`PAUSED
`
`
`
`WDEO
`1824
`TRANSMISSION
`PAUSED
`
`COMMUNICATION
`TRANSMISSION 1831
`
`
`
`
`
`
`
`CONTACT
`MANAGEMENT
`SOFTWARE
`
`PROVIDE
`FOR
`NOTATIONS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PAUSE EXCEEDS
`TIME LIMIT
`
`COMMUNICATION
`COMPLETED
`
`NO
`
`COMMUNICATION
`COMPLETED
`
`PROPORTIONATE
`REPLAY
`
`REPLAY
`FRAMES
`
`DENTIFY
`PRIOR
`POINT
`
`REPLAY
`PORTION OF
`SEGMENT
`
`
`
`
`
`CONTINUE
`FROM PAUSE
`POINT
`
`1869
`
`28
`
`

`

`US 6,408,128 B1
`
`1
`REPLAYING WITH SUPPLEMENTARY
`INFORMATION A SEGMENT OF A VIDEO
`
`BACKGROUND OF THE INVENTION
`
`15
`
`1. Field of the Invention.
`Systems for, and a methods of, processing, random
`accessing, buffering, and playing a Video utilizing the infor
`mation provided by a video map, where the Source of the
`video and video map are, for example, a DVD, a DBS,
`and/or video-on-demand transmission, and where the means
`for playing the Video comprises, for example, a DVD player,
`a personal computer, a set box, and/or a multimedia player.
`2. Description of the Related Art.
`A DVD (Digital Video Disc/Disk or Digital Versatile
`Disk) is a random access optical disc Similar in physical
`dimensions to a CD with Substantially more data Storage
`capacity. A DVD can Store one full length motion picture in
`one of two layers on either side of the disc. Many DVDs
`include Subtitles and/or closed captioning in a plurality of
`languages for a motion picture Stored in a DVD, multiple
`camera angles for a Scene, and the capability to play one of
`a plurality of different content versions (e.g. a director's cut
`or unrated version and an “R” rated versions.
`Devices capable of playing DVDS provide for, among
`other features, capabilities for fast-forward, fast rewind,
`skip-forward to the beginning of the next chapter, and to
`skip-backwards to the beginning of the current chapter and
`a previous chapter, the option to turn on/off the Subtitles, and
`if turned on, to Select the language of the Subtitle, Select
`language of the audio, cameral angles, and multiple content
`versions.
`While the DVD format provides many capabilities and
`functions exceeding those provided by, for example, the
`VHS format, as a result of concepts rooted in the conven
`tional linearity of motion pictures, many of the capabilities
`and functions Specified do not fully realize the potential of
`a randomly accessible digital Video, audio, and data format.
`SUMMARY OF THE INVENTIONS
`By contrast to the passive playing of Videos, as in the
`Viewing of a motion picture, or the interactive playing of a
`Video, as in the playing of a computer game, a primary
`element of the “autoactive' systems and methodologies
`disclosed herein is the incorporation of capabilities and
`environments that automatically customize the playing of
`VideoS to Satisfy the particular video requirement of each of
`a plurality of viewers, and that deliver to each viewer a more
`enjoyable video experience without requiring the level of
`active participation inherent in interactive Systems, the use
`of personal computers, and/or by primitive consumer elec
`tronic products.
`Accordingly, it is an object to provide a multimedia
`multiSource receiver transmitter player comprising random
`access and communications capabilities integrating Video
`and communication Services.
`It is also an object to provide a remote control device with
`the functionality required to control a multimedia multi
`Source receiver transmitter player comprising random acceSS
`60
`and communications capabilities integrating video and com
`munication Services.
`It is also an object to provide for a labeling of keys
`consistent with the functionality required to control a mul
`timedia multiSource receiver transmitter player comprising
`random access and communications capabilities integrating
`Video and communication Services.
`
`45
`
`50
`
`55
`
`65
`
`25
`
`35
`
`40
`
`2
`It is also an object to provide for the automated Selective
`retrieval of non-Sequentially Stored, parallel, transitional,
`and Overlapping Video Segments from a Single variable
`content Video Source, responsive to the viewer's video
`content preferences, and transmits the Selected Segments as
`a logical, Seamless, and continuous version of the Video.
`It is also an object to play a version of a Video that is
`highly responsive to a viewer's content preferences, by
`automatically Selecting among parallel, transitional, and
`overlapping Segments included, within the Video.
`It is also an object to provide content control over, for
`example, a Video's level explicitness, detail, expertise, form
`of expression, Subject matter, element development, and
`program length.
`It is also an object to provide an interactive Video game
`System comprising interactive video game Software, variable
`content game, and a video map defining Segments of the
`variable content game, furnishing a player of the interactive
`Video game the automatic and logical Selection of Video
`Segments responsive to the application of the player's video
`content preferences to the video map, and responsive to the
`logic of the interactive Video game Software.
`It is also an object to provide viewers the means of
`accessing available videos, Segments from a Video, and or
`Segments from a plurality of Videos by the use of keyword
`or a classification tree Structure as would be required by a
`user accessing a very large database of Segments and Videos.
`It is also an object to provide the means for a viewer to
`detail the Subject matter, Story line, and or general content of
`a desired video So that producers of VideoS may elect to
`produce and provide the requested Video.
`It is also an object to provide a variety of reading
`architectures that produce a SeamleSS reading of Sequential
`and non-Sequential Segments of a variable content Video
`from a Single video Source.
`It is also an object to format the Video stream, through
`placement and/or duplication of frames, group of frames,
`and/or Segments to enhance the Seamless playing of non
`Sequential Segments.
`It is also an object to provide for the playing of multiple
`play Segments of a variable content Video.
`It is also an object to provide a device that furnishes a
`previewer, Such a parent, the capability for efficiently pre
`Viewing automatically Selected Segments from the Video,
`responsive to the established preferences of a viewer, Such
`as a child, to permit the previewer to indicate the inclusion
`of the selected segments in the video to be viewed by the
`viewer.
`It is also an object to provide automated capabilities for
`efficiently retrieving and playing only a specified class,
`category, or Subject matter included in Segments within the
`selected video or set of videos as may be available from a
`database of Videos.
`It is also an object to integrate communications capabili
`ties and read/write optical disc player capabilities within a
`Single device to facilitate the downloading of a motion
`picture from a Source remote to the player.
`It is also an object to provide the means for a viewer to
`transport Video and content preferences to enable autocon
`figuring a player.
`It is an object to enhance the viewing of a Video by
`providing an elegant means and method for replaying for a
`Viewer a non-understood Segment of a Video.
`It is an object to utilize a Video's foreground and back
`ground audio elements to provide audio during a video
`pause.
`
`29
`
`

`

`3
`It is an object to provide for a plurality of fast-forward
`variable intermittent content skipping methods.
`It is an object to provide for an automatic display of a
`Viewer defined target within a video as the target moves
`relative to the Video's host image.
`It is an object to provide for a plurality of methods of
`playing presentation Segments of a set of presentation Seg
`ments of a Scene.
`It is an object to provide for the integration of a playing
`of a Video with the acceptance of a communication.
`Briefly, these and other objects are accomplished by
`autoactive capabilities and functions provided by Systems
`comprising, and methodologies utilizing, integrated
`processing, random accessing, and buffering technologies,
`variable content Video structures and configurations, control
`capabilities, Software functions, and Video information, Such
`as information that describes, defines, links, and/or com
`bines Segments of a Video and/or videos, with respect to a
`plurality of content, technical, and presentation categories,
`and playing functions, that facilitate the manipulation of the
`playing of a Video, or a plurality of Videos, to Satisfy a wide
`range of content, technical, and presentation preferences.
`Autoactive capabilities and function being provided,
`whether the source of the video is, for example, a DVD, a
`DBS, and/or a video-on-demand transmission, and whether
`the means for playing the Video comprises, for example, a
`DVD player, a personal computer, a Set box, and/or a
`multimedia player.
`These and other features, advantages, and objects, are
`apparent in the context of the detailed description, accom
`panying drawings, and appended claims, that follow.
`
`15
`
`25
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a Schematic diagram of a multimedia multi
`Source receiver transmitter player comprising random acceSS
`and communications capabilities,
`FIG. 2 is an illustration of a remote control device;
`FIG. 3 is an illustration of a labeling of function keys;
`FIG. 4 is a diagram of a Video and communications
`integrated network;
`FIG. 5A is a flow chart summarizing the steps of produc
`ing a variable content Video;
`FIGS. 5B, 5C, 5D, and 5E, are illustrations of segment
`descriptive Structures,
`FIGS. 6A, 6B, and 6C, are diagrams of three versions of
`a Video and corresponding content codes,
`FIG. 6D is a diagram illustrating the resulting Sequential
`and non-Sequential arrangement of Video Segments within a
`variable content Video;
`FIG. 6E is an illustration of an editing screen in a variable
`content Video nonlinear editing System;
`FIGS. 7A-7G are examples of content preference selec
`tion Screens and Screens utilized in the Selection of Videos,
`FIG. 8A is a schematic of a video disc player's multiple
`reading units architecture;
`FIG. 8B is a diagram representation of a Video reading
`Stream and transmission Stream;
`FIG. 9A is a flow chart detailing the steps of playing a
`variable content Video;
`FIG.9B is a flow chart detailing the steps in the playing
`of multiple play Segments of a variable content video;
`FIG. 10 is a flow chart detailing the steps of previewing
`flagged Segments,
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,408,128 B1
`
`4
`FIG. 11 is a flow chart detailing the steps of retrieving a
`variable content Video from a Video Services provider;
`FIG. 12 is a flow chart of a method of replaying a Segment
`of a Video with Supplementary information;
`FIG. 13 is an illustration of the utilization of foreground
`and background audio to provide audio during a Video
`pause,
`FIGS. 14A, 14B, and 14C, illustrate a plurality of fast
`forward variable intermittent skipping methods,
`FIG. 15 illustrates the automatic display of a viewer
`defined target as the target moves relative to the host image;
`FIG. 16 is a flow chart detailing the steps of creating a
`viewer defined window;
`FIGS. 17A through 17H illustrate a plurality of methods
`of playing a set of presentation Segments,
`FIG. 17I illustrates an example of a fragmenting and
`looping of a playing of at least one Segment of a Video.
`FIG. 18 is a flow chart detailing the steps of integrating
`the playing of a Video with the acceptance of a communi
`cation.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`The following are incorporated herein by reference: i)
`subject-matter-related U.S. Pat. Nos. 5,434,678, 5,589,945,
`5,610,643, 5,634,849, 5,664.046, 5,684,918, 5,696,869,
`5,717.814, and 5,724,472, by the same inventor; ii) U.S. Pat.
`Nos. 5,636,200, 5,630,006, 5,778,142, 5,745,643, 5,598,
`276, 5,644,507, 5,737,479, 5,778,135, and 5,678,012; and
`iii) the RCA DVD player RC5200P user's manual, 1996.
`For purposes of the present disclosure, various terms used
`in the art are defined as follows:
`The term “viewer' as used herein is meant to include and
`be interchangeable with the words “player” (when referring
`to a person), “subscriber', and “user”. That is, the term
`“viewer' is to be understood in the general Sense of a perSon
`passively viewing a Video, interactively playing a Video
`game, retrieving Video from a video provider, and/or
`actively using multimedia, internet, and/or communication
`Services.
`The terms “video', and “video program” are interchange
`able and refer to any Video image regardless of the Source,
`motion, or technology implemented. A video comprises still
`characters, graphics, images, motion pictures, films, inter
`active electronic games, and multimedia productions, full
`motion pictures and television programming, news, Sports,
`cultural, entertainment, commercial, advertisements,
`instructional, and educational programming. A video com
`prises Video, audio, Subpicture information, data and other
`information associated with the video. Unless otherwise
`qualified to mean a computer Software program, the term
`“program' is interchangeable and may be replaced with the
`word video. While a particular feature may be detailed with
`respect to a specified Viewing, gaming, or computing
`application, it is intended to apply the teachings herein
`broadly and harmoniously acroSS the different classes of
`applications that generate a video. The teachings herein with
`respect to a Video applied to, for example, movies, news,
`Sports, educational Videos, advertisements, informationals,
`commercials, and other Videos for the promotion of products
`and Services.
`A video also comprises a “variable content Video' and a
`“variable content game', which are characterized by a
`nonlinear architecture facilitating a variety of possible logi
`cal Sequences of Segments. A video herein comprises a video
`
`30
`
`

`

`S
`map and Video/audio/Subpicture information including
`parallel, transitional, and overlapping Segments to provide
`Viewing of a program's Story-line/interactive action at dif
`ferent levels of forms of expression, levels of detail, and, for
`example, length, with a greater SeamleSS continuity among
`non-Sequential Segments. Additionally a Video may include
`a user interface, Software program routines, and System
`control codes for controlling the playing of the video. The
`term “segment” refers to a part of the video whether one or
`a plurality of frames.
`The terms “non-sequential” and “nonlinear” are intended
`in the Sense that the video includes within a segment (e.g. an
`“R” rated segment) presenting a scene in a manner which
`would normally exclude the playing of another segment (e.g.
`a "PG" rated segment) with a different presentation (e.g.
`version) for the same Scene. In a physical Sense, the Seg
`ments are “linearly' arranged and one would follow the
`other, whether immediately or after one or a plurality of
`other frames and/or Segments.
`The term "seamless” is intended in the sense that the
`transmission of Sequential and non-Sequential frames is
`indiscernible to the eye, and not in the Sense of the natural
`Video Seams that result in the intended changes from one
`Scene to another, from one camera angle to the other, or from
`one gaming Sequence to the other.
`The term “content preferences' refers, Specifically and
`principally, although not exclusively, to a viewer's prefer
`ences for the form of expression, explicitness, the absence of
`objectionable Segments, content categories and levels,
`length, and detail. The Set of content preferences also
`include “technical preferences” and "presentation prefer
`ences” as herein identified. In the broadest Sense, the term
`“content preferences' further comprise S Video,
`programming, and Subject matter preferences
`("programming preferences”). Programming preferences
`refer exclusively to a viewer's preferences as to specific
`videos (e.g. Sega's “Sherlock Holmes Consulting
`Detective'), types of Videos (e.g. interactive video detective
`games), broad Subject matter of Videos (e.g. mysteries),
`and/or time and date that the viewer may desire to view a
`Video. Programming preferences are principally directed at
`the Selection of a Specific video.
`The term “video-on-demand system” refers to any point
`cast capable video delivery System that is responsive to a
`Viewer's programming preferences, i.e. provides a viewer
`Selected Video at a viewer-Selected time. Video-on-demand
`comprises for example movies-on-demand, Video dialtone,
`cellular Video, and digital Satellite Systems that are respon
`Sive to a viewer's time and date preferences and that usually
`provide VCR functionality in the viewing of a video.
`The term “content-on-demand system” refers to a video
`on-demand System that is additionally responsive to a View
`er's content preferences. A content-on-demand System not
`only permits a viewer to obtain a specific program(s) or
`Segment(s) of programs in response to the viewers program
`ming preferences, a content-on-demand System also permits
`the viewer to receive a customized version of the program(s)
`or segment(s) of programs that are responsive to the view
`er's content preferences. A content-on-demand System and
`the delivery of variable content Video Services is indepen
`dent of the Specific hardware and network infrastructure
`employed.
`The term “network' herein refers to any private or public,
`wired and non-wired Video transmission infrastructure, Such
`as may be provided by the utilization of one or a hybrid
`c