United States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,583,560
`
`Florin et al.
`
`[45] Date of Patent:
`
`Dec. 10, 1996
`
`USO05583560A
`
`[54] NIETHOD AND APPARATUS FOR
`AUDIO-VISUAL INTERFACE FOR THE
`SELECTIVE DISPLAY OF LISTING
`INFORMATION ON A DISPLAY
`
`[75]
`
`Inventors: Fabrice Florin, Mill Valley; Michael
`Buetmer’ Burlingame; Glenn Corey’
`San Rafael; Janey Fritsche, Mill
`Valley; Peter Maresca, Palo Alto;
`Peter Miller, Los Altos Hills; Bill
`Purdy, San Anselmo; Stuart Sharpe;
`§cCl.:<a1Vi¥est, both of San Franc1sco, all
`
`[73] Assignee: Apple Computer, Inc., Cupertino,
`Calif.
`
`[21] Appl_ No; 32,081
`
`[22]
`
`Filed;
`
`_]un_ 22, 1993
`
`(Under 37 CFR 1.47)
`
`6
`
`Int. Cl.
`[51]
`[52] U.S. C1.
`
`................................................... .. H04N 7/173
`................................... 348/7; 348/12; 348/13;
`455/5.1; 455/4.2
`[58] Field of Search .................................. .. 348/7, 10, 12,
`348/734, 906, 563, 564, 565, 566, 567;
`455/4.2; H04N 7/16, 7/173, 5/44, 5/445,
`5/00
`
`[56]
`
`References Cited
`
`Re. 32,532
`Re. 34,340
`4,290,142
`
`U‘S' PATENT DOCUMENTS
`
`3/1933 Atkinson ............................... .. 340/709
`8/1993 Freeman ____,_
`
`9/1981 Schnee et al.
`..... .. 455/3
`
`....................
`
`(List continued on next page.)
`
`FOREIGN PATENT DOCUMENTS
`
`0239884
`393555
`420123
`74476
`
`10/1987 Eumpean Pat OE '
`..... .. H04N 7/087
`10/1990 European Pat. Off.
`..... .. H04N 5/782
`4/1991 European Pat. Ofi.
`4/1986
`Japan ............................. H04N 5/445
`
`48879
`3501962
`
`Japan ........................... .. H04N 5/445
`2/1990
`3/1935 WIPO -
`(List Continued on next page.)
`OTHER PUBLICATIONS
`
`42nd Annual Convention and Exposition of the National
`Cable Television Association, Jun. 6, 1993, SF, CA, pp.
`82_89 M 1(1) -1 “Addr
`bl D
`d W- hD nl
`d_
`able dperiiionvsl Y
`essa 6
`Eco at
`1t
`0w 0a
`
`(List C011tiI1115d on next Page-)
`Primary Examl.ner_John K_ Peng
`Assistant Examiner—Chn's Grant
`Attomey, Agent, or Firm—Blakely, Sokoloff, Taylor & Zaf- ‘
`man
`
`[57]
`
`ABSTRACT
`
`An interactive audio-visual (A/V) transceiver is advanta-
`geously coupled to a television and/or telephone (T/T) cable,
`a TV, a video recorder (VCR), and other A/V devices. The
`A/V transceiver switches data between a program/service
`provider and the connected A/V devices. In one embodi-
`ment,
`the transceiver includes three primary modules, a
`main module including a CPU’ 3 system bus’ System
`memory, an infraqed
`control unit, an audio-visua_1 bus,
`an A/V decoder, an AN processor, and an A/V encoder, an
`A/V connect module including a number of tuner/demodu-
`lators and a switch, and an optional CD ROM module. The
`A/V transceiver hardware is complemented with an operat~
`ing system and software program which supports the func-
`tions provided in the AN user interface. Additionally, a
`remote control device is provided to communicate with the
`A/V transceiver to interactively manage selection of pro-
`gram and service sources, selection program and service
`offerings from any selected source, viewing of selected
`Pfogfam Ofienngs, and 1nt<=raCt10n_W1ti} Selected Sen/106
`olferings. The remote control dev1ce 1s advantageously
`provided with a basic AN control button group, an interac-
`tive control button group, an auxiliary control button group
`anda numeric key pad to facilitate control of the transceiver.
`The interactive control button group includes an info button,
`a list button, a categories button, a pix button, a mark button,
`a jump button, and a pointing device consisting of up, down,
`left, and right arrow buttons, and a center select button.
`
`16 Claims, 50 Drawing Sheets
`
`
`
`SE/?l’I6E
`PROVIDE/l
`
`W’//E/I /z////a—y/5z/nL
`DEVICES
`
`
`
`58 TELEV/SIM SET
`
`'50 ‘Wm
`
`1
`
`Comcast,
`
`1
`
`Comcast, Exhibit-1004
`
`

`
`5,583,560
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`FOREIGN PATENT DOCUMENTS
`
`5/1939 WIPO-
`4329553
`12/1989 WIPO-
`8912370
`2/1990 W190-
`9001243
`396092 1“1990 WIPO-
`9118476
`11/1991 WIPO’
`9311640
`6/1993 WIPO _
`9311639
`6,1993 w11,O_
`9311638
`6/1993 WIPO.
`_
`_ OTHER _PI_JBLICAT1O_NS
`18th Triternational Television Symposium and Technical
`EXhib1t19n,
`Jun.
`10, 1993, _ Montreux, Switzerland, p.
`555-567, Hoarly “Multimedia on Cable Television Sys-
`tomS”-
`_
`_
`_
`Technir§:15I%1;(7:los1iIre Bulletin, go]. 34, Ncz 71?, Dec.
`, pp.
`—
`, “ nteractive
`omputer
`on erence
`SorV.or”-
`_
`_
`_
`Tech§i:1caél4DiIsclosure B11.1)11et1I1, fvol.C36, No. 0;, £31.
`-PP-
`-
`Iitoraotivo
`ovioo or onvontion
`5
`to1roProVoFimCtiona1ity”-
`Elektor, No. 4, Apt 1990- PP- 10-121 “D213-Homobus Fiir
`Audio and Video”-
`42nd Annual Convention and Exposition of the NCTA, 6/6,
`1993, SF, CA, pp. 223-236, Bestler “Flexible Data Struc-
`tures and Interface Rituals for Rapid Development of OSD
`App1ications”.
`Alexander, Wsualizing cleared—ofi" desktops, Computer
`world, May 6, 1991, 1,_ 20
`Hiroshi Ishi, Kazuho Arita, Clearface: Translucent Mul-
`tiuser Interfacefor Team Wor/(Station, ECSCW, Sep., 1991,
`1111 6_10_
`Hiroshj Ishii, Naonli
`Workspace’ Communications
`34 N0 12 pp 37_50
`Ar’ticle.entitled-' Learning Considerations In User Interface
`D .
`_ Th R‘ M d l_
`[h _ P Hi k P Ch
`_
`b1._
`99'3"‘
`9 00'"
`" 9’ 9",‘. 9"
`9
`°
`'
`9"’ 9“ 1
`cation of the Software Portability Laboratory, University of
`W0§°r1°°- W0‘0fl00- 0_ma‘10- C_9“00_0v 101- 1934-
`_
`_
`A9101‘? 0111111009 C’w1'0"/04040700/10" 9f 1/19 414110 §"8"a/
`Processor Setup For A PC Audio Editor; publication of
`International Business Machines, IBM Technical Disclosure
`Bulletin, vol. 30, No. 10, Mar. 1988.
`Article entitled: Browsing Within Time—Driven Multimedia
`Documents; authors: Stavros Chiistodoulakis and Steven
`Graham; publication of the Institute for Computer Research,
`University’ of Waterloo, Waterloo, Ontario, Canada, Jul.,
`1988
`Interactive
`An
`Im am“
`entifled_
`Articie
`,
`_
`,
`1,’
`~
`_
`_-
`Natural—Motzon—Pzcture Dedicated Multi—Media Authoring
`System; authors: Hirotada Ueda, Takafumi M1tyat'ci1<o, and
`Satoshi Yoshizawa; Communications of the ACM. Man.
`1991, pp. 343-350.
`
`Toward an Open Shared
`Dec‘,
`vol-
`
`.................................. .. 358/86
`4/1983 Lambert
`8/1985 Sokonick ................................ 340/721
`8/1985 Campbell et al.
`358/122
`11/1985 Pike ........................................ 364/900
`2/1986
`358/86
`11/1986
`.. 340/747
`2/1937
`/
`11/1987
`370/94
`5/1988
`__ 353,147
`6/1933
`340/709
`9/1938
`304/513
`11/1933
`.. 340/721
`3/1989
`372/92
`5/1939
`340,705
`7/1939
`358/343
`7/1989
`364/133
`_
`_
`5/1937
`............................ .. 364/550
`11/1989 Ingle et al.
`12/1989 Monslow et a1.
`380/10
`..................
`2/1990 Beard etal.
`..
`340/706 .
`4/1990 Duflield ............................. .. 358/191.1
`4/1990 Henderson et a1_
`__________________,_ 340/325
`6/1990 Atkinson ......... ..
`340/710
`6/1990 Rowe et al.
`..
`364/188
`7/1990 Beard et al.
`......
`340/706
`1/1991 Johnson et al.
`.. 358/86
`2/1991 Monslow et a1.
`.. 380/10
`4/1991 B1)’ 1‘-131 ------
`354/900
`5/1991 P°°°°k 91 31-
`353/86
`9/1991 Strubbe et a1.
`358/335
`10/1991 Kolnick ........................
`364/521
`1'I6I1(l6l'SOI1, J1‘. Ct
`.......
`340/712
`9/1992 MacKay et al.
`''''" 358/86
`9/1992 Fmaro ''''‘'
`. 358/194.1
`9/1992 Young
`395/157
`10/1992 Hoeber .... ..
`395/157
`10/1992 H061)er at 31
`1/1993 Martinez ................................... 358/86
`3/1993 Wilson et al.
`......................... 370/94.2
`4/1993 Kwan ____________
`__ 353/36
`5/1993 Hearty at 31
`,_ 353/35
`6/1993 Stmbbe ................................... .. 358/86
`8/1993 Thompson, Jr.
`........................ 273/439
`8/1993 Rovira etal.
`-- 370/77
`9/ 1993 Liitefal 01 31-
`-- 353/35
`10/1993 Vogel """"" "
`348/906
`10/1993 Chaney et a1.
`. 358/191.1
`2/1994 Glick et al.
`.... .. 379/90
`10/1994 Young et a1.
`348/563
`10/1994 Banker et 31.
`_ 348/12
`4/1995 Remillard ...................... ..
`.. 379/96
`4/1995 Goldstein .......................... ..
`348/7
`4/1993 Young ..................................... 358/142
`
`
`
`.
`
`
`
`.
`
`4,381,522
`4,533,9i0
`4,536,791
`4,555,775
`4573072
`’
`’
`4,622,545
`416411205
`4,7
`,121
`4,748,618
`4_750,036
`4,772,882
`4_735_403
`4,812,834
`4,329,558
`4,847,604
`4,847,700
`4,373,623
`4,884,223
`4,890,320
`4,899,136
`4,914,517
`4,914,732
`4,931,783
`4,935,865
`4,939,507
`4,987,486
`4,995,078
`59031353
`5-014-125
`5,047,867
`5,062,060
`
`5,148,154
`5’151’782
`5,151,789
`5,155,806
`5,157,768
`5,177,604
`5,195,092
`5,205,722
`5,220,420
`5,223,924
`5,236,199
`5.239540
`5347-347
`51253066
`5,253,067
`5,283,819
`5,353,121
`5,35-7_2-76
`5,404,393
`5,410,326
`B1 4,977,455
`
`2
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 1 of 50
`
`5,583,560
`
`
`
`Em.$33..Eu3
`
`E.._.§,Q9
`
`_.9...
`
`,r|I|.)\|||II\
`
`E8.
`
`kokvaD8228
`2::{V
`
`52%:
`
`8
`
`§..m
`
`
`
`g.\.‘.9:
`
`f.\E
`
`§_§
`
`REESE
`
`
`
`=s%_-s%_\$.$E
`
`mics
`
`3
`
`
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 2 of 50
`
`5,583,560
`
`
`
`ca:._m>_momc.E._.
`
`_IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIa\oH:UoS_
`.o__eEoo_.EH_mo>HmmamU3maAll _.u._u,..HHEHHHHasHHHHE296_..H...H.msH.HHC._HmvoocwH.H3"owmwooiI‘.00HHH>\<>2mHmcoszw_mm_m8HHHH2.auooooHHH>2.coEms_HiHHWmu5H5HumHHeosoz
`HofiauoemoHo_nm0HH>2Hmc9Ho_mCoHHH.......................................--:o_m_>o_m._.
`
`
`
`
`
`
`
`
`
`
`3&0EHH.H.._mm...__..
` ...............I.___HHm_:uo_HHmm8Num..ummm_wEoo>\<,.«THm.¢oEo_2
`
`_o._Eo0H_HHE396
`
`4
`
`
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 3 of so
`
`0/D5,385,5
`
`....m§§...muxfim.§9S§SE.«.8.ms-||+I|
`
`
`
`§..6EQQ<_8..mmzzxaSea“E:Eu3[.411_
`Ea:
`
`
`
`we-@952...3%"§-§.§uB5,:$§ESEE"
`
`H.3.-u.mz§.a_on.o_.._283..ESE"_[LN
`
`
`
`§+.§§.3.wwasES..
`
`.4656..
`
`39.5Esgem:ESE
`
`.o_§§.m.6M35o
`
`__.......E§s2m..<§u..
`
`.mE§E\§§.:§..
`
`
`
`
`
`
`5
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 4 of 50
`
`5,583,560
`
`FIG. 4b
`60
`
`6
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 5 of 50
`
`5,583,560
`
`FIG. 50
`
`HQ 5b
`
`7
`
`

`
`U.S. Patent
`
`o3\\m.0_....
`
`Dec.10,1996
`
`Sheet6of50
`
`5,583,560
`
`8
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 7 of 50
`
`5,583,560
`
`om?
`
`
`
`
`
`o_.N Eum~\D
`
`omFWEE8-._<o_ooEmam>>mWvA/_%__
`
`mom.\I/
`
`K
`
` mowK:.Eos_mi»m.:m>>o:m.
`
`w.@5\fi.
`
`#2.
`
`9
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 8 of 50
`
`5,583,560
`
`
`
`
`
`._mE.._m:.xom_mE05:
`
`mow
`
`.w:=mums_u._._m_co._.l\
`
`
`.£_Ew.62.£5,LQ
`
`
`
`10
`
`m .
`
`o_.._
`
`10
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 9 of 50
`
`5,583,560
`
`om_.
`
`
`
`Emw_.Euoamamm-_2<mwomn_>.__<o\>
`
`.mom
`
`..6%_>Sgo»3.8;25En/D/
`mafia2::2.226.o._w8x@SN
`
`
`
`o_.N
`
`
`
`
`
`I._.zO_>_m=._._.w:m>>o_._m
`
`o3\\
`
`m.O_n_
`
`11
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 10 of 50
`
`5,583,560
`
`00..
`
`
`
`oom_oz<Ez<m.2£20...._.mm.DF
`Sm...mE_m.w>\>mom...96oe=2_._mum...95«mm:/UV
`
`cmsmmmn..m_....=_Ew:.._o_..459
`
`
`
`
`Now
`
`mom
`
`o_.N
`
`_.ON
`
`O_.o_.._
`
`Om_.
`
`12
`
`12
`
`

`
`5,583,560
`
`.._mu._o9.E<s_on.z_.o8.r:8
`
`mom
`
`3%2.5/8.
`
`
`
`
`
`uoEmmm.>au2.2Em._mo._n.2050
`
`U.S. Patent
`
`W.
`
`moom_oz<mM1'.MIto?
`
`13
`
`13
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 12 of 50
`
`065,385,5
`
`93.m_.o_.._
`
`
`>._2w
`
`
`
` _mcow.mn_m.w<mm2<moomn_._4<Q52%mt?>%w:=fiV___:C.J:
`
`ONN
`
`
`
`
`
`
`
`
`
`.€mn_A_v>muo._.xoom>.rzor EEon_m.._mmmcm2.:2mmwl.Io3O___m_.mzmm:mmo5.E_z_.
`
`_.mm
`
`23Ecmi>003__B§ammAvm:_Ea_8s>0»9.
`
` _Efimoi
`.m_:aon_Ozz.om
`
`2328»x2:twamzbemQBzmkE$E_.r5:A_v8..wasxxzm:
`
`
`
`Emgmoi>umEo0mm?mr
`
`
`
`m_>o_2umm_2._.wm_.
`
`14
`
`14
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 13 of 50
`
`5,583,560
`
`Em
`
`mmE_x_.SEAV3,u__>>
`
`H
`
`Efimoi._m_:aoa
`
`._OZZom
`
`m_>oEEmm
`
`m_>o_2no...
`
`EEmo.n_.€mEo0
`
`IE52$E.m3X2:.2
`
`__8§$m_Avm:_EaE§
`
`2:Ecmi
`
`E=._ou_mzmmmcmfi
`
`mzmzbeammo...
`2m»9viz2>003O._.9._._2mNe
`
`
`
`éc.EmaAv.§.£.xuom@2.29B05
`
`_m:ow_mn_
`mw<m
`
`.A___m.18228..
`
`mmo_.—
`
`m_.o_.._
`
`15
`
`15
`
`

`
`U.S. Patent
`
`Dec.10, 1996
`
`Sheet 14 of 50
`
`5,583,560
`
`suemm82:3.5
`
`
`
`5:0225.Qesozm
`
`>___m22‘am:
`E05Ecoemm
`
`mmo:>.rZEmw<m
`
`E:._ou_mzmmmcmz
`
`Emm_.
`
`__B§.$mAv9_EaE_>>
`
`2:§a_n_
`
`maozbogmn_o._.
`
`2m»2V222>0030._.or
`
`
`
`wmE_._.can.A_Vmow25>
`
`
`
`m_>o_>_ao._.
`
`
`
`m_>o_2umm
`
`_>:.m9X21.5
`
`on.
`
`16
`
`16
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 15 of 50
`
`5,583,560
`
`__§e.8mAvm:_:8E§
`
`OHm_
`
`P5Ecmi
`>00vw
`
`
`
`$52.::n_Avmo...25>
`
`xxzm_
`
`EEon_mnmmmcflz
`
`P2mm_
`
`>___m:82.89.
`mmor.
`
`$.62boa8»
`zm»mr
`
`m_>o_2nob
`X21hr
`
`m_>22fimm
`
`Ekmmr
`
`:92mm5m_:o._.6
`
`£8225wfiaogm
`
`bem_mcom.mn_
`mw<m520m
`
`>FZor
`
`om:
`
`17
`
`17
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 16 of 50
`
`5,583,560
`
` .
`
`
`
` cozfimcmomE._.E.>msoégmmEo$co._“mum58.8022vmcamomflmracoo2%m259..2.»xx".N
`
`
`
` 3oéb2m_E8_mn_32m>30...8E_on_Av9.62220m5see"552=o._.,mN>xN
`
`xzflaocwEmSm:mEc_mmoI
`
`AEm>._.zE
`
`18
`
`18
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 17 of 50
`
`5,583,560
`
`HHHHHHHHHH
`
`._
`
`
`
`
`
`m3m_>_mE__mcom.mm.:2mwr
`
`rot).52Av2,32
`
`
`
`xmmm.€mwA_vQsmz
`
`
`
`:o:m._m:m0w_:._..
`
`
`
`__Bsou.omm__oo
`
`
`
`Em_co._.m_m_omam
`
`:mEmco:m
`
`522,0Avxmmz,2;»xoomE2
`
`
`
`sea_m_o8m0:522:.mw<
`
`
`
`wmosommmAvcomccow.v_5:20
`
`m_5_.E_Avxooo585..mmo
`
`of
`
`19
`
`19
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 18 of 50
`
`5,583,560
`
`mmm
`
`
`
`oswiufixoconuu:2.
`
`
`
`
`
`:o«Eu>oxv»:mushxuom
`
`12%ES>m_23mmV__
`
`_m.m_
`
`
`
`m..o_....
`
`
`
`uuuuaomum:Om_.—£OHoox
`
`
`
`xuumhauummnoz
`
`
`
`893duwuoamu«>oxBE.
`
`:08:329.52
`
`
`
`coauuuuauu32$.
`
`
`
`AA<nHOommumdou
`
`a5.om.S:_
`
`zmxuzomam
`
`of
`
`20
`
`20
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 19 of 50
`
`5,583,560
`
`ws_<m.oomn_.:<QA2%522223mmV__im___
`
`:92262Avwzmz
`
`
`
`59:3m_m_omam
`
`
`
`mmosommmAvcowccow.v_.220
`
`
`
`xmmmzzmw.A_VwzmzN>x
`
`
`
`893_m_o8mm_>o_29:.mw<
`
`
`
`cozfimcwomE._._.._.>
`
`
`
`$_<om3.__m._.:m
`
`
`
`cmemcozmxxm
`
`
`
`
`
`__S8on_mmm__oo2%
`
`-r-(\JC')Vl-D(Dl’\®O)
`
`flH||h|I||H|||l||H|||IHH>B
`
`
`
`25:2.Avx805852.mmo:.sm_>.m>OAv5.83mE._.xoom>._.z
`
`C1-
`
`
`
`
`
`m>$_..c2:__m:om.mm._.s_mmm
`
`
`
`m..9“.
`
`of
`
`21
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheét 20 of 50
`
`5,583,560
`
`«mm
`
`ON.o_.._
`
`\\
`om:
`
`22
`
`22
`
`

`
`FIG.2”:
`
`23
`
`

`
`U.S. Patent
`
`W
`
`w5,%
`
`
`
`
`
`amm_>_<mOOmm|_u_<uwm_EOmum._.<0
`
`fl,mwmm5m5EmmomLymsom
`
`E52EEEV
`
`mmoi
`
`24
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 23 of 50
`
`25
`
`5,583,560
`
`25
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 24 of 50
`
`VN.o_.._
`
`\
`om.
`
`26
`
`5,583,560
`
`26
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 25 of 50
`
`5,583,560
`
`27
`
`OQ1
`
`-
`
`FIG.25
`
`27
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 26 of 50
`
`mm.07..
`
`28
`
`5,583,560
`
`28
`
`

`
`U.S. Patent
`
`m,mw.W.
`
`5,583,560
`
`
`8oK:._m_c2.__S$mmA§.mc_::mu:_>>@.69m>___m=323?mmoE\\oJ8msmnolmfi2:3.6N>x5M“.
`$2.00mz_mm%__m£mxwmm
`
`2%mDooo
`
`8oE__8.oou_
`
`29
`
`hm.o_n_
`
`29
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 23 of 50
`
`5,583,560
`
`oom
`
`mom
`
`mm.o_..._
`
`
`
`
`
`2%mtov>muwS.¢2...,”
`
`ao5zm_:o_.__2.8mm2%m
`
`
`
`smumflmfim_...o.3mN>x.\.
`
`own
`
`___m=32.39mmoE
`
`
`aE:_meoou_
`
`omm__oom2.8_m£m%mmmk6:_:=%c_§pm2\\mm.
`
`30
`
`30
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 29 of 50
`
`5,583,560
`
`&
`
`fiflflg
`
`
`
`@._._<mmm<m”m._.moaw
`
`mom
`
`mmor.
`
`
`60559:8.__m8mmm
`am_$.._w69:3.6
`
`zn_mm
`
`N>xN.
`
`EmamzEm:
`mmoE.
`
`2%323555.3
`
`._._<mmw<m
`
`Sm
`
`of
`
`31
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 30 of 50
`
`5,583,560
`
`mum
`
`2%333555.
`
`\Vsmémm8£33.5N>x
`
`28:30xmwE.>mm._._mo><H_
`
`£822535305220
`
`505$25Av35»xoomPE2bogw_m:om.mn_m.w<
`
`m
`
`>___mamz>mm_.mmo:
`
`
`
`
`
`
`
`
`
`__E§_$mA$9._EaE_>>.52E35;m_.mmm:m_2.:2mNF
`
`32
`
`32
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 31 of 50
`
`5,583,560
`
`
`
`$oE_._..€mn_Av>muo._.xoomPEor
`
`
`
`
`1-(DI\w
`
`$399mmBm_=o._.5N>x
`
`w_mnE>Oxwwrr>
`
`
`
`om<om._.=m._.=m
`
`
`
`ms_<m_oomn_pzmsommu
`
`ism_m:ow.mn_mw<m£:o_2m_E.mgesocw.220
`
`oom
`
`
`
`@Fzmnommmnmm:mo>«,.E
`
`D
`
`.._.Z.n._Du
`m5\v
`
`
`
`
`
`E_on_mzmmmcmz.:2mm_.
`
`>___m=32.89wmoI
`
`33
`
`33
`
`
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 32 of 50
`
`5,583,560
`
`36mm.o_.._
`
`
`__m£9ammAvwymceamuczs
`
`
`\y605__m2oo.._mmm__oo
`
`JErrEmaA_v288.xoom\owema6250...6
`
`
`
`m_2<moomn_omv_m<_2
`
`2%mt?>%.u.S£.85
`
`34
`
`34
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 33 of 50
`
`5,583,560
`
`FIG.33
`
`ix0
`
`T-
`
`35
`
`35
`
`

`
`U.S. Patent
`
`Dec. 10,1996
`
`Sheet 34 of 50
`
`5
`
`5,583,560
`
`1-
`
`WC
`
`’)
`
`36
`
`36
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 35 of 50
`
`5,583,560
`
`1-
`
`Q(
`
`0
`
`FIG.35
`
`37
`
`37
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 36 of 50
`
`5,583,560
`
`ONV
`
`
`
`.__<s_mzozmmax:
`
`5©
`
`mo>”:zm2
`
`mm.o_.._
`
`oar
`
`38
`
`NV\
`
`mm¢
`
`FNV
`
`38
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 37 of 50
`
`5,583,560
`
`FIG.37
`
`39
`
`39
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 38 of 50
`
`5,583,560
`
`40
`
`40
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 39 of 50
`
`5,583,560
`
`of
`
`DDEBDEDDEB
`
`mm.o_.._
`
`GZ:u_m<._.w
`
`.
`
`
`
`Z_mO.._..._mo_mm<n_
`
`tfim..:m_.Ez.uF:m>>
`
`>mDMODDOEA
`
`>mom.6mm_o
`
`
`
`z_mo.Emo_mm<n_
`
`can
`
`ZIO_..
`
`.OJIIIIIIIII9QImlinfl .>._._:om
`
`41
`
`
`
`
`

`
`m
`
`5,583,560
`
`
`
`
`P._S.ov.0EU..
`
`
`
`\\muemm2.259:2,
`
`ofmDEEDEEEEDE
`
` am»m:52.van:2.m.EwtoomxmE.>~mm_wmcm;>>m,
`
`.:_Eomwm.:zoE2&<
`
`
`
`foamwmmuom.-mEm.u.__m£man“2uombcou
`
`
`
`
`
`
`
`
`
`
`
`.__m£m>m=mnrgmo>9:cm.96>_._mcmmw_2::m32EmEmwmnmE92.umo.9:._m>oom_ncoow.35Smm.xo:_
`
`
`
`42
`
`42
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 41 of 50
`
`5,583,560
`
`muv
`
`OC
`
`O V
`
`'‘\
`
`EDDEDDDEEE
`
`In.o_u_
`
`H60»
`
`
`
`o.Eo>>mn_nm=.
`
`
`
`
`
` 8%om._.__mmmu_..:>>30>Immmsszmm<.m=>os_m:o>mm._.zmmw<m.E
`
`
`
`
`
`
`
`.-._._@9.._.Ea...w.awmfi
`mamll4£mcm._‘HIM!
`IA:
`
`Imlmlmfi
`
`43
`
`43
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 42 of 50
`
`5,583,560
`
`EEEEDEEBEE
`
`EDEDDBEEBE
`
`NV.2...
`
`
`
`n_.._mO>>mn_n_m:.
`
`wm:.:z__>_mZ_mE<._.m
`
`44
`
`44
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 43 of 50
`
`5,583,560
`
`u
`
`o
`
`DN
`:25m....c2.
`/K
`
`
`
`
`
`__m29_$mAv9__E_mE_>>E2E:._ou_wzmmmcmz.E2mNw
`
`
`
`
`
` wmC._mo><u_QAs_%mi:>mB::Ev__mw.=
`
`mvoi
`
`.25$2.2.
`
`.mE_Eo=mum>E0Ecaw
`
`
`
`>muo._.>mo_o_.Eom._.
`
` ..
`
`
`
`mE_._.ban.Av>muo._.xoom>._.zE.
`
`
`
` hem_mcom._mn_mw<m5%:«EHeszocw220m8.98umam.$30...5N>xN.
`
`m_mnE>oxmm_..r>oom<om.:_m
`
`.:_m_.
`
`45
`
`45
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 44 of 50
`
`5,583,560
`
`46
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 45 of 50
`
`5,583,560
`
`mow
`
`
`
`>._Bom:n_aocm>._.
`
`mnmz-mEm_=_>>
`
`
`
`._m3m._m:50...
`
`47
`
`vo¢
`
`N9.
`
`47
`
`

`
`U.S. Patent
`
`Dec 10, 1996
`
`Sheet 46 of 50
`
`5,583,560
`
`
`
`
`
`5§um:.xm_._m9._.mo_:_,
`
`A5|IL
`
`D
`
`2:.
`
`om:
`
`
`
`ommmammE22wxo:._@Emzxooom_.__m5n_
`
`
`
`bmxmmmEoImtmmmm
`
`
`
`§.mmE39m_963
`
`ofmQE9:
`w_>omo
`
`mv.O_n_
`
`48
`
`48
`
`
`
`

`
`U.S. Patent
`
`Dec 10,1996
`
`Sheet 47 of 50
`
`5,583,560
`
`
`
`
`
`._2om:xm_mm_._.mos:
`
`
`
`bmxmmmEoIwtmmmm
`
`
`
`_9wmE2§mw.xo:._
`
`49
`
`49
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 48 of 50
`
`5,583,560
`
`of
`
`
`
`
`
`.E_owmmzawm_:__>_$33..
`
`mv.o_n_
`
`
`
`o=.oo:uam0ucmowwmaww
`
`
`
`m.n_._o39:E0:.9_wE
`
`
`
`55>.w._mxmEommmfixm
`
`
`
`.2m_NNoc5En_,8otmn_
`
`ucm8:85._mumm_
`
`
`
`.oc_8:aamo69:8
`
`
`
`ummnmoom.:s=._
`
`@33$2oz853mm;
`
`50
`
`50
`
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 49 of 50
`
`5,583,560
`
`mo...
`
`Noowaaov
`
`morn.
`
`
`
`xm._.wmam._m0
` .Em.B=m-mmm-.._.En_E.EEmzazEmu.35
`
`
`
`om.‘
`
`aw.O_.._
`
`51
`
`

`
`U.S. Patent
`
`Dec. 10, 1996
`
`Sheet 50 of 50
`
`5,583,560
`
`Noorasov
`
`morn.
`
`om.o_.._
`
`
`
`
`
`xm._.wmfiw._mo
`
`32050>
`
`
`
`2,o:oE9._.mm.38§__%mg=_3
`
`.s_%923
`
`52
`
`52
`
`

`
`5,583,560
`
`1
`METHOD AND APPARATUS FOR
`AUDIO-VISUAL INTERFACE FOR THE
`SELECTIVE DISPLAY OF LISTING
`INFORMATION ON A DISPLAY
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention:
`
`The present invention relates to the field of audio-visual
`systems. More specifically, the present invention relates to a
`system for selectively viewing and interacting with pro-
`grams and services from a number of prograrn/service
`sources, a control device for controlling the system, and the
`methods and apparatus incorporated in the system for man-
`aging selection, viewing, and interacting with the prograrnl
`service olferings.
`2. Art Background:
`Over the past 40 years, television and motion pictures
`have become an important aspect of everyday life for people
`in the industrialized world. The development of advanced
`technology in the areas of digital and high definition tele-
`vision (HDTV), video recording systems, laser disks and
`compact disc (CD) entertainment systems, coupled with
`satellite, cable television and telephone services, have pro-
`vided opportunities for viewers to store, retrieve and selec-
`tively display a variety of television and audio-visual or
`interactive programming on home entertainment systems.
`Over the past decade, improvements in personal computing
`systems have provided a variety of powerful miniaturized
`personal computers which permit the storage of data and
`control of home appliances, such as entertainment systems,
`through the use of microprocessors. Additionally, a variety
`of graphic user interfaces have been developed to ease
`human interaction with these new personal computer sys-
`tems.
`
`The combination of computer technology with television
`(TV) and audio-visual
`(A/V) systems, has fostered the
`development of multi-media interactive entertainment sys-
`tems. The combination of computer hardware and software
`with audio-visual systems has brought new forms of art and
`entertainment into being. CD read only memory (ROM)
`systems coupled to personal computers permit interactive
`video computer simulations, unique self-paced learning
`environments and interactive “movies”, not possible in
`earlier television systems (See,
`"The Revolution Stans
`Here”, Newsweek, page 42 (Ian. 18, 1993)). It is known that
`consumers desire interactive television and multi-media
`
`applications for home use. One of the biggest challenges for
`an interactive television service is the design of an interface
`that is easy and fun to use by average consumers.
`In the computer industry, a variety of graphic user inter-
`faces have been developed to facilitate human interaction
`with computer systems. Many display systems utilize meta-
`phors in the design of the interface as a way of maximizing
`human familiarity, and conveying information between the
`user and the computer. It is well known that designing
`around a familiar metaphor helps reduce human learning
`time (See for example, Patrick Chan, “Learning Consider-
`ations In User Interface Design: The Room Model”, Report
`CS-84-16, University of Waterloo Computer Science
`Department, Ontario, Canada, July, 1984, and the references
`cited therein). Research in interface design using metaphors
`in man-machine interfaces may be applied to multi-media
`systems, and in particular, to interactive television systems.
`The marriage of video and television technology with
`computer interface technology provides consumers with
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`maximum flexibility in storing, retrieving and viewing tele-
`vision and other audio-visual prograrnnring. As will be‘
`described, the present invention provides methods and appa-
`ratus for presenting an improved audio-visual user interface,
`which includes various user-selectable features for viewing
`and controlling a television, video tape recorder (VCR) and
`other audio-visual devices. As will be disclosed, the present
`invention’s user interface provides a user-friendly mecha-
`nism for consumers to view, record, and play back TV and
`A/V programs, as well as control otherA/V home entertain-
`ment devices using a remote control device. Furthermore,
`information such as TV program listings and additional
`information related to programs as well as selecting and
`controlling categories of interactive programs and services
`may be provided through the user interface of the present
`invention.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides methods and apparatus for
`presenting an improved audio-visual user
`interface for
`selecting and displaying cable television or other audio-
`visual programs, as well as controlling various audio-visual
`devices and interactive services. The present
`invention
`assumes a service provider provides cable television and/or
`telephone (T/T) service to users via a T/I‘ cable, including a
`digital channel of prograrn/service listings, at
`least one
`digital back channel (from the user’s home to a central file
`server), a number of analog TV channels, a number of digital
`pay-per-view channels, and other interactive services trans-
`mitted from remote storage devices such as digital
`file
`servers. Under the present invention an interactive audio-
`visual (AN) transceiver is advantageously coupled to a
`user’s television, video tape recorder, and the T/T cable. In
`one embodiment,
`the transceiver includes three primary
`modules. A main module includes a central processing unit
`(CPU) coupled over a digital system bus to system memory
`and, preferably, an infra-red (IR) or similar wireless control
`unit. The main module is further coupled over an audio-
`visual bus to an A/V decoder, an A/V processor, an AN
`encoder, a TV, an AN connect module connected to the T/T
`cable, and to one or more VCRs and other AN devices. The
`main module may also be coupled to an optional CD ROM
`module, to permit the playback of multi-media CD ROM
`titles, audio CDs, and the like. Other optional modules or
`external devices (such as additional VCRs, CDs, hard disks,
`telephone, fax and answering machines) may be connected
`to this transceiver and controlled through the use of the
`audio-visual user interface of the present invention. The A/V
`transceiver hardware is complemented with an operating
`system software program which supports the functions pro-
`vided in the present invention’s audio-visual user interface.
`In operation,
`the cable or telephone service provider
`transmits an interleaved data stream preferably including a
`television program listing,
`together with program titles,
`program times, categories, channel numbers and the like,
`from a central file server on the digital channel of program
`listings to the transceiver. The data is interleaved such that
`the current day’s data is followed by other weekly data.
`A remote control device is preferably provided for com-
`municating with the transceiver, and includes a number of
`user selectable interactive functions such as: an info button,
`a list button, a categories button, a pix button, a mark button,
`a jump button, a select button, and a pointing device
`consisting of a set of cross-hair arrow buttons (up, down,
`left, right) or other pointing devices capable of providing the
`same functions (such as a mouse, thumb stick or touch pad).
`
`53
`
`53
`
`

`
`5,583,560
`
`3
`While viewing the TV, a user may obtain additional infor-
`mation on a current program by depressing the info button,
`and obtaining more detailed information using the pointing
`device. By depressing the list button on the remote control
`device,
`the transceiver displays a program listing of the
`current programs available for viewing. Through the use of
`the pointing device, viewers can scroll up and down the
`program listing or view a highlighted program in full screen
`by pressing the select button. By pushing the right or left
`arrow buttons on the pointing device, program listings may
`also be viewed for difierent hours, days and up to several
`weeks in advance through the present invention‘ s use of data
`interleaving. The depression of the categories button on the
`remote control device results in the display of a categories
`menu bar on the TV screen, which includes categories such
`as “all”, “sports”, “news" and “favorites”. The selection of
`one of these categories results in the display of only pro-
`grams within that category. Moreover, through the depres-
`sions of both the list button and the categories button, a
`program listing is displayed of the currently available pro-
`grams within the selected category. Through the use of the
`arrow buttons on the pointing device, category listings for
`difierent times and dates up to several weeks in advance may
`also be viewed. The selection of the favorites category
`results in the display of currently available favorite pro-
`grams, including programs that are frequently watched by a
`particular TV household, marked programs, or programs
`that match preselected user preferences.
`The, depression of the pix button results in the display of
`a “visual menu” of multiple picture-in-picture (PIP) win-
`dows along the perimeter of the TV screen. Each of the PIP
`windows displays one of the programs currently being
`received by the A/V transceiver in the last category selected
`by the viewer. The PIP windows are displayed at a variable
`frame rate (N) (depending on the number of windows), and
`are captured by one or more tuners in the transceiver. A
`center PIP window displays a larger video image (along with
`audio) of the currently highlighted PIP window. The other
`PIP windows can be similarly viewed by using the pointing
`device on the remote control. The depression of the mark
`button allows users to “bookmark” a particular program for
`later viewing. The depression of the jump button allows a
`user to jump between previously marked programs.
`Additional features, functions, and interface screens are
`available to the viewer using the teachings of the present
`invention,
`including a menu button for controlling other
`devices connected to the A/V transceiver (such as CD, VCR,
`etc.), a record button for making copies of programs onto
`connected AN recording devices (such as VCRs, hard disks,
`etc.), a plurality of control buttons for playing back, stop-
`ping, rewinding, or fast-forwarding audio-visual programs
`on connected AN playback devices, a numeric keypad for
`dialing charmel numbers, confirming financial transactions
`with personal
`identification numbers, an optional micro-
`phone with talk button for supplementing the present user
`interface with direct voice commands through the use of
`voice recognition technology, a pay-per view interface, and
`a home shopping interface, as well as other
`features
`described herein.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a functional block diagram of the audio-visual
`(A/V) system of the present invention.
`FIG. 2 is a functional block diagram illustrating the major
`components of the audio-visual transceiver of FIG. 1 in
`further detail.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`FIG. 3a diagrarnatically illustrates the electronic spectrum
`utilized by the present invention.
`FIG. 3b conceptually illustrates the present invention’s
`use of an interleaving data stream to download prograrnl
`service listings, and other information, to the present inven-
`tion’s AN transceiver.
`FIG. 4a illustrates one embodiment of the remote control
`device of the present invention with the sliding panel in a
`closed position.
`FIG. 4}; illustrates the first embodiment of the remote
`control device of the present invention shown in FIG. 4a
`with the sliding panel in an open position.
`FIGS. 5a and 5b illustrate a second embodiment of the
`remote control device of the present
`invention with no
`sliding panel and additional features.
`FIG. 6 illustrates the present invention’s full screen view-
`ing function as viewed by a user with a graphic overlay
`panel which includes the current channel number as well as
`the channel logo and identifier.
`FIG. 7 illustrates the present invention’s info function
`showing the first page of the basic information track (“info-
`track").
`FIG. 8 illustrates the present invention’s info function
`with a second page of basic infonnation, obtained by using
`the right arrow button on the remote control device.
`FIG. 9 is an additional illustration of the info function of
`
`the present invention in which the user has requested a third
`page of information using the remote control device.
`FIG. 10 illustrates the present invention’s use of the info
`function providing the user with additional tracks of infor-
`mation (such as the “Tid Bits” information track), by using
`the down arrow button of the remote control device.
`FIG. 11 is a further illustration of additional information
`tracks (“info-tracks"), such as this “Info Mart”, accessible to
`the user of the present invention using the down arrow
`button of the remote control device.
`
`FIG. 12 illustrates the present invention’s use of the list
`function to display current prograrn/service listings and to
`highlight a particular program. Viewers may scroll up or
`down that list by using the up or down arrow buttons on the
`remote control device.
`
`FIG. 13 illustrates the list function of the present inven-
`tion in which the NTV programming during the hour of 8:00
`to 9:00 p.m. includes two programs, one starting at 8:00
`p.m., the other starting after the previous program has ended.
`FIG. 14 illustrates the list function of the present inven-
`tion which allows the user to highlight the second program,
`which starts at 8:10 p.m., using the right arrow button of the
`remote control device.
`FIG. 15 is an additional feature of the list function of the
`present invention where the user marks the highlighted
`program with the mark button of the remote control device.
`FIG. 16 illustrates the selection of an altemate hourly time
`slot beginning at 9 p.m. using the right arrow button of the
`remote control device.
`
`FIG. 17 illustrates the selection of an alternate day for the
`prograrn/service listing by depressing the right arrow button
`of the remote control device.
`
`FIG. 18 illustrates displaying of a record panel confirming
`the title and length of the program to be recorded, along with
`a confirmation of which VCR and tape to record on, by
`depressing a select button in the remote control device.
`FIG. 19 illustrates the present invention’s use of the select
`button on the remote control device while the prograrnl
`
`54
`
`54
`
`

`
`5,583,560
`
`_
`
`5
`
`service listing is displayed to instruct the selected VCR or
`AN recording device to make a copy of the highlighted
`program.
`FIG. 20 illustrates the present invention’s “reminder”
`feature, which automatically displays a live picture-in-pic-
`ture (PIP) window of a previously marked program when the
`program is received by the A/V transceiver.
`FIG. 21 illustrates the present invention’s jump function,
`wherein pressing a jump button one the remote control
`device permits a user to jump directly to a full screen view
`of a previously mar