(12) United States Patent
`Iggulden
`
`(10) Patent N0.:
`(45) Date 0f Patent:
`
`US 6,597,405 B1
`Jul. 22, 2003
`
`US006597405B1
`
`(54) METHOD AND APPARATUS FOR
`AUTOMATICALLY IDENTIFYING AND
`SELECTIVELY ALTERING SEGMENTS OF A
`TELEVISION BROADCAST SIGNAL IN
`REAL_TIME
`
`(76) Inventor: Jerry Iggulden, 21600 Cleardale St.,
`Santa Clarita, CA (US) 91321
`
`EP
`EP
`EP
`EP
`JP
`W0
`W0
`
`2/1987
`0 210 609
`7/1989
`0161512
`3/1993
`0 618 726 A1
`0 574 698 A1 * 12/1993
`402211793 A
`8/1990
`WO 94/16442
`7/1994
`W0 96 08921
`3 1996
`/
`/
`Primary Examiner—John Miller
`
`_
`
`( 4 ) NotIce.
`
`_
`
`_
`
`_
`
`_
`
`SubJect to any d1scla1mer,the term of thIs
`patent is extended or adJusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/452,040
`_
`_
`(22) Flled'
`
`NOV‘ 30’ 1999
`Related US Application Data
`
`(63) Continuation of application NO_ 08/742,983’ ?led on NOV_ 1’
`1996, now Pat. No. 6,002,443.
`(51) 1m. 61.7 ................................................ .. H04N 5/44
`(52) US. Cl. ..................... .. 348/553; 348/907; 348/460;
`
`725/22; 725/14; 725/19; 725/28
`_
`Fleld Of Search ............................... ..
`348/687’ 690’ 552’ 907’ 722’ 465’ 460’
`725/32’ 42’ 22’ 14’ 19’ 20’ 25’ 28’ 34
`References Cited
`
`(56)
`
`U'S' PATENT DOCUMENTS
`3,919,479 A 11/1975 Moon ....................... .. 704/237
`4,229,765 A 10/1980 Sanger ,,,, ,,
`348/234
`4,230,990 A 10/1980 Lert, Jr. .................... .. 348/734
`4,259,689 A
`3/1981 Bonner
`4,333,110 A * 6/1982 Faerber .................... .. 348/632
`4,390,904 A
`6/1983 Johnston et a1‘
`386/53
`4,420,769 A 12/1983 Novak ........ ..
`348/180
`
`. . . .. 386/52
`7/1986 Reese . . . . .
`4,602,297 A
`725/22
`4,677,466 A * 6/1987 Lert 61 a1. ..
`_ _ _ __ 725/19
`4697 209 A
`9/1987 Kiewit _ _ _ _ _ _
`_ _ _ __ 725/22
`4:739j398 A
`4/1988 Thomas _ _ _ _ _
`4,750,052 A
`6/1988 Poppy ,,,,,,,,,,,,,,,,,,,,,,, __ 386/63
`-
`-
`L t
`t
`d
`t
`.
`( 15 Con mue on neX page)
`FOREIGN PATENT DOCUMENTS
`
`Assistant Examiner—Linus M. Lo
`
`(74) Attorney) Agent) Or Firm_B1ake1y, sokoloff, Taylor &
`Zafman LLP
`’
`
`(57)
`
`ABSTRACT
`
`The method and apparatus identi?es selected broadcast
`segments, such as commercial advertisements, of a televi
`sion signal in real-time for the purpose of muting the video
`and audio portions of the television signal during each
`unwanted segment. A signature pattern associated With each
`segment of the television signal is detected and compared to
`3182;353:1115; (g?tjggse $523323; fjn $518381‘; zjggflfrtz
`g.
`'
`g
`pattern matches one of the stored signature patterns, the
`.
`.
`.
`.
`.
`.
`segment is thereby Immediately Identi?ed as bemg one of
`the Selected Segments and is processed in reaptime to mute
`the audio and video portions of the television signal during
`the segment. If the signature pattern of the segment does not
`match any of the stored signature patterns, the segment is
`analyzed to determine Whether the segment is nevertheless
`a selected segment and, if so, its signature pattern is stored
`along With the Stored Signature patterns- The analysis to
`determine Whether the segment is nevertheless a selected
`segment is performed by detecting the length of the segment
`upon its completion and then determining Whether the length
`of the Segment matches one of a predetermined Set of
`permissible selected segment lengths such as standard com
`-
`-
`mercIal advertisement segments of 15 seconds, 30 seconds,
`60
`d If
`th
`t .
`.d f? d
`b .
`or
`5660“ §'
`,5",
`e Segmen ,15 1 em 6 as ,emg a
`selected and Its signature pattern is stored along WIth the
`other stored signature patterns such that, the next time the
`same segment is encountered, its signature Will then match
`the stored si nature and therefore the se ment can be
`g
`g
`immediately identi?ed and muted in real-time.
`
`DE
`
`44 42 264
`
`5/1996
`
`8 Claims, 14 Drawing Sheets
`
`
`
`"iv/i1“ \
`
`
`
`‘°2\ TELEVISION RECEWER j
`
`EVENT Wm";
`
`$TORAGE um
`
`ADVERT/560N121?“
`
`BROADCAST l
`CONIHOWW
`1 AZEDGQTZEKTEJFEQEMHASCKTEO'E
`Km '2”
`858880 3)“
`MuiiéEflt‘ggoN
`SIGNAL
`
`150 l INFORMATION
`51135212153?
`“1
`
`l
`
`l
`
`:
`:
`i
`1
`l
`'
`|
`
`TELEVISION MONITOR
`
`we
`
`104
`
`_ _ _ _ ___l
`
`K COMMERCIAL
`
`ADVERTISEMENT
`DETECTING AND
`MUTING UNIT
`
`Google Ex. 1011
`
`

`
`US 6,597,405 B1
`Page 2
`
`US. PATENT DOCUMENTS
`*
`
`4,752,834 A
`6/1988 Koombes .................. .. 348/634
`135732;? 15/1323 Same‘
`570197899 A
`41991 B5‘; """" "
`571517788 A
`9/1992 Blum
`5’333’O91 A
`7/1994 I
`lden
`'
`7
`7
`ggu
`5,343,251 A
`8/1994 Nafeh ...................... ..
`5 436 653 A
`7/1995 Ellis
`’
`’
`5,450,134 A * 9/1995 Legate ..................... .. 348/467
`5,455,630 A 10/1995 McFarland
`348/476
`5,481,294 A * 1/1996 Thomas et a1.
`725/20
`5,481,296 A
`1/1996 Cragun et a1.
`.. 725/136
`5,488,425 A
`1/1996 Grimes ..................... .. 348/559
`
`4/1996 Ellis
`5,504,518 A
`6/1997 Rothmuller ................ .. 725/46
`5,635,989 A
`8/1997 Kennan ____ __
`__ 72564
`5,659,366 A
`9/1997 Lewine ...................... .. 386/52
`5,668,917 A
`5,692,093 A * 11/1997 Iggulden e181. ............ .. 386/46
`5,696,866 A * 12/1997 Iggulden e181. ..
`386/46
`5,708,477 A * 1/1998 Forbes et a1. ..
`348/552
`5748 263 A
`5/1998 Ball ............ ..
`.348/734
`*
`7
`7
`5,986,692 A 11/1999 Logan et a1. ..
`725/35
`6,400,408 B1 * 6/2002 Berger
`.348/465
`6,404,977 B1 * 6/2002 Iggulden ......... ..
`.. 386/46
`6 469 749 B1 * 100002 D. .t
`t
`1
`348/722
`7
`7
`“n1 “Na 6 a‘ """" "
`
`* cited by examiner
`
`Google Ex. 1011
`
`

`
`wow
`
`wow
`
`US 6,597,405 B1
`
`1 .... -
`
`vow
`
`F GE
`
`00?
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`P.»
`
`300GA
`
`GAtCChS
`
`US 6,597,405 B1
`
`mg:
`
`l.Ezmzommu.mkzmsamm
`
`
`
`
`mhzmzommJNOE»zmsm_m_Em_>o<»zm_>_mm_Em>o<»zm2mm_Em>o<
`
`
`
`
`
`._<_omm__2s_oo4<_omm__2_2oo._<_omm__2_2oo
`
`
`
`ompommos_<mwoEnoozmMbmznoEfimPntomtmo Fzmzomm4»zms_mm_Em>o<Emsmm_Em>o<1pzmsomm2<moomn__asommzzoo_._<_omms_s_oo_Emzwmm
`
`
`
`
`
`\N?HHmoz<z_s5._“Emzomm
`
`«up
`
`.m_2_:om0EO_n5<
`
`ms=._.<
`
`mmon.s__mmn_:m
`
`o_a:<
`
`Google Ex. 1011
`
`cm.$3I§_/§_1«SI1N3m:N:
`
`E.
`
`
`
`
`
`kzmsomm_2<moomn_zmzomms_<moomn_kzmzwmmszmoomm
`
`
`
`
`
`
`
`Google Ex. 1011
`
`
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 3 of 14
`
`US 6,597,405 B1
`
`m><m
`
`mm3<zo_m+>
`mm.mm—
`
`
`bmz
`
`
`
`+2m_>m»zm>m
`
`
`
`EvE<_2mmvE<2
`
`
`
`ZOFOm:m_QZOFOm._.mD
`
`
`
`HZ_On_.~Z_On_
`
`
`
`
`
`.vmV—¢omoommomm:4¢m_m:4¢.omoomm
`
`
`
`.pxmz.momm.mommmm:+<zma
`
`
`
`
`
`
`
`
`>gm<moa2m»mmxm<:mmxm<2>am<moa2m+
`
`»zm>m._zm>m
`
`mm:»<zw5
`
`
`.FX$Z.mowm.momm
`
`
`
`
`.mommmmurmomoommomm34mommagamrm:4¢mm3»<zm:m
`
`
`
`
`
`mm:»<zoa
`
`Fzm>m
`
`mmxm<2
`
`.mommoom34¢
`
`Fxmz
`
`
`
`»zm>m»zm>m
`
`zoxfimflmoom<o@o+200
`
`
`
`
`mm_>mmf>mm3»<zoa*omomoomm
`
`zocomemo
`
`
`
`
`
`
`
`>£I<EO&§mFmmxm<2omoomm¢mxm<2mmxm<2
`
`
`
`»2m>m»zm>m+zm>m>gm<moa2m»
`
`
`
`
`
`Google Ex. 1011
`
`Google Ex. 1011
`
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 4 of 14
`
`US 6,597,405 B1
`
`
`
`
`
`Hmmiomm:.Em_.mD._n_DEOOME
`
`
`
`
`
`om<om5mwxm<2¢wx¢<2>gm<moa2m»
`
`omoomm+>>g¢<moa2m»pzoaFzou
`
`
`
`mm:p<zoaaw?ow?
`
`Fxmz
`
`
`
`F2m>m»zm>m
`
`
`
`mmxm<2mmxm<2
`
`
`
`zo:om»mozo:ompmo
`
`
`
`kzm>mpzm>m
`
`
`
`
`
`
`
`>gm<mo¢2m»»zm>m.momm.mommmm:»<zoa
`
`omomoommmmxm<2
`
`
`
`»zm>m»zm>m
`
`
`
`mmxm<2mmxm<2
`
`F2m>m
`
`mmxm<2
`
`zofiomhmo
`
`+30;
`
`mm:»<zoa
`
`..
`
`3;
`
`
`
`hmglFzomFzou
`
`
`
`
`.m><mZO_._.OmE.mDZO_._.Om_.rmO
`
`>.__m<mon__2m»
`
`pzm>m>em<moa2m»
`
`.momm»zm>m
`
`momm
`
`
`0,mmw
`
`mmxm<2mmxm<2
`
`mm:+<zoa
`
`»zm>mNO_n_8m3n.mm3<zo_m
`
`
`
`mm_vE<_2omOOmEQmOowm
`
`
`Google Ex. 1011
`
`Google Ex. 1011
`
`
`
`
`

`
`U.S. Patent
`
`Jul. 22,2003
`
`Sheet 5 0f 14
`
`US 6,597,405 B1
`
`RECEIVE TELEVISION
`SIGNAL
`
`\\ 200
`
`SEARCH FOR EVENT MARKER
`WITHIN RECEIVED SIGNAL
`
`21°
`
`BROADCAST SEGMENT
`
`[_ _ __ __.
`
`E‘éAR'bLéAgS'gN
`
`f
`
`I
`'
`I
`READ SIGNATURE
`CORRESPONDING TO
`212
`I
`| & CURRENT SEGMENT FROM
`I
`\ BROADCAST SIGNAL (FIG-9)
`
`NO
`
`"_
`
`EVDEQTEEQESER
`'
`
`\
`
`204
`
`|
`I
`
`I
`I
`I
`
`BEGIN SEGMENT
`EVALUATION
`PROCESS
`
`I
`— — '
`
`20a
`
`7
`
`I
`f
`; OUTPUT TELEVISION
`SIGNAL
`
`‘
`?
`DETERMINE IF SEGMENT
`NEVERTHELESS
`IS A COMMERCIAL
`ADVERT'SEMENT (RG12)
`
`\ 206
`
`\
`
`NO
`
`'
`
`\
`218
`
`COMMERCIAL
`ADVERTI EMENT
`75
`'
`
`220
`
`' YES
`STORE SIGNATURE,
`LENGTH OF COMMERCIAL
`ADVERTISEMENT AND OTHER
`BROADCAST PARAMETERS SUCH
`DATE/TIME IN THE DATABASE (FIG.13)
`&
`224
`
`COMPARE SIGNATURE
`wITH PRE-STORED
`SIGNATURES FROM
`THE DATABASE
`
`I
`
`SIGNATURE
`MATCH?
`
`J
`
`NO
`
`21s
`
`226
`
`227
`
`228
`
`230
`
`J
`
`DETECT IF SEGMENT IS THE FIRST
`OF A GROUP
`I
`BEGIN MUTING OUTPUT
`SIGNAL
`'
`READ LENGTH OF
`COMMERCIAL ADVERTISEMENT
`CORRESPONDING To THE
`SIGNATURE FROM THE
`DATABASE
`I
`UPDATE SIGNATURE BROADCAS
`PARAMETERS IN DATABASE SUC
`AS LATEST
`DATE/TIME OF BROADCAST
`i
`
`232
`\
`TERMINATE MUTING AT END OF
`COMMERCIAL ADVERTISEMENT
`0R AT END OF GROUP OR IF
`USER OVER-RIDES MUTING
`
`IF SEGMENT IS THE FIRST OF A
`ROUP, PERFORM GROUP MUTING
`I
`END SEGMENT
`EVALUATION
`
`1 FIG. 8
`
`s
`22
`
`234
`
`\
`
`I
`
`END SEGMENT
`EVALUATION
`
`LO
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 6 0f 14
`
`US 6,597,405 B1
`
`READ SIGNATURE
`CORRESPONDING TO EVENT
`MARKER FROM TELEVISION
`SIGNAL
`
`I
`
`WAIT PREDETERMINED
`NUMBER OF FRAMES FOLLOWING
`EVENT MARKER
`
`212
`
`236
`
`I
`
`RECORD TEST
`FRAME
`
`p38
`
`I
`DETERMINE. FOR EACH OF A P42
`
`PRESELECTED SET OF LINES OF
`THE FRAME, THE AVERAGE
`LUMINANCE LEVEL OF THE LINE
`
`,
`I
`244
`( DETERMINE, FOR EACH LINE,
`WHETHER THE AVERAGE J
`LUMINANCE LEVEL
`OF THE LINE EXCEEDS
`A PREDETERMINED THRESHOLD
`k
`
`I
`
`f
`
`\
`
`ASSIGN A BIT VALUE OF
`246
`1 TO EACH LINE EXCEEDING
`THE THRESHOLD AND A BIT _/
`VALUE OF 0 OTHERWISE
`
`I
`
`248
`CONCATENATE THE BIT VALUES
`TOGETHER YIELDING A SINGLE _/
`BIT STRING WITH ONE BIT FOR
`EACH OF THE SELECTED LINES
`OF THE FRAME
`
`I
`
`[ RETURN TO FIG. 8, STEP 212
`
`249
`
`FIG. 9
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 7 0f 14
`
`US 6,597,405 B1
`
`SIGNATURE
`TEST LINES
`
`SIGNATURE
`TEST FRAME
`
`240
`
`FIG. 10
`
`FIG. 11
`
`BINARY BIT
`STRING
`
`K SIGNATURE
`
`\ 250
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22,2003
`
`Sheet 8 0f 14
`
`US 6,597,405 B1
`
`DETERMINE IF CURRENT
`SEGMENT NEVERTHELESS
`CORRESPONDS TO
`COMMERCIAL ADVERTISEMENT
`
`N .2 0)
`
`II
`READ IN STANDARD
`COMMERCIAL
`ADVERTISEMENT LENGTHS
`
`WAIT PREDETERMINED TIME
`PERIODS BASED UPON
`STANDARD COMMERCIAL
`LENGTHS
`
`II
`DETERMINE IF ANOTHER
`EVENT MARKER OCCURS AT
`THE END OF ANY OF THE
`PREDETERMINED TIME
`PERIODS
`
`254
`
`256
`
`IDENTIFY SEGMENT
`BEGINNING AT FIRST
`EVENT MARKER
`AS NOT BEING A
`COMMERCIAL
`ADVERTISEMENT
`
`J
`
`266
`
`ANOTHER
`EVENT
`MARKER '.7
`
`(
`
`260
`
`II /
`
`w
`
`IDENTIFY SEGMENT
`BEGINNING AT FIRST
`EVENT MARKER AND
`ENDING AT SECOND
`AS A COMMERCIAL
`ADVERTISEMENT
`
`I
`,
`CALCULATE LENGTH OF
`COMMERCIAL ADVERTISEMENT
`
`262
`
`264
`
`)
`
`‘_,[ RETURN TO FIG. 8 STEP 21%‘
`
`FIG. 12
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 9 0f 14
`
`US 6,597,405 B1
`
`STORE SIGNATURE,
`LENGTH OF COMMERCIAL
`ADVERTISEMENT AND
`DATE/TIME IN THE DATABASE /
`
`224
`
`268
`
`272
`
`NO
`
`I
`
`DATABASE
`FULL
`?
`
`SELECT A PREVIOUS SIGNATURE
`ENTRY TO OVERWRITE BASED
`UPON LATEST DATE/TIME OF
`BROADCAST, FREQUENCY OR
`NUMBER OF BROADCASTS, TIME
`OF DAY WEIGHTING FACTOR
`OR SIMPLE FIFO OPERATION
`
`I
`
`RECORD SIGNATURE, LENGTH OF
`COMMERCIAL ADVERTISEMENT
`AND DATE/TIME OF BROADCAST
`
`‘
`
`271
`
`(RETURN TO FIG. 8, STEP 224
`
`270
`
`FIG. 13
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 10 0f 14
`
`US 6,597,405 B1
`
`B" STH'NG
`SIGNATURE SIGNATURE
`
`FIRST BROADCAST
`DATE/TIME
`ENCOUNTERED
`
`LATEST BROADCAST
`ENxéAoTuNTlgIsEo
`
`NUMBER OF BROADCASTS
`ENCOUNTERED
`
`'N'USE
`wEIGI-ITING
`FACTOR
`
`SIGNATURE
`#2
`
`SIGNATURE
`N
`
`275
`
`\ SINGLE ENTRY
`SIGNATURE
`MEMORY UNIT
`
`FIG_ 14
`
`BROADCAST
`DATE/TIME
`
`SIGNATURE
`#11
`
`BIT STRING
`SIGNATURE
`SIGNATURE \, \
`#12
`L
`
`SIGNATURE
`MN
`
`\_
`
`MULTIPLE
`ENTRY
`SIGNATURE
`MEMORY UNIT
`
`FIG.16
`
`276
`__I/
`
`SIGNATURE
`#21
`./ SIGNATURE
`_____/ #22
`
`SIGNATURE
`J #QM
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22,2003
`
`Sheet 11 0f 14
`
`US 6,597,405 B1
`
`NEWEST
`STORED
`SIGNATURE
`
`K‘
`
`BIT STRING
`
`SIGNATURE \
`‘
`
`SIGNATURE
`
`/ FIFO
`
`OLDEST
`STORED
`SIGNATURE
`
`'
`
`274
`
`FIG. 15
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 12 0f 14
`
`US 6,597,405 B1
`
`POWEFI UP SYSTEM WITH
`AUDIONIDEO DISABLED
`
`300
`
`304
`
`WAIT FoR usER
`TO TURN TELEVISION
`MONITOR ON
`
`W
`._‘____E§
`
`I
`
`DATABASE
`SETUP
`COMPLETE
`7
`
`I
`
`306
`
`/
`
`ENABLE VIDEO/AUDIO
`
`YES
`
`I
`
`[308
`
`USER HAS
`TURNED TV
`MONITOR
`ON?
`
`324
`No I
`
`302
`
`322
`
`RECEIVE BROADCAST
`SIGNAL
`
`RECEIVE BROADCAST
`SIGNAL
`
`f 310
`
`I
`
`326
`
`SEARCH FOR EVENT MARKER
`WITHIN RECEIVED SIGNAL
`
`SEARCH FOR EVENT MARKER
`WITHIN RECEIVED SIGNAL
`
`312
`
`_ EVENT MARKER
`DETECTED?
`
`NO
`
`EVENT MARKER
`DETECTED?
`
`BEGIN SEGMENT
`EVALUATION
`
`BEGIN SEGMENT
`EVALUATION
`
`K- 316
`
`328
`
`330
`
`320
`/
`
`DISABLE VIDEO/AUDIO
`
`\
`
`NO
`
`OUTPUT TELEVISION
`SIGNAL
`
`318
`
`USER
`TURNS TV
`MONITOR
`OFF?
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 13 0f 14
`
`US 6,597,405 B1
`
`406
`
`408T:
`EL 410
`
`A
`
`F
`
`{I c
`I’
`
`402
`
`\ 404
`
`400
`K
`
`SELECTIVE
`MUTING UNIT
`
`FIG. 18
`
`Google Ex. 1011
`
`

`
`U.S. Patent
`
`Jul. 22, 2003
`
`Sheet 14 0f 14
`
`US 6,597,405 B1
`
`com
`
`/ 8m
`
`8/
`
`mom
`
`2. GE
`
`Google Ex. 1011
`
`

`
`US 6,597,405 B1
`
`1
`METHOD AND APPARATUS FOR
`AUTOMATICALLY IDENTIFYING AND
`SELECTIVELY ALTERING SEGMENTS OF A
`TELEVISION BROADCAST SIGNAL IN
`REAL-TIME
`
`This application is a continuation of application Ser. No.
`08/742,983 ?led Nov. 1, 1996, now US. Pat. No. 6,002,443.
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The invention relates to television signal processing sys
`tems and in particular to television signal processing sys
`tems for detecting selected segments of a television signal
`such as commercial advertisement segments.
`2. Description of Related Art
`It has long been a desire of many television vieWers to
`have a system Which automatically detects selected seg
`ments of a television signal such as commercial advertise
`ments or other unWanted broadcast segments in real-time
`during a television broadcast for the purpose of, for
`example, muting the audio portion of the television broad
`cast during the unWanted broadcast material. Although a
`Wide variety of techniques have been developed for detect
`ing selected segments of television signals, heretofore there
`have been no suf?ciently effective and economical systems
`that automatically operate in real-time for muting or similar
`purposes.
`Examples of broadcast segment detection systems include
`systems for detecting commercial advertisements during the
`playback of a television program on a video cassette
`recorder (VCR) for the purpose of controlling the VCR to
`skip or scan over the commercial advertisements during
`playback such that a vieWer may Watch the recorded tele
`vision program Without signi?cant commercial advertise
`ment interruptions. One particularly effective system is
`described in US. Pat. No. 5,333,091 entitled “Method and
`Apparatus for Controlling a Videotape Player to Automati
`cally Scan Past Recorded Commercial Messages” and sold
`under the trademark “Commercial Advance.” With the sys
`tem of US. Pat. No. 5,333,091, an automatic editing device
`is coupled betWeen a conventional VCR and a conventional
`television set. As a broadcast program is recorded on a
`videotape by the VCR, the editing device detects and records
`the timing and duration of selected events, such as black
`frames combined With loW audio, that separate segments of
`the broadcast and creates a time-based map of the selected
`events. After the recording is completed, the editing device
`analyZes the time-based map and determines therefrom
`Which segments are commercial advertisement segments
`and Which are program segments. The editing device then
`records control signals on the videotape prior to and fol
`loWing the commercial advertisement segments. Upon
`playback, the control signals control the VCR to fast-scan
`over the commercial advertisement segments so that the
`recorded broadcast can be vieWed substantially Without
`interruption. Related techniques are described in US. Pat.
`No. 5,455,630 and in PCT application PCT/US94/00223
`published as WO 94/16442.
`Although the systems of US. Pat. Nos. 5,333,091 and
`5,455,630 and PCT application PCT/US94/00223 are very
`effective in skipping over commercial advertisements and
`other unWanted broadcast material during the playback of a
`recorded broadcast program on a videotape, the systems do
`not operate in real-time during a received broadcast.
`Other examples of broadcast segment detection systems
`include systems for detecting commercial advertisements
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`2
`during the recording of a television program on a VCR for
`the purposes of controlling the VCR to skip the commercial
`advertisements during recording such that a vieWer may
`later Watch the recorded television program Without any
`commercial advertisement interruption Whatsoever. An
`example is a system described in US. Pat. No. 4,602,297
`entitled “System for Editing Commercial Messages from
`Recorded Television Braodcasts” Which stores and delays
`the output of a received broadcast signal to a VCR until it
`can determine Whether the delayed portion of the broadcast
`signal includes commercial advertisements. To this end, the
`system detects commercial advertisements or groups of
`commercial advertisement Within the delayed portion by
`detecting amplitude drops in the broadcast signal and deter
`mining Whether time intervals betWeen the amplitude drops
`correspond to normal durations of commercial advertise
`ments or groups of commercial advertisements. The system
`then outputs only those portions of the delayed signal to the
`VCR that do not correspond to commercial advertisements
`of groups of commercial advertisements. Hence, the broad
`cast signal is recorded Without commercial advertisements
`for future playback.
`Other examples of systems for detecting commercial
`advertisements during the recording of a television program
`are described in US. Pat. No. 4,750,052 entitled “Apparatus
`and Method for Automatically Deleting Selected Program
`Intervals from Recorded Television Broadcasts” and US.
`Pat. No. 4,782,401 entitled “Editing Method and Apparatus
`for Commercials During Video Recording”, both of Which
`also operate to detect commercial advertisements based
`upon intervals occurring betWeen amplitude drops or other
`fads in a received broadcast signal. Rather than storing and
`deleting the received signals prior to output to a VCR,
`hoWever, the systems of US. Pat. Nos. 4,750,052 and
`4,782,401 record the signals on a video tape using a VCR
`and control the VCR to reWind at the end of each commer
`cial advertisement or group of advertisements such that
`further program signals are recorded over the commercial
`advertisements.
`None of the aforementioned systems for detecting broad
`cast segments during the recording of a television program
`operate in real-time for the purpose of, for example, muting
`the audio during commercial advertisements.
`Still other examples of broadcast segment detection sys
`tems include systems for detecting commercial advertise
`ments in broadcast television signals for the purposes of
`verifying that selected commercial advertisements are being
`broadcast on the proper days and at the proper times.
`Examples include US. Pat. No. 5,504,518 entitled “Method
`and System for Recognition of Broadcast Segments”
`Wherein stored digitiZed segments of broadcast signals are
`processed by an expert system to determine Whether any
`particular segment corresponds to a selected segment of
`interest, such as a selected commercial advertisement. Seg
`ments that cannot be identi?ed by the expert system, such as
`neW or otherWise unknoWn segments, are routed to a human
`operator for verifying that the segment represents a com
`mercial advertisement rather than a portion of a television
`program. Selected segment information such as the date and
`time of broadcast of the segments of interest are recorded
`and tabulated in a report for use by, for example, broadcast
`advertisers for verifying that the selected segments Were
`broadcast on the proper dates and at the proper times. As part
`of its analysis of the segments, the expert system compares
`broadcast signatures With a database of signatures. Avariety
`of techniques are described for de?ning and detecting sig
`natures. The expert system also considers various cues
`
`Google Ex. 1011
`
`

`
`US 6,597,405 B1
`
`3
`indicating the lengths and relative groupings of segments
`and the intervals therebetWeen. No real-time analysis
`appears to be performed by the expert system or by the
`human operator.
`Another system for commercial advertisement broadcast
`veri?cation is described in Japanese Laid-Open Patent
`Application No. 1-174192, of Sony Corporation, entitled
`“Television Signal Processing Unit” Wherein an entire days
`Worth of broadcast programs are recorded on videotape. The
`recorded broadcasts are analyZed to identify commercial
`advertisement segments by intervals therebetWeen as
`detected by luminance signal considerations. The commer
`cial advertisements, and only the commercial
`advertisements, are then output and recorded on a second
`videotape. The second videotape thereby contains a days
`Worth of commercial advertisements in extracted form that
`may be vieWed by a human operator for broadcast veri?ca
`tion purposes. Again, no real-time detection of commercial
`advertisements is performed.
`Examples of systems for detecting broadcast segments,
`such as commercial advertisement segments, Which have
`some real-time detection capability include US. Pat. Nos.
`4,420,769 entitled “Device for the Automatic Editing, at the
`Receiver, of UnWanted Program Material from Broadcast
`Electrical Signals” and 4,750,213 entitled “Method and
`System for Editing UnWanted Program Material from
`Broadcast Signals”. US. Pat. No. 4,420,769 describes a
`system Wherein a human operator identi?es program seg
`ments to be eliminated from a broadcast signal causing a
`digital code Word or signature for the segment to be stored
`in memory. Subsequently, if the same segment is
`encountered, the system automatically compares digitiZed
`portions of the received signal With the stored signatures to
`recogniZe the segment and then to blank out or otherWise
`eliminate the segment in real-time. One signi?cant problem
`With this system is that a human operator must ?rst detect
`program segments to be eliminated. Also, a signi?cant
`amount of data processing is required to establish and
`compare the signatures in a reliable manner. US. Pat. No.
`4,750,213, by the same inventor, sets forth improvements
`intended, in part, to reduce the processing time required. The
`system of US. Pat. No. 4,750,213 also includes a delay or
`other storage device for alloWing a received signal to be
`delayed until a determination can be made by the human
`operator as to Whether a particular segment of the received
`signal should be eliminated. If so, the received, delayed
`signal is output With the selected blanked out or otherWise
`eliminated. Although US. Pat. No. 4,750,213 may represent
`an improvement over US. Pat. No. 4,420,769, it shares the
`disadvantage that a human operator must detect program
`segments to be eliminated.
`Another example of a system Which has some real-time
`detection capability is US. Pat. No. 5,151,788 entitled
`“Method and Apparatus for Identifying and Eliminating
`Speci?c Material from Video Signals” Wherein commercial
`advertisement broadcast segments are distinguished from
`program segments by the relative level of “activity” Within
`the beginning of each segment. If the activity level is high,
`the segment is presumed to be a commercial advertisement.
`The system detects the amount of activity Within the ?rst feW
`seconds of a segment folloWing a blank frame and operates
`to mute or otherWise eliminate the remaining portions of the
`segment if the detected activity level is representative of a
`commercial advertisement. The system may control a VCR
`to stop recording for the remainder of the segment. The
`patent describes that the optimum time period during Which
`the activity level should be detected is 8 seconds. One
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`55
`
`65
`
`4
`disadvantage of the system is that the detection of commer
`cial advertisements based upon an activity level is likely to
`be unreliable for many broadcasts and, even When the
`system does correctly detect a commercial advertisement,
`that detection does not occur until 8 seconds into the
`commercial advertisement.
`Other references that relate to systems for detecting
`selected segments of a broadcast signal such as commercial
`advertisement segments of a television signal are described
`in the folloWing references: US. Pat. No. 3,919,479 entitled
`“Broadcast Signal Identi?cation System”; 4,229,765 entitled
`“Remote Audio and Brightness Control System for a Tele
`vision”; 4,230,990 entitled “Broadcast Program Identi?ca
`tion Method and System”; 4,259,689 entitled “Television
`Advertising Editing System”; 4,333,110 entitled “Television
`Editing System”; 4,390,904 entitled “Automatic Circuit and
`Method for Editing Commercial Messages from Television
`Signals”; 4,667,466 entitled “Broadcast Program Identi?ca
`tion Method and Apparatus”; 4,697,209 entitled “Methods
`and Apparatus for Automatically Identifying Programs
`VieWed or Recorded”; 4,739,398 entitled “Method, Appa
`ratus and System for RecogniZing Broadcast Segments”;
`4,752,834 entitled “Reciprocating Recording Method and
`Apparatus for Controlling a Video Recorder so as to Edit
`Commercial Messages from a Recorded Television Signal”;
`5,019,899 entitled “Electronic Data Encoding and Recog
`nition System” and European Patent Application Publication
`No. 0161512 entitled “Program Identi?cation System”.
`Hence various techniques have been developed for detect
`ing selected broadcast segments such as commercial adver
`tisements Within a television signal. None of the techniques
`described above, hoWever, are effective for reliably and
`automatically detecting broadcast segments in real-time
`Without the need of a human operator. According, it Would
`be desirable to provide a simple, inexpensive system Which
`reliably detects selected segments of a broadcast signal
`Without requiring a human operator and Wherein, either
`immediately or after some period of time, all or most
`selected segments can be detected in real-time.
`
`SUMMARY OF THE INVENTION
`In accordance With the invention, a method and apparatus,
`intended to be marketed under the mark Sound!OffTM, is
`provided for identifying selected segments of a broadcast
`signal in real-time for the purpose of, for example, muting
`the video and audio portions of the signal during the selected
`segment. Hereinafter, “selected segment” refers to any
`selected portion of a broadcast signal including, for
`example, unWanted broadcast segments such a commercial
`advertisement segments. In one embodiment Wherein the
`broadcast signal is a television signal, the method includes
`the steps of detecting a signature pattern associated With the
`beginning of a received segment of the television signal and
`comparing the signature pattern to stored signature patterns
`representative of selected segments. If the signature pattern
`matches one of the stored signature patterns, the received
`segment is thereby immediately identi?ed as being one of
`the selected segments and can be processed accordingly in
`real-time to, for example, mute the audio and video portions
`of the television signal during the received segment as the
`television signal is output to, for example, a television
`monitor. If the signature pattern of the received segment
`does not match any of the stored signature patterns, the
`received segment is analyZed to determine Whether it is
`nevertheless one of the selected segments and, if so, its
`signature pattern is stored along With the stored signature
`patterns. The analysis to determine Whether the received
`
`Google Ex. 1011
`
`

`
`US 6,597,405 B1
`
`5
`segment is one of the selected segments even though its
`signature does not match any of the stored signatures may be
`performed by, for example, detecting the length of the
`received segment upon its completion and then determining
`Whether the length of the received segment matches any of
`a pre-determined set of permissible lengths for selected
`segments. If so, the segment is identi?ed as being a selected
`segment and its signature pattern is stored along With the
`other stored signature patterns such that, the next time the
`same segment is received, its signature Will then match one
`of the stored signatures and therefore the received segment
`Will be identi?ed promptly at a selected segment and muted
`or otherWise altered in real-time. For an embodiment
`Wherein the selected segments are commercial advertise
`ment segments, the permissible lengths are, for example, 15
`seconds, 30 seconds, and 60 seconds. The method may be
`performed by a television set or other device receiving a
`television signal such as a personal computer television
`(PCTV).
`In one implementation, no stored signatures are initially
`provided. Hence, the ?rst time any particular selected seg
`ment is encountered, it is not immediately identi?ed as being
`a selected segment and therefore is not muted in real-time.
`HoWever, the next time the same segment is encountered,
`the segment Will be subject to immediate real-time muting
`because its signature Will have been stored the ?rst time it
`Was received. Thereafter, as long as its signature remains
`stored, the segment Will be subject to real-time muting
`Whenever it is received. As more and more selected seg
`ments are encountered and their respective signature pat
`terns stored, the more and more likely it becomes that any
`particular selected segment encountered While Watching a
`television program Will be promptly detected based upon its
`signature and muted in real-time. Hence, even though no
`stored signatures are initially provided, a database of signa
`tures is soon developed. Also, in the exemplary
`implementation, the selected segment detection method may
`be performed by a television set even While the television set
`is otherWise not in use to thereby detect and record segment
`signatures in a background mode to quickly develop a
`database of signatures. Alternatively, a database of the
`signatures of common segments, such as common commer
`cial advertisement segments, may be initially provided at the
`time of manufacture such that the common segments are
`thereby subject to immediate real-time muting even the ?rst
`time received.
`For many television broadcast systems, commercial
`advertisements appear in groups of tWo or more. When used
`in connection With such systems, the method of the inven
`tion preferably operates to not only mute or otherWise alter
`any detected commercial advertisements of a commercial
`advertisement group but also to mute or otherWise alter other
`unWanted broadcast segments appearing Within the group,
`such as station breaks, neWs bulletins, public service
`announcements, etc. To this end, the method operates to
`identify the ?rst of a group of commercial advertisements,
`then mutes or otherWise alters all segments during a prede
`termined minimum commercial advertisement group time
`period. The minimum group time period may be, for
`example, tWo minutes and tWenty seconds. Hence, all seg
`ments appearing during the period are muted in real-time,
`including commercial advertisements for Which the corre
`sponding signature has not yet been stored (and Which
`therefore Would not otherWise be subject to immediate
`real-time muting) and segments that do not match the
`predetermined permissible lengths (and Which therefore
`Would not otherWise be subject to any muting Whatsoever),
`
`5
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`such as the aforementioned station breaks, neWs bulletins,
`and public service announcements. Thus the method is not
`limited to muting only those segments that have matching
`stored signatures. Rather, the method operates to mute or
`otherWise alter all segments appearing in a detected com
`mercial advertisement group. HoWever, only the signatures
`of segments having lengths matching the permissible
`selected segment lengths are stored. The signatures of a
`non-commercial advertisement segment, such as a station
`break, neWs bulletin, or public service announcement, are
`stored only if the length of the segment happens to match
`one of the predetermined permissible segment lengths.
`Depending upon the implementation, the method identi
`?es the beginning of a received segment by detecting
`pre-determined event markers Which separate segments.
`When implemented for use in North America, the method
`may