IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`Lieberman et al.
`In re Patent of:
`Attorney Docket No.: 50048-0001IP2
`9,691,429
`U.S. Patent No.:
`
`Jun. 27, 2017
`Issue Date:
`
`Appl. Serial No.: 14/708,805
`
`Filing Date:
`May 11, 2015
`Title:
`SYSTEMS AND METHODS FOR CREATING MUSIC
`VIDEOS SYNCHRONIZED WITH AN AUDIO TRACK
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 9,691,429 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`

`

`
`
`
`
`
`I. 
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`TABLE OF CONTENTS
`
`
`REQUIREMENTS FOR IPR .......................................................................... 1 
`A.  Grounds for Standing ................................................................................. 1 
`B.  Challenge and Relief Requested ................................................................. 1 
`THE ’429 PATENT ......................................................................................... 2 
`A.  Brief Description ........................................................................................ 2 
`B.  Summary of the Prosecution History ......................................................... 4 
`C.  Level of Ordinary Skill in the Art .............................................................. 6 
`D.  Claim Construction ..................................................................................... 6 
`III.  THE CHALLENGED CLAIMS ARE UNPATENTABLE ............................ 8 
`A.  GROUND 1: Claims 11-16 are obvious based on Hozumi and Herberger
` .................................................................................................................... 8 
`1.  Overview of Hozumi ............................................................................. 8 
`2.  Overview of Herberger .......................................................................... 9 
`3.  The combination of Hozumi and Herberger ....................................... 10 
`4.  Reasons to combine Hozumi and Herberger ....................................... 13 
`5.  Analysis ............................................................................................... 16 
`B.  GROUND 2A: Claim 17 is obvious based on Hozumi in view of
`Yamamoto ................................................................................................ 42 
`1.  Overview of Yamamoto ...................................................................... 42 
`2.  The combination of Hozumi and Yamamoto ...................................... 46 
`3.  Reasons to combine Hozumi and Yamamoto ..................................... 47 
`4.  Analysis ............................................................................................... 50 
`C.  Ground 2B: Claims 18 and 19 are obvious based on Hozumi in view of
`Yamamoto and Herberger ........................................................................ 60 
`1.  The combination of Hozumi, Yamamoto, and Herberger .................. 60 
`2.  Reasons to combine Hozumi, Yamamoto, and Herberger .................. 61 
`3.  Analysis ............................................................................................... 61 
`INSTITUTION IS APPROPRIATE HERE .................................................. 66 
`A.  The Fintiv Factors favor institution—§314(a) ......................................... 66 
`
`II. 
`
`IV. 
`
`
`
`i
`
`

`

`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`1.  Factor 1: Either party may request stay ............................................... 67 
`2.  Factor 2: District Court schedule ........................................................ 67 
`3.  Factor 3: Petitioner’s investment in IPR outweighs forced investment
`in litigation to date ............................................................................... 68 
`4.  Factor 4: The Petition raises unique issues ......................................... 68 
`5.  Factor 5: Institution would provide the Board an opportunity to
`invalidate claims that could later be asserted against others ............... 69 
`6.  Factor 6: Other circumstances support institution .............................. 69 
`B.  The General Plastic Factors favor institution—§314(a) ......................... 70 
`1.  Factors 1 and 5: This petition is directed to a distinct set of claims ... 71 
`2.  Factors 2 and 4: Petitioner filed this Petition after searching revealed
`the applied art ...................................................................................... 73 
`3.  Factor 3: This Petition Does Not Implicate Road-Mapping Concerns
` ............................................................................................................. 73 
`4.  Factors 6 and 7: Institution would Efficiently Promote Patent Quality
` ............................................................................................................. 74 
`5.  Conclusion ........................................................................................... 74 
`PAYMENT OF FEES ................................................................................... 75 
`V. 
`VI.  CONCLUSION .............................................................................................. 75 
`VII.  MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 75 
`A.  Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) ............................... 75 
`B.  Related Matters Under 37 C.F.R. § 42.8(b)(2) ......................................... 75 
`C.  Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) .................... 76 
`D.  Service Information .................................................................................. 76 
`
`
`
`
`
`ii
`
`

`

`
`
`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`EXHIBITS
`
`U.S. Patent No. 9,691,429 to Leiberman et al. (the “’429
`Patent”)
`
`Excerpts from the Prosecution History of the ’429 Patent
`
`Reserved
`
`U.S. Patent Pub. No. 2013/0163963 to Crosland et al.
`
`U.S. Patent Pub. No. 2012/0148208 to Hozumi (“Hozumi”)
`
`U.S. Patent Pub. No. 2010/0183280 to Beauregard et al.
`
`BD-1001
`
`BD-1002
`
`BD-1003
`
`BD-1004
`
`BD-1005
`
`BD-1006
`
`BD-1007-1008
`
`Reserved
`
`BD-1009
`
`Sarah Perez, “Triller sues TikTok over Patent infringement”,
`Tech Crunch (July 30, 2020), available at:
`https://techcrunch.com/2020/07/30/triller-sues-tiktok-over-
`patent-infringement/
`
`BD-1010-1011
`
`Reserved
`
`BD-1012
`
`BD-1013
`
`BD-1014
`
`BD-1015
`
`Complaint from Triller, Inc. v. Bytedance Ltd. and TikTok
`Inc., Case No. 6:20-cv-00693 (W.D.Tex)
`
`U.S. Application Publication No. 2016/0134785 to Greene
`
`U.S. Patent No. 8,185,297 to MacLeod
`
`U.S. Application Publication No. 2008/0016114 to
`Beauregard et al.
`
`BD-1016
`
`U.S. Patent No. 8,621,355 to Eppolito et al.
`
`iii
`
`

`

`BD-1017
`
`BD-1018
`
`BD-1019
`
`BD-1020
`
`BD-1021
`
`BD-1022
`
`BD-1023
`
`BD-1024
`
`BD-1025
`
`BD-1026
`
`BD-1027
`
`
`
`
`
`
`
`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`U.S. Application Publication No. 2015/0220249 to Snibbe et
`al.
`
`U.S. Patent No. 9,972,357 to Merrill
`
`U.S. Patent No. 7,027,124 to Foote et al.
`
`U.S. Patent No. 6,086,380 to Chu et al.
`
`Declaration of Dr. Jeremy Cooperstock
`
`U.S. Patent No. 7,512,886 to Herberger et al. (“Herberger”)
`
`U.S. Patent Application Publication No. 2012/0076357 to
`Yamamoto et al. (“Yamamoto”)
`
`U.S. Patent Application Publication No. 2010/0238262 to
`Kurtz et al.
`
`The Harvard Dictionary of Music, Fourth Edition (2003)
`
`U.S. Patent Application Publication No. 2015/0243326 to
`Pacurariu
`
`Bytedance Ltd. et al. v. Triller, Inc., IPR2021-00099, Paper 2
`(Petition) (PTAB Oct. 28, 2020)
`
`
`
`iv
`
`

`

`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`Bytedance Ltd. and TikTok Inc. (collectively “Petitioner” or “TikTok”)
`
`petition for Inter Partes Review (“IPR”) under 35 U.S.C. §§311–319 and 37
`
`C.F.R. §42 of claims 11-19 (“the Challenged Claims”) of U.S. Patent 9,691,429
`
`(“the ’429 Patent”). Petitioner respectfully submits that an IPR should be
`
`instituted, and that the Challenged Claims should be canceled as unpatentable.
`
`I.
`
`REQUIREMENTS FOR IPR
`A. Grounds for Standing
`Petitioner certifies that the ’429 Patent is available for IPR and that it is not
`
`barred from requesting this review of the Challenged Claims.
`
`B. Challenge and Relief Requested
`Petitioner requests an IPR of the Challenged Claims on the grounds below.
`
`An explanation of how these claims are unpatentable is provided in the form of a
`
`detailed description. Additional explanation and support are set forth in the
`
`Declaration of Dr. Jeremy Cooperstock (BD-1021), referenced throughout this
`
`Petition.
`
`Ground
`1
`2A
`2B
`
`Claims
`11-16
`17
`18-19
`
`Basis
`§103: Obvious in view of Hozumi and Herberger
`§103: Obvious in view of Hozumi and Yamamoto
`§103: Obvious in view of Hozumi, Yamamoto, and
`Herberger
`
`
`
`Each reference pre-dates the ’429 Patent filing date (May 11, 2015,
`
`1
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`hereinafter the “Critical Date”) and qualifies as prior art:
`
`Reference
`Hozumi (BD-1005)
`
`Herberger (BD-1022)
`
`Yamamoto (BD-1023)
`
`Date
`12/07/2011 (filed)
`06/14/2012 (published)
`4/15/2004 (filed)
`3/31/2009 (issued)
`9/22/2011 (filed)
`3/29/2012 (published)
`
`Prior Art Under
`102(a)(1)
`
`102(a)(1)
`
`102(a)(1)
`
`
`
`These references differ materially from and are not cumulative of those
`
`evaluated by the examiner during prosecution. During prosecution, the examiner
`
`allowed the claims after applicant amended them to recite “wherein synchronizing
`
`further comprises playing, from the beginning, the selected audio track at
`
`substantially the same time as the beginning of capturing each video take of the
`
`plurality of video takes.” BD-1002, 161. Applicant then argued that, in the prior
`
`art evaluated by the examiner, “capturing of the videos... are completely unrelated
`
`to any selected audio track.” BD-1002, 174-175. As set forth below, Hozumi
`
`discloses starting audio track playback and video capture at substantially the same
`
`time. None of the arguments presented in this Petition were presented by the
`
`examiner or addressed by the applicant.
`
`II. THE ’429 PATENT
`A. Brief Description
`The ’429 Patent describes “creating a music video where an audio track is
`
`2
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`synchronized with a plurality of video takes.” BD-1001, 2:29-32, FIG. 8, BD-
`
`1021, [8]. “An audio track, such as a song, may be selected” “as the basis for the
`
`music video that will be created.” BD-1001, 2:31-32, 7:55-60, 13:66-14:10, FIG.
`
`3A, FIG. 8 (step 802). A user begins to “capture video for a music video to be
`
`created by first selecting start button,” and “the selected audio track... may begin to
`
`play, and the image capturing process may capture images until the audio track, or
`
`selected portion of the audio track, has completed.” BD-1001, 9:18-29, 14:32-53,
`
`FIG. 8 (step 806); BD-1021, [9]. The ’429 Patent explains that:
`
`By having the audio track playing while the various video takes are
`captured, the video takes may be captured “in sync” with the audio
`track. For example, at each moment of a video take, a corresponding
`section of the audio track that may be playing at that point in time will
`be known. This way, each video take, whether it begins at the beginning
`of the audio track or at some other point of the audio track, will be
`synchronized to the audio track.
`
`BD-1001, 10:19-26, 14:53-64.
`
`In the ’429 Patent, a music video is “created such that a continuous music
`
`video, having portions of different takes, which may be captured at various times
`
`and at various locations, are paired together to create a professional and
`
`aesthetically unique music video.” BD-1001, 13:59-63; BD-1021, [10]. The ’429
`
`Patent also describes embodiments where “[t]he chorus section of the audio track
`
`3
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`may be paired with a portion of a video take or takes including a large number of
`
`faces determined to be within the video take.” BD-1001, 18:1-5. FIG. 7B below
`
`shows portions of video takes used (darkened portions) and unused (white
`
`portions) when a music video is created. BD-1001, 13:59-63; BD-1021, [10].
`
`BD-1001, Detail of FIG. 7B.
`
`
`
`B.
`Summary of the Prosecution History
`Original claims 11-20 were rejected based on a combination of Pomerantz
`
`(US2014/0160250) in view of Shore (US 8,244,103) and/or Miyasaka
`
`(US2008/0055469), Giordano (US2015/0264573), and Kawamura
`
`(US2008/0037953). BD-1002, 126-136. In response, the applicant amended the
`
`independent claims to include “wherein synchronizing further comprises playing,
`
`from the beginning, the selected audio track at substantially the same time as the
`
`4
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`beginning of capturing each video take of the plurality of video takes.” BD-1002,
`
`164-167. With respect to the above amendment, the applicant argued that:
`
`To ensure that the takes can be played interchangeably, sequenced
`interchangeably, and dispersed throughout the music video however the
`user sees fit, the plurality of captured takes are synchronized with the
`selected audio track while the plurality of video takes are being
`captured. Once these takes are captured, the music video is then created
`using the subset of video takes, which are synchronized to the selected
`audio track so that the takes can be creatively arranged in a multitude
`of ways.
`
`BD-1002, 174. Applicant argued that the claims require “record[ing] multiple
`
`videos and synchroniz[ing] them not merely to audio signals surrounding the
`
`subject of the captured video, but rather to a preselected audio track,” and
`
`“Pomerantz discloses no such preselected audio track.” BD-1002, 174-175. With
`
`respect to Shore, the applicant argued that “the capturing of the videos in Shore are
`
`completely unrelated to any selected audio track in Shore.” BD-1002, 175.
`
`Applicant used a similar argument to distinguish Miyasaka. BD-1002, 178.
`
`With respect to the features relating to a number of faces within each video
`
`take recited in claim 17, the applicant argued that Giordano “only identifies the
`
`presence of any faces in an image” and Kawamura “says nothing about such
`
`‘feature amounts’ relating to the number of faces in any video take.” BD-1002,
`
`182-183. As described in detail below, other prior art references—which were
`
`5
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`never before the examiner—teach or suggest the above-referenced and other
`
`claimed features. BD-1021, [14].
`
`C. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art as of the Critical Date of the ’429 Patent
`
`(“POSITA”) would have been a person with a working knowledge of digital video
`
`and audio technologies. The person would have had a Bachelor of Science, a
`
`Bachelor of Applied Science, or a Bachelor of Engineering degree in an academic
`
`discipline emphasizing the design of electrical, computer, or software technologies,
`
`in combination with training or at least one to two years of related work experience
`
`with capture and processing of data or information, including but not limited to
`
`digital video and audio technologies. BD-1021, [16]-[17]. Alternatively, the
`
`person could have also had a Master of Science, Master of Applied Science, or
`
`Master of Engineering degree in a relevant academic discipline with less than a
`
`year of related work experience in the same discipline. Id.
`
`D. Claim Construction
`Claim terms are construed according to the Phillips standard. Phillips v.
`
`AWH Corp., 415 F.3d 1303 (Fed. Cir. 2005); 37 C.F.R. § 42.100. Based on the
`
`evidence below, striking similarities between the prior art’s description of the
`
`claimed elements and that found within the ’429 Patent specification, and the well-
`
`established principle that “claim terms need only be construed to the extent
`
`6
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`necessary to resolve the controversy,” no formal claim constructions are necessary
`
`in this proceeding. Wellman, Inc. v. Eastman Chem. Co., 642 F.3d 1355, 1361
`
`(Fed. Cir. 2011).
`
`That said, to be clear, the term “synchronizing” has a meaning well
`
`understood to POSITAs. As of the Critical Date, POSITAs would have understood
`
`the term “synchronizing” to include establishing a relationship between each of a
`
`plurality of video takes and an audio track. BD-1021, [20]. Below are examples
`
`of synchronizing video takes with an audio track that demonstrate a POSITA’s
`
`understanding of the term “synchronizing”:
`
` “mouth movements [in a video take] are coordinated with the singing
`
`[in an audio track]” (BD-1006, [0008]; see also BD-1005, [0078];
`
`BD-1013; BD-1014, 7:36-43; BD-1015, [0006]);
`
` “videos... synchronized against a segment of a given audio track”
`
`(BD-1004, [0026]);
`
` “assigns the encoded video data a time stamp” and “assigns the
`
`encoded audio data a time stamp,” and “audio data... is synchronized
`
`with the points time [sic] indicated by the time stamps assigned at the
`
`time of encoding practice captured video data” (BD-1005, [0043],
`
`[0077]);
`
` “[s]ince both the audio information and the video information are
`
`7
`
`

`

`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`recorded by the same video media recording device, the audio and
`
`video information are in sync with one another when recorded” (BD-
`
`1016, 8:60-63);
`
` “metadata structure... includes an audio start time field to synchronize
`
`the audio track with... the media item” (BD-1017, [0148]);
`
` “changes in a presentation of the video data... synchronized to the
`
`transition points in the audio” (BD-1018, 2:47-3:5; see also BD-1015,
`
`[0007]; BD-1019, 2:22-35);
`
` starting playback of audio track and capture of video take at
`
`substantially the same time (with or without a countdown) (BD-1004,
`
`[0031]; BD-1005, [0003], [0008]-[0010], [0067], [0077], [0092]; BD-
`
`1020, 13:41-42, 14:9-19).
`
`Thus, a POSITA would have viewed the term “synchronizing” as covering these
`
`and other examples found in the prior art. BD-1021, [20].
`
`III. THE CHALLENGED CLAIMS ARE UNPATENTABLE
`A. GROUND 1: Claims 11-16 are obvious based on Hozumi and
`Herberger
`1. Overview of Hozumi
`Hozumi describes a “video-audio processing method” for capturing “plural
`
`pieces of video data synchronized with a single piece of audio data.” BD-1005,
`
`Abstract, [0090]. Hozumi’s “video-audio processing apparatus” “acquire[s]
`
`8
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`encoded audio data” from device memory in response to a user selection and
`
`“captur[es] a video image of an object in synchronization with an output of the
`
`audio data by [an] audio output unit.” Id., [0011], [0089]. For example, Hozumi
`
`describes that, “if the user gives an instruction to start capturing video images”
`
`(e.g., by interacting with a user interface), Hozumi’s apparatus “causes [the] audio
`
`output unit 110 to sequentially output the audio data” to which captured video will
`
`be synchronized. Id., [0092]. Hozumi describes that “if plural pieces of video data
`
`are created by using video-audio processing apparatus... to record, or capture,
`
`video images of a single user playing a single music with different musical
`
`instruments, all of the plural pieces of video data can be synchronized with the
`
`audio data of the single music.” BD-1005, [0090]. Hozumi’s techniques make it
`
`“possible to produce a promotion video or the like in which a single player plays as
`
`if playing plural different music instruments simultaneously.” Id.; BD-1021, [21]-
`
`[23].
`
`2. Overview of Herberger
`Herberger describes “automatically aligning audio and video tracks of a
`
`multimedia work, wherein change points or ‘markers’ are located in the
`
`accompanying digital audio work and then transitions are constructed around
`
`‘breaks’ in the video work to give the impression that the two works are in
`
`synchronization.” BD-1022, 2:6-11. Herberger discloses that the “audio work will
`
`9
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`preferably be music.” BD-1022, 2:12. Herberger’s techniques include
`
`“identifying points in the audio work where changes occur” by, for example,
`
`“determination of the musical rhythm (beat/time signature) of the music,
`
`identification of changes in its volume, location within the audio track of a chorus
`
`or refrain, identification of changes in musical key, bar locations, strophe and
`
`refrain, etc.” BD-1022, 2:31-43, 8:8-17, 8:55-9:3.
`
`Herberger also analyzes video data “to locate and mark its natural breaks,”
`
`for example, “between two adjacent video clips, at the start (or end) of a user-
`
`specified effect or transition, or within a video clip (e.g., if the illumination
`
`changes drastically, if the sound level changes, etc.).” BD-1022, 2:50-55, 6:12-16,
`
`6:65-7:10, 8:17-23. “Given a collection of audio breaks and video markers,”
`
`Herberger “aligns the video breaks (e.g., cuts or transitions) with the markers that
`
`have been identified in the audio data” by, for example, “adjusting the start and
`
`stop times of the video transitions to match the occurrence of markers in the audio
`
`works,” “lengthening and shortening the constituent film clips to match the
`
`location of the audio markers” and/or “relocat[ing] video clips in time in order to
`
`synchronize one or more transitions with musical markers.” BD-1022, 3:13-15,
`
`4:38-55, 11:27-49, 11:63-13:10, 13:48-14:5; BD-1021, [24]-[25].
`
`3.
`The combination of Hozumi and Herberger
`Accounting for Herberger, Hozumi’s video-audio processing apparatus
`
`10
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`identifies locations within the recorded audio track of a chorus and creates a music
`
`video by displaying a portion of the captured video takes based on the identified
`
`locations. BD-1021, [26]. Specifically, Hozumi’s encoded audio data stored in
`
`device memory (BD-1005, [0057], [0026], [0043]-[0044], [0051], [0059], [0089])
`
`is analyzed to “identify[] points in the audio work where changes occur,” for
`
`example, “location within the audio track of a chorus or refrain,” “entry into/exit
`
`from a chorus or verse, [and/or] changes in strophe or other musical phrasing”
`
`(BD-1022, 2:31-43, 8:8-17, 8:55-9:3). BD-1021, [26].
`
`Per Herberger, “the user will preferably select one or more video clips for
`
`inclusion in [a video track of] the multimedia work.” BD-1022, 6:47-60, 2:14-30.
`
`“The user will preferably be able to edit and add to the video data using standard
`
`video editing functions.” Id. Herberger then analyzes the video data “to locate and
`
`mark its natural breaks,” for example, “between two adjacent video clips, at the
`
`start (or end) of a user-specified effect or transition, or within a video clip (e.g., if
`
`the illumination changes drastically, if the sound level changes, etc.).” BD-1022,
`
`2:50-55, 6:12-16, 6:65-7:10, 8:17-23. Accounting for Herberger, Hozumi’s
`
`apparatus would incorporate standard video editing functions for the user to select
`
`and edit Hozumi’s “plural pieces of video data” (plurality of video takes) (BD-
`
`1005, [0089]-[0090]) to create video clips (the portion of the plurality of video
`
`takes) for inclusion in a multimedia work such as a music video. BD-1022, 6:47-
`
`11
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`60, 2:14-30; BD-1021, [27]. Hozumi’s apparatus would also analyze the video
`
`data included in the video track to locate and mark its natural breaks. BD-1022,
`
`2:50-55, 6:12-16, 6:65-7:10, 8:17-23, FIG. 3; BD-1021, [27]. Hozumi’s video-
`
`audio processing apparatus “produce[s] a promotion video” using “plural pieces of
`
`video data synchronized with a single piece of audio data” (BD-1005, [0090]),
`
`including “align[ing] the video breaks (e.g., cuts or transitions) with the markers
`
`that have been identified in the audio data” by, for example, “adjusting the start
`
`and stop times of the video transitions to match the occurrence of markers in the
`
`audio works,” “lengthening and shortening the constituent film clips to match the
`
`location of the audio markers” and/or “relocat[ing] video clips in time in order to
`
`synchronize one or more transitions with musical markers” (BD-1022, 3:13-15,
`
`4:38-55, 11:27-49, 11:63-13:10, 13:48-14:5). BD-1021, [27].
`
`Herberger discloses that “given the modern trend toward incorporation of
`
`cameras into other electronic components (e.g., in handheld computers, telephones,
`
`laptops, etc.) those of ordinary skill in the art will recognize that the camera might
`
`be integrated into the computer or some other electronic device and, thus, might
`
`not be a traditional single-purpose video or still camera.” BD-1022, 5:15-20. As
`
`Dr. Cooperstock states in his declaration: “A POSITA would have understood that
`
`Herberger’s techniques are applicable to Hozumi’s video-audio processing
`
`apparatus, which implements functionalities of a camera and a computer, and
`
`12
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`would be implemented in Hozumi’s apparatus in the form of software running on
`
`the central processing unit of Hozumi’s apparatus.” BD-1021, [28] (citing BD-
`
`1005, [0032], [0049], [0088], FIGS. 1, 8; BD-1022, 5:1-6). In combination,
`
`Herberger’s features are implemented in Hozumi’s system in a manner that is
`
`similar to their implementation in Herberger’s system. Id.
`
`4.
`Reasons to combine Hozumi and Herberger
`Hozumi and Herberger are in the same field of art and relate to
`
`synchronizing video data to an audio track. BD-1005, [0003], [0012], [0087]-
`
`[0092]; BD-1022, 2:6-11; BD-1021, [29]. A POSITA would have been motivated
`
`and would have found it obvious and straightforward to combine the apparatus of
`
`Hozumi with the teachings of Herberger to “produce a promotion video” using
`
`“plural pieces of video data synchronized with a single piece of audio data” (BD-
`
`1005, [0090]), including “align[ing] the video breaks (e.g., cuts or transitions) with
`
`the markers that have been identified in the audio data” by, for example, “adjusting
`
`the start and stop times of the video transitions to match the occurrence of markers
`
`in the audio works,” “lengthening and shortening the constituent film clips to
`
`match the location of the audio markers” and/or “relocat[ing] video clips in time in
`
`order to synchronize one or more transitions with musical markers,” as suggested
`
`by Herberger (BD-1022, 3:13-15, 4:38-55, 11:27-49, 11:63-13:10, 13:48-14:5).
`
`BD-1021, [29].
`
`13
`
`

`

`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`A POSITA would have been motivated to apply Herberger’s teaching to
`
`Hozumi to beneficially provide a system with additional capabilities of video
`
`editing for creating a music video that “enable[s] a user to create sophisticated
`
`multimedia compositions in which both audio and video data are in substantial
`
`alignment, which alignment enhances the watching experience of a user created
`
`composition.” BD-1022, 3:44-50; BD-1021, [30]. Applying Herberger’s teaching
`
`to Hozumi would provide for creating “a multimedia work that features video
`
`transitions that occur, for example, on the beat of the music, at tempo changes, at
`
`key changes, etc.” such that “[t]he overall impression will be that the entire work
`
`has been crafted according to the structure of the music.” BD-1022, 4:60-65; BD-
`
`1021, [30].
`
`Further, a POSITA would have found it obvious to combine Hozumi with
`
`Herberger because doing so would have been merely combining prior art elements
`
`according to known methods to yield predictable results. BD-1021, [31]; BD-
`
`1005, [0090]; BD-1022, 3:13-15, 4:38-55, 11:27-49, 11:63-13:10, 13:48-14:5.
`
`“[W]hen a patent ‘simply arranges old elements with each performing the same
`
`function it had been known to perform’ and yields no more than one would expect
`
`from such an arrangement, the combination is obvious.” KSR Int’l Co. v. Teleflex
`
`Inc., 550 U.S. 398, 417 (2007). A POSITA would have recognized that applying
`
`Herberger’s teachings to Hozumi’s apparatus would have led to predictable results
`
`14
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`without significantly altering or hindering the functions performed by the
`
`combined system. BD-1021, [31]; BD-1005, [0090]; BD-1022, 3:13-15, 4:38-55,
`
`11:27-49, 11:63-13:10, 13:48-14:5. In fact, a POSITA would have been motivated
`
`to enhance Hozumi’s apparatus with the well-known technique of “align[ing] the
`
`video breaks (e.g., cuts or transitions) with the markers that have been identified in
`
`the audio data” by, for example, “adjusting the start and stop times of the video
`
`transitions to match the occurrence of markers in the audio works,” “lengthening
`
`and shortening the constituent film clips to match the location of the audio
`
`markers” and/or “relocat[ing] video clips in time in order to synchronize one or
`
`more transitions with musical markers” (BD-1022, 3:13-15, 4:38-55, 11:27-49,
`
`11:63-13:10, 13:48-14:5) to “produce a promotion video” with “plural pieces of
`
`video data synchronized with a single piece of audio data” (BD-1005, [0090]).
`
`BD-1021, [31]. The combined system would include such features to achieve the
`
`predictable benefits, such as those recognized in Herberger. Id.
`
`A POSITA would have had a reasonable expectation of success in making
`
`this modification, and would have reasonably expected to reap benefits of
`
`Herberger’s teachings. BD-1021, [32]; BD-1022, 3:44-50, 4:60-65. Hozumi and
`
`Herberger are in the same field of art and relate to synchronizing video data to an
`
`audio track. BD-1005, [0003], [0012], [0087]-[0092]; BD-1022, 2:6-11; BD-1021,
`
`[32]. Both Hozumi and Herberger describe types of devices that create a music
`
`15
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`video using a plurality of video takes. BD-1005, [0087]-[0092]; BD-1022, 2:50-
`
`55, 6:12-16, 6:65-7:10, 8:17-23; BD-1021, [32]. In combination, Herberger’s
`
`features are implemented in Hozumi’s system in a manner that is similar to their
`
`implementation in Herberger’s system. Id. Accordingly, implementing
`
`Herberger’s teaching in Hozumi’s system would have been routine and
`
`straightforward to a POSITA. Id. It would have been clear that such a
`
`combination would predictably work and provide the expected functionality. Id.
`
`5.
`
`Analysis
`(a) Claim 11
`[11.0]: “A user device, comprising:”
`Per Hozumi, a “video-audio processing apparatus” (a user device)
`
`“captures video images of the user’s actions in synchronization with the normal
`
`audio data.” BD-1005, [0023], [0078], [0088], FIGS. 1, 8; BD-1021, [33]. The
`
`“video-audio processing apparatus” includes an “operation unit 102” that “receives
`
`operation signals inputted by a user such as: a start or a stop of capturing video
`
`images; and a selection of the normal video capture mode, the practice video
`
`capture mode, or the reproduction mode.” BD-1005, [0032]-[0033], [0088]. The
`
`video-audio processing apparatus 100 is a user device because it operates at the
`
`direction of the user. Id.; BD-1021, [33].
`
`16
`
`

`

`
`
`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`User device
`
`Hozumi (BD-1005), FIG. 1 (annotated)
`
`
`
`[11.1]: “at least one audio input component;”
`Per Hozumi, the “video-audio processing apparatus” includes “audio input
`
`unit 142.” BD-1005, [0032], [0088], FIGS. 1, 8. “Audio input unit 142 includes
`
`a microphone. In the case of the normal video capture mode, audio input unit 142
`
`converts the sounds inputted at the time of capturing video images to electric
`
`signals to create normal audio data in response to a control command outputted by
`
`audio controller 132.” BD-1005, [0057], [0026], [0043]-[0044], [0051], [0059].
`
`17
`
`

`

`
`
`
`
`Attorney Docket No. 50048-0001IP2
`IPR of U.S. Patent No. 9,691,429
`
`
`Hozumi’s video-audio proce