PROVISIONAL APPLICATION
`
`eeeEEE
`Attorney Docket No.
`
`19838-000300US
`
`[pio
`/
`
`by:
`
`ASSISTANT COMMISSIONER FOR PATENTS
`
`BOX PROVISIONAL PATENT APPLICATION
`Washington, D.C. 20231
`Sir:
`
`Transmitted herewith for filing is a provisional patent application under CFR 1.53(c) of:
`
`&==
`
`— “Express Mail” Label No.__EL170120073US
`SF
`Date of Deposit:
`2?==sS
`T hereby certify that this is being deposited with the United States
`, So
`Postal Service “Express Mail Post Office to Addressee” service
`3 ——a
`under 37 CFR 1.10 on the date indicated above, addressedto:
`—————
`Assistant Commissionerfor Patents
`Box Provisional Patent Application
`fawn)
`
`
`Washington,
`. 20231
`3
`nn
`hehe
`
`
`a ES
`2=
`SS=>
`me =
`=
`
`
`
`
`
`
`LAST NAME
`
`FIRST NAME
`
`MIDDLEINITIAL
`
`Singhal
`Molloy, Ph.D.
`
`
`
`
`
`RESIDENCE(City/State/Country)
`Palo AR, California, United States
`San Jose, California, United States
`Los Gatos, California, United States
`
`
`
`
`
`
`
`
`
`itle: INTERNET CAMERA VIDEO PRODUCER
`
`Enclosed are:
`
`[2 23___pages of the specification (including description and drawings).
`
`sheet(s) of [
`] formal
`[
`] informal drawing(s).
`
`Abstract.
`A verified statement to establish small entity status under 37 CFR 1.9 and CFR 1.27.
`The invention was made by or under a contract with the following agency of the United States Government:
`under Government contract number
`
`Postcard
`
`
`
`
`Please charge Deposit Account No. 20-1430 as follows:
`
`
`[X]
`[X]
`
`$150.00
`$
`Filing fee
`Any additional fees associated with this paper or during the pendencyofthis application.
`
`2___extra copies of this sheet are enclosed.
`
`Customer No. 20350
`
`TOWNSEND and TOWNSENDand CREW LLP
`Two Embarcadero Center, 8 Floor
`San Francisco, California 94111-3834
`Telephone: (650) 326-2400
`Fax: (650) 326-2422
`PA 3023752 v1
`
`Respectfully submitted,
`
`TOMANSEND and TOWNSEND and CREW LLP
`
`a
`7
`~ ”
`
`‘
`|
`
`
`Richard T. Ogawa
`Reg. No. 37,692
`Attomeys for Applicant
`
`Page | of 1
`
`IPR2018-01413
`Sony EX1018 Page1
`
`IPR2018-01413
`Sony EX1018 Page 1
`
`

`

`-LUXXONCorporation
`
`\
`
`a)
`
`Internet Camera Video
`Producer
`
`
`
`Confidential
`
`Page 1
`
`IPR2018-01413
`Sony EX1018 Page 2
`
`IPR2018-01413
`Sony EX1018 Page 2
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`
`
`Confidential
`
`Page 2
`
`IPR2018-01413
`Sony EX1018 Page 3
`
`IPR2018-01413
`Sony EX1018 Page 3
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`_
`
`Table of Contents
`
`REQUTREMENTS...sscscssscecesesvesosesesesenessecscesesssoesssarszerserersesesoncsesessoassnveeseserssenseasessseconcesesonsensesssasnensosoncsorens4
`DEFINITION .......sssssessessecssessssscsvecncsvecasssssusssessscsscasssussussassncssuseesonvssscaveaseassassassussucsactursnssessusssssuvsusssssescecenceuse 4
`KEY FEATURES......ssssccsssssssscscsessecssssossusssssassssssusssssnscussasescsnssasssesasssssssssssabsuvssssisussussucecessecsececcesceecescescecee4
`SAMPLE NETWORK TOPOLOGY....s.ssssssssessecssessessessscsrssssssesssssuesuecessasssessesessesasssecsessussrssecasssursusancaesecesereeuces 6
`PERFORMANCE......scsseccsssssesesssssssacsesacsussssssucsucsusevensevenssssssssarssscarssssessesassssepsssussusassscussecsesersecsecesciseececerceecees 7
`OPERATING MODES.......-escssssesssscseecesssesscsssssssssusssssesarscussussucssussussussectesassaarssussissavssesepsusssesenssssiseessssesceseecs 8
`Setyp MOde....sesessreesessssesvissesersessssvsssesssesssscsssssssssessussasssessesesasssessesaresnessvssussascavesuessusisssiscseresessuessessssavees 8
`Session Registration Mode ........ssccsssssssssssssssssesssssssssscssssessssusersuessvecsavssssssssuessussesasessvessisssssarstsessssesaseee 8
`Single/Unicast Mode .scccecccceccssesssssessevesssssessssessssssessusssssecsesauarenussssavavsssassteaversaresesissesssersarssesasaveaveseeucenee 8
`Sti] Capture Mode cs. ceccseeccesessssesssssssssssssesssvesssasesssusssssessrsuessuessasesavessasesssessusessusssssesesuessssessssesusesenseses 9
`Monitor M0de......ccsceeecveccscsscesvesssssssssssssssssecssesusssessissussuesussrevarssusasssueasssssresisasssiesessessuseusitsetesseaseaeevesees 9
`Instant Replay MOde.....sescseccssccsscsssssvssssesssssssvesssesssesssusesssccsssssvesssessasenucensensecsueesasasavssuessnsssnecsuesavessesens 10
`CONFIGURATION...sesssssssssssssecsscavesscssesssssssesucssusssussssssssssucaneseuecsussasesussasesessussusaressussssssssssessessurssvesavasecsneeneee 10
`Setup MOdE....sessescsecssessesrsssssesnecsssasssvsssessscsuecsusssesssssssessssssssuesersaveanesuesasarecsassnessucssesssesessussavssuesevesesens 10
`Session Registration Mode ......sscssscesssesssessvesscssssssssssscssisssasaeseseasaseasssnsssessussusssssssesisssissssssuvaresinsesesene il
`Single/Unicast Mode o......sscsecsessvessessssssssssessessvessssssscssssssssusssucsevesucssvsnessusassssssisarestusssessscsussisesevsasseesens il
`Still Capture Mode ...cecscecccsesssesvcssscssesssescssesssesssssssesssesssssssssessvessavasucsassssnsstucsusesucssucsusesstsctivessesesessseee 12
`MOnitOr MOG... eeseccecsescesescssesesesssessvecsessssssssessesessvsussucsusarsassatsucasassasssstsasssssacissrssesearseterssusescaseceseesee. 12
`Instant Replay Mode.....c.cccccscsessesvsssessssssesssssesssssesussssssussssssesavesivscsavenessussestsesesissusstssuesacassssesessesssceasens 12
`ARCHITECTURE & FUNCTION. ....scscccssssssesesstecsosonesersevseccscrescscsecseasuenenronesensenssansnesesvennensssssvsosereseveseses 13
`HIGH LEVEL. .0....cssscessesessssessessscsssssanesssesscsssssnssusssassusarcacausacsusasarssesususitsststisessssasavsususisetstsisessocesscececeseecees 13
`GUL...sssesessssseseessvsssvcavecasessccevecucsnessussusssssasusssssssessscasssssessuesaresuesnusssessussusasscausstussusesessvesuessssaresusseusenscenceeee 14
`LOCA GUT ce cesessencsesesesscessesesesssesesessssssessssscscenescsvsvavsrcisaravsssasasasaseacasssarsvasasararsesusavisicsessareseseseseveveresseeees 14
`Remote Viewer GUI u..eessesesssessessssesesesssssesssssscsessssvarssuseseuenssusisansssassesesissssusavarsasstssavavasessasencissesecanceces 15
`MASTER CONTROL.....csssssssesssessssssssessssssssssccsscsusacacsvecsssesssssesssssaassnassssasssasasavasesessisusavaseceresesceesseceseceseecees 15
`FRAME BUFFERQuo... scsssscssssssssesesessesessevssssussssessssessvevsavasavencsassssussssussesscasssestasassutisassusasateucasseessececeecesecescesees 15
`TRANSCODER......sesscsesessssssscscsssesssscsssseacsssasssssesscsssessesucussecusssaussesasaesncavarsecasaressesassusavaseneasavsaceesesscecesescsececes 16
`CHOPPED seesessesecsesvsssesesevsseerenecuesessestsssessssesussscssssesssnesssssassssnessssusasnssusatsusstsasansasatessssesesessessscassessesessaeeee 16
`DCTSuub-sampler...cccccccceccsssssvecvesvesssesessessssssesssnsesessscsssesssvesssssssuesuvaucasnesssavsresresaussesiesessivasesasseeaees 16
`Frame Rate Reducer......cccececcccessscssesessssvsssssssssssssarsvssivavsnssssivassnsavasessavasisissessitaversivevsuseveveiseeeessessesees 17
`Color Depth Reducer ...cecceccsessecssssssssssessssessesssessesssesssesssessvssssesissasesacsuesuessusansssessuesisssessissisesesssesieseese. 17
`Bitrate CONHOL ..sceceecessscsecsssssssssesessssssvsssssvssessvssssussussvssssussasassasasssusassasasarsavarsssrssissssavarseccussesevessesecesces 17
`STREAM CASTER.....essssssssssscssssescsnenseussssstsnesussassussscscsesssesscsvssssnecsucusssssecusausussscassacassasasasesscsusevarsevssesavansesees 17
`NETWORK INTERFACE......sccssessssssssssscesssssesessrescssssesesncessvecesucsusussassssusasassusutasasssessasssapassueassneececeserceeseescesees 17
`MPEG-1,-2,-4 ENCODER....ssssscsssssssssscsecsecsnesasesessscssssucsssssvssuecoesesensussassaussesassuesassassesssecersevesssnesevesesseseeese 18
`ENCRYPTOR.....sssssssscsesssssssssssessssssssnsssevsscessessssscsssscsesasascuscucurarsuesessssusscassstassssusevavassasevareusaperseseeseceeceescseeces 18
`DESIGN SPECIFICATIONG........scssssscososesscossnesssoesesssesessucsosseccesessessnsnesesecesencesenetersoterevessasenoseseesesenenssooees 19
`MASTER CONTROL.....ssssssccsssssssssssssssssecssesasssssesssesssssecsssesuvssnessucesessusssscsussusssussussavsssssscssecsuvaneseuscesseeeeseeeee 19
`STREAM MANAGER.........ccccscscsssssessesecssesssscsvscesscscsesusscussecsesueaesessesscacarsecacaveeseeveceeeececesenesesesseeeateesecasasenees 20
`Stream Table .cccececcscsecssssssessssessessessssssessssuessssvssucscsnsavsssssucsacausatsacsusaarsavsrssscsssiversesesesvercsteeseesecessesceceeces 21
`RISP. esesssssvscsesesecsesesssessesesssasasessusssssssssnsuessssesessssssvesssassnsnesesaraueassusasavasarsasavassacatsreasaucusssetecsiseesseececescees 22
`LOG saesseecsessessesssesnessesusssucsssssasssesuessucsvecsusssusssncssssssssusssssssusssusssucssessussuesuesanssuesasaressussavesesssessesereseeseeseese 22
`
`
`
`
`
`Confidential
`Page 3
`
`IPR2018-01413
`Sony EX1018 Page 4
`
`IPR2018-01413
`Sony EX1018 Page 4
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`_
`
`
`
`REQUIREMENTS
`
`DEFINITION
`
`The missionof the Intemet Video Producer (IVF)is to:
`1. Camera Control: Provide the interface necessary through which the IVP can control the Sony USB
`camera.
`
`2. Unicast MPEG Video/Still Image: Process the parent stream from Sony USB cameras and seamlessly
`deliver a unique child stream of MPEGvideo ora still image to each display whichis registered to
`display such streams or images.
`3. User Interface: Provide GUIsto the
`3.1. user (at camera end), and
`3.2. viewer (via a website) on the Internet.
`4. Feedback Control: Provide the interface necessary through which the IVP can receive and process
`commands from agents on the network:
`4.1. Viewers
`4.2, WEBSITE!
`4.3. Luxxon Gateway
`
`AWPwNE
`
`To achieve these goals, the IVP works in concert with the Sony-Luxxon USB Driverto acquire sequential
`frames of DCT data from a single parent stream off of the USB camera. It then
`transforms the data to individual child streams of different resolution DCT data,
`modifies the frame rate and color depth of each child stream,
`formats each child stream into either MPEG-1,-2, or 4,
`fragments each stream into TCP/IP frames,
`encrypts the payloads of these frames,
`and sends those frameto specific IP addresses over TCP sockets.
`Additionally, as required, it pauses video streams from the camera to acquire still images ofYUV or
`SJPEG data for processing and transmission as uncompressedstill bitmap or compressedstill JPEG images,
`respectively. It also works with the WEBSITE to register the camera and each session of streaming video.
`
`KEY FEATURES
`
`1. Multiple concurrent MPEG-1,-2,or -4 streams of video and images from a Sony USB camera to
`remote Intemetclient displays.
`e
`The multicasting of streams is enabled by the Luxxon Gateway and Real or MS Media Servers.
`e
`The production of Real Video and MS Media multicast streams are enabled by the Luxxon
`Gateway plus Real and MS Encoders and Servers.
`
`2. Displayed video resolution modifiable based on requirements of user, bandwidth andclient display.
`
`3. Displayed color depth and frame rate reduction performedif required.
`
`4, Optional:
`e
`Storing of video/images on remote website.
`
`' The WEBSITE isthesite on the Internet through which all cameras running the IVP will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras. The Session Registrar will run onthis site to manage the
`association of cameras and their streams.
`
`
`
`Confidential
`
`Page 4
`
`IPR2018-01413
`Sony EX1018 Page 5
`
`IPR2018-01413
`Sony EX1018 Page 5
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`*
`
`Further processing ofvideo on remote server:
`e Additional compression - B & P frames
`e Video analysis - scene monitoring
`e
`Instant Replay
`e
`Targeted advertisements
`
`5. Plug and play operation.
`
`6. Example video resolutions, frame rates and bandwidths (video includes audio).
`
`
`
`
`
`
`
`
`
`
`
`
`
`Confidential
`
`Page 5
`
`IPR2018-01413
`Sony EX1018 Page 6
`
`IPR2018-01413
`Sony EX1018 Page 6
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`SAMPLE NETWORK TOPOLOGY
`
` Real Time
`MPEG-1,2,4
`
` Luxxon Gateway ts
`
`optional; used for archiving
`
`and/orfurther video
`processing.
`
`
`
`S6Kbit
`160x120/5fps/color/
`MPEG4
`
`
`Leereececed
`
`WAP Server + Luxxon Gateway
`
`
`
`
`
`
`
`
`
`PDA, WinCE
`
`
`
`384KDIYDSL
`640x480/1Ofps/color
`MPEGI
`
`
`
`Celiphone/
`microbrowser/
`WAP
`
`
`
`Confidential
`
`Page 6
`
`IPR2018-01413
`Sony EX1018 Page 7
`
`IPR2018-01413
`Sony EX1018 Page 7
`
`

`

`LUXXON Corporation
`internet Camera: Video Producer Specifications
`
`PERFORMANCE
`
`Output DCT chroma modes
`
`MPEGframerates up to 30fps, but fixed for each stream
`From VGA down to any multiple of 8x8, but fixed for each stream
`4:2:0 onl
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Resolution
`
`
`METRIC
`pO
`
`
`
`
`Operating Modes
`Setup Mode
`Still Capture Mode
`
`
`Session Registration Mode
`Monitor Mode
`
`
`
`Sin:
`
`
`
`
`Inputframerates [7.5,15,30fpsCd
`
`
`
`Input resolutions [VGA,QVGAsd
`
`Input DCT chroma modes
`2
`4:2:
`
`
`QVGA: 4:2:2, 4:2:0
`
`
`
`Format
`Bandwidth
`Output Video formats
`
`
`MPEG-1
`VGAorless
`> 64kbps
`
`
`MPEG-2
`VGAorless
`> 64kbps
`
`A gSs a
`MPEG-4
`QVGAorless
`<
`
`
`
`OutputStill Format
`
`
`
`
`
`
`
`
`Stream Parameters
`Resolution
`Bandwidth
`
`
`
`©
`Color Depth
`Format
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`fAudioTBDC—™—OOC*SdY
`
`MPEGTransport Steam[TROC“NNCC*”d
`
`* The output video formatis dependent on the processing capabilities ofthe CPU. In particular, in order to
`achieve 64kbpsor less, we expect that both I & P MPEG-4 framesare needed; this will burden the CPU
`(amount?). If the CPU cannot support the computation of B & P frames, then only I frame MPEG video
`will be generated at either low framerates or higher bandwidths. The Luxxon Gateway will be needed to
`deliver I, B & P frame MPEGstreams to lower bandwidth destinations.
`
` Camera Controls
`
`Reset stream parameters
`e
`Stream Controls
`e=.Start stream
`e
`Pause stream
`e
`End stream
`©
`Change exposure
`®
`Take VGAstill
`
`24bit 4:2:0 YCbCr
`©
`Output Color Depths
`
`e=8bit gray scale
`
`Cropping on 8x8 boundaries supported
`
`
`
`
`
`Confidential
`
`Page 7
`
`IPR2018-01413
`Sony EX1018 Page 8
`
`IPR2018-01413
`Sony EX1018 Page 8
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`
`
`
`
`
`
`
`#of Concurrent Streams
`
`USB Cameras Supported
`
`Supported only in conjunction with the Luxxon Player.
`©
`Stream Encryption
`
`e_Different levels supported: (TBD
`
`
`Specified at registration
`e
`¢ Dependent on stream types, bandwidth & CPU (need benchmark
`results
`
`Sony 3134 based cameras with Luxxon USBCamD driver.
`
`A traffic log is maintained.
`
`
`
`
`
`
`
`
`
`
`
`OPERATING MODES
`
`The are six major modesof operation ofthe IVP:
`
`e
`
`e
`
`e
`
`e
`
`Setup Mode
`
`Session Registration Mode
`
`Single/Unicast Mode
`
`Still Capture Mode
`
`® Monitor Mode (requires the Luxxon Gateway)
`
`e
`
`Instant Replay Mode (requires the Luxxon Gateway)
`
`SETUP MODE
`
`Immediately following installation, the IVP will enter into Setup Mode. In this mode, the IVP
`will guide the user through the General Registration process (see Configuration below). The
`Setup Modecan alwaysbe re-run by the user to modify an existing registration.
`
`SESSION REGISTRATION MODE
`
`Wheneverthe user wantsto initiate a streaming/still picture session, the IVP will be run in
`Session Registration Mode.In this mode, the IVP will guide the user through the Session
`Registration process (see Configuration below). If a streaming/still session is engaged when
`the user wants to run the Session Registration Mode,he/shewill be asked to terminate the
`current session first.
`
`SINGLE/UNICAST MODE
`
`In this mode ofthe IVP:
`
`e Via the WEBSITE, viewers can request new streams from the camera.
`
`Confidential
`
`Page 8
`
`:
`IPR2018-01413
`Sony EX1018 Page 9
`
`IPR2018-01413
`Sony EX1018 Page 9
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`e
`
`Existing streams can be modified/terminated via the WEBSITE.
`
`e A request can be made, via the WEBSITE,to switch the camera into Still Capture Mode.
`
`STILL CAPTURE MODE
`
`In Still Capture Mode, a given viewer can request that either
`
`*
`
`e
`
`an uncompressedstill image (bitmap), or
`
`aJPEG compressedstill image
`
`capture be executed and the data (bitmap or JPEG) be sentto the clients browserfor display
`
`MONITOR MODE
`
`In Monitor Mode,the frames of the stream are checkedto seeif there are any notceable
`changes from frame to frame. Changes due to
`
`®
`
`e
`
`e
`
`camera vibrations,
`
`brightness, and
`
`random noisefluctuations,
`
`are deemed false and are ignored. True changes, such as the movementof an object in the
`field ofview are recognized and an alarm page is sent to the viewer's browser. This page
`plays an alarm sound andflashes an alarm message in the browser. All monitor mode
`processing ofthe child stream is done on the Luxxon Gateway.
`
`The viewerinitiates this mode from the WEBSITE.In so doing, the following information is
`supplied:
`
`®
`
`e
`
`frame rate
`
`recording length (see below).
`
`A new stream is created. This stream terminates at the Gateway with the following
`characteristics:
`
`
`
`
`
`
`
`1-5fps, specified by viewer but must meet bandwidth limitations
`
`
`
`
`
`
`
`Confidential
`
`Page 9
`
`IPR2018-01413
`Sony EX1018 Page 10
`
`IPR2018-01413
`Sony EX1018 Page 10
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`Whena true change is detected, the stream switches overto the highest possible stream
`characteristics supported by the bandwidth (between the camera and Gateway). This stream is
`recorded by the Gatewayfor a period specified by the viewer. A child stream is also sent
`from the Gatewayto the viewer; this stream is set to the highest possible stream
`characteristics supported by the bandwidth between the Gateway andthe viewer.
`
`INSTANT REPLAY MODE
`
`In Instant Replay Mode,each child stream from a camera whichis sent to a Gatewayis
`recorded for 1 minute length loops at the Gateway. Using this, the viewercan elect at any
`time to view the last one minute(or less) of video again and then continue viewingreal time
`video. This modeis an analogofinstant replay in TV broadcasts, exceptthat each viewer
`now has control over when to active and watch the instantreplay.
`
`CONFIGURATION
`
`SETUP MODE
`
`Onceinstallation of the IVP software is complete, the user will run the IVP in Setup Mode.
`This will accomplish the following objectives:
`
`1.
`
`Interrogate and determine the make of the USB camera, validating thatit is supported by
`this IVP.
`
`2. Establish a connection to the Internet via the user’s ISP.
`
`3. Record the IP address assigned to the local machine.
`
`4, Open a TCP/IP socketto the IP addr. and port of the Camera Registrar which runs onthe
`WEBSITE’.
`
`5. Post a request for General Registration to the WEBSITE.
`
`6.
`
`7.
`
`Ifrequest accepted, post a string containing the General Registration Information (TBD);
`else keep posting request until timeout.
`
`Install a cookie, on the user's PC, which will automatically identify the user and camera
`to the Session Registrar which will register the camera each time it connects for a
`streaming session and which runs on the WEBSITE
`
`8. Terminate.
`
`
`
`
`> The WEBSITE is thesite on the Internet through which all cameras running the IVP will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras,
`
`Confidential
`
`Page 10
`
`IPR2018-01413
`Sony EX1018 Page 11
`
`IPR2018-01413
`Sony EX1018 Page 11
`
`

`

`LUXXONCorporation
`internet Camera: Video Producer Specifications °
`
`SESSION REGISTRATION MODE
`
`When a camerais to be used to unicast or even single cast to the Internet. The user will run
`the IVP in Streaming Mode.This will accomplish the following objectives:
`
`1.
`
`Ifthe camerais already registered the process proceeds, else Setup Modeis entered.
`
`2. Establish a connection to the Internetvia the user’s ISP.
`
`3. The cookie (see Setup Mode) will be usedto identify the user and camera with the
`Session Registrar running on the WEBSITE.
`
`4. Record the IP address assigned to the local machine.
`
`5. Open a TCP/IP socketto the IP addr. andport of the Session Registrar which runs on the
`WEBSITE.
`
`6. Post a request for New Session to the Session Registrar.
`
`7.
`
`Ifrequest accepted, negotiate with the Session Registrar to define the session parameters
`(maximum bandwidth, maximum resolutions, maximum framerates,etc.)
`
`8. The Session Registrar will then create a web page for this camera linked to requests for
`that user's camera. This web page will include and link all supported stream
`characteristics.
`
`9. To verify that the connection is working, a 1 minute length of the supportedtest stream is
`sent to the Session Registrar. The bandwidth used bythis test stream is compared to the
`expected bandwidth and any necessary changes to the session parameters are made.
`
`10. The final session parameters are assigned and recorded locally by the IVP and remotely
`by the Session Registrar.
`
`11. Close the socket to the Session Registrar.
`
`12. Wait until the stream is started by the Session Manager running on the WEBSITE.
`
`
`
`SINGLE/UNICAST MODE
`
`Whena viewer requests a child stream via the WEBSITE,the followingactions are taken.
`
`1. The Session Managerchecksto see if a session to that camera is currently open.Ifit is, it
`updates to parameters of the streams which can be sent from the camera assuming that
`the stream currently requested is invoked.
`
`2.
`
`3.
`
`Ifa Luxxon Player is being used by the viewer, then a TCP socket connection is made
`between the IVP and the Luxxon Player.
`Ifa Real or MS Media Player is being used by
`the viewer, then an HTPP connection is made between the IVP andthelatter.
`
`Ifan RTSP connection is not established with the IVP, a connection is made. The IVP is
`then instructed to deliver the stream requested.
`
`Confidential
`
`Page 11
`
`IPR2018-01413
`Sony EX1018 Page 12
`
`IPR2018-01413
`Sony EX1018 Page 12
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`4. The IVP then streams the data to the viewer.
`
`5. A log entry is made on both the IVP machine and by the Session Manager.
`
`6. Whenthe viewer terminates, the socket is closed and the to parameters of the streams
`which can now be sent from the camerais updated.
`
`STILL CAPTURE MODE
`
`Still Capture Mode only operates if there is an active stream being sent to that viewer andthat
`viewer is the only viewer connected to the camera. In this case,
`
`1. The viewer will requesta still image (compressed or uncompressed) from the WEBSITE.
`
`2. The Session Managerwill send a pausecommand to the IVP.
`
`3. Once the IVP has acknowledged the pause command, the Session Managerwill send a
`still capture (uncompressed/compressed) commandto the IVP.
`
`4. The IVP will capture the still image and send the data via HTTP to the viewer's browser.
`
`5. The IVP will inform the Session Managerof the completionofthestill capture.
`
`6. The Session Managerwill instruct the IVP to resumethe live stream.
`
`7. The IVP will continue streaming data to the viewer.
`
`MONITOR MODE
`
`This modeis does notdirectly affect the operations of the IVP. From the IVP's viewpoint,it
`receives instructions via one of the other modes.
`
`INSTANT REPLAY MODE
`
`This modeis does not directly affect the operations of the IVP. From the IVP's viewpoint,it
`receives instructions via one of the other modes.
`
`
`
`Confidential
`
`Page 12
`
`IPR2018-01413
`Sony EX1018 Page 13
`
`IPR2018-01413
`Sony EX1018 Page 13
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications ¢
`
`
`
`ARCHITECTURE & FUNCTION
`
`
`HIGH LEVEL
`
`The IVP consists of the following major elements:
`
`Q GUI
`
`Q Master Control
`
`Q Frame Buffer
`
`Q Transcoder
`
`Q Stream Caster
`
`Q Network Interface.
`
`Its input from the USB Cameradriveris:
`
`®
`
`¢
`
`sequential frames of DCT data in the caseof video,
`
`single frame of SJPEG data or uncompressed YUV datain the caseofstill images.
`
`Its video output is a stream ofIP frames each of which consists of a header and a packet of
`MPEGdata. Each videoIP frameis sent via a specific TCP/IP socketto a particular IP
`address.
`
`Its still image output is a bitmap or JPEGfile sent via TCP/IP to the WEBSITE.
`
`The high level block diagram ofthe IVP is shown in Figure 1.
`
`
`
`Confidential
`
`Page 13
`
`IPR2018-01413
`Sony EX1018 Page 14
`
`IPR2018-01413
`Sony EX1018 Page 14
`
`

`

`
`
`RTSP
`
`Stream Casier
`
`
`
`
`NetworkW/F
`
`
`
`
`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`USB Camera
`Driver
`
`Master
`Control
`
`
`
`Figure 1: High level diagram ofthe IVP.
`
`GUI
`
`There are two GUIs associated with the IVP. Thefirst, called the Local GUI, resides and runs
`on the same PC as the camera. The second, called the Remote Viewer GUI,is used byclients
`to obtain, modify and view streams at their remote locations; this GUI is for development and
`testing purposes only and notpart ofthe product definition.
`
`LOCAL GUI
`
`The camera operator/local user will interface to the IVP via a GUI application whichis
`resident on the PC the camera is connectedto.
`
`This GUI will serve four primary functions:
`
`1.
`
`2.
`
`camera setup,
`
`camera registration,
`
`camera control,
`
`stream monitoring.
`
`< Detailed Spec TBD: Peter>
`
`Confidential
`
`Page 14
`
`IPR2018-01413
`Sony EX1018 Page 15
`
`IPR2018-01413
`Sony EX1018 Page 15
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications-
`
`REMOTE VIEWER GUI
`
`The remote viewer will interface to the IVP via a web page resident on the WEBSITEthe
`camera is registered at.
`
`This GUI will serve three primary fictions:
`
`1.
`
`stream interaction,
`
`2.
`
`camera feedback
`
`3.
`
`activation of the plug-in display for the stream
`
`< Detailed Spec TBD: Peter, Rich, Roger >
`
`MASTER CONTROL
`
`The Master Control manages
`
`e
`
`e
`
`all transactions between the IVP and external agents, and
`
`the processing of individual child streams.
`
`It interfaces to the USB driver to
`
`1.
`
`2.
`
`3.
`
`initiate and terminate streams from the camera,
`
`allows for programmingofthe driver (resolution, frame rate, exposure,etc.), and
`
`cause the driver to place the DCT video data of a given input resolution and framerate
`into the DCT Frame Buffer.
`
`It interfaces to the viewers and the WEBSITE via the Network Interface to the viewers and
`the WEBSITE. Based on commandsit receives from the WEBSITE,it sets up andinitiates a
`parent stream between the USB driver and the Transcoder. It also creates Stream Managers
`which are processes which controlindividual child streams. It communicates with these
`Stream Managers via a Stream Table which contains all of the stream's parameters.
`
`FRAME BUFFER
`
`Datais placed into the frame buffer by the USB driver. The type of data andrate at which the
`buffer is updated is programmedinto the USB driver by the Master Control.
`
`For video, an input stream of DCTdata is read from the DCT Frame Buffer one frame at a
`time and sent to the Transcoder whereit is turned into multiple child streams of either MPEG-
`1,-2, or -4 data which are then sent to the Network Interface for distribution via TCP/IP. The
`entire process is orchestrated by the Master Control unit.
`
`Forstill images, an input frame of either uncompressed YUV data or compressed SJPEG data
`is read from the Frame Buffer.
`
`
`
`Confidential
`
`Page 15
`
`IPR2018-01413
`Sony EX1018 Page 16
`
`IPR2018-01413
`Sony EX1018 Page 16
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`TRANSCODER
`
`The block diagram ofthe transcoder is shown in Figure 2. Its job is to produce multiple
`MPEG-xstreams at different resolutions, frame rates and color depths.
`
`
`
`
`DCT
`
`Frame Rate
`Color Depth
`Reducer
`Reducer
`
`
`Subsampler
`
`
`
`
`
`
`
`
`Bitrate
`MPEG-1,2,4
`
`
`Encryptor
`Control
`Encoder
`
`
`
`
`
`Figure 2: Transcoder block diagram.
`
`CROPPER
`
`The function of the cropperis to extract a rectangular region of data from the Frame Buffer.
`The coordinates of the comers of this rectangle are specified in the Stream Table. This data is
`sent to the DCT Sub-samplerfor further processing. Note that the cropper only functions in
`video mode.
`
`DCT SUB-SAMPLER
`
`The function of the DCT Sub-sampleris to down scale, in DCT space,the resolution of the
`input stream producing only streams with those resolutions which have been requested.
`
`If a given resolution stream from the DCT Sub-sampleris not to be split into multiple color
`depth streams, then it is automatically frame rate reduced before DCT Sub-sampling.
`Otherwise,it is not frame rate reduced before DCT sub-sampling but each sub-sampled
`stream is then sent to the Frame Rate Reducer and then onto the Color Depth Reducer.
`
`
`
`Confidential
`
`Page 16
`
`IPR2018-01413
`Sony EX1018 Page 17
`
`IPR2018-01413
`Sony EX1018 Page 17
`
`

`

`LUXXONCorporation
`Internet Camera: Video Producer Specifications
`
`FRAME RATE REDUCER
`
`The Frame Rate Reducer drops the appropriate framesin order to match the required outgoing
`framerate. It maintains a running counter of which framesto send for each stream.
`
`COLOR DEPTH REDUCER
`
`The Color Depth Reduceris a drastically simplified version ofthe Color Depth Reducer
`whichlies in the gateway. Whereasthe focusofthelatter is primarily to deal with the color
`characteristics of diverse display, the job ofthe CDRin the IVP is to reduce the density of
`data in the stream while makingsure notto tax the CPU. Thus,in this first implementation,
`the CDReither passes the stream through untouchedor simply sets the Cb and Cr channel
`data to 0 (or 128) thereby producinggray scale video.
`
`BITRATE CONTROL
`
`Thebit rate control block finds the quantization scale factor which will most closely result in
`the current frame compressing to the desired numberofbits. Thelatteris computed as
`follows:
`
`1. The numberofbits for the previous frame is compared with the target numberofbits per
`frame.
`
`2.
`
`Ifthe numberofbits is abovethe target, the scale factor for the stream is increased bya
`default value (TBD). If the numberofbits is below thetarget, the scale factor for the
`stream is decreased by the default value.
`
`3. The new scale factor is used as the base scale factor for the next frame and the process
`repeats,
`
`STREAM CASTER
`
`The function of the Stream Casteristo:
`
`1.
`
`Fragmentthe stream data, presumably by image frame, for transmission via TCP/IP.
`
`2. Create a header for each TCP frameto be sent. This header will include the destination
`IP address and payloadlength ofthe fragmentbeing sent.
`
`3.
`
`4.
`
`Place the IP fragmentdata into the Network Interface for the associated TCP socket.
`
`Send the fragment.
`
`NETWORK INTERFACE
`
`The function of the Network Interfaceisto:
`
`1. Manage the creation, monitoring and destruction ofall TCP/IP sockets.
`
`2. Manage the creation, monitoring and destruction of all RTSP connections.
`
`
`
`ie
`
`Confidential
`
`Page 17
`
`IPR2018-01413
`Sony EX1018 Page 18
`
`IPR2018-01413
`Sony EX1018 Page 18
`
`

`

`LUXXON Corporation
`internet Camera: Video Producer Specifications
`
`3.
`
`Send and receive data to/from the network via all TCP/IP sockets.
`
`4. Pass data received from the network to the Master Controlfor processing.
`
`MPEG-1,-2,-4 ENCODER
`
`The function of the MPEG-1,-2,-4 Encoder is to compress the DCT data inputtoit into
`MPEGstream data. At a minimum this will be I-frame data for MPEG-1,-2 streams.
`Depending on CPU capability andthe available bandwidth, B & P frameswill be included for
`MPEG-1,-2 streams. For MPEG-4data, P frameswill be required.
`
`ENCRYPTOR
`
`The function of the Encryptoris to encrypt the data using a TBD methodology. This
`encryption is done onlyif the Luxxon Gatewayis the immediate receiver of the data.
`
`
`
`Confidential
`
`Page 18
`
`IPR2018-01413
`Sony EX1018 Page 19
`
`IPR2018-01413
`Sony EX1018 Page 19
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`
`DESIGN SPECIFICATIONS
`
`
`MASTER CONTROL
`
`Input: WEBSITE HTTP command
`Output: Commandsto the Stream Manager and USB Driver
`
`This block
`e Directs Stream Managers via changing the stream parameters in the Stream Table.
`© Managesthe registration and dropping ofdisplays andtheir associated RTSP
`connections.
`Initializes the Stream Table entry to be used by each Stream Manager.
`*
`¢ Monitors the bandwidths, source and display channels and sends changesto the stream
`registers used by Stream Managers.
`It is implemented as a state machine with the following states and events:
`
`FROM STATE|TRANSITION EVENT|TO STATE
`[Bon
`
`
`
`
`[Running_|StreamComplete|Running|
`m9KO
`
`
`
`[Running__|StreamPauseRequest___[Running|
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`|StreamPauseRequest
`
`
`treamContinueRequest|
`[StreamStopRequest|
`[SteamError|
`
`|StreamStop|
`Stream Change Request
`
`||S
`
`pPp
`
`
`
`e
`
`
`
`Internal Failure
`
`
`
`
`
`< Detailed Spec TBD: Avnish >
`
`Confidential
`
`Page 19
`
`IPR2018-01413
`Sony EX1018 Page 20
`
`ee
`
`Qaine
`”aq5 tiria i
`
`|SteamEnor
`
`i
`
`
`
`
`
`
`
`
`
`
`Internal Failure
`
`EVENT
`
`DESCRIPTION
`
`Stream Change Request|Running_|
`PT
`
`
`
`
`Reset
`
`
`
`Start
`|NewStreamRequest_|
`
`IPR2018-01413
`Sony EX1018 Page 20
`
`

`

`LUXXONCorporation
`Internet Camera:
`Video Producer Specifications
`
`STREAM MANAGER
`
`Input: Commands from