ASSISTANT COMMISSIONER FOR PATENTS
`BOX PROVISIONAL PATENT APPLICATION
`
`Washington, DC. 20231
`Sir:
`
`Jgeyér
` uwmmm
`
`
`' l6"‘2
`ll!!!
`“WitW
`
`_
`
`mung:
`
`t.
`
`.
`
`OE
`£-a
`
`V
`
`‘ Em
`fi§$
`:33
`j 3%
`”I,g“
`11:
`
`
`PROVISIONAL APPLICATION
`Attorney Docket No.
`
`l9838-000300US
`
`A
`/'
`l
`(’ ra/
`/
`
`E 17 120073US
`
`“Express Mail” Label No.
`Date of Deposit:
`I hereby certify that this is being deposited with the United States
`Postal Service “Express Mail Post Office to Addressee” service
`under 37 CFR 1.10 on the date indicated above, addressed to:
`Assistant Commissioner for Patents
`
`Box Provismnal Patent Application
`Washington,
`.2023 l
`
`Transmitted herewith for filing is a provisional patent application under CFR 1.5 3(c) of:
`
`
`
`
`LAST NAME
`Singhal
`Molloy, Ph.D.
`
`
`
`FIRST NAME
`
`KMIDDLE INITIAL
`
`RESIDENCE (C1ty/State/Country)
`Palo Alto, California, United States
`San Jose, California, United States
`Los Gatos, California, United States
`
`
`
`
`
`
`itle: INTERNET CAMERA VIDEO PRODUCER
`
`
`pages of the specification (including description and drawings).
`sheet(s) of [
`] formal
`[
`] informal drawing(s).
`
`Enclosed are:
`[7X]
`23
`1
`
`Abstract.
`
`
`
`
`
`A verified statement to establish small entity status under 37 CFR 19 and CFR 1.27.
`The invention was made by or under a contract with the following agency of the United States Government:
`under Government contract number
`
`Postcard
`
`Piaease charge Deposit Account No. 20-1430 as follows:
`
`
`
`
`[X]
`[ X ]
`
`$150.00
`$
`Filing fee
`Any additional fees associated with this paper or during the pendency of this application.
`
`2
`
`extra copies of this sheet are enclosed.
`
`Customer No. 20350
`TOWNSEND and TOWNSEND and CREW LLP
`Two Embarcadero Center, 8fll Floor
`San Francisco, California 94111-3834
`Telephone: (650) 326-2400
`Fax: (650) 326-2422
`PA 3023752 v1
`
`Respectfully submitted,
`
`TO
`\
`
`SEND and TOWNSEND and CREW LLP
`
`
`
`Richard T. Ogawa
`Reg. No. 37,692
`Attorneys for Applicant
`
`Page 1 of l
`
`|PR2018-01413
`
`Sony EX1018 Page 1
`
`IPR2018-01413
`Sony EX1018 Page 1
`
`

`

`
`
`Producer
`
`
`
`Confidential
`
`Page 1
`
`|PR2018—O1413
`
`Sony EX1018 Page 2
`
`IPR2018-01413
`Sony EX1018 Page 2
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`
`
`
`
`Confidential
`
`Page 2
`
`|PR2018—O1413
`
`Sony EX1018 Page 3
`
`IPR2018-01413
`Sony EX1018 Page 3
`
`

`

`LUXXON Corporation
`
`Internet Camera: Video Producer Specifications
`
`_
`
`Table of Contents
`
`REQUIREMENTS...................
`
`................
`
`..........
`
`........................4
`
`DEFINITION .................................................................................................................................................. 4
`KEY FEATURES ............................................................................................................................................4
`SAMPLE NETWORK TOPOLOGY .................................................................................................................... 6
`PERFORMANCE ............................................................................................................................................. 7
`OPERATING MODES ..................................................................................................................................... 8
`Setup Mode.............................................................................................................................................. 8
`Session Registration Mode ...................................................................................................................... 8
`Single/Unicast Mode ............................................................................................................................... 8
`Still Capture Mode .................................................................................................................................. 9
`Monitor Mode.......................................................................................................................................... 9
`Instant Replay Mode.............................................................................................................................. I 0
`CONFIGURATION ........................................................................................................................................ 10
`Setup Mode............................................................................................................................................ 10
`Session Registration Mode .................................................................................................................... 11
`Single/Unicast Mode ............................................................................................................................. 1 I
`Still Capture Mode ................................................................................................................................ 12
`Monitor Mode........................................................................................................................................ 12
`Instant Replay Mode.............................................................................................................................. 12
`
`ARCHITECTURE & FUNCTION............................................................................................................ 13
`
`HIGH LEVEL ............................................................................................................................................... 13
`GUI............................................................................................................................................................ 14
`Local GUI. ............................................................................................................................................. 14
`Remote Viewer GUI .............................................................................................................................. I5
`MASTER CONTROL ..................................................................................................................................... 15
`FRAME BUFFER .......................................................................................................................................... 15
`TRANSCODER ............................................................................................................................................. 1 6
`Cropper ................................................................................................................................................. I 6
`DCTSub—sampler.................................................................................................................................. 16
`Frame Rate Reducer.............................................................................................................................. I 7
`Color Depth Reducer ............................................................................................................................ I 7
`Bitrate control ....................................................................................................................................... I 7
`STREAM CASTER ........................................................................................................................................ 17
`NETWORK INTERFACE ................................................................................................................................ 17
`MPEG~1,-2,—4 ENCODER ........................................................................................................................... 18
`ENCRYPTOR ............................................................................................................................................... 1 8
`
`DESIGN SPECIFICATIONS ..................................................................................................................... 19
`
`MASTER CONTROL ..................................................................................................................................... 19
`STREAM MANAGER .........................................................................................; .......................................... 20
`Stream Table ......................................................................................................................................... 2 I
`RTSP...................................................................................................................................................... 22
`Log ........................................................................................................................................................ 22
`
`
`
`
`
`Confidential
`Page 3
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 4
`
`IPR2018-01413
`Sony EX1018 Page 4
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`
`
`REQUIREMENTS
`
`DEFINITION
`
`The mission of the Internet Video Producer (IVP) is to:
`1. Camera Control: Provide the interface necessary through which the NP can control the Sony USB
`camera.
`
`2. Unicast NIPEG Video/Still Image: Process the parent stream from Sony USB cameras and seamlessly
`deliver a unique child stream of MPEG video or a still image to each display which is registered to
`display such streams or images.
`3. User Interface: Provide GUIs to the
`
`3.1. m (at camera end), and
`3.2. viewer (via a web site) on the Internet.
`4. Feedback Control: Provide the interface necessary through which the NP can receive and process
`commands from agents on the network:
`4.1. Viewers
`4.2. WEBSITEI
`
`4.3. Luxxon Gateway
`
`awewwr
`
`To achieve these goals, the IV? works in concert with the Sony-Luxxon USB Driver to acquire sequential
`frames of DCT data from a single parent stream off of the USB camera. It then
`transforms the data to individual child streams of different resolution DCT data,
`modifies the frame rate and color depth of each child stream,
`formats each child stream into either MPEG-1,-2, or —4,
`fragments each stream into TCP/1P frames,
`encrypts the payloads of these frames,
`and sends those frame to specific IP addresses over TCP sockets.
`Additionally, as required, it pauses video streams from the camera to acquire stillimages of YUV or
`SJPEG data for processing and transmission as uncompressed still bitmap or compressed still JPEGimages,
`respectively. It also works with the WEBSITE to register the camera and each session of streaming video.
`
`KEY FEATURES
`
`1. Multiple concurrent MPEG—1,-2, or -4 streams of video and images from a Sony USB camera to
`remote Internet client displays.
`0
`The multicasting of streams is enabled by the Luxxon Gateway and Real or MS Media Servers.
`0
`The production of Real Video and MS Media multicast streams are enabled by the Luxxon
`Gateway plus Real and MS Encoders and Servers.
`
`2. Displayed Video resolution modifiable based on requirements of user, bandwidth and client display.
`
`3. Displayed color depth and frame rate reduction performed if required.
`
`4. Optional:
`0
`Storing of video/images on remote web site.
`
`1 The WEBSITE is the site on the Internet through which all cameras running the NP will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras. The Session Registrar will run on this site to manage the
`association of cameras and their streams.
`
`
`
`Confidential
`
`Page 4
`
`
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 5
`
`IPR2018-01413
`Sony EX1018 Page 5
`
`

`

`LUXXON Corporation
`internet Camera: Video Producer Specifications
`
`0
`
`Further processing of video on remote server:
`0 Additional compression - B & P frames
`0 Video analysis - scene monitoring
`0
`Instant Replay
`0
`Targeted advertisements
`
`5. Plug and play operation.
`
`6. Example video resolutions, frame rates and bandwidths (video includes audio).
`
`
`
`
`
`d
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Confidential
`
`Page 5
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 6
`
`IPR2018-01413
`Sony EX1018 Page 6
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`SAMPLE NETWORK TOPOLOGY
`
`MPEG-1, 2,4
`
` Luxxon Gateway IS
`
`optional; used for archiving
`and/or further video
`processing.
`
`
`
`
`
`
`
`56Kbit
`
`160x120/5fps/color/
`-________
`WAP Sen/er + uxxon Gateway
`MPEG4
`
`
`14.4Kbps
`5Ips/160x120/color/
`
`MPEG4
`
`
`Cellphone]
`
`-
`microbrowser/
`PDA. WinCE
`WAP
`
`
`
`384Kbit/DSL
`640x480/1Ofps/color
`MPEG1
`
`
`
`
`iii
`
`
`
`Confidential
`
`Page 6
`
`|PR2018-01413
`
`Sony EX1018 Page 7
`
`IPR2018-01413
`Sony EX1018 Page 7
`
`

`

`LUXXON Corporation
`intemet Camera: Video Producer Specifications
`
`PERFORMANCE
`
`
`
`Setup Mode
`Still Capture Mode
`Monitor Mode
`
`Session Registration Mode
`
`
`Sin
`
`
`
`
`
`
`
`
`Ou nut DCT chroma modes
`
`MPEG frame rates u- to 30 us, but fixed for each stream
`From VGA down to an multiole of 8X8, but fixed for each stream
`4:220 on]
`
`Output Video formats
`
`Output Still Format
`
`Stream Parameters
`
`Format
`MPEG-1
`MPEG-2
`
`MPEG-4
`
`7d
`esolution
`
`<
`
`0<
`
`_
`
`Bandwidth
`>64kbs
`V g30" 5;
`S 64ka
`
`Resolution
`
`Mode
`Uncomressed
`Comressed
`
`Resolution
`
`JPEG
`
`
`
`
`
`
`
`
`
`Bandwidth
`
`Format
`Color Depth
`0
`
`
`Frame Rate
`0
`__
`Stream Controls
`0
`Reset stream parameters
`0
`Start stream
`0
`Pause stream
`
` End stream
`Change exposure
`Take VGA still
`
`MPEG Trans ort Steam
`
`
`
`2 The output video format is dependent on the processing capabilities of the CPU. In particular, in order to
`achieve 64kbps or less, we expect that both I & P MPEG-4 frames are needed; this will burden the CPU
`(amount?). If the CPU cannot support the computation of B & P frames, then only I frame MPEG Video
`will be generated at either low frame rates or higher bandwidths. The Luxxon Gateway will be needed to
`deliver I, B & P frame MPEG streams to lower bandwidth destinations.
`
`
`Confidential
`Page 7
`
`|PR2018—O1413
`
`Sony EX1018 Page 8
`
`
`
`
`
`
`0 O
`
`0
`
`o
`0
`
`24bit 4:2:0 YCbCr
`8bit 1 a scale
`
`Cro- am; on 8x8 boundaries sun aorted
`
`
`
`
`
`
`
`
`
`
`
`
`Camera Controls
`
`Output Color Depths
`
`IPR2018-01413
`Sony EX1018 Page 8
`
`

`

`LUXXON Corporation
`
`Internet Camera: Video Producer Specifications
`
`
`
`
`
`
`
`
`
`
`
`Stream Encryption
`
`
`
`
`
`
`
`#of Concurrent Streams
`
`
`
`
`Supported only in conjunction with the Luxxon Player.
`0
`0 Different levels su sorted: TBD
`
`Specified at registration
`o
`0 Dependent on stream types, bandwidth & CPU (need benchmark
`results
`
`
`
`
`
`
`
`
`32:3:
`
`USB Cameras Sun orted
`
`Son 3134 based cameras with Luxxon USBCamD driver.
`
`A traffic lo is maintained-
`
`
`
`OPERATING MODES
`
`The are six major modes of operation of the IVP:
`
`0
`
`0
`
`0
`
`0
`
`Setup Mode
`
`Session Registration Mode
`
`Single/Unicast Mode
`
`Still Capture Mode
`
`0 Monitor Mode (requires the Luxxon Gateway)
`
`0
`
`Instant Replay Mode (requires the Luxxon Gateway)
`
`SETUP MODE
`
`Immediately following installation, the IV? will enter into Setup Mode. In this mode, the IV?
`will guide the user through the General Registration process (see Configuration below). The
`Setup Mode can always be re-run by the user to modify an existing registration.
`
`SESSION REGISTRATION MODE
`
`Whenever the user wants to initiate a streaming/still picture session, the WP will be run in
`Session Registration Mode. In this mode, the NP will guide the user through the Session
`Registration process (see Configuration below). If a streaming/still session is engaged when
`the user wants to run the Session Registration Mode, he/she will be asked to terminate the
`current session first.
`
`SINGLE/UNICAST MODE
`
`In this mode of the IVP:
`
`0 Via the WEBSITE, viewers can request new streams from the camera.
`
`
`
`Confidential
`
`Page 8
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 9
`
`IPR2018-01413
`Sony EX1018 Page 9
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`0
`
`Existing streams can be modified/terminated via the WEBSITE.
`
`0 A request can be made, via the WEBSITE, to switch the camera into Still Capture Mode.
`
`STILL CAPTURE MODE
`
`In Still Capture Mode, a given viewer can request that either
`
`0
`
`o
`
`an uncompressed still image (bitmap), or
`
`a JPEG compressed still image
`
`capture be executed and the data (bitmap or IPEG) be sent to the clients browser for display
`
`MONITOR MODE
`
`In Monitor Mode, the frames of the stream are checked to see if there are any notceable
`changes from frame to frame. Changes due to
`
`0
`
`o
`
`0
`
`camera Vibrations,
`
`brightness, and
`
`random noise fluctuations,
`
`are deemed false and are ignored. True changes, such as the movement of an object in the
`field of view are recognized and an alarm page is sent to the viewer‘s browser. This page
`plays an alarm sound and flashes an alarm message in the browser. All monitor mode
`processing of the child stream is done on the Luxxon Gateway.
`
`The viewer initiates this mode from the WEBSITE. In so doing, the following information is
`supplied:
`
`0
`
`0
`
`frame rate
`
`recording length (see below).
`
`A new stream is created. This stream terminates at the Gateway with the following
`characteristics:
`
`
`
`
`
`
`
`1-5fps, specified by viewer but must meet bandwidth limitations
`
`
`
`
`
`
`
`Confidential
`
`Page 9
`
`IPR2018-01413
`
`Sony EX1018 Page 10
`
`IPR2018-01413
`Sony EX1018 Page 10
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`When a true change is detected, the stream switches over to the highest possible stream
`characteristics supported by the bandwidth (between the camera and Gateway). This stream is
`recorded by the Gateway for a period specified by the viewer. A child stream is also sent
`from the Gateway to the viewer; this stream is set to the highest possible stream
`characteristics supported by the bandwidth between the Gateway and the viewer.
`
`INSTANT REPLAY MODE
`
`In Instant Replay Mode, each child stream from a camera which is sent to a Gateway is
`recorded for 1 minute length loops at the Gateway. Using this, the viewer can elect at any
`time to View the last one minute (or less) of video again and then continue viewing real time
`video. This mode is an analog of instant replay in TV broadcasts, except that each viewer
`now has control over when to active and watch the instant replay.
`
`CONFIGURATION
`
`SETUP MODE
`
`Once installation of the IVP software is complete, the user will run the IVP in Setup Mode.
`This will accomplish the following objectives:
`
`1.
`
`Interrogate and determine the make of the USB camera, validating that it is supported by
`this IVP.
`
`2. Establish a connection to the Internet via the user’s ISP.
`
`3. Record the IP address assigned to the local machine.
`
`4. Open a TCP/IP socket to the IP addr. and port of the Camera Registrar which runs on the
`WEBSITE3.
`
`5. Post a request for General Registration to the WEBSITE.
`
`6.
`
`7.
`
`If request accepted, post a string containing the General Registration Information (TBD);
`else keep posting request until timeout.
`
`Install a cookie, on the user's PC, which will automatically identify the user and camera
`to the Session Registrar which will register the camera each time it connects for a
`streaming session and which runs on the WEBSITE
`
`8. Terminate.
`
`
`
`
`3 The WEBSITE is the site on the Internet through which all cameras running the IVP will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras.
`
`
`
`Confidential
`
`Page 10
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 11
`
`IPR2018-01413
`Sony EX1018 Page 11
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications ’
`
`SESSION REGISTRATION MODE
`
`When a camera is to be used to unicast or even single cast to the Internet. The user will run
`the IVP in Streaming Mode. This will accomplish the following objectives:
`
`1.
`
`If the camera is already registered the process proceeds, else Setup Mode is entered.
`
`2. Establish a connection to the Internet via the user’s ISP.
`
`3. The cookie (see Setup Mode) will be used to identify the user and camera with the
`Session Registrar running on the WEBSITE.
`
`4. Record the IP address assigned to the local machine.
`
`5. Open a TCP/[P socket to the IP addr. and port of the Session Registrar which runs on the
`WEBSITE.
`
`6. Post a request for New Session to the Session Registrar.
`
`7.
`
`If request accepted, negotiate with the Session Registrar to define the session parameters
`(maximum bandwidth, maximum resolutions, maximum frame rates, etc.)
`
`8. The Session Registrar will then create a web page for this camera linked to requests for
`that user’s camera. This web page will include and link all supported stream
`characteristics.
`
`9. To verify that the connection is working, a 1 minute length of the supported test stream is
`sent to the Session Registrar. The bandwidth used by this test stream is compared to the
`expected bandwidth and any necessary changes to the session parameters are made.
`
`10. The final session parameters are assigned and recorded locally by the IVP and remotely
`by the Session Registrar.
`
`11. Close the socket to the Session Registrar.
`
`12. Wait until the stream is started by the Session Manager running on the WEBSITE.
`
`
`
`SINGLE/UNICAST MODE
`
`When a viewer requests a child stream via the WEBSITE, the following actions are taken.
`
`1. The Session Manager checks to see if a session to that camera is currently open. If it is, it
`updates to parameters of the streams which can be sent from the camera assuming that
`the stream currently requested is invoked.
`
`2.
`
`3.
`
`If a Luxxon Player is being used by the viewer, then a TCP socket connection is made
`between the IVP and the Luxxon Player.
`If a Real or MS Media Player is being used by
`the viewer, then an HTPP connection is made between the IV? and the latter.
`
`If an RTSP connection is not established with the NP, a connection is made. The IVP is
`then instructed to deliver the stream requested.
`
`Confidential
`
`Page 11
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 12
`
`IPR2018-01413
`Sony EX1018 Page 12
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications .
`
`4. The IVP then streams the data to the viewer.
`
`5. A log entry is made on both the IVP machine and by the Session Manager.
`
`6. When the viewer terminates, the socket is closed and the to parameters of the streams
`which can now be sent from the camera is updated.
`
`STILL CAPTURE MODE
`
`Still Capture Mode only operates if there is an active stream being sent to that viewer and that
`viewer is the only viewer connected to the camera. In this case,
`
`1. The viewer will request a still image (compressed or uncompressed) from the WEBSITE.
`
`2. The Session Manager will send a pause‘command to the IVP.
`
`3. Once the IVP has acknowledged the pause command, the Session Manager will send a
`still capture (uncompressed/compressed) command to the IVP.
`
`4. The IVP will capture the still image and send the data via HTTP to the viewer's browser.
`
`5. The IVP will inform the Session Manager of the completion of the still capture.
`
`6. The Session Manager will instruct the IVP to resume the live stream.
`
`7. The IVP will continue streaming data to the viewer.
`
`MONITOR MODE
`
`This mode is does not directly affect the operations of the IVP. From the IVP's viewpoint, it
`receives instructions via one of the other modes.
`
`INSTANT REPLAY MODE
`
`This mode is does not directly affect the operations of the IVP. From the IVP's viewpoint, it
`receives instructions via one of the other modes.
`
`
`
`
`
`Confidential
`
`Page 12
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 13
`
`IPR2018-01413
`Sony EX1018 Page 13
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications (
`
`
`
`ARCHITECTURE 8c FUNCTION
`
`
`HIGH LEVEL
`
`The IVP consists of the following major elements:
`
`El GUI
`
`Cl Master Control
`
`E] Frame Buffer
`
`D Transcoder
`
`CI
`
`Stream Caster
`
`El Network Interface.
`
`Its input from the USB Camera driver is:
`
`0
`
`0
`
`sequential frames of DCT data in the case of video,
`
`single frame of SJPEG data or uncompressed YUV data in the case of still images.
`
`Its video output is a stream ofIP frames each of which consists of a header and a packet of
`MPEG data. Each video IP frame is sent via a specific TCP/IP socket to a particular IP
`address.
`
`Its still image output is a bitmap or JPEG file sent via TCP/IP to the WEBSITE.
`
`The high level block diagram of the IVP is shown in Figure l.
`
`
`
`
`
`Confidential
`
`Page 13
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 14
`
`IPR2018-01413
`Sony EX1018 Page 14
`
`

`

`LUXXON Corporation
`
`lntemet Camera: Video Producer Specifications
`
`USB Camera
`Driver
`
`
`
`
`
`
`
`
`NetworkUP
`
`
`I StreamCaster
`
`
`
`RTSP
`
`Figure I: High [earl diagrmn qftbe IVP.
`
`GUI
`
`There are two GUIs associated with the NP. The first, called the Local GUI, resides and runs
`on the same PC as the camera. The second, called the Remote Viewer GUI, is used by clients
`to obtain, modify and View streams at their remote locations; this GUI is for development and
`testing purposes only and not part of the product definition.
`
`LOCALGUI
`
`The camera operator/local user will interface to the NP via a GUI application which is
`resident on the PC the camera is connected to.
`
`This GUI will serve four primary functions:
`
`1.
`
`camera setup,
`
`2.
`
`3.
`
`camera registration,
`
`camera control,
`
`4.
`
`stream monitoring.
`
`< Detailed Spec TBD: Peter>
`
`
`
`Confidential
`
`Page 14
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 15
`
`
`
`IPR2018-01413
`Sony EX1018 Page 15
`
`

`

`LUXXON Corporation
`
`lntemet Camera: Video Producer Specifications '
`
`REMOTE VIEWER GUI
`
`The remote viewer will interface to the NP via a web page resident on the WEBSITE the
`camera is registered at.
`
`This GUI will serve three primary fimctions:
`
`1.
`
`2.
`
`stream interaction,
`
`camera feedback
`
`3.
`
`activation of the plug-in display for the stream
`
`< Detailed Spec TBD: Peter, Rich, Roger >
`
`MASTER CONTROL
`
`The Master Control manages
`
`0
`
`0
`
`all transactions between the WP and external agents, and
`
`the processing of individual child streams.
`
`It interfaces to the USB driver to
`
`l.
`
`initiate and terminate streams from the camera,
`
`2.
`
`3.
`
`allows for programming of the driver (resolution, frame rate, exposure, etc.), and
`
`cause the driver to place the DCT video data of a given input resolution and frame rate
`into the DCT Frame Buffer.
`
`It interfaces to the viewers and the WEBSITE via the Network Interface to the viewers and
`
`the WEBSITE. Based on commands it receives from the WEBSITE, it sets up and initiates a
`parent stream between the USB driver and the Transcoder. It also creates Stream Managers
`which are processes which control individual child streams. It communicates with these
`Stream Managers via a Stream Table which contains all of the stream's parameters.
`
`FRAME BUFFER
`
`Data is placed into the frame buffer by the USB driver. The type of data and rate at which the
`buffer is updated is programmed into the USB driver by the Master Control.
`
`For video, an input stream of DCT data is read from the DCT Frame Buffer one frame at a
`time and sent to the Transcoder where it is turned into multiple child streams of either MPEG-
`l,-2, or —4 data which are then sent to the Network Interface for distribution via TCP/IP. The
`
`entire process is orchestrated by the Master Control unit.
`
`For still images, an input frame of either uncompressed YUV data or compressed SJPEG data
`is read from the Frame Buffer.
`
` ”
`
`‘Hii
`
`
`
`
`
`Confidential
`
`Page 15
`
`|PR2018-01413
`
`Sony EX1018 Page 16
`
`IPR2018-01413
`Sony EX1018 Page 16
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`TRANSCODER
`
`The block diagram of the transcoder is shown in Figure 2. Its job is to produce multiple
`MPEG-x streams at different resolutions, frame rates and color depths.
`
`DCT
`Frame Rate
`Color Depth
`
`
`
`
`Reducer
`Reducer
`Subsampier
`
`
`
`
`
`
`Encryptor
`
`MPEG-1 ,2,4
`
`Figure 2: Transcoder block diagram.
`
`The fimction of the cropper is to extract a rectangular region of data from the Frame Buffer.
`The coordinates of the comers of this rectangle are specified in the Stream Table. This data is
`sent to the DCT Sub-sampler for further processing. Note that the cropper only functions in
`video mode.
`
`
`
`CROPPER
`
`DCT SUB-SAMPLER
`
`The function of the DCT Sub-sampler is to down scale, in DCT space, the resolution of the
`input stream producing only streams with those resolutions which have been requested.
`
`If a given resolution stream from the DCT Sub-sampler is not to be split into multiple color
`depth streams, then it is automatically frame rate reduced before DCT Sub—sampling.
`Otherwise, it is not frame rate reduced before DCT sub-sampling but each sub-sampled
`stream is then sent to the Frame Rate Reducer and then onto the Color Depth Reducer.
`
`
`
`Confidential
`
`Page 16
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 17
`
`IPR2018-01413
`Sony EX1018 Page 17
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`FRAME RATE REDUCER
`
`The Frame Rate Reducer drops the appropriate frames in order to match the required outgoing
`frame rate. It maintains a running counter of which frames to send for each stream.
`
`COLOR DEPTH REDUCER
`
`The Color Depth Reducer is a drastically simplified version of the Color Depth Reducer
`which lies in the gateway. Whereas the focus of the latter is primarily to deal with the color
`characteristics of diverse display, the job ofthe CDR in the IVP is to reduce the density of
`data in the stream while making sure not to tax the CPU. Thus, in this first implementation,
`the CDR either passes the stream through untouched or simply sets the Cb and Cr channel
`data to O (or 128) thereby producing gray scale video.
`
`BITRATE CONTROL
`
`The bit rate control block finds the quantization scale factor which will most closely result in
`the current frame compressing to the desired number of bits. The latter is computed as
`follows:
`
`1. The number ofbits for the previous flame is compared with the target number ofbits per
`frame.
`
`2.
`
`If the number of bits is above the target, the scale factor for the stream is increased by a
`default value (TBD). If the number ofbits is below the target, the scale factor for the
`stream is decreased by the default value.
`
`3. The new scale factor is used as the base scale factor for the next frame and the process
`repeats.
`
`STREAM CASTER
`
`The function of the Stream Caster is to:
`
`1. Fragment the stream data, presumably by image frame, for transmission via TCP/IP.
`
`2. Create a header for each TCP flame to be sent. This header will include the destination
`IP address and payload length of the fragment being sent.
`
`3.
`
`Place the IP fragment data into the Network Interface for the associated TCP socket.
`
`4.
`
`Send the fragment.
`
`NETWORK INTERFACE
`
`The function of the Network Interface is to:
`
`1. Manage the creation, monitoring and destruction of all TCP/IP sockets.
`
`2. Manage the creation, monitoring and destruction of all RTSP connections.
`
`
`
`932235
`
`
`Confidential
`Page 17
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 18
`
`IPR2018-01413
`Sony EX1018 Page 18
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`3.
`
`Send and receive data to/from the network via all TCP/IP sockets.
`
`4. Pass data received from the network to the Master Control for processing.
`
`MPEG-1,~2,-4 ENCODER
`
`The function of the MPEG-l,-2,-4 Encoder is to compress the DCT data input to it into
`MPEG stream data. At a minimum this will be I-frame data for MPEG—1,-2 streams.
`Depending on CPU capability and the available bandwidth, B & P frames will be included for
`MPEG-1,-2 streams. For MPEG4 data, P frames will be required.
`
`ENCRYPTOR
`
`The function of the Encryptor is to encrypt the data using a TBD methodology. This
`encryption is done only if the Luxxon Gateway is the immediate receiver of the data.
`
`
`
`
`
`Confidential
`
`Page 18
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 19
`
`IPR2018-01413
`Sony EX1018 Page 19
`
`

`

`LUXXON Corporation
`intemet Camera: Video Producer Specifications
`
`
`DESIGN SP ECIFICATIONS
`
`
`MASTER CONTROL
`
`Input: WEBSITE HTTP command
`Output: Commands to the Stream Manager and USB Driver
`
`This block
`
`0 Directs Stream Managers via changing the stream parameters in the Stream Table.
`0 Manages the registration and dropping of displays and their associated RTSP
`connections.
`
`Initializes the Stream Table entry to be used by each Stream Manager.
`a
`0 Monitors the bandwidths, source and display channels and sends changes to the stream
`registers used by Stream Managers.
`It is implemented as a state machine with the following states and events:
`
`
`
`EE_——
`
`
`
`
`
`
`
`
`
`
`
`
`__—
`
`
`
`
`
`
`
`
`
`
`
`
`_—
`_—
`_—
`__-
`_-
`
`__—
`_—
`_—
`—-_
` Internal Failure
`
`
`
`
`
`—_
`
`
`
`
`
`
`
`
`
`
`
`< Detailed Spec TBD: Avnish >
`
`Confidential
`
`Page 19
`
`|PR201 8-0141 3
`
`Sony EX1018 Page 20
`
`__—
`
`m—n
`-__Im:-
`
`_m-
`
`
`
`
`
`
`IPR2018-01413
`Sony EX1018 Page 20
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`STREAM MANAGER
`
`Input: Commands from the Master Control, DCT data from the Master Control
`Output: XPEG stream/frame to the appropriate Frame Buffer
`
`This block initiates, controls and terminates the processing of