.
`
`PROVISIONALAPPLICATION
`
`. *v—-—--——-—-—--—-—.
`
`Attorney Docket No.
`
`l9838—000300US
`
`I ’t/
`.r r
`fV/’ 1’
`
`B 17 120073US
`
`“Express Mail" Label No.
`Date of Deposit:
`I hereby certify that this is being deposited with the United States
`Postal Service “Express Mail Post Office to Addresscc” service
`under 37 CFR 1.10 on the date indicated above, addressed to:
`Assrstant Commrssroner for Patents
`)
`Box Proviswnal Patent Application
`Washington/515:0?“
`<-
`2‘
`,
`r
`c E
`447» (Java;
`/
`>
`t: E
`By.
`
`JD
`fig: =~—_\
`:33: E4
`«\g
`gin E
`a E
`
`‘
`
`O “
`
`E
`Em
`lg: '
`DES
`:33
`' 33"
`
`'
`ASSISTANT COMIVHSSIONER FOR PATENTS
`BOX PROVISIONAL PATENT APPLICATION
`Washington, D.C. 20231
`Sir:
`
`Transmitted herewith for filing is a provisional patent application under CFR l.53(c) of:
`
`
`
`
`
`
`fflitle: INTERNET CAMERA VIDEO PRODUCER
`as;
`
`
`
`
`
`
`Enclosed are:
`[5X]
`23
`E4]
`E]
`F3]
`Z-
`_
`]
`|
`If} ]
`
`Postcard
`
`pages of the specification (including description and drawings).
`sheet(s) of [
`] formal
`[
`] informal drawing(s).
`
`Abstract.
`A verified statement to establish small entity status under 37 CFR 1.9 and CPR 1.27.
`The invention was made by or under a contract with the following agency of the United States Government:
`under Government contract number
`
`[ x ]
`[ X ]
`
`$150.00
`$
`Filing fee
`Any additional fees associated with this paper or during the pendency of this application.
`
`2
`
`extra copies of this sheet are enclosed.
`
`Respectfully submitted,
`
`Customer No. 20350
`
`WNSEND and CREW LLP
`
`San Francisco, California 94111-3834 Richard T. Ogawa
`
`TOWNSEND and TOWNSEND and CREW LLP
`Two Embarcadero Center, 8‘h Floor
`
`Telephone: (650) 326-2400
`Fax: (650) 326-2422
`PA 3023752 v1
`
`Reg. No. 37,692
`Attorneys for Applicant
`
`Page I of 24
`
`GOOGLE EXHIBIT 1007
`
`Page 1 of 1
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`Producer
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Confidenfial
`
`
`
`Page 1
`
`
`
`Page 2 of 24
`
`Page 2 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`.1:23:
`.==—;
`
`35.1is;
`J"
`
`
`
`"5'33"if"!A53...
`
`:nkll
`“iI}
`
`
`
`Confidential
`
`Page 2
`
`Page 3 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: \fideo Producer Specifications
`
`Table of Contents
`
`REQUIREMENTS........................................................................................................................................4
`
`DEFINITION .................................................................................................................................................. 4
`KEY FEATURES ............................................................................................................................................ 4
`SAMPLE NETWORK TOPOLOGY .................................................................................................................... 6
`PERFORMANCE ............................................................................................................................................. 7
`OPERATING MODES ..................................................................................................................................... 8
`
`Setup Mode.............................................................................................................................................. 8
`Session Registration Mode ...................................................................................................................... 8
`Single/Unicast Mode ............................................................................................................................... 8
`Still Capture Mode .................................................................................................................................. 9
`Monitor Mode.......................................................................................................................................... 9
`
`Instant Replay Mode.............................................................................................................................. 10
`CONFIGURATION ........................................................................................................................................ 10
`
`Setup Mode............................................................................................................................................ 10
`Session Registration Mode .................................................................................................................... II
`Single/Unicost Mode ............................................................................................................................. 1 I
`Still Capture Mode ................................................................................................................................ 12
`Monitor Mode........................................................................................................................................ 1 2
`
`Instant Replay Mode.............................................................................................................................. 12
`
`ARCHITECTURE & FUNCTION............................................................................................................13
`
`HIGH LEVEL ............................................................................................................................................... 13
`GUI............................................................................................................................................................ 14
`Local GUI. ............................................................................................................................................. 14
`Remote Viewer GUI .............................................................................................................................. 15
`MASTER CONTROL..................................................................................................................................... 15
`
`FRAME BUFFER .......................................................................................................................................... 15
`TRANSCODER ............................................................................................................................................. 16
`
`Cropper ................................................................................................................................................. 16
`DCTSub-sampler.................................................................................................................................. 16
`Frame Rate Reducer.............................................................................................................................. 1 7
`
`Color Depth Reducer ............................................................................................................................ 1 7
`Bitrate control ....................................................................................................................................... I 7
`STREAM CASTER........................................................................................................................................ 17
`NETWORK INTERFACE................................................................................................................................ l7
`
`MPEG-1,-2,-4 ENCODBR ........................................................................................................................... 18
`ENCRYI’I‘OR ............................................................................................................................................... 1 8
`
`DESIGN SPECIFICATIONS........................................................
`
`.............................................. 19
`
`MASTER CONTROL..................................................................................................................................... l9
`STREAM MANAGER .........................................................................................'. .......................................... 20
`Stream Table ......................................................................................................................................... 21
`RISP...................................................................................................................................................... 22
`
`Log ........................................................................................................................................................ 22
`
`
`
`z
`
`::
`3:5
`'==
`
`”f
`5.;
`f
`
`Confidential
`
`Page 3
`
`Page 4 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`REQUIREMENTS
`
`DEFINITION
`
`
`
`1.
`L
`
`The mission of the Internet Video Producer (IVP) is to:
`1. Camera Control: Provide the interface necessary through which the NP can control the Sony USB
`camera.
`
`2. Unicast MPEG Video/Still Image: Process the parent mam from Sony USB cameras and seamlessly
`deliver a unique child 5% of MPEG video or a still image to each display which is registered to
`display such streams or images.
`3. User Interface: Provide GUIs to the
`
`3.1. m (at camera end), and
`3.2. xjewer (via a web site) on the Internet.
`4. Feedback Control: Provide the interface necessary through which the N? can receive and process
`commands from agents on the network:
`4.1. Viewers
`4.2. WEBSITE‘
`
`4.3. Luxxon Gateway
`
`awewwr
`
`To achieve these goals, the [VP works in concert with the Sony-Luxxon USB Driver to acquire sequential
`frames of DCT data from a single parent stream off of the USB camera. It then
`transforms the data to individual child streams of different resolution DCT data,
`modifies the frame rate and color depth of each child stream,
`formats each child stream into either MPEG-1;; or —4,
`fragments each stream into TCP/IP frames,
`encrypts the payloads of these frames,
`and sends those frame to specific IP addresses over TCP sockets.
`Additionally, as required, it pauses video streams from the camera to acquire stillimages of YUV or
`SJPEG data for processing and transmission as uncompressed still bitmap or compressed still JPEGimages,
`respectively. It also works with the WEBSITE to register the camera and each session of streaming video.
`
`KEY FEATURES
`
`1. Multiple concurrent MPEG-l,-2, or -4 streams of video and images from a Sony USB camera to
`remote Internet client displays.
`0
`The multicasting of streams is enabled by the Luxxon Gateway and Real or MS Media Sewers.
`o
`The production of Real Video and MS Media multicast streams are enabled by the Luxxon
`Gateway plus Real and MS Encoders and Servers.
`
`2. Displayed video resolution modifiable based on requirements of user, bandwidth and client display.
`
`3. Displayed color depth and frame rate reduction performed if required.
`
`4. Optional:
`0
`Storing of video/images on remote web site.
`
`1 The WEBSITE is the site on the Internet through which all cameras running the [VP will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras. The Session Regisn'ar will run on this site to manage the
`association of cameras and their streams.
`
`Confidential
`
`Page 4
`
`Page 5 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: \fideo Producer Specifications
`
`0
`
`Further processing of video on remote server:
`0 Additional compression - B & P frames
`0 Video analysis - scene monitoring
`0
`Instant Replay
`0
`Targeted advertisements
`
`5. Plug and play operation.
`
`6. Example video resolutions, frame rates and bandwidths (video includes audio).
`
`
`—_ Bandwidth
`0
`IF 176x144/MPEG4
`
`
`
`CIF/MPEG4 __
`
`
`[—
`
`-—
`
`
`
`
`
`Confidential
`
`Page 5
`
`Page 6 of 24
`
`

`

`LUXXON Corporation
`
`lntemet Camera: Video Producer Specifications
`
`SAMPLE NETWORK TOPOLOGY
`
`WES-1,2,4
`
`option‘al; used for archiving
`and/or funher video
`
`
`
`
`
`
`
`
`
`14.4KDDS
`abs/16¢! 12Wcolor/
`
`FDA. WinCE
`
`WAP Swen mnGateway
`
`Cellphone/
`mmrobrmer/
`mp
`
`Confidential
`
`Page 6
`
`Page 7 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`P ERFORMANCE
`
`.—~ . .—i_,__. ,
`
`,fi____.._~_
`
`iw_wArfi_—.~_..m fi—_.——-——-—1
`
`TARGET
`
`3
`
`2
`
`
`
`
`
`
`‘ Operating Modes
`Setup Mode
`Still Capture Mode
`|
`0
`Session Registration Mode
`0 Monitor Mode
`
`Sin_1e/Unicast Mode
`Instant R- ola Mode
`——l-‘_ 7.5 15, 30 s
`
`l-_ VGA VGA
`,
`Input DCI‘ chroma modes
`VGA: 4:2:2
`
`OVGA: 4:2:2 4:2:0
`a
`. to30 s, butfixedforeachstream
`‘_ MPEGframerates
`I
`_ From VGA down to an multile of8x8 but fixed for each stream
`‘
`I On utDCT chroma modes
`4:2:0 onl
`c——
`l Output Video formats
`Im——_;
`MPEG-1
`> 64ka
`.
`MPEG-2
`1
`QVGA or less
`MPEG-4
`l
`_———i
`' Output Still Format E_—_|
`Windows Birma (bm _
`VGA
`
`
`
`
`‘
`
`‘
`
`Resolution
`
`Color Depth
`Frame Rate
`
`0
`
`0
`
`Bandwidth
`
`Format
`
`Reset stream parameters
`Start stream
`Pause stream
`
`Change exposure
`Take VGA still
`
`-
`
`0
`0
`
`0
`0
`-
`
`0 O
`
`0
`
`o
`0
`
`24bit 412:0 YCbCr
`8bit 1 .
`scale
`
` Stream Parameters
`
`Stream Controls
`
`Camera Controls
`
`
`
`Output Color Depths
`
`
`e
`'5
`‘*
`
`‘ Cro um; Modes
`
`Cro .m; on 8x8 boundaries 3 ya rted
`
`xi
`
`
`
`Audio
`
`
`MPGE Trans' rt S.
`
`2 The output video format is dependent on the processing capabilities of the CPU. In particular, in order to
`achieve 64kbps or less, we expect that both I & P MPEG-4 frames are needed; this will burden the CPU
`(amount?). If the CPU cannot support the computation of B & P frames, then only I frame MPEG video
`will be generated at either low frame rates or higher bandwidths. The Luxxon Gateway will be needed to
`deliver I, B & P frame MPEG streams to lower bandwidth destinations.
`
`Confidential
`
`Page 7
`
`Page 8 of 24
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`l I
`
`
`
`l
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`
`0 Different levels su- orted: TBD
`
`
`
`
`
`l
`
`
`
`
`
`A traffic 10 is maintained.
`L .LL. _
`_____
`.
`--___L._J
`|
`
`l #of Concurrent Streams
`3
`
`Specified at registration
`o
`0 Dependent on stream types, bandwidth & CPU (need benchmark
`results )
`
`USB Cameras Su yo orted
`
`Son 3134 based cameras With Luxxon USBCamD driver.
`
` !
`
`OPERATING MODES
`
`The are six major modes of operation of the IVP:
`
`0
`
`0
`
`Setup Mode
`
`Session Registration Mode
`
`0
`
`0
`
`Single/Unicast Mode
`
`Still Capture Mode
`
` SETUP MODE
`
`0 Monitor Mode (requires the Luxxon Gateway)
`
`0
`
`Instant Replay Mode (requires the Luxxon Gateway)
`
`
`
`7 Z
`“'
`
`Immediately following installation, the [VP will enter into Setup Mode. In this mode, the [VP
`will guide the user through the General Registration process (see Configuration below). The
`Setup Mode can always be re-run by the user to modify an existing registration.
`
`SESSION REGISTRATION MODE
`
`Whenever the user wants to initiate a streaming/still picture session, the IVP will be run in
`Session Registration Mode. In this mode, the NP will guide the user through the Session
`Registration process (see Configuration below). If a streaming/still session is engaged when
`the user wants to run the Session Registration Mode, he/she will be asked to terminate the
`current session first.
`
`SINGLE/UNICAS'I‘ MODE
`
`In this mode of the IVP:
`
`0
`
`Via the WEBSITE, viewers can request new streams fiom the camera.
`
`Confidential
`
`Page 8
`
`Page 9 of 24
`
`i Stream Encryption
`
`Supported only in conjunction with the Luxxon Player.
`
`
`
` -
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`0
`
`Existing streams can be modified/terminated via the WEBSITE.
`
`o A request can be made, via the WEBSITE, to switch the camera into Still Capture Mode.
`
`STILL CAPTURE MODE
`
`In Still Capture Mode, a given viewer can request that either
`
`0
`
`0
`
`an uncompressed still image (bitmap), or
`
`a JPEG compressed still image
`
`capture be executed and the data (bitmap or JPEG) be sent to the clients browser for display
`
`MONITOR MODE
`
`In Monitor Mode, the frames of the stream are checked to see if there are any notceable
`changes from frame to frame. Changes due to
`
`0
`
`0
`
`0
`
`camera vibrations,
`
`brightness, and
`
`random noise fluctuations,
`
`are deemed false and are ignored. True changes, such as the movement of an object in the
`field of view are recognized and an alarm page is sent to the viewer's browser. This page
`plays an alarm sound and flashes an alarm message in the browser. All monitor mode
`processing of the child stream is done on the Luxxon Gateway.
`
`The viewer initiates this mode from the WEBSITE. In so doing, the following information is
`supplied:
`
`0
`
`0
`
`flame rate
`
`recording length (see below).
`
`A new stream is created This stream terminates at the Gateway with the following
`characteristics:
`
`
`
`illiifi
`
`"iv"inu-"x; 14..
`
`
`
`5':
`
`""4"
`
`lift:«2112.1
`
`CHAmunERmnm”
`
`Resolution
`1 Color Space
`Color Depth
`
`l ’
`
`1 i
`
`Delta Frames sent as SJPEG data l
`
`Frame Rate
`
`l-5fps, specified by viewer but must meet bandwidth limitations
`
`i
`
`Format
`l _—
`
`.__._
`
`.
`
`. .___--....
`
`Confidential
`
`Pqu
`
`Page 10 of 24
`
`

`

`LUXXON Corporation
`Internet Camera: \fideo Producer Specifications
`
`When a true change is detected, the stream switches over to the highest possible stream
`characteristics supported by the bandwidth (between the camera and Gateway). This stream is
`recorded by the Gateway for a period specified by the viewer. A child stream is also sent
`from the Gateway to the viewer; this stream is set to the highest possible stream
`characteristics supported by the bandwidth between the Gateway and the viewer.
`
`INSTANT REPLAY MODE
`
`In Instant Replay Mode, each child stream from a camera which is sent to a Gateway is
`recorded for 1 minute length loops at the Gateway. Using this, the viewer can elect at any
`time to view the last one minute (or less) of video again and then continue viewing real time
`video. This mode is an analog of instant replay in TV broadcasts, except that each viewer
`now has control over when to active and watch the instant replay.
`
`CONFIGURATION
`
`SETUP MODE
`
`Once installation of the NP sofiware is complete, the user will run the IVP in Setup Mode.
`This will accomplish the following objectives:
`
`1.
`
`Interrogate and determine the make of the USB camera, validating that it is supported by
`this NP.
`
`2. Establish a connection to the Internet via the user’s ISP.
`
`3. Record the IP address assigned to the local machine.
`
`4. Open a TCP/IP socket to the IP addr. and port of the Camera Registrar which runs on the
`WEBSITE3.
`
`5. Post a request for General Registration to the WEBSITE.
`
`6.
`
`7.
`
`If request accepted, post a string containing the General Registration Information (TBD);
`else keep posting request until timeout
`
`Install a cookie, on the user's PC, which will automatically identify the user and camera
`to the Session Registrar which will register the camera ach time it connects for a
`streaming session and which runs on the WEBSIT'E
`
`8. Terminate.
`
`
`
`
`
`'h
`:‘1':
`
`"um
`
`353
`
`3 The WEBSITE is the site on the Internet through which all cameras running the IV? will register and
`make themselves available, etc. The Camera Registrar will run on this site maintaining the active
`registration, including logs, etc., of all cameras.
`
`Confidential
`
`Page 10
`
`Page 11 of 24
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications ’
`
`SESSION REGISTRATION MODE
`
`When a camera is to be used to unth or even single cast to the Internet. The user will nm
`the [VP in Streaming Mode. This will accomplish the following objectives:
`
`1.
`
`If the camera is already registered the process proceeds, else Setup Mode is entered.
`
`2. Establish a connection to the Internet via the user’s lSP.
`
`3. The cookie (see Setup Mode) will be used to identify the user and camera with the
`Session Registrar running on the WEBSITE.
`
`4. Record the IP address assigned to the local machine.
`
`5. Open a TCP/1P socket to the IP addr. and port of the Session Registrar which runs on the
`WEBSITE.
`
`6. Post a request for New Session to the Session Registrar.
`
`7.
`
`If request accepted, negotiate with the Session Registrar to define theW
`(maximum bandwidth, maximum resolutions, maximum frame rates, etc.)
`
`8. The Session Registrar will then create a web page for this camera linked to requests for
`that user's camera. This web page will include and link all supported stream
`characteristics.
`
`9. To verify that the connection is working, a 1 minute length of the supported test stream is
`sent to the Session Registrar. The bandwidth used by this test stream is compared to the
`expected bandwidth and any necessary changes to the session parameters are made.
`
`10. The final session parameters are assigned and recorded locally by the NP and remotely
`by the Session Registrar.
`
`11. Close the socket to the Session Registrar.
`
`12. Wait until the stream is started by the Session Manager running on the WEBSITE.
`
`
`
`SINGLE/UNICAST MODE
`
`When a viewer requests a child stream via the WEBSITE, the following actions are taken.
`
`1. The Session Manager checks to see if a session to that camera is currently open. Ifit is, it
`updates to parameters of the streams which can be sent from the camera assuming that
`the stream currently requested is invoked.
`
`2.
`
`3.
`
`Ifa Luxxon Player is being used by the viewer, then a TCP socket connection is made
`between the [VP and the Luxxon Player. If a Real or MS Media Player is being used by
`the viewer, then an HTPP connection is made between the [VP and the latter.
`
`If an RTSP connection is not established with the NP, 3 connection is made. The IVP is
`then instructed to deliver the stream requested.
`
`Confidential
`
`Page 11
`
`Page 12 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications ‘
`
`4. The IVP then streams the data to the viewer.
`
`5. A log entry is made on both the IVP machine and by the Session Manager.
`
`6. When the viewer terminates, the socket is closed and the to parameters of the streams
`which can now be sent from the camera is updated.
`
`STILL CAPTURE MODE
`
`Still Capture Mode only operates if there is an active stream being sent to that viewer and that
`viewer is the only viewer connected to the camera In this case,
`
`l. The viewer will request a still image (compressed or uncompressed) from the WEBSITE.
`
`2. The Session Manager will send a pausecommand to the NP.
`
`3. Once the IVP has acknowledged the pause command, the Session Manager will send a
`still capture (uncompressed/compressed) command to the IVP.
`
`4. The IVP will capture the still image and send the data via HTTP to the viewer's browser.
`
`5. The IVP will inform the Session Manager of the completion of the still capture.
`
`6. The Session Manager will instruct the IVP to resume the live stream.
`
`7. The IVP will continue streaming data to the viewer.
`
`MONITOR MODE
`
`This mode is does not directly affect the operations of the IVP. From the IVP's viewpoint, it
`receives instructions via one of the other modes.
`
`INSTANT REPLAY MODE
`
`This mode is does not directly affect the operations of the IVP. From the IVP's viewpoint, it
`receives instructions via one of the other modes.
`
`
`
`Confidential
`
`Page 12
`
`Page 13 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Wdeo Producer Specifications c
`
`ARCHITECTURE 8c FUNCTION
`
`HIGH LEVEL
`
`The IVP consists of the following major elements:
`
`El GUI
`
`0 Master Control
`
`0 Frame Buffer
`
`D Transcoder
`
`El
`
`Stream Caster
`
`CI Network Interface.
`
`Its input from the USB Camera driver is:
`
`0
`
`0
`
`sequential frames of DCT data in the case of video,
`
`single frame of SJPEG data or uncompressed YUV data in the case of still images.
`
`Its video output is a stream of IP frames each of which consists of a header and a packet of
`MPEG data. Each video IP frame is sent via a specific TCP/[P socket to a particular IP
`address.
`
`Its still image output is a bitmap or JPEG file sent via TCP/IP to the WEBSITE.
`
`The high level block diagram of the [VP is shown in Figure 1.
`
`
`
`.§ 2};
`
`
`
`M;-..“m”nah
`
`3::
`.9?
`
`Confidential
`
`Page 13
`
`Page 14 of 24
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`USB Camera
`
`
`
`
`
`Driver
` Stream Caster
`
`
`
`Figure I: High lebddlhgrmn 9fthe 1 VP.
`
`GUI
`
`There are two GUIs associated with the WP. The first, called the Local GUI, resides and runs
`on the same PC as the camera. The second, called the Remote Viewer GUI, is used by clients
`to obtain, modify and view streams at their remote locations; this GUI is for development and
`testing purposes only and not part of the product definition.
`
`LOCALGUI
`
`The camera operator/local user will interface to the [VP via a GUI application which is
`resident on the PC the camera is connected to.
`
`This GUI will serve four primary functions:
`
`1.
`
`camera setup,
`
`2.
`
`3.
`
`camera registran'on,
`
`camera control,
`
`4.
`
`stream monitoring.
`
`< Detailed Spec TBD: Peter>
`
`
`
`tEIEHEZZHu-it”)!iijii£I=
`
`IF"!-
`:unm
`‘1'"
`n“
`
`trun-
`
`Confidential
`
`Page 14
`
`Page 15 0f 24
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`REMOTE VIEWERGUI
`
`The remote viewer will interface to the NP via a web page resident on the WEBSITE the
`camera is registered at.
`
`This GUI will serve three primary fimctions:
`
`I.
`
`stream interaction,
`
`2.
`
`camera feedback
`
`3.
`
`activation of the plug-in display for the stream
`
`< Detailed Spec TBD: Peter, Rich, Roger >
`
`MASTER CONTROL
`
`The Master Control manages
`
`0
`
`0
`
`all transactions between the NP and external agents, and
`
`the processing of individual child streams.
`
`It interfaces to the USB driver to
`
`l.
`
`initiate and terminate streams from the camera,
`
`2.
`
`allows for programming of the driver (resolution, frame rate, exposure, etc), and
`
`3.
`
`cause the driver to place the DCT video data of a given input resolution and frame rate
`into the DCT Frame Buffer.
`
`It interfaces to the viewers and the WEBSITE via the Network Interface to the viewers and
`
`the WEBSITE. Based on commands it receives from the WEBSITE, it sets up and initiates a
`parent stream between the USB driver and the Transcoder. It also creates Stream Managers
`which are processes which control individual child streams. It communicates with these
`Stream Managers via a Stream Table which contains all of the stream's parameters.
`
`FRAME BUFFER
`
`Data is placed into the frame buffer by the USB driver. The type of data and rate at which the
`buffer is updated is programmed into the USB driver by the Master Control.
`
`For video, an input stream of DCI‘ data is read from the DCT Frame Buffer one frame at a
`time and sent to the Transcoder where it is turned into multiple child streams of either MPEG-
`l,-2, or —4 data which are then sent to the Network Interface for distribution via TCP/IP. The
`entire process is orchestrated by the Master Control unit.
`
`For still images, an input flame of either uncompressed YUV data or compressed SJPEG data
`is read from the Frame Buffer.
`
`
`
`
`
`
`tl’"hit"‘h"W“I"It2155b-ll..llILA?lie
`
`
`
`Confidential
`
`Page 15
`
`Page 16 of 24
`
`

`

`LUXXON Corporation
`Internet Camera: Video Producer Specifications
`
`TRANSCOD ER
`
`The block diagram of the transcoder is shown in Figure 2. Its job is to produce multiple
`MPEG-x streams at different resolutions, frame rates and color depths.
`
`
`
`
`
`Color Depth
`Reducer
`
`
`
`Figure 2: Trmm’er block diagram
`
`The fimction of the crapper is to extract a rectangular region of data from the Frame Buffer.
`The coordinates of the comers of this rectangle are specified in the Stream Table. This data is
`sent to the DCT Sub-sampler for further processing. Note that the cropper only functions in
`video mode.
`
`
`
`CROPPER
`
`g;
`
`
`DCT SUB-SAMI’LER
`
`The function of the DCT Sub-sampler is to down scale, in DCT space, the resolution of the
`input stream producing only streams with those resolutions which have been requested.
`
`If a given resolution stream from the DCT Sub-sampler is not to be split into multiple color
`depth streams, then it is automatically frame rate reduced before DCT Sub-sampling.
`Otherwise, it is not frame rate reduced before DCT sub-sampling but each sub-sampled
`stream is then sent to the Frame Rate Reducer and then onto the Color Depth Reducer.
`
`Confidential
`
`Page 16
`
`Page 17 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`FRAME RATE REDUCER
`
`The Frame Rate Reducer drops the appropriate frames in order to match the required outgoing
`frame rate. It maintains a running counter of which frames to send for each sneam.
`
`COLOR DEPTH REDUCER
`
`The Color Depth Reducer is a drastically simplified version of the Color Depth Reducer
`which lies in the gateway. Whereas the focus of the latter is primarily to deal with the color
`characteristics of diverse dISplay, the job of the CDR in the [VP is to reduce the density of
`data in the stream while making sure not to tax the CPU. Thus, in this first implementation,
`the CDR either passes the stream through untouched or simply sets the Cb and Cr channel
`data to 0 (or 128) thereby producing gray scale video.
`
`BI'I'RA'I'E CONTROL
`
`The bit rate control block finds the quantization scale factor which will most closely result in
`the current frame compressing to the desired number of bits. The latter is computed as
`follows:
`
`1. The number of bits for the previous frame is compared with the target number of bits per
`frame.
`
`2.
`
`If the number of bits is above the target, the scale factor for the stream is increased by a
`default value ('I‘BD). If the number of bits is below the target, the scale factor for the
`stream is decreased by the default value.
`
`3. The new scale factor is used as the base scale factor for the next frame and the process
`repeats.
`
`STREAM CASTER
`
`The function of the Stream Caster is to:
`
`1. Fragment the stream data, presumably by image frame, for transmission via TCP/1P.
`
`2. Create a header for each TCP frame to be sent. This header will include the destination
`
`IP address and payload length of the fragment being sent.
`
`3. Place the [P fragment data into the Network Interface for the associated TCP socket.
`
`4. Send the fragment.
`
`NETWORK INTERFACE
`
`The function of the Network Interface is to:
`
`1. Manage the creation, monitoring and destruction of all TCP/[P sockets.
`
`2. Manage the creation, monitoring and destruction of all RTSP connections.
`
`
`
`
`
`llIZill
`
`
`
`
`Confidential
`
`Page 18 of 24
`
`Page 17
`
`

`

`LUXXON Corporation
`Internet Camera: Wdeo Producer Specifications
`
`3. Send and receive data to/from the network via all TCP/IP sockets.
`
`4. Pass data received from the network to the Master Control for processing.
`
`MPEG-1,-2,-4 ENCODER
`
`The function of the MPEG-1,-2,-4 Encoder is to compress the DCl" data input to it into
`MPEG stream data. At a minimum this will be I-fiame data for MPEG-l,-2 streams.
`Depending on CPU capability and the available bandwidth, B & P frames will be included for
`MPBG-L-Z streams. For MPEG-4 data, P flames will be required.
`
`ENCRYPTOR
`
`The function of the Encryptor is to encrypt the data using a TBD methodology. This
`encryption is done only if the Luxxon Gateway is the immediate receiver of the data.
`
`
`
`F:
`
`".m'
`
`Confidential
`
`Page 18
`
`Page 19 of 24
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifications
`
`DESIGN SPECIFICATIONS
`
`MASTER CONTROL
`
`Input: WEBSITE HTTP command
`Output: Commands to the Stream Manager and USB Driver
`
`This block
`
`0 Directs Stream Managers via changing the stream parameters in the Stream Table.
`0 Manages the registration and dropping of displays and their associated RTSP
`connections.
`
`Initializes the Stream Table entry to be used by each Stream Manager.
`Monitors the bandwidths, source and display channels and sends changes to the stream
`registers used by Stream Managers.
`It is implemented as a state machine with the following states and events:
`
`
`TO STATE ;
`
`0M STATE
`
`TRANSITION EVENT
`
`
`
`EE__—{
`
`——-
`
`
`f
`
`
`
`
`:5.ll!!...3[in
`
`
`
`
`9-.
`
`
` im- streams” Re est
`
`
`m—-;
`
`Rumnn_
`1——
`
`.m Stream Chan_e Reuest Im-
`—_—;
`__nma-
`
`
`_i
`
`
`W_
`
`
`
`
`
`
`
`
`
`
`
`
`[_EVEN '
`
`_ESCRIPT0N
`
`
`
`3:1
`
`New Stream Re - uest
`
`Stream Com lete
`
`‘ Stream Pause Re-uest
`Stream Continue Re-uest
`Stream Sto- Re- est
`Stream Error
`‘ Stream S I o
`Stream Chan e Re- uest
`
`
`
`
`
`Internal Failure LI
`
`< Detailed Spec TBD: Avnish >
`
`Confidential
`
`Page 19
`
`Page 20 of 24
`
`
`
`
`
`
`

`

`LUXXON Corporation
`lntemet Camera: Video Producer Specifimtions
`
`STREAM MANAGER
`
`Input: Commands from the Master Control, DCT data from the Master Control
`Output: xPEG stream/frame to the appropriate Frame Buffer
`
`This block initiates, controls and terminates the processing of a child stream, given a
`parent stream and parameters defined in the Stream Table.
`
`Stream Managers are processes that are spawned by the Master Control. Before
`spawning a Stream Manager, the Master Control first sets up an entry table in the Stream
`Table containing all of the parameters that define and are needed for the processing of
`this stream.
`
`The Stream Manager is a state machine whose states and events are specified in the tables
`below.
`
`T0 STE
`TRANSITION EVENT
`: FROM STATE
`mam——
`—m—m-
`—_—-
`
`
`§”E_
`‘-—I_
`—_—5
`
`
`
`
`
`
`——_lur
`.
`,_I_'
`
`i-_ Internal Failure _i
`
`
`
`
`
`
`
`
`
`
`E’—D