~\
`
`PROVISIONAL
`APPLICATION
`NUMBER
`
`1 CL¥'-GJ Itt;·
`17 /t;s;.:;J
`UJU
`~f
`
`.......
`
`,·, .•
`
`······:
`
`~ .. '..
`
`.': .·
`
`Form PT0-1625
`(Rev. 6/99)
`
`(FACE)
`
`Exhibit 2004
`Bradium Technologies LLC - patent owner
`Microsoft Corporation - petitioner
`IPR2016-01897
`1
`
`

`
`PATENT APPLICATI,QN SERIAL NO .. _. - - - - - - - -
`
`U.S. DEPARTMENT OF COMMERCE ,
`PATENT AND TRADEMARK OFFICE
`FEE RECORD SHEET
`
`..
`
`PT0-1556
`(5/87)
`
`'U.S. GPO: 1999-459-082/19144
`
`2
`
`

`
`j
`
`Page I oU""'-....:-..
`
`UNITED .STATES PATENT AND ThADEMARK OFFICE
`
`COMMISSIONER FOR PAT::NTS
`UNITED STATES PATENT AND TRADEMARK OFFICE
`WASHING10N, D.C. 202.31
`www.uspto.gov
`
`CONFIRMATION NO. 3877
`
`CLASS
`
`GROUP ART UNIT
`
`ATTORNEY
`DOCKET NO.
`FLVT3001
`
`1111~111111111111 ~11111111 ~m 111111111111111111111111111111! 1111
`
`Bib Data Sheet
`
`SERIAL NUMBER
`60/258,489
`
`FILING DATE
`12/27/2000
`RULE
`
`~PPLICANTS
`Isaac Levanon, Ramat Hasham, ISRAEL;
`Yoni Lavi, Raanana, ISRAEL;
`
`** CONTINUING DATA*************************
`
`** FOREIGN APPLICATIONS ********************
`
`IF REQUIRED, FOREIGN FILING LICENSE GRANTED
`** 05/11/2001 .
`
`CJ yes CJ no
`Foreign Priority claimed
`~5 USC 119 (a-d) conditions CJ yes CJ no CJ Met after
`met
`Allowance
`Verified and
`Acknowledged
`ADDRESS
`23488
`
`Examiner's Signature
`
`Initials
`
`STATE OR
`COUNTRY
`ISRAEL
`
`SHEETS
`DRAWING
`5
`
`TOTAL
`CLAIMS
`
`INDEPENDENT
`CLAIMS ..
`
`TITLE
`Optimization of image parcel download sequence for fast quality build-up of image data streamed over limited
`and narrowband communication channels with different viewing frustum from a dynamic three-dimensional
`viewpoint
`
`FILING FEE FEES: Authority .has been given in Paper
`to charge/credit DEPOSIT ACCOUNT
`RECEIVED No.
`for following:
`No.
`150
`
`11:1 All Fees
`11:11.16 Fee~ (Filing)
`1:11.17 Fees (Processing Ext. of
`time)
`
`11:11.18 Fees (Issue)
`
`11:1 Other
`
`11:1 Credit
`
`I
`I
`
`I
`I
`I
`
`3
`
`

`
`IN T8t uNITED STATES PATENT AND TRADEMAKI\ OFFlCE
`REQUEST FOR FILING A PROVISIONAL APPLICATION FOR PATENT
`UNDER 37 CFR §1.53 (c)
`
`INVENTOR(S)
`
`ell
`
`1.
`
`Isaac
`
`2. Yoni
`
`Levan on
`
`La vi
`
`3 Nacho! Besor St., Ramat Hasharn, Israel
`
`21 Bar llan St., Raanana, Israel
`
`TITLE OF THE INVENTION
`
`OPTIMIZATION OF IMAGE PARCEL DOWNLOAD SEQUENCE FOR
`FAST QUALITY BUILD-UP OF IMAGE DATA STREAMED OVER LIMITED
`AND NARROWBAND COMMUNICATION CHANNELS WITH
`DIFFERENT VIEWING FRUSTUM FROM A DYNAMIC THREE-
`DIMENSIONAL VIEWPOINT
`_x_ Direct all correspondence to Customer Number 23488.
`
`Gerald B. Rosenberg, Esq. (Reg No.: 30,320)
`NewTechLaw
`285 Hamilton Avenue, Suite 520
`Palo Alto, California 94301
`
`Telephone:
`Facsimile:
`
`650.325.2100
`650.325.2107
`
`1111111111111111111111111111111111111111
`23488
`PATENT TRADEMARK OFFICE
`
`ENCLOSED APPLICATION PARTS (check all that apply)
`
`_ x_ Specification
`_ x_ Drawings
`
`No. of pages: _10 _
`
`No. of sheets: _5 _
`
`-- Declaration
`Other: Return-Recei12t Post Card.
`X
`
`-
`-
`-
`
`Small Entity Statement
`
`Power of Attorney
`
`Assignment and Cover Sheet
`
`~ 1ft
`~ b-
`p
`
`qo,
`c
`!D:'
`·0
`t':J
`
`..
`
`!..
`
`:: .,
`
`METHOD OF PAYMENT OF FILING FEES FOR THIS PROVISIONAL APPLICATION FOR PATENT
`
`Provisional Basic Filing Fee:$ 150.00 (Small Entity: $75.00)
`_x_ A check is enclosed to cover the Filing Fees.
`_x_ The Commissioner is hereby authorized charge Filing Fees or credit any
`overpayment to: Deposit Account Number: 50-0890.
`_x_ This invention was not made by or under contract with a US Government agency.
`
`Filing Fee Amount: $ 150.00
`
`US Government agency and Contract:
`
`Signature: ~Q~·~ r-(f'\A o SJ JO.rt-e. Date: December 26, 2000
`~Application Docket No:
`Express Mail Label No.:
`
`Gerald B. Rosenberg
`Reg. No.: 30,320
`
`FLVT3001
`
`EL 661 534 265 US
`
`Address To: Box Provisional Application, Assistant C~mmissioner for Patents, Washington, DC 20231
`
`gbrlflvt/3001.002.prov.xmittal.wpd
`
`4
`
`

`
`-1-
`
`OPTIMIZATION OF IMAGE PARCEL DOWNLOAD
`SEQUENCE FOR FAST QUALITY BUILD-UP OF
`IMAGE DATA STREAMED OVER LIMITED AND
`NARROWBAND COMMUNICATION CHANNELS
`WITH DIFFERENT VIEWING FRUSTUM FROM A
`DYNAMIC THREE-DIMENSIONAL VIEWPOINT
`
`Inventors:
`Isaac Levanon
`Yoni Lavi
`
`Background of the Invention
`
`The present invention is generally related to the delivery of high-resolution
`
`highly featured graphic images over limited and narrowband communications
`
`. channels.
`
`Summary of the Invention
`
`The objective is to display a two-dim!;3nsional pixel map,· a 16-Bit RGB color
`
`image in the preferred embodiments, of very large dimensions and permitting the
`
`viewing ofthe image from a dynamic three-dimensional viewpoint. Multiple such
`
`Attorney Docket No.: FL VT300 1
`gbr/flvt/3001 .OOO.provisionaLwpd
`
`12/26/2000
`
`j
`
`1
`.2
`3
`4
`5
`6
`7
`8
`9
`1 0
`1 1
`1 2
`1 3
`1 4
`1 5
`1 6
`1 7
`
`1 8
`
`1 9
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`26
`
`·=
`~g
`n o:;::r;::o
`r1~~
`tr~
`::=:::;
`~ ~ ::
`r.-~
`~tq
`
`~I:"':
`
`- ~
`
`:
`U)
`--
`
`·-.:.~~
`
`;-:
`::;:.,..;:
`:=~
`S:;;:'l
`
`5
`
`

`
`1
`2
`3
`4
`5
`6
`
`7
`
`8
`
`9
`1 0
`1 1
`1 2
`1 3
`1 4
`1 5
`1 6
`1 7
`1 8
`1 9
`20
`21
`22
`23
`.24
`25
`
`.:::.u:.
`:..:::
`;::::
`
`CJ
`
`~~
`~ =::::,
`~l
`E}3
`r--=
`'T= w
`
`~n
`
`-=.::..:::::
`
`;::
`~
`n~
`E~
`
`r~
`~rJ.
`CJ
`g
`
`-2-
`
`images are remotely hosted for on-demand ·selection and transfer to a client
`
`system for viewing.
`
`Images, as stored by the server, may individually range from gigabytes to
`
`multiple terabyte in total size. A correspondingly large server storage and
`
`processing system is contemplated. Conversely, client systems are contemplated
`
`to be conventional personal computer systems and, in particular, mobile, cellular,
`
`embedded, and handheld computer systems, such as personal digital assistants
`
`(PDAs) and internet-capable digital phones, with relatively limited to highly
`
`constrained network communications capabilities. For most wireless applications,
`
`conventional narrowband communications links have a bandwidth of less than
`
`approximately three kilobytes of data per second. Consequently, transmittal of
`
`entire images to a client system in reasonable time is infeasible as a practical
`
`matter.
`
`Overview:
`
`Description of the Invention
`
`For purposes of the present invention, each image (Figure 1) is at Jeast
`
`logically defined in terms of multiple grids of image parcels with various levels of
`
`resolutions (Figure 2) that are created through composition of information from
`
`all level of resolutions, and stor.ed by th~ server to provide an image for transfer
`
`to a client system (Figure 3). Composed and separate static and dynamically
`
`created layers are transferred to client system in parcels in a p·rogram selectable
`
`order to pptimize for fast quality build-up of the image presented to a user of the
`
`client system, particularly when the parcels are streamed over a narrowband
`
`..
`
`communication link.
`
`Attorney Docket No.: FL VT300 1
`gbr/fl¥t/300l.OOO.provisional.wpd
`
`12/26/2000
`
`6
`
`

`
`-3-
`
`The multiple layers of an 1mage allow the selectivity to incorporate
`
`· topographical, geographical, orientational, ·and other terrain and mapping
`
`related information into the image delivered. Other layers, such as geographic
`
`grids, graphical text overlays, and hyperlink selection· areas, separately provided
`
`or composed, aid in the useful presentation and navigation of the image as
`
`presented by the client system and viewed by the user.
`
`Com positing of layers on the server enables the data transfer burden to be
`
`reduced, particularly in analysis of the requirements and capabilities of the client
`
`system and the connecting communications link. Separate transfer of layers to the
`
`client system allows the client system selectivity in managing and presentation of
`
`the data to the user.
`
`The system and methods of 'the present invention are designed to, on
`
`demand, select, process and immediately transfer data parcels to the client
`
`system, which immediately processes and displays a low-detail representation of
`
`the image requested by the client system. The system and.methods immediately
`
`continue to select, process and sequentially transfer data parcels that, in turn, are
`
`processed and displayed by the client system to augment the presented image
`
`..
`
`and thereby provide a continuously improving image to the user.
`
`Selection of the sequentially transferred data is, in part, dependent on the
`
`progressive transl.ation of the three-dimensional viewpoint as dynamically
`
`modified on the client system during the transfer process. This achieves the
`
`above-stated objective while concurrently achieving a good rendering quality for
`
`continuous fly-over of the image as fast as possible, yet continuously building the
`
`image quality to the highest resolution of the image as stored by the server.
`
`1
`2
`3
`4
`5
`6
`7
`8
`9
`10
`1 1
`1 2
`1 3
`14
`15
`16
`1 7
`18
`1 9
`20
`21
`22
`23
`24
`
`~,
`
`b~
`fJ
`
`f]j
`F
`-;-··
`~~}
`¥5
`=
`~-:5:
`~u
`
`~J
`~ ·=
`
`Attorney Docket No.: FLVr3001
`gbr/flvt/300l.OOO.provisional.wpd
`
`12/26/2000
`
`7
`
`

`
`-4-
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`1 1
`
`12
`
`13
`
`14
`
`15
`
`16
`
`1 7
`
`18
`
`1 9
`
`20
`
`21
`
`22
`
`23
`
`24
`
`To optimize image quality build-up over limited and narrowband
`
`communication links, the target image, as requested by the client system, is
`
`represented by multiple grids of 64x64 image pixels (Figure 4) with each grid
`
`having some corresponding level of d~tail. That is, each grid is treated as a
`
`sparse data array that can be progressively revised to increase the resolution of
`
`the grid and thereby the leve.l of detail presented by the grid. The reason for
`
`choosing the 64x64 pixel dimension is that, using current image compression
`
`· algorithms, a 16-bit 64x64 pixel array irnage can be presented as a 2KByte data
`
`parcel.
`
`In turn, this 2KByte parcel is the optimal size, subject to conventional
`
`protocol and overhead requirements, to be transmitted through a 3KByte per
`
`second narrowband transmission channel. Using a smaller image array, such as
`
`32x32, would create a 0.5KByt.e parcel, hence causing inefficiencies due to packet
`
`..
`
`transmission overhead, given the nature of current wireless communications
`
`protocols.
`
`Image array dimensions are preferably powers of two so that they can be
`
`used in texture mapping efficiently. Each parcel, as received by the client system,
`
`is preferably immediately processed and incorporated into the presented image.
`
`To do so efficiently, according to the present invention, each data parcel is
`
`independently processable by the client system, which is enabled by the selection
`
`and server-side processing used to prepare a parcel for transmission. In addition,
`
`each data parcel is sized appropriate to fit within the level- 1 cache, or equivalent,
`
`of the client system processor, thereby enable the data processing intensive
`
`operations needed to process the data pa'rcel to be performed without extended
`
`. memory access delays.
`
`In the preferred embodiment of the present invention, ·
`
`~§
`r..;::. b;:
`=:J::
`~ ~~
`~-~
`
`~:;
`~X~
`=JJ
`
`:.::
`~b
`
`~~
`
`;_::::::;~
`:::..-=:1
`
`Attorney Docket No.: FLVT3001
`gbr/flvt/300l.OOO.provisional.wpd
`
`12/26/2000
`
`8
`
`

`
`-5-
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`data parcels are also processed for texture mapping and other image features,
`
`such as topographical detailing.
`
`Currently, with regard to conventional client systems, a larger image array,·
`
`such as 128x128, is too large to be fully placed within the Ievei-l cache of many
`
`ofthe smaller conventional current processors, such as used by personal digital
`
`assistants {PDAs) and cellular phones. Since· access to cache memory 1s
`.
`.
`substantially faster than to RAM this will likely result in lower frame rate.
`
`Different and larger data parcel sizes may be optimal as transmission
`
`protocols and micro-architectures of the client computers change. For purposes
`
`ab?ve, the data content was a pixel array representing image data. Where the
`
`data parcel content is vector, text or other data that may subject to different client
`
`system design factors, other parcel sizes may be used.
`
`In the process implemented by the present invention, data parcels maybe
`
`selected for sequential transmission based on a prioritization of the importance
`
`of the data contained. The criteria of importance maybe defined as suitable for
`
`particular applications and may directly relate to the presentation ·of image
`
`quality, provision of a textual overlay of a low-quality image to quickly provide a
`
`navigational orientation, or the addition of topography information at a rate or
`
`timing different from the rate of image quality improvement. Thus, image data
`
`layers reflecting l)avigational cues, text overlays, and topography can be
`
`composed into .data packets for transmission subject to prioritizations set by the
`
`server alone, based on the nature and type of the client system, and interactively
`
`influenced by the actions and commands provided by the user ofthe client system
`
`(Figure 5).
`
`..
`
`Attorney Docket No.: FLVT3001
`gbr/flvt/3001 .OOO.provisional.wpd
`
`12/26/2000
`
`9
`
`

`
`.-6-
`
`Progressive transmission of image parcels is performed in an iterative
`
`process involving selection of an image data grid within the target image of the
`
`client system, which is a portion of a potentially multi-layered source image stored
`
`by the server. The selection parameters are preferably dependent on the client
`
`navigation viewpoint, effective velocity, and height, and the effective level of detail
`
`currently presented in each grid. Once a grid is selected, the serv~r selects the
`
`source data to be logically composed into the selected grid to complement the
`
`effective resolution of that grid, processing the grid data to produce the optimally
`
`sized size grid data parcels, and sequentially transmitting the parcels to the client
`
`system. Preferably, the detail of a grid array is sequentially enhanced by division
`
`of the grid into sub-grids related by a power of two (Figure 6). Thus, a given grid.
`
`is preferably updated using four data parcels having twice the data resolution of
`
`..
`
`the existing grid. Whatever number of parcels are used, each data parcel is
`
`rendered by the client system into the target image. Additional client system
`
`image data processing to provide texturing and three-dimensional representation
`
`of the data may be performed as part of the parcel rendering and integration into
`
`the target image.
`
`Image Parcel Download Sequence:
`
`The ser.ier of the present invention supports the download of parcel data
`
`to a client system by providing data parcels in response to network requests
`
`originated by client systems. Each requested data parcel is identified within a grid
`
`coordinate system relative to an image stored by the server.
`
`A client system implementing the process of the present invention is
`
`responsible for identifying and requesting parcel data, then rendering the parcel
`
`Attorney Docket No.: FL VT300 1
`gbr/flvt/300 l.OOO.provisional.wpd
`
`12/26/2000
`
`~:::: ::
`
`rra
`~d
`r~~
`
`... ~.:=-M
`~-:-\:.g
`~~;
`rr:~
`':'::":::'
`
`1
`2
`3
`4
`5
`6
`7
`8
`9
`10
`1 1
`12
`13
`~ 14
`15
`16
`1 7
`18
`19
`20
`21
`22
`23
`24
`25
`
`-~
`
`~f::
`
`~~
`bl
`t:J
`
`10
`
`

`
`-7-
`
`data into the target image at the correct locatio11. The client system is also
`
`responsible for managing navigational and other interaction with the user.
`
`In
`
`identifying the parcel data to be requested, the client system operates to select
`
`grids within the coordinate system, corresponding to portions ofthe target image,
`
`for which to request a corresponding data parcel. The requests are issued over
`
`the network to the server and rendering performed asynchronously as data
`
`parcels are received·. The order of data parcel requests is defined as a sequence
`
`that will provide for the optimal build-up of the target image as presented to the
`
`user. The rate of optimal build up of the target image is dependent on the nature
`
`ofthe target image requested, such as the supported parcel size and depth ofthe
`
`target image that can be rendered by the client system.
`
`The client identifies and requests the download of data parcels in the
`process as follows. Denote the target image as 10 and its size in pixels as (X, Y).
`Let N be the smallest power of 2 that is equal or greater than max {X,Y}.
`
`..
`
`Construct the grid of 64x64 pixel grid-images lo,i,i that together compose the
`target image 10• The rectangle [64i,64i + 64] x [64j,64 j + 64] of 10 i~ mapped
`to 10 . ..
`,1,1
`
`In order to view a large portion of the image, the target image, without
`
`downloading the substantial bulk of the target image, mip-maps of 10 are created,
`representing a collection of images to be used as surface textures when rendering
`
`a two-dimensional representation of a three-dimensional scene, and which are
`
`defined recursively as:
`
`lk+l (i,j} = avg(lk (2i,2j}, lk(2i + 1,2j}, lk(2i,2j + 1 }, lk(2i + 1,2j + 1 ))
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`1 1
`
`12
`
`13
`
`14
`
`15
`
`1 6
`
`1 7
`
`1 8
`
`1 9
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`~g
`[J
`::::~ a;J
`=~
`~~
`::::.:~
`~
`
`··~
`
`:?'-:~
`~=i
`:J:~
`"'
`
`~~ ~ n ::~f
`
`~J. =
`
`Attorney Docket No.: FLVT3001
`gbr/flvt/3001 .OOO.provisional.w.pd
`
`12/26/2000.
`
`11
`
`

`
`~n w
`
`~u
`Lf~
`~~
`::=
`
`~~ ro
`:::.f1 =
`" ~~~
`
`~==
`at,~
`i:=:~
`; ~~
`:~~~
`F~ ·=::
`.::::::=.
`~J
`
`-8-
`
`Such mip-maps are created up to IMtM = log 2 (N) - 6. At this point, IM is
`a 64x64 image containing the enti.re area of the original image, hence no further
`
`mip-inapping is required.
`
`The methods of the present invention then proceed by constructing the
`
`respective grids or cells (lk,i,i) for each mip-map. Each nonempty image cell lk,i,i
`
`now may be downloaded. Larger values of k cover more area within the original
`image but provide lower detail on that area. The task at hand is n~w to
`
`determine, given the view_ing frustum and the list of previously downloaded image
`
`cells lk,i,i' downloading w~ich grids will improve the quality. of the display as fast
`
`as possible, considering the download rate as fixed. The scheme used to
`
`implemented the downloading sequence of these cells is by constructing a tree,
`
`starting from IN-6,0,0 and expanding a quadtree towards the lower mip-map levels.
`(Quadtrees are data structures in which each node can have up to four child
`
`..
`
`nodes. As each 64x64 pixel image in the grid 1 k has exactly four matching 64x64
`pixel images on the grid 1 k- 1 covering the same area, the data structure is built
`accordingly.)
`
`For every frame that is rendered , begin with the cell that covers the area
`
`of the entire original image, IN-6,0,0• For each cell under consideration, compute
`the principle mip-map level that should be used to draw it. If it is lower than the
`
`mip-map level of the cell, subdivide the cell to four smaller cells and use
`
`recursion.
`
`If this operation attempts to draw over areas that do not yet have
`
`image cells at a low enough mip-map level to use with them, the recursion stops.
`
`If the principle mip-map level is equal or higher than the level of the cell,
`
`then the cell is rendered using the cell ofthe principle mip-map level, which is the
`
`parent of that cell in the Quad-tree, at the appropriate level. Then download th'e
`
`1
`
`2
`
`3
`
`4
`
`5
`6
`
`7
`
`8
`
`9
`
`1 0
`1 1
`1 2
`
`1 3
`
`14
`
`1 5
`1 6
`
`1 7
`1 8
`
`1 9
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Attorney Docket No.: FLVT3001
`gbr/flvt/300l.OOO.provisional.wpd
`
`12/26/2000
`
`12
`
`

`
`ffi
`:::=:::
`
`~~
`~¥
`~J
`
`~
`
`1
`2
`3
`4
`5
`6
`7
`8
`9
`~r~
`~ 10
`1 1
`UH
`12
`13
`14
`15
`16
`1 7
`18
`19
`20
`21
`22
`23
`24
`25
`
`:::::1!
`P.J
`
`Q
`b.~
`
`-9-
`
`cells in which the difference between the principle mip-map level to the mip-map
`
`level of the image cell actually used is the highest. Downloading is asynchronous;
`
`the renderer maintains a prioritY queue of download requests, and s·eparate
`
`threads are downloading imag.es. Whenever a download is complete, another
`
`download is initiated im111ediately, based on the currently highest-priority request.
`
`The principle mip-map level of an image cell is determined by the screen
`
`resolution, FOV (field of view) angle, the angle formed between the image's plane
`
`normal and the line connecting between the camera and the position within the
`
`cell that is closest to the camera, and a few other factors. The equation, which
`
`uses the above information, approximates the general mip-mapping level
`
`equation:
`
`I = max(O, log4 (T/S))
`
`..
`
`in which S is the surface of the cell as displayed on the screen during rendering
`
`(in pixels), and T is the surface of the cell within the texture being mapped (in
`
`pixels).
`
`When rendering a cell of the grid lk,
`
`and
`
`S = xycos(a)ctg2(0.5FOV)t2 T I z2
`
`I
`
`where xis the display's x-resolution, y is the display's y-resolution, FOV is the field(cid:173)
`
`of-view ·angle, a is the angle between the image's plane normal and the line
`
`Attorney Docket No.: FL VT300 1
`gbr/flvt/300l.OOO.provisional.wpd
`
`12/26!2000
`
`-------- -------
`
`13
`
`

`
`-10-
`
`connecting the viewpoint and the point in. the cell of shortest distance to it, tis the
`
`len·gth of the square each pixel in the original image is assigned to in 3D, and z
`
`is the height of the camera over the image's plane.
`
`This arrives at the equation:
`I = log4 {z2 /(xycos(a)ctg 2(0.5FOV)f))
`I = max(O, min(l, M))
`
`For example, using a 64x64 target grid display to render the image from a view
`
`of height N with FOV angle of 90 degrees, with the length of each pixel in space
`
`being one, the entire target· image can be fitted precisely to the display as
`
`demonstrated by:
`
`..
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`n =
`
`Attorney Docket No.: FLVT3001
`gbr/flvt/3001.000.provisional.wpd
`
`12/26/2000
`
`14
`
`

`
`li:::!l n:::n ,~;::: ;i:::!! ;1:::!! ··n::· Ill n:::u ij;::1: "h :EI: ~!:::H ;;::::: !l:::ll'!:::n
`
`/
`'1
`
`Graphicallayerfortext
`
`,
`
`'
`
`2D and 3D elements
`
`CeiiiD, G PS, Etc. ·
`
`• Perspective
`3D with six
`degrees of
`freedom
`
`• 2D in Mobile
`Terminals
`
`Geograph~
`Grid
`
`Unlimited MB
`Unlimited oixels
`
`~UKQ3
`
`'F ~ L\rt.t r .
`
`......
`
`.............
`Ctt
`
`·,
`
`_)
`
`15
`
`

`
`. z/s
`
`.. --\
`i
`
`M«W.
`
`16
`
`n.
`_
`
`E:_
`
`3 T
`
`awe
`
`EM3:.-
`...=_.mu
`MWA_
`
`«S.:“W
`:2%..
`.._..mm
`_w
`«
`1.
`
`“WWm...”
`
`16
`
`
`
`
`
`

`
`64
`
`64
`
`64
`
`64
`
`64
`
`64
`
`64
`
`64
`
`FIG. 4
`
`..
`
`17
`
`

`
`,...
`
`:::;:=:: awEmmE.“mm3%mgmpmmmm“Ma.SS
`
`18
`
`

`
`I.
`
`E
`
`E
`
`§§§§§;§§a%§E
`
`ifiii
`
`19
`
`19
`
`

`
`· ... ·
`
`PTO/SB/68 (02-10)
`Approved for use through 07131/2012. OMS 0651-0031
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`nd to a collection of information unless it dis Ia sa valid OMS control number.
`Under the Pa
`rwork Reduction Act of 1995, no
`REQUEST FOR ACCESS TO AN ABANDONED APPLICATION UND~R 37 CFR 1.14
`
`Bring completed form to:
`File Information Unit, Suite 3A20
`2800 South ·Randolph Street
`Arlington, VA 22206
`
`~elephone: (703) 756-1800
`
`In re Application of
`
`I hereby request access under 37 CFR'1.14(a}(1 ){iv) to the application file record of the above-identified ABANDONED
`application, which is not within the file jacket of a pending Continued Prosecution Application {CPA) (37 CFR 1.53(d)) and
`which is identified in, or to which a bene~it is claimeq, in the 'following document {a~ shown in the attachment):
`'.
`United States Patent Application Publication No. - - - - - - - - - - - ' page, ____ line _____ ,
`
`United States Patent Number ] / 3 7 7 7 L(
`
`... h--=tt-=t!c:.<-=---· line,
`WIPO Pub. No. ___________ ,page _____ ,line
`
`, column
`
`Related Information About Access to Applications Maintained in the Image File ·
`Wrapper System (IFW) ~nd Access to Pending Applications in General
`A. member of the public, acting without a power to inspect, ·cannot order applications maintained. in the IFW system through
`the FlU. If the member of the pllblic is entitled to a ccipy of the application file, then the file is made available through the
`Public'P.atent Application Information Retrieval system (Public PAIR) on the L!SPTO internet web site (www.uspto.gov).
`Terminals that allow access to Public PAIR are available in the Public Search Room. The member of the public may also
`· be entitled to obtain a copy of all or part of the application file upon payment of the appropriate fee. Such copies must be
`·
`purchased through the Office of Public Records upon payment of the appropriate fee (37 CFR 1.19(b)).
`For published applications that are still pending,' a member of the public may obtain a copy of:
`·the file contents; the pending application as· originally filed; or any document in the file of the pending application.
`,,
`For unpublished applications that are still pending:
`(1)
`If the benefit of the pending application is claimed under 35 U.S.C. 119(e), 120, 121, or 365 in another application
`that has: (a) issued as a U.S. patent, or (b) published as a statutory invention registration, a U.S. patent' .
`application publication; or an international patent application publicatio.n in accordance with PCT Article 21 (2), a
`member of the public may obtain a copy of: the file cont~nts; the pending application as originally filed; ·or any
`document in the file of the pending application.
`If the application is incorporated by reference or otherwise identified in a U.S. patent,-a statutory invention
`registration, a U.S. patent application publication, ·or an international patent application publication in accordance
`with PCT Article 21 (2), a member of the public may obtain a copy of the pending application as originally filed.
`
`(2)
`
`..
`
`lfdi cJ,a~l 1J . h, f!-ftJYJ
`
`Signatuf]
`
`,
`
`·
`
`.
`
`Typed of printed name
`
`Registration Number, if applicable
`
`7o 3 -· ss-3-· (j-(7Cf()
`
`Telephone Number
`
`•·
`
`Approved by:
`
`Unit:
`
`This collection of information is required by 37 CFR 1,11 and 1.14. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to
`process) an application. Confidentiality is governed by 35 U.S.C. 122 and ';>7 CFR 1.11 and 1.14. This collection is estimated to take 12 minutes to complete, including
`gathering, preparing, and submitling the completed application fBrm to the USPTO. 'Time will vary depending upon the individual case. Any comments on the amount of time
`you require to complete this form and/or suggestions for redudng this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S.
`Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. BRING TO: File
`Information Unit, Suite 3A20, 2800 South Randolph Street, Arlington, Virginia.
`
`If you need assistance in completing the form, call 1-800-PT0-9199 and select option 2.
`
`20
`
`

`
`\
`lllllllllllllllllllllllllllllllllllb lllllllllllllllllllllllll~ llllllllllll
`
`US007139794B2
`
`c12) United States Patent
`Levanon et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,139,794 B2
`Nov. 21, 2006
`
`(54) SYSTEM AND METHODS FOR NETWORK
`IMAGE DELIVERY WITH DYNAMIC
`VIEWING FRUSTUM OPTIMIZED FOR
`LIMITED BANDWIDTH COMMUNICATION
`CHANNELS
`
`(75)
`
`Inventors: Isaac Levanon, Ramat Hasham (IL);
`Yoni Lavi, Raanana (lL)
`
`(73) Assignee: 3-D-V-U Israel (2000) Ltd., Rannana
`(IL)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 918 days.
`
`(58) Field of Classification Search ................ 709/203,
`709/246-247, 217-219; 345/418-420, 581,
`345/619; 382/103, 232, 235, 240
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,698,689 A ,.
`5,966,672 A ,.
`6,182,114 B1*
`6,397,259 B1.,
`6,671,424 B1 *
`6,704,024 B1 *
`6,874,012 Bl*
`
`10/1987 Tzou ..................... 3581426.14
`1011999 Knupp ........................ 702116
`1/2001 Yap et al. ................... 709/203
`5/2002 Lincke et al. . ... ... .. . . . .. . 709/236
`1212003 Skoll et al. ................. 382/305
`3/2004 Robotham et al ........... 345/581
`3/2005 St. Pierre ................... 709/207
`
`(21) Appl.. No.: 10/035,981,
`
`(22) Filed:
`
`Dec. 24, 2001
`
`"' cited by examiner
`
`Primary Examiner-Philip B. Tran
`
`(57)
`
`ABSTRACT
`
`(65)
`
`Prior Publication Data
`
`US 2002/0118224 A1
`
`Aug. 29,2002
`
`(60)
`
`Related U.S. Application Data
`
`Provisional application No. 60/258,488, filed on Dec.
`27, 2000, provisional application No. 60/258,489,
`filed on Dec. 27, 2000, provisional application No.
`60/258,465, filed on Dec. 27, 2000, provisional appli(cid:173)
`cation No. 60/258,468, filed on Dec. 27, 2000, pro(cid:173)
`visional application No. 60/258,466, filed on Dec. 27,
`2000, provisional application No. 60/258,467, filed
`on Dec. 27, 2000.
`'
`
`(51)
`
`(52)
`
`Int. Cl.
`G06F 15116
`(2006.01)
`U.S. CI. ...................... 709/203; 709/246; 709/217;
`709/219; 345/420; 345/581; 382/232; 382/235
`
`Dynamic visualization of image data provided through a
`·network communications channel is performed by a client
`system including a parcel request subsystem and a parcel
`rendering subsystem. The parcel request subsystem includes
`a parcel request queue and is operative to request discrete
`image data parcels in a priority order and to store received
`image data parcels in a parcel data store. The parcel request
`subsystem is responsive to an image parcel request of
`assigned priority to place the image parcel request in the
`parcel request queue ordered in correspondence with the
`assigned priority. The parcel rendering subsystem is coupled
`to the parcel data store to selectively retrieve and render
`received image data parcels to a display memory. The parcel
`rendering system provides the parcel request subsystem with
`the image parcel request of the assigned priority.
`
`2 Claims, 5 Drawing Sheets
`
`..
`
`42
`
`PARCEL DATA
`RECEIVE/REQUEST
`
`NEtWORK
`
`52
`
`PRIORITY
`REQUEST
`QUEUE
`
`DISPLAY
`
`48
`
`PARCEL DATA
`RENOERING
`
`r·-;,
`
`50
`
`NAVIGATION COMMANDS
`
`VIEWING fRUSlUM
`CONTROL
`
`"-40
`
`21
`
`

`
`POSITION
`
`INITIALS
`
`..
`
`IDNO.
`
`FEE DETERMINATION
`O.I.P.E. CLASSIFIER
`FORMALITY REVIEW
`RESPONSE FORMALITY REVIEW
`
`. .,.-..._.-)
`C"'··~·
`~7
`
`~
`
`I _(Jdf1
`
`DATE
`
`.
`r:;;z I J )O .l
`
`..
`
`(LEFT INSIDE)
`
`22
`
`

`
`' ·. J
`
`I
`
`..
`
`.... :,. '< ·.,
`:
`'
`
`,, .
`
`.
`
`•
`
`. ,:
`-· ·~ . :
`.
`
`.
`
`(---. ·-'-···PATEN'f·-·APP~LICATION
`
`.
`
`.
`
`.
`
`I 1\\11\\1111 \\II\ II\\\ \11\11\111 \I\\ I 1\11\1\111\\1
`60258489
`---·----'-- -----~
`
`\
`\
`I
`\...______. ·--~-----------
`Date
`Entered
`or
`Counted
`
`CONTENTS
`
`. Date
`· Received ·
`·· ..
`.· .. · ,: or
`Mailed
`
`'
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`10.
`
`11.
`
`12.
`
`13.
`
`14.
`
`15.
`
`16.
`
`17.
`
`18.
`
`19.
`
`20.
`
`21.
`
`22.
`
`23.
`
`24.
`
`25.
`
`26.
`
`27.
`
`28.
`
`29.
`
`30.
`
`31.
`
`32.
`
`. ' ............
`
`.'•
`
`:--:-·
`.. ,--
`
`(LEFT OUTSIDE)
`
`23