U5006496189B1
`
`
`
`
`
`
`
`
`US 6,496,189 B1
`(10) Patent N0.:
`(12) Umted States Patent
`
`
`
`
`
`
`
`
`
`Yaron et al.
`(45) Date of Patent:
`Dec. 17, 2002
`
`
`
`
`
`
`
`
`
`(54) REMOTE LANDSCAPE DISPLAY AND
`PILOT TRAINING
`
`
`
`
`
`(75)
`
`
`
`
`( * ) Notice:
`
`
`
`
`
`
`
`
`
`Inventors: Ronnie Yaron, Kadima; Ofer ShOI‘, Tel
`
`
`
`AVIv, both of (IL)
`
`
`
`
`
`
`(73) Assignee: Skyline Software Systems Ltd., Givat
`
`
`Shmuel (IL)
`
`
`
`
`
`
`Subject to any disclaimer, the term Of this
`patent is extended or adjusted under 35
`
`
`
`
`
`
`
`U.S.C. 154(b) by 0 days.
`
`
`
`
`
`(63)
`
`
`
`
`
`
`
`(21) Appl. No.: 09/258,663
`
`
`
`
`
`Flled:
`Feb. 26’ 1999
`(22)
`-
`-
`t Dt
`thdU.S.A 1
`
`
`
`
`
`
`
`
`
`pp lca 10n
`e a e
`a a
`
`
`
`
`
`Continuation—in—part of application No. 08/939,948, filed on
`Sep. 29, 1997, now Pat. No. 6,111,583.
`
`
`
`
`
`
`
`
`
`
`
`
`(51)
`Int. Cl.7
`G06T 17/00
`
`
`
`
`(52) U S Cl
`iiiiiiiiiiiiiiiiiiiiiiiiii345/428345/582 345/419
`f Siiiiiiiihiiiiiiiiiiii
`345/419 582
`F'- 1d
`58
`’
`
`
`
`
`
`
`
`
`
`0 3227:28342/364435701 456‘ 701/207’
`(
`)
`1e
`
`
`
`
`
`
`
`’
`’
`’
`’
`’
`’
`
`
`
`
`
`
`
`
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`
`
`......... 364/200
`4,070,705 A *
`1/1978 Lockwood et al.
`
`
`
`
`
`
`4,359,733 A * 11/1982 O’Neill
`.................... 342/36
`
`
`
`
`
`
`4,940,972 A
`7/1990 Mouchot et al.
`..
`340/747
`
`
`
`
`
`
`
`
`5,550,959 A
`8/1996 Freeman ...........
`395/127
`
`
`
`
`
`
`
`5,566,073 A
`10/1996 Margolin ............. 364/449
`
`
`
`
`
`5,652,863 A *
`7/1997 Asenio et al.
`......... 395/497.04
`
`
`
`
`
`
`
`(56)
`
`
`
`
`
`
`
`
`
`
`
`
`
`.............. 345/430
`6/1998 Migdal et al.
`5,760,783 A *
`8/1998 Viebahn et al. .......... 340/974
`5,798,713 A *
`
`
`
`
`
`
`3/1999 Migdal et al.
`........... 345/423
`5,886,702 A *
`
`
`
`
`
`
`
`
`
`
`8/2000 Reber et al.
`................ 345/327
`6,111,568 A *
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`“ZBuffer,” Caroline Jelinek, Mar. 24, 1996—Materials col-
`
`
`
`
`
`
`
`lected from the World Wide Web—Addressed http://www.c-
`
`s.unc.edu/~bryan/bsg/TechMan/Zbufferhtml.
`
`
`
`
`
`
`
`“1.1 Photo—realistic Image Synthesis,” Gary Shea, May 27,
`1995—Materials collected from the World Wide Web—
`
`
`
`
`
`
`
`
`
`http://www.xmission.com/~shea/thl/
`Addressed:
`node5 .html#Section0041.
`
`
`
`
`
`* cited by examiner
`
`
`
`
`Primary Examiner—Mark Zimmerman
`Assistant Examiner—Lance W. Seale
`Y
`
`
`
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—Duane Morris LLP
`
`
`ABSTRACT
`
`(57)
`
`
`
`
`
`
`
`
`
`
`
`A method of providing data blocks describing three-
`
`
`
`
`
`
`
`dimensional terrain to a renderer. The data blocks belong to
`a hierarchical structure which includes blocks at a plurality
`
`
`
`
`
`
`
`of different resolution levels. The method includes receiving
`
`
`
`
`
`
`
`
`from the renderer one or more coordinates in the terrain
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`along with indication of a respective resolution level, pro-
`
`
`
`
`
`
`
`
`
`viding the renderer with a first data block which includes
`
`
`
`
`
`
`
`data corresponding to the one or more coordinates, from a
`
`
`
`
`
`
`
`
`local memory, and downloading from a remote server one or
`more additional data blocks which include data correspond-
`
`
`
`
`
`
`
`
`ing to the one or more coordinates if the provided block from
`
`
`
`
`
`
`
`
`
`the local memory is not at the indicated resolution level.
`
`
`
`
`
`
`
`
`
`
`
`
`24 Claims, 9 Drawing Sheets
`
`
`
`
`PILOT TRATNER
`ON PROCESSOR
`
`
`7
`
`
`
`NO
`
`FETCH PTLOT
`RATNER PACKAGE
`
`
`
`
`FROM SERVER
`
`
`___L
`FE’CH ROUTE Efimc
`
`
`
`A D HEADER RECORD
`
`
`
`
`
`
`
`
` PROMPT USER
`
`TO CHOOSE
`
`
`ROUTE FROM
`
`
`CATALOG
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`.
`T
`
`
`
`
`
`
`
`ACTIVATE
`RENDERER
`
`
`
`
`
`
`
`ACTTVATE CACHE
`MANAGER
`
`WTTH STARTING
`
`POTNT OF
`
`ROUTE
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 1 of 20
`
`Unified Patents Exhibit 1028
`
`Page 1 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`US. Patent
`
`Dec. 17, 2002
`
`Sheet 1 0f 9
`
`US 6,496,189 B1
`
`
`
`
`
`Page 2 of 20
`
`Unified Patents Exhibit 1028
`
`Page 2 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 2 0f 9
`
`
`
`US 6,496,189 B1
`
`
`
`42A
`
`
`
`.(o. 20.000;
`
`
`0, 30.000
`i20000, 30.000)
`
`
`
`
`
`
`
`
`
`LEVEL
`
`\44A
`
`
`
`1
`
`
`
`
`
`40
`/
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 2
`
`44c
`
`
`
`EE...
`
`I LEVEL
`
`
`
`
`
`
`3
`43 ll. “fig:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IIIIIIIIIII /
`
`IIIIIIIIIII
`IIIIIIIIIII
`IIIIIIIIIII
`
`lllllllllll
`
`
`
`
`
`LEVEL
`
`
`
`4
`
`
`
`Page 3 of 20
`
`Unified Patents Exhibit 1028
`
`Page 3 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 3 0f 9
`
`
`
`US 6,496,189 B1
`
`
`FIG. 3
`
`
`RECEIVE
`
`
`IMAGES
`
`
`
`CUT IMAGES
`
`
`INTO BLOCKS
`
`
`
`
`
`
`
`
`
`
`
`
`PREPARE HIERARCHY
`
`
`OF RESOLUTION LEVELS
`
`
`
`
`ALIGN BLOCKS
`
`
`
`
`
`
`
`
`
`
`
`
`COMPRESS BLOCKS
`
`
`
`
`ADD OBJECTS
`
`
`
`
`
`
`
`
`
`Page 4 of 20
`
`Unified Patents Exhibit 1028
`
`Page 4 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 4 0f 9
`
`
`
`US 6,496,189 B1
`
`
`FIG. 4
`
`/
`
`
`
`64
`
`
`
`
`
`
`
`
`
`
`
`(10367.8,9512.8,749)
`
`
`
`SPEED;152 _,.
`
` (103622, 9457.6,750)
`
`
`
`
`
`
`KENNEDY AIRPORT: LANDING
`
` GREAT CANYON: FLIGHT
`
`
`
`
`
`
`
`
`
` YMS AIRPORT LANDING
`
`
`
`
`
`
`
`
`BERMUDA HORROR FLIGHT
`
`
`[IIL__I
`
`
`
`
`
`
`
`III
`
`\I/IIII
`
`A
`
`\I‘I \‘
`
`Page 5 of 20
`
`Unified Patents Exhibit 1028
`
`Page 5 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 5 0f 9
`
`
`
`US 6,496,189 B1
`
`vmk
`
`Ego
`
`”$040
`
`
`
`
`
`9LrNN
`
`EOE;
`
`
`
`”$65
`
`mmo<z<2
`
`
`
`Emmozmmmoh<o_><z
`
`
`
`
`
`mop
`
`zozomzzoo
`
`QB
`
`zozomzzoo
`
`
`
`2502
`
`EimmOH
`
`om
`
`Page 6 of 20
`
`Unified Patents Exhibit 1028
`
`Page 6 of 20
`
`Unified Patents Exhibit 1028
`
`
`
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 6 0f 9
`
`
`
`US 6,496,189 B1
`
`YES
`
`
`
`
`IS
`
`
`PILOT TRAINER
`
`ON PROCESSOR
`
`
`?
`
`
`
`NO
`
`
`
`
`FIG. 6
`
`
`
`FETCH PILOT
`
`TRAINER PACKAGE
`
`
`FROM SERVER
`
`
`
`._.__‘___.._____1
`
`
`
`
`FETCH ROUTE CATALOG
`
`
`
`AND HEADER RECORD
`
`
`
`
`
`
`PROMPT USER
`
`
`TO CHOOSE
`
`
`ROUTE FROM
`
`CATALOG
`
`ACTIVATE
`
`
`
`NAVIGATOR
`
`
`WITH ROUTE
`
`
`ACTIVATE
`RENDERER
`
`
`
`ROUTE
`
`
`
`
`ACTIVATE CACHE
`
`MANAGER
`
`WITH STARTING
`
`
`POINT OF
`
`
`
`
`Page 7 of 20
`
`Unified Patents Exhibit 1028
`
`Page 7 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 7 0f 9
`
`
`
`US 6,496,189 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`90Km ,I’/,r'/////
`
`/ /
`
`
`
`
`
`//,//
`
`
`
`
`
`7 F
`
`IG.
`
`
`
`
`Page 8 of 20
`
`Unified Patents Exhibit 1028
`
`Page 8 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 8 0f 9
`
`
`
`US 6,496,189 B1
`
`
`FIG. 8
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DOWNLOAD 4 LEVEL-I BLOCKS [4'02
`
`
`
`
`SURROUNDING STARTING POINT_
`
`
`
`
`
`RECEIVE BLOCK Y
`RENDERER REQUEST
`
`
`
`L_<BB>DK2L_LEVEL=N)
`FROM J98. MNEQILOLL
`1.04
`L—_—_—— .
`
`
`
`
`
`d=LEVEL OF HIGHEST LEVEL
`
`
`
`OF ANCESTOR IN CACHE MEMORY
`
`106\
`
`
`
`
`
`PROVIDE RENDERER WITH LEVEL
`
`
`
`
`
`J ANCESTOR OF BLOCK X
`
`
`
`
`
`
`ADD ANCESTORS OF LEVELS
`
`
`
`N TO DOWNLOAD QUEUE
`£1,
`
`\Ioa
`
`
`
`:
`
`
`
`
`
`
`
`
`
`
`
`
`
`ALL TC'
`CONNECTIONS
`
`
`IN USE?
`
`
`
`YES
`
`
`
`
`
`I22
`
`
`
`
`
`
`
`
`
`
`
`
`BLOCK Y
`
`NO
`
`
`CURRENTLY NEEDED
`
`
`
`Y RENDERER
`
`9
`
`
`
`PROVIDE BLOCK
`
`
`Y TO RENDERER
`
`
`
`124
`
`
`
`
`
`STORE BLOCK Y
`
`
`
`NO
`IN CACHE MEMORY
`
`
`126/
`
`
` NO
`CACHE
`
`
`
`MEMORY
`
`
`
`FULL?
`
`
`
`
`
`
`DISCARD AN OUT
`
`
`
`OF RANGE BLOCK
`
`
`128
`
`
`
`
`
`ALL
`
`
`SURROUNDING
`QUEUE
`
`
`BLOCKS IN
`
` EMPTY”)
`
`I NO
`
`
`my
`
`
`
`
`
`
`112
`
`
`
`
`
`K=LOWEST LEVEL OF
`
`
`BLOCK IN QUEUE
`
`
`
`
`
`
`
`
`
`K=LOWEST LEVEL FOR WHICH NOT ALL
`
`
`
`
`SURROUNDING BLOCKS ARE IN CACHE
`
`
`
`
`
`
`
`114 BLOCK x = LAST ENTERED
`,_I
`
`
`
`C
`LEVEL—K BLOCK
`
`
`
`_._____I_—____I
`
`
`
`
`SEND DOWNLOAD ORDER FOR
`
`
`
`
`
`A LEVEL—K BLOCK NOT IN CACHE
`
`NO
`
`
`
`
`116’\
`
`
`IS BLOC
`
`
`X WITHIN
`
`
`
`RANGE?
`
`
` SEND DOWNLOAD
`
`
`
`
`118
`
`
`
`\«QPLDER FOR BLOCK x
`
`
`
`
`
`
`
`
`
`120
`
`
`
`
`REMOVE ORDERED
`
`
`BLOCK FROM QUEUE
`
`
`
`Page 9 of 20
`
`Unified Patents Exhibit 1028
`
`Page 9 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`US. Patent
`
`
`
`
`
`Dec. 17, 2002
`
`
`
`
`Sheet 9 0f 9
`
`
`
`US 6,496,189 B1
`
`0
`
`
`
`~
`.
`
`152
`
`
`
`LEVEL J
`
`LEVEL J-I—t
`
`
`
`
`158
`
`
`
`
`
`
`
`150
`
`
`
`LEVEL N
`
`
`FIG. 9
`
`
`15*
`
`
`
`ORDER BLOCKS OF
`
`
`
`LEVELS J+1 TO N
`
`
`
`
`
`Page 10 of 20
`
`Unified Patents Exhibit 1028
`
`Page 10 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`1
`REMOTE LANDSCAPE DISPLAY AND
`
`
`
`PILOT TRAINING
`
`
`CROSS-REFERENCE TO RELATED CASE
`
`
`
`
`
`
`
`
`This is a continuation-in-part of US. patent application
`Ser. No. 08/939,948, filed on Sep. 29, 1997 now US. Pat.
`
`
`
`
`
`
`
`
`
`
`No. 6,111,583.
`
`
`FIELD OF THE INVENTION
`
`
`
`
`
`
`
`
`
`The present invention relates generally to terrain displays,
`
`
`
`
`
`
`and specifically to computerized displaying of real-life
`
`
`
`three-dimensional images of terrain.
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`Pilots frequently wish to plan their flight route ahead of
`
`
`
`
`
`
`
`
`
`time and familiarize themselves with the points they plan to
`
`
`
`
`
`
`
`
`
`
`pass and approach. Maps and flight charts are used to plan
`
`
`
`
`
`
`
`
`
`routes before actually flying. However, not all pilots can
`
`
`
`
`
`
`
`
`
`easily appreciate all the features and difficulties of a route
`
`
`
`
`
`
`
`from a map. Furthermore, some information may not appear
`
`
`
`
`
`
`
`
`
`on the map in sufficient detail. For this reason, Jeppesen
`
`
`
`
`
`
`
`Sanderson Inc., of Englewood, Colorado, provides pilots
`
`
`
`
`
`
`with actual photographs of runway environments together
`
`
`
`
`
`
`
`with graphical presentations of terrain and other obstacles,
`
`
`
`
`
`
`
`as well as other pertinent local traffic information.
`
`
`
`
`
`
`
`
`Flight simulators allow pilots to practice flying without
`
`
`
`
`
`
`
`
`
`leaving the ground. The simulator shows the view seen by
`
`
`
`
`
`
`
`
`the pilot at every point of the flight. However, most existing
`
`
`
`
`
`
`
`
`
`simulators do not show real images, but rather animated
`
`
`
`
`
`
`
`
`views, and they cannot provide full acquaintance with
`real-life terrain.
`
`
`In addition, there exist Internet sites which display still
`
`
`
`
`
`
`
`
`images and/or video streams of different locations, for those
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`who wish to see the location before visiting there. Although
`
`
`
`
`
`
`
`streaming of video images over the Internet is known in the
`art, Internet video streams are limited in the information they
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`provide and would not allow the pilot to interactively probe
`
`
`
`
`
`
`the terrain over which he is going to fly.
`
`
`
`
`
`Computer rendering of three-dimensional terrain images
`is known in the art. US. Pat. No. 4,940,972, which is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`incorporated herein by reference, suggests displaying to a
`
`
`
`
`
`
`
`
`pilot of an aircraft a synthetic image of the ground terrain
`over which the pilot is flying. US. Pat. No. 5,566,073 which
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`is incorporated herein by reference, describes a method for
`
`
`
`
`
`
`
`allowing a pilot to preview a route either in flight or on the
`
`
`
`
`
`
`
`
`
`ground. The ’073 patent also suggests allowing the pilot to
`
`
`
`
`
`
`
`
`
`take over and try out different flight strategies. However, the
`
`
`
`
`
`
`
`data volume required to display flight routes in accordance
`
`
`
`
`
`
`
`
`with the above-mentioned patents is very large. The ’073
`
`
`
`
`
`
`
`patent suggests representing the terrain as polygons in order
`
`
`
`
`
`
`
`
`
`
`to save computer storage space, but even so, the amount of
`
`
`
`
`
`
`
`
`storage on most home computers allows only a limited area
`
`
`
`
`
`
`
`
`
`and/or resolution level to be displayed. The ’073 patent
`
`
`
`
`
`
`
`suggests using a CD-ROM to store the required data.
`However,
`the delivery of the CD-ROM to home users
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`requires time and prevents pilots from preparing for last
`
`
`
`
`
`
`minute flight scheduling. Furthermore, changes in the
`
`
`
`
`
`
`
`terrain, such as seasonal changes, require frequent updates
`of the data on the CD-ROM.
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`It is an object of some aspects of the present invention to
`
`
`
`
`
`
`
`provide methods and apparatus for displaying on a remote
`
`
`
`
`
`
`
`computer actual images of an area as seen from a viewpoint
`
`
`
`which is chosen interactively.
`
`
`2
`
`
`
`
`
`
`
`
`It is another object of some aspects of the present inven-
`
`
`
`
`
`
`
`
`tion to provide methods and apparatus for training a pilot to
`
`
`
`
`
`
`
`
`
`fly a preplanned flight course while allowing the pilot to see
`
`
`
`
`
`
`
`
`
`
`
`the view seen at any point along the flight course at
`
`
`
`
`substantially any desired angle.
`
`
`
`
`
`
`
`
`
`It is still another object of some aspects of the present
`
`
`
`
`
`
`
`invention to provide methods and apparatus for displaying
`
`
`
`
`
`
`
`on a client computer images of ground terrain stored in a
`
`
`
`
`
`
`
`
`remote server, which are conveyed to the client via a
`
`
`
`
`network, preferably the Internet.
`
`
`
`
`
`
`
`
`It is still another object of some aspects of the present
`
`
`
`
`
`
`invention to provide methods and apparatus for streaming
`
`
`
`
`
`
`data required for rendering three-dimensional terrain images
`on a remote computer.
`
`
`
`
`
`
`
`
`
`invention, a
`In preferred embodiments of the present
`
`
`
`
`
`
`
`processor simulates flight of a virtual airplane along a
`
`
`
`
`
`
`
`
`selected route. At substantially any viewpoint along the
`
`
`
`
`
`
`
`
`
`
`route, the processor displays the view seen from the view-
`
`
`
`
`
`
`in three-dimensional real-life images. A user may
`point
`
`
`
`
`
`
`
`
`
`select at substantially each point along the route the direc-
`
`
`
`
`
`
`
`
`
`tion of view and may change the direction dynamically.
`
`
`
`
`
`
`
`Preferably, the user controls the speed of progress along
`
`
`
`
`
`
`
`
`
`the route and may stop the progress and/or reverse direction
`
`
`
`along the course.
`
`
`
`
`
`
`
`Preferably, the user views the three-dimensional images
`
`
`
`
`
`
`
`without having to store a large database of terrain images on
`
`
`
`
`
`
`
`
`
`the user’s processor, and without requiring that the processor
`
`
`
`
`
`
`
`
`have an unusually large memory capacity. Rather, the user
`
`
`
`
`
`
`
`
`connects to a pilot
`training server, which stores all the
`
`
`
`
`
`
`
`
`
`required data, and the user’s processor downloads the data
`
`
`
`
`
`
`
`dynamically according to the data required to display each
`
`
`
`
`
`
`
`specific scene. Preferably,
`the processor connects to the
`
`
`
`
`
`
`server via a communication link, preferably a public
`network, such as the Internet. Preferably, the data is con-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`veyed by a standard modem at sufficient speed for relatively
`
`
`
`
`
`smooth display of the images.
`
`
`
`
`
`
`Alternatively or additionally, the required data is down-
`
`
`
`
`
`
`
`
`
`loaded prior to the displaying of the scene, or the data is
`
`
`
`
`
`
`retrieved from a CD or other memory apparatus associated
`
`
`
`with the processor.
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`
`
`
`
`
`
`
`the user may change the parameters of the movement of the
`
`
`
`
`
`
`
`
`virtual airplane, including the speed, height and angle of the
`
`
`
`
`
`
`
`
`airplane, and even deviate from the preplanned route.
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`
`
`
`
`
`
`
`the route includes access to an airport according to the
`
`
`
`
`
`
`access rules of the airport. Alternatively or additionally, the
`
`
`
`
`
`
`route may include military maneuvers for bombing or access
`
`
`
`
`
`
`to enemy locations. Further alternatively or additionally, the
`
`
`
`
`
`
`
`route may be selected from routes of access to canyons,
`mountain areas or any other hard-to-access points,
`for
`
`
`
`
`
`
`
`
`
`
`
`
`
`example, in order to train rescue helicopter pilots.
`
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`an operator of the server and/or the user may add to the data
`
`
`
`
`
`
`
`
`
`
`
`which describes the terrain annotations which describe spe-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`cific points in the scene, such as runways or other destina-
`
`
`
`
`
`
`
`
`tions. Alternatively or additionally, the operator may add
`
`
`
`
`
`
`three-dimensional objects, such as planned buildings,
`
`
`
`
`
`
`
`vehicles, etc. In a preferred embodiment of the present
`
`
`
`
`
`
`
`invention, a group of vehicles are tracked according to their
`
`
`
`
`
`
`
`
`position, and they are constantly superimposed on the land-
`
`
`scape images.
`
`
`
`
`
`
`
`U.S. patent application Ser. No. 08/939,948, which is
`
`
`
`
`
`
`
`
`assigned to the assignee of the present application and is
`
`
`
`
`
`
`incorporated herein by reference, describes a method of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`US 6,496,189 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`5
`
`
`
`10
`
`15
`
`
`
`20
`
`25
`
`
`
`30
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`Page 11 of 20
`
`Unified Patents Exhibit 1028
`
`Page 11 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`
`
`US 6,496,189 B1
`
`
`3
`
`
`
`
`
`
`terrain images. The method
`rendering three-dimensional
`
`
`
`
`
`
`
`allows fast rendering of images by comparison with other
`
`
`
`
`
`
`
`
`methods known in the art. The method of US. patent
`
`
`
`
`
`
`
`application Ser. No. 08/939,948 uses a hierarchical database
`
`
`
`
`
`
`
`in which substantially each terrain area is described in a
`
`
`
`
`
`
`
`
`plurality of blocks at different resolution levels. The blocks
`at lower resolution levels include less detail per unit area,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`while the blocks of higher resolution levels include more
`
`
`
`
`
`
`
`
`
`detail per unit area. The additional storage space required in
`
`
`
`
`
`
`
`
`
`order to store the additional data required due to the hier-
`
`
`
`
`
`
`
`archical structure is “paid back” in improved time perfor-
`
`
`
`mance of the rendering.
`
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`the data is stored at the remote server in accordance with the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`hierarchical structure described in the ’948 application.
`
`
`
`
`
`
`
`When data describing a specific area is needed in order to
`
`
`
`
`
`
`
`
`render the current view,
`the processor first downloads a
`
`
`
`
`
`
`
`
`
`block with a low level of resolution, which requires less data
`
`
`
`
`
`
`
`
`
`
`
`volume per area unit, and renders the view using the lower
`resolution block. Afterwards, if available bandwidth on the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`communication line permits, blocks which cover the specific
`
`
`
`
`
`
`
`
`
`area from higher resolution levels are downloaded, and the
`
`
`
`
`
`
`
`
`processor uses these blocks to improve the rendered view.
`
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`
`
`
`
`
`
`
`
`the processor manages a local cache memory in which the
`
`
`
`
`
`
`
`
`
`processor stores blocks which cover terrain areas in the
`
`
`
`
`
`
`
`vicinity of the current viewpoint. Preferably, the processor
`
`
`
`
`
`
`
`
`stores in the cache memory all blocks downloaded from the
`
`
`
`
`
`
`
`
`server. Preferably, when the communication link is not
`
`
`
`
`
`
`
`needed to download blocks required for currently rendered
`
`
`
`
`
`
`
`
`images, the processor downloads blocks in the vicinity of the
`
`
`
`
`
`
`
`
`viewpoint. Preferably,
`the processor stores in the cache
`
`
`
`
`
`
`
`memory an equal number of blocks from each resolution
`level. Further preferably, the number of blocks in the cache
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`memory is such that the cache memory has all the required
`
`
`
`
`
`
`
`
`
`blocks to render the view from the current viewpoint in any
`
`
`
`
`
`
`
`view direction. Preferably, when downloading blocks, the
`processor first downloads blocks from lower resolution
`
`
`
`
`
`
`
`levels.
`
`
`
`
`
`
`
`Preferably, when the processor requires a number of
`blocks, the first block sent is the block of the lowest level.
`
`
`
`
`
`
`
`
`
`
`If two blocks of the same level are required, the one which
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`is requested last is sent first. Thus, when the viewpoint
`
`
`
`
`
`
`
`
`
`
`
`changes, the blocks sent first are for the new viewpoint, and
`
`
`
`
`
`
`
`
`
`only afterwards are blocks sent for the old viewpoint.
`
`
`
`
`
`
`
`Preferably, if a block requested earlier is relatively far from
`
`
`
`
`
`
`
`
`
`the viewpoint, it is not sent, since it is obviously not needed
`anymore.
`
`
`
`
`
`
`
`Preferably, when the processor needs to render a new
`
`
`
`
`
`
`
`
`
`image, the processor first uses the data blocks stored in the
`
`
`
`
`
`
`
`cache memory and concurrently sends download orders for
`higher resolution level blocks.
`
`
`
`
`
`
`
`
`
`
`
`In some preferred embodiments of the present invention,
`
`
`
`
`
`
`
`
`the processor runs a rendering program which displays the
`
`
`
`
`
`
`
`three dimensional images based on the viewpoint, the size of
`
`
`
`
`
`
`
`
`
`the displayed image and the blocks received from the server.
`
`
`
`
`
`
`
`
`The rendering program orders the blocks it needs using a
`
`
`
`
`
`
`cache manager, which is preferably a software routine
`
`
`
`
`
`
`
`
`
`
`running on the processor. If the cache manager has the
`
`
`
`
`
`
`
`ordered block,
`it provides it
`to the rendering program.
`However, if the block is not carried by the cache manager,
`
`
`
`
`
`
`
`
`it is ordered from the server, and a replacement block from
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a lower resolution level is passed to the rendering program.
`
`
`
`
`
`
`When the rendering program requires a block including a
`
`
`
`
`
`
`
`
`
`
`new point or area for display,
`the cache manager first
`requests the block of the lowest resolution level which
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`15
`
`
`
`20
`
`25
`
`
`
`30
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`
`4
`
`
`
`
`
`
`
`
`
`
`covers the area with the least detail and then requests
`
`
`
`
`
`
`
`subsequent blocks with successively increasing detail, until
`
`
`
`
`
`
`
`
`the block with the level of detail required by the rendering
`
`
`
`
`
`
`
`
`program is sent. The rendering program renders the three-
`
`
`
`
`
`
`
`
`dimensional image using the blocks it has on hand. Each
`
`
`
`
`
`
`
`
`time another block is received, the image is rendered again.
`
`
`
`
`
`
`
`
`
`Thus, the user sees an image at substantially all times and is
`
`
`
`
`
`
`
`not prevented from moving the viewpoint while additional
`
`
`
`
`
`
`data is being sent from the server.
`
`
`
`
`
`
`There is therefore provided in accordance with a preferred
`
`
`
`
`
`
`embodiment of the present invention, a method of providing
`
`
`
`
`
`
`data blocks describing three-dimensional
`terrain to a
`
`
`
`
`
`
`
`renderer, the data blocks belonging to a hierarchical struc-
`
`
`
`
`
`
`
`
`ture which includes blocks at a plurality of different reso-
`
`
`
`
`
`
`
`
`
`lution levels, the method including receiving from the ren-
`
`
`
`
`
`
`
`
`derer one or more coordinates in the terrain along with
`
`
`
`
`
`
`
`indication of a respective resolution level, providing the
`renderer with a first data block which includes data corre-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sponding to the one or more coordinates, from a local
`
`
`
`
`
`
`
`
`memory, and downloading from a remote server one or more
`
`
`
`
`
`
`
`additional data blocks which include data corresponding to
`
`
`
`
`
`
`
`
`
`the one or more coordinates if the provided block from the
`
`
`
`
`
`
`
`local memory is not at the indicated resolution level.
`
`
`
`
`
`
`
`
`Preferably, providing the first data block includes provid-
`
`
`
`
`
`
`
`
`
`
`ing the data block from the highest resolution level which
`
`
`
`
`
`
`
`includes data corresponding to the one or more coordinates.
`
`
`
`
`
`
`
`Preferably, downloading the one or more additional data
`
`
`
`
`
`
`blocks includes downloading a block at a resolution level
`
`
`
`
`
`
`
`
`higher than the resolution level of the first block.
`
`
`
`
`
`
`
`Further preferably, downloading the one or more addi-
`
`
`
`
`
`
`
`
`tional data blocks includes downloading the blocks from a
`
`
`
`
`
`
`
`succession of resolution levels, from the level immediately
`
`
`
`
`
`
`
`
`
`
`higher than the resolution level of the first block up to the
`maximal existent resolution level on the server not above the
`
`
`
`
`
`
`
`
`indicated resolution level.
`
`
`
`
`
`
`
`
`
`
`Preferably, receiving from the renderer the one or more
`
`
`
`
`
`coordinates includes receiving a plurality of coordinates
`
`
`
`
`
`
`included in a plurality of respective distinct blocks, and
`
`
`
`
`
`
`downloading the one or more blocks includes downloading
`
`
`
`
`
`
`blocks including data corresponding to at least some of the
`
`
`
`plurality of coordinates.
`
`
`
`
`Preferably, downloading the blocks includes downloading
`the blocks in an order determined according to their reso-
`
`
`
`
`
`
`
`lution levels.
`
`
`
`
`
`
`
`Further preferably, downloading the blocks includes
`downloading blocks of lower resolution levels before blocks
`
`
`
`
`
`
`
`
`
`
`of higher resolution levels.
`
`
`
`
`Preferably, downloading the blocks includes downloading
`first the block for which the coordinates were provided last
`
`
`
`
`
`
`
`
`
`among blocks at a common resolution level.
`
`
`
`
`
`
`
`
`
`Preferably, downloading the blocks includes downloading
`the blocks according to the order in which the coordinates
`
`
`
`
`
`
`
`
`
`were provided.
`
`
`
`
`
`Preferably, downloading the blocks includes downloading
`first the block for which the coordinates were provided last.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Preferably,
`the method includes downloading excess
`
`
`
`
`
`
`
`
`
`blocks not currently needed by the renderer to fill up the
`
`
`
`
`
`
`local memory when not downloading blocks required by the
`renderer.
`
`Preferably, the renderer renders a view from a current
`
`
`
`
`
`
`
`
`
`
`
`
`viewpoint, and downloading the excess blocks includes
`
`
`
`
`
`
`
`
`
`filling the local memory with substantially all of the blocks
`
`
`
`
`
`
`
`surrounding a point in the terrain seen from the current
`
`
`
`
`
`viewpoint within a predetermined distance range.
`
`
`
`
`
`
`Preferably, downloading excess blocks includes filling the
`
`
`
`
`
`
`
`local memory with substantially the same number of blocks
`from each different resolution level.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 12 of 20
`
`Unified Patents Exhibit 1028
`
`Page 12 of 20
`
`Unified Patents Exhibit 1028
`
`

`

`5
`
`
`
`
`
`
`
`
`
`Preferably, filling the local memory includes filling the
`
`
`
`
`
`
`
`
`
`memory with substantially all the blocks within the range
`
`
`
`
`
`
`
`from a lower resolution level before downloading blocks of
`
`
`
`higher resolution levels.
`
`
`
`
`
`
`Preferably, downloading the data blocks includes down-
`
`
`
`
`
`
`loading the blocks via the Internet.
`
`
`
`
`
`There is further provided in accordance with a preferred
`
`
`
`
`
`embodiment of the present invention, a method of display-
`
`
`
`
`
`
`ing three dimensional images, including establishing a com-
`
`
`
`
`
`
`munication link between a local processor and a server,
`
`
`
`
`
`
`
`transferring data blocks describing terrain over the commu-
`
`
`
`
`
`
`
`
`nication link from the server to the local processor; and
`
`
`
`
`
`rendering a three-dimensional
`terrain image at
`the local
`
`
`
`
`
`processor responsive to the data blocks.
`
`
`
`
`
`Preferably, establishing the communication link includes
`
`
`
`
`establishing a low-speed communication link.
`
`
`
`
`
`
`Preferably, transferring the data blocks includes transfer-
`
`
`
`
`
`
`ring the blocks via the Internet.
`
`
`
`
`
`
`Further preferably, transferring the data blocks includes
`
`
`
`
`
`
`transferring the blocks responsive to a list of coordinates
`
`
`
`generated by the processor.
`
`
`
`
`
`
`Preferably, the list of coordinates is prepared responsive
`
`
`
`
`
`
`
`to a viewpoint from which the image is rendered.
`
`
`
`
`
`
`
`Preferably, the viewpoint changes over time following a
`
`
`predetermined course.
`
`
`
`
`
`
`Preferably, the predetermined course is received from the
`server.
`
`
`
`
`
`
`Preferably,
`the predetermined course describes a sug-
`
`
`
`
`
`
`gested course for landing in an airport.
`
`
`
`
`
`
`
`
`Preferably, a user of the processor changes the view
`
`
`
`
`
`
`
`
`direction from the viewpoint without removing the view-
`
`
`
`
`
`point from the predetermined course.
`
`
`
`
`
`
`Preferably, the viewpoint is controlled by a user of the
`processor.
`
`
`
`
`
`
`
`Preferably, transferring the data blocks includes transfer-
`ring blocks which include altitude data of the terrain.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Preferably, transferring the data blocks includes transfer-
`
`
`
`
`
`
`
`ring blocks which include optional objects to be overlaid on
`the terrain.
`
`
`
`
`
`
`
`
`Preferably, rendering the images includes rendering the
`
`
`
`
`
`
`
`images with at least some of the optional objects according
`
`
`
`
`
`
`
`to settings of the user of the local processor.
`
`
`
`
`
`
`There is further provided in accordance with a preferred
`embodiment of the present invention, a method of pilot
`
`
`
`
`
`
`
`
`
`
`
`
`
`training, including loading a course of a flight vehicle into
`
`
`
`
`
`
`a local processor, establishing a communication link
`50
`
`
`
`
`
`
`
`
`between the processor and a server, transferring data blocks
`describing terrain viewed from the course over the commu-
`
`
`
`
`
`
`
`
`
`nication link from the server to the local processor, and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`rendering a three-dimensional
`terrain image at
`the local
`
`
`
`
`
`processor responsive to the data blocks.
`
`
`
`
`
`
`Preferably, the course describes a suggested course for
`
`
`
`landing in an airport.
`
`
`
`
`
`
`There is further provided in accordance with a preferred
`
`
`
`
`
`
`
`embodiment of the present invention, apparatus for provid-
`
`
`
`
`
`
`
`ing data blocks describing three-dimensional terrain to a
`renderer, the data blocks belonging to a hierarchical struc-
`
`
`
`
`
`
`
`ture which includes blocks at a plurality of different reso-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`lution levels, the apparatus including a local memory, which
`
`
`
`
`
`
`stores data blocks corresponding to coordinates proximal to
`a current viewpoint of the renderer, a communication link,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`through which the memory receives the data blocks from a
`remote server, and a processor which receives one or more
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`specified coordinates along with indication of a respective
`
`
`
`
`
`
`10
`
`
`
`
`
`15
`
`
`
`
`
`
`
`
`
`
`
`
`20
`
`25
`
`30
`
`
`
`
`
`
`
`
`
`
`35
`
`40
`
`45
`
`55
`
`
`
`60
`
`65
`
`
`
`US 6,496,189 B1
`
`
`6
`
`
`
`
`
`
`
`
`resolution level from the renderer, provides the renderer
`
`
`
`
`
`
`
`
`with a first data block which includes data corresponding to
`
`
`
`
`
`
`
`
`
`the one or more specified coordinates from the local
`
`
`
`
`
`
`
`
`memory, and downloads over the communication link one or
`
`
`
`
`
`
`
`
`more additional data blocks which include data correspond-
`ing to the one or more coordinates if the first block is not
`
`
`
`
`
`
`
`
`
`from the indicated level.
`
`
`
`
`
`
`
`
`
`
`
`
`Preferably,
`the processor provides the first data block
`
`
`
`
`
`
`
`
`
`from the highest resolution level which includes the one or
`
`
`
`
`
`
`
`more coordinates currently available in the local memory.
`
`
`
`
`
`
`Preferably, the processor downloads a block of a resolu-
`
`
`
`
`
`
`
`
`
`
`tion level higher than the resolution level of the first block.
`
`
`
`
`
`
`
`Preferably,
`the processor downloads blocks from the
`
`
`
`
`
`
`
`
`resolution level immediately higher than the resolution level
`of the first block up to a maximal resolution level of blocks
`
`
`
`
`
`
`
`
`stored on the server that is not above the indicated resolution
`
`
`
`
`
`
`
`
`level.
`
`
`
`
`
`
`
`
`Preferably,
`the processor receives from the renderer a
`
`
`
`
`
`plurality of coordinates included in a plurality of respective
`
`
`
`
`
`
`
`
`distinct blocks and downloads blocks including at least some
`
`
`
`
`of the plurality of coordinates.
`
`
`
`
`
`Preferably, the processor downloads the blocks in an order
`
`
`
`
`