`DocumentDescription: TrackOne Request
`
`PTO/AIA/424 (04-14)
`
`CERTIFICATION AND REQUESTFOR PRIORITIZED EXAMINATION
`UNDER 37 CFR 1.102(e) (Page 1 of 1)
`
`Isaac LevanontonLT
`
`Nonprovisional Application Number(if
`
`First Named
`
`forms are submitted.
`
`OPTIMIZED IMAGE DELIVERY OVER LIMITED BANDWIDTH COMMUNICATION CHANNELS
`
`APPLICANT HEREBYCERTIFIES THE FOLLOWING AND REQUESTSPRIORITIZED EXAMINATION FOR
`THE ABOVE-IDENTIFIED APPLICATION.
`
`1. The processing fee set forth in 37 CFR 1.17(i)(1) and the prioritized examination fee set forth in
`37 CFR 1.17(c) have been filed with the request. The publication fee requirementis met
`becausethat fee, set forth in 37 CFR 1.18(d), is currently $0. The basic filing fee, search fee,
`and examination fee arefiled with the request or have been already been paid.
`| understand
`that any required excess claims fees or application size fee must be paid for the application.
`
`| understand that the application may not contain, or be amendedto contain, more than four
`independentclaims, more than thirty total claims, or any multiple dependent claims, and that
`any request for an extension of time will cause an outstanding Track | request to be dismissed.
`
`3. The applicable box is checked below:
`
`Original Application (Track One)
`
`- Prioritized Examination under § 1.102(e)(1
`
`i.
`
`(a) The application is an original nonprovisional utility application filed under 35 U.S.C. 111(a).
`This certification and requestis being filed with the utility application via EFS-Web.
`---OR---
`(b) The application is an original nonprovisional plant application filed under 35 U.S.C. 111(a).
`This certification and requestis being filed with the plant application in paper.
`
`An executed inventor’s oath or declaration under 37 CFR 1.63 or 37 CFR 1.64 for each
`inventor, or the application data sheet meeting the conditions specified in 37 CFR 1.53(f)(3)(i) is
`filed with the application.
`
`Request for Continued Examination - Prioritized Examination under § 1.102(e)(2)
`
`A request for continued examination has beenfiled with, or prior to, this form.
`If the application is a utility application, this certification and requestis being filed via EFS-Web.
`iii. The application is an original nonprovisionalutility application filed under 35 U.S.C. 111(a), or is
`a national stage entry under 35 U.S.C. 3771.
`iv. This certification and requestis being filed prior to the mailing of a first Office action responsive
`to the request for continued examination.
`No prior requestfor continued examination has been granted prioritized examination status
`under 37 CFR 1.102(e)(2).
`
`signature/ANatoly S. Weiser/
`Name
`yMnatoly S. Weiser
`
`(Print/Typed)
`
`pete O3 November 2016
`Practitioner
`43229
`
`Registration Number
`
`This form must be signed in accordance with 37 CFR 1.33. See 37 CFR 1.4(d) for signature requirements and certifications.
`Note:
`Submit multiple forms if more than one signature is required.”
`
`[|
`
`*Total of
`
`1 of 150
`
`Microsoft Corp. Exhibit 1016
`
`1 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`Privacy Act Statement
`
`The Privacy Act of 1974 (P.L. 93-579) requires that you be given certain information in connection with your
`submission of the attached form related to a patent application or patent. Accordingly, pursuant to the requirements of
`the Act, please be advised that: (1) the general authority for the collection of this information is 35 U.S.C. 2(b)(2); (2)
`furnishing of the information solicited is voluntary; and (3) the principal purpose for which the information is used by the
`U.S. Patent and Trademark Office is to process and/or examine your submission related to a patent application or
`patent.
`If you do not furnish the requested information, the U.S. Patent and Trademark Office may not be able to
`process and/or examine your submission, which may result in termination of proceedings or abandonmentof the
`application or expiration of the patent.
`
`The information provided by you in this form will be subject to the following routine uses:
`
`1. The information on this form will be treated confidentially to the extent allowed under the Freedom of
`Information Act (5 U.S.C. 552) and the Privacy Act (5 U.S.C 552a). Records from this system of records may
`be disclosed to the Departmentof Justice to determine whether disclosure of these records is required by the
`Freedom ofInformation Act.
`A record from this system of records maybe disclosed, as a routine use, in the course of presenting evidence
`to a court, magistrate, or administrative tribunal, including disclosures to opposing counsel in the course of
`settlement negotiations.
`A record in this system of records maybe disclosed, as a routine use, to a Member of Congress submitting a
`request involving an individual, to whom the record pertains, when the individual has requested assistance from
`the Member with respect to the subject matter of the record.
`A record in this system of records maybe disclosed, as a routine use, to a contractor of the Agency having
`needfor the information in order to perform a contract. Recipients of information shall be required to comply
`with the requirements of the Privacy Act of 1974, as amended, pursuantto 5 U.S.C. 552a(m).
`A record related to an International Application filed under the Patent Cooperation Treaty in this system of
`records maybe disclosed, as a routine use, to the International Bureau of the World Intellectual Property
`Organization, pursuant to the Patent Cooperation Treaty.
`A record in this system of records maybe disclosed, as a routine use, to another federal agency for purposes
`of National Security review (35 U.S.C. 181) and for review pursuantto the Atomic Energy Act (42 U.S.C.
`218(c)).
`A record from this system of records may be disclosed, as a routine use, to the Administrator, General
`Services, or his/her designee, during an inspection of records conducted by GSAaspart of that agency’s
`responsibility to recommend improvements in records managementpractices and programs, under authority of
`44 U.S.C. 2904 and 2906. Such disclosure shall be made in accordance with the GSA regulations governing
`inspection of records for this purpose, and any other relevant (/.e., GSA or Commerce) directive. Such
`disclosure shall not be used to make determinations about individuals.
`A record from this system of records maybe disclosed, as a routine use, to the public after either publication of
`the application pursuant to 35 U.S.C. 122(b) or issuance of a patent pursuant to 35 U.S.C. 151. Further, a
`record may be disclosed, subject to the limitations of 37 CFR 1.14, as a routine use, to the public if the record
`wasfiled in an application which became abandonedor in which the proceedings were terminated and which
`application is referenced by either a published application, an application open to public inspection or an issued
`patent.
`A record from this system of records may be disclosed, as a routine use, to a Federal, State, or local law
`enforcement agency, if the USPTO becomesawareof a violation or potential violation of law or regulation.
`
`2 of 150
`
`Microsoft Corp. Exhibit 1016
`
`2 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-l-
`
`OPTIMIZED IMAGE DELIVERY OVER LIMITED BANDWIDTH COMMUNICATION
`
`CHANNELS
`
`Isaac Levanon
`
`Yonatan Lavi
`
`Priority Claims/Related Applications
`
`This application is a continuation of and claimspriority to U.S. Patent Application Serial
`
`No. 14/970,526, filed December 15, 2015, entitled OPTIMIZED IMAGE DELIVERY OVER
`
`LIMITED BANDWIDTH COMMUNICATION CHANNELS; this application is also a
`
`continuation of and claimspriority to U.S. Patent Application Serial No. 15/281,037, filed
`
`September 29, 2016, entitled OPTIMIZED IMAGE DELIVERY OVER LIMITED
`
`BANDWIDTH COMMUNICATION CHANNELS; eachof the U.S. Patent Applications Serial
`
`No. 14/970,526 and Serial No. 15/281,037 is a continuation of and claimspriority to U.S. Patent
`
`Application Serial No. 14/547,148, filed November 19, 2014, entitled OPTIMIZED IMAGE
`
`DELIVERY OVER LIMITED BANDWIDTH COMMUNICATION CHANNELS, now U.S.
`
`Patent No. 9,253,239; which is a continuation of and claimspriority to U.S. Patent Application
`
`Serial No. 13/027,929, filed February 15, 2011, entitled OPTIMIZED IMAGE DELIVERY
`
`OVER LIMITED BANDWIDTH COMMUNICATION CHANNELS, now U.S. Patent No.
`
`8,924,506; which is a continuation-in-part of and claimspriority to U.S. Patent Application
`
`Serial No. 12/619,643, filed on November 16, 2009, entitled OPTIMIZED IMAGE DELIVERY
`
`OVER LIMITED BANDWIDTH COMMUNICATION CHANNELS, now U.S. Patent No.
`
`7,908,343; which is a continuation of and claimspriority to U.S. Patent Application Serial No.
`
`10/035,987, filed on December 24, 2001, entitled OPTIMIZED IMAGE DELIVERY OVER
`
`LIMITED BANDWIDTH COMMUNICATION CHANNELS, now U.S. Patent No. 7,644,131;
`
`which claims the benefit under 35 U.S.C. $119(e) of U.S. Provisional Application Nos.
`
`60/258,488, 60/258,489, 60/258,465, 60/258,468, 60/258,466, and 60/258,467, all filed
`
`December 27, 2000. The disclosuresofall the foregoing patent documents are incorporated
`
`herein by referenceas if fully set forth herein, including Figures, Claims, and Tables. The
`
`present application is also related to application Serial No. 10/035,981, entitled SYSTEM AND
`
`METHODS FOR NETWORKIMAGE DELIVERY WITH DYNAMIC VIEWING FRUSTUM
`
`OPTIMIZED FOR LIMITED BANDWIDTH COMMUNICATION CHANNELS, Levanon er
`
`3 of 150
`
`Microsoft Corp. Exhibit 1016
`
`3 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-2-
`
`al., filed on December 24, 2001, now U.S. Patent No. 7,139,794, issued on November 21, 2006,
`
`whichis assigned to the Assignee of the present Application.
`
`Field
`
`The disclosure is related to network based, image distribution systems and,in particular,
`
`to a system and methodsfor efficiently selecting and distributing image parcels through a
`
`narrowband or otherwise limited bandwidth communications channel to support presentation of
`
`high-resolution images subject to dynamic viewing frustums.
`
`Background
`
`The Internet and or other network systems may provide a unique opportunity to transmit
`
`for example complex images, typically large scale bit-maps, particularly those approaching
`
`photo-realistic levels, over large area and or distances. In common application, the images may
`
`be geographic, topographic, and or other highly detailed maps. The data storage requirements
`
`and often proprietary nature of such imagescould be such that conventional interests may be to
`
`transfer the images on an as-needed basis.
`
`In conventional fixed-site applications, the image data may be transferred over a
`
`relatively high-bandwidth network to client computer systemsthat in turn, may render the
`
`image. Client systems may typically implement a local image navigation system to provide
`
`zoom and or pan functions based on userinteraction. As well recognized problem with such
`
`conventional systems could be that full resolution image presentation may be subject to the
`
`inherent transfer latency of the network. Different conventional systems have been proposed to
`
`reduce the latency affect by transmitting the image in highly compressed formats that support
`
`progressive resolution build-up of the image within the current client field of view. Using a
`
`transform compressed image transfer function increasesthe field of the image that can be
`
`transferred over a fixed bandwidth network in unit time. Progressive image resolution
`
`transmission, typically using a differential resolution method, permits an approximate image to
`
`be quickly presented with image details being continuously added overtime.
`
`4 of 150
`
`Microsoft Corp. Exhibit 1016
`
`4 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-3-
`
`Tzou, in U.S. Pat. No. 4,698,689, describes a two-dimensional data transform system
`
`that supports transmission of differential coefficients to represent an image. Subsequent
`
`transmitted coefficient sets are progressively accumulated with prior transmitted sets to provide a
`
`succeedingly refined image. The inverse-transform function performedby the client computer
`
`is, however, highly compute intensive. In order to simplify the transform implementation and
`
`further reduce the latency of presenting any portion of an approximate image, images are sub-
`
`divided into a regular array. This enables the inverse-transform function on the client, which is
`
`time-critical, to deal with substantially smaller coefficient data sets. The array size in Tzou is
`
`fixed, which leads to progressively larger coefficient data sets as the detail level of the image
`
`increases. Consequently, there is an inherently increasing latency in resolving finer levels of
`
`detail.
`
`An image visualization system proposed by Yapet al., U.S. Pat. No. 6,182,114,
`
`overcomes someofthe foregoing problems. The Yapet al. system also employs a progressive
`
`encoding transform to compress the image transfer stream. The transform also operates on a
`
`subdivided image, but the division is indexed to the encoding level of the transform. The
`
`encoded transform coefficient data sets are, therefore, of constant size, which supports a modest
`
`improvementin the algorithmic performanceof the inverse transform operation required on the
`
`client.
`
`Yap et al. adds utilization of client image panning or other image pointing input
`
`information to support a foveation-based operator to influencethe retrieval order of the
`
`subdivided image blocks. This two-dimensional navigation information is used to identify a
`
`foveal region that is presumed to be the gaze point of a client system user. The foveation
`
`operator defines the corresponding image block as the center point of an orderedretrieval of
`
`coefficient sets representing a variable resolution image. The gaze point image block represents
`
`the area of highest image resolution, with resolution reduction as a function of distance from the
`
`gaze point determined bythe foveation operator. This technique thus progressively builds image
`
`resolution at the gaze point and succeedingly outward based on a relatively compute intensive
`
`function. Shifts in the gaze point can be respondedto with relative speed by preferentially
`
`retrieving coefficient sets at and near the new foveal region.
`
`5 of 150
`
`Microsoft Corp. Exhibit 1016
`
`5 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-4-
`
`Significant problems remain in permitting the convenient and effective use of complex
`
`images by manydifferent types of client systems, even with the improvements provided by the
`
`various conventional systems. In particular, the implementation of conventional image
`
`visualization systems is generally unworkable for smaller, often dedicated or embedded,clients
`
`where use of image visualization would clearly be beneficial. Conventional approaches
`
`effectively presumethat client systems have an excess of computing performance, memory and
`
`storage. Small clients, however, typically have restricted performance processors with possibly
`
`no dedicated floating-point support, little general purpose memory, and extremely limited
`
`persistent storage capabilities, particularly relative to common imagesizes. A mobile computing
`
`device such as mobile phone, smart phone, tablet and or personal digital assistant (PDA)is a
`
`characteristic small client. Embedded, low-cost kiosk, automobile navigation systems and or
`
`Internet enabled I connected TV are other typical examples. Such systemsare not readily
`
`capable, if at all, of performing complex, compute-intensive Fourier or wavelet transforms,
`
`particularly within a highly restricted memory address space.
`
`As a consequenceof the presumption that the client is a substantial computing system,
`
`conventional image visualization systems also presumethat the client is supported by a complete
`
`operating system. Indeed, many expect and require an extensive set of graphics abstraction
`
`layers to be provided by the client system to support the presentation of the delivered image data.
`
`In_general, these abstraction layers are conventionally considered required to handle the mapping
`
`of the image data resolution to the display resolution capabilities of the client system. Thatis,
`
`resolution resolved image data provided to the client is unconstrained by any limitation in the
`
`client system to actually display the corresponding image. Consequently, substantial processor
`
`performance and memory can be conventionally devoted to handling image data thatis not or
`
`cannot be displayed.
`
`Another problem is that small clients are generally constrained to generally to very
`
`limited network bandwidths, particularly when operating under wireless conditions. Such
`
`limited bandwidth conditions may exist due to either the direct technological constraints dictated
`
`by the use of a low bandwidth data channel or indirect constraints imposed onrelatively high-
`
`bandwidth channels by high concurrent user loads. Cellular connected PDAs and webphonesare
`
`examples of small clients that are frequently constrained by limited bandwidth conditions. The
`
`6 of 150
`
`Microsoft Corp. Exhibit 1016
`
`6 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`_5-
`
`conventionally realizable maximum networktransmission bandwidth for such small devices may
`
`range from below onekilobit per second to several tens of kilobits per second. While Yapetal.
`
`states that the described system can work over low bandwidthlines, little more than utilizing
`
`wavelet-based data compression is advanced as permitting effective operation at low
`
`communications bandwidths. While reducing the amountof data that must be carried from the
`
`server to the client is significant, Yap et al. simply relies on the data packet transfer protocols to
`
`provide for an efficient transfer of the compressed image data. Reliable transport protocols,
`
`however, merely mask packetlosses and the resultant, sometimes extended recovery latencies.
`
`Whensuch coverederrors occur, however, the aggregate bandwidth of the connection is reduced
`
`and the client system can stall waiting for further image data to process.
`
`Consequently, there remains a need for an image visualization system that can support
`
`small client systems, place few requirements on the supporting client hardware and software
`
`resources, and efficiently utilize low to very low bandwidth network connections.
`
`Summary
`
`Thus, a general purpose of the present invention is to provide an efficient system and
`
`methods of optimally presenting image data on client systems with potentially limited processing
`
`performance, resources, and communications bandwidth.
`
`This is achieved in the present invention by providing for the retrieval of large-scale
`
`images over network communications channels for display on a client device by selecting an
`
`update image parcel relative to an operator controlled image viewpoint to display via the client
`
`device. A request is prepared for the update image parcel and associated with a request queue
`
`for subsequent issuance over a communications channel. The update imageparcel is received
`
`from the communications channel and displayed as a discrete portion of the predetermined
`
`image. The update image parcel optimally has a fixed pixel array size, is received in a single and
`
`or plurality of network data packets, and were the fixed pixel array may be constrained to a
`
`resolution less than or equalto the resolution of the client device display.
`
`An advantageof the present invention is that both image parcel data requests and the
`
`rendering of image data are optimized to address the display based on the display resolution of
`
`the client system.
`
`7 of 150
`
`Microsoft Corp. Exhibit 1016
`
`7 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-6-
`
`Another advantage of the present inventionis that the prioritization of image parcel
`
`requests is based on an adaptable parameter that minimizes the computational complexity of
`
`determining request prioritization and, in turn, the progressive improvementin display resolution
`
`within the field of view presented on a client display.
`
`A further advantage of the present invention is that the client software system requires
`
`relatively minimal client processing powerand storage capacity. Compute intensive numerical
`
`calculations are minimally required and image parcel data is compactly stored in efficient data
`
`structures. The client software system is very small and easily downloaded to conventional
`
`computer systems or embeddedin conventional dedicated function devices, including portable
`
`devices, such as PDAs, tablets and webphones.
`
`Still another advantage of the present invention is that image parcel data requests and
`
`presentation can be readily optimized to use low to very low bandwidth network connections.
`
`The software system of the present invention providesfor re-prioritization of image parcel data
`
`requests and presentation in circumstances wherethe rate of point-of-view navigation exceeds
`
`the data requestrate.
`
`Yet another advantage of the present invention is that image parcel data rendering is
`
`performed without requiring any complex underlying hardware or software display subsystem.
`
`The client software system of the present invention includes a bit-map rendering engine that
`
`draws directly to the video memoryof the display, thus placing minimal requirements on any
`
`underlying embedded or disk operating system and display drivers. Complex graphics and
`
`animation abstraction layers are not required.
`
`Still another advantage of the present invention is that image parcel block compression is
`
`used to obtain fixed size transmission data blocks. Image parcel data is recoverable from
`
`transmission data using a relatively simple client decompression algorithm. Using fixed size
`
`transmission data blocks enables image data parcels to be delivered to the client in bounded time
`
`frames.
`
`A yet further advantage of the present invention is that multiple data forms can be
`
`transferred to the client software system for concurrent display. Array overlay data, correlated
`
`positionally to the image parcel data and generally insensitive to image parcel resolution, can be
`
`8 of 150
`
`Microsoft Corp. Exhibit 1016
`
`8 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-7-
`
`initially or progressively provided to the client for parsing and parallel presentation on a client
`
`display image view.
`
`Brief Description of the Drawings
`
`These and other advantages and features of the present invention will become better
`
`understood upon consideration of the following detailed description of the invention when
`
`considered in connection with the accompanying drawings, in which like reference numerals
`
`designate like parts throughout the figures thereof, and wherein:
`
`FIG. 1 depicts a preferred system environment within which various embodimentsof the
`
`present invention can be utilized;
`
`FIG. 2 is a block diagram illustrating the preparation of image parcel and overlay data set
`
`that are to be stored by and served from a network server system in accordance with a preferred
`
`embodimentof the present invention;
`
`FIG. 3 is a block diagram of a client system image presentation system constructed in
`
`accordance with a preferred embodimentofthe present invention;
`
`FIG. 4 provides a data block diagram illustrating an optimized client image block
`
`processing path constructed in accordance with a preferred embodimentof the present invention;
`
`FIG. 5 is a process flow diagram showing a main processing thread implemented in a
`
`preferred embodimentof the present invention;
`
`FIG. 6 provides a process flow diagram showing a network request thread implemented
`
`in a preferred embodimentof the present invention;
`
`FIG. 7 provides a process flow diagram showing a display image rendering thread
`
`implemented in a preferred embodimentof the present invention;
`
`FIG. 8 provides a process flow diagram showing the parcel map processing performed
`
`preliminary to the rendering of image data parcels in accordance with a preferred embodiment of
`
`the present invention;
`
`9 of 150
`
`Microsoft Corp. Exhibit 1016
`
`9 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-8-
`
`FIG. 9 provides a process flow diagram detailing the rendering and progressive
`
`prioritization of image parcel data download requests in accordance with a preferred embodiment
`
`of the present invention; and
`
`FIG. 10 provides a process flow diagram detailing the determination of an optimal detail
`
`level for image parcel presentation for a current viewing frustum in accordance with a preferred
`
`embodimentof the present invention.
`
`Detailed Description of One or More Embodiments
`
`The preferred operational environment 10 of the present invention is generally shown in
`
`FIG. 1. A network server system 12, operating as a data store and server of imagedata, is
`
`responsive to requests received through a communications network, such as the Internet 14
`
`generally and various tiers of internet service providers (ISPs) including a wireless connectivity
`
`provider 16. Client systems, including conventional workstations and personal computers 18 and
`
`smaller, typically dedicated function devices often linked through wireless network connections,
`
`such as PDAs, webphones 20, and automobile navigation systems, source image requests to the
`
`network server 12, provide a client display and enable imagenavigational input by a user of the
`
`client system. Alternately, a dedicated function client system 20 may be connected through a
`
`separate or plug-in local network server 22, preferably implementing a small, embedded Web
`
`server, to a fixed or removable storage local image repository 24. Characteristically, the client
`
`system 18, 20 displays are operated at somefixed resolution generally dependent on the
`
`underlying display hardware of the client systems 18, 20.
`
`The image navigation capability supported by the present invention encompassesa
`
`viewing frustum placed within a three-dimensional space over the imaged displayed onthe client
`
`18, 20. Client user navigational inputs are supported to control the x, y lateral, rotational and z
`
`height positioning of the viewing frustum overthe image as well as the camera angle of
`
`incidencerelative to the plane of the image. To effect these controls, the software implemented
`
`on the client systems 18, 20 supports a three-dimensional transform of the image data provided
`
`from the server 12, 22.
`
`In accordance with the preferred embodiments of the present invention, as generally
`
`illustrated in FIG. 2, a network image server system 30 stores a combination of source image
`
`10 of 150
`
`Microsoft Corp. Exhibit 1016
`
`10 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`_9-
`
`data 32 and source overlay data 34. The source image data 32 is typically high-resolution bit-
`
`map raster map andorsatellite imagery of geographic regions, which can be obtained from
`
`commercial suppliers. The overlay image data 34 is typically a discrete data file providing
`
`image annotation information at defined coordinatesrelative to the source image data 32. In the
`
`preferred embodiments of the present invention, image annotations include, for example, street,
`
`building and landmark names, as well as representative 2 and 3D objects, graphical icons, decals,
`
`line segments, and or text and or other characters, graphics and or other media.
`
`The network image server system 30 preferably pre-processes the source image data 32
`
`and or source overlay data 34 to forms preferred for storage and serving by the network server
`
`12, 22. The source image data 32 is preferably pre-processed to obtain a series Kin of
`
`derivative images of progressively lower image resolution. The source image data 32,
`
`corresponding to the series image Ko, is also subdivided into a regular array such that each
`
`resulting image parcel of the array has for example a 64 by 64 pixel resolution where the image
`
`data has a coloror bit per pixel depth of 16 bits, which represents a data parcel size of 8K bytes.
`
`The resolution of the series K1-n of derivative imagesis preferably related to that of the source
`
`image data 32 or predecessor imagein the series by a factor of four. The array subdivision is
`
`likewise related by a factor of four such that each imageparcelis of a fixed 8K byte size.
`
`In the preferred embodimentof the present invention, the image parcels are further
`
`compressed and stored by the network server 12, 22. The preferred compression algorithm may
`
`implement for example a fixed 4:1 compression ratio such that each compressed and stored
`
`image parcel has a fixed 2K byte size. The image parcels are preferably stored inafile of
`
`defined configuration such that any imageparcel can be located by specification of a Kp, X, Y
`
`value, representing the imageset resolution index D and corresponding image array coordinate.
`
`In other implementations, the image array dimensions (which as 64 X 64 above) may be
`
`powersof two so that the imagearray can be used in texture mappingefficiently. To
`
`accommodate different data parcel size than the 2KByte associated with 64x64 pixel parcel
`
`dimension described above and other communication protocol and overhead requirements, to
`
`accommodate transmission through other than a 3KByte per second transmission channel, the
`
`present invention may use larger compressionratios that takes, for example, a 128x128 or
`
`11 of 150
`
`Microsoft Corp. Exhibit 1016
`
`11 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-10-
`
`256x256 pixel parcel dimension and compresses it to meet the 3KByte per second transmission
`
`channel, or other communication bandwidth used to stream the parcel.
`
`The system may also accommodate different and larger data parcel sizes as transmission
`
`protocols, compression ratio achieved and micro-architectures of the client computers change.
`
`For purposes above, the data content was a pixel array representing image data. Where the data
`
`parcel content is vector, text or other data that may subject to different client system design
`
`factors, other parcel sizes may be used. Furthermore, the parcel sizes can be different between
`
`the server and the client. For example the server may create parcels or hold parcels, for
`
`streaming with 256x256 pixel parcel dimension and the client my render them as 64x64. In
`
`addition, parcels sizes on different servers may vary from oneserver to another and from the
`
`client side rendering. In the system, each grid is treated as a sparse data array that can be
`
`progressively revised to increase the resolution of the grid and therebythe levelof detail
`
`presented bythe grid.
`
`The source overlay data 34 is preferably pre-processed 36 into either an open XML
`
`format, such as the Geography Markup Language (GML), which is an XMLbased encoding
`
`standard for geographic information developed by the OpenGIS Consortium (OGC;
`
`www.opengis.org), or a proprietary binary representation. The XML/GMLrepresentationis
`
`preferred as permitting easier interchange between different commercial entities, while the
`
`binary representation is preferred as more compact and readily transferable to a client system 18,
`
`20. In both cases, the source overlay data 34 is pre-processed to contain the annotation data
`
`preferably in a resolution independent form associated with a display coordinate specification
`
`relative to the source image data 32. The XML, GMLorbinary overlay data may be compressed
`
`prior to storage on the network server 12, 22.
`
`The preferred architecture 40 of a client system 18, 20, for purposes of implementing the
`
`present invention, is shown in FIG. 3. The architecture 40 is preferably implemented by
`
`software plug-in or application executed bythe client system 18, 20 and that utilizes basic
`
`software and hardwareservices provided by the client system 18, 20. A parcel request client 42
`
`preferably implements an HTMLclient that supports HTML-based interactions with the server
`
`12, 22 using the underlying network protocol stack and hardware network interface provided by
`
`the client systems 18, 20. A central parcel processing control block 44 preferably implements
`
`12 of 150
`
`Microsoft Corp. Exhibit 1016
`
`12 of 150
`
`Microsoft Corp. Exhibit 1016
`
`
`
`-ll-
`
`the client process and control algorithms. The control block 44 directs the transfer of received
`
`image parcels and XML/GML/binaryoverlay data to a local parcel data store 46. Local parcel
`
`data store 46 mayalso act for example as local cache weather the entire data or part of it is in
`
`dynamic and/orstatic cache. Preferably image data parcels are stored in conventional quad-tree
`
`data structures, where tree nodes of depth D correspondto the stored imageparcels of a
`
`derivative image of resolution KD. The XML/GML/binary overlay data is preferably stored as a
`
`data object that can be subsequently read by an XML/GML/binary parser implemented as part of
`
`the control block 44.
`
`The control block 44 is also responsible for decompressing and directing the rendering
`
`of image parcels to a local display by a rendering engine