(12) Ulllted States Patent
`Du et a].
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,734,729 B2
`Jun. 8, 2010
`
`US007734729B2
`
`(54) SYSTEM AND METHOD FOR OBTAINING
`INFORMATION RELATING TO AN ITEM OF
`COMMERCE USING A PORTABLE IMAGING
`DEVICE
`
`(75) Inventors: Kavin Du, Seattle, WA (US); Milen
`Nankov, Redmond, WA (US)
`
`(73) Assignee: agglzon Technologies, Inc., Reno, NV
`
`EP
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days,
`
`(21) App1_ NO; 10/749,473
`
`.
`(22) Frled:
`
`Dec. 31, 2003
`
`(65)
`
`PI‘iOI‘ PllblicatiOIl Data
`Us 2005/0198095 A1
`Sep. 8, 2005
`
`(51) Int CL
`(200601)
`G06F 15/16
`(200601)
`G06F 1 7/30
`(200601)
`G06Q 30/00
`(52) US. Cl. ....................... .. 709/219; 709/203; 705/26;
`705/273 707/10
`0f Classi?cation Search ............... ..
`_
`_
`709/203, 219; 705/26,_27; 707/ 10
`See aPPhCaUOn ?le for Complete Search 11151013’ -
`References Cited
`
`(56)
`
`US. PATENT DOCUMENTS
`
`5,818,028 A * 10/1998 Meyerson et a1. .... .. 235/472.01
`5,848,413 A 12/1998 Wolff
`5,918,213 A *
`6/1999 Bernard et a1. .............. .. 705/26
`5,933,829 A
`8/1999 Durst et a1.
`5,978,773 A 11/1999 HudetZ et a1.
`
`6,053,408 A *
`6,108,656 A
`6,199,048 B1
`6,434,561 B1
`6,542,933 B1
`
`4/2000 Stoner ................. .. 235/462.22
`8/2000 Durst et a1.
`3/2001 HudetZ 6t ?l~
`8/2002 Durst, Jr. et a1.
`4/2003 Durst, Jr. et a1.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`1465071 A2 100004
`(Continued)
`
`OTHER PUBLICATIONS
`Baig, E.C., “Radical GiZmos Arrive at Demo Conference,” USA
`Today.‘ CyberSpeak, Feb. 13, 2002, <WWW.usatoday.com/tech/col
`umnist/2002/02/13/baig.htm> [retrieved Oct. 15,2004].
`
`d
`C t'
`( on “we )
`Primary ExamineriRanodhi N Serrao
`(74) Attorney, Agent, or Firm4Christensen O’Connor
`Johnson Kmdness PLLC
`
`ABSTRACT
`(57)
`A method, system, and apparatus are provided for allowing
`users to readily obtain information associated With a selected
`item from a remote location. More speci?cally, a user at the
`location Ofthe ?rst entity Operates a portable imaging device
`to capture an image of identifying data, such as a barcode, that
`identi?es a Selected item_ The Captured image is then Com
`municated to a server operated by a second entity that is
`different than the ?rst entity to obtain item information (e. g.,
`price, availability, etc.) associated With the selected item. The
`item information is communicated back to the portable imag
`ing device for display to the user While the user remains at the
`location of the ?rst entity. In other embodiments, the infor
`mation extracted from the captured image may also be used to
`forecast future purchasing activity for the selected item.
`
`35 Claims, 6 Drawing Sheets
`
`SERVER
`
`103
`
`Page 1 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US 7,734,729 B2
`Page 2
`
`US. PATENT DOCUMENTS
`
`4/2003 Ogasawara
`6,543,052 B1
`6,611,881 B1* 8/2003 Gottfurcht et al. .......... .. 710/18
`6,616,049 B1
`9/2003 Barkan et al.
`6,832,729 B1 * 12/2004 Perry et al. .......... .. 235/47201
`2001/0011276 A1
`8/2001 Durst, Jr. et al.
`2001/0032252 A1 10/2001 Durst, Jr. et al.
`2001/0044747 A1* 11/2001 Ramachandran et al. .... .. 705/16
`2001/0047428 A1 11/2001 Hunter
`2001/0054008 A1 12/2001 Miller et al.
`2002/0002496 A1
`1/2002 Miller et al.
`2002/0003166 A1
`1/2002 Miller et al.
`2002/0007307 A1
`1/2002 Miller et al.
`2002/0007331 A1* 1/2002 Lo et a1. ..................... .. 705/36
`2002/0016750 A1
`2/2002 Attia
`2002/0022963 A1
`2/2002 Miller et al.
`2002/0022992 A1
`2/2002 Miller et al.
`2002/0022993 A1
`2/2002 Miller et al.
`2002/0022994 A1
`2/2002 Miller et al.
`2002/0022995 A1
`2/2002 Miller et al.
`2002/0023959 A1
`2/2002 Miller et al.
`2002/0026357 A1
`2/2002 Miller et al.
`2002/0026358 A1
`2/2002 Miller et al.
`2002/0026369 A1
`2/2002 Miller et al.
`2002/0029181 A1
`3/2002 Miller et al.
`2002/0030105 A1
`3/2002 Miller et al.
`2002/0046093 A1
`4/2002 Miller et al.
`2002/0065717 A1
`5/2002 Miller et al.
`2002/0068991 A1* 6/2002 FitZsimmons, Jr. ........ .. 700/214
`2002/0072970 A1
`6/2002 Miller et al.
`2002/0082931 A1* 6/2002 Siegel et al. ................ .. 705/26
`2002/0107861 A1* 8/2002 Clendinning et al. ...... .. 707/101
`2002/0125324 A1* 9/2002 Yavid et al. .
`. 235/462.45
`370/265
`2003/0063575 A1* 4/2003 Kinjo ....... ..
`
`2003/0115152 A1* 6/2003 Flaherty . . . . .
`. . . .. 705/65
`2003/0121981 A1* 7/2003 Slutsky et al. ........ .. 235/462.45
`2003/0208396 A1 11/2003 Miller et al.
`2005/0015311 A1* 1/2005 FrantZ et a1. ................ .. 705/26
`2005/0049907 A1
`3/2005 Kumar et a1.
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`W0
`W0
`
`0142894 A1
`WO 02/15118 A1
`WO 02/15120 A1
`
`6/2001
`2/2002
`2/2002
`
`OTHER PUBLICATIONS
`
`“Bangonet: Camera-Enabled Mobiles and PDAs Scan Barcodes for
`Instant Access to WAP,” Cambridge 3G, Sep. 10, 2002, <WWW.
`
`cambridge3g.com/pooled/articles/BFiNEWSART/vieW.asp?Q:
`BFiNEWSARTi36234> [retrieved Oct. 15, 2004].
`“Bookstores Tap AmaZon.com Info,” Jul. 25, 2003, Channel
`NeWsAsia,
`<WWW.channelneWsasia.com/stories/technologynews/
`vieW/45292/1/html> [retrieved Jul. 28, 2003].
`“Camera Phone Clicks as a Credit Card,” Jul. 30, 2003, CNETAsia
`[online], [retrieved Aug. 1, 2003].
`“Camera Phones Used to Check Football Membership Cards,” Jan.
`28,
`2004,
`<WWW.Celll1l3I-I1GWS.COIIl/SIOI'Y/l0508ipf1IlLShUI1l>
`[retrieved Feb. 17, 2004].
`Eye,”
`the
`Meets
`Than
`“Finnish
`LaplandiMore
`e.FinlandiTechnology Serving People, Mar. 20, 2003, We
`?nland.?/netcomm/news/showarticle.asp?intNWSAID:13208
`[retrieved Oct. 15, 2004].
`GeraCap BarReader, GeraCap Oy (Symbian Platinum Partner),
`Seinajoki, Finland, WWW.barreader.com, 2-page brochure.
`Gilbert, A., “MIT to Uncork Futuristic Bar Code,” Aug. 29, 2003,
`CNE T News. corn.‘ Tech News First [online], [retrieved Sep. 3, 2003].
`“Lavasphere SDKiSO?WBIG Enabling Mobile Camera Phones to
`Read Bar Codes and 2d Codes,” Press Release, © 2004 Gavitec
`AGiMobile Digit, Feb. 21, 2002 [retrieved Oct. 15, 2004].
`Marek, S., “Camera Phones Capture Bar Codes,” Feb. 15, 2004,
`Wireless
`Week,
`<WWW.WirelessWeek.com/indeX.
`asp?layout:articlePrint&aIticleID:CA381641> [retrieved Feb. 17,
`2004].
`Red Ferret Journal, The [online blog], Feb. 23, 2002, <WWW.enorgis.
`com/arc20020217.htm> [retrieved Oct. 15, 2004], 7 pages.
`ScanImageTM 1470B, Intermec, wwwintermec.com/cgi-bin/ASP/
`PrintiProductsShoWdetail .asp?Category:SCAN&Family:SCA. ..
`[retrieved Dec. 30, 2003], 3 pages.
`“ScoutPal.com: Advanced Tools for Amazon Sellers,” © 2002-2005
`ScannerPal, Inc., <WWW.scoutpal.com> [retrieved Sep. 9, 2005].
`Soto Ouchi, M., “Small Sellers Get the AmaZon.com Edge,” Jun. 7,
`2004, The Seattle Times, <seattletimes.nWsource.com/html/
`businesstechnology/2001949623iamaZonWeb07.h5ml> [retrieved
`Sep. 9, 2005].
`Sundgot, J. (ed), “Point and Receive With a Twist,” infoSync World,
`Jan.
`8,
`2002,
`<WWW.infosyncWorld.com/system/print.
`php?id:2131> [retrieved Oct. 15, 2004].
`“VisionSpy 2300MiThe Off-the-Shelf Code Reader to Read
`Barcodes and 2d Codes From Any Mobile Phone Display,” Press
`Release, © 2004 Gavitec AGiMobile Digit, Feb. 21, 2002 [retrieved
`Oct. 15,2004].
`Supplementary European Search Report mailed Jan. 23, 2009, issued
`in corresponding Application No. EP 04815119, ?led Dec. 22, 2004,
`3 pages.
`
`* cited by examiner
`
`Page 2 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US. Patent
`
`Jun. 8, 2010
`
`Sheet 1 of6
`
`US 7,734,729 B2
`
`SERVER
`
`103
`
`5%
`
`@
`
`w 0 1
`
`102C
`
`Fig. 1.
`
`Page 3 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US. Patent
`
`Jun. 8, 2010
`
`Sheet 2 of6
`
`US 7,734,729 B2
`
`
`~/\ @556
`NE 595 uzawmuoi
`
`
`Page 4 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`4|.n64|.aPs”U
`
`00
`
`0
`
`3whS
`
`2B927,437,7SU
`
`.muznv$5
`
`6b<m>§_SSo
`
`J55:32
`mm9$:ibE
`
`mwe§E:>:2,E.:be\,Sm>:
`
`
`
`DZNRVKNEQ
`
`239%
`
`zmmkcmmNW5
`
`UZNWMWDQEN
`
`Zbhm—\D~N.~.~u\
`
`$2
`
`MN§~
`
`DN§~
`
`QNQN
`
`NVNHZND
`
`
`
`HE/Suzaunmufifi
`
`«NME
`
`Page 5 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`Page 5 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US. Patent
`
`Jun. 8, 2010
`
`Sheet 4 0f 6
`
`US 7,734,729 B2
`
`300x
`
`STARTINFORIWATION
`PROCESSING METHOD
`
`OBTAIN IMAGE
`(FIG. 4A.)
`
`V
`
`COMMUNICA TE IIMAGE TOA
`SERVER
`
`V
`
`EXTRACTIDENTIFYING DATA
`FROM IAMGE
`
`V
`
`OBTAINITEM INFORAM TION
`USING IDENTIFYING DA TA
`
`V
`COMMUNICA TE AND DISPLA Y
`ITEM INFORMA TION
`(FIG. 4B.)
`
`V
`PROCESS INF ORIlM T I ON T O
`GENERA TE FORECASTING
`DA TA
`
`v
`END INF ORIWA T I ON
`PROCESSING METHOD
`
`Fig. 3.
`
`Page 6 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US. Patent
`
`Jun. 8, 2010
`
`Sheet 5 of6
`
`US 7,734,729 B2
`
`411
`
`400
`
`Page 7 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US. Patent
`
`Jun. 8, 2010
`
`Sheet 6 of6
`
`US 7,734,729 B2
`
`400
`
`Fig. 4B.
`
`Page 8 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US 7,734,729 B2
`
`1
`SYSTEM AND METHOD FOR OBTAINING
`INFORMATION RELATING TO AN ITEM OF
`COMMERCE USINGA PORTABLE IMAGING
`DEVICE
`
`FIELD OF THE INVENTION
`
`The present invention is directed to the ?elds of electronic
`commerce and software, and in particular, to a method and
`system for communicating and obtaining information related
`to an item of commerce using a portable imaging device.
`
`BACKGROUND OF THE INVENTION
`
`In the retail industry, effective and timely communication
`of product information is essential to both merchants and
`consumers. For consumers, there is a need for effective and
`timely communication of product information for those in
`search of the best price for a desired product. For merchants,
`there is a need for effective and timely communication of
`product information for effective advertising of competitive
`prices, sales, promotions, neW products, etc. In addition, both
`merchants and consumers bene?t When merchants are able to
`collect and analyZe sales and other information to forecast
`future sales. Accurate sales forecasting enables a merchant to
`effectively select and stock products to meet consumer
`demands. In response to a long-standing need for improved
`communication betWeen merchants and consumers, many
`companies have contributed to the groWth and development
`of the Internet and electronic commerce systems. Given the
`current state of the art, it is Well knoWn that existing electronic
`commerce systems have greatly enhanced many aspects of
`the retail industry and commerce in general.
`While electronic commerce has improved the Way con
`sumers shop for products, services, or other commerce items,
`existing systems present some limitations. For instance, most
`consumers have access to detailed product information only
`at their home or of?ce because they search for such informa
`tion using a personal computer or a television set-top box.
`Consumers desiring to search for such information remotely
`must use a Wireless-enabled personal digital assistant (PDA)
`or a mobile telephone. Since most PDAs and mobile phones
`require the tedious task of entering data by the use of an
`abbreviated keypad, users may not be able to easily or quickly
`search for product information.
`Given the above-described limitations, consumers cannot
`readily compare prices of on-line products While shopping in
`a local retail store. For example, if a consumer is shopping at
`a retail book store, he or she may be interested in comparing
`the price of a particular book to prices offered by on-line
`retailers. To do so using a portable Wireless device, the con
`sumer Would have to manually enter one or more pieces of
`identifying information, e.g., the title, author, ISBN, etc., in
`order to search for the book. Such data entry can be a long and
`tedious process. Alternatively, the consumer can Write doWn
`all of the information and look up the book When they return
`to their home or o?ice computer. Such method is inherently
`inef?cient, hoWever, since the consumer may then have to
`return to the store if the price at the store is subsequently
`found to be loWer than prices available online.
`In addition to the above-described limitations, existing
`systems are also limited to speci?c types of data collection
`used for purchase forecasting. As Will be appreciated by those
`of ordinary skill in the art, many merchants ?nd it helpful to
`have an accurate forecast of future purchasing activity for
`products or other commerce items.
`
`20
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`SUMMARY OF THE INVENTION
`
`The present invention provides interactive merchandising
`programs that, in some embodiments, are particularly suited
`for improved communication of information associated With
`commerce items. In accordance With one aspect, the present
`invention provides a system and method for alloWing con
`sumers to obtain information associated With a particular
`item, e.g., price, availability, revieWs, etc., by the use of a
`portable imaging device, such as a digital camera, mobile
`telephone, portable computing device (e.g., PDA), etc., hav
`ing a component capable of capturing an image. In one exem
`plary embodiment, a consumer may use the camera of a
`mobile telephone to capture an image of a desired item. The
`image is communicated to one or more servers, and data
`identifying the desired item is extracted from the image. The
`identifying data extracted from the image may be information
`such as a Universal Product Code (UPC), text on the box of
`the product, indicia on the product itself, etc. The identifying
`data may be extracted from the image using any one of many
`standard image recognition and processing programs. The
`data identifying the selected item is then used to query one or
`more resources to obtain item information related to the
`selected item, such as the item’ s price, ratings or revieWs, and
`availability. The obtained information is then communicated
`to the mobile phone. The item information may be displayed
`on a display screen of the mobile phone or may be issued as an
`audible signal via a speaker of the mobile phone. Among
`many other bene?ts, the present invention alloWs users to
`readily obtain item information, even if they are located in a
`remote location.
`The foregoing embodiment describes one implementation
`of a method for communicating information regarding a
`desired item. The method includes obtaining an image of an
`item using an imaging device, Wherein the image contains
`identifying data associated With the item as provided by a ?rst
`entity (e.g., a brick-and-mortar retail store). The method then
`extracts identifying data for the item from the image. Item
`information, such as the item’s price, availability, etc., are
`obtained from a resource by a second entity (e.g., an online
`retail store) using the identifying data. The second entity is
`different than the ?rst entity and may be situated at a location
`remote from the ?rst entity. The item information is then
`communicated from the second entity back to the imaging
`device for communication through an output of the imaging
`device, such as a display or a speaker, While the imaging
`device (and user) remain present at the location of the ?rst
`entity.
`In accordance With another aspect of the present invention,
`data received from a number of different portable imaging
`devices is collected and processed to facilitate sales forecast
`ing analysis. Data is collected Whenusers of portable imaging
`devices submit a request for information related to an item.
`The collected data may be used to generate sales forecasting
`reports to determine market demand for various items. In one
`example, a consumer may use the camera of a mobile phone
`to capture an image of a barcode or other identifying data
`associated With a desired item. The image is then communi
`cated to a server, Where the identifying data is extracted from
`the captured image to identify the selected item. The number
`of times that all consumers take a picture of a particular item
`is stored in a database and used to generate sales forecast
`reports. Such reports may assist merchants accurately fore
`cast future purchasing activity for products or other com
`merce items. The fact that an individual consumer identi?ed
`a particular item may also be stored in a database associated
`With the consumer and used to predict the purchasing behav
`
`Page 9 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US 7,734,729 B2
`
`3
`ior of the consumer, thereby improving future shopping expe
`riences for the consumer by anticipating, as best possible, the
`consumer’s purchasing interests.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The foregoing aspects and many of the attendant advan
`tages of this invention will become more readily appreciated
`as the same become better understood by reference to the
`following detailed description, when taken in conjunction
`with the accompanying drawings, wherein:
`FIG. 1 is a pictorial diagram of a number of devices con
`nected to a network, including a number of imaging devices in
`communication with a server;
`FIG. 2A is a schematic block diagram of an exemplary
`computer architecture for the server depicted in FIG. 1;
`FIG. 2B is a schematic block diagram of an exemplary
`architecture for one imaging device in FIG. 1;
`FIG. 3 is a ?ow diagram illustrating an exemplary method
`for communicating price information and projecting future
`purchasing activity for an item;
`FIG. 4A is a pictorial diagram illustrating an exemplary
`imaging device for obtaining an image of information iden
`tifying an item of commerce; and
`FIG. 4B is a pictorial diagram illustrating an exemplary
`display of price and product information related to an item of
`commerce on the imaging device shown in FIG. 4A.
`
`DETAILED DESCRIPTION
`
`The following description of the present invention pro
`vides an overview of a computer system and method for
`communicating information regarding a selected item by
`using an imaging device in communication with a server. In
`accordance with one aspect, the present invention provides a
`system and method that allows consumers to obtain informa
`tion associated with a particular item, e. g., price, availability,
`ratings, reviews, etc ., by using an integrated portable imaging
`device, such as a digital camera, mobile telephone, portable
`computing device (e.g., PDA), etc. For example, in one
`implementation of the invention, a consumer at the location of
`a ?rst entity (e.g., a brick-and-mortar retail store) may use the
`camera of a mobile telephone to capture an image of a desired
`item as provided by the ?rst entity. The image is then com
`municated to a server operated by a second entity (e.g., an
`online retail store) that is different than the ?rst entity. The
`image is analyZed and data identifying the selected item is
`extracted from the image. Such identifying data may include
`the Universal Product Code (UPC), text on the packaging of
`the product, indicia on the product itself, etc. The identifying
`data may be extracted from the image using any one of many
`standard image recognition and analysis programs. The iden
`tifying data is then used to query one or more resources to
`obtain information related to the selected item, such as the
`item’ s price and availability. The obtained item information is
`communicated from the second entity to the mobile phone,
`where it may be displayed on a display screen or issued as an
`audible signal via a speaker, while the mobile phone (and the
`user) remain at the location of the ?rst entity. In addition to
`communicating the item information to the mobile phone,
`information related to the selected item is processed by the
`server to generate sales forecast reports. The illustrative
`examples described herein are not intended to be exhaustive
`or to limit the invention to the precise forms disclosed. Simi
`larly, any process steps described herein may be interchange
`able with other steps, or several combinations of steps, in
`order to achieve the same result.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`Referring to FIG. 1, the following description is intended to
`provide an exemplary overview of one environment in which
`the invention may be implemented. In this environment, a
`plurality of imaging devices 102A, 102B, 102C, 102D
`(present at the location of one or more ?rst entities) are
`con?gured to electronically communicate with a server 103
`operated by a second entity via a network 100. Although a
`speci?c network con?guration is shown in FIG. 1, it will be
`appreciated that the imaging devices may be con?gured to
`communicate with the server 103 through other communica
`tion con?gurations. For example, an imaging device may
`communicate data with a computer (not shown) via a port,
`such as a USB or a IEEE 1394 port. In turn, the computer may
`then communicate the data communicated through the port
`with the server 103 via a network or other data connection.
`The server 103 may be a computer that is associated with a
`store or other entity that provides goods and/or services to
`others, whether retail, wholesale or otherwise, or any other
`entity that provides information about goods available to
`consumers. The network 100 may be a local area network
`(LAN) or a larger network, such as a wide area network
`(WAN) or the Internet. In FIG. 1, the imaging devices are
`illustrated as portable computers 102A and 102C, and mobile
`telephones 102B and 102D. However, the imaging devices
`102A, 102B, 102C, 102D shown in FIG. 1 may take the form
`of any one of a number of different computer products that
`includes appropriate hardware and software components for
`capturing an image. An image, in this context, is a two
`dimensional array of pixels, and does not include one-dimen
`sional data obtained from a laser scanner or light pen. For
`example, the imaging device may be a digital camera, a
`mobile telephone, a portable computing device, or any other
`device having a component capable of capturing an image.
`One example of a digital camera that may be used in the
`invention is Fuji Fine Pix 30i that is capable of immediately
`downloading and communicating an image using a compat
`ible cell phone. Examples of mobile telephones with imaging
`components include the Motorola T722i, Samsung e715,
`Nokia 3650, and Ericsson T616 phones. An example of a
`portable computing device with an imaging component is the
`Handspring Treo 600 available from palmOne, Inc. Obvi
`ously, these imaging devices should be considered as exem
`plary and not limiting to the invention. As will also be appre
`ciated by those skilled in the art, the con?guration shown in
`FIG. 1 provides a simpli?ed example for one implementation
`of the present invention. The present invention is not limited
`to this type of con?guration.
`As will be described in more detail below, a consumer can
`use an imaging device, such as device 102B, to capture an
`image of an item, with identifying data such as a barcode or
`product title of the item, while shopping at a local retail store.
`The imaging device then communicates the captured image to
`the server 103 (operated by an entity that is different than the
`local retail store), where the identifying data is extracted from
`the image using any one of many standard image recognition
`and analysis programs. For example, where the identifying
`data in the image is in an alphanumeric text format, an optical
`character recognition (OCR) program may be employed to
`extract the identifying data. Where the identifying data is a
`graphical code such as a bar code, a standard barcode extrac
`tion program may be used to read the barcode image and
`translate it to an alphanumeric code, such as a universal
`product code, for the item. The identifying data is thenused to
`query one or more resources to obtain item information (e. g.,
`price, reviews, availability, etc.) associated with the selected
`item. The server 103 communicates the item information
`back to the imaging device for display to the consumer while
`
`Page 10 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US 7,734,729 B2
`
`5
`the consumer remains at the location of the local retail store.
`Using the invention, the consumer may readily compare the
`price offered by the local retail store to using to prices offered
`by other on-line retailers, or read the revieWs of the product to
`help the consumer in making a purchasing decision.
`The various hardWare and softWare components of the
`server 103 and the imaging devices 102A, 102B, 102C and
`102D that are used to receive, store, and process the previ
`ously described data Will noW be discussed in more detail.
`FIG. 2A depicts an exemplary computer architecture of the
`server 103 shoWn in FIG. 1. Those skilled in the art Will
`appreciate that the server 103 may include more or feWer
`components than those shoWn in FIG. 2A. HoWever, it is not
`necessary that all of these generally conventional components
`be shoWn in order to disclose an enabling embodiment of the
`present invention. As shoWn in FIG. 2A, the server 103 is
`connected to the netWork 100 (FIG. 1) via a netWork interface
`160. Those skilled in the art Will appreciate that the netWork
`interface 160 includes the necessary hardWare and softWare
`for alloWing the server 103 to communicate With other com
`puters connected to the netWork by the use of one or more
`suitable communication protocols, such as the TCP/IP proto
`col.
`The server 103 also includes a processing unit 162, a video
`display adapter 164, and memory 166, all connected together
`and to the netWork interface 160 by a bus 168. The memory
`166 generally comprises RAM, ROM, and permanent
`memory, such as a hard disk drive, tape drive, optical drive,
`?oppy disk drive, or combination thereof. The memory 166
`stores an operating system 172 for controlling the operation
`of the server 103. As is knoWn to those skilled in the art, the
`operating system may be formed by a general-purpose server
`operating system such as a Microsoft® server operating sys
`tem, UNIX, or LINUXTM. A binary input/output system
`(“BIOS”) 188 for controlling the loW-level operation of
`server 103 is also stored in the memory 166.
`The memory 166 may also store program code and data for
`providing a Web site that alloW users to request, receive, and
`vieW information and data ?les stored in the server 103. Thus,
`the memory 166 may store a Web server application 178 that
`may be any one of a number of commercially available soft
`Ware packages. The Web server application 178 comprises
`computer executable instructions that, When executed by the
`server 1 03, generate con?gurable markup documents, such as
`the sample markup document shoWn in FIG. 4B, Which are
`described in more detail beloW. The memory 166 also stores
`other softWare components, such as a processing application
`180, to facilitate various functions of the present invention. As
`Will be described in more detail beloW, the processing appli
`cation 180 is con?gured to receive images from imaging
`devices, extract identifying data from the images, query a
`database for item information and communicate database
`query results to the imaging devices. The processing applica
`tion 180 is also con?gured to store data, Whether it be the
`received image, extracted identifying data, or item informa
`tion, for the generation of forecast reports.
`The server 103 may also include an input/output interface
`190 connected to the bus 168 for communicating With exter
`nal devices, such as a mouse, keyboard, scanner, or other
`input devices not shoWn in FIG. 2A. LikeWise, the server 103
`may further include additional mass storage facilities, such as
`CD-ROM/DVD-ROM drive 192, and large capacity mass
`memory 194, also connected to the bus 168. The mass
`memory 194 may be utiliZed by the server 103 to store several
`databases. Inparticular, the mass memory 194 may contain or
`have access to an item database 198 for use by the processing
`application 180. As Will be appreciated by those skilled in the
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`art, the item database 198 may be con?gured to contain data
`in any one of a number of formats suitable for storing infor
`mation related to commerce items. For instance, the item
`database 198 may be con?gured to store item information
`describing the price of an item, the availability of an item,
`customer revieWs of an item, or any other like data. The item
`database 198 may also contain advertisements about a par
`ticular item, neWs articles, and other product-related informa
`tion.
`The mass memory 194 may also contain or have access to
`a general database 199 for use by the processing application
`180. The general database 199 may be con?gured to contain
`other types of information, such as information related to
`users of the Website operated by the Web server 178, the
`purchase history or query history of the users, and other like
`information.
`As described in more detail beloW, information stored in
`the item database 198 and general database 199 may be com
`municated to one or more imaging devices for display to
`users. Although this illustration includes examples of speci?c
`softWare applications and databases, this illustration should
`be taken as exemplary and not limiting.
`As Will be appreciated by those skilled in the art, the
`architecture of the imaging devices 102A, 102B, 102C, 102D
`may take on any suitable form, such as the architecture illus
`trated in FIG. 2B, if appropriate. For example, an imaging
`device may include a netWork interface 210 for providing
`communication With the netWork 100. The netWork interface
`210 may be con?gured for use With any Wired or Wireless
`netWork connection, and may be used With any suitable com
`munication protocol, such as the TCP/IP protocol. In general,
`the imaging device includes a central processing unit 206, an
`imaging componcnt 207, an input/output interface 208 and
`memory 201, all connected together and to the netWork inter
`face 210 by a bus 209. The memory 201 stores the program
`code, such as an operating system 202, necessary for operat
`ing the imaging device and for generating an interface, such
`as a graphical user interface (GUI) on a display of the imaging
`device. The memory 201 may also store a Web broWser appli
`cation 203, such as Microsoft Internet Explorer®, for broWs
`ing Web pages generated by remote servers, such as server
`103.
`The imaging component 207 may include a charge coupled
`device (CCD) or any other circuit suitable for capturing an
`image of an item that includes identifying data, such as a
`barcode or text information. Portable electronics With imag
`ing components, such as mobile phones, PDAs, game
`machines, and digital cameras, are presently knoWn and
`understood in the art of image capture and communication.
`The imaging device may further comprise one or more input/
`output devices 208, Which may include a display, a speaker, or
`any other device for communicating information. Accord
`ingly, the imaging device may also contain softWare compo
`nents, such as a processing application 204, for converting
`electronic signals into audible signals suitable for communi
`cating information through a speaker.
`Referring noW to the How diagram of FIG. 3 and the pic
`torial diagrams of FIGS. 4A and 4B, one exemplary embodi
`ment of an information processing method 300 Will noW be
`described. In this description, the information processing
`method 300 processes an image to identify the item identi?ed
`by the image and returns to the user information pertaining to
`the identi?ed item. The information processing method 300
`also collects item information for use in generating individual
`user pro?les or aggregate sales forecasting reports. The infor
`mation processing method 300 begins at block 301, Where an
`image of an item is obtained from an imaging device, such as
`
`Page 11 of 16
`
`ROTHSCHILD EXHIBIT 1007
`
`

`
`US 7,734,729 B2
`
`7
`a mobile phone (as shown in FIG. 4A). The image contains
`identifying data, such as a barcode of a UPC, the text of the
`UPC, and/or any other suitable identi?er, such as a tWo
`dimensional barcode, text describing the item, etc.
`FIG. 4A illustrates a perspective vieW of one suitable
`arrangement for capturing an image containing identifying
`data about a selected item. As shoWn, a mobile phone 400
`having a camera 413 is used to capture an image 412 of
`identifying data 402 of an item 401. In this example, the
`camera 413 is directed to the barcode 402 of the item 401 for
`capturing the image of the barcode 402. In other instances, the
`camera 413 may be directed to a label, sign, or advertising
`that includes identifying data of a selected item, even if the
`selected item is physically present or not. In yet other
`instances, the camera 413 may be directed to the product
`itself.
`During operation of the camera 413, the image is captured
`by the camera 413 and stored in the memory of the camera
`413. The cap