`Thompson etal.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,767,354 B2
`Sep. 19, 2017
`
`US009767354B2
`
`(54)
`
`GLOBAL GEOGRAPHIC INFORMATION
`RETRIEVAL, VALIDATION, AND
`NORMALIZATION
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`(71)
`
`Applicant: Kofax, Inc., Irvine, CA (US)
`
`(72)
`
`Inventors: Stephen Michael Thompson,
`Oceanside, CA (US); Jan W. Amtrup,
`
`1,660,102 A
`3,069,654 A
`
`2/1928 Appelt et al.
`12/1962 Hough
`(Continued)
`
`Petitioner Apple Inc., Ex. 1106, p. 1
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 2
`
`Related U.S. Application Data
`
`which is a continuation of application No. 14/176,
`O06, filed on Feb. 7, 2014, now Pat. No. 8,958,605,
`which is a continuation-in-part of application No.
`13/948,046, filed on Jul. 22, 2013, now Pat. No.
`8,855,425, which 1s a continuation of application No.
`13/691,610, filed on Nov. 30, 2012, now Pat. No.
`8,926,739, which 1s a continuation of application No.
`12/368,685, filed on Feb. 10, 2009, now Pat. No.
`
`5,652,663
`5,668,890
`5,680,525
`5,696,611
`5,696,805
`5,699,244
`5,717,794
`5,721,940
`5,757,963
`5,760,912
`5,781,665
`
`7/1997
`9/1997
`10/1997
`12/1997
`12/1997
`12/1997
`2/1998
`2/1998
`5/1998
`6/1998
`7/1998
`
`Zelten
`Winkelman
`Sakai et al.
`Nishimura et al.
`Gaborskiet al.
`Clark, Jr. et al.
`Kogaet al.
`Luther et al.
`Ozaki et al.
`Itoh
`Cullen et al.
`
`errrrrrrrrrrrrrrrrrPPPPPPPPSPEPPPEPerrPEPPrEPPS
`
`Petitioner Apple Inc., Ex. 1106, p. 2
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 3
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`Kotovich et al.
`7/2007
`7,249,717
`Valtchev et al.
`7/2007
`7,291,777
`Suzuki et al.
`8/2007
`7,253,836
`Moriwaki
`8/2007
`7,263,221
`Ferlitsch et al.
`9/2007
`7,206,768
`5/2003 Matama
`Bl
`6,563,531
`10/2007
`Cooper
`7,286,177
`7/2003 Appelt et al.
`B2
`6,601,026
`Dominguez etal.
`11/2007
`7,298,897
`9/2003 Agnihotri et al.
`Bl
`6,614,930
`Suzuki et al.
`1/2008
`7,317,828
`9/2003 Fan etal.
`Bl
`6,621,595
`Woolf et al.
`2/2008
`7,337,389
`9/2003 Hsu et al.
`Bl
`6,628,416
`Verstraelen et al.
`3/2008
`7,339,585
`9/2003 Bach et al.
`Bl
`6,628,808
`3/2008
`Goodman
`7,340,376
`Bl=10/2003 Tipping
`6,633,857
`
`Petitioner Apple Inc., Ex. 1106, p. 3
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 4
`
`U.S. PATENT DOCUMENTS
`
`B2=12/2016 Macciola et al.
`
`9,514,357
`References Cited
`B2
`2/2017
`Macciola et al.
`9,576,272
`Amtrup et al.
`B2
`2/2017
`9,584,729
`200 1/0027420
`Al
`10/2001
`Boublik et al.
`Kinjo
`2002/003083 1
`Al
`3/2002
`8/2012 Jin et al.
`B2
`$238,880
`2002/0054693
`Al
`$/2002
`Elmenhurst
`8/2012 Schmidtler et al.
`B2
`$8,239,335
`2002/00692 18
`Al
`6/2002
`Sull et al.
`8/2012 Cho et al.
`B2
`8,244,031
`Reavy et al.
`2002/0113801
`Al
`8/2002
`9/2012 Jin
`Bl
`$,265,422
`2002/0122071
`Al
`9/2002
`Camara et al.
`B2=10/2012 Couchman
`8,279,465
`2002/01263 13
`Al
`9/2002
`Namizuka
`B2=10/2012 Katougi et al.
`8,295,599
`
`
`2002/0165717 Al—11/2002 Solmer et al.
`B2~~11/2012 +Filatov et al.
`8,311,296
`
`Petitioner Apple Inc., Ex. 1106, p. 4
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 5
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2006/00 178 10
`2006/0023271
`2006/003 1344
`2006/0047704
`2006/0048046
`2006/007482 1
`2006/0089907
`
`1/2006
`2/2006
`2/2006
`3/2006
`3/2006
`4/2006
`4/2006
`
`Kurzweil et al.
`Boayetal.
`Mishimaet al.
`Gopalakrishnan
`Joshi et al.
`Cristianini
`Kohlmaier et al.
`
`2008/00823 52
`2008/0086432
`2008/0086433
`2008/0095467
`2008/0097936
`2008/0130992
`2008/0133388
`2008/0137971
`2008/0144881
`2008/0147561
`2008/0147790
`
`4/2008
`4/2008
`4/2008
`4/2008
`4/2008
`6/2008
`6/2008
`6/2008
`6/2008
`6/2008
`6/2008
`
`Schmidtler et al.
`Schmidtler et al.
`Schmidtler et al.
`Olszak et al.
`Schmidtler et al.
`Fujii
`Alekseevet al.
`King et al.
`Fortuneet al.
`Euchneret al.
`Malaneyetal.
`
`Petitioner Apple Inc., Ex. 1106, p. 5
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 6
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/2014 Schmidtler et al.
`2014/0164914 Al
`6/2014 Chirehdast
`2014/0172687 Al
`6/2014 Poornachandran et al.
`2014/0181691 Al
`7/2014 Buttner et al.
`2014/0201612 Al
`7/2014 Schmidtler et al.
`2014/0207717 Al
`1/2012 Heweset al.
`9012/0008856 Al
`8/2014 Borreyet al.
`2014/0233068 Al
`1/2012 Murrayetal.
`9012/0019614 Al
`
`
`2012/0038549 Al 2014/0254887 Al=9/2014 Amtrupetal.2/2012 Mandella etal.
`
`2012/0057756 Al 2014/0270349 Al=9/2014 Amtrupetal.3/2012. -Yoonetal.
`
`
`2012/0069131 Al
`3/2012. Abelow
`2014/0270439 Al
`9/2014 Chen
`9012/0075442 Al
`3/2012 Vujic
`2014/0270536 Al
`9/2014 Amtrup et al.
`2012/0077476 Al
`3/2012. Paraskevakosetal.
`2014/0316841 Al
`10/2014 Kilbyet al.
`
`Petitioner Apple Inc., Ex. 1106, p. 6
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 7
`
`(56)
`
`References Cited
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`
`2008134683 A
`2011034387 A
`2011055467 A
`2012009033 A
`2012517637 A
`2013196357 A
`5462286 B2
`
`6/2008
`2/2011
`3/2011
`1/2012
`8/2012
`9/2013
`4/2014
`
`International Search Report and Written Opinion from International
`Application No. PCT/US2014/026569, dated Aug. 12, 2014.
`Gllavata, et al., “Finding Text in Images Via Local Thresholding,”
`International Symposium on Signal Processing and Information
`Technology, Dec. 2003, pp. 539-542.
`Zunino, et al “Vector Quantization for License-Plate Location and
`Image Coding,” IEEE Transactions on Industrial Electronics, vol.
`47, Issue 1, Feb. 2000, pp. 159-167.
`Bruns, E. et al., “Mobile Phone-Enabled Museum Guidance with
`Adaptive Classification,” Computer Graphics and Applications,
`
`Petitioner Apple Inc., Ex. 1106, p. 7
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 8
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Non-Final Office Action from U.S. Appl. No. 13/740,127, dated
`Feb. 23, 2015.
`International Search Report and Written Opinion from International
`Application No. PCT/US2015/021597, dated Jun. 22, 2015.
`U.S. Appl. No. 14/340,460, filed Jul. 24, 2014.
`Requirement for Restriction from U.S. Appl. No. 14/177,136, dated
`
`Su et al., “Stereo rectification of calibrated image pairs based on
`geometric transformation,” I.J.Modern Education and Computer
`Science, vol. 4, 2011, pp. 17-24.
`Malis et al., “Deeper understanding of the homography decompo-
`sition for vision-based control,” [Research Report] RR-6303, Inria,
`sep. 2007, pp. 1-90.
`Notice of Allowance from U.S. Appl. No. 14/491,901, dated Aug.
`4, 2015.
`Final Office Action from U.S. Appl. No. 14/491,901, dated Apr. 30,
`2015.
`
`Petitioner Apple Inc., Ex. 1106, p. 8
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`Page 9
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Office Action from ‘Tarwan Patent Application No. 102101177,
`dated Dec. 17, 2014.
`Non-Final Office Action from U.S. Appl. No. 13/740,141, dated
`Oct. 16, 2015.
`Notice of Allowance from U.S. Appl. No. 13/740,145, dated Sep. 8,
`2015.
`
`Non-Final Office Action from U.S. Appl. No. 15/390,321, dated
`Mar. 17, 2017.
`Extended European Search Report from European Application No.
`14881675.4, dated Jun. 7, 2017.
`Notice of Allowance from U.S. Appl. No. 15/191,442, dated Apr.
`24, 2017.
`Notice of Allowance from U.S. Appl. No. 15/234,969, dated May8,
`2017.
`Notice of Allowance from U.S. Appl. No. 14/814,455, dated May
`26, 2017.
`Corrected Notice of Allowance from U.S. Appl. No. 15/191,442,
`
`Petitioner Apple Inc., Ex. 1106, p. 9
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 1 of 6
`
`US 9,767,354 B2
`
`100
`
`FROM FIRST DOCUMENT |
`
`102
`
`Petitioner Apple Inc., Ex. 1106, p. 10
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 2 of 6
`
`US 9,767,354 B2
`
`200
`
`
`
`Petitioner Apple Inc., Ex. 1106, p. 11
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 3 of6
`
`US 9,767,354 B2
`
`308
`
`OCR invoice
`
`Petitioner Apple Inc., Ex. 1106, p. 12
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 4 of 6
`
`US 9,767,354 B2
`
`404 —~
`
`Petitioner Apple Inc., Ex. 1106, p. 13
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 5 of6
`
`US 9,767,354 B2
`
`Petitioner Apple Inc., Ex. 1106, p. 14
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`U.S. Patent
`
`Sep. 19, 2017
`
`Sheet 6 of6
`
`US 9,767,354 B2
`
` 602
`
`Petitioner Apple Inc., Ex. 1106, p. 15
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`1
`
`GLOBAL GEOGRAPHIC INFORMATION
`RETRIEVAL, VALIDATION, AND
`NORMALIZATION
`
`FIELD OF THE INVENTION
`
`The present invention relates to document analysis sys-
`tems, methods, and computer program products, and more
`particularly, this invention relates to systems, methods, and
`computer program products for retrieving, determining
`
`2
`particularly for validation and normalization of address
`information according to various international standards.
`
`SUMMARY
`
`According to one embodiment, a computer-implemented
`method includes: capturing an image of a document using a
`camera of a mobile device; performing optical character
`recognition (OCR) on the image of the document; extracting,
`0 an identifier of the document from the image basedat least
`in part on the OCR; comparing the identifier with content
`
`Petitioner Apple Inc., Ex. 1106, p. 16
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`3
`FIG. 3 showsa representative hardware environmentthat
`may be associated with the servers and/or clients of FIG.4,
`in accordance with one embodiment.
`FIG. 6 is a flowchart of a method, according to one
`embodiment.
`
`DETAILED DESCRIPTION
`
`The following description is the best mode presently
`contemplated for carrying out the present invention. This
`description is madefor the purposeofillustrating the general
`
`4
`receiver has to retrieve or extract the information from the
`received document and compare it to the corresponding
`information stored in its database. This, for example, can be
`achieved by a human reading the document, encoding its
`data, and comparing it to the corresponding content of the
`receiver’s database. The extraction of the information can
`be, at least to some extent, automated by utilizing technolo-
`gies that automatically extract the relevant information from
`the document.
`Today many documents still are recerved on paper and are
`
`Petitioner Apple Inc., Ex. 1106, p. 17
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`5
`process simultaneously analyzes the information from these
`sources and uses the complementary information to validate
`the interaction.
`Several exemplary embodiments and descriptions thereof
`are provided below in the context of a business transaction
`involving a document such as an invoice or otherfinancial
`document. Those having ordinary skill in the art will appre-
`ciate that the inventive concepts presented herein are equally
`applicable to retrieval, validation, and/or normalization of
`geographic
`information such as partial or complete
`addresses, which may be obtained from any suitable source
`
`6
`the need of having a custom-built template for every vendor.
`Yet the information held by the line items is important to
`validate the invoice. Similarly,
`information held in an
`address line may be important to validate the invoice or
`other types of documents depicting addresses, such as IDs,
`etc. as set forth herein.
`Additionally, for the validation of the invoice, a large
`portion of the extracted information may be irrelevant.
`Given the described process, the knowledge of which infor-
`mation is important for invoice validation and which infor-
`mation can be disregarded is not available to the operator
`
`Petitioner Apple Inc., Ex. 1106, p. 18
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`7
`as well as electronic records (e.g. in a database) memorial-
`izing, representing, or including information derived from
`physical and/or electronic documents, etc. as would be
`understood by a person having ordinary skill in the art upon
`reading the present disclosures. For
`instance,
`in one
`approach where the document is an invoice, corresponding
`documents may include physical and/or electronic records
`such as a purchase order, delivery notes, etc. as would be
`understood by a person having ordinary skill in the art upon
`reading the present descriptions. In other approaches, cor-
`responding documents may include bills, checks, deeds, etc.
`
`8
`skilled artisan upon reading the present descriptions. Simi-
`larly, corresponding documents and/or information con-
`tained therein may be derived from on-boarding documents,
`in various approaches.
`Additionally, in one embodiment, the scanned image may
`be generated by scanning or otherwise imaging the first
`document. For example,
`the document may be scanned
`using a personal or commercial hardware scanning device,
`using scanning software, by capturing image data using a
`camera, e.g. of a mobile device, etc.
`Further, the scanned image may include any imagethat
`
`Petitioner Apple Inc., Ex. 1106, p. 19
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`9
`identified in the first document may be advantageous
`because the fuzzy matching process is provided more data
`from which to characterize and/or validate the document,
`enabling a more robust analysis of the content (e.g. textual
`information per se) and/or context of the document(e.g. the
`intended origin of the document, intended destination of the
`document, intended purpose of the document, etc. as would
`be understood by one having ordinary skill in the art upon
`reading the present descriptions).
`Further, as shown in operation 106, a complementary
`document (or documents) associated with the first document
`
`10
`the generation of the list of hypotheses for later use. For
`instance, in one approach address validation, retrieval, nor-
`malization, etc. may include expanding any identified abbre-
`viations into full wordings, e.g. “St.” becomes “Street,”
`“Ave.” becomes “Avenue,” “Blvd.” becomes “Boulevard,”
`etc. as would be understood by a person having ordinary
`skill in the art upon reading the present disclosures.
`In addition, the list of hypotheses may be generated using
`non-textual
`information from the first document and the
`complementary document, such as lines, colors, symbols,
`holograms, pictures, etc. Further, the list of hypotheses may
`
`Petitioner Apple Inc., Ex. 1106, p. 20
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`1]
`the predefined business rules, and/or the complementary
`document, the extracted textual information is optionally
`altered. For example, numbers, letters, and other field items
`may be altered according to information obtained from the
`predefined business rules and the complementary document.
`After the alteration has occurred, an additional analysis is
`performedutilizing the altered extracted textual information,
`the predefined business rules, and the complementary docu-
`ment. In this way, the extracted textual information may be
`fine-tuned to more accurately relate to the complementary
`document. Similarly, and as described in further detail
`
`12
`image of the document was capturedat or in proximity to the
`physical location corresponding to the geographic informa-
`tion represented on the document, the extracted geographic
`information may be replaced with complementary geo-
`eraphic information in a complementary document
`for
`which the complementary location information matched the
`location information stored in association with the image.
`Alternatively, an OCR correction may be suggested based on
`the complementary geographic information.
`As will be appreciated by a person having ordinary skull
`in the art upon reading the present disclosures, the foregoing
`
`Petitioner Apple Inc., Ex. 1106, p. 21
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`13
`another embodiment, the determination may be additionally
`based on a confidence level of the hypotheses.
`Further, in one embodiment, an alert may be generated
`upon encountering a potential problem when determining
`the validity of the first document. For example, the alert may
`include an identification of a mismatch in expected similar
`or identical values in the first and complementary docu-
`ments. Additionally, in another embodiment, user input may
`be received indicating at least one of a correction and a
`validation of items such as a line item, header field item,
`partial or complete address, etc. of the first document.
`
`14
`validity of the first document. The human operation may be
`notified via a message, e.g. an electronic mail message,
`indicating that unresolvable errors exist with the first docu-
`ment. After human correction has been performed,
`the
`method may then be repeated on the corrected first docu-
`ment.
`In another embodiment, a notification to access the rec-
`onciliation screen may be sent
`to a sender of the first
`document. Further, a modification to the first document may
`be received by a user viewing the reconciliation screen.
`Further still, re-validation of the modified first document
`
`Petitioner Apple Inc., Ex. 1106, p. 22
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`15
`option, the method 200 may be carried out in the context of
`the architecture and environment of FIG. 1. Of course,
`however, the method 200 may be carried out in any desired
`environment. As will be appreciated by skilled artisans upon
`reading the instant descriptions, while methods 200 and 300
`are described in the context of determining validity of an
`invoice, the principles set forth are equally applicable to
`determining validity of other document types using other
`information commonly included in the respective document
`type. Such information may include the content of the
`
`16
`tor may provide information about extracted data such as the
`unit price, quantity, description, line item price,etc.
`In addition, it 1s determined by the integrated matching
`and extraction algorithm 220 in operation 222 whether the
`invoice is valid. For example, it may be determined whether
`the invoice contains incomplete or incorrect data. If it is
`determined in operation 222 that the invoice is valid, then in
`operation 224 the invoice is further processed given its
`validity. If it is determined in operation 222 that the invoice
`is invalid,
`then in operation 226 the invoice is further
`processed according to one or more errors detected by the
`
`Petitioner Apple Inc., Ex. 1106, p. 23
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`17
`other. For example, improved OCR may result in improved
`extraction, which in turn may yield better matching, and so
`forth.
`FIG. 3 shows a method 300 for determining a validity of
`an invoice without the use of an intelligent agent, in accor-
`dance with yet another embodiment. As an option,
`the
`method 300 may be carried out
`in the context of the
`architecture and environment of FIGS. 1 and/or 2. Of course,
`however, the method 300 may be carried out in any desired
`environment.
`As shown in operation 302, an invoice is scanned. Addi-
`
`18
`Also, a position match candidate (PMC) includes a combi-
`nation of line-itemsthat is a candidate to match to a purchase
`order position. In one embodiment, PMCs may map oneto
`one to positions, whereas line-1tems do not necessarily have
`a one to one mapping to positions.
`The matching and extraction algorithm validates invoices
`by comparing the information given on an invoice with the
`corresponding purchase order. lo this end the algorithm
`performs the following tasks. First, the algorithm validates
`line-items by associating the line-items on a given invoice
`with the open purchase order positions of this invoice.
`
`Petitioner Apple Inc., Ex. 1106, p. 24
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`19
`As shown in Table 1, cPMC indicates the cost of gener-
`ating a specific set of PMCs and cMAP1s the cost associated
`with a specific one to one mapping of the generated PMCset
`to positions and the validation of the invoice. The cost cPMC
`is factored into the following sum, as shown in ‘lable 2.
`
`TABLE 2
`
`cPMC = cprior + cline + cextraction +
`cOCR + csequence + calignment
`
`20
`sum over the individual matching costs of matching a single
`PMCto a position. The single matching costs are derived
`from the cost of fuzzy matching the individual line-item
`fields’ description, quantity, unit price, and extended price to
`the corresponding entries in the position. The fuzzy match-
`ing takes into account the OCR confidence of the individual
`characters in the extracted line-item fields.
`Similarly, for embodiments in which geographic infor-
`mation is the subject of validation, fuzzy matching may
`include evaluating the single matching costs for individual
`components of the geographic information, e.g. individual
`
`Petitioner Apple Inc., Ex. 1106, p. 25
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`21
`As will be appreciated by skilled artisans upon reading the
`present disclosure, the “APT BLK”prefix exhibits regularity
`and may be leveraged to validate addresses or other geo-
`graphic information in Singapore. Similarly, the unit number
`(14-358) is formatted as floor-unit. The postal code 1s also
`correlated to the block number. Specifically, the last 3 digits
`are the same. Conventions such as these for other interna-
`tional locations may be similarly leveraged without depart-
`ing from the scope of the present disclosures, in various
`embodiments. Accordingly,
`it should be understood by a
`
`22
`for
`matching hypothesis becomes quickly unpractical
`invoices with more than a dozen of line-items and positions
`when using prior art methods. The developed algorithm
`approximates the search efficiently and effectively. The
`elected approach is described in the following paragraphs.
`The number of possible PMC sets is factorial
`in the
`numberof line-1tems. Similarly, the number of possible one
`to one mappings to positions given a specific PMCset is
`factorial in the numberof positions and line-items. Accord-
`ingly, the numberof resulting possible matching hypotheses
`
`Petitioner Apple Inc., Ex. 1106, p. 26
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`23
`
`TABLE 5-continued
`
`US 9,767,354 B2
`
`24
`
`Algorithm 1 Matching algorithm to find best association of line-items to purchase
`order positions.
`
`10:
`11:
`12:
`13:
`
`minCost :=c
`bestMatch := (M,setOfPMCs)
`endif
`wupdateAnnealingSchdedule( ) {Procedure that monitors the changes in
`the individual costs that constitute the cost cp,,- and their relation with
`the overall cost c. It updates the annealing schedules needed in the routine
`nextPMC accordingly.}
`14: end while
`
`Petitioner Apple Inc., Ex. 1106, p. 27
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`25
`memory and other portable memory cards,etc.), etc. Further,
`such software can be downloadable or otherwise transfer-
`able from one computing device to another via network,
`wireless link, nonvolatile memory device, etc.
`FIG. 4 illustrates a network architecture 400, in accor-
`dance with one embodiment. As shown, a plurality of
`networks 402 is provided. In the context of the present
`network architecture 400, the networks 402 may each take
`any form including, but not limited to a local area network
`(LAN), a wireless network, a wide area network (WAN)
`such as the Internet, peer-to-peer network, etc.
`
`26
`that the concepts are equally applicable to mobile devices,
`for example any “scanning” operation discussed herein may
`be applied to a mobile device and/or mobile computing
`environment, for example by capturing an image using a
`mobile device camera rather than “scanning” the image or
`document.
`Those having ordinary skill in the art will appreciate that
`image data generated using a scanner and image data
`generated using a camera may have unique aspects or
`characteristics in some approaches. For example, an image
`captured using a mobile device camera may includeartifacts
`
`Petitioner Apple Inc., Ex. 1106, p. 28
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`27
`In operation 604, method 600 includes performing OCR
`on the image. The OCR may be performed in any suitable
`manner, preferably as described hereinabove.
`Method 600 also includes operation 606, where an iden-
`tifier of the document extracted from the image of the
`document, optionally based at least 1n part on the OCRresult
`obtained from performing operation 604.
`Method 600 also includes comparing the extracted iden-
`tifier with content from one or more data sources in opera-
`tion 608. The data sources preferably comprise one or more
`relational databases, but may also include other data sources
`
`23
`information in the reference data sources. As will be under-
`stood by persons having ordinary skill
`in the art upon
`reviewing these disclosures, such parsing based on heuristic
`rules and normalization facilitates accurate comparisons of
`identifiers against the geographic information in the refer-
`ence data source(s), bolstering the accuracy of the presently
`described validation processes.
`Accordingly,
`in operation 610, method 600 includes
`determining whether the extracted identifier is valid based
`on the comparison performed in operation 608.
`
`Petitioner Apple Inc., Ex. 1106, p. 29
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`29
`The identifier may additionally and/or alternatively be
`encoded on the document, for example, in a hologram or
`barcode (including one-,
`two- and/or three-dimensional
`holograms/barcodes), in some approaches. In more complex
`examples, the identifier may include personal identification
`information such as a name, social security number (SSN),
`tax ID number, date of birth (DOB), place of residence, a
`logo, a unique imageor photograph (e.g. a photograph ofthe
`resident or owner’s face), etc. as would be understood by
`one having ordinary skill in the art upon reading the present
`descriptions.
`
`30
`a state name or abbreviation, etc. as would be understood by
`one having ordinary skill in the art upon reading the present
`descriptions.
`Upon extracting the document identifier, the presently
`disclosed techniques may leverage a number of advanta-
`geous features to provide a document owner with useful
`information and/or services regarding the document. For
`example, in one approach the documentidentifier comprises
`one or more of a street name and a ZIP code. A request may
`be submitted to a remote resource for information corre-
`sponding to the document using the identifier as a query.
`
`Petitioner Apple Inc., Ex. 1106, p. 30
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`3]
`of OCR errors with respect to the extracted identifier, rather
`than a discrepancy between the “true” identifier and the
`corresponding identifier information from the complemen-
`tary document
`(e.g.
`the “textual
`information” in some
`approaches). Similarly, in embodiments where an identifier
`or other information is input by a user, a partial match may
`be indicative of erroneous data input rather than a discrep-
`ancy between the “true” identifier and the corresponding
`identifier information from the complementary document.
`To account for, and automatically correct, such OCR
`and/or data input errors, in some approaches the extracted
`
`32
`to determining the existence of the OCR error(s). Most
`preferably, user input is received via a mobile device, and
`relates to one or more of OCRerrors, the textual information
`from the imaged financial document and/or the complemen-
`tary document, and the predefined businessrules.
`Similarly, other discrepancies not arising from either of
`OCR errors or data input errors as described above may
`nonetheless be automatically handled using the present
`techniques. In one embodiment, according to textual infor-
`mation from a complementary document(e.g. an electronic
`record in a reference database) an identifier 1s expected to be
`
`Petitioner Apple Inc., Ex. 1106, p. 31
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`34
`33
`quently perform OCR on the region to extract the geo-
`rather than true mismatch between the identifier on the
`eraphic information, and/or perform a second OCRiteration
`imaged document and the corresponding data from the one
`to improve OCRresults.
`or more data sources.
`Returning to the notion of identifier characteristics, in a
`in some
`OCR errors of this nature may be corrected,
`preferred embodiment
`identifier characteristics may be
`approaches, by determining one or more characteristics of
`determined based on a location from which an identifier is
`data correspondingto the identifier. In one embodiment, the
`extracted being located below data depicting related infor-
`first OCR iteration may extract the identifier in an unaccept-
`mation, such as an identifier being located belowastreet
`able format(e.g. the data is not properly normalized) and/or
`address line, which typically corresponds to a city, state,
`perform the OCR in a manner such that the extracted
`and/or zip code, particularly in documents depicting a mail-
`identifier contains one or more OCRerrors. As a result, the
`ing address. In another preferred embodiment,
`identifier
`
`Petitioner Apple Inc., Ex. 1106, p. 32
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`35
`records for comparison. Restrictions of this type may be
`implemented using any suitable technique that would be
`understood by a person having ordinary skill in the art upon
`reading the present descriptions.
`In addition, in some approaches geocoding may be uti-
`lized to facilitate the retrieval of additional
`information
`regarding a particular location of piece of geographic infor-
`mation. For instance, 1n a scenario wherea partial address is
`depicted and extracted from a document, missing portions of
`the address may be retrieved based on correlation between
`
`36
`formula, a pattern, convention,
`expression (e.g. a rule,
`structure, organization, etc. or any number or combination
`thereof).
`Those having ordinary skill in the art will appreciate that
`similar business rules may inform an OCRprocess regarding
`how to define the extracted identifier string in a variety of
`situations differing from the numeral/character distinction
`exemplified above.
`For example,
`in one embodiment a business rule may
`indicate that a particular alphabet of symbols should be used,
`e.g. as opposed to a more complete or different alphabet of
`
`Petitioner Apple Inc., Ex. 1106, p. 33
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`37
`wherein the predefined alphabet consists of one or more of
`numerals, alphabetic characters, and symbols.
`3. The method as recited in claim 1, wherein the identifier
`comprises a partial or complete address.
`4. The methodas recited in claim 1, wherein the identifier
`comprises one or more of:
`a street name, a street number, a block number, a unit
`number, a city name, a county name, a municipality
`name, a state name, a state abbreviation, a country
`name, a country abbreviation, and a ZIP code.
`5. The method as recited in claim 1,
`the comparing
`
`38
`detecting one or more OCRerrors based at least in part on
`textual information from the complementary document
`and one or more of the predefined businessrules;
`correcting at least one detected OCR error using one or
`more of the predefined business rules;
`correcting at least one detected OCR error using textual
`information from the complementary document;
`correcting at least one detected OCR error using textual
`information from the complementary document and
`one or more of the predefined business rules;
`normalizing data from a complementary document using
`
`Petitioner Apple Inc., Ex. 1106, p. 34
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`US 9,767,354 B2
`
`39
`obtaining the geographic information from one or more
`of the proprietary address database and an open
`source address database; and
`parsing the geographic information according to a set
`of predefined heuristic rules, wherein the set of 5
`predefined heuristic rules are configured to normal-
`ize the global address information obtained from the
`one or more sources accordingto a single convention
`for representing address information,
`determining whether the identifier is valid based at least
`in part on the comparison;
`
`10
`
`40
`
`Petitioner Apple Inc., Ex. 1106, p. 35
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`CERTIFICATE OF CORRECTION
`
`PATENT NO.
`APPLICATION NO.
`DATED
`INVENTOR(S)
`
`» 9,767,354 B2
`: 15/146848
`: September 19, 2017
`: Stephen Michael Thompsonetal.
`
`Page 1 of 1
`
`It is certified that error appears in the above-identified patent and that said Letters Patent is hereby corrected as shown below:
`
`In the Specification
`
`Petitioner Apple Inc., Ex. 1106, p. 36
`Apple Inc. v. MemoryWeb, LLC, IPR2022-00031
`
`