Doc Code: TR.PROV
`
`Document Description: Provisional Cover Sheet (SB16)
`
`PTOfSBl16 (11-08)
`Approved for use through 01f31I2014 OMB 0551 -0032
`U.S. Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number
`
`Provisional Application for Patent Cover Sheet
`This is a request for filing a PROVISIONAL APPLICATION FOR PATENT under 37 CFR 1.53(c)
`
`lnventor(s)
`
`Inventor 1
`
`Given Name
`
`Middle Name
`
`Family Name
`
`City
`
`Nepomniachtchi
`
`San Diego
`
`Grigori
`
`Inventor 2
`
`Inventor 3
`
`Given Name
`
`Middle Name
`
`Family Name
`
`City
`
`Josh
`
`Roach
`
`San Diego
`
`All Inventors Must Be Listed — Additional Inventor Information blocks may be
`generated within this form by selecting the Add button.
`
`Add
`
`Title of Invention
`
`SYSTEMS FOR MOBILE IMAGE CAPTURE AND REMITTANCE
`PROCESSING
`
`Attorney Docket Number (if applicable)
`
`117313-011PRV
`
`Correspondence Address
`
`Direct all correspondence to (select one):
`
`(Q The address corresponding to Customer Number O Firm or Individual Name
`
`Customer Number
`
`27189
`
`The invention was made by an agency of the United States Government or under a contract with an agency of the United
`States Government.
`
`6) No.
`
`Q Yes, the name of the U.S. Government agency and the Government contract number are:
`
`EFS — web1?.age 1 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 1 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Doc Code: TR.PROV
`
`Document Description: Provisional Cover Sheet (SB16)
`
`PTOfSBl16 (11-08)
`Approved for use through 01f31I2014 OMB 0551 -0032
`U.S. Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number
`
`Entity Status
`Applicant claims small entity status under 37 CFR 1.27
`
`6) Yes, applicant qualifies for small entity status under 37 CFR 1.27
`
`O No
`
`Petitioner/applicant is cautioned to avoid submitting personal information in documents filed in a patent application that may
`contribute to identity theft. Personal information such as social security numbers, bank account numbers, or credit card
`numbers (other than a check or credit card authorization form PTO—2038 submitted for payment purposes) is never required
`by the USPTO to support a petition or an application.
`If this type of personal information is included in documents submitted
`to the USPTO, petitioners/applicants should consider redacting such personal information from the documents before
`submitting them to USPTO. Petitionerlapplicant is advised that the record of a patent application is available to the public
`after publication of the application (unless a non-publication request in compliance with 37 CFR 1_213(a) is made in the
`application) or issuance of a patent. Furthermore, the record from an abandoned application may also be available to the
`public if the application is referenced in a published application or an issued patent (see 37 CFR1.14). Checks and credit
`card authorization forms PTO—2038 submitted for payment purposes are not retained in the application file and therefore are
`not publicly available.
`
`Signature
`
`Please see 37 CFR 1.4(d) for the form of the signature.
`
`/Noel c. Gillespie.’
`Noel 0
`
`First Name
`
`Last Name
`
`Gillespie
`
`Date (YYYY-MM-DD)
`Registration Number
`(If appropriate)
`
`2011-11-18
`47596
`
`This collection of information is required by 37 CFR 1.51. The information is required to obtain or retain a benefit by the public which is to
`file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.11 and 1.14. This collection
`is estimated to take 8 hours to complete, including gathering, preparing, and submitting the completed application form to the USPTO.
`Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form andlor
`suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department
`of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. This
`form can only be used when in conjunction with EFS-Web. If this form is mailed to the USPTO, it may cause delays in handling
`the provisional application.
`
`EFS — web1?.age 2 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 2 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Privacy Act Statement
`
`The Privacy Act of 1974 (P.L. 93-579) requires that you be given certain information in connection with your submission of
`the attached form related to a patent application or paten. Accordingly, pursuant to the requirements of the Act, please be
`advised that :
`(1) the general authority for the collection of this information is 35 U.S_C_ 2(b)(2); (2) furnishing of the
`information solicited is voluntary; and (3) the principal purpose for which the information is used by the U_S. Patent and
`Trademark Office is to process and/or examine your submission related to a patent application or patent.
`If you do not
`furnish the requested information, the U.S. Patent and Trademark Office may not be able to process and/or examine your
`submission, which may result in termination of proceedings or abandonment of the application or expiration of the patent.
`
`The information provided by you in this form will be subject to the following routine uses:
`
`1 .
`
`The information on this form will be treated confidentially to the extent allowed under the Freedom of Information
`Act (5 U.S.C. 552) and the Privacy Act (5 U.S.C 552a). Records from this system of records may be disclosed to the
`Department of Justice to determine whether disclosure of these records is required by the Freedom of Information
`Act.
`
`A record from this system of records may be disclosed, as a routine use, in the course of presenting evidence to
`a court, magistrate, or administrative tribunal, including disclosures to opposing counsel in the course of settlement
`negotiations.
`A record in this system of records may be disclosed, as a routine use, to a Member of Congress submitting a
`request involving an individual, to whom the record pertains, when the individual has requested assistance from the
`Member with respect to the subject matter of the record.
`A record in this system of records may be disclosed, as a routine use, to a contractor of the Agency having need
`for the information in order to perform a contract. Recipients of information shall be required to comply with the
`requirements of the Privacy Act of 1974, as amended, pursuant to 5 U.S.C. 552a(m).
`A record related to an International Application filed under the Patent Cooperation Treaty in this system of
`records may be disclosed, as a routine use, to the International Bureau of the World Intellectual Property
`Organization, pursuant to the Patent Cooperation Treaty.
`A record in this system of records may be disclosed, as a routine use, to a n other federal agency for purposes
`of National Security review (35 U.S.C. 181) and for review pursuant to the Atomic Energy Act (42 U.S.C. 218(c)).
`A record from this system of records may be disclosed, as a routine use, to the Administrator, General Services,
`or his/her designee, during an inspection of records conducted by GSA as part of that agency's responsibility to
`recommend improvements in records management practices and programs, under authority of 44 U.S.C. 2904 and
`2906. Such disclosure shall be made in accordance with the GSA regulations governing inspection of records for this
`purpose, and any other relevant (i.e., GSA or Commerce) directive. Such disclosure shall not be used to make
`determinations about individuals.
`
`A record from this system of records may be disclosed, as a routine use, to the public after either publication of
`the application pursuant to 35 U.S.C. 122(b) or issuance of a patent pursuant to 35 U.S.C. 151. Further, a record
`may be disclosed, subject to the limitations of 37 CFR 1.14, as a routine use, to the public if the record was filed in an
`application which became abandoned or in which the proceedings were terminated and which application is
`referenced by either a published application, an application open to public inspection or an
`issued patent.
`A record from this system of records may be disclosed, as a routine use, to a Federal, State, or local law
`enforcement agency, if the USPTO becomes aware of a violation or potential violation of law or regulation.
`
`Page 3 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 3 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`SPECIFICATION
`
`SYSTEMS FOR MOBILE IMAGE CAPTURE AND REMITTANCE PROCESSING
`
`RELATED APPLICATIONS INFORMATION
`
`[0001]
`
`This application is a continuation in part of copending U.S. Patent Application Serial
`
`No. 12/906,036 filed on October 15, 2010, which itself is a continuation in part of copending
`
`U.S. Patent Application Serial No. 12/778,943 filed on May 12, 2010 as well as a continuation in
`
`part of U.S. Patent application Serial Number 12/346,026 filed December 30, 2008, now U.S.
`
`Patent No. 7,978,900, which in turn claims the benefit of U.S. Provisional application Serial
`
`Number 61/022,279 filed January 18, 2008, all of which are incorporated herein by reference in
`
`their entirety as if set forth in full. This application is also related to U.S. Patent Application
`
`Serial No. 12/717,080 filed March 3, 2010, which is now U.S. Patent No. 7,778,457, which is
`
`incorporated herein by reference in its entirety as if set forth in full.
`
`BACKGROUND
`
`1.
`
`Technical Field
`
`[0002]
`
`The embodiments described herein generally relate to automated document
`
`processing and more particularly, to systems and methods for document image processing that
`
`enhances an image for data extraction from images captured on a mobile device with camera
`
`capabilities.
`
`2.
`
`Related Art
`
`[0003]
`
`Remittance processing services provide payment processing services for other
`
`businesses. A remittance processing service may be set up by a bank. Often these services set
`
`up one or more post office boxes for receiving payments from customers in the form of paper
`
`checks. Typically the user will include a remittance slip or remittance coupon with the payment.
`
`[0004]
`
`Banks and other businesses often provide remittance slips or coupons that a customer
`
`can include with a payment. These coupons may be included with an invoice or provided to a
`
`117313/000001/142519801
`
`Page 4 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 4 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: l 17313-01 lPRV
`
`PATENT
`
`customer in advance. For example, some mortgage lenders or automobile lenders provide a book
`
`of remittance coupons to customers who then tear out a mail a remittance coupon with each
`
`scheduled payment. The remittance coupons generally include customer account information, an
`
`amount due, and a due date for the payment. The customer account information might include
`
`an account holder name, mailing address, and a customer account number. Other information,
`
`such as the mailing address of the bank or business may also be included on the remittance
`
`coupon. Some remittance coupons also include computer-readable bar codes or code lines that
`
`include text or other computer-readable symbols that can be used to encode account-related
`
`information that can be used to reconcile a payment received with the account for which the
`
`payment is being made. The bar code or code line can be detected and decoded by a computer
`
`system to extract the information encoded therein.
`
`[0005]
`
`In general, the remittance coupon is mailed with a check or other form of payment to
`
`the bank or other business or to a payment processing company designated by the bank or other
`
`business.
`
`In some systems, data entry clerks may manually enter payment information into a
`
`payment processing system.
`
`In other systems, the remittance coupon and/or the check may be
`
`scanned by a computer system for automated processing.
`
`SUMMARY
`
`[0006]
`
`Systems and method are provided for capturing and processing images of remittance
`
`coupons using a mobile device, such as a mobile phone. Embodiments of the systems and
`
`methods described herein provide image optimization and enhancement such that data can be
`
`extracted from the remittance coupon for processing by a remittance processing service. In some
`
`embodiments, an image of a check can also be captured to be processed as a payment associated
`
`with the remittance coupon.
`
`Some embodiments described herein involve a mobile
`
`communication device capturing an image of a document and transmitting the captured image to
`
`a server for image optimization and enhancement. Techniques for assessing the quality of
`
`images of documents captured using the mobile device are also provided. The tests can be
`
`selected based on the type of document that was imaged, the type of mobile application for
`
`which the image quality of the mobile image is being assessed, and/or other parameters such as
`
`117313/000001/142519801
`
`Page 5 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 5 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 1173 13-01 IPRV
`
`PATENT
`
`the type of mobile device and/or the characteristics of the camera of the mobile device that was
`
`used to capture the image. In some embodiments, the image quality assurance techniques can be
`
`implemented on a remote server, such as a mobile phone carrier’s server or a web server, and the
`
`mobile device routes the mobile image to be assessed and optional processing parameters to the
`
`remote server processing and the test results can be passed from the remote server to the mobile
`
`device.
`
`[0007]
`
`In one exemplary aspect, a computer-readable medium is disclosed.
`
`In one
`
`embodiment, the computer-readable medium comprises instructions which, when executed by a
`
`computer, perform aprocess comprising:
`
`receiving an image of a document captured by a
`
`mobile device; identifying a set of characteristics of the document in relation to said image;
`
`processing the image to correct one or more graphical defects, wherein said processing is
`
`adapted to generate a processed image based at least in part upon the set of characteristics;
`
`executing one or more mobile image quality assurance tests on the processed image to assess the
`
`quality of the processed image; and extracting data fiom the processed image.
`
`[0008]
`
`In another exemplary aspect, an apparatus for processing an image captured from a
`
`mobile device is disclosed. In one embodiment, the apparatus comprises: a
`
`receiving module
`
`adapted to receive an image of a document captured by a mobile device; an identification module
`
`in electrical communication with said receiving module, the identification module adapted to
`
`identify a set of characteristics of the document in relation to said image; a processing module in
`
`electrical communication with said identification module, the processing module adapted to
`
`process the image to correct one or more graphical defects present within the image, wherein
`
`said processing is based at least in part upon said set of characteristics; a testing module in
`
`electrical communication with the processing module, the testing module adapted to execute one
`
`or more mobile image quality assurance tests on a processed image in order to assess the quality
`
`of the processed image; and a data extraction module in electrical communication with the
`
`testing module, the data extraction module adapted to extract data fiom the processed image.
`
`117313/000001/142519801
`
`Page 6 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 6 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`[0009]
`
`Other features and advantages of the present invention should become apparent from
`
`the following description of the preferred embodiments,
`
`taken in conjunction with the
`
`accompanying drawings, which illustrate, by way of example, the principles of the invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0010]
`
`The various embodiments provided herein are described in detail with reference to the
`
`following figures. The drawings are provided for purposes of illustration only and merely depict
`
`typical or example embodiments.
`
`These drawings are provided to facilitate the reader’s
`
`understanding of the invention and shall not be considered limiting of the breadth, scope, or
`
`applicability of the embodiments. It should be noted that for clarity and ease of illustration these
`
`drawings are not necessarily made to scale.
`
`[0011]
`
`FIG.
`
`1
`
`is an image of a remittance coupon 100 illustrating an example remittance
`
`coupon that might be imaged with the systems and methods described herein.
`
`[0012]
`
`FIG. 2 is a geometrically corrected image created using image processing techniques
`
`disclosed herein using the mobile image of the remittance coupon 100 illustrated in FIG. 1.
`
`[0013]
`
`FIG. 3 is high level block diagram of a system that can be used to implement the
`
`systems and methods described herein.
`
`[0014]
`
`FIG. 4 is a flow diagram of an example method for capturing an image of a
`
`remittance coupon using a mobile device according to an embodiment.
`
`[0015]
`
`FIG. 5 is a flow diagram of an example method for processing an image of a
`
`remittance coupon captured using a mobile device according to an embodiment.
`
`[0016]
`
`FIG. 6 is a flow diagram illustrating a method for correcting defects to mobile image
`
`according to an embodiment.
`
`[0017]
`
`FIG. 7 and its related description above provide some examples of how a perspective
`
`transformation can be constructed for a quadrangle defined by the comers A, B, C, and D
`
`according to an embodiment.
`
`117313/000001/142519801
`
`Page 7 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 7 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`[0018]
`
`FIG. 8 is a diagram illustrating an example original image, focus rectangle and
`
`document quadrangle ABCD in accordance with the example of FIG. 7.
`
`[0019]
`
`FIG. 9 is a flow chart for a method that can be used to identify the comers of the
`
`remittance coupon in a color image according to an embodiment.
`
`[0020]
`
`FIG. 10 is a flow diagram of a method for generating a bi-tonal image according to an
`
`embodiment.
`
`[0021]
`
`FIG. 11 illustrates an example image binarized image of a remittance coupon
`
`generated from the remittance geometrically corrected remittance coupon image illustrated in
`
`FIG. 2.
`
`[0022]
`
`FIG. 12 is a flow diagram of a method for converting a document image into a
`
`smaller color icon image according to an embodiment.
`
`[0023]
`
`FIG. 13A is a mobile image of a check according to an embodiment.
`
`[0024]
`
`FIG 13B is an example of a color icon image generated using the method of FIG. 12
`
`on the example mobile image of a check illustrated in FIG. 13A according to an embodiment.
`
`[0025]
`
`FIG. 14 is a flow diagram of a method for reducing the color depth of an image
`
`according to an embodiment.
`
`[0026]
`
`FIG. 15A depicts an example of the color “icon” image of FIG. 13B after operation
`
`1302 has divided it into a 3x3 grid in accordance with one embodiment of the invention.
`
`[0027]
`
`FIG. 15B depicts an example of the color “icon” image of FIG. 13B converted to a
`
`gray “icon” image using the method illustrated in FIG. 14 according to an embodiment.
`
`[0028]
`
`FIG. 16 is a flowchart illustrating an example method for finding document comers
`
`from a gray “icon” image containing a document according to an embodiment.
`
`[0029]
`
`FIG. 17 is a flowchart that illustrates an example method for geometric correction
`
`according to an embodiment.
`
`117313/000001/1425198.01
`
`Page 8 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 8 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`[0030]
`
`FIG. 18A is an image illustrating a mobile image of a check that is oriented in
`
`landscape orientation according to an embodiment.
`
`[0031]
`
`FIG. 18B example gray-scale image of the document depicted in FIG. 13A once a
`
`geometrical correction operation has been applied to the image according to an embodiment.
`
`[0032]
`
`FIG. 19 is a flow chart illustrating a method for correcting landscape orientation of a
`
`document image according to an embodiment.
`
`[0033]
`
`FIG. 20 provides a flowchart illustrating an example method for size correction of an
`
`image according to an embodiment.
`
`[0034]
`
`FIG. 21 illustrates a mobile document image processing engine (MDIPE) module
`
`2100 for performing quality assurance testing on mobile document images according to an
`
`embodiment.
`
`[0035]
`
`FIG. 22 is a flow diagram of a process for performing mobile image quality assurance
`
`on an image captured by a mobile device according to an embodiment.
`
`[0036]
`
`FIG. 23 is a flow diagram of a process for performing mobile image quality assurance
`
`on an image of a check captured by a mobile device according to an embodiment.
`
`[0037]
`
`FIG. 24A illustrates a mobile image where the document captured in the mobile
`
`document image exhibits view distortion.
`
`[0038]
`
`FIG. 24B illustrates an example of a grayscale geometrically corrected subimage
`
`generated from the distorted image in FIG. 24A according to an embodiment.
`
`[0039]
`
`FIG. 25A illustrates an example of an in-focus mobile document image.
`
`[0040]
`
`FIG. 25B illustrates an example of an out of focus document.
`
`[0041]
`
`FIG. 26 illustrates an example of a shadowed document.
`
`117313/000001/142519801
`
`Page 9 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 9 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 1173 13-01 IPRV
`
`PATENT
`
`[0042]
`
`FIG. 27 illustrates an example of a grayscale snippet generated from a mobile
`
`document image of a check where the contrast of the image is Very low according to an
`
`embodiment.
`
`[0043]
`
`FIG. 28 illustrates a method for executing a Contrast IQA Test according to an
`
`embodiment.
`
`[0044]
`
`FIG. 29A is an example of a mobile document image that includes a check that
`
`exhibits significant planar skew according to an embodiment.
`
`[0045]
`
`FIG. 29B illustrates an example of a document subimage that exhibits View skew
`
`according to an embodiment.
`
`[0046]
`
`FIG. 30 is a flow chart illustrating a method for testing for View skew according to an
`
`embodiment.
`
`[0047]
`
`FIG. 31 illustrates an example of a mobile document image that features an image of
`
`a document where one of the comers of the document has been cut off in the picture.
`
`[0048]
`
`FIG. 32 illustrates a Cut-Off Comer Test that can be used for testing whether comers
`
`of a document in a document subimage have been cut off when the document was imaged
`
`according to an embodiment.
`
`[0049]
`
`FIG. 33 illustrates an example of a mobile document image that features a document
`
`where one of the ends of the document has been cut off in the image.
`
`[0050]
`
`FIG. 34 is a flow diagram of a method for determining whether one or more sides of
`
`the document are cut off in the document subimage according to an embodiment.
`
`[0051]
`
`FIG. 35 illustrates an example of a mobile document image where the document is
`
`warped according to an embodiment.
`
`[0052]
`
`FIG. 36 is a flow diagram of a method for identifying a warped image and for scoring
`
`the image based on how badly the document subimage is warped according to an embodiment.
`
`117313/000001/142519801
`
`Page 10 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 10 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`[0053]
`
`FIG. 37 illustrates an example of a document subimage within a mobile document
`
`image that is relatively small in comparison to the overall size of the mobile document image
`
`according to an embodiment.
`
`[0054]
`
`FIG. 38 is a flow diagram of a process that for performing an Image Size Test on a
`
`subimage according to an embodiment.
`
`[0055]
`
`FIG. 39A is a flow chart of a method for executing a MICR-line Test according to an
`
`embodiment.
`
`[0056]
`
`FIG. 39B is a flow chart of a method for executing a code line Test according to an
`
`embodiment.
`
`[0057]
`
`FIG. 40 illustrates a method for executing an Aspect Ratio Test according to an
`
`embodiment.
`
`[0058]
`
`FIG. 41 illustrates a method for performing a front-as-rear test on a mobile document
`
`image.
`
`[0059]
`
`FIG 42 is a flow chart of a method for processing an image using form identification
`
`according to an embodiment.
`
`[0060]
`
`FIG 43 is a flow chart of a method for processing an image using dynamic data
`
`capture according to an embodiment.
`
`[0061]
`
`FIG. 44 is a block diagram of various functional elements of a mobile device that can
`
`be used with the various systems and methods described herein according to an embodiment.
`
`[0062]
`
`FIG. 45 is a block diagram of functional elements of a computer system that can be
`
`used to implement the mobile device and/or the servers described in the systems and methods
`
`disclosed herein.
`
`117313/000001/142519801
`
`Page 11 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 11 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-01lPRV
`
`PATENT
`
`[0063]
`
`FIG. 46A-46F are images representing application that can reside on mobile device
`
`and be using in according with the systems and method disclosed herein according to an
`
`embodiment.
`
`[0064]
`
`FIG. 47 is a flow diagram illustrating an exemplary method of configuring a recurring
`
`payment schedule according to an embodiment.
`
`[0065]
`
`FIG. 48 is a flow diagram illustrating an exemplary method of selecting a specific
`
`scheduling preference according to an embodiment.
`
`[0066]
`
`FIG. 49 is a flow diagram illustrating an exemplary method of enabling a user to set
`
`one or more reminders associated with a recurring bill payment according to an embodiment.
`
`DETAILED DESCRIPTION
`
`[0067]
`
`The embodiments described herein are directed towards automated document
`
`processing and systems and methods for document image processing using mobile devices.
`
`System and methods are provided for processing a remittance coupon captured using a mobile
`
`device. Generally, in some embodiments an original color image of a document is captured
`
`using a mobile device and then converted the color image to a bi-tonal image. More specifically,
`
`in some embodiments, a color image of a document taken by a mobile device is received and
`
`converted into a bi-tonal image of the document that is substantially equivalent in its resolution,
`
`size, and quality to document images produced by “standard” scanners.
`
`[0068]
`
`The term “standard scanners” as used herein, but is not limited to, transport scanners,
`
`flat-bed scanners, and specialized check-scanners. Some manufacturers of transport scanners
`
`include UNISYS®, BancTec®, IBM®, and Canon®. With respect to specialized check-scanners,
`
`some models include the TellerScan® TS200 and the Panini® My Vision X. Generally,
`
`117313/000001/142519801
`
`Page 12 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 12 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`standard scanners have the ability to scan and produce high quality images, support resolutions
`
`from 200 dots per inch to 300 dots per inch (DPI), produce gray-scale and bi-tonal images, and
`
`crop an image of a check from a larger full-page size image. Standard scanners for other types
`
`of documents may have similar capabilities with even higher resolutions and higher color-depth.
`
`[0069]
`
`The term “color images” as used herein, but is not limited to, images having a color
`
`depth of 24 bits per a pixel (24 bit/pixel), thereby providing each pixel with one of 16 million
`
`possible colors. Each color image is represented by pixels and the dimensions W (width in
`
`pixels) and H (height in pixels). An intensity function I maps each pixel in the [W x H] area to
`
`its RGB-value. The RGB-value is a triple (R,G,B) that determines the color the pixel represents.
`
`Within the triple, each of the R(Red), G(Green) and B(Blue) values are integers between 0 and
`
`255 that determine each respective color’s intensity for the pixel.
`
`[0070]
`
`The term, “gray-scale images” as used herein, but is not limited to, images having a
`
`color depth of 8 bits per a pixel (8 bit/pixel), thereby providing each pixel with one of 256 shades
`
`of gray. As a person of ordinary skill in the art would appreciate, gray-scale images also include
`
`images with color depths of other various bit levels (e. g. 4 bit/pixel or 2 bit/pixel). Each gray-
`
`scale image is represented by pixels and the dimensions W (width in pixels) and H (height in
`
`pixels). An intensity function I maps each pixel in the [Wx H] area onto a range of gray shades.
`
`More specifically, each pixel has a value between 0 and 255 which determines that pixel’s shade
`
`of gray.
`
`[0071]
`
`Bi-tonal images are similar to gray-scale images in that they are represented by pixels
`
`and the dimensions W (width in pixels) and H (height in pixels). However, each pixel within a
`
`bi-tonal image has one of two colors: black or white. Accordingly, a bi-tonal image has a color
`
`depth of 1 bit per a pixel (l bit/pixel). The similarity transformation, as utilized by some
`
`embodiments of the invention, is based off the assumption that there are two images of [WX H]
`
`and [W’ X H’] dimensions, respectively, and that the dimensions are proportional (i.e. W/ W’ =
`
`H / H’). The term “similarity transformation” may refer to a transformation ST from [WX H]
`
`area onto [W’ XH’] area such that ST maps pixelp =p(x,y) on pixelp’ =p’(x’,y’) with x’ = x *
`
`W’/Wandy =y *H’/H.
`
`117313/000001/1425198.01
`
`Page 13 of 139
`
`10
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 13 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: l 173 13-01 lPRV
`
`PATENT
`
`[0072]
`
`The systems and methods provided herein advantageously allow a user to capture an
`
`image of a remittance coupon, and in some embodiments, a form of payment, such as a check,
`
`for automated processing. Typically, a remittance processing service will scan remittance
`
`coupons and checks using standard scanners that provide a clear image of the remittance coupon
`
`and accompanying check. Often these scanners produce either gray-scale and bi-tonal images
`
`that are then used to electronically process the payment. The systems and methods disclosed
`
`herein allow an image of remittance coupons, and in some embodiments, checks to be captured
`
`using a camera or other imaging device included in or coupled to a mobile device, such as a
`
`mobile phone. The systems and methods disclosed herein can test the quality of a mobile image
`
`of a document captured using a mobile device, correct some defects in the image, and convert the
`
`image to a format that can be processed by remittance processing service.
`
`[0073]
`
`FIG.
`
`l
`
`is an image illustrating an example remittance coupon l900 that can be
`
`imaged with the systems and methods described herein.
`
`The mobile image capture and
`
`processing systems and methods described herein can be used with a variety of documents,
`
`including financial documents such as personal checks, business checks, cashier’s checks,
`
`certified checks, and warrants. By using an image of the remittance coupon l00, the remittance
`
`process can be automated and performed more efficiently. As would be appreciated by those of
`
`skill in the art, remittance coupons are not the only types of documents that might be processed
`
`using the system and methods described herein. For example, in some embodiments, a user can
`
`capture an image of a remittance coupon and an image of a check associated with a checking
`
`account from which the remittance payment will be funded.
`
`[0074]
`
`FIG. 2 is a geometrically corrected image created using image processing techniques
`
`disclosed herein and using the mobile image of the remittance coupon 100 illustrated in FIG. l.
`
`A remittance coupon may include various fields, and some fields in the documents might be
`
`considered “primary” fields. For example, some remittance coupons also include computer-
`
`readable bar codes or code lines 205 that include text or other computer-readable symbols that
`
`can be used to encode account-related information. The account-related information can be used
`
`to reconcile a payment received with the account for which the payment is being made. Code
`
`line 205 can be detected and decoded by a computer system to extract the information encoded
`
`117313/000001/l425l98.0l
`
`Page 14 of 139
`
`ll
`
`ROTHSCHILD EXHIBIT 1010
`
`Page 14 of 139
`
`ROTHSCHILD EXHIBIT 1010
`
`

`
`Atty. Docket No.: 117313-011PRV
`
`PATENT
`
`therein. The remittance coupon can also include an account number field 210 and an amount due
`
`field 215. Remittance coupons can also include other fields, such as the billing company name
`
`and address 220, a total outstanding balance, a minimum payment amount, a billing date, and
`
`payment due date. The examples are merely illustrative of the types of information that may be
`
`included on a remittance coupon and it will be understood that other types of information can be
`
`included on other types of remittance co