Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`SPECIFICATION
`
`SYSTEMS AND METHODS FOR MOBILE IMAGE CAPTURE AND REMITTANCE
`
`PROCESSING
`
`RELATED APPLICATIONS INFORMATION
`
`[001]
`
`This application claims priority to U.S. Provisional Patent Application No.
`
`61/561,772, filed November 18, 2011, now pending, and is a continuation in part of copending
`
`U.S. Patent Application Serial No. 12/906,036 filed on October 15, 2010, now pending, which
`
`itself is a continuation in part of copending U.S. Patent Application Serial No. 12/778,943 filed
`
`on May 12, 2010, now pending, as well as a continuation in part of U.S. Patent Application
`
`Serial Number 12/346,026 filed December 30, 2008, now U.S. Patent No. 7,978,900, which in
`
`turn claims the benefit of U.S. Provisional Application Serial Number 61/022,279 filed January
`
`18, 2008, now expired, all of which are incorporated herein by reference in their entirety as if set
`
`forth in full. This application is also related to U.S. Patent Application Serial No. 12/717,080
`
`filed March 3, 2010, which is now U.S. Patent No. 7,778,457, which is incorporated herein by
`
`reference in its entirety as if set forth in full.
`
`BACKGROUND
`
`1.
`
`Technical Field
`
`[002]
`
`The embodiments described herein generally relate to automated processing of an
`
`image of a financial document captured by a mobile device, and more particularly to capturing,
`
`processing and obtaining data fiom an image of a remittance coupon captured by a mobile device
`
`in order to set up or pay a bill associated with the remittance coupon.
`
`2.
`
`Related Art
`
`[003]
`
`Remittance processing services provide payment processing services for other
`
`businesses. A remittance processing service may be set up by a bank. Often these services set
`
`117313/011UTL/1579235.01
`
`Page 1 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 1 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`up one or more post office boxes for receiving payments from customers in the form of paper
`
`checks. Typically the user will include a remittance slip or remittance coupon with the payment.
`
`[004]
`
`Banks and other businesses often provide remittance slips or coupons that a customer
`
`can include with a payment. These coupons may be included with an invoice or provided to a
`
`customer in advance. For example, some mortgage lenders or automobile lenders provide a book
`
`of remittance coupons to customers who then tear out a mail a remittance coupon with each
`
`scheduled payment. The remittance coupons generally include customer account information, an
`
`amount due, and a due date for the payment. The customer account information might include
`
`an account holder name, mailing address, and a customer account number. Other information,
`
`such as the mailing address of the bank or business may also be included on the remittance
`
`coupon. Some remittance coupons also include computer-readable bar codes or code lines that
`
`include text or other computer-readable symbols that can be used to encode account-related
`
`information that can be used to reconcile a payment received with the account for which the
`
`payment is being made. The bar code or code line can be detected and decoded by a computer
`
`system to extract the information encoded therein.
`
`[005]
`
`In general, the remittance coupon is mailed with a check or other form of payment to
`
`the bank or other business or to a payment processing company designated by the bank or other
`
`business.
`
`In some systems, data entry clerks may manually enter payment information into a
`
`payment processing system.
`
`In other systems, the remittance coupon and/or the check may be
`
`scanned by a computer system for automated processing.
`
`[006]
`
`Other means of paying bills include using online banking, paying a bill directly with
`
`the biller by accessing the biller’s website, or calling the biller to pay the bill over the phone.
`
`Online banking provides convenience to some customers, but the process is cumbersome.
`
`Typically, the consumer needs to first set up the biller on the consumer’s bank’s website, which
`
`means manually entering the biller information, including account number and mailing address.
`
`Then, the consumer must enter the payment information (e.g., the payment amount, payment
`
`date, etc...). This manual data entry is susceptible to user error, can be time consuming, and
`
`inconvenient for consumers, which can inhibit the adoption rate of online banking bill payment.
`
`117313/011UTL/157923501
`
`Page 2 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 2 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: l 17313-01 lTRl
`
`PATENT
`
`[007]
`
`What is needed is a convenient, easy-to-use method for paying a bill which also
`
`maintains or increases the accuracy of the information gathered from the remittance coupon.
`
`SUMMARY
`
`[008]
`
`Systems and methods are provided for capturing and processing images of remittance
`
`coupons using a mobile device, and obtaining data from the captured image which is used to set
`
`up or carry out payment of a bill that corresponds to the remittance coupon. Optimization and
`
`enhancement of image capture and image processing are provided on the mobile device to
`
`improve the initial quality of the captured image and provide a user with real time feedback
`
`regarding the quality of the captured image. The image is then transferred from the mobile
`
`device to a remote server, which performs additional image processing before extracting data
`
`from the remittance coupon to determine one or more parts of the bill, including the biller,
`
`payment address, user account number, amount due, etc. This information is then used to initiate
`
`payment of the bill to a biller or to initiate creation of a new biller account for payment of the
`
`bill.
`
`[009]
`
`In some embodiments, an image of a check can also be captured to be processed as a
`
`payment associated with the remittance coupon. Some embodiments described herein involve a
`
`mobile communication device capturing an image of a document and transmitting the captured
`
`image to a server for image optimization and enhancement. Techniques for assessing the quality
`
`of images of documents captured using the mobile device are also provided. The tests can be
`
`selected based on the type of document that was imaged, the type of mobile application for
`
`which the image quality of the mobile image is being assessed, and/or other parameters such as
`
`the type of mobile device and/or the characteristics of the camera of the mobile device that was
`
`used to capture the image. In some embodiments, the image quality assurance techniques can be
`
`implemented on a remote server, such as a mobile phone carrier’s server or a web server, and the
`
`mobile device routes the mobile image to be assessed and optional processing parameters to the
`
`remote server processing and the test results can be passed from the remote server to the mobile
`
`device.
`
`117313/OIIUTL/l579235.0l
`
`Page 3 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 3 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[010]
`
`In on embodiment, a method of processing a remittance coupon captured by a mobile
`
`device comprises: receiving an image of a remittance coupon captured by a mobile device;
`
`correcting at least one aspect of the image to produce a corrected image; performing a first
`
`content recognition pass on the corrected image to extract content from the remittance coupon;
`
`identifying an address of a biller on the remittance coupon by comparing address content in the
`
`extracted content with an address database; determining biller profile information of the biller,
`
`including an identity of the biller on the remittance coupon, by comparing the identified address
`
`of the biller with a database of biller profile information; and producing a set of billing
`
`information,
`
`including the extracted content and the identity of the biller, for processing a
`
`payment of the bill.
`
`[011]
`
`The method may also comprise using the biller profile information of the biller to
`
`perform a second content recognition pass on the corrected image to extract content from the
`
`remittance coupon, wherein the biller profile information includes at least one of a remittance
`
`coupon format, a remittance coupon mask, a location of at least one field on the remittance
`
`coupon and a format of at least one field.
`
`[012]
`
`The method may also comprise reading a code line on the remittance coupon and
`
`correcting a scale of the remittance coupon based on a size of the code line.
`
`[013]
`
`The correcting of the at least one image may include at least one of a perspective
`
`correction, an aspect ratio correction, a warping correction and a shadow correction.
`
`[014]
`
`The first content
`
`recognition pass may be performed using optical character
`
`recognition (OCR).
`
`[015]
`
`The method may also comprise comparing the address content in the extracted
`
`content with address content extracted by reading a barcode on the remittance coupon before
`
`comparing the address content with the address database.
`
`[016]
`
`The address database may be populated with a plurality of addresses from a United
`
`States Postal Service (USPS) database.
`
`117313/011UTL/1579235.01
`
`Page 4 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 4 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[017]
`
`Comparing the address content in the extracted content with the address database may
`
`include comparing a “zip code plus four digit” field in the extracted content with a zip code plus
`
`four digit field in the address database.
`
`[018]
`
`Comparing address content in the extracted content with an address database may
`
`involve performing a fuzzy search of the address database using the address content.
`
`[019]
`
`The method may also comprise transmitting the set of billing information to the
`
`mobile device to display to a user.
`
`[020]
`
`In another embodiment, a method of processing a remittance coupon on a mobile
`
`device comprises: activating an image capture device on the mobile device; detecting at least one
`
`position setting of the mobile device; capturing an image of a remittance coupon when at least
`
`one of the position settings meets a threshold value; and transmitting the image to a remote
`
`S€I'V€I'.
`
`[021]
`
`The at least one position setting may include at least one of: motion of the mobile
`
`device, an angle of the mobile device with respect to the remittance coupon, and the size of the
`
`remittance coupon within a field of view of the image capture device.
`
`[022]
`
`The threshold value of the motion of the mobile device may be met when there is no
`
`motion of the mobile device for a period of time.
`
`[023]
`
`The threshold value of the angle of the mobile device with respect to the remittance
`
`coupon may be when the angle is approximately zero degrees.
`
`[024]
`
`The threshold value of the size of the remittance coupon within a field of view of the
`
`image capture device may be when all of the edges of the remittance coupon are visible in a
`
`viewfinder of the mobile device.
`
`[025]
`
`The viewfinder of the mobile device may be a display screen of the mobile device,
`
`and wherein the display screen displays a quadrilateral outline to help the user capture the image
`
`of the remittance coupon which meets the threshold value of size.
`
`117313/011UTL/1579235.01
`
`Page 5 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 5 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 1173 13-01 lTRl
`
`PATENT
`
`[026]
`
`The method may further comprise providing feedback to a user of the mobile device
`
`if at least one of the position settings does not meet a threshold value.
`
`[027]
`
`The feedback may include an instruction for correcting at least one of the position
`
`settings of the mobile device.
`
`[028]
`
`The method may further comprise performing at least one image quality test on the
`
`image to determine whether the image meets at least one threshold value.
`
`[029]
`
`The at
`
`least one image quality test may include at
`
`least one of document
`
`identification, de-warping and shadow removal.
`
`[030]
`
`In one exemplary aspect, a computer-readable medium is disclosed.
`
`In one
`
`embodiment, the computer-readable medium comprises instructions which, when executed by a
`
`computer with a processor and a memory, perform a process comprising:
`
`receiving an image of
`
`a remittance coupon captured by a mobile device; correcting at least one aspect of the image to
`
`produce a corrected image; performing a first content recognition pass on the corrected image to
`
`extract content from the remittance coupon; identifying an address of a biller on the remittance
`
`coupon by comparing address content
`
`in the extracted content with an address database;
`
`determining biller profile information of the biller, including an identity of the biller on the
`
`remittance coupon, by comparing the identified address of the biller with a database of biller
`
`profile information; and producing a set of billing information, including the extracted content
`
`and the identity of the biller, for processing a payment of the bill.
`
`[031]
`
`Other features and advantages of the present invention should become apparent from
`
`the following description of the preferred embodiments,
`
`taken in conjunction with the
`
`accompanying drawings, which illustrate, by way of example, the principles of the invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[032]
`
`The various embodiments provided herein are described in detail with reference to the
`
`following figures. The drawings are provided for purposes of illustration only and merely depict
`
`typical or example embodiments.
`
`These drawings are provided to facilitate the reader’s
`
`117313/011UTL/1579235.01
`
`Page 6 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 6 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`understanding of the invention and shall not be considered limiting of the breadth, scope, or
`
`applicability of the embodiments. It should be noted that for clarity and ease of illustration these
`
`drawings are not necessarily made to scale.
`
`[033]
`
`FIG. 1 is a block diagram which illustrates one embodiment of a system for mobile
`
`image capture and remittance processing, according to one embodiment of the invention.
`
`[034]
`
`FIG. 2 illustrates one embodiment of a method of mobile image capture and
`
`remittance processing, according to one embodiment of the invention.
`
`[035]
`
`FIGS. 3A-3F illustrate one embodiment of a plurality of graphical user interfaces
`
`(GUIs) which may be presented to the user on a display screen of the mobile device during the
`
`mobile image capture and remittance processing, according to one embodiment of the invention.
`
`[036]
`
`FIG. 4 is a block diagram which illustrates a workflow of a server-side system for
`
`remittance processing and related components, according to one embodiment of the invention.
`
`[037]
`
`FIG.
`
`5
`
`is a flowchart
`
`illustrating a biller
`
`lookup process, according to one
`
`embodiment of the invention.
`
`[038]
`
`FIG. 6 is a flow diagram illustrating a process for a second data recognition process
`
`on a remittance coupon, according to one embodiment of the invention.
`
`[039]
`
`FIGS. 7A and 7B are images of remittance coupons which illustrate the results of a
`
`first data recognition process and a second data recognition process, according to one
`
`embodiment of the invention.
`
`[040]
`
`FIG. 8 is an image of a remittance coupon captured by a mobile device.
`
`[041]
`
`FIG. 9 is a geometrically corrected image created using image processing techniques
`
`disclosed herein using the mobile image of the remittance coupon illustrated in FIG. 8.
`
`117313/011UTL/1579235.01
`
`Page 7 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 7 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: ll73 13-01 lTRl
`
`PATENT
`
`[042]
`
`FIG. 10 and its related description above provide some examples of how a
`
`perspective transformation can be constructed for a quadrangle defined by the comers A, B, C,
`
`and D according to an embodiment.
`
`[043]
`
`FIG. ll is a diagram illustrating an example original image, focus rectangle and
`
`document quadrangle ABCD in accordance with the example of FIG. l0.
`
`[044]
`
`FIG. l2 is a flow diagram illustrating a method for correcting defects to mobile image
`
`according to an embodiment.
`
`[045]
`
`FIG. l3 is a flow chart for a method that can be used to identify the comers of the
`
`remittance coupon in a color image according to an embodiment.
`
`[046]
`
`FIG. l4 is a flow diagram of a method for generating a bi-tonal image according to an
`
`embodiment.
`
`[047]
`
`FIG.
`
`l5 illustrates a binarized image of a remittance coupon generated from the
`
`geometrically corrected remittance coupon image illustrated in FIG. 9, according to one
`
`embodiment.
`
`[048]
`
`FIG.
`
`l6 is a flow diagram of a method for converting a document image into a
`
`smaller color icon image according to an embodiment.
`
`[049]
`
`FIG. l7A is a mobile image of a check according to an embodiment.
`
`[050]
`
`FIG l7B is an example of a color icon image generated using the method of FIG. l2
`
`on the example mobile image of a check illustrated in FIG. l3A according to an embodiment.
`
`[051]
`
`FIG.
`
`l8 is a flow diagram of a method for reducing the color depth of an image
`
`according to an embodiment.
`
`[052]
`
`FIG. l9A depicts an example of the color “icon” image of FIG. l7B after operation
`
`l302 has divided it into a 3x3 grid in accordance with one embodiment of the invention.
`
`117313/OIIUTL/l579235.0l
`
`Page 8 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 8 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[053]
`
`FIG. 19B depicts an example of the color “icon” image of FIG. 17B converted to a
`
`gray “icon” image using the method illustrated in FIG. 18 according to an embodiment.
`
`[054]
`
`FIG. 20 is a flowchart illustrating an example method for finding document comers
`
`from a gray “icon” image containing a document according to an embodiment.
`
`[055]
`
`FIG. 21 is a flowchart that illustrates an example method for geometric correction
`
`according to an embodiment.
`
`[056]
`
`FIG. 22A is an image illustrating a mobile image of a check that is oriented in
`
`landscape orientation according to an embodiment.
`
`[057]
`
`FIG. 22B example gray-scale image of the document depicted in FIG. 17A once a
`
`geometrical correction operation has been applied to the image according to an embodiment.
`
`[058]
`
`FIG. 23 is a flow chart illustrating a method for correcting landscape orientation of a
`
`document image according to an embodiment.
`
`[059]
`
`FIG. 24 provides a flowchart illustrating an example method for size correction of an
`
`image according to an embodiment.
`
`[060]
`
`FIG. 25 illustrates a mobile document image processing engine (MDIPE) module for
`
`performing quality assurance testing on mobile document images according to an embodiment.
`
`[061]
`
`FIG. 26 is a flow diagram of a process for performing mobile image quality assurance
`
`on an image captured by a mobile device according to an embodiment.
`
`[062]
`
`FIG. 27 is a flow diagram of a process for performing mobile image quality assurance
`
`on an image of a check captured by a mobile device according to an embodiment.
`
`[063]
`
`FIG. 28A illustrates a mobile image where the document captured in the mobile
`
`document image exhibits view distortion.
`
`[064]
`
`FIG. 28B illustrates an example of a grayscale geometrically corrected subimage
`
`generated from the distorted image in FIG. 28A according to an embodiment.
`
`117313/011UTL/1579235.01
`
`Page 9 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 9 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[065]
`
`FIG. 29A illustrates an example of an in-focus mobile document image.
`
`[066]
`
`FIG. 29B illustrates an example of an out of focus document.
`
`[067]
`
`FIG. 30 illustrates an example of a shadowed document.
`
`[068]
`
`FIG. 31 illustrates an example of a grayscale snippet generated from a mobile
`
`document image of a check where the contrast of the image is Very low according to an
`
`embodiment.
`
`[069]
`
`FIG. 32 illustrates a method for executing a Contrast IQA Test according to an
`
`embodiment.
`
`[070]
`
`FIG. 33A is an example of a mobile document image that includes a check that
`
`exhibits significant planar skew according to an embodiment.
`
`[071]
`
`FIG. 33B illustrates an example of a document subimage that exhibits View skew
`
`according to an embodiment.
`
`[072]
`
`FIG. 34 is a flow chart illustrating a method for testing for View skew according to an
`
`embodiment.
`
`[073]
`
`FIG. 35 illustrates an example of a mobile document image that features an image of
`
`a document where one of the comers of the document has been cut off in the picture.
`
`[074]
`
`FIG. 36 illustrates a Cut-Off Comer Test that can be used for testing whether comers
`
`of a document in a document subimage have been cut off when the document was imaged
`
`according to an embodiment.
`
`[075]
`
`FIG. 37 illustrates an example of a mobile document image that features a document
`
`where one of the ends of the document has been cut off in the image.
`
`[076]
`
`FIG. 38 is a flow diagram of a method for determining whether one or more sides of
`
`the document are cut off in the document subimage according to an embodiment.
`
`117313/011UTL/1579235.01
`
`Page 10 of 157
`
`10
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 10 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[077]
`
`FIG. 39 illustrates an example of a mobile document image where the document is
`
`warped according to an embodiment.
`
`[078]
`
`FIG. 40 is a flow diagram of a method for identifying a warped image and for scoring
`
`the image based on how badly the document subimage is warped according to an embodiment.
`
`[079]
`
`FIG. 41 illustrates an example of a document subimage within a mobile document
`
`image that is relatively small in comparison to the overall size of the mobile document image
`
`according to an embodiment.
`
`[080]
`
`FIG. 42 is a flow diagram of a process that for performing an Image Size Test on a
`
`subimage according to an embodiment.
`
`[081]
`
`FIG. 43 is a flow chart of a method for executing a code line test according to an
`
`embodiment.
`
`[082]
`
`FIG. 44 illustrates a method for executing an Aspect Ratio Test according to an
`
`embodiment.
`
`[083]
`
`FIG. 45 is a flow chart of a method for processing an image using form identification
`
`according to an embodiment.
`
`[084]
`
`FIG 46 is a flow chart of a method for processing an image using dynamic data
`
`capture according to an embodiment.
`
`[085]
`
`FIG. 47 is a flow diagram illustrating an exemplary method of configuring a recurring
`
`payment schedule according to an embodiment.
`
`[086]
`
`FIG. 48 is a flow diagram illustrating an exemplary method of selecting a specific
`
`scheduling preference according to an embodiment.
`
`[087]
`
`FIG. 49 is a flow diagram illustrating an exemplary method of enabling a user to set
`
`one or more reminders associated with a recurring bill payment according to an embodiment.
`
`117313/011UTL/1579235.01
`
`Page 11 of 157
`
`ll
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 11 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`[088]
`
`FIG. 50 is a block diagram of various functional elements of a mobile device that can
`
`be used with the various systems and methods described herein according to an embodiment.
`
`[089]
`
`FIG. 51 is a block diagram of functional elements of a computer system that can be
`
`used to implement the mobile device and/or the servers described in the systems and methods
`
`disclosed herein.
`
`DETAILED DESCRIPTION
`
`[090]
`
`The embodiments described herein are directed to processing and payment of a bill
`
`with a mobile device, and more particularly to systems and methods for capturing, processing
`
`and obtaining data from an image of the bill captured by the mobile device to pay the bill
`
`through user interaction with the mobile device. The mobile device includes systems and
`
`methods for capturing the image of the bill and processing the image to ensure adequate image
`
`quality. The image is then sent from the mobile device to a remote server, where additional
`
`image processing is performed to improve the quality of the image and then extract data from the
`
`image that is relevant to paying the bill. The extracted data may be verified through comparisons
`
`with databases which store information on billers (payees), bill formats and other relevant
`
`content that will appear on the bill.
`
`[091]
`
`The embodiments described herein provide an end-to-end solution for capturing
`
`information from a bill via a camera on a mobile device and using that information to automate
`
`the payment of the bill.
`
`[092]
`
`FIG. 1 illustrates one embodiment of a system 100 for mobile image capture and
`
`remittance processing. The system 100 includes a mobile device 102, such as a cellular phone,
`
`smartphone, tablet, personal digital assistant (PDA) or other portable electronic device that may
`
`be connected with a communications network. The mobile device 102 will include an image
`
`capture device (not shown), such as a digital camera or a portable scanning device, which is
`
`capable of capturing an image of a document. The mobile device 102 is connected with a remote
`
`server 104 over a network so that the mobile device 102 can transmit captured images to the
`
`remote server 104. The remote server 104 performs additional image processing and data
`
`117313/011UTL/1579235.01
`
`Page 12 of 157
`
`12
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 12 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`extraction, as will be described in further detail below, in order to determine information about
`
`the remittance coupon and identify the appropriate biller and payment information.
`
`In one
`
`embodiment, the remote server 104 may be connected with an address database 106 which is
`
`used to verify address information obtained from the remittance coupon, as will be described in
`
`further detail below. The remote server 104 may also be connected with a biller database 108
`
`which stores information on billers, such as address information and billing formats for the
`
`remittance coupons. Once the remote server 104 has extracted and identified all of the relevant
`
`data from the image of the remittance coupon, the extracted data and the captured and processed
`
`images may be stored in a content database 110 connected with the remote server 104.
`
`The
`
`extracted data may then be transmitted to a banking server 112 for processing the payment from
`
`a bank account belonging to the user of the mobile device 102. The extracted data may also be
`
`first sent back to the mobile device 102 to display the data to a user for confirmation before a bill
`
`is paid.
`
`[093]
`
`The mobile device can comprise a mobile telephone handset, Personal Digital
`
`Assistant, or other mobile communication device. The mobile device can include a camera or
`
`other imaging device, such as a scanner, or might include functionality that allows it to connect
`
`to a camera or other imaging device. The connection to an external camera or other imaging
`
`device can comprise a wired or wireless connection.
`
`In this way the mobile device can connect
`
`to an external camera or other imaging device and receive images from the camera or other
`
`imaging device.
`
`[094]
`
`Images of the documents taken using the mobile device or downloaded to the mobile
`
`device can be transmitted the remote server via a network. The network can comprise one or
`
`more wireless and/or wired network connections. For example, in some cases, the images can be
`
`transmitted over a mobile communication device network, such as a code division multiple
`
`access (“CDMA”) telephone network, or other mobile telephone network. The network can also
`
`comprise one or more connections across the Internet.
`
`Images taken using, for example, a
`
`mobile device’s camera, can be 24 bit per pixel (24 bit/pixel) JPG images. It will be understood,
`
`however, that many other types of images might also be taken using different cameras, mobile
`
`devices, etc.
`
`117313/011UTL/1579235.01
`
`Page 13 of 157
`
`13
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 13 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: l 17313-01 lTRl
`
`PATENT
`
`[095]
`
`The remote server can be configured to perform various image processing techniques
`
`on images of remittance coupons, checks, or other financial documents captured by the mobile
`
`device. The remote server can also be configured to perform various image quality assurance
`
`tests on images of remittance coupons or financial documents captured by the mobile device to
`
`ensure that the quality of the captured images is sufficient to enable remittance processing to be
`
`performed using the images. Examples of various processing techniques and testing techniques
`
`that can be implemented on the remote server are described in detail below.
`
`[096]
`
`According to an embodiment, the remote server can be configured to communicate to
`
`one or more bank servers via the network. The bank server can be configured to process
`
`payments in some embodiments. For example, in some embodiments, mobile device can be used
`
`to capture an image of a remittance coupon and an image of a check that can be used to make an
`
`electronic payment of the remittance payment. For example, the remote server can be configured
`
`to receive an image of a remittance coupon and an image of a check fiom the mobile device.
`
`The bank server can electronically deposit the check into a bank account associated with the
`
`entity for which the electronic remittance is being performed (payor). According to some
`
`embodiments, the bank server and the remote server can be implemented on the same server or
`
`same set of servers.
`
`[097]
`
`In other embodiments, the remote server can handle payment. For example, the
`
`remote server can be operated by or on behalf of an entity associated with the coupon of FIG. 8,
`
`such as a utility or business. The user’s account can then be linked with a bank, Paypal®, or
`
`other account, such that when remote server receives the remittance information, it can charge
`
`the appropriate amount to the user’s account.
`
`I.
`
`MOBILE DEVICE CAPTURE AND REMITTANCE PROCESSING
`
`[098]
`
`The capturing of the image of the financial document is the first step of an end-to-end
`
`solution for processing financial documents using mobile device cameras, which can be utilized
`
`to provide the user with tools and information to improve the quality of the image and decrease
`
`the chance of errors from poor image quality. However, by having the mobile device carry out
`
`several image processing steps, the overall user experience may be improved, due to the fact that
`
`117313/0llUTL/l579235.0l
`
`Page 14 of 157
`
`l4
`
`ROTHSCHILD EXHIBIT 1015
`
`Page 14 of 157
`
`ROTHSCHILD EXHIBIT 1015
`
`

`
`Atty. Docket No.: 117313-011TR1
`
`PATENT
`
`the image of the financial document which is eventually sent to the remote server will be of
`
`substantially higher quality. A higher quality image means it is much less likely that the image
`
`will be rejected by the remote server, which would otherwise require the user to capture another
`
`image of the financial document. By running an application on the mobile device, problems with
`
`the captured image can be immediately identified and corrected without waiting for transmission
`
`of the image to the remote server, analysis at the server, and feedback from the remote server to
`
`the user.
`
`[099]
`
`FIG. 2 illustrates one embodiment of a method of mobile image capture and
`
`remittance processing, as will be further described herein. The workflow of the methods
`
`described herein start with the mobile device, the steps of which are illustrated on the left portion
`
`of FIG. 2.
`
`In a first step S202, a mobile billing application is initialized on the mobile device.
`
`The mobile billing application may be a software application stored on the mobile device which
`
`is configured to coordinate with an image capture device and which displays various graphical
`
`user interfaces (GUIs) on a display of the mobile device that leads the user through the mobile
`
`capture process, such as verification of user credentials, and then presents them with the option
`
`of paying a bill or setting up a new bill for payment.
`
`[0100]
`
`In a next step S204, the mobile device activates an image capture device to capture an
`
`image of a remittance coupon with the image capture device, such as a camera, that is coupled
`
`with, or embedded within, the mobile device.
`
`In one embodiment, the user may manually
`
`capture the image by depressing an appropriate button or command on the mobile device which
`
`operates the image capture device and captures an image of the remittance coupon.
`
`In another
`
`embodiment, the mobile billing application may be programmed to control the image capture
`
`device and automatically capture an image when certain requirements or thresholds are met
`
`which will provide for better image quality. Additional features of the automatic capture process
`
`are provided below.
`
`[0101]
`
`In step S206, one