PATENT
`
`APPLICATION
`
`of
`
`Grigori Nepomniachtchi
`
`For
`
`UNITED STATES LETTERS PATENT
`
`On
`
`SYSTEMS FOR MOBILE IMAGE CAPTURE AND PROCESSING OF CHECKS
`
`Attorney Docket No.: 11JN—144857
`
`Sheets of Drawings: Twenty-one (21)
`
`Attorneys
`
`SHEPPARD MULLIN RICHTER & HAMPTON LLP
`
`333 South Hope Street
`Forty-Eighth Floor
`Los Angeles, CA 90071
`Telephone:
`(858) 720-8900
`Facsimile:
`(858) 509-3691
`
`Page I of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 1 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`Related Application
`
`[0001]
`
`This application claims the benefit of US, Provisional application(s) Serial
`
`Number 61/022,279 filed January 1, 2008, which is hereby incorporated herein by reference.
`
`Field of the Invention
`
`[0002]
`
`The present invention relates generally to automated document processing and
`
`more particularly, to systems and methods for document image processing that enhances an
`
`image for data extraction from images captured on a mobile device with camera capabilities.
`
`Background of the Invention
`
`[0003]
`
`In general, financial institutions have automated most check processing systems
`
`by printing financial information, such as account numbers and bank routing numbers, onto the
`
`checks. Before a check amount is deducted from a payer’s account, the amount, account
`
`number, and other important information must be extracted from the check. This highly
`
`automated form of extraction is done by a check processing control system that captures
`
`information from the Magnetic Ink Character Recognition (“MICR”) line. The MICR line
`
`consists of specially designed numerals that are printed on the bottom of a check using magnetic
`
`ink. The MICR data fields include the bank routing number, bank transit number, account
`
`number, check serial number, check amount, process code and extended process code.
`
`[0004]
`
`Checks and other documents may be processed by banks and other financial
`
`institutions in large numbers. The documents that may be processed might include checks,
`
`deposit slips, payment slips, etc. In some cases the banks or other financial institutions may be
`
`required to use the actual physical documents. For example, checks might need to be transported
`
`between multiple banks or other financial institutions. This may slow down the processing of
`
`financial documents. In addition, other types of documents that are non-financial in nature may
`
`be processed by businesses and other institutions in large volumes.
`
`WO2—WEST:6AXS1\401 1881917
`
`-2-
`
`Page 2 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 2 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`Summary of the Invention
`
`[0005]
`
`In order to facilitate processing of a document depicted in an image captured by a
`
`mobile device, embodiments of the systems and methods described herein provide image
`
`optimization and enhancement such that data can be extracted from the document. Some
`
`systems and methods described herein specifically involve a mobile communication device
`
`capturing an image of a document and then transmitting that image to a server for image
`
`optimization and enhancement.
`
`[0006]
`
`The present invention relates to automated document processing and more
`
`particularly, to methods and systems for document image capture and processing using mobile
`
`devices.
`
`In accordance with various embodiments, methods and systems for document image
`
`capture on a mobile communication device are provided such that the image is optimized and
`
`enhanced for data extraction from the document as depicted. These methods and systems may
`
`comprise capturing an image of a document using a mobile communication device; transmitting
`
`the image to a server; and processing the image to create a bi—tonal image of the document for
`
`data extraction. For example, a mobile communication devices, such as a camera phone, would
`
`transmit the image of the document to the server, where the image is processed and results in a
`
`bi—tonal image of the document.
`
`[0007]
`
`Some embodiments of the invention may allow the users to transmit images of the
`
`documents using a mobile communication device. Additionally, methods and systems are
`
`disclosed that allow the transmission of such information using a mobile communication device
`
`such as, for example, a mobile telephone handset with a camera (also known as a camera phone).
`
`Many people may benefit from these systems and methods because a large number of people
`
`currently carry and use handheld mobile communication devices.
`
`[0008]
`
`In accordance with some embodiments of the invention, methods and systems for
`
`document capture on a mobile communication device further comprise requiring a user to login
`
`into an application. In this way access to the document capture system using a mobile
`
`communication device might be limited to authorized users. The methods and systems may
`
`further comprise selecting a type of document and entering an amount. Some systems may
`
`receive a status at the mobile communication device.
`
`WO2—WEST:6AXS1\401188191.?
`
`-3-
`
`Page 3 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 3 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0009]
`
`In other various embodiments, processing the image may comprise processing the
`
`image on the mobile communication device, processing the image on the server or processing the
`
`image on the mobile communication device and the server. Processing the image may comprise
`
`converting the image to gray~scale, detecting a quadrangle and correcting the image. In some
`
`embodiments, processing the image may comprise converting the image to a bi-tonal image.
`
`[0010]
`
`In yet other embodiments, the methods and systems in accordance with the
`
`invention may comprise capturing an image of a document using the mobile communication
`
`device; automatically detecting the document within the captured image; geometrically
`
`correcting the image; binarizing the captured image; correcting the orientation of the captured
`
`image; correcting the size of the captured image; and outputting the modified captured image of
`
`the document.
`
`[0011]
`
`In further embodiments, the automatic detection of the document may comprise
`
`determining a plurality of corners belonging to the document depicted within the captured image.
`
`In yet further embodiments, the automatic detection of the document may comprise converting
`
`the first image to a color “icon” image; reducing color within the color “icon” image, thereby
`
`resulting in a gray~scale “icon” image; and determining the plurality of corners belonging to the
`
`document depicted within the captured image.
`
`[0012]
`
`In other embodiments, the geometric correction comprises reducing color within
`
`the captured image, resulting in a gray~scale image; building a projective transformation model
`
`that maps the document within the gray-scale image to a gray-scale document image; and
`
`applying the projective transformation model to the first image, resulting in the gray-scale
`
`document image. Further embodiments include a geometric correction further comprising
`
`correcting the orientation of the document within a gray-scale “icon” image if the document
`
`within the captured image is in landscape orientation; and building the projective transformation
`
`model from the gray-scale “icon” image.
`
`[0013]
`
`In some embodiments, correcting the orientation of the captured image comprises
`
`correcting the orientation of the document within the third image if the document is in upside~
`
`down orientation. In some such embodiments, correcting the orientation of the captured image
`
`further comprises determining the orientation of the document within the third image using an
`
`MICR-line on the document.
`
`W02-WEST:6AXS1\401188191.7
`
`"4"
`
`Page 4 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 4 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0014]
`
`In other embodiments, correcting the size of the fourth image comprises reading a
`
`relevant object of a known position on the document within the captured image; computing an
`
`average width of the relevant object; computing a scaling factor based on the average width of
`
`the relevant object; using the scaling factor to determine whether the captured image needs a size
`
`correction; applying a size correction to the captured image, resulting in a resized image;
`
`geometrically correcting the resized captured image, resulting in a corrected captured image;
`
`binarizing the corrected captured image, resulting in a binarized image; and outputting the
`
`binarized modified captured image.
`
`[0015]
`
`In various embodiments, the captured image is a color image. In other
`
`embodiments, the outputted modified captured image is a bi—tonal image of the document. In yet
`
`further embodiments, the outputted modified captured image is a gray—scale image of the
`
`document.
`
`[0016]
`
`In further embodiments, the mobile communication device is a camera phone. In
`
`yet further embodiments, the mobile communication device tranmists the image od the document
`
`to the server. In some of these embodiments, once the server receives the image, the image
`
`processed, resulting in a optimized and enhanced image.
`
`[0017]
`
`In some embodiments of the invention, a computer program product is provided,
`
`comprising a computer useable medium having computer program code embodied therein for
`
`enabling a computing device to perform operations in accordance with some of the methods
`
`described herein.
`
`[0018]
`
`Other features and advantages of the present invention should become apparent
`
`from the following description of the preferred embodiments, taken in conjunction with the
`
`accompanying drawings, which illustrate, by way of example, the principles of the invention.
`
`Brief Description of the Drawings
`
`[0019]
`
`The present invention, in accordance with one or more various embodiments, is
`
`described in detail with reference to the following figures. The drawings are provided for
`
`purposes of illustration only and merely depict typical or example embodiments of the invention.
`
`These drawings are provided to facilitate the reader’s understanding of the invention and shall
`
`W02-WEST:6AXS1\4011881917
`
`‘S’
`
`Page 5 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 5 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`not be considered limiting of the breadth, scope, or applicability of the invention. It should be
`
`noted that for clarity and ease of illustration these drawings are not necessarily made to scale.
`
`[0020]
`
`FIG. 1 is a diagram illustrating an example check that might be imaged with the
`
`systems and methods described herein.
`
`[0021]
`
`FIG. 2 is a diagram illustrating an example payment coupon that might be
`
`imaged using the systems and methods described herein.
`
`[0022]
`
`FIG. 3 is a diagram illustrating an example out-of-focus image of the check
`
`illustrated in FIG. 1.
`
`[0023]
`
`FIG. 4 is a diagram illustrating an example out-of-focus image of the payment
`
`coupon illustrated in FIG. 2.
`
`[0024]
`
`FIG. 5 is a diagram illustrating an example of perspective distortion in an image
`
`of a rectangular shaped document.
`
`[0025]
`
`FIG. 6 is a diagram illustrating an example original image, focus rectangle and
`
`document quadrangle ABCD in accordance with the example of FIG. 5.
`
`[0026]
`
`FIG. 7 is a flowchart illustrating an example method in accordance with the
`
`systems and methods described herein.
`
`[0027]
`
`FIG. 8 is a diagram illustrating an example bi-tonal image of the check of Figures
`
`1 and 3 in accordance with the systems and methods described herein.
`
`[0028]
`
`FIG. 9 is a diagram illustrating an example bi-tonal image of the payment coupon
`
`of Figures 2 and 4 in accordance with the systems and methods described herein.
`
`[0029]
`
`FIG. 10 is a flowchart of an example method in accordance with the invention
`
`that is used during image processing stages in accordance with the systems and methods
`
`described herein.
`
`[0030]
`
`FIG. 11a is a flowchart illustrating an example method for automatic document
`
`detection within a color image from a mobile device in accordance with the systems and methods
`
`described herein.
`
`[0031]
`
`FIG. 11b is an example mobile image depicting a check where the corners have
`
`been detected in accordance with the systems and methods described herein.
`
`W02-WEST:6AXS1\4011881917
`
`-6-
`
`Page 6 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 6 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0032]
`
`FIG. 12a is a flowchart illustrating an example method for converting a color
`
`image to a smaller “icon” image in accordance with the systems and methods described herein.
`
`[0033]
`
`FIG. 12b is a mobile image depicting an example of the mobile image of FIG. 11b
`
`after being converted into a color “icon” image in accordance with the systems and methods
`
`described herein.
`
`[0034]
`
`FIG. 13a is a flowchart illustrating an example method for color depth reduction
`
`in accordance with the systems and methods described herein.
`
`[0035]
`
`FIG. 13b is a mobile image depicting an example of the color “icon” image of
`
`FIG. 12b after a color depth reduction operation has divided it into a 3x3 grid in accordance with
`
`the systems and methods described herein.
`
`[0036]
`
`FIG. 13c is a mobile image depicting an example of the of the color “icon” image
`
`of FIG. 12b once it has been converted to a gray “icon” image by a color depth reduction
`
`operation in accordance with the systems and methods described herein.
`
`[0037]
`
`FIG. 14 is a flowchart illustrating an example method for finding document
`
`corners from a gray “icon” image in accordance with the systems and methods described herein.
`
`[0038]
`
`FIG. 15a is a flowchart illustrating an example method for geometric correction in
`
`accordance with the systems and methods described herein.
`
`[0039]
`
`FIG. 15b is an example mobile image depicting a check in landscape orientation.
`
`[0040]
`
`FIG. 15c is a mobile image depicting an example of the mobile image of FIG. 11b
`
`after a geometric correction operation in accordance with the systems and methods described
`
`herein.
`
`[0041]
`
`FIG. 16a is a flowchart illustrating an example method for binarization in
`
`accordance with the systems and methods described herein.
`
`[0042]
`
`FIG. 16b is a mobile image depicting an example of the mobile image of FIG. 15c
`
`after it has been converted to a bi—tonal image by a binarization operation in accordance with the
`
`systems and methods described herein.
`
`W02-WEST:6AXSl\40l 1881917
`
`-7-
`
`Page 7 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 7 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0043]
`
`FIG. 17a is a flowchart illustrating an example method for correcting the upside-
`
`down orientation of a document within a mobile image in accordance with the systems and
`
`methods described herein.
`
`[0044]
`
`FIG. 17b is an example bi-tonal image depicting a check in an upside-down
`
`orientation.
`
`[0045]
`
`FIG. 18 is a flowchart illustrating an example method for size correction of an
`
`image in accordance with the systems and methods described herein.
`
`[0046]
`
`FIG. 19 is a simplified block diagram illustrating an example-computing module
`
`in accordance with one embodiment of the invention.
`
`Detailed Description of the Preferred Embodiments
`
`[0047]
`
`The present invention is directed towards automated document processing and
`
`systems and methods for document image processing using mobile devices. Generally, some
`
`embodiments of the invention capture an original color image of a document using a mobile
`
`device and then converted the color image to a bi-tonal image. More specifically, some
`
`embodiments accept a color image of a check taken by a mobile device and convert it into a bi-
`
`tonal image of the check that is substantially equivalent in its resolution, size, and quality to
`
`check images produced by “standar ” scanners.
`
`[0048]
`
`Before describing the invention in greater detail, it would be useful to define some
`
`of the common terms used herein when describing various embodiments of the invention.
`
`[0049]
`
`The term “standard scanners” includes, but is not limited to, transport scarmers,
`
`flat-bed scanners, and specialized check-scanners. Some manufacturers of transport scanners
`
`include UNISYS®, BancTec®, IBM®, and Canon®. With respect to specialized check-
`
`scanners, some models include the TellerScan® TS200 and the Panini® My Vision X.
`
`Generally, standard scanners have the ability to scan and produce high quality images, support
`
`resolutions from 200 dots per inch to 300 dots per inch (DPI), produce gray-scale and bi-tonal
`
`images, and crop an image of a check from a larger full-page size image. Standard scanners for
`
`other types of documents may have similar capabilities with even higher resolutions and higher
`
`color-depth.
`
`W02-WEST:6AXS1\4011881917
`
`'8'
`
`Page 8 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 8 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0050]
`
`The term “color images” includes, but is not limited to, images having a color
`
`depth of 24 bits per a pixel (24 bit/pixel), thereby providing each pixel with one of 16 million
`
`possible colors. Each color image is represented by pixels and the dimensions W(width in
`
`pixels) and H (height in pixels). An intensity function I maps each pixel in the [Wx H] area to
`
`its RGB—value. The RGB—value is a triple (R,G,B) that determines the color the pixel represents.
`
`Within the triple, each of the R(Red), G(Green) and B(Blue) values are integers between 0 and
`
`255 that determine each respective color’s intensity for the pixel.
`
`[0051]
`
`The term, “gray—scale images” includes, but is not limited to, images having a
`
`color depth of 8 bits per a pixel (8 bit/pixel), thereby providing each pixel with one of 256 shades
`
`of gray. As a person of ordinary skill in the art would appreciate, gray—scale images also include
`
`images with color depths of other various bit levels (e. g. 4 bit/pixel or 2 bit/pixel). Each gray-
`
`scale image is represented by pixels and the dimensions W (width in pixels) and H (height in
`
`pixels). An intensity function I maps each pixel in the [Wx H] area onto a range of gray shades.
`
`More specifically, each pixel has a value between 0 and 255 which determines that pixel’s shade
`
`of gray.
`
`[0052]
`
`Bi—tonal images are similar to gray—scale images in that they are represented by
`
`pixels and the dimensions W(width in pixels) and H (height in pixels). However, each pixel
`
`within a bi—tonal image has one of two colors: black or white. Accordingly, a bi—tonal image has
`
`a color depth of 1 bit per a pixel (1 bit/pixel). The similarity transformation, as utilized by some
`
`embodiments of the invention, is based off the assumption that there are two images of [WX H]
`
`and [W’ X H’] dimensions, respectively, and that the dimensions are proportional (i.e. W/ W’ =
`
`H /H’). The term “similarity transformation” may refer to a transformation ST from [WX H]
`
`area onto [W’ X H’] area such that ST maps pixel p = p(x,y) on pixel p ’ = p’(x’,y’) with x’ = x *
`
`W’/Wandy =y *H’/H.
`
`[0053]
`
`Figure 1 is a diagram illustrating an example check 100 that might be imaged with
`
`the systems and methods described herein. The mobile image capture and processing systems
`
`and methods may be used with a variety of documents, including financial documents such as
`
`personal checks, business checks, cashier’s checks, certified checks, warrants or other
`
`documents. By using an image of the check 100, the check clearing process is performed more
`
`efficiently. As would be appreciated by those of skill in the art, checks are not the only type of
`
`W02-WEST:6AXS1\401188191.7
`
`-9-
`
`Page 9 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 9 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`documents that may be used with these systems. For example, other documents, such as deposit
`
`slips, might also be processed using the systems and methods described herein. Figure 2 is a
`
`diagram illustrating an example payment coupon 200 that might be imaged using the systems
`
`and methods described herein.
`
`[0054]
`
`In some embodiments, checks 100, payment coupons 200, or other documents
`
`might be imaged using a mobile device. The mobile device may be a mobile telephone handset,
`
`Personal Digital Assistant, or other mobile communication device. The mobile device may
`
`include a camera, or might include functionality that allows it to connect to a camera. This
`
`connection might be wired or wireless. In this way the mobile device may connect to an external
`
`camera and receive images from the camera.
`
`[0055]
`
`Images of the documents taken using the mobile device or downloaded to the
`
`mobile device may be transmitted to a server. For example, in some cases, the images may be
`
`transmitted over a mobile communication device network, such as a code division multiple
`
`access (“CDMA”) telephone network, or other mobile telephone network. Images taken using,
`
`for example, a mobile device’s camera, may be 24 bit per pixel (24 bit/pixel) JPG images.
`
`It will
`
`be understood, however, that many other types of images might also be taken using different
`
`cameras, mobile devices, etc.
`
`[0056]
`
`Various documents may include various fields. Some of the fields in the
`
`documents might be considered “primary” fields. For example, the primary fields of interest on
`
`a check 100 might include the legal 102 and courtesy 104 amounts and the MICR line 106.
`
`Other fields of interest may include the payee 108, date 110 and the signature 112. The primary
`
`fields of interest for the payment coupon 200 might include the payment amounts 202, such as
`
`the balance, minimum payment and interest. The billing company name and address 204, the
`
`account number 206 and the code—line 208 may also be fields of interest. In some embodiments
`
`it may be necessary to electronically read various information from these fields on a document.
`
`For example, in order to process a check that is to be deposited, it might be necessary to
`
`electronically read the legal 102 and courtesy 104 amounts, the MICR line 106, the payee 108,
`
`date 110 and the signature 112 on the check. In some cases, this information is difficult to read
`
`because, for example, the check or other document is out of focus or is otherwise poorly imaged.
`
`W02-WEST:6AXS1\4011881917
`
`‘I0’
`
`Page 10 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 10 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0057]
`
`Figure 3 is a diagram illustrating an example out-of-focus image of the check
`
`illustrated in Figure 1. In some cases, document images might be out of focus. An image of a
`
`document that is out of focus may be difficult or impossible to read, electronically process, etc.
`
`For example, it might be difficult to read the amount 302 and 304 or the payee 306 on the image
`
`300 of the check 100. Figure 4 is a diagram illustrating an example out-of-focus image of the
`
`payment coupon illustrated in Figure 2. Because the image 400 of the payment coupon 200 is
`
`out of focus it might be difficult to properly credit the payment. For example, the payment might
`
`be credited to the wrong account or an incorrect amount might be credited. This may be
`
`especially true if a check and a payment coupon are both difficult to read or the scan quality is
`
`poor.
`
`[0058]
`
`Many different factors may affect the quality of an image and the ability of a
`
`mobile device based image capture and processing system. Optical defects, such as out-of-focus
`
`images (as discussed above), unequal contrast or brightness, or other optical defects, might make
`
`it difficult to process an image of a document (e. g., a check, payment coupon, deposit slip, etc.)
`
`The quality of an image may also be affected by the document position on a surface when
`
`photographed or the angle at which the document was photographed. This affects the image
`
`quality by causing the document to appear, for example, right side up, upside down, skewed, etc.
`
`Further, if a document is imaged while upside-down it might be impossible or nearly impossible
`
`to for the system to determine the information contained on the document.
`
`[0059]
`
`In some cases, the type of surface might affect the final image. For example, if a
`
`document is sitting on a rough surface when an image is taken, that rough surface might show
`
`through. In some cases the surface of the document might be rough because of the surface below
`
`it. Additionally, the rough surface may cause shadows or other problems that might be picked up
`
`by the camera. These problems might make it difficult or impossible to read the information
`
`contained on the document.
`
`[0060]
`
`Lighting may also affect the quality of an image, for example, the location of a
`
`light source and light source distortions. Using a light source above a document might light the
`
`document in a way that improves the image quality, while a light source to the side of the
`
`document might produce an image that is more difficult to process. Lighting from the side
`
`might, for example, cause shadows or other lighting distortions. The type of light might also be
`
`W02-WEST:6AXS1\4011881917
`
`-11-
`
`Page 11 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 11 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`a factor, for example, sun, electric bulb, florescent lighting, etc. If the lighting is too bright, the
`
`document might be washed out in the image. On the other hand, if the lighting is too dark, it
`
`might be difficult to read the image.
`
`[0061]
`
`The quality of the image might also be affected by document features, such as, the
`
`type of document, the fonts used, the colors selected, etc. For example, an image of a white
`
`document with black lettering may be easier to process than a dark colored document with black
`
`letters. Image quality may also be affected by the mobile device used. Some mobile camera
`
`phones, for example, might have cameras that save an image using a greater number of mega
`
`pixels. Other mobile cameras phones might have an auto-focus feature, automatic flash, etc.
`
`Generally, these features may improve an image when compared to mobile devices that do not
`
`include such features.
`
`[0062]
`
`A document image taken using a mobile device might have one or more of the
`
`defects discussed above. These defects or others may cause low accuracy when processing the
`
`image, for example, when processing one or more of the fields on a document. Accordingly, in
`
`some embodiments, systems and methods using a mobile device to create images of documents
`
`may include the ability to identify poor quality images. If the quality of an image is determined
`
`to be poor, a user may be prompted to take another image.
`
`[0063]
`
`A variety of metrics might be used to detect an out-of-focus image. For example,
`
`a focus measure may be employed. The focus measure may be the ratio of the maximum video
`
`gradient between adjacent pixels measured over the entire image and normalized with respect to
`
`an image’s gray level dynamic range and "pixel pitch". The pixel pitch may be the distance
`
`between dots on the image. In some embodiments a focus score might be used to determine if an
`
`image is adequately focused. If an image is not adequately focused, a user might be prompted to
`
`take another image.
`
`[0064]
`
`An image focus score might be calculated as a function of maximum video
`
`gradient, gray level dynamic range and pixel pitch. For example, in one embodiment:
`
`Image Focus Score = (Maximum Video Gradient)*(Gray Level Dynamic Range)*(Pixel Pitch)
`
`(eq. 1)
`
`W02-WEST:6AXS1\4011881917
`
`-12-
`
`Page 12 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 12 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`[0065]
`
`The video gradient may be the absolute value of the gray level for a first pixel "i"
`
`minus the gray level for a second pixel "i+l ". For example:
`
`Video Gradient = ABS[(Grey level for pixel "i") — (Gray level for pixel "i+l")] (eq. 2)
`
`[0066]
`
`The gray level dynamic range may be the average of the "n" lightest pixels minus
`
`the average of the "n" darkest pixels. For example:
`
`Gray Level Dynamic Range = [AVE("N" lightest pixels) — AVE("N" darkest pixels)] (eq. 3)
`
`[0067]
`
`In equation 3 above, N may be defined as the number of pixels used to determine
`
`the average darkest and lightest pixel gray levels in the image. In some embodiments, N might
`
`be chosen to be 64. Accordingly, in some embodiments, the 64 darkest pixels are averaged
`
`together and the 64 lightest pixels are averaged together to compute the gray level dynamic range
`
`value.
`
`[0068]
`
`The pixel pitch may be the reciprocal of the image resolution, for example, in dots
`
`per inch.
`
`Pixel Pitch = [1 / Image Resolution] (eq. 4)
`
`[0069]
`
`In other words, as defined above, the pixel pitch is the distance between dots on
`
`the image because the Image Resolution is the reciprocal of the distance between dots on an
`
`image.
`
`[0070]
`
`Figure 5 is a diagram illustrating an example of perspective distortion in an image
`
`of a rectangular shaped document. An image may contain perspective transformation distortions
`
`500 such that a rectangle might become a quadrangle ABCD 502, as illustrated in the figure.
`
`The perspective distortion may occur because an image is taken using a camera that is placed at
`
`an angle to a document rather than directly above the document. When directly above a
`
`rectangular document it will generally appear to be rectangular. As the imaging device moves
`
`from directly above the surface, the document distorts until it can no longer be seen and only the
`
`edge of the page may be seen.
`
`[0071]
`
`The dotted frame 504 comprises the image frame obtained by the camera. The
`
`image frame is be sized h x W, as illustrated in the figure. Generally, it may be preferable to
`
`contain an entire document within the h x W frame of a single image. It will be understood,
`
`WO2—WEST:6AXSl\40l 1331917
`
`-13-
`
`Page 13 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 13 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`however, that some documents might be too large or include too many pages for this to be
`
`preferable or even feasible.
`
`[0072]
`
`In some embodiments, an image might be processed, or preprocessed, to
`
`automatically find and "lift" the quadrangle 502. In other words, the document that forms
`
`quadrangle 502 might be separated from the rest of the image so that the document alone might
`
`be processed. By separating quadrangle 502 from any background in an image, it may then be
`
`fi1rther processed.
`
`[0073]
`
`The quadrangle 502 might be mapped onto a rectangular bitmap in order to
`
`remove or decrease the perspective distortion. Additionally, image sharpening might be used to
`
`improve the out—of—focus score of the image. The resolution of the image may then be increased
`
`and the image converted to a black—and—white image. In some cases, a black—and—white image
`
`might have a higher recognition rate when processed using an automated document processing
`
`system in accordance with the systems and methods described herein.
`
`[0074]
`
`An image that is bi—tonal, e.g., black—and—white, might be used in some systems.
`
`Such systems might require an image that is at least 200 dots per inch resolution. Accordingly, a
`
`color image taken using a mobile device might need to be high enough quality so that the image
`
`may successfully be converted from, for example, a 24 bit per pixel (24 bit/pixel) RGB image to
`
`a bi—tonal image. The image may be sized as if the document, e.g., check, payment coupon, etc.,
`
`was scanned at 200 dots per inch.
`
`[0075]
`
`Figure 6 is a diagram illustrating an example original image, focus rectangle and
`
`document quadrangle ABCD in accordance with the example of Figure 5. In some embodiments
`
`it may be necessary to place a document for processing at or near the center of an input image
`
`close to the camera. All points A, B, C and D are located in the image, and the focus rectangle
`
`602 is located inside quadrangle ABCD 502. The document might also have a low out—of—focus
`
`score and the background surrounding the document might be selected to be darker than the
`
`document. In this way, the lighter document will stand out from the darker background.
`
`[0076]
`
`Figure 7 is a flowchart illustrating an example method 700 in accordance with the
`
`systems and methods described herein. Referring now to Figure 7, in operation 701 a user logs
`
`into a document capture system on a mobile communication device. In accordance with various
`
`embodiments, methods and systems for document capture on a mobile communication device
`
`W02-WEST:6AXS1\4011881917
`
`-14-
`
`Page 14 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`Page 14 of 62
`
`ROTHSCHILD EXHIBIT 1013
`
`

`
`may further comprise requiring the user to log into an application. In this way, access to the
`
`document capture system using a mobile communication device might be limited to authorized
`
`USCIS.
`
`[0077]
`
`In operation 702, in the illustrated embodiment, the type of document is selected.
`
`For example, a user might select a document type for a check, payment coupon or deposit slip.
`
`By entering the type of document, a mobile device might be able to scan specific parts of an
`
`image to determine, for example, payee, check amount, signature, etc. In some embodiments,
`
`however, a device might determine what type of document is being imaged by processing the
`
`image.
`
`[0078]
`
`In operation 704, an im