(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`(19) World Intellectual Property
`Organization
`International Bureau
`
`
`
`(43) International Publication Date
`29 December 2005 (29.12.2005)
`
`(10) International Publication Number
`
`WO 2005/124657 A1
`
`(51) International Patent Classification7:
`
`G06K 7/015
`
`(21) International Application Number:
`PCT/FI2005/000291
`
`(22) International Filing Date:
`
`17 June 2005 (17.06.2005)
`
`(25) Filing Language:
`
`(26) Publication Language:
`
`(30) Priority Data:
`20040858
`
`Finnish
`
`English
`
`18 June 2004 (18.06.2004)
`
`Fl
`
`(71) Applicant (for all designated States except US): VAL-
`TION TEKNILLINEN TUTKIMUSKESKUS [Fl/FT];
`Vuorimiehentie 5, FI-02150 Espoo (Fl).
`
`(72) Inventors; and
`(75) Inventors/Applicants (for US only): BACKSTROM,
`Christer [FT/PI]; Pihatorma 6 D 18, ”—02240 Espoo (FT).
`SODERGARD, Caj [PI/FT]; Nakinkaari 5 C 1, 131—02320
`Espoo (F1).
`
`(74) Agent: SEPPO LAINE OY; Itainerenkatu 3 B, FI—00180
`Helsinki (FD.
`
`(81) Designated States (unless otherwise indicated, for every
`kind ofnational protection available): AE, AG, AL, AM,
`AT, AU, AZ, BA, BB, BG, BR, BW, BY, BZ, CA, CH, CN,
`CO, CR, CU, CZ, DE, DK, DM, DZ, EC, EE, EG, ES, FI,
`GB, GD, GE, GH, GM, HR, HIT, ID, IL, IN, IS, JP, KE,
`KG, KM, KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MA,
`MD, MG, MK, MN, MW, MX, MZ, NA, NG, N1, NO, NZ,
`OM, PG, PH, PL, PT, RO, RU, SC, SD, SE, SG, SK, SL,
`SM, SY, TJ, TM, TN, TR, T1", TZ, UA, UG, US, UZ, VC,
`VN, YU, ZA, ZM, ZW,
`
`(84) Designated States (unless otherwise indicated. for every
`kind of regional protection available): ARIPO (BW, GH,
`GM, KE, LS, MW, MZ, NA, SD, SL, SZ, TZ, UG, ZM,
`ZW), Eurasian (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM),
`European (AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI,
`FR, GB, GR, IIU, IE, IS, IT, LT, LU, MC, NL, PL, PT, RO,
`SE, SI, SK, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN,
`GQ, GW, ML, MR, NE, SN, TD, TG).
`
`[Continued on next page]
`
`(54) Title: METHOD FOR DETECTING A CODE WITH THE AID OF A MOBILE STATION
`
`Activate sofiware
`
`
`
`Low Res
`
`
`mode
`
`bar code
`
`in view ? 2
`enter coord
`
`
`
`
`
`
`
`High Res image capture
`
`Edge detection
`3
`Estimation of comer points
`Perspective transformation
`Decoding ofduta
`Salomon Reed error correction
`
`
`
`05/124657A1||||l|||l||||l||llllll|||||||||||||||||l|||||||||||||||||l|||||||||||||I|||||||||||||||||||||||
`
`c (57) Abstract: The invention relates to a method for detecting a code (40), and a mobile station adapted to the method. In the method,
`N the mobile station (50) is used to capture an image of the subject (40) and the image is ana1ysed and decoded into a numerica] form
`in the mobile station (50). According to the invention, before the decoding of the bar code in the mobile station the edges of the
`subject (40) are determined from a high—resolution image, in the case of a 2D code, the location of the corner point of the subject
`(49) are estimated on the basis of adaptive thresholding, and, in the case of a 2D code, a perspective transformation is formed.
`
`Mitek Exhibit 1011, Page 001
`Mitek Exhibit 1011, Page 001
`
`

`

`WO 2005/124657 A1
`
`||||||||||l|||||||l||l|||||||l|||||||||||Illl||||||||||l||||||||||l||||||||||||||||||I||l||||||
`
`Published:
`— with international search report
`— before the expiration of the time limit for amending the
`claims and to be republished in the event of receipt of
`amendments
`
`For two-letter codes and other abbreviations, refer to the ”Guid—
`anee Notes on Codes andAbbreviations ” appearing at the begin
`ning ofeaeh regular issue ofthe PCT Gazette.
`
`Mitek Exhibit 1011, Page 002
`Mitek Exhibit 1011, Page 002
`
`

`

`W0 2005/124657
`
`PCT/FIZOOS/(Nlfl29l
`
`Method for detecting a code with the aid of a mobile station
`
`1
`
`The present invention relates to a method, according to the preamble of Claim 1, for
`
`detecting a code with the aid of a mobile station.
`
`The invention also relates to a mobile station and a computer program product.
`
`According to the prior art, camera telephones are known, which can be used to detect bar
`
`codes from various products and use this bar code to connect the mobile station to
`
`product information on the product, with the aid of telecommunications connections.
`
`‘
`
`A drawback of the prior art is that the detection event is time consuming and the end
`
`result often uncertain. The user may have to try several times before reading succeeds,
`
`which does not motivate the user. The uncertainty of the reading increases, if the code
`
`sign is not read in precisely the correct direction.
`
`Most known technologies demand the user to actively capture an image of the code by
`
`pressing a key. This clearly makes the service more labourious to use.
`
`In devices of the prior art, the user is given insufficient indication of the success of the
`
`reading. Thus the reading event remains vague and slow, in which case the power
`
`consumption of the reading device also increases during the reading event.
`
`The invention is intended to eliminate the defects of the prior art described above and for
`
`this purpose create an entirely new type of method for detecting a code and a mobile
`
`station for this purpose.
`
`The invention is based on automatic, real~time image analysis, which decides when the
`
`camera is sufficiently well positioned to successfully interpret the code. By means of this
`
`method, a bar code can even be read successfully in difficult conditions, for instance,
`
`from a slanting direction, under poor lighting, and on shiny objects. In addition, the
`
`interface of the invention is implemented in such a way that it tells the user of the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`Mitek Exhibit 1011, Page 003
`Mitek Exhibit 1011, Page 003
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`2
`
`current stage of the imaging situation and instructs them how to proceed. Based on the
`
`instruction, the user can very easily set the camera telephone at the correct distance from
`
`the imaging subject. In addition, in one preferred embodiment of the invention, the >
`
`software controls the lens arrangement of a camera telephone equipped with zoom
`
`optics, in such a way that the optimal magnification ratio is found automatically. Thus
`
`the usability is further improved, because the user does not have to search for the correct
`
`imaging distance - it is enough for the code to be visible to the camera.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`More specifically, the method according to the invention is characterized by what is
`
`stated in the characterizing portions of Claims 1 and 8.
`
`The mobile station according to the invention is, in turn, characterized by what is stated
`
`in the characterizing portions of Claims 15 and 22.
`
`Considerable advantages are gained with the aid of the invention.
`
`The method according to the invention seeks for the optimal imaging distance, so that
`
`the imaging situation is not as demanding as in solutions according to the prior art. An
`
`acceptable image is obtained even if the code to be read is tilted relative to the camera.
`
`In addition, neither poor lighting nor shiny codes will prevent successful interpretation
`
`because, in the case of a 1D code, the algorithm retrieves the best out of hundreds of
`
`horizontal image lines. In addition, the imaging situation is facilitated, as the user is
`given real-time instructions for finding a suitable imaging distance and is informed of
`
`the success of the imaging situation.
`
`In the following, the invention is examined with the aid of examples and With reference
`
`to the accompanying drawings.
`
`Figure 1 shows a flow diagram of the method according to the invention.
`
`Figure 2 shows a perspective view of a problem situation according to the invention.
`
`Mitek Exhibit 1011, Page 004
`Mitek Exhibit 1011, Page 004
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`Figure 3 shows a perspective View of a second problem situation according to the
`
`invention.
`
`3
`
`Figure 4 shows the interface according to the invention
`
`Figures 5 — 8 show photographs of the interface according to the invention.
`
`10
`
`15
`
`20
`
`25
`
`The invention exploits the cameras, which are typically according to the VGA standard,
`
`in present—day mobile stations. One application according to the invention is
`
`implemented using the Symbian Series 60 software platform. Using this application
`
`according to the invention, both one-dimensional bar codes and two-dimensional matrix
`
`codes can be read.
`
`One purpose of the invention is to provide a method and means for data search that takes
`
`place with the aid of product codes. In this case, the code is read using a mobile station.
`
`The numerical form of the code connects the mobile station over a communications
`
`network to a databank, from which the background data relating to the read code can be .
`
`found. The mobile station can contain a preset user profile, so that always only
`
`information that interests the user will be retrieved according to the user’s profile. The
`user profile can target, for example, some property of feedstuffs, such as calorie content,
`
`salt content, or some other health~related factor. The information retrieved over the
`
`telecommunications network is displayed to the user by means of the display of the
`
`mobile station.
`
`The functionality according to the invention can be implemented, for example, in
`
`existing cameras, purely with the aid of software. In some cases, the optics may need
`
`changes, to allow close-up reading to succeed with sufficient accuracy. The change in
`
`the optics can be, for example, an extra lens in front of the normal optics. The new-
`
`generation variableLfocal—length lenses can be applied as such to the solution of the
`
`30
`
`invention.
`
`In the manner of implementation according to the invention, detection takes place in the
`
`Mitek Exhibit 1011, Page 005
`Mitek Exhibit 1011, Page 005
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`,
`
`4
`
`actual mobile station. In solutions according to the prior art, implementations are also
`
`known, in which the code is sent as such for analysis, i.e. as image data, over a
`
`telecommunications network. This procedure strains telecommunications networks and,
`
`at existing data—transfer speeds, is a form of operation that is slow and quite difficult to
`use.
`
`The solution according to the invention is fully automatic once the imaging subject has
`
`been found.
`
`10
`
`The operational module of the camera determines when the code, for example a bar
`code, is within’the camera’s field of vision. If the camera has zoom optics, the optics are
`
`guided to the optimal magnification ratio. After this, the camera captures a full-size
`
`colour-VGA image, analyses it and decodes the contents of the code, for example a bar
`
`code. The contents can be a network address, product number, or, for example, a service
`
`15
`
`request.
`
`20
`
`25
`
`30
`
`According to Figure 1, the module according to the invention is activated in stage 1, for
`
`example, by selecting a suitable functional icon on the display of the mobile station.
`
`After activation 1, the mobile station begins to analyse the image continuously in a
`
`lower-resolution (SIF) operating state and to search for code signs, such as a bar code in
`
`block 2. If controllable zoom—optics are found, a suitable magnification ratio is
`
`automatically sought. Once the identifying signs of a bar code are found and the
`
`conditions for successful imaging are realized, the module changes the camera to a
`
`higher resolution state (VGA), captures an image, and analyses the contents of the code
`according to block 3. In block 3, detection of the edge of the code, estimation of the
`
`location of the corner points, perspective transformation, code-data decoding, and
`
`Salomon Reed error correction are carried out.
`
`Once an image of the code 40, such as a bar code, has been captured according to Figure
`
`2, it is nearly impossible to position the camera in such a way that the camera’s image
`
`place would be parallel to the plane 20 of the code 40. For the planes to be parallel, the
`
`optical axis of the camera would have to be aligned precisely at right angles to the plane
`
`Mitek Exhibit 1011, Page 006
`Mitek Exhibit 1011, Page 006
`
`

`

`W0 2005/124657
`
`PCT/F12005/000291
`
`_ 20 of the code. For a normal user this almost an impossible task. Figure 2 shows this
`
`_
`
`problem field and depicts the corrected image plane 10 relative to the code plane 20.
`
`5
`
`4 In the case of a 1D code, the descriptors of the bar code are calculated in real time, along
`
`horizontal segments of the image. Once the descriptors exceed a threshold, a change is
`
`made to a higher resolution form and the high—resolution image is examined. The error
`
`check in the 1D code ensures the correct result of the interpretation.
`
`The defining of the imaging of each point of the corrected image plane 10, relative to the
`
`10
`
`plane 20 of the code, such as a bar code 40, requires the solution of the following
`
`homogeneous system of equations:
`
`XW
`
`abc x
`
`YW = def
`
`W
`
`ghl
`
`y
`
`l
`
`(1)
`
`‘15
`
`X and Y represent the co-ordinates of the image plane 10 while x‘and y for their part
`
`represent the co-ordinates of the plane 20 or the code 40. In order to determine the
`
`unknown coefficients, the system of equations is rearranged, so that the following group
`
`(2)
`
`of equations is obtained:
`
`" xyiooo—xXL yX '1 X
`
`Y
`
`X
`
`Y
`
`X Y X Y
`
`wwxmmnwa
`
`OOOxyl —- xY — yY
`
`xleOO — xX — yX
`
`OOOxyl — xY -— yY
`
`xleOO — xX — yX
`‘ OOOerl — xY —- yY
`
`xleOO — xX - yX
`
`OOOxyl — xY — yY
`
`20
`
`This group of equations solved using four pairs of points from the image plane 10 and
`
`correspondingly from the code level 20, for example, the corner points of a two-
`
`dimensional code (trapezium).
`
`Mitek Exhibit 1011, Page 007
`Mitek Exhibit 1011, Page 007
`
`

`

`WO 2005/124657
`
`PCT/FI2005/000291
`
`Because the aim is to obtain the bar code as a square on the image plane 10 from the
`
`trapezium of the bar—code level 20, according to the invention the following formula is
`
`used to implement the imaging:
`
`X=((C-X)(Tn-e)-(f-Y)OGI-b))/((Yh-e)(Xg-a)-(Xh-b)(Yg-d)
`
`Y=((C-X)(Yg-d)-(f-Y)(Xg-a))/((Yg-d)(Xh-b)-(Xg-a)(Yh-e))
`
`Conversion of the perspective of the image is essential in the decoding of the code, for
`
`example, a bar code. This is shown schematically in Figure 3, in which the code 40
`
`according to the trapezium is converted to a rectangular set of co—ordinates in the right-
`
`hand picture. The perspective conversion can be used in several ways, all of which are
`
`important in the decoding of a two—dimensional data-matrix code. These methods are:
`
`10
`
`15
`
`1. Scaling. The accuracy of the image analysis will improve significantly, if the
`
`size of the data elements of the bar code is several image pixels.
`
`2. Rotation. The image is rotated to be in accordance with a rectangular set of co-
`
`20
`
`ordinates.
`
`3; Perspectivetransformation; This permits correction of a perspective error that
`
`would otherwise seriously impede decoding of the code, or even make it
`
`impossible.
`
`For a successful conversion, it is important to estimate precisely the number of comer
`
`points. This is implemented by expressing the edge. This expression method utilizes a
`
`second derivative, the exceeding of zero.
`
`A method, in which there is no positioning error (in the case of step edges), is used in the
`
`case of a two—dimensional matrix sign, to increase accuracy. In the case of a one-
`
`dimensional bar code, it is sufficient to use a simple derivation procedure.
`
`25
`
`30
`
`Mitek Exhibit 1011, Page 008
`Mitek Exhibit 1011, Page 008
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`According to Figure 4, one interface of the invention’s preferred mobile station 50
`
`includes a display 55, on which there is an image 40 of the imaging subject and, in
`
`addition to this, an additional indicator 35, which can, in the case of a 1D code, be a
`
`horizontal line 35 that changes colour according to the image. A red colour of the line 35
`
`indicates that the imaging event is in the preparation stage and the code has not yet been
`
`found. A change of colour in the line 35, for example, to yellow indicates that the code
`
`has been found and the analysis stage may have started. The change of the code to
`
`completely green indicates the successful interpretation. In addition to colour, a sound
`
`signal notifies of the successful interpretation of the code. The colour variations can be
`
`continuous, i.e. bright red « light red - light yellow - bright yellow — light green - bright
`
`green. An additional indicator 35 can naturally be, for example, a graphical Sign, or a
`
`change in the colour of the entire display, or even a change in the display from black—
`
`and-white to coloured or vice versa. The most important aspect is to use a sign 35 to
`
`indicate to the user the stage of the process, in other words the state in which the user
`
`should keep the mobile station as completely stationary as possible.
`
`According to one alternative embodiment of the invention, the additional indicator 35 is
`
`a coloured bar,'which changes form red to yellow and green, depending on the quality of
`
`the image. Altematively, there can also be a sign indicating the direction of the zooming,
`
`for example ‘Zoom in’ is a change from red to green. In the ‘Zoom out’ situation, the
`
`. colounchange takes place in the opposite direction.
`
`In a 2D case (matrix sign), a red box surround the centre—point of the sign 40, telling
`
`how large the sign should be. The colour coding is the same as in the 1D case.
`
`Once the bar code has been interpreted, the system returns automatically to the bar-code
`
`search mode, after a specific delay.
`
`When new camera-phone generations appear on the market, the use of low resolution in
`
`the code-search stage will not be absolutely necessary, due to their increased calculation
`
`capacity. On the other hand, the number of pixels of camera phones will also increase,
`
`10
`
`15
`
`20
`
`25
`
`30
`
`Mitek Exhibit 1011, Page 009
`Mitek Exhibit 1011, Page 009
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`which will increase the calculation requirement.
`
`8
`
`According to one preferred embodiment of the invention, polarizing filtering can be used
`
`in the camera’s lens system, to reduce reflections.
`
`The invention is typically implemented using a computer program stored in a data means
`
`to be run using a computer (the telephone’s processor).
`
`According to one preferred embodiment of the invention, a calculated quality indicator
`
`is defined in real time from each camera image, and depicts the probability that the bar
`
`code is in the image field and can be interpreted correctly, the quality indicator being
`
`calculated in the case of a 1D bar code from the variance of the grey tones of the
`
`horizontal lines of the image. In the case of a 2D bar code, the quality indicator is
`
`calculated form the variance of the grey tones of the image, the value of the quality
`
`indicator being shown to the user using coloured icons on the display of the phone, or by
`
`graphical signs, for example, in such a way that a lower probability is shown by read and
`
`a higher probability by yellow and green, and a high-resolution image is captured once
`the quality indicator of the bar code exceeds a set threshold value and the defined Line ‘
`
`of Interest is interpreted in the low—resolution stage.
`
`Further, one preferred embodiment of the invention includes means for indicating to the
`
`user, for example, by a sound signal, when the code Sign 40~has been detected.
`
`According to Figures 5 — 8, the display’s indicator changes from the red of Figure 5
`
`through the orange of Figure 6 and the green of Figure 8, to the imaging-ready display of
`
`Figure 8.
`
`10
`
`15
`
`20
`
`25
`
`Mitek Exhibit 1011, Page 010
`Mitek Exhibit 1011, Page 010
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`Claims:
`
`1. A method for detecting a code (40), in which method a mobile station (50) is used to
`
`capture an image of the subject (40) and the image is analysed and decoded into a
`
`numerical form in the mobile station (50),
`
`characterized in that, before the decoding of the bar code in the mobile station
`
`- a calculated quality indicator is defined in real time from each camera image
`
`from the Variance of the grey tones, which depicts the probability that the bar code is in
`
`the image field and can be interpreted correctly,
`
`- on the basis of the quality indicator, a decision is made as to whether the bar
`
`code is ID or 2D,
`
`- in the case of a 1D bar code, the quality indicator is calculated from the Line of
`
`Interest, in which the differences between the grey tones along the line are greatest,
`
`- in the case of a 2D bar code, the quality indicator is calculated on the basis of
`
`the difference histogram of the grey tones of the image above the assumed bar code,
`
`‘-‘the value-of the-quality indicator is shown to the user on the display of the
`
`telephone, by using graphical signs, for example, in such a way that poorer visibility of
`
`the code is shown with red and higher Visibility with yellow and green, and
`
`- a high-resolution image is captured once the quality indicator of the bar code
`
`exceeds a specified threshold level and the Line of Interest defined in the low-resolution
`
`stage is interpreted.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`2. A method according to Claim 1, characterized in that the subject is repeatedly
`
`imaged in low-resolution mode, using the mobile station (40), prior to the capture of the
`
`image to be analysed.
`
`Mitek Exhibit 1011, Page 011
`Mitek Exhibit 1011, Page 011
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`10
`
`3. A method according to any of the above Claims, characterized in that the interface is
`
`implemented in such a way that it instructs the user as to the stage in which the imaging
`
`situation is and how the user should proceed, so that on the basis of the instruction the
`
`user can easily set the camera telephone at the correct distance from the imaging subject.
`
`4. A method according to Claim 1 or 2, characterized in that, in the case [of a 2D code,
`
`the image is rotated according to a rectangular set of co-ordinates before it is analysed.
`
`5. A method according to any of the above Claims, characterized in that an indication
`
`that a code sign (40) has been detected is given to the user, for example, with the aid of
`
`an icon and sound.
`
`6. A method according to any of the above Claims, characterized in that the optical
`
`signal coming to the camera is filtered using a polarization filter.
`
`7. A method according to any of the above Claims, characterized in that the software is
`
`arranged to control the lens arrangement of a camera telephone equipped with zoom
`
`optics, in such a way that the optimal magnification ratio is found automatically, thus
`
`further improving usability, because the user need not seek the correct imaging distance
`
`and it is sufficient for the code to be within the area of visibility of the camera.
`
`10
`
`15
`
`20
`
`8. A method for detecting a code (40), in which method a mobile station (50) is used to
`
`capture an image of the subject (40) and the image is analysed and decoded into a
`
`25
`
`numerical form in the mobile station (50),
`
`characterized in that before the decoding of the bar code in the mobile station
`
`— the edges of the subject (40) are determined from a high—resolution image,
`
`30
`
`- in the case of a 2D code, the location of the comer point of the subject (40) are
`
`estimated on the basis of adaptive thresholding, and
`
`Mitek Exhibit 1011, Page 012
`Mitek Exhibit 1011, Page 012
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`11
`
`~ a perspective transformation is formed in the case of a 2D code.
`
`9. A method according to Claim 9, characterized in that using the mobile station (50)
`
`the subject is repeatedly imaged in low~resolution mode prior to capturing the image to
`
`be analysed.
`
`10. A method according to any of Claims 8 — 9, characterized in that the interface is
`
`implemented in such a way that it instructs the user as to the stage in which the imaging
`
`10
`
`situation is and how the user should proceed, so that on the basis of the instruction it is
`
`very easy for the user to set the camera telephone at the correct distance from the
`
`imaging subject.
`
`11. A method according to any of Claims 8 — 10, characterized in that, in the case of a
`
`2D code, the image is rotated according to a rectangular set of co—ordinates before it is
`
`analysed.
`
`12. A method according to any of Claims 8 — 11, characterized in that an indication that
`
`a code sign (40) has been detected is given to the user, for example, with the aid of an
`
`20
`
`icon and sound.
`
`13. A'method according to any of Claims 8 — 12, characterized in‘that the optical signal
`
`coming to the camera is filtered using a polarization filter.
`
`14. A method according to any of Claims 8 - 13, characterized in that the software is
`
`arranged to control the lens arrangement of a camera telephone equipped with zoom
`
`optics, in such a way that the optimal magnification ratio is found automatically, thus
`
`further improving usability, because the user need not seek the correct imaging distance
`
`and it is sufficient for the code to be within the area of visibility of the camera.
`
`25
`
`30
`
`15. A mobile station for detecting a code (40), which mobile station includes means for
`
`capturing an image of the subject (40) and means for analysing the subject (40) decoding
`
`Mitek Exhibit 1011, Page 013
`Mitek Exhibit 1011, Page 013
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`it into a numerical form,
`
`12
`
`characterized in that the mobile station (50) includes means
`
`- for defining a quality indicator in real time from each camera image from the
`
`variance of the grey tones, which indicator depicts the probability that the bar code is in
`
`the image field and can be interpreted correctly,
`
`— in the case of a 1D bar code, for calculating a quality indicator from the
`
`10
`
`variance of the grey tones of the horizontal lines of the image,
`
`— in the case of a 2D bar code, for calculating a quality indicator from the
`
`variance of the grey tones of the image,
`
`15
`
`- for showing the value of the quality indicator to the user on the display of the
`
`telephone, by using graphical signs, for example, in such a way that poorer visibility of
`
`the code is shown with red and higher visibility with yellow and green, and
`
`- for capturing a high—resolution image once the quality indicator of the bar code
`
`20
`
`exceeds a specified threshold level.
`
`16. A mobile station according to Claim 15, characterized it includes means for
`
`imaging the subject repeatedly in low-resolution mode, prior to the capture of the image
`
`to be analysed.
`
`25
`
`17. A mobile station according to any of the above Claims, characterized in that it
`
`includes means for implementing the interface in such a way that it instructs the user as
`
`to the stage in which the imaging situation is and how the user should proceed, so that on
`
`the basis of the instruction the user can easily set the camera telephone at the correct
`
`30
`
`distance from the imaging subject.
`
`18. A mobile station according to any of the above Claims, characterized in that, in the
`
`Mitek Exhibit 1011, Page 014
`Mitek Exhibit 1011, Page 014
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`case of a 2D code, it includes means for rotating the image according to a rectangular set
`
`of co-ordinates before it is analysed.
`
`13
`
`19. A mobile station according to any of the above Claims, characterized in that it
`
`includes means for indicating to the user, for example with a sound signal, once a code
`
`sign (40) has been detected.
`
`20. A mobile station according to any of the above Claims, characterized in that it
`
`includes a polarizing filter for filtering the optical signal coming to the camera, in order
`
`10‘
`
`to avoid reflections.
`
`15
`
`20
`
`25
`
`30
`
`21. A mobile station according to any of the above Claims, characterized in that the
`
`software is arranged to control the lens arrangement of a camera telephone equipped
`
`with zoom optics, in such a way that the optimal magnification ratio is found
`
`automatically, thus further improving usability, because the user need not seek the
`
`correct imaging distance and it is sufficient for the code to be within the area of visibility
`
`of the camera.
`
`22. A mobile station for detecting a code (40), which mobile station includes means for
`
`capturing an image of the subject (40) and means for analysing the subject (40) and
`
`decoding it into a numerical form,
`
`characterized in that the mobile station (50) includes means
`
`— for determining the edges of the subject (40) from a high-resolution image,
`
`- in the case of a 2D code, for estimating the location of the corner-point of the
`
`subject (40) on the basis of adaptive thresholding, and
`
`— for forming a perspective transformation in the case of a 2D code.
`
`23. A computer program product, to be stored on a memory medium and to be run using
`
`Mitek Exhibit 1011, Page 015
`Mitek Exhibit 1011, Page 015
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`a computer, for implementing the mobile station, or method according to any of the
`
`above Claims.
`
`14
`
`Mitek Exhibit 1011, Page 016
`Mitek Exhibit 1011, Page 016
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`1/3
`
`Activate sofiware
`
`
`
`Low Res
`
`
`mode
`
`bar code
`
`in view ? 2
`
`
`enter coord
`
`
`
`
`
`High Res image capture
`Edge detection
`Estimation of comer points
`
`Perspective transformation
`
`Decoding of data
`
`Salomon Reed error correction
`
`
`
`Mitek Exhibit 1011, Page 017
`Mitek Exhibit 1011, Page 017
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`2/3
`
`
`
`Mitek Exhibit 1011, Page 018
`Mitek Exhibit 1011, Page 018
`
`

`

`WO 2005/124657
`
`PCT/F12005/000291
`
`3/3
`
`Fig. 5
`
`
`Mitek Exhibit 1011, Page 019
`Mitek Exhibit 1011, Page 019
`
`

`

`INTERNATIONAL SEARCH REPORT
`
`Intel-flagella application No.
`
`.PCT/FI 2005/000291
`
`A. CLASSIFICATION OF SUBJECT MATTER
`
`IPC7: 606K 7/015
`According to International Patent Classification (IPC) or to both national classification and [PC
`B. FIELDS SEARCHED
`
`Minimum documentation searched (classification system followed by classification symbols)
`
`
`IPC7: 606K, 606T, H04N
`Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched
`
`SE,DK,FI,N0 classes as above
`
`Electronic data base consulted during the international search (name of data base and, where practicable, se arch terms used)
`
`INSPEC COMPENDEX TDB
`PAJ
`EPO- INTERNAL WPI DATA
`C. DOCUMENTS CONSIDERED TO BE RELEVANT
`
`Citation of document, with indication, where appropriate, of the relevant passages
`
`Relevant to claim No.
`
`us 20020102966 A1 (TSVI H. LEV ET AL),
`1 August 2002 (01. 08. 2002), abstract,
`[01061—[0113]
`
`[0037],
`
`1—7, 9-21, 23
`
`US 20030218069 A1 (TIMOTHY P. MEIER ET AL),
`27 November 2003 (27.11.2003), abstract,
`[0042],[0046]
`
`(DAVID J. NELSON ET AL),
`US 6688525 Bl
`10 February 2004 (10.02.2004), column 2,
`line 18 - line 27; column 2,
`line 53 - line 64,
`figure 2, abstract
`
`4,8—14,22
`
`I] See patent family annex.
`'3 Further documents are listed in the continuation of Box C.
`*
`Special categories of cited documents:
`"T"
`later document published after the international filing date or priority
`date and not in conflict with the application but cited to understand
`"A” document defining the general state of the art which is not considered
`the principle or theory underlying the invention
`to be of particular relevance
`document of particular relevance: the claimed invention cannot be
`earlier application or patent but published on or after the international
`filing date
`document which may throw doubts on priority claim(s) or which'Is
`cited to establish the publication date of another citation or other
`special reason (as specified)
`document referring to an oral disclosure, use, exhibition or other
`means
`
`"E"
`”L”
`
` considered novel or cannot be considered to involve an inventive
`
`IIXIA
`
`"Y”
`
`step when the document is taken alone
`document of particular relevance: the claimed invention cannot be
`considered to involve an inventive step when the document is
`combined with one or more other such documents, such combination
`being obvious to aperson skilled in the art
`”6;” document member of the same patent family
`
`document published pIior to the international filing date but later than
`the priority date claimed
`
`Date of the actual completion of the international search
`
`Date of mailing of the international search report
`
`2 November 2005
`
`Name and mailing address of the ISA/
`Swedish Patent Office
`Box 5055, 8—102 42 STOCKHOLM
`Facsimile No. +46 8 666 02 86
`Form PCT/ISA/ZIO (second sheet) (April 2005)
`
`8 9 ~11» 2005
`Authorized officer
`
`Alexander Lakic/MN
`Telephone N0.
`+ 46 8 782 25 00
`
`Mitek Exhibit 1011, Page 020
`Mitek Exhibit 1011, Page 020
`
`

`

`INTERNATIONAL SEARCH REPORT
`
`
`
`
`International application No.
`
`
`
`PCT/FI 2005/000291
`
`
`
`C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT
`
`Citation of document, with indication, where appropriate, of the relevant passages
`
`Relevant to claim No.
`
`US 20030116628 A1 (TSUTDMU, NAKAZANA ET AL),
`26 June 2003 (26.06.2003), figure 2,
`abstract, [0011]-[0014],[003