1797_I
` UNITED STATES DEPARTMENT OF COMMERCE
`
`
`
`II" IIII'III:IIII..I' I‘||I_|': III IIIII I[.-III_IIIIIIIIIIILILIJIIILI_!IIIII_IIIIIIIIIIIIJII'I.‘ '.I|II||.II.III'.IIIII|'I|l:|III_-.IIIIII
`
`United States Patent and Trademark Office
`
`THIS IS TO CERTIFY THAT ANNEXED IS A TRUE COPY FROM THE
`
`
`
`November 01, 2018
`
`
`RECORDS OF THIS OFFICE OF THE FILE WRAPPER AND CONTENTS
`
`OF:
`
` APPLICATION NUMBER: 09/736,825
`
`
`
`FILING DATE: December 14, 2000
`
`PATENT NUMBER: 6,654,507
`
`ISSUE DATE: November 25, 2003
`
`
`
`
`Under Secretary of Commerce for Intellectual Property
`
`
`and Director of the United States Patent and Trademark Office '-
`
`
`
`
`
`L1." \..
`
`PART (,2) OF (.2) PART(S)
`\.“NM
`
`
`
`
`
`
`
`
`
`
`
`
`
`.n'iss-....Iu...wuuI
`I‘I-kuvyv-v-vauu‘.u1nv-.uw-n...-.-.
`._.._._.—_____v.‘..__w..__.
`
`nweuvunnw-I
`__-:!_gnu-“uh...“
`
`
`By Authority of the
`
`M. TARVER
`
`Certifying Officer
`
`--¢
`MmLI‘J’H—Ii:I|_|_IIl:i|-_I1.II_I_I|_IDLIIII‘IHI_I|I_III_III_IIl|IsIII]r.I_-IIII _I:I:II.IIIIIIu_sII
`
`
`
`Page 1 of 309
`
`BOSCH EXHIBIT 1002 Part 2 of 2
`
`Page 1 of 309
`
`BOSCH EXHIBIT 1002 Part 2 of 2
`
`

`

`US 6,335,985 B1
`
`51
`basis of a prediction error amount such that the extraction
`result obtained by said second object extraction section is
`used as an object region when the prediction error caused by
`said second object extraction section falls within a prede-
`termined range, and the extraction result obtained by said
`first object extraction section is used as an object region
`when the prediction error exceeds the predetermined range.
`15.
`'lhe apparatus according to claim 11, wherein said
`second object extraction section performs inter-flame pre-
`diction in a sequence different flom an input flame sequence
`such that a flame interval between the reference frame and
`the current flame as the object extraction target is set to not
`less than a predetermined number of flames.
`16. An object extraction apparatus comprising:
`a picture input section which inpu1s moving picture data
`and shape data representing an object region on a
`predetermined flame of a plurality of flames corre-
`sponding to the moving picture data;
`a segmentation section which segments a current frame
`into a plurality of blocks;
`a search section which searches a reference flame for a
`similar block, for each of the blocks, which is similar
`in figtrre represented by picttn'e data to a current block
`and is larger in area than the current block, the refer—
`ence flame being temporally difierent flom the current
`flame;
`a paste section which pastes shape data obtained by
`extracting and reducing shape data of each similar
`block from the reference flame on each block of the
`current flame; and
`an output section which outputs the pasted shaped data as
`shape data of the current flame.
`17. An object extraction apparatus comprising:
`a picture input section which inputs picture data repre-
`senting a picture including at least one object and a
`background surrounding the object and shape data
`representing the object;
`a setting section which sets blocks on a contour portion of
`the object, and searches for a similar block, for each of
`the blocks, which is similar in graphic figure repre-
`sented by the picture data to each block and is larger
`than the block, flom the same picture to obtain a
`plurality of similar blocks;
`a replacement section which replaces the shape data of
`each of the blocks with reduced shape data obtained by
`reducing the shape data of each of the similar blocks;
`a repeat section which repeats the replacement by a
`predetermined number of times; and
`an output section which outputs shape data obtained by
`repeating the replacement as corrected shape data.
`18. The apparatus according to claim 17, further com—
`prising a second repeat section which repeats the searching
`for the similar block and the predetermined number of
`replacements of the shape data by a plurality of times while
`decreasing a block size every repetition.
`19.A method for extracting a moving object flom an input
`moving picture, comprising the steps of:
`determining a first background region common to a
`current flame containing a target object to be extracted
`flom a moving picture signal and a first reference frame
`that temporally diflers from the current flame on the
`basis of a difference between the current flame and the
`first reference frame;
`determining a second background region common to the
`current flame and a second reference frame that tem—
`
`5
`
`ID
`
`15
`
`20
`
`3D
`
`35
`
`45
`
`St]
`
`55
`
`60
`
`65
`
`52
`porally dill'ers flom the current flame on the basis of a
`diflemnce between the current frame and the second
`reference frame; and
`extracting a region, in a picture on the current flame,
`which belong to neither the first background region
`not the sewnd background region as an object region,
`the first background region and the second background
`region indicating a background in the input moving
`picture.
`20. The method according to claim 19, which comprises
`a step of determining pixels of the current flame as the
`object region when pixels of one of the first and second
`reference flames belongs to the object region, and deter-
`mining the pixels of the current flame as the background
`region when the pixels of one of the first and second
`reference flames belongs to the background region, using a
`predetermined shape of the object of one of the first and
`second reference flames in a case of that the difi'erence
`between the pixels of the current flame and the pixels the
`one of the first and second reference flames is small.
`21. The method according to claim 20, wherein the step
`of determining the pixels of the current flame uses the
`predetermined shape of the object, when the shape of the
`object of one of the first and second reference flames has
`already been extracted, and a shape of the block of the one
`of the first and second reference flames which is created
`flom the flame, floor which the shape of the object has been
`extracted, by a block matching method, when the object
`region is not extracted.
`22. The method according to claim 19, further comprising
`a step of correcting motion of a background on one of the
`first and second reference flames or the current flame such
`that the motion of the background between each of the first
`and second reference flames and the current flame becomes
`relatively zero.
`23. The method according to claim 19, wherein the
`background region determining step includes determining
`the common background region using a predetermined
`threshold value.
`24. The method according to claim 23, wherein the
`background region determining step includes setting the
`threshold value to a larger value than the predetermined
`threshold value when the difference of the current flame is
`larger than a predetermined value, and to a smaller value
`than it when the difference is smaller.
`25. The method according to claim 23, wherein the
`background region determining step includes dividing the
`current flame into a plurality of regions, measuring a dif-
`ference between each of the regions and each of correspond-
`ing regions of one of the first and second reference flames,
`and setting the threshold value to a larger value than a
`predetermined value when the difierence is larger than a
`predetermined value and to a smaller value when it is
`smaller.
`26. The method according to claim 19, further comprising
`the step of predicting aposition or shape of the object on the
`current frame from a frame from which the object region has
`already been extracted, and selecting the first and second
`reference frames to be used by said background region
`determining step on the basis of the position or shape of the
`object on the current flame which is predicted by said
`predicting step.
`27. The method according to claim 19, which further
`comprises a step of setting a figure surrounding the target
`object on an initial flame of the moving picture signal, and
`a step of setting on one of the first and second reference
`flames a figure surrounding a region on each input flame of
`
`Page 2 of 309
`
`Page 2 of 309
`
`

`

`US 6,335,985 B1
`
`53
`the moving picture signal which corresponds to an image
`inside figure of one of the first and second reference frames
`that temporally difl'ers from the input frame on the basis of
`a correlation between the input frame and the image inside
`figure, and said object region extracting step extracts a
`region, in the image inside figure, which belongs to neither
`the first background region nor the second background
`region as an obiect region.
`28. The method according to claim 26, wherein said
`setting step sets a figure surrounding the target object on the
`basis of an external input.
`29. A method for extracting an object from an input
`picture comprising the steps of:
`setting a figure surrounding a target object on an initial
`flame of a moving picture signal;
`setting on an input frame a figure surrounding a region on
`the input frame of the moving picture signal and
`corresponding to an image inside figure of a reference
`flame that temporally differs flom the input frame on
`the basis of a correlation between the input frame and
`the image inside figure;
`determining a first background region common to a
`current flame as an object extraction target and a first
`reference flame that temporally differs flom the current
`flame on the basis of a difi'erence between the current
`flame and the first reference frame, and determining a
`second background region common to the current
`frame and a second reference frame that temporally
`diflers from the current flame on the basis of a difl'er—
`ence between the current flame and the second refer-
`ence frame, the first background region and the second
`background region indicating a background in the input
`picture
`extracting a region, in the image inside figure of the
`current flame, which belongs to neither the first back—
`ground region nor the second background region, as an
`object region;
`extracting an object region flom the image inside figure
`on the current flame as the object extraction target by
`using a method different from that used by said extract-
`ing steps; and
`selectively switching the extracting steps.
`30. The method according to claim 29, further comprising
`the step of extracting a feature value of a picture in at least
`a partial region of the current frame as the object extraction
`target from the current frame, and wherein said switching
`step selectively switches said extracting steps on the basis of
`the extracted feature value.
`31. The method according to claim 29, wherein said
`extracting an object region step includes predicting a posi-
`tion or shape of the object on the current flame as the object,
`using a frame, flom which an object region has already been
`extracted, as a reference frame, to predict a position or shape
`of the object on the current frame as an object extraction
`target from the reference flame.
`32. The method according to claim 31, wherein said
`extracting steps are selectively switched and used in units of
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`54
`blocks of each frame on the basis of a prediction error
`amount such that the extraction result obtained by said
`extracting an object region step is used as an object region
`when the prediction error caused by said extracting an object
`region step falls within a predetermined range, and the
`extraction result obtained by said extracting a region step is
`used as an object region when the prediction error exceeds
`the predetermined range.
`33. The method according to claim 29, wherein said
`extracting an object region step performs inter-frame pre—
`diction in a sequence different flom an input frame sequence
`such that a frame interval between a reference frame and the
`current frame as the object extraction target is set to not less
`than a predetermined number of flames.
`34.Amethod of extracting an object from an input picture
`comprising the steps of:
`inputting moving picture data and shape data representing
`an object region on a predetermined flame of a plurality
`of flames constituting the moving picture data;
`segmenting a currently processed flame into a plurality of
`blocks;
`searching for a similar block, for each of the blocks,
`which is similar in figure represented by picture data to
`the currently processed block and is larger in area than
`the currently processed block, from the reference
`flame;
`pasting shape data obtained by extracting and reducing
`shape data of each similar block from the reference
`flame on each block of the currently processed flame;
`and
`outputting the pasted shaped data as shape data of the
`currently processed frame.
`35.A method of extracting an object from an input picture
`comprising:
`inputting picture data and shape data representing an
`object region on the picture;
`setting blocks on a contour portion of the shape data;
`searching for a similar block, for each of the blocks,
`which is similar in graphic figure represented by the
`picture data to each block and is larger than the block,
`from the same picture;
`replacing the shape data of each of the blocks with shape
`data obtained by reducing the shape data of each of the
`similar blocks;
`repeating the replacement by a predetermined number of
`times; and
`outputting shape data obtained by repeating the replace—
`ment as corrected shape data
`36‘ The method according to claim 35, further comprising
`a step of secondly repeating the searching for the similar
`block and the predetermined number of replacements of the
`shape data by a plurality of times while decreasing a block
`size every repetition.
`
`Page 3 of 309
`
`Page 3 of 309
`
`

`

`ark? E Jpfix
`.‘Q
`Ga
`l
`5 1.09% E}
`l m ‘3
`°
`(23%}
`9163':
`\‘3ffL5:
`“‘ W193 "
`
`#2;
`
`Customer No. 01333
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`In re Application of:
`
`Jiebo Luo
`
`AUTOMATICALLY PRODUCING
`AN IMAGE OF A PORTION OF A
`PHOTOGRAPHIC IMAGE
`
`Serial No. 09/736,825 /
`
`Filed 14 December 2000/
`
`Commissioner for Patents
`Washington, DC. 20231
`
`Sir:
`
`Group Art Unit: 2625
`
`Examiner: Aaron W. Carter
`
`RECEIVED
`MAR 0 5 2003
`
`Technology Center 2600
`
`ASSOCIATE POWER OF ATTORNEY
`
`I hereby appoint Patrick J. Finnan, Registration No. 39,189, and
`
`Andrew J. Aldag, Registration No. 40,483, whose address is Edell, Shaprio,
`Finnan & Lytle, LLC, 1901 Research Boulevard, Suite 400, Rockville, Maryland
`20850-3164, as associate attorneys in the above-entitled application with full
`power to prosecute this application and to transact all business in the Patent Office
`connected therewith.
`
`Please address all written communications to Thomas H. Close, at
`Eastman Kodak Company, Patent Legal Staff, Rochester, New York 14650 - 2201
`
`and direct all telephone communications to Raymond L. Owens at 585-477-4653.
`
`Respectfully submitted,
`
`
`
`é’z OWL
`
`Attorney for Applicants
`Registration No. 22,3 63
`
`Raymond L. Owens/phw
`Rochester, NY 14650
`
`Telephone: (585) 477—4653
`Facsimile: (585) 477—4646
`
`A:\81595\poweraty.doc
`
`Page 4 of 309
`
`Page 4 of 309
`
`

`

`.'
`
`33“?
`" I."
`
`dd;‘\
`9'31
`100% a?
`5,!
`33'
`TN: 35"?“
`
`00% “ 5
`
`2023‘
`
`Atty Docket: 81595RLO
`0505.0018C
`
`EDELL, SHAPIRO & FINNAN, LLC
`1901 Research Boulevard
`Suite 400
`
`Rockville, Maryland 2085 0-3 164
`(301) 424-3640
`
`In re the PATENT application of:
`
`Jiebo LUO
`
`Examiner: Aaron W. Carter
`
`Serial No.: 09/736,825
`
`Group Art Unit: 2625
`
`Filed: December 14, 2000
`
`For: Automatically Producing an Image of a Portion of a Photographic Image
`
`COMMISSIONER FOR PATENTS
`Washington, D. c. 20231
`
`HECEIVED
`.
`,
`MAR 0 6 2003
`
`TRANSMITTAL LETTER
`
`T
`mm'ogy center 2590
`
`Sir:
`
`Transmitted herewith for filing in the subject application is an Associate Power of
`
`Attorney.
`
`Respectfully submitted,
`
` P trick Jr
`innan
`
`Registration No. 39,189
`
`Hand-delivered: 3 lg ] MD?)
`
`Page 5 of 309
`
`Page 5 of 309
`
`

`

`Jiebo Luo
`
`Appln. No. 09/736,825
`
`computer storage medium having instructions stored therein causing one or more
`
`computers to perform the method of claim 1.
`
`
`
`_____,___,_______..——-u
`
`15.
`
`(Currently Amended) A method of producing an image of a
`
`portion of at least a portion of a photographic image onto a photographic receiver,
`
`comprising thestepsrof:
`
`a)
`
`receiving a digital image corresponding to the photographic image,
`
`the digital image comprising including pixels;
`
`b)
`
`computing abeliefmap ofthe digital I'mde;byusing the pixels of
`
`the digital image to determine a series of features; and using such features to
`
`assign the g probability of the g location of a main subject of the digital image in
`
`the belief map;
`
`c)
`
`determining a crop window having a shape M and a zoom
`
`factor, the shape and th_e zoom fee-tarm determining a size of the crop
`
`window; and
`
`d)
`
`locating the a relative optical position of a photographic image, a
`
`lens assembly, and a photographic receiver in response to the belief map and
`
`illuminating a portion of the photographic image of high subject content to
`
`produce an image of such portion onto g the photographic receiver.
`
`16.
`
`(Currently Amended) The method of claim 15 wherein stepe)
`
`fiartheneempfises—the—steps-etl determining a crop window includes
`
`Page 6 of 309
`
`Page 6 of 309
`
`

`

`_
`-
`‘
`ww
`
`-‘.
`,
`'\
`WW
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`i)
`
`computing a weighted center-of-mass of the belief map, th_e
`
`weighted center-of-rnass weighted by the belief values of
`
`the belief map;
`
`ii)
`
`computing weighted central moments of the belief map,
`
`relative to the center-of-mass and weighted by a weighting
`
`function of each belief value of the belief map;
`
`iii)
`
`computing an effective rectangular bounding box according
`
`to the central moments; and
`
`iv)
`
`determining a crop Window having a shape factor and a
`
`6's
`
`zoom factor, the shape and fig zoom fee—tor factors
`
`determining a size of the crop window.
`
`17.
`
`(Currently Amended) The method of claim 15 wherein step-d)
`
`{asthma-West"— locating the relative optical position of a
`
`photographic image, a lens assemblyJ and a photographic receiver includes
`
`i)
`
`selecting an initial position of the crop window at a location which
`
`includes the center;of;mass;
`
`ii)
`
`using the belief values corresponding to the crop window to select
`
`the position of the crop window to include a portion of the image of high subject
`
`content in response to the belief map; and
`
`iii)
`
`cropping the digital image according to the position of the crop
`
`window.
`
`Page 7 of 309
`
`Page 7 of 309
`
`

`

`WW
`
`@W
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`18.
`
`(Currently Amended) The method of claim 16 wherein step—d)
`
`fufiheeeompases—fiiesteps-ef— locating the relative optical position of a
`
`photographic image, a lens assembly, and a photographic receiver includes
`
`i)
`
`selecting a crop window of a rectangular shape and of an identical
`
`aspect ratio to the (unempped) digital image; and
`
`ii)
`
`selecting a zoom factor to determine the size of the crop window
`
`such that it the crop window encompasses the effective bounding box.
`
`19.
`
`(Currently Amended) The method of claim 16 wherein the
`
`fl)
`:
`
`weighting function instep-b) of comguting weighted central moments of the
`belief map is a linear weighting function.
`
`20.
`
`(Currently Amended) The method of claim 16 wherein the
`
`weighting function iii-steels) of computing weighted central moments of the
`
`belief map is a constant function.
`
`21.
`
`(Currently Amended) The method of claim 17 wherein step—b)
`
`further—eem-paws—Hte—steps—e-f: comguting a belief map of the digital image
`
`includes
`
`i)
`
`calculating a subject content index value for the crop window
`
`derived from the belief values;
`
`Page 8 of 309
`
`Page 8 of 309
`
`

`

`as
`
`@W
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`ii)
`
`following a positioning pfeeedew process of repeating step-i)
`
`
`selecting an initial position of the crog window at a location which includes the
`
`center of the mass for at least two positions of the crop window; and
`
`6’s
`
`iii)
`
`using the subject content index values to select the crop window
`
`osition.
`
`
`
`
`22.
`
`(Original) The method of claim 15 wherein the crop window is
`
`completely within the digital image.
`
`23.
`
`(Currently Amended) The method of claim 16 wherein step-b)
`
`Mber—cem-pfises—thestepofi-pesformmg-a computing a belief map of the digital
`
`image includes clustering of the belief map to identify at least a cluster of highest
`
`belief values corresponding to me main subject, a cluster of intermediate belief
`
`values corresponding to secondary subjects, and a cluster of lowest belief values
`
`corresponding to the background.
`
`24.
`
`(Currently Amended) The method of claim 23 wherein said
`
`clustering includes setting said background portions to a zero belief value.
`
`25.
`
`(Currently Amended) The method of claim 19 further comprising
`
`positioning said th_e crop window such that the subject content index value of said
`
`th_e crop window is at an optimum.
`
`Page 9 of 309
`
`Page 9 of 309
`
`

`

`ww
`
`ow
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`26.
`
`(Currently Amended) The method of claim 17 further comprising
`
`positioning said m crop window such that said th_e crop window includes all of
`
`said main subject cluster.
`
`%
`
`27.
`
`(Currently Amended) The method of claim 26 further comprising
`
`positioning said th_e crop window to include a buffer around said main subject
`
`cluster.
`
`28.
`
`(Currently Amended) A computer storage product having at least
`
`one computer storage medium having instructions stored therein causing one or
`
`more computers to perform the method of claim i_1_5.
`
`10
`
`Page 10 of 309
`
`Page 10 of 309
`
`

`

`ww
`
`aw
`
`/
`
`/
`
`l Amendments to the Abstract:
`e-__.—-""-
`A method of producing an image of at least a portion of a digital imagerthe-di-gital
`
`imageeem-prisifig-pixela—eempfisiag—Ehe—Steps—efflhat includes pixels includes
`
`a}—computing a belief map of the digital image, by using the pixels of the digital
`
`image to determine a series of features; and using such features to assign the probability of
`
`A 5 thelocationofamainsubjectofthedigitalimageinthebeliefmapeba—determininga
`
`crop window having a shape and a zoom factor, the—shape—and—Eeem—fafier—detemfining
`
`which determine a size of the crop window: and e—)—cropping the digital image to include a
`
`portion of the image of high subject content in response to the belief map and the crop
`
`window.
`
`r-
`
`Page 11 of 309
`
`Page 11 of 309
`
`

`

`Jiebo Luo
`
`w V
`
`Appln. No. 09/736,825
`
`U w
`
`REMARKS
`
`Favorable reconsideration of this application in view of the above amendments and
`
`the following remarks is respectfiJlly requested. By this amendment, claims 1-7, 9—13, 15-21
`
`and 23-28 have been amended. Applicants submit that no new matter has been added, and
`
`formal acknowledgement of such is respectfully requested. Currently, claims 1 28 are
`
`pending of which claims 1 and 15 are independent.
`
`The Examiner is thanked for his indication of allowability of claims 2, 4—7, 9-11, 16,
`
`18-21, and 23-25. Formal notice of such is solicited.
`
`Claim 28 was rejected under 35 USC 112, second paragraph, as being indefinite for
`
`failing to particularly point out and distinctly claim the subject matter which applicant
`
`regards as his invention. Claim 28 has been amended to change “claim 1” to --claim 15--, as
`
`suggested by the Examiner. Applicant submits that the amendment to claim 28 addresses the
`
`Examiner’s concern, and overcomes this rejection. Accordingly, withdrawal of this rejection
`
`is respectfully requested.
`
`Claims 1, 3, 8, 12-15, 17, 22, and 26-28 were rejected under 35 USC 102(6) as
`
`anticipated by Jia (US. Patent No. 6,430,320). This rejection is respectfully traversed.
`
`Claims 1 and 15 are independent and claims 3, 8, and 12-14, and claims 17, 22, and
`
`26-28 depend therefrom, respectively.
`
`In general, this application relates to automatically cropping a digital image without
`
`manual techniques, but rather based on scene content as indicated by a belief map.
`
`Generally, a method of cropping a digital image having pixels to produce a cropped digital
`
`image includes developing a belief map of a photographic image by using the pixels to
`
`determine a series of features and using the features to assign a probability of a location of a
`
`-12-
`
`Page 12 of 309
`
`Page 12 of 309
`
`

`

`~
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`J
`
`V V
`
`.-- - .
`
`W W
`
`main subject of the digital image in the belief map, and cropping the digital image to include
`
`main subjects indicated by the belief map to produce the cropped digital image.
`
`In one aspect, a method of producing an image of at least a portion of a digital image
`
`includes providing a digital image having pixels, computing a belief map of the digital image,
`
`determining a crop window having a shape and a zoom factor, and cropping the digital imagc
`
`to include a portion of the image of high subject content in response to the belief map and the
`
`crop window. The belief map is computed by using the pixels of the digital image to
`
`determine a series of features and using the features to assign a probability of a location of a
`
`main subject of the digital image in the belief map. The shape and the zoom factors
`
`determine a size of the crop window.
`
`In another aspect, a method of producing an image of a portion of at least a portion of
`
`a photographic image onto a photographic receiver includes receiving a digital image
`
`including pixels that corresponds to the photographic image, computing a belief map of the
`
`digital image, determining a crop window having a shape factor and a zoom factor, and
`
`locating a relative optical position of a photographic image, a lens assembly, and a
`
`photographic receiver in response to the belief map and illuminating a portion of the
`
`photographic image of high subject content to produce an image of such portion on the
`
`photographic receiver. The belief map is computed by using the pixels of the digital image to
`
`determine a series of features and using such features to assign a probability of a location of a
`
`main subject of the digital image in the belief map. The shape and the zoom factors
`
`determine a size of the crop window.
`
`Jia relates to a system and method for automatically determining unwanted extraneous
`
`material, i.e., background or scanner background information.
`
`-13-
`
`Page 13 of 309
`
`Page 13 of 309
`
`

`

`‘
`
`Jiebo Luo
`
`Appln. No. 09/736,825
`
`Jia fails to describe of suggest the method of independent claim 1. Firstly, Jia fails to
`
`describe or suggest “computing a belief map of the digital image by using the pixels of the
`
`digital image to determine a series of features and using such features to assign a probability
`
`to a location of a main subject of the digital image in the belief map,” as recited. In the
`
`instant application, the belief map is created using MSD, for example, as described in the
`
`specification at pages 6, 7, and 10.
`
`In fact, there is m Mmap in Jia. Jia addresses cropping background from a
`
`scanned document. In Jia, the background is assumed to be largely uniform, to have known
`
`characteristics, and to form a clear boundary between the background and the real image.
`
`Consequently, in Jia, a pixel of the scan line is checked against a reference background pixel,
`
`is treated as an image pixel when the pixel color is different from the color of the reference
`
`background pixel by more than a predetermined threshold value and when the color of the
`
`adjacent pixel is also different from the color of the reference background pixel by more than
`
`a predetermined threshold value, and is not cropped. (See Jia, col. 13, lines 38-39).
`
`Otherwise, it is cropped. thus, there is no belief map in Jia, and no claimed “computing a
`
`belief map of the digital image” in Jia.
`
`Further, Jia fails to describe or suggest the claimed “cropping the digital image to
`
`include a portion of the image of high subject content in response to the belief map and the
`
`crop window” of independent claim 1. Since Jia does not provide a belief map, as explained
`
`above, Jia cannot “crop[] the digital image to include a portion of the image of high subject
`
`content in response to the belief map and the crop window,” as claimed (emphasis added).
`
`Also, in the instant application, the belief map uses the pixels of the original image to
`
`determine features, uses these features to assign a probability as to the location of the main
`
`subject of the digital image in the belief map, as recited above. In Jia, however, elements
`
`-14-
`
`Page 14 of 309
`
`Page 14 of 309
`
`

`

`Jiebo Luo
`
`w W
`
`Appln. No. 09/736,825
`
`Q
`
`702, 710, 712, 714, 716, 718, 720 shown in Fig. 8 of Jia are the six (6) predetermined
`
`boundary pixels used by the Jia system to crop an image. The cropped image then includes
`
`only the portion of the original image located Within the six (6) boundary pixels, everything
`
`else is trimmed away. (See Jia, col. 11, lines 5 - 14). Thus, Jia does not “crop the digital
`
`image to include a portion of the image of high subject content in response to the belief 111a];
`
`and the crop window,” as claimed (emphasis added). Therefore, Jia fails to describe or
`
`suggest the subject matter of independent claim 1, and independent claim 1, and claims 3, 8,
`
`and 12-14, are allowable over the cited art.
`
`Similarly, as to independent claim 15, for at least the reasons presented above, Jia
`
`fails to describe or suggest the “computing a belief map of the digital image by using the
`
`pixels of the digital image to determine a series of features and using such features to assign a
`
`probability of a location of a main subject of the digital image in the belief map,” as claimed.
`
`There is no belief map in Jia; no belief map is computed. Therefore, independent claim 15,
`
`and claims 17, 22, and 26-28 which depend therefrom, are allowable over the cited art.
`
`Thus, Applicants submit that none of claims 1, 3, 8, 12—15, 17, 22, and 26-28 are
`
`described or suggested by Jia, and accordingly, withdrawal of this rejection is respectfially
`
`requested.
`
`Claims 1, 8, and 14 were rejected under the judicially created doctrine of
`
`obviousness—type double-patenting as being unpatentable over claims 44-48 of co—pending
`
`Application Serial No. 09/490,915.
`
`Firstly, Applicants do not understand this rejection. Applicants submit that this
`
`rejection is more properly a provisional double-patenting rejection as two co-pending
`
`applications are noted by the Examiner, and neither application has issued as a patent to date.
`
`-15-
`
`Page 15 of 309
`
`Page 15 of 309
`
`

`

`Jiebo Luo
`
`Appln. No. 09/736,825
`
`(”3"“)
`'
`
`O 0--
`
`Applicants hereby submit a draft Terminal Disclaimer. Upon an indication of
`
`allowability of claims 1, 8, and 14, should such a terminal disclaimer disclaiming a portion of
`
`a patent issuing from this application, which would extend beyond the term of any patent
`
`issuing from co-pending Application Serial No. 09/490,915, become necessary, such a
`
`document will be submitted. Accordingly, withdrawal of this rejection is respectfully
`
`requested.
`
`Applicants submit that all pending claims are in condition for allowance, and formal
`
`notice of such is solicited. If the Examiner has any questions, the Examiner is respectfillly
`
`requested to contact the undersigned at the number indicated below.
`
`Respect
`
`1y submitted,
`
`
`
`Patrick J. Finnan
`
`Registration No. 39,189
`
`EDELL, SHAPIRO & FINNAN, LLC
`1901 Research Boulevard, Suite 400
`Rockville, Maryland 2085 0-3 1 64
`(301) 424-3640
`
`Hand Delivered:
`
`fl g- 2:!03
`-'
`
`-l6-
`
`Page 16 of 309
`
`Page 16 of 309
`
`

`

`.,
`
`"
`
`L:
`
`.
`
`'
`
`(3 D
`DEAF?
`
`“"7543
`50
`1"
`APR 02 2003 g)
`PTO/5B125 (10-00)
`’3.
`@657
`Approved for use through 10/31/2002. 0MB 0651-0031
`\55”!
`15‘";
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`‘ #312 [thnl -‘
`Under the Paperwork Reduction Ac BTTQEE’no persons are required to respond to a coliection of information unless it displays a valid OMB control number.
`TERMINAL DISCLAIMER TO OBVIATE A PROVISIONAL DOUBLE PATENTING DOCKEI Number (Optional)
`
`
`81595RLO/0505.00180
`
`
`RECEI\/ED
`
`.
`,
`APR 0 3 7.903
`
`
`REJECTION OVER A PENDING SECOND APPLICATION
`
`In re Application of: Luo et al.
`
`09/736,825
` Application No.:
`
`December 14, 2000
`Filed:
`For: Automatically Producing An Image of a Portion of a Photographic Image
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Technology Center 2600
`
`the instant application hereby
`in
`interest
`The owner*,E‘:"_"‘_"_K°_d“_‘ffi"fTfi______ , of lgflmercent
`disclaims, except as provided below, the terminal part of the statutory term of any patent granted on the instant
`application, which would extend beyond the expiration date of the full statutory term defined in 35 U.S.C. 154 to
`156 and 173 as shortened by any terminal disclaimer filed prior to the grant of any patent granted on pending
`second Application Number leflgflé___________ , filed onJEEaWJE-_2929___, of any patent on the pending
`second application.The owner hereby agrees that any patent so granted on the instant application shall be
`enforceable only for and during such period that it and any patent granted on the second application are
`commonly owned. This agreement runs with any patent granted on the instan