Case 1:15-cv-00516-LPS-CJB Document 1 Filed 06/19/15 Page 1 of 5 PageID #: 1
`
`IN THE UNITED STATES DISTRICT COURT
`DISTRICT OF DELAWARE
`
`§
`§
`§
`§
`§
`§
`§
`§
`§
`
`Case No. _______________
`
`JURY TRIAL DEMANDED
`
`
`
`ADVANCED MICROSCOPY INC.,
`
`
`
`
`
`
`
`
`v.
`
`CARL ZEISS MICROSCOPY, LLC,
`
`Plaintiff,
`
`Defendant.
`
`
`
`PLAINTIFF’S ORIGINAL COMPLAINT
`
`Plaintiff ADVANCED MICROSCOPY INC. (“Plaintiff”) files this Original Complaint
`
`against Defendant CARL ZEISS MICROSCOPY, LLC, alleging as follows:
`
`I. THE PARTIES
`
`1.
`
`ADVANCED MICROSCOPY INC. (“Plaintiff”) is a corporation organized and
`
`existing under the laws of the State of Delaware, with a principal place of business in Costa
`
`Mesa, California.
`
`2.
`
`Upon information and belief, Defendant CARL ZEISS MICROSCOPY, LLC
`
`(“Carl Zeiss” or “Defendant”) is a corporation organized and existing under the laws of the State
`
`of New York, with a principal place of business in Thornwood, NY. Carl Zeiss may be served
`
`with process by serving an officer, a managing or general agent, or any other authorized agent.
`
`II. JURISDICTION AND VENUE
`
`3.
`
`This is an action for infringement of a United States patent. Federal question
`
`jurisdiction is conferred to this Court over such action under 28 U.S.C. §§ 1331 and 1338(a).
`
`4.
`
`Upon information and belief, Defendant has had minimum contacts with the
`
`District of Delaware such that this venue is fair and reasonable. Defendant has committed such
`
`purposeful acts and/or transactions in this district that it reasonably should know and expect that
`
`
`
`1
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1 Filed 06/19/15 Page 2 of 5 PageID #: 2
`
`it could be hauled into this Court as a consequence of such activity. Upon information and
`
`belief, Defendant has transacted and, at the time of the filing of this Complaint, is transacting
`
`business within the District of Delaware.
`
`5.
`
`Further, upon information and belief, Defendant manufactures (directly or
`
`indirectly through third party manufacturers) and/or assembles products that are and have been
`
`used, offered for sale, sold, and/or purchased in the District of Delaware. Defendant directly
`
`and/or through its distribution network, places infringing products or systems within the stream
`
`of commerce, which stream is directed, at least in part, at this district, with the knowledge and/or
`
`understanding that those products will be sold and/or used in the State of Delaware.
`
`6.
`
`For these reasons, personal jurisdiction exists and venue is proper in this Court
`
`under 28 U.S.C. §§ 1391(b) and (c) and 28 U.S.C. § 1400(b).
`
`III. PATENT INFRINGEMENT
`
`7.
`
`On November 6, 2001, United States Patent No. 6,313,452 (“the ’452 Patent”)
`
`was duly and legally issued for a “MICROSCOPY SYSTEM UTILIZING A PLURALITY OF
`
`IMAGES FOR ENHANCED IMAGE PROCESSING CAPABILITIES.” A true and correct
`
`copy of the ’452 Patent is attached hereto as Exhibit “A” and made a part hereof.
`
`8.
`
` By way of assignment, Plaintiff is the owner of all right, title and interest in and
`
`to the ’452 Patent, with all rights to enforce it against infringers and to collect damages for all
`
`relevant times, including the right to prosecute this action.
`
`9.
`
`On information and belief, Defendant, without authority, consent, right, or
`
`license, and in direct infringement of the ’452 Patent, manufactures, has manufactured, makes,
`
`has made, uses, imports, has imported, markets, sells, or offers for sale systems or products that
`
`directly infringe one or more claims of the ’452 Patent. By way of example only, Defendant’s
`
`
`
`2
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1 Filed 06/19/15 Page 3 of 5 PageID #: 3
`
`use, sale, or offer for sale of a system that includes a Zeiss microscope (for example, Axio
`
`Imager 2, Axio Observer, Axio Scan.Z1, or LSM 510 Confocal) with a motorized stage and
`
`image capture functionality (or with a digital camera or video capture device, for example, Axio
`
`Cam, coupled to the microscope) and the ZEN microscope software (for example, ZEN Blue or
`
`Black edition) with the ZEN Tiles and Positions module for installation on a Windows PC,
`
`infringes at least Claim 1 of the ’452 Patent.
`
`10.
`
`Further, upon information and belief, Defendant induces and/or contributes to the
`
`infringement of one or more of the claims of the ’452 patent by others and is therefore liable for
`
`its indirect infringement. Defendant has had knowledge of the ’452 Patent and knowledge of its
`
`infringement since at least as early as the filing of this suit, at the latest.
`
`11.
`
`For example, on information and belief, the University of Texas at Arlington
`
`(“UTA”), through its Genomics Core Facility, directly infringes at least claim 1 of the ’452
`
`Patent by using in the U.S. an infringing product or system like the ones described above.
`
`12.
`
`On information and belief, Defendant possessed a specific intent to induce
`
`infringement, as evidenced by, at a minimum, its efforts to provide or make available
`
`instructions on how to use the product or system in a way that would infringe the ’452 Patent.
`
`For example, Defendant provides instructions for how to use its infringing systems to combine
`
`multiple images. That intent is also evidenced by its decision to provide equipment to a
`
`laboratory or research facility in which to conduct infringing activities.
`
`13.
`
`Upon information and belief, Defendant actively induces at least UTA, its faculty,
`
`and/or its students to infringe the ’452 Patent by advertising or publicizing an infringing use of
`
`its infringing product or system. For example, Defendant advertises an infringing use of an
`
`infringing product or system by providing product manuals and literature to explain the operation
`
`
`
`3
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1 Filed 06/19/15 Page 4 of 5 PageID #: 4
`
`of its equipment and by providing equipment to a laboratory or research facility in which to
`
`conduct infringing activities.
`
`14.
`
`Upon information and belief, the Defendant’s ZEN Tiles and Positions module,
`
`with its image-combining capabilities, has no substantial non-infringing use. Further, upon
`
`information and belief, Defendant knows that certain of its products, such as the ZEN Tiles and
`
`Positions module, with its image-combining capabilities, is especially made or especially
`
`adapted for use in a system that infringes the ’452 Patent.
`
`16.
`
`17.
`
`Plaintiff expressly reserves the right to assert additional claims of the ’452 Patent.
`
`Plaintiff has been damaged as a result of Defendant’s infringing conduct.
`
`Defendant is, thus, liable to Plaintiff in an amount that adequately compensates for its
`
`infringement, which, by law, cannot be less than a reasonable royalty, together with interest and
`
`costs as fixed by this Court under 35 U.S.C. § 284.
`
`18.
`
`Upon information and belief, Defendant will continue its infringement of the ’452
`
`Patent unless enjoined by the Court. Defendant’s infringing conduct has caused Plaintiff
`
`irreparable harm and will continue to cause such harm without the issuance of an injunction.
`
`IV. JURY DEMAND
`
`19.
`
`Plaintiff hereby requests a trial by jury pursuant to Rule 38 of the Federal Rules of
`
`Civil Procedure.
`
`V. PRAYER FOR RELIEF
`
`
`
`WHEREFORE, Plaintiff respectfully requests that the Court find in its favor and against
`
`Defendant, and that the Court grant Plaintiff the following relief:
`
`a.
`
`Judgment that one or more claims of the ’452 Patent have been directly infringed,
`either literally or under the doctrine of equivalents, by Defendant, or judgment
`that one or more of the claims of the ’452 patent have been directly infringed by
`
`
`
`4
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1 Filed 06/19/15 Page 5 of 5 PageID #: 5
`
`others and indirectly infringed by Defendant, to the extent Defendant contributed
`to or induced such direct infringement by others;
`
`Judgment that Defendant account for and pay to Plaintiff all damages to and costs
`incurred by Plaintiff because of Defendant’s infringing activities and other
`conduct complained of herein;
`
`That Plaintiff be granted pre-judgment and post-judgment interest on the damages
`caused by Defendant’s infringing activities and other conduct complained of
`herein;
`
`That the Court declare this an exceptional case and award Plaintiff its reasonable
`attorney’s fees and costs in accordance with 35 U.S.C. § 285;
`
`That Defendant be permanently enjoined from any further activity or conduct that
`infringes one or more claims of the ’452 Patent; and
`
`That Plaintiff be granted such other and further relief as the Court may deem just
`and proper under the circumstances.
`
`
`b.
`
`
`c.
`
`
`d.
`
`
`e.
`
`
`f.
`
`
`
`
`
`Respectfully submitted,
`
`FARNAN LLP
`
`
`
`/s/Brian E. Farnan
`Brian E. Farnan (Bar No. 4089)
`Michael J. Farnan (Bar No. 5165)
`919 N. Market St., 12th Floor
`Wilmington, DE 19801
`(302) 777-0300
`Fax: (302) 777-0301
`bfarnan@farnanlaw.com
`mfarnan@farnanlaw.com
`
`ATTORNEYS FOR PLAINTIFF
`ADVANCED MICROSCOPY INC.
`
`
`
`
`
`Dated: June 19, 2015
`
`Of Counsel:
`
`David A. Skeels
`Todd I. Blumenfeld
`FRIEDMAN, SUDER & COOKE
`604 East 4th Street, Suite 200
`Fort Worth, TX 76102
`817-334-0400
`Fax: 817-334-0401
`skeels@fsclaw.com
`blumenfeld@fsclaw.com
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`5
`
`
`
`
`
`
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 1 of 12 PageID #: 6
`Case 1:15—cv—00516—LPS—CJB Document 1-1 Filed 06/19/15 Page 1 of 12 Page|D #: 6
`
`
`
`
`
`EXHIBIT A
`
`EXHIBIT A
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 2 of 12 PageID #: 7
`
`(12) United States Patent
`Paragano et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 6,313,452 B1
`Nov. 6, 2001
`
`US006313452B1
`
`(54) MICROSCOPY SYSTEM UTILIZINGA
`PLURALITY OF IMAGES FOR ENHANCED
`IMAGE PROCESSING CAPABILITIES
`
`(58) Field of Search ............................ .. 250/201.1, 201.2,
`250/2013, 548, 216, 559.08; 359/368, 380,
`391, 393; 348/79, 80; 382/284, 294
`
`(75) Inventors: Vincent Paragano, LaWrenceville;
`Douglas Fremont Dixon, Hopewell;
`Michael R. Piacentino, Princeton;
`Rakesh Kumar, Monmouth JCL;
`Harpreet S. SaWhney, Cranbury;
`Lambert E. Wixson, Rocky Hill, all of
`NJ (Us)
`
`.
`
`_
`
`(73) Assrgnee. (Slagio?' Corporation, Princeton, NJ
`
`.
`
`.
`
`(56)
`
`References Cited
`
`*
`
`U'S' PATENT DOCUMENTS
`4,673,988 * 6/1987 Jansson et a1. ...................... .. 348/79
`_
`_
`cued by examlner
`Primary Examiner—John R. Lee
`Assistant Examiner—Kevin Pyo
`
`(74) Attorney) Agent) Or Firm_W?1iam J‘ Burke
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U_S_C_ 154(k)) by 0 days_
`
`(21) APPL No. 09/217 315
`’
`Dec. 21, 1998
`
`(22) Filed:
`
`Related U-S- Application Data
`(60) Provisional application NO- 60/088,779, ?led on Jun- 10,
`1998'
`(51) Int. Cl.7 ................................................... .. G02B 27/64
`(52) US. Cl. ........................ .. 250/201.3; 359/368; 348/79
`
`(57)
`
`ABSTRACT
`
`A microscopy system comprising a microscope including a
`stage, at least one magnifying lens, and a lens controller, a
`video capture device coupled to the microscope capturing a
`plurality of images of an object on the stage of the micro
`scope and a processing subsystem receiving the plurality of
`images from the video capture device, in Which the process
`ing subsystem generates at least one resultant image as a
`function of the plurality of images. The at least one resultant
`image generated by the processing subsystem may include a
`mosaic, submosaic or a sequence of mosaics.
`
`20 Claims, 4 Drawing Sheets
`
`VIDEO CAPTURE
`DEVICE
`
`2”
`
`40
`/
`
`M -—
`
`USER
`INTERFACE
`DEVICE
`
`4a
`
`42
`\
`
`12 __ MOTORIZED
`STAGE
`
`CONTROLLER/INTERFACE
`DEVICE
`
`DEVICE
`
`PROCESSING
`SUBSYSTEM
`
`LENS
`14 — POWER-SETTING
`CONTROLLER
`
`MICROSCOPE
`
`\m
`
`STORAGE
`DEvmE
`
`DISPLAY
`DEVICE
`
`43
`COMPUTER WORKSTATION
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 3 of 12 PageID #: 8nu
`£
`m
`e
`aC
`0.aP
`1.
`2.
`CSPI__
`.51.IQ.
`.m
`
`M4OMm.1.
`._Hwh1..S
`
`eU
`
`9M...a,pmHm,mS
`
`fin.mD2
`
`R..44co1OmU
`
`
`
` M4t5mnu4tmuavP
`
`umC0.0,D6BW
`
`1
`
`E»:
`
`m9Em:z_
`
`ma_>ma
`
`fa:
`
`m:=.E<u82>
`
`m_u_>mn
`
`mz_§u2_._
`
`smztmmam
`
`zo:<E.=:=s
`
`m_u<EEz_
`
`3.5..
`
`mu<#_Ez_>_u._._2_=.8
`
`3.5..
`
`S~_.::os_
`
`SE»
`
`::._m:_
`
`3.3:.
`
`
`
`uu<=ohmmzma
`
`
`
`3.5..uz_E_m.E>>o._
`
`
`
`zo_::e_.=;>:m._.D.=2oo
`
`3
`
`5.3528
`
`u._8m2_u__2
`
`
`
`
`
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 4 of 12 PageID #: 9
`
`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 2 0f 4
`
`US 6,313,452 B1
`
`FIG. 2
`
`SEND REQUEST TO
`c/I DEVICE
`
`_ 20°
`
`SEND COMMAND T0
`MOTORIZED STAGE _ Z05
`
`SCAN
`IMAGES _z'°
`
`SEND IMAGES TO
`THE M DEVICE _ 2"’
`
`SEND IMAGES TO THE
`PROCESSING SUBSYSTEM _ 22“
`
`CREATE
`MOSAIC _ 225
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 5 of 12 PageID #: 10
`
`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 3 0f 4
`
`US 6,313,452 B1
`
`DISPLAY
`MOSAIC _3°0
`
`FIG. 3
`
`usER SELECTS PORTION
`OF MosAIc
`
`" 305
`
`REnuEsT c/l DEVICE
`To POLL THE LPC — 31°
`
`REauEsT C/l nEvlcE To POLL _ 315
`THE MOTORIZED STAGE
`
`DETERMINE
`SCANNING AREA
`
`_ m
`
`DETERMINE _ 3Z5
`TRAvEl. PATHS
`
`MOVE THE MOTORIZED STAGE _
`ALONG TRAVEL PATHS
`33°
`
`STEP 210
`OF FIG. 2
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 6 of 12 PageID #: 11
`
`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 4 0f 4
`
`US 6,313,452 B1
`
`\ 8
`
`5»:
`
`353
`
`555;: I:
`
`22225“
`
`/
`
`2 2.
`
`\
`
`@5352: SE55. Al
`555225
`mus:
`
`_ S
`
`3 SEE
`BEES. 82>
`
`
`
`
`
`352 $5“ '1 35.55.5435,...» :55: l .2
`
`‘we
`
`
`
`ZQhEhvEES "GEES-cu
`
`no /
`
`S / E8855
`
`25 2225. .
`5:358 I 2
`
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 7 of 12 PageID #: 12
`
`US 6,313,452 B1
`
`1
`MICROSCOPY SYSTEM UTILIZING A
`PLURALITY OF IMAGES FOR ENHANCED
`IMAGE PROCESSING CAPABILITIES
`
`This application claims the bene?t of the ?ling date of
`US. Provisional Application No. 60/088,779, ?led Jun. 10,
`1998.
`
`FIELD OF THE INVENTION
`
`The present invention relates to the ?eld of microscopy.
`More particularly, the present invention relates to a micro
`scope coupled to a computer to provide enhanced image
`processing capabilities.
`
`BACKGROUND INFORMATION
`
`10
`
`15
`
`2
`object and providing the captured magni?ed image to an
`image display system for display.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides an interactive microscopy
`system comprising a computer Workstation coupled to a
`microscope and a video capture device. The computer
`display system may include a processing subsystem, a
`display device, and a user interface device. A user may
`interact With the microscopy system via the user interface
`device.
`The microscope may include a motoriZed stage, a lens
`poWer setting controller, and a focus adjustment controller.
`The video capture device may be coupled to the microscope
`to capture a plurality of images of an object mounted on the
`motoriZed stage of the microscope, and provides the plural
`ity of images to the computer Workstation.
`The computer Workstation may then perform various
`image enhancements on the plurality of images based upon
`a user input received via the user interface device. In a ?rst
`operation of the system of the present invention, the com
`puter display system may create an overall image mosaic
`from the plurality of images. An image mosaic is an image
`With a larger ?eld-of-vieW than can be seen in a single
`image.
`In a second operation of the system of the present
`invention, the computer Workstation may create a sub
`mosaic of the overall image mosaic as a function of a user
`input.
`In a third operation of the system of the present invention,
`the user may indicate a speci?c area of interest in the overall
`image mosaic. The computer Workstation may then cause
`the video capture device to capture the plurality of images of
`a scanning area approximating the area of interest in a
`predetermined manner. The computer Workstation may then
`create an enhanced image of the area of interest at a
`resolution greater than a current resolution setting of the
`microscope.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 shoWs an exemplary embodiment of an interactive
`microscopy system according to the present invention.
`FIG. 2 is a ?oWchart depicting a ?rst operation of the
`microscopy system in accordance With the present inven
`tion.
`FIG. 3 is a ?oWchart depicting a second operation of the
`microscopy system in accordance With the present inven
`tion.
`FIG. 4 shoWs an alternative embodiment of the interactive
`microscopy system according to the present invention.
`
`DETAILED DESCRIPTION
`
`FIG. 1 shoWs an exemplary embodiment of an interactive
`microscopy system according to the present invention,
`Which comprises a microscope 10, a video capture device
`20, a controller/interface device 30, and a computer Work
`station 40. The microscope 10 may be of a conventional type
`except that it includes a motoriZed stage (or platform) 12 and
`a lens poWer-setting controller 14.
`In conventional microscopes, a user places an object
`(such as a specimen on a slide) on the stage, and then
`manipulates the stage using a manual control system, such
`as knobs, located directly on the microscope to vieW speci?c
`areas of the object. In contrast, the present invention
`
`Microscopes provide a means for vieWing objects under
`varying levels of magni?cation. Earlier conventional micro
`scopes Were typically stand-alone devices so that all obser
`vations and manipulations of the microscope and the object
`being vieWed Were performed directly on the microscope.
`These conventional microscopes basically consisted of an
`eyepiece through Which a specimen may be vieWed, lenses
`that provided varying levels of magni?cation, a stage for
`holding a slide containing the specimen, a ?rst mechanical
`means for moving the stage, and a second mechanical means
`for adjusting the magni?cation level in order to focus the
`lenses. The user Would use these microscopes by ?rst
`placing a specimen on a slide and attaching the slide to the
`stage of the microscope. The user Would then use the ?rst
`mechanical means to position the slide under the lenses, and
`use the second mechanical means to focus on and magnify
`a portion of the specimen.
`US. Pat. No. 5,627,442 to Engelse et al. describes another
`conventional type of microscope con?guration, in Which a
`microscope is coupled With an image display system. The
`image display system includes a camera coupled to the
`microscope and a computer Workstation. The computer
`Workstation may include a monitor, a keyboard and a remote
`platform controller. In this con?guration, the camera pro
`vides an image of the object being vieWed on the microscope
`to the Workstation. The user may then manipulate the remote
`platform controller at the Workstation to move the platform
`on the microscope in order to change the vieW of the object
`displayed on the monitor of the Workstation.
`One common shortcoming in these conventional micro
`scope systems is their limited ?eld-of-vieW capabilities. A
`?eld-of-vieW is the total amount of an area of interest that
`these microscope systems alloW the user to vieW at any
`given moment in time. In these conventional microscope
`systems, the ?eld-of-vieW is limited to What could be seen
`through the microscope at any speci?c instant in time. To see
`other vieWs of the area of interest, for example, an area to the
`left, to the right, above or beloW the current vieW, the user
`must either
`reduce the lens magni?cation level or (ii)
`move the platform to obtain a different vieW. HoWever, if the
`user reduces the lens magni?cation level, this necessarily
`reduces the amount of detail that can be seen. If the user
`moves the platform, the user can see a different vieW, but
`loses part of the original vieW (i.e., if the user moves the
`platform to the left a predetermined amount, then the user
`Will no longer be able to see an amount of the original vieW
`corresponding to the movement to the left of the platform).
`Another common shortcoming is the lack of image enhance
`ment capabilities of these conventional microscope systems.
`More particularly, conventional microscope systems are
`generally limited to capturing a magni?ed image of an
`
`25
`
`35
`
`45
`
`55
`
`65
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 8 of 12 PageID #: 13
`
`US 6,313,452 B1
`
`3
`includes a motorized stage 12 Which includes at least one
`motor to cause a movement of the motoriZed stage 12 in a
`selectable direction in response to a motor control signal
`provided to the motoriZed stage via the controller/interface
`device 30. The motoriZed stage 12 may, upon request from
`the controller/interface device 30, provide a motor status
`signal, Which may indicate information such as, for
`example, a current position of the motoriZed stage 12. In the
`exemplary embodiment, the motoriZed stage 12 may
`include, for example, tWo stepper or DC motors (not shoWn)
`Which drive the stage along tWo directions. For example, a
`?rst motor may drive the stage along an X-axis, and the
`second motor may drive the stage along a Y-axis. These axes
`may be determined With reference to a vieW seen through an
`eyepiece (not shoWn) of the microscope. Of course, the
`motoriZed stage 12 may be moved using other means Within
`the scope of the present invention.
`As in most conventional microscopes, the microscope 10
`may have a plurality of lenses of different magni?cation
`poWers (not shoWn). The lens poWersetting controller 14
`may be used to change betWeen the plurality of lenses to
`select a speci?c magni?cation level. The lens poWer-setting
`controller 14 may be a conventional control system such as
`a sWitch or knob (not shoWn) located on the microscope 10,
`or a remote control system so that a user may select a
`magni?cation level via the computer Workstation 40 and
`have the lens poWer-setting controller 14 set the microscope
`10 to that magni?cation level. In this exemplary
`embodiment, the lens poWer-setting controller 14 may be
`controlled by a lens control signal provided by the
`controller/interface device 30. The lens poWer-setting con
`troller 14 may also provide the controller/interface device 30
`With a lens status signal upon a request signal from the
`controller/interface device 30, Which provides the controller/
`interface device 30 With information such as, for example,
`the lens poWer-setting controller’s 14 current magni?cation
`level.
`The video capture device 20 may be coupled to the
`microscope 10 in order to capture a plurality of images of an
`object being vieWed through the microscope 10, and provide
`a digitiZed form of the plurality of images to the controller/
`interface device 30. The video capture device 20 may be, for
`example, a CMOS imager or other loW-cost digital imager
`such as, for example, a charge-coupled device (“CCD”)
`camera, etc. Due to the image processing/enhancements
`performed in accordance With the present invention, the
`CMOS imager may be a loW-resolution imager having such
`loW resolutions such as, for example, 160x120. These types
`of video capture devices are relatively loW-cost and Well
`knoWn; thus, they Will not be discussed further. The video
`capture device 20 may be controlled in response to a video
`capture device control signal via the controller/interface
`device 30 to capture the plurality of images at predetermined
`intervals.
`The controller/interface device 30 may be, for example, a
`?eld programmable gate-array (“FPGA”) or an application
`speci?c integrated circuit (“ASIC”). Using conventional
`semiconductor technology that are knoWn to those skilled in
`the art, the controller/interface device 30 may be custom
`programmed to provide the capabilities to control the motor
`iZed stage 12, the lens poWer-setting controller 14 and the
`video capture device 20. The controller/interface device 30
`may be coupled to the computer Workstation 40 via a
`Workstation interface device 48, so that a user Working on
`the computer Workstation 40 may control the various
`devices connected to the controller/interface device 30 via
`the controller/interface device 30. The controller/interface
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`4
`device 30 may provide the plurality of digitiZed images
`originating in the video capture device 20 to the computer
`Workstation 40 via the Workstation interface device 48, and
`receive commands via the Workstation interface device 48
`Which are used to control the motoriZed stage 12 and the lens
`poWer-setting controller 14. The Workstation interface
`device 48 may include, for example, an interface port (i.e.,
`serial, parallel, universal serial bus, etc.) or an interface card.
`In this exemplary embodiment, the Workstation interface
`device 48 may be, for example, a parallel port. Furthermore,
`although the controller/interface device 30 is illustrated as
`being separate from the computer Workstation 40 in this
`exemplary embodiment, the controller/interface device 30
`may also be an add-on that connects directly into the
`computer Workstation 40 such as an interface card, plug-in
`device, etc. The controller/interface device 30 may also be
`integrated into the computer Workstations 40.
`The computer Workstation 40 may also include a process
`ing subsystem 42, a storage device 43, a user interface
`device 44 and a display device 46. The user interface device
`44 may include devices such as, for example, a keyboard, a
`mouse, a trackball, etc., to alloW the user to interact With the
`computer Workstation 40, and thereby control the micro
`scope 10 and the video capture device 20. The processing
`subsystem 42 may run a softWare driver Which alloWs the
`processing subsystem 42 to communicate With the
`controller/interface device 30 to generate the video capture
`device control signals, the motor control signals, the lens
`control signals, and the request signal for the lens poWer
`setting controller 14. The processing subsystem 42 may also
`run a softWare application that provides the user With an
`easy-to-use interface, Which alloWs the user to control the
`microscopy system. For example, the softWare application
`may provide a graphic user interface (“GUI”). These type of
`GUIs are Well-knoWn to those skilled in the art and Will
`therefore not be discussed further herein.
`FIG. 2 is a ?oWchart depicting a ?rst operation of the
`microscopy system in accordance With the present
`invention, in Which the microscopy system constructs a
`mosaic of an object being vieWed through the microscope.
`Mosaics arc formed by combining a plurality of images from
`a speci?c video capture device to provide a combination
`image having a larger ?eld-of-vieW than Would be possible
`in any single image from the speci?c video capture device.
`By obtaining a plurality of images of a particular scene or
`area of interest, and correlating the images to each other or
`the overall scene, the plurality of images may be combined
`to form the mosaic. Using this mosaicing process, the
`microscopy system in accordance With the present invention
`may construct a mosaic from the plurality of images
`received from the video capture device 20. This addresses
`the shortcomings in the conventional microscopy systems
`Which require that the user to either reduce the magni?cation
`level or move the microscope platform to vieW areas outside
`of the present vieW through the microscope. The microscopy
`system according to the present invention may receive a ?rst
`image from the video capture device 20, and as the user
`moves the platform, instead of losing information in the ?rst
`image, the system Would add the information from a second
`image received from the video capture device 20 shoWing a
`different overall area of interest. Since, the second image
`may also be obtained by the video capture device 20 Without
`changing the magni?cation level, there Would be no loss of
`detail as there Would be the case if the user had reduced the
`magni?cation level.
`In a ?rst step 200, the processing subsystem 42 sends a
`request via the Workstation interface device 48 to the
`
`

`
`Case 1:15-cv-00516-LPS-CJB Document 1-1 Filed 06/19/15 Page 9 of 12 PageID #: 14
`
`US 6,313,452 B1
`
`5
`controller/interface device 30, so that the controller/interface
`device 30 Will then command the video capture device 20
`and the motorized stage 12, at step 205, to cooperatively
`capture a plurality of images of the object. At step 210, the
`motoriZed stage 12 may move in a back and forth travel path
`along a ?rst axis, for example, the X-axis, and at the end of
`each travel path, the motoriZed stage 12 may move along an
`axis orthogonal to the ?rst axis, for example, the Y-axis, a
`predetermined amount, so that the video capture device 20
`may capture a plurality of images covering at least the entire
`object to be vieWed.
`At step 215, the video capture device 20 Will send the
`plurality of images in a predetermined digitiZed format to
`the controller/interface device 30. At step 220, the
`controller/interface device 30 Will then send the plurality of
`images to the processing subsystem 42 via the Workstation
`interface device 48. At step 225, the processing subsystem
`42 may then create a mosaic from the plurality of images. It
`is possible to send the images individually from the
`controller/interface device 30 to the processing subsystem
`42, so that the mosaic creation may be performed in real
`time. It is also possible to store the plurality of images at the
`controller/interface device 30, and then send all of the
`images at once to the processing subsystem 42.
`In this exemplary embodiment, the processing subsystem
`42 may create the mosaic using the exemplary techniques
`and algorithms described in US. Pat. No. 5,649,032 to Burt
`et al. (the “Burt ’032 Patent”), Which is expressly incorpo
`rated herein by reference. Using these techniques and
`algorithms, the microscopy system according to the present
`invention may create either a dynamic or static mosaic. In a
`dynamic mosaic, the motoriZed stage 12 and the video
`capture device 20 may continue to operate even after the
`mosaic has been created, so that there is a relatively con
`tinuous How of images processed by the processing sub
`system 42. Each image may then be appropriately placed in
`the mosaic to provide a continually updating mosaic image
`of the object to be vieWed.
`Once the processing subsystem 42 has completed forming
`the mosaic, the processing subsystem 42 may cause the
`mosaic to be displayed on the display device 46. The
`techniques and steps involved in displaying images on a
`display system are Well-knoWn and Will therefore not be
`discussed further herein. The processing subsystem 42 may
`also store the mosaic to the storage device 43 for retrieval,
`or further processing at a later time.
`In a ?rst example of further processing that may be
`performed, the processing subsystem 42 may create a plu
`rality of time-sequential mosaics. The processing subsystem
`42 may then display each one of the plurality of mosaics in
`sequence to effectively provide a time-lapse video of the
`plurality of time-sequential mosaics. Alternatively, the pro
`cessing subsystem 42 may process the plurality of time
`sequential mosaics to detect a moving object Within the
`plurality of timesequential mosaics. This may be performed
`by looking at the change in the images betWeen the plurality
`of images and assuming that any change in the plurality of
`images is due to a moving object. Then the processing
`subsystem 42 may then extract the moving object from the
`rest of the image to separate out and isolate the background.
`An exemplary process that may be used is described in detail
`in US. Pat. No. 5,649,032 to Burt et al., Which is expressly
`incorporated herein by reference. An alternative process that
`may be used in accordance With the present invention is
`described in US. Pat. No. 5,768,447 to Irani et al., Which is
`expressly incorporated herein by reference. Using this
`process, the processing subsystem 42 may detect and track
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`a moving object in the plurality of images, and create a
`timelapse video of the moving obj ect’s motion over time and
`superimpose this time-lapse video of the moving object on
`a selected background.
`Another example of further processing that the processing
`subsystem 42 may perform Would be to use the lens control
`signal from the lens poWer-setting controller 14, Which has
`the operative magni?cation level at Which the image Was
`taken, and provide measurements of a section selected by the
`user. For ex