`Case 1:12—cv—00398—GMS Document 27-2 Filed 02/21/13 Page 1 of 19 Page|D #: 345
`
`EXHIBIT B
`
`EXHIBIT B
`
`
`
`
`
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 2 of 19 PageID #: 346
`
`UNITED STATES DEPARTMENT OF COMMERCE
`United States Patent and Trademark Office
`
`February 14, 2012
`
`THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COPY FROM
`THE RECORDS OF THIS OFFICE OF:
`
`U.S. PATENT: 7,477,284
`ISSUE DATE: January 13, 2009
`
`By Authority of the
`Under Secretary of Commerce for Intellectual Property
`and Director of the United States Patent and Trademark Office
`
`T. LAWRENCE
`Certifying Officer
`
`HET00000279
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 3 of 19 PageID #: 347
`
`111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US007477284B2
`
`(12) United States Patent
`Peleg et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,477,284 B2
`Jan.13,2009
`
`(54) SYSTEM AND METHOD FOR CAPTURING
`AND VIEWING STEREOSCOPIC
`PANORAMIC IMAGES
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`(75)
`
`Inventors: Shmuel Peleg, Mevaseret Zion (IL);
`Moshe Ben-Ezra, Jerusalem (IL); Yael
`Pritch, Jerusalem (IL)
`(73) Assignee: Yissum Research Development
`Company ofthe Hebrew University of
`Jerusalem, Jerusalem (IL)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 894 days.
`(21) Appl. No.: 09/861,859
`(22) Filed:
`Apr. 19, 2001
`Prior Publication Data
`(65)
`
`( * ) Notice:
`
`Nov. 11,2004
`US 2004/0223051 A1
`Jul. 21, 2005
`US 2005/0157166A9
`Related U.S. Application Data
`
`(60) Provisional application No. 60/230,562, filed on Aug.
`30, 2000, provisional application No. 60/217,152,
`filed on Jul. 6, 2000, provisional application No.
`60/198,381, filed on Apr. 19, 2000, provisional appli(cid:173)
`cation No . 60/113,962, filed on Dec. 28, 1998, provi(cid:173)
`sional application No . 60/102,720, filed on Sep. 29,
`1998, provisional application No. 60/100,721, filed on
`Sep. 17, 1998.
`Int. CI.
`(2006.01)
`H04N 13100
`(52) U.S. CI. ....... ................. ............................... . 348/53
`(58) Field of Classification Search ............. ...... 348/36,
`348/42, 51, 158, 218.1, 98, 44, 46, 48, 49,
`348/52, 53; 382/154, 232, 294, 299, 284,
`382/268; 352/57, 69- 70; 396/324; H04N 13/ 00
`See application file for complete search history.
`
`(51)
`
`4,134,644 A
`5,455,689 A *
`5,594,498 A *
`5,649,032 A •
`5,657,073 A
`5,721,5 85 A
`5,764,871 A •
`5,825,466 A
`5,963,664 A •
`5,986,668 A •
`5,991,444 A *
`
`1/1979 Marks eta! ................. 350/132
`10/1995 Taylor eta! .......... ....... 358/450
`111997 Fraley ........................ 348/158
`7/1997 Burt eta! . ................... 382/284
`8/1997 Henley et al.
`2/1998 Keast eta! .................... 348/36
`6/1998 Fogel ......................... 345/427
`10/1998 Lo eta! ........................ 355/33
`10/ 1999 Kumar et al.
`. .............. 382/154
`11/1999 Szeliski eta! . .............. 345/634
`11/1999 Burt eta! .
`................... 382/232
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`0361297
`
`4/1990
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Peleg, Schmuel et a!., "Stereo Panorama with a Single Camera",
`IEEE, 1999, pp. 395401.
`
`(Continued)
`
`Primary Examiner- Tung Vo
`(74) Attorney, Agent, or Firm- Browdy and Neimark,
`P.L.L.C.
`
`(57)
`
`ABSTRACT
`
`Method and apparatus for generating images of a scene from
`image data of the scene and displaying the images to provide
`a sense of depth. In some embodiments of the method and
`apparatus the generated images are mosaics .
`
`55 Claims, 6 Drawing Sheets
`
`a, b, c, d, e,
`
`IDDJ50(3)
`51al._ __ _..}.._a, .... r _____ ~_os_G~-~<:_a· _ __,
`51b~._' __ _.1.._~ .... r _____ ~_OS-GA-~C-"b_" - - '
`_ _..i .... c, ..... f _____ ~_~S-GA-~C-"c· _ __,
`51c~._ _
`51dll... _ _ _.'l ... d, ... [ _ __ __ ~_os_:_~c_.d_· _ ___.
`51el'-_ __.I .... · ... l _____ ~_s_G~-~~_e·---'
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000280
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 4 of 19 PageID #: 348
`
`US 7,477,284 B2
`Page2
`
`U.S. PATENT DOCUMENTS
`
`6,075,905 A *
`6,078,701 A *
`6,083,353 A *
`6,141 ,145 A
`6,275,206 Bl •
`6,278,480 Bl *
`6,377,294 B2 *
`6,434,280 Bl *
`6,435,969 Bl *
`6,512,892 Bl *
`6,522,787 Bl *
`6,532,036 Bl *
`6,532,298 Bl *
`6,535,650 Bl *
`6,643,413 B1 *
`6,665,003 B1 *
`6,717,608 Bl *
`6,750,860 B1 *
`6,831 ,677 B2 •
`6,850,210 B1 *
`6,956,573 Bl *
`200110010546 A1 *
`200110020976 A1
`
`6/2000 Herman et al . .............. 382/284
`612000 Hsu et al.
`. .................. 382/294
`7/2000 Alexander, Jr .
`............. 202/158
`10/2000 Nalwa
`8/2001 Tsai et al. . .................... 345/88
`8/2001 Kurahashi et al. ............. 348/59
`4/2002 Toyofuku et al.
`. .. .......... 348/36
`8/2002 Peleg et al .
`....... .. ... ..... 382/299
`8/2002 Tanaka et al. .......... ....... 463/44
`112003 Montgomery et al . ....... 396/326
`212003 Kumar eta! .
`............... 382/268
`3/2003 Peleg et al.
`. .................. 348/36
`3/2003 Cambier et al .............. 382/117
`3/2003 Poulo et al. ................. 382/284
`11/2003 Shum et al .................. 382/284
`1212003 Peleg et al. . .................. 348/36
`4/2004 Mancuso et al. . ......... .... 348/36
`6/2004 Shum et al .................. 345/419
`12/2004 Peleg et al. . ..... ............. 348/36
`212005 Lipton et al. ... ..... ... ........ 345/6
`10/2005 Bergen et al. . .............. 345/473
`8/2001 Chen .......................... 348/218
`9/200 1 Peleg et al.
`
`1112001 Peleg et al.
`200110038413 A1
`200110043264 AI * 1112001 Sinclair et al .
`
`................ 348/36
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`IL
`JP
`JP
`JP
`JP
`wo
`wo
`
`0 989 436 A2
`0989436
`W098/34195
`11146425
`11146425
`11164326
`11164326
`W099/ 17555
`wo 99/ 18725
`
`3/2000
`3/2000
`
`.. 8/1998
`
`5/1999
`5/1999
`6/1999
`6/1999
`4/1999
`4/1999
`
`OTHER PUBLICATIONS
`
`Ishiguro, H. et al.; "Correspondence, Omni-Directional Stereo";
`IEEE Transactions on Patern Analysis and Machine Intelligence, vol.
`14,Feb. 1992;pp.257-262.
`Gluckman, J. et al.; "Real-Time Omnidirectional and Panoramic
`Stereo"; Proc . Of Image Understanding Workshop, IUW98;1998; 5
`Pages; Retrieved from the Internet: <http://cis.poly.edu/-gluckman/
`papers/uw98a.pdf>.
`* cited by examiner
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000281
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 5 of 19 PageID #: 349
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 1 of6
`
`US 7,477,284 B2
`
`FIG. I
`
`11N)
`Stereo dat
`
`10~
`
`Distribution
`Channel
`
`12A
`
`Viewing
`device
`
`Viewing
`device
`12M
`
`FIG. 2
`
`;23
`I
`
`2 r-
`
`3
`
`}--24
`;22
`L
`4 lD 0
`DO
`DO
`DO
`
`!0~1
`
`f.--20
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000282
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 6 of 19 PageID #: 350
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 2 of6
`
`US 7,477,284 B2
`
`·34
`
`Comrmmicati.on
`tmit
`
`OPR CTRL
`PANEL22
`
`FIG. 3
`30
`
`Capture
`u..-rllt
`
`l
`; '
`I Processing
`
`unit
`
`4
`
`memory
`mrit
`
`t
`
`I
`
`,~2 5
`
`i
`
`Local'
`Display
`i (lMno/stereo)
`i
`
`33A
`
`338
`
`FIG. 4
`
`;::_4_0_.;:'\,...__ _ ____,
`
`(wireless or
`non -wi n~less)
`
`Receiver H Decoder
`
`d
`
`4~
`
`Direction
`Control
`mrit
`
`j
`
`I
`
`(;
`
`; 42
`
`Display
`unit
`
`I
`
`!
`
`r
`J
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000283
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 7 of 19 PageID #: 351
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 3 of6
`
`US 7,477,284 B2
`
`FIG. 5
`
`;
`
`50(1)
`
`-
`
`"' ~
`
`lli I
`I
`I
`
`~
`
`51 a
`
`51b
`
`51c
`
`51d
`
`51e
`
`I ,.,...
`
`"'
`
`50(2)
`•••
`
`if
`
`~
`
`- 50(3)
`
`-
`
`;
`I
`
`I
`
`MOSAIC
`IMAGE na"
`
`MOSAIC
`IMAGE "b"
`
`MOSAIC
`IMAGE "c"
`
`MOSAIC
`IMAGE "d ..
`
`MOSAIC
`IMAGE .. e ..
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000284
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 8 of 19 PageID #: 352
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 4 of6
`
`US 7,477,284 B2
`
`F /G. 6
`
`/61(2)
`LonllcUor Lens · (,A
`
`62(3 )(A)---- .
`-.
`
`62(3)(8)-
`
`62(3)(C)----·101
`
`62(2)(8)-~
`
`62(2)(C)---
`
`62(1 )(A)---
`
`62(1 )(8)--- 1
`
`62(1)(C)--~--f
`
`60--'
`
`61
`
`61(1)
`
`63(A)
`
`63(8)
`
`/104A
`
`1048
`
`104C
`
`100 __:,
`
`FIG. 7
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000285
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 9 of 19 PageID #: 353
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 5 of6
`
`US 7,477,284 B2
`
`F IG.JO
`
`FIG. 8
`
`111
`
`110
`
`116A-=
`9
`__..:;, 11681
`
`1448
`
`142
`
`F IG.ll
`
`146A-=
`
`140__A
`
`1468/
`
`150_..:;,
`
`F IG.12
`
`164A
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000286
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 10 of 19 PageID #: 354
`
`U.S. Patent
`
`Jan.13,2009
`
`Sheet 6 of6
`
`US 7,477,284 B2
`
`FIG. 9
`
`124C 122A
`
`124A
`
`120 ___.,;,
`
`170~
`
`171K
`
`F IG.14
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000287
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 11 of 19 PageID #: 355
`
`US 7,477,284 B2
`
`1
`SYSTEM AND METHOD FOR CAPTURING
`AND VIEWING STEREOSCOPIC
`PANORAMIC IMAGES
`
`RELATED APPLICATIONS
`
`2
`in the art. In one technique, described in Joshua Gluckman, et
`al., "Real-Time Omnidirectional And Panoramic Stereo,"
`DARPA Image Understanding Workshop, 1998, two omnidi(cid:173)
`rectional cameras, vertically displaced along a common axis,
`5 record panoramic images ofthe surrounding scene. Since the
`cameras are displaced, the pair of images recorded by the
`cameras, when considered in combination, will provide depth
`information for objects in the scene surrounding the cameras.
`However, since the displacement is vertical, the recorded
`1 o images are inappropriate for human stereo panoramic percep(cid:173)
`tion.
`
`SUMMARY OF THE INVENTION
`
`This application is a continuation-in-part of U.S. applica(cid:173)
`tion Ser. No. 09/792,638 filed On Feb. 24, 2001, U.S. appli(cid:173)
`cation Ser. No. 09/726,198 filed on Nov. 29, 2000 now U.S.
`Pat. No. 6,795,109, and U.S. application Ser. No. 09/396,248
`filed on Sep. 16, 1999, now U.S. Pat. No. 6,665,003. U.S.
`application Ser. No. 09/726,198 is a continuation-in-part of
`U.S. application Ser. No. 09/396,248. This application also
`claims the benefit under 119(e) of U.S. provisional applica(cid:173)
`tion60/198,381 filedonApr. 19, 2000, U.S. provisional appli- 15
`cation 60/217,152 filed Jul. 6, 2000 and U.S. provisional
`application 60/230,562 filed on Aug. 30,2000.
`
`INCORPORATION BY REFERENCE
`
`U.S. patent application Ser. No. 09/396,248, filed Sep. 16,
`1999, in the names of Shmuel Peleg, et al., entitled "System
`and Method for Generating and Displaying Panoramic
`Images and Movies,"(hereinafter referred to as "the Peleg I
`patent application") assigned to the assignee of the present
`application, incorporated herein by reference.
`U.S. patent application Ser. No. 09/726,198, filed Nov. 29,
`2000, in the names of Shmuel Peleg, et al., entitled "Stereo
`Panoramic Camera Arrangements For Recording Panoramic
`Images Useful In A Stereo Panoramic Image Pair," (herein- 30
`after referred to as "the Peleg II patent application") assigned
`to the assignee of the present application, incorporated herein
`by reference.
`U.S. patent application Ser. No. 09/792,638, filed Feb. 24,
`2001, in the names of Shmuel Peleg, et al., entitled "System 35
`And Method For Facilitating The Adjustment Of Disparity In
`A Stereoscopic Panoramic Image Pair;' (hereinafter referred
`to as "the Peleg III patent application") assigned to the
`assignee of the present application, incorporated herein by
`reference.
`
`FIELD OF THE INVENTION
`
`The invention provides a new and improved system and
`method for generating and displaying a stereoscopic pan(cid:173)
`oramic image set, comprising respective at least two pan(cid:173)
`oramic images of a scene, each having a different viewing
`direction, for contemporaneous viewing by respective left
`20 and right eyes of a viewer to provide an apparent stereoscopic
`image of the scene to the viewer.
`In brief summary, the invention provides an arrangement
`for recording images for use in generating and utilizing
`images comprising a stereoscopic image set. The arrange-
`25 ment includes at least one stereoscopic data source, at least
`one utilization device and a distribution channel. The at least
`one stereoscopic data source records images from which a
`stereoscopic image set can be generated. The utilization
`device is provided to facilitate viewing, printing, or otherwise
`utilizing the stereoscopic image set. The distribution channel
`facilitates transferring information between the stereoscopic
`data source and the utilization device, the information includ(cid:173)
`ing the images as recorded by the stereoscopic data source or
`the stereoscopic image set itself. If a stereoscopic image set is
`to comprise a plurality of mosaic images, the mosaic images
`may be generated by the stereoscopic data source, the image
`utilization device and/or the distribution channel.
`A number of stereoscopic data sources of diverse configu(cid:173)
`rations are disclosed, including fixed and moving mirrors,
`40 prisms, and lenses, which may be used in the stereoscopic
`data source in the stereoscopic image arrangement.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention relates generally to the field of recording and
`generating images, and more particularly to the generation, 45
`displaying and printing of panoramic images stereoscopi(cid:173)
`cally. The invention specifically provides a system and
`method for generating and displaying a stereoscopic pan(cid:173)
`oramic image set, comprising respective at least two pan(cid:173)
`oramic images of a scene, each having a different viewing 50
`direction, for contemporaneous viewing by respective left
`and right eyes of a viewer to provide an apparent stereoscopic
`image of the scene to the viewer.
`
`BACKGROUND OF THE INVENTION
`
`55
`
`Panoramic images are images of a scene having a wide
`field of view, up to a full 360°. Panoramic images may be
`recorded using a wide angled lens, a mirror, or the like,
`providing a wide field of view. Panoramic images having a 60
`wider field of view can be generated by, for example, record(cid:173)
`ing a plurality of images around a particular point and, using
`conventional mosaicing techniques, generating a single
`mosaic image. Panoramic images may also be generated of
`simulated scenes using conventional computer graphics tech(cid:173)
`niques. Stereoscopic panoramic images can also be generated
`from images using various techniques known to those skilled
`
`This invention is pointed out with particularity in the
`appended claims. The above and further advantages of this
`invention may be better understood by referring to the fol(cid:173)
`lowing description taken in conJunction with the accompany-
`ing drawings, in which:
`FIG. 1 schematically depicts a stereoscopic panoramic
`image arrangement for recording, generating and displaying
`stereoscopic panoramic images, constructed in accordance
`with the invention;
`FIG. 2 schematically depicts an exterior plan view of an
`illustrative stereoscopic data source for use in connection
`with the arrangement depicted in FIG. 1;
`FIG. 3 depicts a functional block diagram of the stereo(cid:173)
`scopic data source depicted in FIG. 2;
`FIG. 4 depicts a functional block diagram of an illustrative
`viewing device depicted in FIG. 3;
`FIG. 5 is useful in understanding the operations performed
`by the stereoscopic panoramic image arrangement depicted
`in FIG. 1 in connection with generating a stereoscopic pan-
`65 oramic set;
`FIG. 6 is useful in understanding the arrangement for gen(cid:173)
`erating and displaying lenticular prints; and
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000288
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 12 of 19 PageID #: 356
`
`US 7,477,284 B2
`
`3
`FIGS. 7 through 14 depict illustrative image recording
`arrangements that may be used in connection with the stereo(cid:173)
`scopic data source described in connection with FIGS. 2 and
`3.
`
`DETAILED DESCRIPTION OF AN
`ILLUSTRATIVE EMBODIMENT
`
`FIG. 1 schematically depicts a stereoscopic panoramic
`image arrangement 10 for recording, generating and display(cid:173)
`ing stereoscopic panoramic images, constructed in accor(cid:173)
`dance with the invention. With reference to FIG. 1, the ste(cid:173)
`reoscopic panoramic image arrangement 10 includes one or
`more stereoscopic data sources llA through UN (generally
`identified by reference numeral11n) and one or more viewing
`devices 12A through 12M (generally identified by reference
`numeral 12m). Generally, each stereoscopic data source lln
`records images and generates therefrom at least one stereo(cid:173)
`scopic panoramic image set comprising a set of panoramic
`images. Each stereoscopic data source lln can also transmit
`respective stereoscopic panoramic image sets that it generates
`through a distribution channel13 to one or more of the view(cid:173)
`ing devices 12m for viewing, generation of lenticular prints,
`or other disposition as will be apparent to those skilled in the
`art.
`An illustrative stereoscopic data source lln will be
`described below in connection with FIGS. 2 and 3. Generally,
`the stereoscopic data source lln will include an image
`recording arrangement, an image processing arrangement
`and a communication arrangement. The image recording
`arrangement records one or more images from which a ste(cid:173)
`reoscopic panoramic image set is generated. The image
`recording arrangement may be similar to those described in
`the Peleg I or Peleg II patent applications. If the image record(cid:173)
`ing arrangement is similar to that described in the Peleg I
`patent application, or to those described in the Peleg II patent
`application for which the arrangement is rotated and/or trans(cid:173)
`lated to facilitate recording of images from which a pan(cid:173)
`oramic image is generated, at least the image recording
`arrangement records a series of images as it is rotated around
`an axis comprising a center of rotation, translated along a
`path, or any combination of rotation and translation. As the
`image recording arrangement is rotated and/or translated, it
`records a series of images from which the image processing
`arrangement generates at least one set of panoramic images
`comprising a stereoscopic panoramic image set as described
`in the Peleg I patent application, or as described below in
`connection with FIG. 5. Generally, in generating a stereo(cid:173)
`scopic panoramic image set, the image recording arrange(cid:173)
`ment will record a series of images. For each panoramic
`image in a stereoscopic panoramic image set, the image pro(cid:173)
`cessing arrangement will generate the panoramic image by
`mosaicing together strips from successive ones of the images.
`The strips obtained from the respective images may all have
`the same displacement from the center of the respective
`images, as described in the Peleg I patent applications, or they
`may have different displacements, as described in the Peleg
`III patent application. Alternatively, as noted above, the pan(cid:173)
`oramic images may be generated as described below in con(cid:173)
`nection with FIG. 5.
`Alternatively, the image recording arrangement may be
`similar to those described in the Peleg II patent application.
`For several of the image recording arrangements that are
`described in the Peleg II patent application, the respective
`arrangements may also be rotated and/or translated to facili(cid:173)
`tate recording of a series of images from which strips may be
`mosaiced together to generate a set of panoramic images
`
`4
`comprising a stereoscopic panoramic image set. On the other
`hand, for others of the image recording arrangement
`described in the Peleg II patent application, the respective
`arrangements may be those including omni-cameras, which
`5 arrangements can directly record panoramic images for a
`stereoscopic panoramic image set without requiring mosaic(cid:173)
`ing.
`The communication arrangement facilitates transmission
`of the stereoscopic panoramic image set to the distribution
`10 channel 13 for distribution to one or more of the viewing
`devices 12m. Generally, information defining the stereo(cid:173)
`scopic panoramic image set transferred by the communica(cid:173)
`tion arrangement will be in digital form, and the communi(cid:173)
`cation arrangement may include any arrangement that
`15 facilitates transfer of digital data between two devices, which
`may be at the same location or at different locations. Illustra(cid:173)
`tive communication arrangements include, for example, a
`direct connection, such as a wire, cable or optical fiber con(cid:173)
`nection, a wireless connection, any other arrangement for
`20 facilitating the transfer of information in digital form, or any
`combination thereof. A direct connection may include, for
`example, a direct network connection, an indirect connection
`to a network through, for example, a computer, a connection
`through the public switched telephony network, and the like.
`25 A wireless connection may include, for example, a radio
`connection, a cellular telephone connection, an infrared con(cid:173)
`nection, and the like
`An illustrative viewing device will be described below in
`connection with FIG. 4. Generally, the viewing device will
`30 include an a communication arrangement, an image storage
`arrangement and a display, printer or other device for gener(cid:173)
`ating images for viewing by a viewer. The viewing device's
`communication arrangement facilitates reception of the ste(cid:173)
`reoscopic panoramic image set from the distribution channel
`35 13. Generally, information defining the stereoscopic pan(cid:173)
`oramic image set received by the communication arrange(cid:173)
`ment will be iri digital form, and the communication arrange(cid:173)
`ment may include a direct connection, such as a wire, cable or
`optical fiber connection, a wireless connection, such as a
`40 cellular telephone connection, or any other arrangement for
`facilitating the transfer of information in digital form.
`The image storage arrangement stores the digital informa(cid:173)
`tion comprising a stereoscopic panoramic image set as
`received by the viewing device's communication arrange-
`45 ment from the distribution channel 13 for later display.
`Depending on the amount of information that can be stored in
`the image storage arrangement and the amount of information
`comprising an stereoscopic panoramic image set, the image
`storage arrangement may have the capacity to store informa-
`50 tion comprising only one stereoscopic panoramic image set,
`or it may have the capacity to store information comprising a
`plurality of stereoscopic panoramic image sets.
`The display, if provided, will display at least a portion of
`one or more of the images comprising a stereoscopic pan-
`55 oramic image set. It will be appreciated that, if the display
`displays at least a portion of one of the images, the image,
`when viewed, will not be stereoscopic. On the other hand, if
`the display displays at least a portion ofboth images using, for
`example, a lenticular lens, as will be described below in
`60 connection with FIG. 6, the images, when viewed, will be
`stereoscopic. If a printer is provided, the printer can generate
`hardcopy prints of one or more of the images comprising a
`stereoscopic panoramic image set, and may generate, for
`example, lenticular prints, as will be described below in con-
`65 nection with FIG. 6, which can be viewed by a viewer using
`a lenticular lens to provide a stereoscopic image of the scene
`represented by the stereoscopic panoramic image set.
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000289
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 13 of 19 PageID #: 357
`
`US 7,477,284 B2
`
`5
`It will be appreciated that both each stereoscopic data
`source lln and viewing device 12m may also include control
`arrangements for facilitating control thereof by a respective
`operator. For example, if the stereoscopic data source lln
`includes an image recording arrangement that is to be rotated 5
`and/or translated, it can include a control to enable the image
`recording arrangement to rotate and/or translate. In addition,
`the stereoscopic data source 11n can include controls to allow
`an operator to actuate the image recording arrangement to
`facilitate recording of images of a scene, to enable generation 1 o
`by the image processing arrangement of a stereoscopic pan(cid:173)
`oramic image set for the scene, and to enable the communi(cid:173)
`cation arrangement to transmit an stereoscopic panoramic
`image set to the distribution channel13 for transfer to one or
`more of the viewing devices.
`Similarly, the respective viewing device 12m can include a
`control to facilitate receiving of stereoscopic panoramic
`image sets from the distribution channel13 and storage by the
`image storage arrangement. In addition, the respective view(cid:173)
`ing device 12m can include a control to facilitate selection of 20
`a stereoscopic panoramic image set for display, selection of
`display mode, and selection of a portion of the stereoscopic
`panoramic image set to be displayed. The viewing device 12m
`may be able to selectively display, as well as at least portions
`of a stereoscopic panoramic image set, at least portions of 25
`individual panoramic images of the stereoscopic panoramic
`image set as respective display modes, and the selection of the
`display mode can facilitate display of one or more of the
`panoramic images of the selected stereoscopic panoramic
`image set. In addition, the viewing device 12m can include a 30
`control to facilitate selection of a respective portion of an
`stereoscopic panoramic image set that is to be displayed, or
`the portion of an individual image from the stereoscopic
`panoramic image set that is to be displayed of the viewing
`device 12m provides such display modes.
`As noted above, a viewing device 12m can, instead of or in
`addition to providing a display for displaying an stereoscopic
`panoramic image set with a lenticular lens to provide for
`stereoscopic viewing, provide a printer or similar device for
`generating a hardcopy print of at least a portion of one or both 40
`of the images comprising a stereoscopic panoramic image set,
`or a hardcopy print of at least a portion of the stereoscopic
`panoramic image set that, when viewed through a lenticular
`lens, would provide a stereoscopic panorama. A viewing
`device 12m that provides a printer or similar device can 45
`include controls for enabling it to generate such hardcopy
`prints.
`With this background, an illustrative stereoscopic data
`source lln will be described in connection with FIGS. 2 and
`3. FIG. 2 depicts an exterior plan view of the illustrative 50
`stereoscopic data source lln, and FIG. 3 depicts a functional
`block diagram of the illustrative stereoscopic data source.
`Generally, the illustrative stereoscopic data source lln is a
`portable device that can be, for example, hand-held by an
`operator, mounted on a vehicle, or the like, and rotated and/or 55
`translated to facilitate recording of the series of images from
`which the stereoscopic panoramic image set is generated. In
`addition, the transfers the digital information comprising the
`stereoscopic panoramic image set over a cellular telephone
`communication link. With reference to FIG. 2, the illustrative 60
`stereoscopic data source lln includes a housing 20 that
`houses and supports a video camera 21, an operator control
`panel22, a display 23 and an antenna 24. The video camera 21
`provides the image recording arrangement described above,
`and the operator control panel 22 comprises a plurality of 65
`controls that may be actuated to enable the video camera 21 to
`perform predetermined operations. The controls can be
`
`6
`implemented in a number of ways, including, for example,
`push buttons that may be depressed by the operator to actuate
`the respective control.
`After the operator energizes the stereoscopic data source
`lln by actuating a respective control on the operator control
`panel 22, he or she can directs the video camera 21 in a
`particular direction. The display 23 can display an image
`indicating what the video camera 21 can record. The operator
`can enable the video camera by actuating a control on the
`operator control panel 22. As the operator actuates the image
`recording control, and rotates and/or translates the stereo-
`scopic data source lln, he or she can enable the video camera
`21 to record a series of images from which a stereoscopic
`panoramic image set can be generated. After the stereoscopic
`15 data source 11n has recorded a series of images, the operator
`can release the image recording control, at which point the
`video camera 21 can stop recording of images.
`After the stereoscopic data source lln has recorded a series
`of images, the operator can actuate another control on the
`operator control panel to enable the stereoscopic data source
`lln to generate the panoramic images comprising the stereo-
`scopic panoramic image set. After at least one of the pan(cid:173)
`oramic images has been generated, the operator can actuate
`another control on the operator control panel22 to enable the
`stereoscopic data source lln to display at least a portion the
`generated panoramic image in the display 23, with the portion
`being selectable by use of the same or another control on the
`operator control panel 22. Similarly, after at least two pan(cid:173)
`oramic images of the stereoscopic panoramic image set have
`been generated, the operator can actuate a control on the
`operator control panel 22 to enable the stereoscopic data
`source lln to display them in a manner so that they can be
`viewed stereoscopically. The stereoscopic data source lln
`may display the stereoscopic panoramic image set in a man-
`35 ner such that any a number of kinds of appliances may be
`required to facilitate stereoscopic viewing, including, for
`example, a lenticular lens, glasses with polarized lenses or
`lenses of different color, or other appliances as will be appre-
`ciated by those skilled in the art, with the stereoscopic data
`source lln displaying the stereoscopic panoramic image set
`in a corresponding manner.
`In addition, after the stereoscopic data source lln has
`generated the panoramic images comprising the stereoscopic
`panoramic image set, it can transmit them to the distribution
`channel for distribution to respective ones of the viewing
`devices 12m. As noted above, the illustrative stereoscopic
`data source lln makes use of a cellular telephone connection
`to transfer information to the distribution channel13. Accord(cid:173)
`ingly, the operator can actuate one or more other controls on
`the operator control panel 22 to enable the stereoscopic data
`source lln to establish a cellular telephone call to the distri-
`bution channel13 through a cellular provider (not shown). In
`that case, the stereoscopic data source lln can establish a
`cellular link through the antenna 24.
`FIG. 3 depicts a functional block diagram of the stereo(cid:173)
`scopic data source lln described above in connection with
`FIG. 2. With reference to FIG. 3, the stereoscopic data source
`lln includes an image capture unit 30, a local memory unit
`31, a processing unit 32, one or more local displays 33A,
`338, ... , and a communication unit 34, as well as the operator
`control panel 22. The image capture unit 30, local memory
`unit 31 and processing unit 3 2 together form the video camera
`21 described above in connection with FIG. 2, and the local
`displays 33A, 338, . .. form the display 23 described above in
`connection with FIG. 2. The capture unit 30 will include, for
`example, an image sensor, aperture, lenses, and/or the like to
`facilitate capturing or acquiring of the respective images. The
`
`Copy provided by USPTO from the PIRS Image Database on 02/13/2012
`
`HET00000290
`
`
`
`Case 1:12-cv-00398-GMS Document 27-2 Filed 02/21/13 Page 14 of 19 PageID #: 358
`
`US 7,477,284 B2
`
`7
`image sensor may be any of a number of conventional image
`sensors, including, for example, CCD (charge coupled
`devices), film, and the like.
`Generally, after the operator energizes the stereoscopic
`data source lln by actuating a respective c