Exhibit 1008
`Exhibit 1008
`
`

`
`111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
`US 20050078052Al
`
`(19) United States
`(12) Patent Application Publication
`Morichika
`
`(10) Pub. No.: US 2005/0078052 Al
`Apr. 14, 2005
`(43) Pub. Date:
`
`(54) DISPLAY IMAGE GENERATING DEVICE
`COMPRISING CONVERTING FUNCTION OF
`RESOLUTION
`
`(30)
`
`Foreign Application Priority Data
`
`Oct. 14, 2003
`
`(JP)
`
`2003-353621
`
`(75)
`
`Inventor: Kazumasa Morichika, Tokyo (JP)
`
`Correspondence Address:
`FRISHAUF, HOLTZ, GOODMAN & CHICK,
`PC
`767 THIRD AVENUE
`25TH FLOOR
`NEW YORK, NY 10017-2023 (US)
`
`(73)
`
`Assignee: CASIO COMPUTER CO., LTD.,
`Tokyo (JP)
`
`(21)
`
`Appl. No.:
`
`10/964,427
`
`(22)
`
`Filed:
`
`Oct. 13, 2004
`
`Publication Classification
`
`Int. Cl.7
`(51)
`(52) U.S. Cl.
`
`G06K l/00
`345/3.3
`
`(57)
`
`ABSTRACT
`
`A high resolution photographed image, picked up by a
`camera device, is projected by using a projector. In a case
`where there is a magnifying operation of a display image
`that is projected, image data of the display image that is
`reduced in accordance with the resolution at the projector
`side, is not magnified, but the image data of a display region
`corresponding to the size of a requested magnification ratio
`in the original photographed image, is reduced, to project a
`magnified image that is a display image based on the image
`data.
`
`2
`
`27
`
`26
`
`PROJECTION UNIT
`(INCLUDING OPTICAL
`SYSTEM)
`
`15
`
`12
`
`

`
`FIG. 1
`FIG. 1
`
`4c
`
`6
`~
`
`s
`
`
`
`
`
`5.85>E.:numo=_.=c=8.§.>3.E.38$8.HaEcm~....m33m..m~>H
`
`

`
`FIG. 2
`
`2
`,;
`27
`
`DISPLAY
`DEVICE
`
`PROJECTION UNIT
`(INCLUDING OPTICAL
`SYSTEM)
`
`28
`18
`,---/14
`RGB
`'I CABLE ~ IMAGE INPUT_
`UNIT
`21
`~
`,.--/
`100
`55
`I CONTROL!
`,-/
`MEANS
`53
`
`~9
`
`' - - -
`
`11
`~
`
`CPU
`
`1
`,;
`_,-..../ 15
`
`12
`~
`ROM
`)3
`RAM
`
`17
`I
`16
`~
`-A INFRARED
`SIGNAL
`~NPUT RECEIVING
`UNIT
`T
`
`26
`.~
`VRAM
`
`22
`~
`-
`RAM
`
`,_35
`
`VIDEO
`ADAPTER
`
`51
`,-/
`IMAGE INPUT
`MEANS
`
`CONVERTING
`CPU MEANS
`
`-./54
`
`CHANGING
`REQUEST
`MEANS
`
`~
`
`T
`E
`
`23
`
`-
`
`,-
`r--
`.......
`STORING
`DEVICE
`_..,.
`..........
`
`USB
`~ CABLE
`
`)
`r- rl
`29 200
`{~
`
`4
`~
`CAMERA
`DEVICE
`
`

`
`VIDEO
`ADAPTER
`
`FIG. 3
`
`j5
`
`21
`~
`~5
`I CONTROL I
`MEANS
`53
`
`26
`,--
`VRAM
`
`51
`~
`IMAGE INPUT
`MEANS
`CPU CONVERTING
`MEANS
`
`22
`,J
`
`RAM
`
`-
`
`___..54
`
`CHANGING
`REQUEST
`MEANS
`
`50
`,_;
`15
`
`12
`~
`ROM
`
`)3
`RAM
`
`'
`
`PROJECTION UNIT
`~NCLUDING OPTICAL
`YSTEM)
`)4
`IMAGE INPUT
`UNIT
`
`11
`~
`
`CPU
`
`-------- 2 0 1
`42
`/
`
`I
`
`34 L-
`
`nPUT
`EVICE
`
`23
`
`r-
`_..,.
`""'-
`:-...._
`STORING
`DEVICE
`.........
`
`.-'
`
`SIGNAL
`PROCESSING
`UNIT
`
`.. ceo
`
`)7
`I
`16
`41 ~ INFRARED
`SIGNAL
`~NPUT RECEIVING
`UNIT
`T
`
`

`
`Patent Application Publication Apr. 14, 2005 Sheet 4 of 11
`
`US 2005/0078052 Al
`
`FIG. 4
`
`41
`
`I
`
`ceo G2
`
`4
`
`(
`
`47
`
`46
`
`SIGNAL
`PROCESSING I
`UNIT
`
`I ROM
`
`I
`
`I USB· 1/F
`
`44 G
`
`KEY INPUT
`UNIT
`
`49
`
`~ RAM
`
`43
`
`48
`
`IMAGE
`MEMORY
`
`45
`
`

`
`Patent Application Publication Apr. 14, 2005 Sheet 5 of 11
`
`US 2005/0078052 Al
`
`FIG. 5
`
`r
`
`""
`START j
`
`PHOTOGRAPH DOCUMENT ----
`
`BY CAMERA DEVICE, AND
`SEND TO PC
`
`SA1
`
`DISTORTION AMENDMENT
`
`ROTATION
`.
`CUTTING
`
`SEND IMAGE DATA TO
`PROJECTOR
`
`-- - SA2
`r- _ . / SA3
`r- ~ SA4
`- r--'"" SA5
`
`1
`
`PROJECT TO SCREEN BY
`PROJECTOR
`
`- - - . / SA6
`
`I' END
`
`....._
`
`'
`
`~
`
`

`
`FIG. 6A
`FIG. 6A
`
`FIG. 68
`FIG. 6B
`
`G1
`~
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`0
`
`•
`
`0 • • • • •
`
`• • • • • • • • 0 • • 0 • • • • • • • •
`
`. . ~-----------------•....
`
`• •• ·1
`·
`·1
`·
`·I
`.•.. 1
`· ·I
`.·.·~
`• ·• ·1
`• ·1
`· ·I
`·.·.I
`· ·I
`.·.·1
`. •• ·1
`. - - . , . . ~.., - . - - . , . . .... - . - . - , . . . T"' " ' ' ' - - . - . •
`
`Ga
`
`0
`
`r. ·.
`1·
`·
`I·
`·
`1.·.·
`I· ·
`I· .
`r. ·.
`1·
`·
`I···.
`1.·.·
`I· ·
`(.· .
`r. ·.
`
`FIG. 6C
`Ga
`________________ ) __
`
`FIG. 60
`FIG. 6D
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`r---G2(Ga)
`G2(Ga)
`
`
`
`
`
`5.8:”>E._§:§._.=c_§:§.>3.3.33$8.aaEcm~....m§:m..m~>H
`
`
`

`
`Patent Application Publication Apr. 14, 2005 Sheet 7 of 11
`
`US 2005/0078052 Al
`
`FIG. 7
`
`START
`
`!
`
`OBTAIN SIZE (LENGTHWISE/CROSSWISE)
`OF IMAGE DATA
`
`!
`
`CALCULATE LENGTHWISE AND CROSSWISE
`RATIO OF IMAGE DATA
`
`~
`
`OBTAIN RESOLUTION (SIZE OF LENGTHWISE/
`CROSSWISE) OF PC MONITOR
`
`~
`
`CALCULATE LENGTHWISE AND CROSSWISE
`RATIO OF PC MONITOR
`
`~
`
`COMPARE LENGTHWISE AND CROSSWISE
`RATIO OF PC MONITOR WITH LENGTHWISE
`AND CROSSWISE RATIO OF IMAGE DATA,
`AND SPECIFY LONGER SIDE OF IMAGE
`DATA TOWARDS PC MONITOR
`
`~
`
`CALCULATE MAGNIFYING/REDUCING RATIO
`OF IMAGE CONCERNING SPECIFIED LONGER
`SIDE OF IMAGE DATA TOWARDS PC MONITOR,
`FROM LENGTH OF SAME SIDE AS PC MONITOR
`
`~
`
`MAGNIFY/REDUCE IMAGE DATA WITH
`CALCULATED RATIO
`~·
`MATCH CENTER POSITION OF IMAGE WITH
`CENTER POSITION OF PC MONITOR, DISPLAY
`IMAGE DATA TO PC MONITOR, AND OUTPUT
`TO PROJECTOR
`
`~
`
`END
`
`r
`..._
`
`'
`
`~
`
`881
`
`882
`
`883
`
`884
`
`885
`
`886
`
`887
`
`S88
`
`

`
`Patent Application Publication Apr. 14, 2005 Sheet 8 of 11
`
`US 2005/0078052 Al
`
`FIG. 8
`
`1 ..
`
`X1
`
`T IMAGE DATA (G2)
`1
`
`Y1
`
`.. I
`
`X2
`
`.. I
`
`PC MONITOR
`
`I
`
`T
`l
`
`Y2 I
`
`

`
`[IMAGE DATA]
`
`[PC MONITOR SCREEN]
`
`X1
`,--------------------------
`
`FIG. 9A Y1
`
`G2
`
`¢
`
`Y2
`
`X2
`
`G3
`
`--------------------
`
`X1
`--------------------------,
`
`FIG. 98 Y1
`
`G2
`
`¢v2
`
`-------------------------~
`
`FIG. 9C
`
`Y1
`
`X1
`
`G2
`
`¢v2
`
`X2
`
`G3
`
`X2
`
`G3
`
`J
`
`

`
`Patent Application Publication Apr. 14, 2005 Sheet 10 of 11
`
`US 2005/0078052 A1
`
`FIG. 10
`
`START ./
`
`OBTAIN MAGNIFICATION POSITION~OkDESGINATED BY
`OPERATION KEYS OR REMOTE CO T OL UNIT
`
`SPECIFY MAGNIFYING POSITION (P~ CORRESPONDING
`TO MAGNIFICATION POSITION, IN P OTOGRAPHED
`IMAGE
`
`OBTAIN IMAGE DATA OF REGION ~Q) IN ACCORDANCE WITH
`MAGNIFYING RATIO, SETTING SP CIFIED POSITION (P) AS
`CENTER
`
`MATCH IMAGE DATA WITH RESOLUTION OF IMAGE DATA
`
`DISPLAY MAGNFIED IMAGE BASED ON IMAGE DATA AFTER
`PROCESSING
`
`SC1
`
`SC2
`
`SC3
`
`SC4
`
`SC5
`
`END
`
`'
`
`~
`
`"""
`
`

`
`0
`
`-f G3
`
`I
`
`R 0
`(
`'
`+ G3
`-
`
`0
`0
`I
`
`1
`
`PHOTOGRAPHED
`IMAGE G2
`
`FIG. 11B
`
`FIG. 110
`
`FIG.11C
`
`+
`
`+
`
`G2
`
`-fG4
`
`-- ----------- _........_ ___ __,
`
`IVZS08L00/SO0ZSI]
`
`
`
`
`
`
`
`IIJ0II1991lSSO0Z‘i7I°JdVuoneonqndu0neo!1ddV1ua1ed
`
`

`
`US 2005/0078052 A1
`
`Apr. 14, 2005
`
`1
`
`DISPLAY IMAGE GENERATING DEVICE
`COMPRISING CONVERTING FUNCTION OF
`RESOLUTION
`
`BACKGROUND OF THE INVENTION
`
`pixel interpolation on an image of 640x480 pixels. Espe(cid:173)
`cially in a case where an arbitrary region of an image of
`document reference, etc, imaged by a document camera is
`magnified, because the image is a natural image, the image
`quality after magnification drops remarkably.
`
`[0001] 1. Field of the Invention
`
`SUMMARY OF THE INVENTION
`
`[0002] The present invention relates to a display image
`generating device having a converting function of image
`resolution.
`
`[0003] 2. Description of the Related Art
`
`In recent years, magnifying projection of images of
`[0004]
`documents (characters and charts, etc.) created by a personal
`computer (hereinafter referred to as PC) to a screen, using a
`projector, is carried out during a presentation, or a meeting,
`etc. The projector comprises a structure of converting an
`image signal input from the PC, etc., as an image signal, to
`a projection light by an image converting element such as a
`liquid crystal panel or micro mirror array, etc. There are
`projectors that comprise magnification display functions
`(hereinafter referred to as digital zoom function) that mag(cid:173)
`nify and project images sent from the PC, etc., by carrying
`out data processing.
`
`[0005] As a device for projecting for example, document
`references (newspapers, articles in magazines, or hand(cid:173)
`written manuscripts), etc., or a three dimensional reference
`or object, a document camera is used. The document camera
`has a structure of transferring the image data generally
`photographed (imaged) by a CCD or MOS type imaging
`element to the projector via the PC, or directly to the
`projector.
`
`In a case where the above described document
`[0006]
`camera is used, when an arbitrary region is requested to be
`projected by magnifying the region, a method of nearing the
`document camera to the document reference, etc., raising a
`zoom ratio if the document camera has an optical zoom, or
`a method of utilizing the above described magnification
`display function of the projector, is used. In that case of
`nearing the document camera to the document reference, an
`operation of matching the imaging region of the document
`camera with the to-be-displayed part of the document ref(cid:173)
`erence, etc. is carried out. This operation is burden for the
`user. Therefore, in most cases where a projector having a
`digital zoom function is used, the method using that function
`is adopted.
`
`[0007] However, in a case where the digital zoom function
`that the projector has is used, there are problems such as
`below. The resolution of the image that the projector can
`project, differs according to the model, but is generally
`SVGA(800x600 pixels) to SXGA(1,024xl,024 pixels), and
`for example, in a case where an image data is sent from a PC
`to a projector, a processing of matching the resolution of the
`image to the resolution of the image that the projector can
`project, is carried out in advance.
`
`[0008] Therefore, in a case where an arbitrary region of an
`image is projected magnified by the above described digital
`zoom function, the resolution of the magnified image
`reduces being proportional to the magnifying ratio. For
`example, in a case where the projector projects an image of
`SVGA and a magnification ratio of 1.25 times at SVGA, an
`image data of 800x600 pixels is generated by carrying out
`
`[0009] The present invention has been made in consider(cid:173)
`ation of the above problem.
`
`[0010] Accordingly, the display image generating device
`of the present invention comprises:
`
`image input means for inputting first image data
`[0011]
`representing a first image with a first resolution;
`
`image output means for outputting second image
`[0012]
`data representing a second image with a second resolution;
`
`[0013] converting means for sequentially converting a
`to-be-displayed part of the first image data which represents
`an image to-be-displayed in the first image data input by the
`image input means, to the second image data with the second
`resolution, and sequentially outputting the second image
`data generated by the conversion, by the image output
`means;
`
`inputting change
`[0014] changing request means for
`request for changing the to-be-displayed part of the first
`image data; and
`
`[0015] control means for specifying changed to-be-dis(cid:173)
`played part of the first image data in response to the change
`request input by the changing request means, and specifying
`a part of the first image data to be converted to the second
`image data by the converting means.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0016] These objects and other objects and advantages of
`the present invention will become more apparent upon
`reading of the following detailed description and the accom(cid:173)
`panying drawings in which:
`
`[0017] FIG. 1 is a system structure diagram showing a
`photographed image projection system according to the
`present invention;
`
`[0018] FIG. 2 is a block diagram showing an outline of an
`electric structure of mainly a projector and a PC in the
`photographed image projection system;
`
`[0019] FIG. 3 is a block diagram showing the structure of
`a projector that has a magnifying processing function of an
`image embedded thereto;
`
`[0020] FIG. 4 is a block diagram showing an electric
`structure of a camera device;
`
`[0021] FIG. 5 is a flowchart showing basic operation of
`the photographed image projection system;
`
`[0022] FIGS. 6A to 6D are diagrams describing the con(cid:173)
`tent of cutting processing by the PC;
`
`[0023] FIG. 7 is a flowchart showing operation content
`concerning display and output of an image, by the PC;
`
`[0024] FIG. 8 is a diagram showing the lengthwise and
`crosswise ratio of a photographed image after the cutting
`processing, and the lengthwise and crosswise ratio of a PC
`monitor;
`
`

`
`US 2005/0078052 Al
`
`Apr. 14, 2005
`
`2
`
`[0025] FIGS. 9A to 9C are diagrams showing the relation
`of photographed image after the cutting processing and the
`display screen of the PC monitor and the projection image;
`
`[0026] FIG. 10 is a flowchart showing movement content
`concerning magnifying processing of an image by the PC;
`and
`
`[0027] FIGS. 11A to 11C are diagrams describing the
`content of the magnifying processing.
`
`DETAILED DESCRIPTION OF 1HE
`PREFERRED EMBODIMENT
`
`[0028] A picked-up photographed) image projection sys(cid:173)
`tem according to an embodiment of the present invention
`comprises a projector 1, a laptop personal computer (here(cid:173)
`inafter referred to as PC) 2, and a camera device 4. The
`projector 1 and the PC 2 are connected with each other via
`a RGB cable 100 and a USB cable 200. The PC2 and the
`camera device 4 are connected with each other via another
`USB cable 201. Each device is connected to the cables via
`a connector, in accordance with need. In the present embodi(cid:173)
`ment, the PC 2 serves as a display image generating device
`of the present invention.
`
`[0029] The projector 1 is provided at the front end portion
`of the main body. The projector 1 has an optical system such
`as a projection lens, etc., and projects a light corresponding
`to an image such as a document, etc., sent from the PC 2, to
`a screen S (or a white board, etc.) from the projection unit
`1a. The projector 1 has a remote control unit 5. The remote
`control unit 5 send commands and data to the projector 1
`through the infrared light. Operation of the projector 1 from
`a distant place, becomes possible by the infrared remote
`control unit 5.
`
`[0030] The camera device (document camera) 4 com(cid:173)
`prises: a camera base 4a; a strut 4b provided on the camera
`base 4a; and a digital camera 4c that is attached to an
`attaching unit which is provided at an upper end portion of
`the strut 4b and supports the camera 4c so that it can rotate
`up and down and right to left. The digital camera 4c has a
`general specification including AE (Auto Exposure) func(cid:173)
`tion, etc., and a predetermined photographing (imaging)
`mode that includes PC camera function for outputting image
`data obtained by photographing (picking-up image; or imag(cid:173)
`ing).
`
`[0031] Arbitral document material Asuch as articles of
`newspapers or magazines, presentation documents, and
`document manuscripts, etc., or various three-dimension
`materials are placed on the camera base 4a.
`
`[0032] The camera 4c imaging (photographing) the mate(cid:173)
`rial A, and sends (outputs) the photographed (picked-up)
`image (more specifically, image data representing picked-up
`image) to the projector 1 via the PC 2, and the projector 1
`projects, the image of the document A, etc., defined by the
`supplied image data, on a screen S.
`
`[0033]
`
`(Structure of the Projector)
`
`image input unit 14, a display unit 15, a key input unit 16,
`and an infrared signal receiving unit 17.
`
`[0035] An input terminal18 of an REG signals that has the
`RGB cable 100 connected thereto, is connected to the image
`input unit 14. The image input unit 14 receives the RGB
`signals and generates a display signal for displaying and
`projecting an image based on the RGB signals (image input
`signals) and supplies the display signal to the display unit
`15. The display unit 15 includes a light source such as a
`krypton lamp, etc.; image conversion elements (serve as
`projection means of the present invention), such as a liquid
`crystal panel and micro mirror array, etc., for converting
`light from the light source to a projection light correspond(cid:173)
`ing to the display signal; a drive circuit for driving the image
`conversion elements, based on the display signal sent from
`the image input unit 14; and an optical system such as a
`projection lens, etc., provided at the projection unit 1a, and
`projects a display image based on the image data sent from
`the PC 2 to the screen S.
`
`[0036] The CPU 11 operates the RAM 13 as a work area,
`in accordance with a program stored in the ROM 12, to
`control the entire projector 1. The key input unit 16 com(cid:173)
`prises a plurality of operation (operable) keys prepared for
`the user to operate the projector 1. The infrared signal
`receiving unit 17 receives infrared light sent from the remote
`control unit 5, decodes the operation signal that includes
`coded information, and sends the decoded signal to the CPU
`11. A USB interface circuit (not shown) is connected to the
`CPU 11. A USB terminal 19 that has the USB cable 200
`connected thereto, is provided on the USB interface circuit.
`In the present embodiment, operation signals sent from the
`operation keys of the key input unit 16, and the infrared
`remote control unit 5, are sent to the PC 2, in accordance
`with need.
`
`[0037]
`
`(Structure of PC)
`
`[0038] The PC 2 comprises a CPU 21, a RAM 22, a
`storing device 23, an input device 24, a video adapter 25, a
`VRAM 26, and a display device 27. The PC 2 comprises two
`USB ports. A first USB connection terminal29, and a second
`USB connection terminal 30, are provided at a not shown
`USB Interface circuit connected to the USB ports of the
`CPU 21. The USB cable 200 is connected to the first USB
`connection terminal 29, and another USB cable 201 is
`connected to the second USB connection terminal 30.
`
`[0039] The video adapter 25 generates a video signal
`(RGB signals) for display, and outputs the video signal to the
`display device 27 that comprises an LCD panel and a drive
`circuit, etc., thereof. The VRAM 26 continually stores the
`image data for display, that the video adapter 25 generates.
`An ROB output terminal 28 that has the RGB cable 100
`connected thereto, is provided at the video adapter 25. The
`video adapter 25 has a function for outputting image data for
`displaying an image to be projected on the screen S, which
`is generated other than the image data that is output to the
`display device 27 in accordance with instructions from the
`PC 21, to the projector 1, via the RGB cable 100.
`
`[0034] FIG. 2 is a block diagram mainly showing an
`electronic structure of the projector 1 and the PC 2 in the
`above image projection system. The projector 1 comprises a
`CPU (Central Processing Unit) 11, a ROM (Read Only
`Memory) 12, a RAM (Random Access Memory) 13, an
`
`[0040] The input device 24 includes a plurality of keys
`provided at the main body of the PC 2, and a mouse, etc.,
`connected to the main body, and the storing device 23 is a
`hard disk, etc., with a relatively large storing capacity. The
`storing devices stores a predetermined presentation program
`
`

`
`US 2005/0078052 Al
`
`Apr. 14, 2005
`
`3
`
`for generating and editing documents for presentations, an
`image processing program for carrying out later-described
`various image processing to the to-be-projected images, and
`image display program for controlling the video adapter 25
`to generating the image data for displaying the above
`documents and images, etc., to the display device 27, and the
`image data to be output to the projector 1, etc.
`
`In the PC 2, by the CPU 21 operating, based on the
`[0041]
`above image display program, the CPU 21 functions as the
`image input means, image output means, converting means,
`change request means and control means of the present
`invention.
`
`[0042]
`(Structure of Digital Camera)
`[0043] FIG. 4 is a block diagram showing a structure of
`the digital camera 4c that serves as the camera device.
`
`[0044] The digital camera 4c comprises a CCD 41 and a
`signal processing unit 42. The CCD 41 serves as the imaging
`means of the present invention and photographs (images) the
`document reference A, etc., via an un-shown lens. The signal
`processing unit 42 carries out various signal processing to an
`analog image signal output from the CCD 41 and outputs a
`digital image signal representing a processed image. In the
`present embodiment, the CCD 41 has a high resolution, for
`example, it has four million effective pixels.
`
`[0045] The image signal output from the signal processing
`unit 42 is sent to an LCD 43 at a photographing wait state
`(stand-by state), to be displayed as the photographed
`(picked-up; imaged) subject image. At an ordinary photo(cid:173)
`graphing mode, the image photographed by the CCD 41 in
`accordance with a photographing operation is compressed
`by a CPU 44, following a predetermined format such as
`JPEG, etc., and stored to an image memory 45. On the other
`hand, at the predetermined photographing mode, data of the
`image photographed by the CCD 41 is output to the PC 2
`from a USB interface 46, via the USB cable 5.
`
`[0046] The compressed image data that is stored to the
`image memory 45, is read by the CPU 44, in accordance
`with need, and is re-displayed as a still image in the LCD 43,
`after being decompressed. The image memory 45 is a
`non-volatile memory such as a flash memory, etc., that is
`embedded or can be attached to/detached from the digital
`camera 4c. A ROM 47 that stores various control programs
`necessary for the CPU 44 to compress/decompress the above
`image data, and to control the entire device, a RAM 48 that
`is an operation memory of the CPU 44, and a key input unit
`49 that comprises a plurality of operation keys such as a
`shutter key, and a mode switching key, etc., are provided to
`the digital camera 4c.
`
`(Description of Operation of the Photographed
`[0047]
`Image Projection System)
`
`[0048] An operation of projecting a document reference A,
`such as shown in FIG. 1 to a screen S, in the photographed
`image projection system structured as above, will be
`described.
`
`or modifies the photographed image by performing various
`image processing, such as distortion amendment processing,
`rotating processing, and cutting (extracting) processing, etc.,
`on the input image data, in accordance with the image
`processing program. The image data after processed is
`output to the projector 1, by a display program (steps SA2
`to SA5). Then, the projector 1 projects an image based on the
`image data input from the PC 2, to the screen S (step SA6).
`
`[0050] The above image processing by the PC 2 is carried
`out concerning a part or the entire image, in accordance with
`need. In the below description, it is assumed that in the
`above cutting processing, from an image Gl sent from the
`camera device (refer to FIG. 6A), for example, a designated
`(specified) area Ga excluding the unnecessary circumfer(cid:173)
`ence part designated (specified) by the user, is cut (extracted)
`(as shown in FIGS. 6A and 6B), to output an image data of
`an image G2 comprising only the designated region Ga, to
`the projector 1.
`
`[0051] FIG. 7 is a flowchart showing operation of the PC2
`for displaying and outputting the image during the above
`operation. In the below, it is assumed that the resolution set
`at the display device 27 (hereinafter referred to as PC
`monitor) and the resolution of the projector 1 are the same.
`
`[0052] When displaying the image G2 that is ultimately
`obtained in the above image processing (hereinafter referred
`to as photographed image), first, the PC 2 obtains the
`crosswise size (pixel number) and lengthwise size (pixel
`number) of the image data (SBl), and calculates the ration
`of the crosswise size to the lengthwise size (step SB2).
`Further, the PC 2 obtains the resolution set in the PC monitor
`(step SB3), and calculates the ration of the crosswise size to
`lengthwise size thereof (step SB4). FIG. 8 is a diagram
`showing examples of the lengthwise and crosswise of the
`image data (photographed image) and display of PC 2. In
`FIG. 8, the ration of crosswise size to lengthwise size of
`image data of image G2 is Xl/Yl, and that of the PC
`monitor is X2/Y2. The ordinary ratio of crosswise size to
`lengthwise size of the PC monitor is%, and if the resolution
`is XGA, the lengthwise and crosswise size is 768xl,024
`pixels (it is the same for the projector 1 ).
`
`[0053] The ratio of crosswise size to lengthwise size of the
`PC monitor and the ratio of crosswise to lengthwise of the
`image data are compared, and the longer side of the image
`data (image G2) with respect to the PC monitor is specified
`(step SB5). FIGS. 9A to 9C are diagrams showing the
`patterns of the specified result, The specified result becomes
`one of the below three results.
`
`[0054] FIG. 9 A ... (Xl/Yl)>(X2/Y2), and the crosswise
`side (Xl) is longer.
`
`[0055] FIG. 9B ... (X/Yl)=(X2/Y2), and the ratio of
`crosswise size to lengthwise size is the same as that of the
`PC monitor.
`
`[0056] FIG. 9C ... (Xl/Yl)<(X2/Y2), and the lengthwise
`side (Yl) is longer.
`
`[0049] FIG. 5 is a flowchart showing a basic operation in
`the present system. By the predetermined mode being set,
`the camera device 4 (concretely, the digital camera 4c)
`photographs (picks-up image, images) an arbitrary docu(cid:173)
`ment reference A, etc., and outputs the image data of the
`imaged document to the PC 2 (step SAl). The PC 2 amends
`
`[0057] Next, concerning the specified longer side, from
`the ratio of the length of the same side of the PC monitor to
`that of the image G2, the magnification for enlarging or
`reducing the photographed image G2 is calculated (step
`SB6). With the calculated magnification, namely, (X2/Xl)
`times in FIG. 9A, and (Y2/Yl) times in FIGS. 9A and 9B,
`
`

`
`US 2005/0078052 Al
`
`Apr. 14, 2005
`
`4
`
`the data of the photographed image G2 is reduced (step
`SB7). Namely, the resolution of the photographed image G2
`is reduced. The reduced image (display image) G3 is dis(cid:173)
`played to the PC monitor, so that the center of the reduced
`image matches the center display screen of the PC monitor
`27, and the image data of the above display image G3 is
`output to the projector 1 (step SB8). By this, the display
`image G3 displayed at the PC monitor 27, is displayed on the
`screen S. In this case, the pixel number of the images
`displayed on the PC monitor 27 and the pixel number of the
`image projected on the screen S are same.
`
`[0058] On the other hand, FIG. 10 is a flowchart showing
`operation for magnifying an image that the PC 2 executes,
`when there is an instruction to enlarge the display image G3
`from the user, by operation of operation keys or the infrared
`remote control unit 5, while the display image G3 is pro(cid:173)
`jected by the projector 1. Here, descriptions will be made
`assuming that the size of the photographed image G2 is
`1,500x2,00 pixels (lengthwise (vertical; y) to crosswise
`(horizontal; x) ratio of 3:4), and the magnification rate is
`1.25 times, in the x axis direction and the y axis direction
`respectively.
`
`In accordance with the magnification instruction
`[0059]
`from the operation keys or the infrared remote control unit
`5, the PC 2 displays a cross shape point (cross cursor) that
`can be operated to be moved by the user using the operation
`keys or the infrared remote control unit 5, to the display
`image G3, by an OSD (On Screen Display) function, etc.,
`and obtains a magnification position (center position of
`to-be-magnified area (region)) 0 indicated by the user by the
`cross shape point (step SC1: refer to FIG. 11A).
`
`[0060] Next, the PC2 specifies the magnifying position P
`that corresponds to the magnifying position 0 of the display
`image G3 position homothetic with magnifying position 0)
`in the photographed image G2 (step SC2) as shown in FIG.
`llB. The positions 0 and P represent same points in the
`respective images. Then, the PC 2 sets the magnifying
`position P in the photographed image G2 as the center of
`to-be-magnified area, and specifies the display region Q
`which has sizes in accordance with the magnification rate
`designated by the user as a display region and center position
`corresponding to the magnifying position P. The PC 2
`obtains the image data of the display region Q (step SC3:
`refer to FIG. llB).
`
`[0061] Here, because the magnification rate is 1.25 times,
`the lengthwise and crosswise size of the display region Q
`(Y3xX3) is a size (1,200x1,600 pixels) that reduced the
`lengthwise and crosswise size of the photographed image
`G2 (1,500x2,000 pixels), by 80 (lf1.2sxlOO) percent. The
`display region Q is a region that is homo the tic (analogous)
`with a region that is to be magnified in accordance with the
`above magnification ratio, setting the magnifying position 0
`in the display image G3 designated by the cross-shape point,
`as the center.
`
`screen S by the projector 1 (step SC5). By this, magnifying
`of the display image G3 is completed.
`
`[0063] After the magnifying processing is completed, in a
`case where there is an instruction for scrolling (or moving)
`the displayed image, by the user from the operation keys or
`the infrared remote control unit 5, the PC 2 performs a
`scrolling operation. In the scrolling operation, the PC 2
`moves the position of the display region Q in the photo(cid:173)
`graphed image G2 (or re-obtain the display region Q in the
`photographed image G2) (refer to FIG. llB), in accordance
`with the instructed direction and moving amount and carries
`out the above described steps SC3 to SC5, to update the
`magnified image G4. Thereafter, the PC 2 repeats the above
`processing each time there is an instruction for scrolling (or
`moving). For example, if the instruction is instruction for
`continuous scrolling, the above operation is repeated at a
`predetermined interval corresponding, for example. Further,
`in a case where there is a reducing instruction for reducing
`image size by the user, from the operation keys or the
`infrared remote control unit 5, the PC 2 updates the display
`image G3 by carrying out a reducing processing of setting
`the region having a size which corresponds to the reducing
`rate designated by the user in the photographed image G2,
`as the display region Q (refer to FIG. llB), in the above
`described step SC3.
`
`[0064] As the above, in the photographed image projec(cid:173)
`tion system of the present embodiment, in a case where the
`resolution (the number of pixels of the image obtained by
`photographing or imaging) of the camera device 4 (digital
`camera 4c) exceeds the resolution of the projector 1 (the
`number of pixels that the projector 1 can projects and
`display), when an arbitrary area of the projection image is
`magnified while the photographed image is projected by the
`projector 1, if the magnifying rate is within a predetermined
`range, the magnified image that is to be projected is prepared
`or generated by carrying out reducing processing of the data
`of the photographed image G2. Therefore, degrading of
`image quality of the projection after magnification does not
`occur, and the same image quality as the image before
`magnifying can be ensured.
`
`[0065] This kind of effect is not limited to a case where an
`image photographed by the camera device 4 is projected
`real-time, and for example, when an image stored in the
`storing device 23 of the PC 2 is projected, the same kind of
`effect can also be obtained in a case where the number of
`pixels of the stored image data, exceeds the number of pixels
`that the projector 1 can display.
`
`[0066] The function of the above projector 1, the image
`magnifying processing function in the above described PC
`2, and the function of the camera device can be embedded
`in one device. FIG. 3 is a block diagram showing the outline
`structure in this case. The operation of each part is substan(cid:173)
`tially same as the operation in the above described photo(cid:173)
`graphed image projection system.
`
`[0062] The PC 2 performs the processing of matching the
`resolution of the image data of the display region Q with the
`resolution of the PC monitor 27, namely, the projector 1 (in
`the above example, reducing processing of 64 (768/1,200x
`100 and 1,024/1,600x100) percent) (step SC4: refer to FIG.
`llC). Then, the PC 2 displays the image based on the
`processed image data, namely, displays the magnified image
`G4 to the PC monitor 27, and projects the image to the
`
`[0067] The magnifying processing of an image by the PC
`2 may be carried out at the projector 1, if the CPU 11 has
`enough power or ability compared to those the projector 1
`requires to project images. Or, output processing of the
`image to the projector 1 and magnifying processing of the
`image by performed by the PC2, can be carried out by the
`camera device 4, if the CPU 44 of the digital camera 4c has
`enough processing ability. In that case, the PC 2 may be
`
`

`
`US 2005/0078052 Al
`
`Apr. 14, 2