(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`(19) World Intellectual Property
`Organization
`International Bureau
`
`1111111111111111 IIIIII IIIII 11111111111111111111 lllll lllll 11111111111111111111111111111111111111
`
`( 43) International Publication Date
`5 August 2004 (05.08.2004)
`
`PCT
`
`(10) International Publication Number
`WO 2004/066615 Al
`
`H04N 5/225,
`(51) International Patent Classification 7:
`7/14, H04M 1/725, H04N 5/232, H04M 1/247
`
`WALSTIJN, Bartholomeus, Gerardus, Gosewinus
`[NL/DK]; Parkovsvej 3, DK-2820 Gentofte (DK).
`
`(21) International Application Number:
`PCT/EP2003/000607
`
`(22) International Filing Date: 22 January 2003 (22.01.2003)
`
`(74) Agent: VAN WALSTI.JN, B., Gerard, G.; Walstijn In(cid:173)
`tellectual Property ApS, Parkovsvtj 3, DK-2820 Gentofte
`(DK).
`
`(81) Designated States (national): AE, AG. AL, AM, AT (util(cid:173)
`ity model), AT, AU, AZ, BA, BB, BG, BR, BY, BZ, CA,
`CH, CN, CO, CR, CU, CZ (utility model), CZ, DE (util(cid:173)
`ity model), DE, DK (utility model), DK, DM, DZ, EC, EE
`(utility model), EE, ES, FI (utility model), FI, GB, GD, GE,
`GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ,
`LC LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN,
`MW, MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU, SC,
`SD, SE, SG, SK (utility model), SK, SL, TJ, TM, TN, TR,
`TT, TZ, UA, UG, US, UZ, VC, VN, YU ZA, ZM, ZW.
`
`(84) Designated States (regional): ARIPO patent (GH, GM,
`KE, LS, l\lIW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW),
`
`[Continued on next page]
`
`(57) Abstract: The present invention relates to the
`field of hand-held devices that are equipped wit a
`processor and a digital camera for capturing motion
`video or still images. Images captured by the cam(cid:173)
`era are used to determine motion of the hand-held
`device. A resulting motion signal is used as input to
`a user interface. Displayed images can be scrolled,
`zoomed or rotated by moving the hand-held device.
`The motion signal can also be used as input for a
`graphical user interface to move a cursor or other
`object of the graphical user interface over the dis(cid:173)
`play. The invention relates further to a hand-held
`device provided with means for sensing motion, a
`display, a keypad with at least a first- and a second
`key, and a graphical user interface with objects and a
`cursor. The hand-held device also comprises means
`for transforming the sensed motion of the handheld
`device into a signal suitable for moving the cursor
`over the display.
`
`(25) Filing Language:
`
`(26) Publication Language:
`
`English
`
`English
`
`(71) Applicant (for all designated States except US): NOKIA
`CORPORATION [FI/Fl]; Keilalahdentie 4, FIN-02150
`Espoo (FI).
`
`(72) Inventors; and
`(75) Inventors/Applicants (for US only): SYRBE, Hanno
`[DE/DE]; Buchenweg 23, 89197 Weidenstetten (DE).
`KRISCHEU, Klaus [DE/DE]; Wiesenweg 9, 89160
`Tomerdingen (DE). KOIVUNEN, Rami, Arto [FI/FI];
`Munstenpellonkatu 7 B 11, FIN-20740 Turku (FI). VAN
`
`--
`= (54) Title: IMAGE CONTROL
`
`\
`
`\ V \_
`
`-iiiiiiiiiiiiiii -iiiiiiiiiiiiiii -iiiiiiiiiiiiiii --
`
`-iiiiiiiiiiiiiii
`iiiiiiiiiiiiiii --
`'l"""""i <
`an
`'l"""""i
`\0
`\0
`\0
`
`0 --~
`
`0
`0
`M
`0
`~
`
`0338
`
`Exhibit 1002 page 338 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615 A 1
`
`1111111111111111 IIIIII IIIII 11111 IIIII IIII I II llllllll 1111111111111111111111111111111 IIII IIII IIII
`
`Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM),
`European patent (AT, BE, BG, CH, CY, CZ, DE, DK, EE,
`ES, Fl, FR, GB, GR, HU, IE, IT, LU, MC, NL, PT, SE, SI,
`SK, TR), OAPI patent (BF, BJ, CF, CG, CI, CM, GA, GN,
`GQ, GW, ML, MR, NE, SN, TD, TG).
`
`Published:
`-
`with international search report
`
`For two-letter codes and other abbreviations, refer to the "Guid(cid:173)
`ance Notes on Codes and Abbreviations" appearing at the begin(cid:173)
`ning of each regular issue of the PCT Gazette.
`
`0339
`
`Exhibit 1002 page 339 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`IMAGE CONTROL
`
`5
`
`The present invention relates to the field of hand-held
`devices that are equipped with a processor and a digital
`camera for capturing motion video or still images,
`in
`particular such devices that further comprise a display
`for displaying images or a graphical or character based
`user interface.
`
`BACKGROUND ART
`
`Hand-held devices provided with digital image capturing
`equipment, digital processing power and high resolution
`displays are becoming increasingly more common in a wide
`variety of uses.
`
`For example,
`small mobile phones are recently being
`equipped with digital cameras and relatively small high
`resolution LCD
`screens. Hand-held computers
`commonly
`called "personal digital assistants"
`(PDA)
`are also
`available and are
`typically equipped with small high
`resolution display screens and have slots for receiving
`e.g.
`a digital camera. Similarly, communicators having
`both cellular communication and computer capabilities are
`available, typically having small display screens and an
`inbuilt or detachable digital camera. These small, hand(cid:173)
`held devices do not,
`and cannot,
`conveniently have
`conventional input devices, such as a computer mouse and
`other control keys. Therefore,
`conventional personal
`computer interfaces, which also have their own drawbacks,
`are not suited for these small hand-held devices.
`
`As a result, there are significant limitations on using
`such small hand-held devices in both obtaining output,
`e.g. viewing data on the display screen, and in inputting
`
`15
`
`20
`
`25
`
`30
`
`35
`
`CONFIRMATION COPY
`
`0340
`
`Exhibit 1002 page 340 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`2
`
`commands, e.g. changing the area viewed on the display
`screen or controlling the performance of a particular
`parameter associated with the device. Further, given the
`limited area available, not only on the display screen but
`also on the entire device, adding additional control keys,
`etc., is both difficult and burdensome to a user requiring
`two hand operation of the device.
`
`US
`6466198 discloses
`a
`system and method
`for view
`navigation and magnification of the display of hand-held
`devices in response to the orientation changes along only
`two axes of rotation as measured by sensors inside the
`devices. The view navigation system can be engaged and
`controlled by simultaneously pressing switches on both
`sides of
`the hand-held device. Miniature sensors
`like
`accelerometers,
`tilt
`sensors,
`or magneto-resistive
`direction sensors sense
`the orientation changes. These
`miniature sensors are presently not typically standard
`equipment for hand-held devices. Thus, such sensors add
`cost, use precious space and add weight.
`
`The present invention. is directed toward overcoming one or
`more of the above-identified problems.
`
`5
`
`10
`
`15
`
`20
`
`25 DISCLOSURE OF THE INVENTION
`
`On
`this background,
`it is an object of
`the present
`invention to provide a hand-held device of
`the kind
`referred to initially, which allows user input with the
`same hand that holds the device, without requiring the
`dedicated sensatory equipment used by prior art hand-held
`devices.
`
`30
`
`This object is achieved in accordance with claim l, by
`35 providing a hand-held device comprising a processor,
`a
`digital camera for capturing motion video or still images,
`
`0341
`
`Exhibit 1002 page 341 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`3
`
`and means for transforming a signal from the camera into a
`motion signal indicative of the motion of the hand-held
`device.
`
`5 Thus, by using a sensor that is available to start with in
`many hand-held devices -- namely a digital camera -- for a
`secondary use, namely creating a motion signal indicative
`of the motion of the hand-held device, a hand-held device
`with motion sensing is provided
`in a
`economical and
`reliable manner.
`
`10
`
`15
`
`20
`
`The hand-held device may further comprise a user interface
`in which motion of the hand-held device is -
`through the
`motion signal derived thereof - used as a user input.
`
`The hand-held may further comprise a display, preferably a
`display suitable for displaying captured images.
`
`Motion of a given type of the hand-held device can be used
`to manipulate
`images
`shown at least
`in part on
`the ,,
`display, preferably by moving
`the
`images
`in a manner
`substantially corresponding to the movement of the hand(cid:173)
`held device.
`
`25 Different types of motion the hand-held device can e.g. be
`used to move, and/or zoom, and/or expand/collapse and/or
`rotate images displayed on the display.
`
`Motion substantially parallel to the plane of the display
`of the hand-held device can be used to scroll an image
`displayed
`on
`the
`display. Motion
`substantially
`perpendicular to the plane of the display can be used to
`zoom an image displayed on the display. Rotational motion
`of the hand-held device can be used to rotate an image
`displayed on the display.
`
`30
`
`35
`
`0342
`
`Exhibit 1002 page 342 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`4
`
`The images that are manipulated can e.g. be images that
`were previously captured by the camera.
`
`The movement of
`image can be inverted with respect to
`5 motion of
`the hand-held device,
`since some users may
`prefer this.
`
`10
`
`15
`
`The user interface may comprise a graphical user interface
`and motion of the hand-held device can be used as an input
`to the graphical user interface.
`
`Motion of the hand-held device can be used to manipulate
`an object displayed by
`the graphical user
`interface,
`preferably by moving the object in a manner substantially
`corresponding to the mot~on or to the inverted motion of
`the hand-held device, whereby the object displayed by the
`graphical user interface can be an icon, a dialogue box, a
`window, a menu or a pointer.
`
`20 Motion of a given type of the hand-held device can be used
`to move, and/or zoom, and/or expand/collapse and/or rotate
`objects displayed by the graphical user interface.
`
`Motion substantially parallel to the plane of the display
`of the hand-held device can be used to scroll an object
`displayed by
`the graphical user
`interface. Motion
`substantially perpendicular to the plane of the display
`can be used to zoom an object displayed by the graphical
`user interface. Rotational motion of the hand-held device
`can be used to rotate an object displayed by the graphical
`user interface.
`
`The digital camera can be an inbui 1 t camera or can be a
`detachable camera. The camera may be movable relative to
`the hand-held device.
`
`25
`
`30
`
`35
`
`0343
`
`Exhibit 1002 page 343 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`5
`
`The means for transforming a signal from the camera into a
`motion signal preferably derives the motion signal from
`changes between succeeding images captured by the camera.
`
`5 The camera can be equipped with an autofocus system,
`whereby the focusing setting of the autofocus system can
`be used for detecting movement in the camera direction.
`
`The
`hand-held device may further comprise at least one
`key.
`The functionality of a motion type can be dependent
`on the state of the at least one key.
`
`Rotational motion of the hand-held device about an axis
`substantially perpendicular to the display may be used to
`cause an
`inverse rotational movement of
`the
`image or
`graphical user interface object relative to the display,
`preferably in a manner such that the image or object is
`static with respeGt to the fixed coordinate system in
`which the hand-held device is situated.
`
`The motion signal can be used to adjust device settings,
`such as
`sound settings, keypad settings and display
`settings.
`
`The hand-held device may further comprise a keypad with at
`least a first and a second key and the graphical user
`interface comprising a cursor, whereby motion of the hand(cid:173)
`held device can be used to position the cursor over an
`object of
`the graphical user
`interface and primary
`functions associated with
`the object concerned can be
`activated by pressing
`the
`first key and
`secondary
`functions associated with the object of the concerned can
`be activated by pressing the second key.
`
`15
`
`20
`
`25
`
`30
`
`35
`
`The functionality of the first key can be associated with
`selection and activation of objects of the graphical user
`
`0344
`
`Exhibit 1002 page 344 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`6
`
`interface, and the functionality of the second key can be
`associated with calling up a context-sensitive menu.
`
`The selection of the object concerned can be performed by
`5 pressing and releasing the first key. Activation of the
`object concerned
`can be performed by pressing and
`releasing the first key twice in rapid succession. Moving
`or resizing of the object concerned can be performed by
`holding down
`the first key while moving the hand-held
`device to move the cursor.
`
`10
`
`The first key and the second key can be softkeys, whereby
`the current functionality of the softkeys is shown in the
`display, preferably in dedicated fields of the display.
`
`The first key can be placed below the display on the left
`side of the latter, preferably proximate to lower edge of
`the display, and the second key can be placed below the
`display on
`the right side of
`the
`latter, preferably
`proximate to lower edge of the display.
`
`It is another object of the present invention to provide
`an
`improved method for proving user input to hand-held
`devices. This object is achieved by providing a method for
`creating user input for a hand-held device that has a
`processor,
`a user interface and a digital camera for
`capturing motion video or still images comprising
`the
`steps of:
`determining motion of the hand-held device from
`the camera signal;
`using the determined motion of the hand-held device
`as an input for the user interface.
`
`15
`
`20
`
`25
`
`30
`
`It is yet another object of
`the present
`invention to
`35 provide a use of a digital camera of a hand-held device
`
`0345
`
`Exhibit 1002 page 345 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`7
`
`that has a processor to produce a motion signal indicative
`of motion of the hand-held device.
`
`A further object of the invention is to provide a hand-
`s held device with an
`improved graphical user interface.
`This object is achieved by providing a hand-held device
`comprising a processor, means for sensing motion of the
`hand-held device, a display, a keypad with at least a
`first- and a second key, a graphical user interface with
`10 objects and a cursor, and means for
`transforming
`the
`sensed motion of
`the handheld device
`into a
`signal
`suitable for moving the cursor over the display.
`
`15
`
`By controlling the position of a cursor through motion of
`the handheld device it becomes possible to provide a user(cid:173)
`friendly cursor controlled graphical user interface for
`hand-held devices.
`
`Preferably, motion of the hand-held device is used to
`20 position the cursor over objects of the graphical user
`interface and primary functions associated with the object
`concerned are activated by pressing the first key and
`secondary functions associated with the object concerned
`are activated by pressing the second key.
`
`25
`
`The functionality of the first key can be associated with
`selection and activation of objects of the graphical user
`interface, and the functionality of the second key can be
`preferably associated with calling up a context-sensitive
`30 menu.
`
`Selection of the object concerned is preferably performed
`by pressing and releasing the first key, and activation of
`the object concerned is preferably performed by pressing
`and releasing the first key twice in rapid succession.
`
`35
`
`0346
`
`Exhibit 1002 page 346 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`8
`
`Moving or resizing of
`the object concerned can be
`performed by holding down the first key while moving the
`hand-held device
`to move
`the cursor and
`the object
`concerned in unison therewith.
`
`The first key and the second key can be softkeys, whereby
`the current functionality of the softkeys is shown in the
`display, preferably in dedicated fields of the display.
`
`The first key can be placed below the display on the left
`side of the latter, preferably proximate to lower edge of
`the display, and the second key can be placed below the
`display on
`the right side of
`the latter, preferably
`proximate to lower edge of the display.
`
`The means for transforming motion of the hand.held device
`into a signal suitable for moving the cursor over the
`display may further comprise a tilt sensor and/or an image
`capturing device and/or an accelerometer.
`
`The hand-held device according may further comprise means
`to transform a signal from the image capturing device,
`i.e. camera and/or tilt sensor and/or accelerometer into a
`position signal for the cursor.
`
`The means for transforming a signal from the camera into a
`motion signal preferably derives. the motion signal from
`changes between succeeding images, or parts of succeeding
`images captured by the camera.
`
`The camera may have an autofocus system, whereby
`the
`focusing setting of
`the autofocus system is used for
`detecting movement in the camera direction.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`0347
`
`Exhibit 1002 page 347 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`9
`
`The graphical user interface may include one or more of
`the
`fallowing object
`types :
`icons,
`dialogue boxes,
`windows, menus, pointers.
`
`5 Further objects, features, advantages and properties of
`the hand-held device, method for proving user input and
`use of a digital camera in a hand-held device according to
`the
`invention will become apparent
`from
`the detailed
`description.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`In
`the
`following detailed portion of
`the present
`description,
`the
`invention will be explained
`in more
`15 detail with reference to the exemplary embodiments shown
`in the drawings, in which:
`
`25
`
`Fig. l illustrates a preferred embodiment of a hand-held
`device according to the invention,
`20 Fig. 2 shows a block diagram of the hand-held device of
`the embodiment of Fig. l,
`Fig. 3 indicates the axes of movement and rotation along
`which the hand-held device is moved and rotated in order
`to create user input in accordance with the present
`invention,
`Figs. 3.l
`to 3.7
`illustrate
`the use of
`the present
`invention for zooming, scrolling and rotating images shown
`on the display,
`Figs. 3.8
`to 3.10
`illustrate the use of
`the present
`invention for user input to a graphical user interface to
`scroll a table in a window and to resize a window,
`Figs. 3. 11
`to 3 .13 illustrate the use of the present
`invention for scrolling a magnifying window over
`the
`display,
`
`30
`
`0348
`
`Exhibit 1002 page 348 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`10
`
`Figs. 3 .14 and 3. 15 illustrate the use of the present
`invention for moving a part of an image by cutting an
`pasting,
`Fig. 3 .16 illustrates the use of the present invention
`5 with a text editing application,
`Fig. 3. l 7
`illustrates the use of the present invention
`with an application for entering musical notes in a stave,
`Fig. 3 .18 illustrates the use of the present invention
`with a labyrinth game
`10 Fig. 3.19 illustrates the use of the present invention for
`controlling a video game,
`Fig. 3. 20 illustrates the use of the present invention
`with an application for controlling the sound settings of
`a music player funtion,
`15 Fig. 4 outlines a
`software
`flow diagram for zooming,
`scrolling and rotating images shown on the display, and
`Figs. 5 and 6 show an alternative prefe~red embodiment of
`a hand-held device according to the invention.
`
`20 DETAILED DESCRIPTION
`
`This invention allows hand-held communication or computing
`devices with a relatively small display to receive user
`input by moving or rotating the device. In particular with
`devices having a display, the invention allows convenient
`navigation of a large stored virtual display or of objects
`in a graphical user interface. Such devices may include
`PDA devices, camcorders, digital photo cameras, digital
`binoculars
`(solid-state
`stereoscopic
`imaging
`system
`incorporated for viewing and digitally recording magnified
`stereo
`images), mobile hand-held
`terminals,
`advanced
`pagers, mobile telephones, and communicators.
`
`According to a preferred embodiment, the hand-held device
`is a hand portable phone, preferably a cellular/mobile
`phone.
`
`25
`
`30
`
`35
`
`0349
`
`Exhibit 1002 page 349 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`11
`
`Fig. 1 shows a mobile phone according to the invention,
`and it will be seen that the phone, which is generally
`designated by l, comprises a user interface having a
`keypad 2, a display 3, an on/off button 4
`, a speaker 5
`(only the openings are shown), and a microphone 6
`(only
`the opening is shown). The phone 1 according
`to
`the
`preferred embodiment is adapted for communication via a
`cellular network, such as the GSM 900/1800 MHz network.
`
`The keypad 7 has a first group of keys 8 as alphanumeric
`keys, by means of which the user can enter a
`telephone
`number, write
`a
`text message
`(SMS), write
`a
`name
`(associated with
`the phone number) , etc. Each of
`the
`twelve alphanumeric keys 8 is provided with a figure 11 0-9 11
`or a sign 11 # 11 or"*", respectively. In alpha mode each key
`is associated with a number of letters and special signs
`used in the text editing.
`
`The keypad 7 additionally comprises
`two softkeys 9,
`two
`call handling keys 12, and an arrow key 10. The function
`of the softkeys depends on the state of the phone and the
`navigation in the menu can be performed by using the
`navigation-key. The present function of the softkeys 9 is
`shown in separate fields in the display 3, just above keys
`9. The two call handling keys 12 are used for establishing
`a call or a conference call,
`terminating a call or
`rejecting
`an
`incoming
`call. This
`key
`layout
`is
`characteristic for e.g. the Nokia 6210™ phone.
`
`5
`
`15
`
`20
`
`25
`
`30
`
`The arrow key 10 is an up/down key which can be used for
`cursor movement and scrolling and is placed centrally on
`the front surface of the phone between the display 3 and
`the group of alphanumeric keys 7. A battery pack 14 is
`35 mounted on the,back of the phone and supplies electrical
`power for the electronic components of the mobile phone.
`
`0350
`
`Exhibit 1002 page 350 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`12
`
`The phone has a flat display 3 that
`LCD with optional back lighting,
`capable of displaying color images.
`
`is typically made of a
`such as a TFT matrix
`
`The phone is equipped with a digital camera 35 of which
`only the lens 36 is visible in Fig. 1. The camera is
`arranged at the rear of the phone above the battery pack
`14. The
`camera direction
`is
`therefore substantially
`perpendicular to the rear surf ace of the phone 1. Just
`under
`the
`camera
`an
`infrared port
`38
`for
`IrDA
`communication is provided
`( only window of the port is
`shown) . The camera may alternatively have a
`rotatable
`connection to the phone (not shown),
`to allow adjustment
`of the camera direction relative to
`the housing of the
`phone.
`
`Fig. 2 schematically shows the most important parts of a
`preferred embodiment of the phone, in the form of a block
`diagram. A processor 18 controls the communication with
`the network via the transmitter/receiver circuit 19 and an
`internal antenna 20. A microphone 6 transforms the user's
`speech into analogue signals, the analogue signals formed
`thereby are A/D converted in an A/D converter (not shown)
`before
`the
`speech
`is encoded
`in
`a digital signal
`processing unit 14
`(DSP)
`The encoded speech signal is
`transferred to the processor 18, which i.e. supports the
`GSM
`terminal software. The processor 18 also forms
`the
`interface
`to
`the peripheral units of
`the apparatus,
`including a RAM memory 17a and.a Flash ROM memory 17b, a
`SIM card 16, and the keypad 2
`(as well as data, power
`supply, etc.). The digital signal-processing unit 14
`speech-decodes the signal, which is transferred from the
`processor 18 to the earpiece 5 via a D/A converter (not
`shown). The processor communicates in two directions with
`the
`IrDA port 38
`(infrared port)
`that allows data
`
`5
`
`IO
`
`15
`
`20
`
`25
`
`30
`
`35
`
`0351
`
`Exhibit 1002 page 351 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`13
`
`communication with other devices that are equipped with
`such a port,
`such a PC's,
`laptops, personal digital
`assistants
`(FDA) and other mobile phones. The phone may
`further be equipped with a short range RF
`transmitter
`receiver
`(not
`shown) , e.g. according
`to
`the Bluetooth
`standard,
`for data
`transmission with other devices as
`mentioned for IR data communication.
`
`The processor 18 also forms the interface for the display
`controller 31. The image controller 31 receives the image
`files from the processor 18 and renders images for showing
`on the display 3.
`
`A camera controller 37 is also connected to the processor
`lB and controls
`the digital camera 35. The camera
`controller 37 sets the resolution, the refresh rate, the
`focus,
`and zoom factor of
`the camera 35. The camera
`controller 37 sets the focus automatically through any of
`the well-known auto focus techniques available. The output
`signal of the camera 35 is connected to the processor 18.
`
`When the camera is used to detect motion of the hand-held
`device, the camera controller 37 automatically selects the
`appropriate resolution and refresh rate,
`so
`that
`the
`refresh rate is high enough to derive a
`smooth motion
`signal from
`the changes in the succeeding
`images . For
`motion detection it is not usually necessary to use the
`complete
`image captured by
`the digital camera. The
`software can pick out a particular section of the image
`for the motion detection so that the amount of data that
`has to be processed is reduced. These two measures
`(low
`resolution and using a section of the image) allow higher
`sampling rates and reduced power consumption because of
`lower data processing power demands.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`0352
`
`Exhibit 1002 page 352 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`l4
`
`Some surfaces in the camera view may not be particularly
`suited for detecting motion, e.g. because of a uniform
`surface, or because the distance to the objects in the
`camera view is too large to determine changes in distance
`accurately. Such problems may be solved (in the embodiment
`with the adjustable camera direction) by directing the
`camera
`to another available object with sufficient
`texture, such as the user. The camera direction is thus
`reversed compared to the "normal II direption. The motion
`signal derived
`from
`the camera signal
`is
`therefore
`automatically inversed when the camera is directed to the
`user.
`
`The
`camera 35
`is
`a
`conventional digital camera and
`therefore not all the features of the camera 35 are shown.
`The image sensor of the digital camera can be any of the
`known configurations for solid-state image sensors, such
`as
`frame
`transfer,
`interline
`transfer CCDs, or diode
`arrays.
`
`Standard CCD devices are sensitive to both visual light
`and near infrared light. Conventional digital cameras for
`capturing images with visual light-are therefore provided
`with an
`infrared filter for preventing
`influences of
`infrared light on the captured image. The mobile phone l
`is also provided with an infrared filter (not shown). The
`infrared filter can be moved out of the path of the light
`into the camera for capturing infrared images. The IrDA
`port 38 can be set to irradiate continuously to function
`as a
`light source for the camera 35 when it is in the
`infrared mode. Thus, the camera 35 can be used when there
`is little or no ambient light. The use of an IrDA port as
`a light source for a digital camera and the details of a
`device to move
`the infrared filter in and out of the
`camera path as well adjustments to the auto focus system
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`0353
`
`Exhibit 1002 page 353 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`15
`
`are disclosed in US patent application with serial nr.
`10/029,968, hereby incorporated by reference.
`
`The lens 36 is preferably a fixed focal length lens with
`5 movable lens group to allow auto focus, however, lens 36
`could be any type of lens providing for adjustment to
`focus on different parts of the image received, as will be
`understood to those skilled in the art.
`
`The characteristics of visible light and near infrared
`light with respect to focusing are slightly different.
`Therefore, the auto focus system has two settings; a first
`setting for capturing
`images with visual light and a
`second setting for infrared light.
`
`10
`
`15
`
`Standard software for processing, storing and recalling
`pictures captured with visual light and captured with
`infrared light is installed on the phone 1. This software
`may as such be conventional and commercially available.
`20 The software is also able to control the refresh rate of
`the images shown on the display.
`
`25
`
`Optionally, the phone l may also comprise one or two tilt
`sensors 39 which determine the direction and magnitude of
`the displacement relative to vertical using the planetary
`gravitational field. Such sensors could be any of well
`known types such as those operating with an encoding disk
`on a freely rotatable shaft connected to a weight, or of
`the type that uses sphere provided with an asymmetrical
`30 weight that floats in a
`liquid. Alternatively the tilt
`sensor could by of the gyroscopic type. The signals of the
`camera the tilt sensors can be combined for generating the
`motion signal.
`
`35 The phone
`l
`in accordance with e pref erred embodiment
`employs two operational modes associated with the use of
`
`0354
`
`Exhibit 1002 page 354 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`16
`
`motion of
`the phone
`to generate user
`input, which
`throughout this document are referred to as navigation
`mode and fixed mode. When set to
`the
`navigation mode,
`motion of
`the phone 1
`is used
`input,
`as
`and when
`5 displaying an
`image,
`the display view is automatically
`scrolled, zoomed and rotated to follow movements of the
`holding hand. The navigation mode is activated by pressing
`the left softkey 9 "Navigate". Thus,
`the navigation mode
`is activated and the functionality of the left softkey 9
`changes to "Fixed". When set back to the fixed mode by
`pressing the left softkey "Fixed" again, the display view
`becomes stationary and no longer follows the movements of
`the hand.
`
`10
`
`15 Fig. 3
`indicates the relative axes of orientation along
`which the phone 1
`is rotated or translated in order to
`navigate an
`image on
`the display 3. Throughout
`this
`document, axis 40 will be referred to as the Y-axis, and
`,motion of the phone in the direction of the Y-axis is in a
`20 preferred embodiment used
`to scroll
`images
`in the Y(cid:173)
`direction. Similarly, axis 41 will be referred to as the
`X-axis and motion of the phone in the direction of the X(cid:173)
`axis is in a preferred embodiment used to scroll images in
`the X-direction. Motion in the camera direction measured
`along the Z-axis 42 is in a preferred embodiment used to
`control the zoom factor of the images shown on the display
`3. Rotation about the z-axis is used to rotate images
`shown on the display 3 . Though these are the pref erred
`functions assigned with movements along the X-
`,Y- and Z-
`axes, any other functionality can be assigned to movement
`in the direction of these axes or to rotational movement
`about these axes.
`
`30
`
`25
`
`35
`
`While the scrolling, zooming, and rotation of the display
`follows
`the movements of
`the device,
`the
`rate of
`scrolling/zooming/rotation as well as
`the
`amount of
`
`0355
`
`Exhibit 1002 page 355 of 1005
`DENTAL IMAGING
`
`

`

`WO 2004/066615
`
`PCT /EP2003/000607
`
`17
`
`scrolling/zooming/rotation need not
`follow
`the exact
`amount of change in position, and can be inverted. As will
`be discussed below,
`the control software of the present
`invention can smooth the movements of the phone to provide
`a convenient effect on the display.
`
`5
`
`Figs. 3.1 to 3.13 show an overview of the operation of the
`phone l to scroll, zoom and rotate images.
`
`10
`
`With reference to Fig. 3.1, a high resolution image of a
`holiday snapshot comprising
`lake,
`a bridge and
`a
`a
`mountain peak stored in