(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2004/0021643 A1
`Hoshino et al.
`(43) Pub. Date:
`Feb. 5, 2004
`
`US 2004.0021643A1
`
`(54) DISPLAY UNIT WITH TOUCH PANEL AND
`INFORMATION PROCESSING METHOD
`
`(76) Inventors: Takeshi Hoshino, Kodaira (JP);
`Takeshi Minemoto, Kawasaki (JP);
`Yujin Tsukada, Tokyo (JP)
`Correspondence Address:
`MATTINGLY, STANGER & MALUR. P.C.
`1800 DIAGONAL ROAD
`9
`SUTE 370
`ALEXANDRIA, VA 22314 (US)
`(21) Appl. No.:
`10/230,100
`(22) Filed:
`Aug. 29, 2002
`
`(30)
`
`Foreign Application Priority Data
`
`Aug. 2, 2002 (JP)...................................... 2002-226131
`
`Publication Classification
`(51) Int. Cl." ....................................................... G09G 5/00
`(52) U.S. Cl. .............................................................. 345/173
`(57)
`ABSTRACT
`A display unit with a touch panel to move a touch-driven
`member Such as a functional button to a desired position
`through a touch operation. By touching a draggable icon
`displayed on a Screen with a finger, the icon is changed in
`color to indicate that the user can drag the icon. Then, by
`pushing the icon, the icon is changed in size and color to
`indicate that the user can drag the icon. AS by reducing the
`pushing force to touch on the icon, and moving the finger,
`the icon is dragged following the finger. AS by pushing the
`icon with the finger at a predetermined position, the icon is
`Set at this position and changed in size and color. When
`releasing the finger from the icon, the icon is Settled at this
`position.
`
`
`
`
`
`
`
`9
`
`TOUCHSENSOR,
`PRESSURE SENSOR
`
`DISPLAY UNIT
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 1
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 1 of 16
`
`US 2004/0021643 A1
`
`FIG. A
`
`FIG. 1B
`
`
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 2
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 2 of 16
`
`US 2004/0021643 A1
`
`
`
`2ZZ% RSS -
`SNN
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 3
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 3 of 16
`
`US 2004/0021643 A1
`
`16
`
`16
`
`---
`
`16
`
`o
`
`9
`
`FIG. 3A
`
`FIG. 3B
`
`FIG. 3C
`
`FIG. 4
`
`DETERMINATION
`
`CPUSH-IN
`
`
`
`B TOUCH
`
`
`
`A NO REACTION
`O
`
`P1
`
`P2
`
`PUSHING FORCEP
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 4
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 4 of 16
`
`US 2004/0021643 A1
`
`FIG. 5A
`
`
`
`
`
`
`
`8,12
`
`TOUCHSENSOR,
`PRESSURE SENSOR
`
`18
`
`STORAGE UNIT
`
`FIG. 5B
`
`
`
`
`
`TOUCH SENSOR
`(FORDETECTING
`POSITION)
`
`
`
`10,11,15
`
`2
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 5
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 5 of 16
`
`US 2004/0021643 A1
`
`y
`
`obj2
`
`obj4
`
`
`
`
`
`
`
`
`
`ym - I - I
`
`-
`
`I -
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 6
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 6 of 16
`
`US 2004/0021643 A1
`
`FIG. 7
`
`100
`
`OPERATION END
`CONDITION SET
`
`
`
`
`
`
`
`TERMINATE
`FUNCTION
`
`
`
`TOUCHICON
`ASSOCATED WITH
`FUNCTION?
`
`
`
`
`
`MOVE OBJECT
`TO TOUCHED
`POSITION
`
`SETTLE OBJECT
`AT CURRENT
`POSITION
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 7
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 7 of 16
`
`US 2004/0021643 A1
`
`FIG. 8
`
`(1)
`
`(2)
`
`TOUCH
`
`PUSHICON INTO
`(3) ACTIVATEFUNCTION
`
`RELEASEFINGER TIP TO
`(4) TERMINATEFUNCTION
`
`)
`
`6-20
`
`)
`
`FIG. 9
`TOUCH DETERMINE
`N -
`(2) (3)
`(1)
`(4)
`
`
`
`TIME
`(FLOW)
`
`P1 ......------- ir - - - - - - - -
`
`- - - - - - - /
`
`- . . .
`
`. .
`
`.
`
`.
`
`.
`
`.
`
`. . . . .
`
`. .
`
`. .
`
`. . . .
`
`.
`
`. .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. a- - - - - - - - - we r r
`
`w w w w - w w w w w w - - -
`
`P2 ...................Y i-/-r - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`-
`
`PRESSUREP
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 8
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 8 of 16
`
`US 2004/0021643 A1
`
`(1)
`
`G-20
`(x0,y0)
`
`(2)
`
`TOUCH
`
`&
`
`)
`
`MOVEFINGERTPIN
`TOUCH WITHICON
`(5) (SO-CALLEDDRAG)
`
`RELEASEFINGER TIP
`(6) TOSETTLEICON
`
`)
`
`20
`(x1,y1)
`
`)
`
`(1)
`
`
`
`TOUCH
`(2)
`
`DRAG
`(5)
`
`DETERMINE
`(6)
`
`TIME
`(FLOW)
`
`P1
`
`P2
`
`PRESSUREP
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 9
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 9 of 16
`
`US 2004/0021643 A1
`
`
`
`- a YES
`
`TOUCH MOVABLE
`OBJECT?
`
`YES E
`
`
`
`
`
`
`
`
`
`SETTLE OBJECTAT
`CURRENT POSITION
`
`- - - as
`
`- m am us - m a
`
`2O3
`
`204
`
`205
`
`-20 7
`-
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 10
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 10 of 16 US 2004/0021643 A1
`
`FIG. 13
`
`(1)
`
`(2)
`
`TOUCH
`
`PUSH CON INTO
`(3) STARTMOVINGICON
`
`(x0,y0)
`
`)
`
`)
`
`MOVEFINGERTPIN
`TOUCHWTHICON
`(4) (SO-CALLEDDRAG)
`
`RELEASEFINGER TIP
`(5) TOSETTLEICON
`
`)
`
`2O
`
`)
`
`16
`
`2O
`6
`(x2,y2)
`
`F.G. 14
`
`TOUCH
`(1) (2)
`
`DETERMINE
`(3)
`
`DRAG
`(4)
`
`DETERMINE
`(5)
`
`TIME
`(FLOW)
`
`
`
`P1 ................Y vrrior, M.
`
`
`
`P2 -------------------------...------------ i-l- - - - w - - - - - - - a
`
`PRESSURE P
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 11
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 11 of 16 US 2004/0021643 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`- YES
`
`TOUCH MOVABLE
`OBJECT?
`
`S
`YE
`
`DISPLAY THAT OBJECT
`IS MOVABLE
`
`
`
`
`
`
`
`YES
`
`DISPLAY THAT USER CAN
`START MOVING OBJECT
`
`
`
`303
`
`304
`(2)
`
`305
`
`306
`
`RETURN OBJECT TO
`INTIAL POSITION
`
`MOVE OBJECT TO
`TOUCHED POSITION
`
`
`
`
`
`NO
`
`YES
`
`DISPLAY OBJECTAT
`SETTLED POSITION
`
`SETTLE POSITION
`OF OBJECT
`
`310
`
`311
`(6)
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 12
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 12 of 16 US 2004/0021643 A1
`
`(1)
`
`(2)
`
`TOUCH
`
`PUSH CONNTO
`(3) START MOVINGICON
`
`6-20
`(x0,y0)
`
`&
`
`)
`
`)
`
`MOVEFINGER TIPN
`TOUCH WITH CON
`(4) (SO-CALLEDDRAG
`)
`
`PUSHICON INTO
`(5) DETERMINEPOSITION
`
`RELEASEFINGER TIP
`(6) TOSETTLEICON
`
`)
`
`)
`
`20
`
`)
`
`16
`
`20
`
`(x3,y3)
`
`FIG. 17
`
`TOUCH DETERMINE DRAG
`(2)
`(3)
`(4)
`
`(1)
`
`DETERMINE
`(5)
`
`(6)
`
`
`
`TIME
`(FLOW)
`
`P1 ...............Y vri - - - - - - - - .
`
`. .
`
`.
`
`.
`
`. .
`
`.
`
`- - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`P2 --------................-- i.l. i-l-
`
`PRESSURE P
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 13
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 13 of 16 US 2004/0021643 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TOUCH MOVABLE
`OBJECT?
`
`403
`
`YES
`
`RETURN OBJECT TO
`INITIAL POSITION
`
`DISPLAY THAT OBJECT
`S MOVABLE
`
`405
`
`NO
`
`a P-pe DNC
`
`TOUCHED POSITION
`WITHINFUNCTION
`SETING REGION?
`YES
`ACTIVATE FUNCTION
`ASSOCATED WITH
`OBJECT
`
`o- FUNCTION
`
`TERMINATED?
`
`YES
`
`408
`(5)
`
`409
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 14
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 14 of 16 US 2004/0021643 A1
`
`DISPLAY
`COM
`MENTS
`
`
`
`
`
`MOVEFINGERTIPIN
`TOUCH WITHICON
`(3) 22 (SO-CALLEDDRAG)
`
`21
`
`PUSH CON INTO
`DETERMINE
`(4) 23' 22 FUNCTION
`
`2
`
`DISPLAY
`COM
`MENTS
`
`
`
`ARTIST.
`DURATION: 5 MINUTES
`20 SECONDS
`ECASE
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 15
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 15 of 16 US 2004/0021643 A1
`
`FIG. 20
`
`(1)
`
`TOUCH
`(2)
`
`DRAG
`(3)
`
`DETERMINE
`(4)
`
`(5)
`
`P1 ................... \rior lo
`P2 ----------------------------------------------------------------------- -lor.
`
`PRESSUREP
`
`FG 22
`
`(1)
`AAA
`
`27a
`
`(2)
`
`TOUCH
`
`
`
`MOVEFINGERTPIN
`TOUCH WITHMENU
`OBJECT
`(3) (SO-CALLEDDRAG)
`
`PUSH MENU. OBJECTIN
`(4) TODETERMINEFUNCTION
`
`(5)
`
`
`
`FIG. 23 (1)
`
`
`
`TOUCH
`(2)
`
`DRAG
`(3)
`
`DETERMINE
`(4)
`
`(5)
`
`TIME
`(FLOW)
`
`P1 ................... \ri - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`- - - -
`
`P2 ...............---------------------------............------------- .N.I.
`
`PRESSURE P
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 16
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`Patent Application Publication
`
`Feb. 5, 2004 Sheet 16 of 16 US 2004/0021643 A1
`
`FIG 21
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`503
`
`TOUCH MOVABLE
`OBJECT?
`YES
`
`RETURN OBJECT TO
`INTIAL POSITION
`
`
`
`
`
`MOVE SELECTION FRAME
`TO TOUCHED POSITION
`
`TOUCHED
`POSITION WITHINAREA
`OF OPTION?
`YES
`
`SELECT OPTION FOR
`ACTIVATION
`
`508
`
`CHANGE DISPLAY
`
`509
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 17
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`US 2004/0021643 A1
`
`Feb. 5, 2004
`
`DISPLAY UNIT WITH TOUCH PANEL AND
`INFORMATION PROCESSING METHOD
`
`BACKGROUND OF THE INVENTION
`0001. The present invention relates to a display unit with
`a touch panel which is used in a terminal device Such as a
`desktop type, a portable device and the like.
`0002. In recent years, terminal devices have been used
`more and more widely in a variety of businesses. For
`example, automatic teller machines (ATM) have been
`installed in banks and the like, while automatic ticket
`vending machines and map guiding machines have been
`installed in Stations and the like, So that Some of businesses
`involved in the banks, Stations and the like can also be
`processed on the terminal devices. In addition, establish
`ments Such as fast food restaurants also employ terminal
`devices for processing orders of articles made by customers
`(for example, see JP-A-5-216587). Moreover, there have
`been practiced or proposed a terminal device for receiving
`delivered contents, and a terminal device for browsing web
`Sites utilizing a communication network Such as the Internet.
`0.003 Such a terminal device as mentioned above is
`provided with a display unit which displays information
`Such as a message presented on its display Screen, So that a
`user operates an input means Such as a keyboard, while
`Viewing the information, to proceed with a variety of opera
`tions as mentioned. There has been a growing trend to
`employ a display device with a touch panel on a display
`Screen, which has functions of input means, permitting the
`user to make operations on the Screen in accordance with
`messages, menus and the like displayed thereon to execute
`a variety of operations.
`0004 Such a display unit with a touch panel allows the
`user to directly touch his finger tip on the display Screen for
`operations, thereby providing a high operability which
`includes easy operations and leSS errors in operations. In
`addition, Since the number of functional buttons can be
`reduced in an input unit Such as a keyboard, the terminal
`device itself can be reduced in size, resulting in advantages
`Such as a reduction in the area required for installation, a
`higher degree of freedom for a place at which the terminal
`device is installed in a Store, a precinct, or the like.
`0005. A conventional display unit with a touch panel,
`however, displays an image representative of an input device
`comprised of touch-driven memberS Such as functional
`buttons (touch buttons) on its display Screen Such that a
`customer or user Visually recognizes the input device, and
`under this recognition touches a desired position on the
`image representative of the input device, causing the display
`device to execute predetermined processing associated with
`the touched position.
`0006 A personal computer is provided with a mouse
`based drag-and-drop function. Since the user can freely
`move the cursor through the mouse and push a click button
`(buttons), the user can move an icon with the click button
`held down, or place the cursor on an icon and double clickS
`the click button to instruct the personal computer to execute
`a preset function associated with the icon.
`0007. However, since the conventional touch panel, by
`Sensing one touch at a time, executes each process only one
`proceSS can be executed. For example, with a display unit
`
`with a touch panel, typically employed in ATM and the like,
`as the user touches a touch-driven member on a display
`Screen, a function associated with the touch-driven member
`is activated at this time, So that the user cannot perform the
`drag-and-drop function through a touch operation. While
`Some compact information devices with a touch panel
`provide the drag-and-drop function, these devices require
`the user to touch an icon twice at preset timings in order to
`activate a function previously associated with the icon. With
`operations on a mouse, the user indirectly operates the
`mouse while viewing the cursor displayed on the display
`Screen, So that no Serious problems arise in the visibility and
`operability. On the contrary, a touch panel, which forces the
`user to directly handle a touch-driven member displayed on
`a display Screen, implies challenges in the prevention of
`erroneous operations and the operability because the user
`must touch a position once touched by him again at the same
`timing. Particularly, when public devices are concerned,
`Such as ATM, automatic machines, and the like which can be
`operated by indefinite users, a problem Still remains in
`forcing every user to touch a position twice at the same
`timing.
`
`SUMMARY OF THE INVENTION
`0008 To solve the problem mentioned above, it is an
`object of the present invention to provide a display unit with
`a touch panel which is capable of arbitrarily moving a
`touch-driven member Such as a functional button to a
`position desired by a user, and is capable of activating a
`function previously Set to each touch-driven member.
`0009. To achieve the above object, the present invention
`provides a display unit with a touch panel which is capable
`of detecting a pushing force applied to a touch-driven
`member displayed on a display Surface of the display panel
`in two Stages, wherein the touch-driven member is made
`movable on the display Screen in response to a first pushing
`condition, and a function associated with the touch-driven
`member is made executable in response to a Second pushing
`condition.
`0010 Specifically, a representative means according to
`the present invention includes Sensing means for Sensing a
`pushing force P produced by indicating means when the
`indicating means touches the touch-driven member, and a
`control unit for performing first processing associated with
`the touch-driven member pushed by the indicating means
`when the pushing force P detected by the Sensing means is
`equal to or larger than a first Set pressure P1 and Smaller than
`a Second Set pressure P2 larger than the first Set pressure P1
`(P1-P-P2), and for performing second processing associ
`ated with the touch-driven member pushed by the indicating
`means when the pushing force P is equal to or larger than the
`second set pressure P2 (P2<=P), wherein the first processing
`includes dragging the touch-driven member following the
`indicating means pursuant to a movement of the indicating
`means, and the Second processing includes activating a
`function previously assigned to the touch-driven member.
`0011. Other objects, features and advantages of the
`invention will become apparent from the following descrip
`tion of the embodiments of the invention taken in conjunc
`tion with the accompanying drawings.
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 18
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`US 2004/0021643 A1
`
`Feb. 5, 2004
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0012 FIGS. 1A to 1C are diagrams showing one
`embodiment of a display unit with a touch panel according
`to the present invention;
`0013 FIGS. 2A and 2B are diagrams generally illustrat
`ing a specific example of a display means within a housing
`in FIG. 1;
`0.014 FIGS. 3A to 3C are diagrams illustrating how an
`operator touches a finger tip on a touch panel in FIG. 1;
`0.015
`FIG. 4 is a graph showing how to determine from
`a pressure detected by a pressure Sensor whether the finger
`tip illustrated in FIG. 3 is pushing the touch panel in, or is
`in touch with the touch panel, or is released from the touch
`panel;
`0016 FIGS. 5A and 5B are block diagrams illustrating
`exemplary Specific circuit configurations of a main portion
`in the embodiment illustrated in FIG. 1;
`0017 FIGS. 6A and 6B are tables schematically show
`ing Specific examples of databases in a storage unit in FIG.
`5;
`0018 FIG. 7 is a flow chart illustrating a first specific
`example of a function control performed by a control unit in
`FIG. 5;
`FIG. 8 is a diagram illustrating a first specific
`0.019
`example of images displayed on a display Screen in FIG. 1
`in the process of a control operation illustrated in FIG. 7;
`0020 FIG. 9 is a diagram showing a first specific
`example of a change in pressure produced by a finger tip for
`executing the control operation illustrated in FIG. 7;
`0021
`FIG. 10 is a diagram illustrating a second specific
`example of images displayed on a display Screen in FIG. 1
`in the process of the control operation illustrated in FIG. 7;
`0022 FIG. 11 is a diagram showing a second specific
`example of a change in pressure produced by a finger tip for
`executing the control operation illustrated in FIG. 7;
`0023 FIG. 12 is a flow chart illustrating a second specific
`example of the function control performed by the control
`unit in FIG. 5;
`0024 FIG. 13 is a diagram illustrating a specific example
`of images displayed on the display screen in FIG. 1 in the
`process of a control operation illustrated in FIG. 12;
`0.025
`FIG. 14 is a diagram showing a specific example
`of a change in preSSure produced by a finger tip for execut
`ing the control operation illustrated in FIG. 12;
`0.026
`FIG. 15 is a flow chart illustrating a third specific
`example of the function control performed by the control
`unit in FIG. 5;
`0.027
`FIG. 16 is a diagram illustrating a specific example
`of images displayed on the display screen in FIG. 1 in the
`process of the control operation illustrated in FIG. 15;
`0028 FIG. 17 is a diagram showing a specific example
`of a change in preSSure produced by a finger tip for execut
`ing the control operation illustrated in FIG. 15;
`
`0029 FIG. 18 is a flow chart illustrating a fourth specific
`example of the function control performed by the control
`unit in FIG. 5;
`0030 FIG. 19 is a diagram illustrating a specific example
`of images displayed on the display screen in FIG. 1 in the
`process of a control operation illustrated in FIG. 18;
`0031
`FIG. 20 is a diagram showing a specific example
`of a change in pressure produced by a finger tip for execut
`ing the control operation illustrated in FIG. 18;
`0032 FIG. 21 is a flow chart illustrating a fifth specific
`example of the function control performed by the control
`unit in FIG. 5;
`0033 FIG.22 is a diagram illustrating a specific example
`of images displayed on the display screen in FIG. 1 in the
`process of a control operation illustrated in FIG. 21; and
`0034 FIG. 23 is a diagram showing a specific example
`of a change in pressure produced by a finger tip for execut
`ing the control operation illustrated in FIG. 21.
`
`DETAILED DESCRIPTION OF THE
`EMBODIMENTS
`0035) In the following, the present invention will be
`described in connection with several embodiments thereof
`with reference to the accompanying drawings.
`0036 FIGS. 1A to 1C are diagrams showing one
`embodiment of a display unit with a touch panel according
`to the present invention, and FIG. 1A is a perspective view,
`FIG. 1B is a side view, and FIG. 1C is a cross-sectional
`View. The display unit comprises a housing 1; an opening 1a;
`display Screen 2, a mount 3, a Stand 4; a rotating shaft 5; pin
`holes 6; a pin 7; a display panel 8; a display Surface 8a; a
`touch panel 9, and a Supporting member 10.
`0037 AS can be seen in FIG. 1A, the box-shaped housing
`1 is formed with a rectangular opening 1a through a front
`face thereof, and the display Screen 2 is fitted in the opening
`1a. Though not shown, displayed on the display Screen 2 are
`members and areas for activating functions assigned to
`touch-driven memberS Such as functional buttons and mes
`Sages (hereinafter collectively called the "touch-driven
`members”). As a user touches Such a touch-driven member
`with a finger tip, the nib of a pen, or an instructing means
`Such as a pen-shaped input device, the user can operate a
`device which incorporates the display unit according to this
`embodiment (for example, a personal computer, portable
`terminal, ATM, ticket vending machine, and the like).
`0038. The mount 3 is integrally formed on the back of the
`housing 1, opposite to the opening 1a. The mount 3 is
`mounted to the stand 4 through the rotating shaft 5. With the
`mount 3, the housing 1 is rotatable about the rotating shaft
`5 with respect to the Stand 4, So that the display Screen 2 can
`be changed in orientation by rotating the housing 1.
`0039 While the display screen 2 may be continuously
`changed in orientation, the display Screen 2 in this embodi
`ment is designed to permit a gradual change in orientation.
`For this purpose, the mount 3 is formed with a plurality of
`pin holes 6 on a side face arranged about the rotating shaft
`5, and the pin 7 is provided opposite to one point on a line
`along which the pin holes 6 are arranged, as illustrated in
`FIG. 1B, such that the pin 7 can be pushed into the stand 4
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 19
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`US 2004/0021643 A1
`
`Feb. 5, 2004
`
`through one of the pin holes 6. AS the housing 1 is rotated
`about the rotating Shaft 5 to place the display Screen 2 in a
`predetermined orientation, the pin 7 can be fitted into a
`nearby pin hole 6 to fix the display Screen 2 Substantially in
`this predetermined orientation.
`0040. In this manner, the pin holes 6 and pin 7 implement
`a means for adjusting the orientation (angle) of the display
`Screen 2, So that the orientation of the display Screen 2 can
`be changed in as many Stages as the number of pin holes 6.
`0041 As illustrated in FIG. 1C, the display panel 8 as a
`display means is Supported by the Supporting member 10
`and incorporated in the housing 1. The display Surface 8a of
`the display panel 8 for displaying an image is fitted in the
`opening 1a of the housing 1, as the display Screen 2, together
`with the touch panel 9 which covers the entirety of the
`display Surface 8a.
`0.042 FIGS. 2A and 2B are diagrams generally illustrat
`ing Specific examples of the display means contained in the
`housing 1 in FIGS. 1A to 1C. Specifically, FIG. 2A illus
`trates a display means which may be used when the display
`panel 8 is implemented by a liquid crystal panel, a Braun
`tube, a plasma display panel, or the like, while FIG. 2B
`illustrates a display means which may be used together with
`a projector Such as a liquid crystal projector. The display
`means comprises a Supporting member 11, a projection unit
`12, a mirror 13, a Screen 14; and a Supporting member 15.
`Components corresponding to those in FIGS. 1A to 1C are
`designated the Same reference numerals, and repetitive
`description thereon is omitted.
`0043. In FIG. 2A, the display panel 8 is supported by the
`Supporting means 11 in the housing 1 as described above,
`while the touch panel 9 is supported on the display panel 8
`by the Supporting means 11.
`0044) The touch panel 9 comprises a transparent film that
`covers the entire display Surface 8a, and Senses a touch
`thereon by the user with a finger tip (touch sensing) to detect
`the touched position (position detection). In this specific
`example, the touch panel 9 further comprises a function of
`detecting a preSSure when the user touches the touch panel
`9 (pressure detecting function). This function of detecting a
`preSSure may be implemented by three methods as shown in
`the following Table 1.
`
`TABLE 1.
`
`Example 1
`Touch Sensing
`Position
`Detection
`Pressure
`Detection
`
`(Support)
`
`(Support)
`
`Example 2
`Touch Sensing
`Position
`Detection
`
`Example 3
`Touch Sensing
`Position
`Detection
`
`Pressure
`Detection
`
`(Support)
`
`(Support)
`
`Pressure
`Detection
`
`Touch
`panel 9
`
`Supporting
`Member
`1O
`Supporting
`Member
`11
`
`0.045. In Table 1, Example 1 provides the touch panel 9
`with the function of Sensing a touch, and associated func
`tions of detecting a touched position and detecting a preSSure
`at the touched position, wherein the Supporting members 10,
`11 only have their inherent functions of Supporting the
`
`display panel 8, and Supporting the touch panel 9, respec
`tively. In Example 2, the Supporting member 10 for Sup
`porting the display panel 8 is provided with a pressure Sensor
`to have the function of detecting a pressure, while the touch
`panel 9 and supporting member 11 only have their inherent
`functions. In Example 3, the Supporting member 11 for
`Supporting the touch panel 9 is provided with a preSSure
`Sensor to have the function of detecting a preSSure, while the
`touch panel 9 and supporting member 10 only have their
`inherent functions.
`0046. In the specific example illustrated in FIG. 2B, the
`housing 1 contains a projector which is composed of the
`projection unit 12 including a liquid crystal panel, a Braun
`tube, or the like for generating a Video image, the mirror 13,
`and the Screen 14, and the touch panel 9 is disposed outside
`of and integral with the screen 14. The screen 14 and touch
`panel 9 are Supported in the opening 1a of the housing 1 by
`the Supporting member 15.
`0047 The projection unit 12 comprises a projecting lens,
`not shown, to project a Video image from the projection unit
`12 onto the screen 14 through the mirror 13, wherein the
`Video image projected onto the Screen 14 is enlarged by the
`projecting lens. The mirror 13 may be removed when the
`projection unit 12 is arranged to directly oppose the Screen
`14.
`0048 Likewise, in this specific example, the touch panel
`9 is provided with a function of detecting a pressure pro
`duced when the user touches the touch panel 9 (pressure
`detecting function). This function of detecting a pressure
`may be implemented by two methods as shown in the
`following Table 2.
`
`TABLE 2
`
`Example 4
`Touch Sensing
`Position Detection
`Pressure Detection
`
`Example 5
`Touch Sensing
`Position Detection
`
`(Support)
`
`Pressure Detection
`
`Touch panel 9
`
`Supporting
`Member 15
`
`0049. In Table 2, Example 4 provides the touch panel 9
`with the function of Sensing a touch, and associated func
`tions of detecting a touched position and detecting a preSSure
`at the touched position, whereas the Supporting member 15
`only has its inherent function of Supporting the display panel
`8. In Example 5, the Supporting member 15 is provided with
`a pressure Sensor to have the function of detecting a pres
`Sure, whereas the touch panel 9 only has its inherent
`function.
`0050. In the foregoing Tables 1, 2, “touch sensing” refers
`to a conventional function of a touch panel for Sensing a
`finger tip 16 which gets into touch with the touch panel 9.
`0051 Alternatively, in either of the foregoing examples,
`the touch panel 9 may determine that the finger tip 16 gets
`into touch with the touch panel 9 when a position touched by
`the finger tip 16 is detected on the touch panel 9 (i.e., when
`P>=P1).
`0.052 FIGS. 3A to 3C are diagrams illustrating how the
`user touches a finger tip 16 on the touch panel 9, wherein
`FIG. 3A illustrates that the finger tip 16 is not in touch with
`
`Neonode Smartphone LLC, Exhibit 2051
`Page 2051 - 20
`IPR2021-01041, Google LLC v. Neonode Smartphone LLC
`
`

`

`US 2004/0021643 A1
`
`Feb. 5, 2004
`
`the touch panel 9; and FIGS. 3B and 3C respectively
`illustrate that the finger tip 16 is in touch with the touch
`panel 9. More specifically, FIG. 3B illustrates that the finger
`tip 16 is lightly in touch with the touch panel 9, while FIG.
`3C illustrates that the finger tip 16 is Strongly pushing the
`touch panel 9 in.
`0.053
`FIG. 4 is a graph showing how to determine from
`a pressure detected by a pressure Sensor whether the finger
`tip illustrated in FIG. 3 is pushing the touch panel in, or is
`in touch with the touch panel, or is released from the touch
`panel (this determination is made by a control unit, later
`described), wherein the horizontal axis represents a pressure
`P acting on the touch panel 9 (a force pushing the touch
`panel 9, hereinafter called the “pushing force”), and the
`Vertical axis represents the result of determination on the
`pushing force P.
`0054.
`In FIG. 4, a low pressure P1 and a high pressure P2
`have been previously Set, Such that the control unit deter
`mines that no pushing force P is acting on the touch panel
`9 (without reaction) when the pushing force Ponto the touch
`panel 9 is lower than P1 (P<P1); that the finger tip 16 gets
`into touch with the touch panel 9 when the pushing force P
`is equal to or larger than P1 and smaller than P2 (P1-P<P2)
`(corresponding to the "touch sensing in the aforementioned
`Tables 1, 2) to detect the position on the touch panel 9 at
`which the finger tip 16 is in touch with the touch panel 9; and
`that the finger tip 16 is pushing the touch panel 9 in when the
`pushing force P is equal to or higher than P2 (P2<=P)
`because a strong pushing force is acting on the touch panel
`9 to push the same in (corresponding to the "pressure
`detection” in the aforementioned Tables 1, 2).
`0055 FIG. 3A shows P-P1 in FIG. 4, in which case the
`control unit determines that no finger tip is in touch with the
`touch panel 9. FIG. 3B shows that the finger tip 16 is in
`touch with the touch panel 9, in which case the control unit
`determines that the finger tip 16 is not in touch with the
`touch panel 9 when 0<=P<P1, and that the finger tip 16 is
`only in touch with the touch panel 9 but not pushing the
`touch panel 9 in when P1<=P<P2. In this manner, erroneous
`operations can be reduced.
`0056 FIG. 3C shows that the user touches the touch
`panel 9 with the finger tip 16 to push the touch panel 9 in,
`so that FIG. 3C corresponds to P2<=P in FIG. 4. In this
`event, the control unit determines that the touch panel 9 is
`pushed in.
`0057. As described above, this embodiment enables the
`Sensing of a touch on the touch panel 9 in two stages. This
`two-stage Sensing mode can offer the user an operation
`feeling close to the operability provided by a keyboard. For
`example, the user can feel as if he is typing on a keyboard
`in Such a manner that the user recognizes the position of a
`key with a lighter touch (P1-P<P2) and strongly pushes the
`key in (P2<=P). Therefore, this two-stage Sensing mode can
`provide the user with a Secure typing feeling which cannot
`So far been achieved by conventional touch panels. Thus,
`according to the display unit of this embodiment which
`employs the two-stage Sensing mode, the user can instruct an
`asSociated device to activate a particular function by touch
`ing a position on the display Screen corresponding thereto,
`as well as perform a function Similar to the drag-and drop,
`as described later, through a touch operation. Moreover,
`according to the display unit of this embodiment, “activation
`
`of a particular function' can be distinguished from a "drag
`and-drop' operation based on a pressure with which the user
`touches the touch panel, without requiring the user to once
`release the finger tip from the touch panel. This can reduce
`possible erroneous operations, in comparison with a touch
`panel which activates Similar functions relying on the timing
`of touch, as previously described in the prior art example,
`thus providing a Satisfactory typing feeling.
`0.058 While this embodiment is described in connection
`with a touch panel which is touched by a finger tip, taken as
`an example, the display unit with a touch panel according to
`the present invention can be operated with a rod-shaped
`member Such as a pen, a pen-shaped input device, and the
`like, other than the finger tip.
`0059. It should be noted that the aforementioned pressure
`P1 (>0) i