as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2006/0034042 Al
`(43) Pub. Date: Feb. 16, 2006
`
`Hisano et al.
`
`US 20060034042A1
`
`(54) ELECTRONIC APPARATUS HAVING
`UNIVERSAL HUMAN INTERFACE
`
`(75)
`
`Inventors: Katsumi Hisano, Matsudo-shi (JP);
`Masanori Ozawa, Kawasaki-shi (JP);
`Ryo Furukawa, Kawasaki-shi (JP);
`Minoru Mukai, Tokyo (JP)
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`GO6F 1/16
`(52) US. Che
`acscsscsssssesense 361/681; 345/905; 248/917
`
`(57)
`
`ABSTRACT
`
`Correspondence Address:
`FINNEGAN, HENDERSON, FARABOW,
`GARRETT & DUNNER
`LLP
`
`901 NEW YORK AVENUE, NW
`WASHINGTON, DC 20001-4413 (US)
`
`(73) Assignee: KABUSHIKI KAISHA TOSHIBA
`
`(21) Appl. No.:
`
`11/196,462
`
`(22)
`
`Filed:
`
`Aug. 4, 2005
`
`(30)
`
`Foreign Application Priority Data
`
`An electronic apparatus includes first and second display
`devices having first and second display screens heldin first
`and second housings,
`respectively. The second display
`screen is provided with a sensor which generates an output
`signal determining an input area in response to an external
`predetermined input to the second display screen. Thefirst
`and second housingsare connected together by a connecting
`mechanism so that an opening angle between thefirst and
`second housings can be adjusted. A first interface image is
`displayed on the seconddisplay screen. An instruction input
`to the interface imageis determined onthe basis of a sensor
`output signal. In response to the instruction input, a second
`display image is displayed in placeofthe first display image.
`In responseto the instruction input, a second interface image
`is displayed in place of the first interface image.
`
`HP Inc. - Exhibit 1027 - Page 1
`
`Aug. 10, 2004
`
`(JP) one ecesessesseeseesneene 2004-233643
`
`14A
`
`
`
`HP Inc. - Exhibit 1027 - Page 1
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 1 of 10
`
`US 2006/0034042 Al
`
`
`XN~- foeeeHe*
`
`
`36
`
`34
`
`N \
`
`RooO
`
`32
`
`HP Inc. - Exhibit 1027 - Page 2
`
`HP Inc. - Exhibit 1027 - Page 2
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 2 of 10
`
`US 2006/0034042 Al
`
`8h Column driver
`
`J@AUPMOY
`
`pS
`
`
`
`JOSUSB}EUINJOODX
`
`a8}
`
`
`
`ColsVk
`
`Y coordinate sensor
`
`~c
`wo
`
`HP Inc. - Exhibit 1027 - Page 3
`
`
`
`CcNddsoydes9
`
`9p
`
`
`
`UONIaSyINDWIO
`
`Aejdsiq
`
`
`
`uonsesBuissedoud
`
`ajeulp100/
`
`
`
`
`
` Ov]puewwosf\q4Apogurew0}
`
`
`
`
`
`Aejdsipabew\Ndjnopuewwoy
`
`HP Inc. - Exhibit 1027 - Page 3
`
`
`
`
`
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 3 of 10
`
`US 2006/0034042 Al
`
`X-Y sensor detects area
`touched by user's finger
`
`$14
`
`Not determined that key
`input has been provided
`
`$16
`
`Yes
`
`during predetermined period
`
`
` Contact area at
`
`least predetermined
`size ?
`
`518
`
`Contact
`
`than
`area smaller
`
`
`predetermined size
`9
`
`Yes
`
`$12
`
`$20
`
`$22
`
`Determine touch to be
`keyboard input and measure
`coordinates of center of area
`
`Compare center coordinates
`wg
`with key position
`
`Output key code
`
`$24
`
`F | G. 4
`
`End
`
`S26
`
`HP Inc. - Exhibit 1027 - Page 4
`
`HP Inc. - Exhibit 1027 - Page 4
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 4 of 10
`
`US 2006/0034042 Al
`
`
`
`FIG.5
`
`18
`
`8
`
`64
`
`72
`
`62
`
`2
`
`4
`
`68
`
`68A
`
`70
`
`66
`
`=~60
`
`FIG.6
`
`HP Inc. - Exhibit 1027 - Page 5
`
`HP Inc. - Exhibit 1027 - Page 5
`
`

`

`US 2006/0034042 Al
`
`Patent Application Publication Feb. 16,2006 Sheet 5 of 10
`
`HP Inc. - Exhibit 1027 - Page 6
`
`HP Inc. - Exhibit 1027 - Page 6
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 6 of 10
`
`US 2006/0034042 Al
`
`130A
`
`HP Inc. - Exhibit 1027 - Page 7
`
`HP Inc. - Exhibit 1027 - Page 7
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 7 of 10
`
`US 2006/0034042 Al
`
` FIG.11A
` FIG.11B
`
`HP Inc. - Exhibit 1027 - Page 8
`
`HP Inc. - Exhibit 1027 - Page 8
`
`

`

`Patent Application Publication Feb. 16, 2006 Sheet 8 of 10
`
`US 2006/0034042 Al
`
`146
`
`HP Inc. - Exhibit 1027 - Page 9
`
`HP Inc. - Exhibit 1027 - Page 9
`
`

`

`Patent Application Publication Feb. 16,2006 Sheet 9 of 10
`
`US 2006/0034042 Al
`
`image display
`
`Remote conference
`
`HP Inc. - Exhibit 1027 - Page 10
`
`HP Inc. - Exhibit 1027 - Page 10
`
`

`

`Patent Application Publication Feb. 16, 2006 Sheet 10 of 10
`
`US 2006/0034042 Al
`
`interface
`
`Optical
`
`HP Inc. - Exhibit 1027 - Page 11
`
`HP Inc. - Exhibit 1027 - Page 11
`
`

`

`US 2006/0034042 Al
`
`Feb. 16, 2006
`
`ELECTRONIC APPARATUS HAVING UNIVERSAL
`HUMANINTERFACE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application is based upon and claims the
`benefit of priority from prior Japanese Patent Application
`No. 2004-233643,filed Aug. 10, 2004, the entire contents of
`which are incorporated herein by reference.
`
`BACKGROUNDOF THE INVENTION
`
`[0002]
`
`1. Field of the Invention
`
`[0003] The present invention relates to an electronic appa-
`ratus having a universal human interface, and in particular,
`to improvements in a user interface for a portable informa-
`tion apparatus having a planar display device and a device
`that allowsa user to provide inputs using the user’s finger or
`a stylus.
`
`[0004]
`
`2. Description of the Related Art
`
`interface is very important for the notebook personal com-
`puter. Thus, notebook personal computers are classified into
`a category including whatis called full-size notebook per-
`sonal computers which are inferior in portability and which
`have large housings,a large screen, and almost the same key
`pitch as that of desktop computers and a category including
`notebook personal computers having mobile housings which
`are small and easy to carry out but which have a small-sized
`screen, a small key pit, and a small touch pad area,thatis,
`an inferior humaninterface. The user selects the category to
`whichthe personal computerthe useris to purchase belongs,
`assuming a situation in which the personal computer is most
`frequently used. Also in this regard, for portable electronic
`apparatuses, portability must be compatible with ease of use.
`
`[0007] For portable electronic apparatuses,it is desirable
`to improve the operability of the pointing device. To move
`the mouse cursor on the screen using the touch pad, the user
`must move the mouse cursor within a plane much smaller
`than the screen by moving his or her finger. This pointing
`device is considered to offer operability inferior to that of a
`mouse connected to the notebook personal computer and
`which can be movedto a larger extent than the touch pad in
`controlling the mousecursor. Similarly, to move the mouse
`cursor using the stick, the user must use the force of his or
`her finger to adjust
`the movement of the position of the
`mouse cursor. This pointing device is considered to offer
`operability inferior to that of a mouse connected to the
`notebook personal computer and which can be movedto a
`larger extent than the stick in controlling the mouse cursor.
`
`[0005] Notebook personal computers, PDAs, cellular
`phones, and the like are knownastypical electronic appa-
`ratuses having user interfaces. For example, a conventional
`notebook personal computer comprises, as a user interface,
`a display such as a liquid crystal display which displays
`characters, graphics s, and the like on a plane surface, and
`a keyboard that allowsa user to input characters by depress-
`ing keys arranged in a predetermined manner, as described
`in Jpn. Pat. Appin No. 1997-130935. In such a notebook
`[0008] The mouse, which is externally installed, is easier
`personal computer, the display and the keyboard are held in
`to operate than the touch pad, whichis incorporated into the
`respective independent housings, which are foldably con-
`notebook personal computer because it allows the user to
`nected together using hinges. When the notebook personal
`use a larger area than the touch pad. However, disadvanta-
`computer is used, the keyboard is placed on a plane almost
`geously, the user must move his or her arm quite a long
`parallel to a desk surface by appropriately adjusting the
`distance in a horizontal direction from the keyboard, from
`hinge angle of one of the housingsto that of the other, both
`which the user commonly provides inputs using both hands,
`housings being placed on the desk surface. The display is
`to the mouse, which is operated with the user’s rightor left
`placed so as to face the user. The user can input characters
`hand. Further, when it is difficult to operate the computer
`from the keyboard and movea cursor by operating keys on
`using the mouse, hardware suchasajoy stick or a game pad
`which arrowsare printed. A pointing device called a touch
`must be mounted on the computer as an external input
`pad is provided on the surface of the housing in which the
`device, the hardware being specialized for software. Such
`keyboard is installed; the touch pad is located farther from
`hardware maynotoffer sufficient general-purpose properties
`the hinge than the keyboard. The user can move the mouse
`andis likely to be infrequently used. Disadvantageously, the
`cursor by touching this pointing device with his or her
`cost effectiveness of the device introduced by the user is
`fingertip. Available pointing devices include a stick system
`virtually low. Further, manufacturing many devices infre-
`that allows the user to move the mouse cursor by using his
`quently used may lead to contamination of environment and
`or her finger to exert a force on a stick provided near the
`consumption of resources. Also in this regard, the use of
`center of the keyboard.
`such hardware is not preferable.
`
`for portable electronic apparatuses,
`In general,
`[0006]
`portability is inconsistent with operability and visibility
`associated with ease of use. It has thus been desirable to
`
`[0009] Notebook personal computers are also commer-
`cially available which have an electromagnetic or pressure-
`sensitive touch panel lying on top of an LCD panelso that
`the
`make portability consistent with ease of use. That is,
`direct touch with the screen enables the position on the
`notebook personal computeris an information apparatus that
`screen to be input. However, with such a notebook personal
`the user carries with him or her for use. Accordingly, the
`computer, when inputting characters, while editing the input
`sizes of the housings are important specification items.
`characters using the pointing device,
`the user must fre-
`Obviously, the display area of the display decreases consis-
`quently move his or her armarelatively long distance
`tently with housing size. Further, when the notebook per-
`against gravity from the keyboard to the screen. This is
`sonal computer is equipped with a keyboard having a key
`disadvantageously inappropriate if both keyboard andpoint-
`arrangement and a key numbersignificantly different from
`ing device are frequently operated.
`the standards, the general-purpose properties of the note-
`book personal computer are degraded. Consequently,
`the
`key pitch of the keyboard decreases consistently with hous-
`ing size. The consistence betweenportability and the human
`
`such a notebook personal computer
`{0010] Further,
`enablesthe user to input characters by displaying a software-
`based keyboard on the screen and depressing virtual keys.
`
`HP Inc. - Exhibit 1027 - Page 12
`
`HP Inc. - Exhibit 1027 - Page 12
`
`

`

`US 2006/0034042 Al
`
`Feb. 16, 2006
`
`However, the keyboard is almost two-dimensionally placed
`on a desk surface, and the display surface is placed at an
`angle with the desk surface so as to facethe user. As is easily
`understood from this arrangement,
`the angle of a plane
`suitable for inputis different from that of a plane suitable for
`viewing. There is a limit to the improvement of the input
`environment based on the addition of the touch panel
`function to one display.
`
`is integrated with the main body. Thus, disadvantageously,
`the user cannot adjust the key pitch or arrangement.
`
`Theinterface for the electronic apparatus can desir-
`{0012]
`ably provide a quieter environment. That is, when the user
`inputs characters from the keyboard, typing sounds occur.
`Accordingly, the keyboard is inconvenient for taking notes
`in a situation where making noise is not preferable, for
`example, in a library or during a class in school. It is thus
`desirable to enable input while maintaining a quiet environ-
`ment.
`
`BRIEF SUMMARY OF THE INVENTION
`
`Itis an object of the present invention to provide an
`[0013]
`electronic apparatus which is excellent in portability, oper-
`ability, visibility, and reliability.
`
`[0014] According to an aspect of the present invention,
`there is provided an electronic apparatus comprising:
`
`the portable
`is desirable to adapt
`it
`[0011] Moreover,
`electronic apparatus to a particular situation so that
`the
`apparatus can be usedeasilyin that situation. In general, the
`keyboard arrangement is specified. Accordingly, when the
`keyboard is utilized for a particular task, the fixed arrange-
`ment may disadvantageously be unsuitable for improving
`workefficiency. For example, the user may carry the note-
`book personal computer with him orher in order to process
`personal information. In this example, information stored in
`the notebook personal computer may be shown to a person
`[0015] a first display device comprisingafirst housing
`
`different from the user. In some cases, a person different
`and a first display screen which is held in the first
`from the user may be allowed to perform a simple operation
`housing;
`to execute a specified process on the basis of a procedure
`stored in a server on the network to which the notebook
`
`thus providing outputs
`personal computer is connected,
`constituting services. In other cases, subsequent business
`activities for articles or insurances may be based on these
`outputs. In such an example, notall the keys of the keyboard
`are required in the site where the user carries the notebook
`personal computer with him orher. Further, the client may
`provide incorrect input. If it is inconvenientto be able to use
`all the key inputs, the notebook personal computers manu-
`factured by the current techniques offer no solutions. The
`possibility of incorrect operations can be reduced byallow-
`ing the client to operate only the previously described touch
`panel-installed display. However, the presence of the key-
`board may hinder fundamental solutions. To solve this, a
`portable personal computer has been provided which has no
`keyboard and which is composed of an externally rectan-
`gular housing comprising a display and the minimum
`required input device such as a power supply switch. How-
`ever, adapting the hardware only to a particular situation
`may degrade general-purpose properties. Specifically, when
`the user carries the apparatus to the client with him or her
`and if the client suddenly changes the contents of the
`operation, it is difficult to deal with this change with the
`above computer. Further, the user normally places paper flat
`on the desk when writing characters on the paper with a
`pencil. When reading a book, the user holds the book in his
`or her hand so that his or her line of sight is perpendicular
`to the print surface of the book. As is apparent from this
`behavior, the angle of a plane at which the user can easily
`input characters with his or her finger or a pen is different
`from that of a plane at which the user can easily read
`information displayed on the plane. This may disadvanta-
`geously tire the user. Moreover, for the electronic appara-
`tuses, the user interface desirably meets users’ individual
`requirements. That
`is, the input device needs to flexibly
`conform to the physical properties or preferencesofusers.It
`is well knownthat the size of each part of the human body
`varies. The input device such as a common keyboard or
`mouse has fixed dimensions. Consequently, a comfortable
`work environment cannot be realized unless the user pre-
`pares an input deviceof a size suitable for himself or herself.
`However, in the notebook personal computer, the keyboard
`
`a second display device comprising a second
`[0016]
`housing and a second display screen whichis received
`and held in the second housing,
`the second display
`screen comprising a sensor which generates a sensor
`output signal depending on an input on the second
`display screen;
`
`a connecting mechanism configured to connect
`[0017]
`the first and second housings together and capable of
`adjusting an opening angle of the first housing with
`respect to the second housing;
`
`a display processing section configured to dis-
`[0018]
`play a first display image onthefirst display screen and
`a first interface image on the second display screen;
`
`a determining section configured to determine an
`[0019]
`instruction to the interface image on the basis of the
`sensor output signal; and
`
`a control section configured to control the dis-
`[0020]
`play processing section in responseto the instruction to
`cause a second display image to be displayed in place
`of the first display image and control
`the display
`processing section in response to the instruction to
`cause a second interface image to be displayed in place
`of the first interface image.
`
`[0021] According to another aspect of the present inven-
`tion, there is provided a foldable display device comprising:
`
`a first device having a first display screen to
`[0022]
`display application images;
`
`a second device having a second display screen
`[0023]
`to display interface images and an input sensor pro-
`vided around the second display screen to sense an
`input position of touching or a movementon the second
`display screen;
`
`a connecting mechanism configured to arbi-
`[0024]
`trarily set an angle of the first display screenofthe first
`device and the second display screen of the second
`device and to be capable to be folded;
`
`HP Inc. - Exhibit 1027 - Page 13
`
`HP Inc. - Exhibit 1027 - Page 13
`
`

`

`US 2006/0034042 Al
`
`Feb. 16, 2006
`
`an input processor to converting an input posi-
`[0025]
`tion sensed by the sensor into an input data to the
`display processor, and
`
`FIG.6 is a sectional view schematically showing
`[0040]
`another structure of the illuminating mechanism incorpo-
`rated into the notebook personal computer shownin FIG.1;
`
`to generate application
`a display processor
`[0026]
`images to be displayed onthefirst display screen and
`interface images to be displayed on the second display
`screen,
`
`[0041] FIG.7 is a perspective view schematically show-
`ing how a documentis displayed on twoscreens according
`to another usage of the notebook personal computer shown
`in FIG.1;
`
`[0042] FIG.8 is a perspective view schematically show-
`[0027] wherein the display processor generates a first
`ing an interface display accordingto yet another usageof the
`interface image indicating distinctly displaying the
`input position after receiving the input data on the
`notebook personal computer shown in FIG.1;
`second display screen, and generatesafirst application
`[0043] FIG. 9 is a perspective view schematically show-
`image reflecting the input data on the first display
`screen.
`ing usage of a notebook personal computer as an example of
`a portable electronic apparatus according to another embodi-
`ment of the present invention;
`
`[0028] According to yet another aspect of the present
`invention, there is provided a portable computer, compris-
`ing:
`
`a flat panel having a first display panel and a
`[0029]
`second display panel, that is foldable along a centerline
`and that
`a
`surface of the second display panel
`approachesto a surface of the first display panel when
`they are folded;
`
`least on a
`a touch panel sensor provided at
`[0030]
`second display panel to detect a user’s input operation;
`
`[0031] display processor to supply display data to the
`first display panel and the second display panel; and
`
`a display controller to control the display pro-
`[0032]
`cessor in at least two modes comprising,
`
`a first mode wherethe display processor supplies
`[0033]
`an application image to the first display panel while
`supplying an interface image to the second display
`panel so as to use the second display panel as an input
`device, and
`
`a second mode where an application image is
`[0034]
`divided into two part and one part of the application
`image is supplied to the second display panel andthe
`other part of the application image is supplied to the
`first display panel so as to use the first and second
`display panels as a one extendedscreen.
`
`BRIEF DESCRIPTION OF THE SEVERAL
`VIEWS OF THE DRAWING
`
`[0035] FIG. 1 is a perspective view schematically show-
`ing a notebook personal computer as an example of a
`portable electronic apparatus according to an embodimentof
`the present invention;
`
`FIG.2 is a sectional view schematically showing
`[0036]
`the internal structure of the notebook personal computer
`shownin FIG. 1;
`
`FIG.3 is a block diagram showinga drivingcircuit
`[0037]
`that drives a touch panel-equipped LCD panelof the note-
`book personal computer shown in FIG.1;
`
`FIG.4 is a flowchart showing control with which
`[0038]
`a key code is output in response to an input to a virtual
`keyboard displayed on the touch panel-installed LCD panel
`of the notebook personal computer shown in FIG.1;
`
`FIG.5 is a sectional view schematically showing
`[0039]
`the structure of an illuminating mechanism incorporatedinto
`the notebook personal computer shown in FIG.1;
`
`[0044] FIG.10 is a perspective view schematically show-
`ing usage of a notebook personal computer comprising a
`gravity sensor as a portable electronic apparatus according
`to yet another embodimentof the present invention;
`
`[0045] FIGS. 11A and 11Bare perspective views sche-
`matically showing usage of a notebook personal computeras
`an example of a portable electronic apparatus according to
`still another embodimentof the present invention;
`
`[0046] FIG. 12 is a perspective view schematically show-
`ing another usage of the notebook personal computer shown
`in FIG.1;
`
`[0047] FIG. 13 is a perspective view schematically show-
`ing a notebook personal computer comprising a silicon on
`glass structure as an example of a portable electronic appa-
`ratus according to further another embodimentof the present
`invention;
`
`[0048] FIG.14 is a perspective view schematically show-
`ing usage of a notebook personal computer comprising an
`input display function a as an example of a portable elec-
`tronic apparatus according to further another embodiment of
`the present invention;
`
`FIG.15 is a perspective view schematically show-
`[0049]
`ing usage of a notebook personal computer comprising an
`input display function a as an example of a portable elec-
`tronic apparatus according to yet another embodimentof the
`present invention;
`
`[0050] FIG. 16 is a perspective view schematically show-
`ing usage of a camera-installed notebook personal computer
`as an example of a portable electronic apparatus according
`to still another embodiment of the present invention; and
`
`{0051] FIG. 17 is a perspective view schematically show-
`ing usage of a camera-installed notebook personal computer
`as an example of a portable electronic apparatus according
`to further another embodiment of the present invention.
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`[0052] With reference to the drawings, description will be
`given of an electronic apparatus having a universal human
`interface according to an embodimentof the present inven-
`tion.
`
`[0053] FIG. 1 is a perspective view schematically show-
`ing a notebook personal computer as an example of a
`portable electronic apparatus according to an embodimentof
`the present invention.
`
`HP Inc. - Exhibit 1027 - Page 14
`
`HP Inc. - Exhibit 1027 - Page 14
`
`

`

`US 2006/0034042 Al
`
`Feb. 16, 2006
`
`[0054] As shown in FIG. 1, the notebook personal com-
`puter is madeportable by foldably connectinga first housing
`2 and a second housing 4 together using hinges 6A and 6B
`and folding the first housing 2 over the second housing 4 to
`lay the first housing 2 on top of the second housing 4.
`Further, as shown in FIG. 1, the first housing 2 is opened
`with respect to the second housing 4 so that a user can
`operate the notebook personal computer while viewing the
`first housing 2 and the second housing4. Thatis, the user can
`place the second housing 4 on a desk surface and use the
`hinges 6A and 6B to appropriately adjust the angle of the
`first housing 2 so that the user can easily view an LCD panel
`in the first housing 2.
`
`[0055] The first housing 2 has the LCD (liquid crystal
`display) panel 8 installed in its frame 10 to display images,
`characters, and the like. A driving circuit (not shown) is
`stored in the housing 2 to drive the LCD panel8. Further, in
`addition to the LCD panel 8, an antenna for wireless LAN
`or radio communication such as Bluetooth is stored in the
`
`first housing 2. The LCD panel8 displays a pointer 12 used
`to select any of icons shown as images or characters. The
`pointer 12 can be used to make any of the icons active to
`operate the related application.
`
`In the second housing 4, rubber pads 14A and 14B
`[0056]
`are applied to the surface of the frame 10 as a cushioning
`material so as to preventthe first housing 2 and the second
`housing 4 from being shocked when they are folded.
`Accordingly,
`the rubber pads 14A and 14B absorb an
`external shock applied when the notebook personal com-
`puter is folded or carried. This prevents the first housing 2
`and the second housing 4 from being damaged bythe shock.
`
`[0057] The second housing 4 has a touch panel-installed
`LCD panel 18 installed in its frame 16. The touch panel-
`installed LCD panel 18 includes a pressure-sensitive touch
`panel laminated to an LCD panel (liquid crystal display
`device) used to display images, characters, and the like. The
`second housing 4 stores a driving circuit (not shown) that
`drives the LCD panelof the touch panel-installed LCD panel
`18 and a detecting circuit (not shown) that detects an input
`to the touch panel. Further,
`the second housing 4 has a
`built-in motherboard on which a CPU and a memoryare
`mounted, and a drive device such as an HDD. The housings
`2 and 4 supply power to each other and exchange informa-
`tion with each other via a bundle of electric wires passing
`through the neighborhood of the center of a shaft of each of
`the hinges 6A and 6B.
`
`[0058] The touch panel-installed LCD panel 18 displays a
`virtual keyboard 20 corresponding to a keyboard imagethat
`can be used to input characters or the like by providing key
`input as with common keyboards. That is, an input to the
`touch panel is replaced with a code or the like which is
`generated when the corresponding key of the common
`keyboard is depressed, on the basis of the relationship
`between the input and the corresponding display image. The
`code is then input
`to a CPU in the notebook personal
`computer main body, where the code is then subjected to
`software processing. Then, the resulting image is displayed
`on the LCD panel 8.
`In this manner, emulation of the
`mechanical keyboard may be implemented by a special
`electronic circuit that controls the touch panel-installed LCD
`panel. Alternatively, it may be implemented by processing
`executed by software such as firmware. The user maycreate
`
`the touch
`the keyboard image in bitmap form. Further,
`panel-installed LCD panel can be used as an interface in a
`form different from the keyboard as described later.
`
`(0059] The touch panel-installed LCD panel 18 displays
`not only the virtual keyboard 20 but also a virtual mouse 22
`operated similarly to a common mouse to movea pointer
`position or make any icon active. That is, an image corre-
`sponding to the mouse 22 is displayed in a screen on which
`the keyboard 20 is displayed. The user uses his or her hand
`to touch and depressa part of the touch panel corresponding
`to the displayed position of the virtual mouse 22, to move the
`virtual mouse 22. This results in a setting such that the entire
`touch panel is used as an area in which the virtual mouse can
`move. In this case, even if the virtual keyboard is depressed
`during the operation of the virtual mouse 22, this is not
`recognized as a key input. Whenthe user’s hand remainsoff
`the virtual mouse 22 for a specified time, the virtual mouse
`movesto the lowerright of the touch panel-installed display
`18, shownin FIG. 1, to wait for an operation. Then, an input
`can be provided to the virtual keyboard 20. Of course, the
`standbyposition of the virtual mouse 22is not limited to the
`lowerright of the touch panel-installed display 20 but can be
`arbitrarily set by the user. Further, by making the displayed
`virtual mouse 22 smaller while it is standing by than while
`it is being used, as in the case of the icons,it is possible to
`reduce the possibility that the virtual mouse 22 will interfere
`with an input area for the virtual keyboard 20. If the
`standing-by virtual mouse 22 is displayed as an icon, then
`tapping the icon enablesthe virtual mouse 22 to be displayed
`near the icon in the use-timesize.
`
`[0060] The following action may be taken instead of
`causing the virtual mouse 22 to stand by whileit is not used:
`upon sensing the thenar eminence, hypothenar eminence,
`index finger, and middle finger of one hand touching the
`touch panel and moving over the touch panel 20 while
`maintaining the same positions,
`the computer may deter-
`minethatthe virtual mouse 22 is being operated. In this case,
`the virtual mouse is displayed on the touch panel-installed
`LCDpanel 20 so that a pointer on the screen can be moved.
`Further, the computer can be allowed to determine, on the
`basis of the pressure of the finger, whether a button is being
`pressed or the finger is off the button, to perform a click
`operation. Further, when one of the fingers moves in a way
`that the distances from this finger to the thenar eminence and
`to hypothenar eminence change, the computer may deter-
`mine that this input indicates that a wheel has been moved.
`In this manner, as in the case of the common mouse, an
`output from the virtual mouse is providedto the CPU in the
`personal computer main body, where it
`is subjected to
`software processing. The results of the software processing
`are displayed as an image on the LCD panel8.
`
`In this case, when the virtual mouse 22 moves onto
`[0061]
`the virtual keyboard 20, it may be displayed on top of the
`virtual keyboard 20 so as to appear translucent or opaque.
`Further, when the virtual mouse 22 or the virtual keyboard
`20 is touched, the brightness, color, or image contrast of the
`touched input area may be changed depending on the
`intensity of the input to the touch panel. Furthermore,if any
`line or character is already displayed in an input area when
`the virtual mouse 22 or the virtual keyboard 20 is touched,
`the thickness of the line, the size of the character, or the font
`may be changed. A sound may be generated which corre-
`spondsto the virtual key displayed in the input area.
`
`HP Inc. - Exhibit 1027 - Page 15
`
`HP Inc. - Exhibit 1027 - Page 15
`
`

`

`US 2006/0034042 Al
`
`Feb. 16, 2006
`
`[0062] Further, the touch panel-installed LCD panel 18
`can display other icons 24A and 24Bandthelike as required
`and also display images, characters, and the like without
`showing the virtual keyboard 20 similarly to the LCD panel
`8. Acombination of the LCD panel 8 and the LCD panel 18
`provides a larger work area than with common personal
`computers. On the touch panel-installed LCD panel 18, the
`icons 24A and 24B may be displayed on top of another
`image. When displayed on top of another image, the icons
`24A and 24B maybe translucent or opaque.
`
`In the above embodiment, the touch panel-installed
`[0063]
`LCD panel 18 senses a contact pressure on the basis of the
`contact area. However, obviously, the touch panel-installed
`LCD panel 18 maybe able to sense the pressure exerted
`directly on a touch panel 18T. Similarly, the touch panel-
`installed LCD panel 18 may beable to electromagnetically
`sense a touch with the touch panel 18T. The electromagnetic
`type adopts a sensor that can detect the intensity of electro-
`magnetic waves on the touch panel 18T to sense a touch on
`the basis of a change in intensity.
`
`[0064] The second housing4 is assembledin the following
`manner: the panel surface of the touch panel-installed LCD
`panel 18,installed in the second housing4,is flush with the
`top surface of the frame 16 of the second housing 4 so that
`no step is formed between the panel surface of the LCD
`panel 18 and the frame 16 of the housing 14. Since there is
`almost no step between the surface of the LCD panel in the
`housing and the surface of the second housing, the move-
`mentof the user’s handis not hindered bysteps at the ends
`of the panel when the virtual mouseis operated.
`
`[0065] With the conventional notebook personal com-
`puter, the pointer movingall over the screen is operated by
`moving the user’s finger in a small area such as the touch
`pad, which is at most several centimeters by several centi-
`meters square. This is considered to be disadvantageous in
`the accuracy of operations. However, with such a notebook
`personal computer as shownin FIG. 1, an area almost equal
`to the size of the housing can be usedas a pointing device.
`Consequently, operability is expected to be drastically
`impr