Tilting Operations for Small Screen Interfaces
`(Tech Note)
`
`Jun Rekimoto
`Inc.
`Sony Computer
`Science Laboratory
`3-14-13 Higashi-Got
`anda, Shinagawa-ku,
`Tokyo 141, Japan
`rekimot o(lcsl.sony.co.jp
`sony.co.jp/person/rekimoto.html
`
`http://www.csl.
`
`tech-
`
`ABSTRACT
`for
`techniques
`new interaction
`introduces
`This TechNote
`small screen devices such as palmtop computers
`or hand-
`held electric devices,
`including
`pagers and cellular phones.
`Our proposed method uses the tilt of
`the device itself as
`input. Using both tilt and buttons,
`it
`is possible to build
`several
`interaction techniques ranging from menus and scroll
`bars, to more complicated
`examples such as a map browsing
`system and a 3D object viewer. During operation, only one
`hand is required to both hold and control
`the device. This
`feature is especially useful
`for field workers.
`KEYWORDS:
`small
`screen interfaces,
`interaction
`niques, palmtop computers
`INTRODUCTION
`difference between desktop computers
`The most significant
`and hand-held computers is not
`the computation
`power, but
`the size of
`the screen.
`It
`is often impractical
`to simply
`apply the user interface for desktop computers
`to palmtop
`sized computers.
`For example,
`the desktop metaphor and
`the multiple windows metaphor are not so effective for
`the
`small screen of a Personal Digital Assistant
`(PDA). Many
`PDAs use a different
`user-interface metaphors,
`such as the
`notebook or cardfile metaphor.
`Another major difference
`between PCs and small-screen
`devices is the input method. Pens and touch panels are often
`used for PDAs, but
`there are several disadvantages.
`These
`interfaces require two hands for operation (one for holding
`the display unit and the other for holding the pen). Pens tend
`to get lost and the pen hides the screen area. With very small
`electronic devices such as a pager,
`it
`is near impossible
`to
`use a pen interface due to its limited screen size.
`TILTING AS INPUT
`there are
`Instead of using a pen or a touch sensitive display,
`other possibilities
`to operate hand-held
`devices.
`Tlhe idea
`of using positions and orientations
`of palmtop computers as
`input was first
`introduced by the Chameleon system [2].
`In
`as the input
`this paper, we explore the idea of using tilt
`method.
`for
`thk material
`copies of all or part of
`to make digitai/hard
`Fernnssion
`personal or clasaroom uae ia granted without
`fee provided
`that
`the copies
`afe not made or dlatributed
`for profit or commercial
`advantage,
`tlhe copy-
`right notice,
`the title of
`the publication
`and its date appear, and notice is
`given that copyright
`is by permission
`of
`the ACM,
`Inc. To copy otherwise,
`to republish,
`to post on servers or to redktribute
`to lists,
`requires specific
`permission
`and/or
`fee.
`USA
`UIST ’96 Seattle Washington
`01996 ACM 0-S9791-798-7/96/11
`
`..$3.50
`
`the device is much easier than sensing
`of
`Sensing rotation
`motions,
`by using small
`solid-state
`angular
`rate sensors
`(gyros). We can also use an electric
`inclinometer
`or a
`compass to assist
`the other sensors.
`As these types of
`sensors become used in other devices, we believe that
`the
`tilting
`interface becomes much more practical. Unlike pen
`interfaces,
`tilting allows single hand operation (i.e., only one
`hand is required to hold and operate the device). This feature
`is particularly
`useful
`for very small electronic devices such
`as pagers.
`
`Figure 1: The operational
`face
`
`concept of the tilting inter-
`
`Figure 2: Two variations of tilting menus
`
`THE PROTOTYPE SYSTEM
`interface, we built a
`the tilting
`To thoroughly
`investigate
`prototype
`system consisting of a palmtop display (an LCD
`TV), a FASTRAK
`position and orientation
`sensor, and two
`buttons, The back-end workstation
`(SGI
`Indigo2)
`takes input
`from the sensor and the buttons through serial cables, and
`generates corresponding
`screen images for
`the display. The
`FASTRAK
`reports the absolute orientation
`of
`the sensor in
`the world coordinates,
`and the system converts it
`into the
`relative rotation about
`the body coordinate system (t9, ~, and
`@in Figure l).
`
`167
`
`SCEA Ex. 1048 Page 1
`
`

`
`(3) Button Release \ -------
`\
`\
`(2)Tdt + Button Pres~\
`
`(4) .4mmated Movement
`---
`--.
`,
`
`1
`(1) Irutial Viewpoint
`/,\
`
`‘
`
`/’
`
`(5)Final Vlewpomt
`
`‘\\
`
`‘
`A.
`@“&g W&lie
`Map Surface
`Figure 3: Navigating a map by tilting the display
`
`the viewpoint by
`view of the map, and allow the user to control
`tilting the display. Unlike other proposed methods to achieve
`“focus and context”
`[3], our system does not use nonlinear
`projection, but simply relies on perspective projection. When
`a user wishes to view the right side of a map,
`for example,
`the user presses the button and tilts the display toward the
`right.
`The system shows a map from a viewpoint
`which
`is located left-hand
`side of
`the current viewpoint.
`At
`the
`same time, a rectangle cursor representing the target position
`appears on the surface of
`the map. The user controls
`the
`location of
`the cursor and looks around the map by keeping
`button depressed and rotating the display left and right, or up
`and down (Figure 3). When the user released the button,
`the
`viewpoint
`smoothly
`approaches to the designated position
`and the user sees animated movement
`toward the destination.
`This interface is more than just a 3D visualization
`of the map.
`When a user tilts the display,
`the following
`transformation
`is applied to the map surface for visualization
`(the map is
`placed on the X-Y plane and the viewpoint
`is looking
`into
`the negative Z axis):
`Trans(C,,
`Cv, –Al~a~(l@l,l~l))*
`–CV, O)
`Rot(Az6’,
`Aj4,0)
`* T~ans(–Cz,
`where Trans(z,
`y, z) is a translation, Rot(d, ~, ~) is a rota-
`tion at an angle of 13about
`the X axis, @ about
`the Y axis,
`and # about
`the Z axis. (CZ, Cv) is the center of rotation, and
`A1,2 are constants. This equation simultaneously
`controls the
`three effects of viewing
`transformation
`(panning,
`zooming,
`and rotating)
`in an interrelated manner. The actual viewpoint
`movement
`is shown in Figure 3.
`3D OBJECT INSPECTION
`the
`In this example,
`viewer.
`We also built
`a 3D object
`user inspects a 3D object model using a palmtop display.
`While pressing the button,
`the orientation
`of the 3D object
`is
`coupled to the orientation
`of
`the display so that
`it looks as if
`the object were fixed to the frame of
`the screen. Users can
`see 3D models from different
`directions just
`like inspecting
`a small object
`in their hand.
`CONCLUSION
`We have presented a new interaction technique that takes the
`tilt of a small display as input commands. This interface is
`being used and evaluated by our hand-held augmented reality
`device (HARD) project. Future plans include integration with
`other
`interaction
`techniques
`(e.g.,
`tilt marking menus), and
`more human-factor
`evaluations
`(e.g., how Fits’
`law applies
`to tilting operations).
`ACKNOWLEDGMENTS
`I thank Toshiyuki Masui
`REFERENCES
`1.
`and Ben
`Jack Callahan, Don Hopkins, Mark Weiser,
`Shneiderman. An empirical
`comparison of pie vs. linear
`menus.
`In CHI’88, pp. 95--100, 1988.
`George W. Fitzmaurice,
`Chignell.
`Virtual
`Reality
`Transactions
`on Information
`pp. 197--218, July 1993.
`George Furnas. Generalized fisheye views.
`pp. 16--23, 1986.
`
`H.
`and Mark
`Zhai,
`Shumin
`for palmtop
`computers.
`ACM
`Systems, Vol.
`11, No. 3,
`
`for helpful discussions.
`
`In CHI
`
`’86,
`
`TILT MENUs
`two
`The first example is a menu interface. We implemented
`variations,
`the cylindrical menu and the pie menu (Figure 2).
`The basic interactions
`are the same; a user first presses the
`button and moves a menu item to the center of
`the screen
`by tilting
`the display while keeping the button depressed,
`then releases the button.
`The cylindrical menu is a 1D
`menu and users select an item by rotating the display about
`the horizontal
`axis. Menu items are placed on the cylinder
`surface to make a selected item larger while displaying
`all
`items. We adapted an already developed pie menu [1]
`to our
`tilting interface. Tilting
`scroll-bars can also be implemented.
`An informal evaluation suggested that users could control
`the
`tilt quite precisely if visual
`feedback was adequately designed
`and provided. With the cylindrical menu,
`for example,
`the
`user was able to control
`the menu selection from one item to
`the next with only a 2 degree tilt. Normally,
`the user normally
`tilts the display 10-15 degrees during operation. Within
`this
`angle,
`the visibility
`of the LCD screen is not affected by the
`tilting.
`There are several alternatives on menu designs. Our current
`implementation
`maps rotation
`about
`the vertical
`axis to
`horizontal movement of
`the menu items, and rotation about
`the horizontal
`axis to vertical movement.
`Instead of moving
`menu items,
`it could also be possible to control
`a cursor
`by tilting while menu items are fixed on the screen. This
`is much like a typical mouse interface, but we selected the
`cursor-fixed
`& menu-floating
`approach because it allows
`larger menu items than the screen size.
`This would
`be
`a desirable feature for small screens that have a restricted
`screen resolution and size.
`NAVIGATING A LARGE 2D SPACE
`Generally,
`browser.
`The
`second
`example
`is
`a map
`within
`fit
`large
`to
`entire
`area
`of
`a map
`is
`too
`are often
`screen,
`but
`simple
`scrolling
`interfaces
`navigation
`Therefore,
`we need a quick
`and intuitive
`for effective
`viewing.
`Our
`solution
`is to provide
`
`the
`a small
`frustrating.
`technique
`a perspective
`
`2.
`
`3.
`
`168
`
`SCEA Ex. 1048 Page 2