Proceedings
`
`Fifth International Syniposiurn on
`Wearable Computers
`
`8-9 October 2001
`
`Zurich, Switzerland
`
`Sponsored by
`
`IEEE-Computer Society
`
`IEEE~
`
`COMPUTER
`SOCIETY
`♦
`
`Los Alamitos, Calif omia
`Brussels = Tokyo
`Washington
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 1 of 10
`
`

`

`Copyright © 2001 by The Institute of Electrical and Electronics Engineers, Inc.
`All rights reserved
`
`Copyright and Reprint Permissions: Abstracting is permitted with credit to the source. Libraries may
`photocopy beyond the limits of US copyright law, for private use of patrons, those articles in this volume
`that carry a code at the bottom of the first page, provided that the per-copy fee indicated in the code is paid
`through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.
`
`Other copying, reprint, or republication requests should be addressed to: IEEE Copyrights Manager, IEEE
`Service Center, 445 Hoes Lane, P.O. Box 133, Piscataway, NJ 08855-1331.
`
`The papers in this book comprise the proceedings of the meeting mentioned on the cover and title page.
`They reflect the authors' opinions and, in the interests of timely dissemination, are published as presented
`and without change. Their inclusion in this publication does not necessarily constitute endorsement by the
`editors, the IEEE Computer Society, or the Institute of Electrical and Electronics Engineers, Inc.
`
`IEEE Computer Society Order Number PRO l 318
`ISBN 0-7695-1318-2
`ISBN 0-7695-1319-0 (case)
`ISBN 0-7695-1320-4 (microfiche)
`ISSN 1530-0811
`
`Additional copies may be ordered from:
`
`IEEE Computer Society
`Customer Service Center
`10662 Los Vaqueros Circle
`P.O. Box 3014
`Los Alamitos, CA 90720-1314
`Tel: + 1714821 8380
`Fax: + I 714 821 4641
`http ://computer.org/
`csbooks@computer.org
`
`IEEE Service Center
`445 Hoes Lane
`P.O. Box 1331
`Piscataway, NJ 08855-1331
`Tel: + I 732 981 0060
`Fax: + I 732 981 9667
`http://shop.ieee.org/store/
`customer-service@ieee.org
`
`IEEE Computer Society
`Asia/Pacific Office
`Watanabe Bldg., 1-4-2
`Minami-Aoyama
`Minato-ku, Tokyo 107-0062
`JAPAN
`Tel: + 81 3 3408 3118
`Fax: + 81 3 3108 3553
`tokyo.ofc@computer.org
`
`Editorial production by Danielle C. Young
`
`Cover art production by Joe Daigle/Studio Productions
`
`Printed in the United States of America by The Printing House
`
`~~ -~_rie ra~ Library System
`t ,rnvers ,ty of Wisconsin - Madison
`728 State Street
`Madison, WI 53706-1494
`U.S.A.
`
`IEEE~
`COMPUTER
`SOCIETY
`♦
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 2 of 10
`
`

`

`This material may be protected by Copyright law {Title 17 U.S. Code)
`
`Energy trade-offs in the IBM Wristwatch computer
`
`\
`
`Noboru Kamijoh, Tadanobu Inoue) C. Michael Olsen+, M. T. Raghunath+, Chandra Narayanaswami+
`{kamijo,inouet}@jp.ibm.com, {cmolsen,mtr,chandrasj@us.ibm.com
`IBM Research Division, Tokyo, Japan
`<+> IBM Research Division, Yorktown Heights, NY, USA
`
`Abstract
`
`We recently demonstrated a high function wrist watch
`computer prototype that runs the Linux operating system
`and also Xii graphics libraries. in this paper we describe
`the unique energy related challenges and tradeoffs we
`encountered while building this watch. We show that the
`usage duty factor for the device heavily dictates which of
`the powers, active power or sleep power, needs to be
`minimized more aggressively in order to achieve the
`longest perceived battery life. We also describe the energy
`issues that percolate through several layers of software all
`the way from de vice usage scenarios, applications, user
`interfaces, system level software to de vice drivers. All of
`these need to be systematically addressed to achieve the
`battery life dictated by the hardware components and the
`capacity of the battery in the de vice.
`
`1 Introduction
`
`We built the high function IBM wrist watch computer
`prototype to study several areas of mobile computing such
`as user interfaces [I] , high resolution displays [2] , system
`software, wireless
`communication,
`security,
`and
`interaction patterns between various pervasive devices. We
`view this watch as a wearable computing platform rather
`than a special purpose device. Therefore our goals differ
`quite significantly from those posed to the designer of a
`traditional wrist watch with just time keeping functions.
`We chose a watch form factor (Figure 1) because
`watches are easily accessible and get misplaced less often
`than PDAs and ceir phones since they are worn, not
`carried. It is also commonly believed that many people
`glance at their watches up to forty times a day and this we
`think is a good reason to put some additional useful
`information, such as upcoming appointments on the watch
`face. The watch is also an ideal device for conveying
`information alerts to the user, since it is instantly viewable.
`takes many of the
`The watch form
`factor also
`packaging, user interface, and power problems to the
`exfreme which appealed to several of us.
`
`1530-0811/01 /$10.00 © 2001 IEEE
`
`133
`
`Figure 1. Wrist watch prototypes
`
`Our watch has a touch sensitive screen and a roller
`wheel for user interaction. The main card has an ARM 7
`based Cirrus EP 7211 processor, 8MB of Flash memory
`and 8MB of DRAM, and serial, IrDA, and expansion
`interfaces. We have two monochrome displays, a 96xl20
`reflective LCD and a 640x480 Organic LED (OLED)
`display [2] . Bluetooth™ functionality is provided by an
`auxiliary card that connects to the main card in some
`watches. The watch is powered by a rechargeable Lithium
`polymer batter with 60mAh capacity at a nominal voltage
`of 3.7V. Figure 2 shows the hardware block diagram.
`We chose the Linux operating system for our watch
`because of the availability of source code and a wide
`variety of software tools, and programmer familiarity .
`Third party software developers are less likely to be
`interested in learning new APis unless the platform is
`already widely deployed. For the same reason we chose
`the XI 1 graphics library instead of defining a new library.
`People are often impressed by the amount of function
`we have managed to fit
`into such a small package.
`However, a question that often gets asked is how long the
`batteries last. Obviously the battery life on a watch like
`ours will not compare favorably with conventional watches
`that have limited functionality. Notwithstanding, battery
`life is an important aspect that we paid attention to, and is
`at the heart of many trade-offs in the design of the entire
`system [3] . In the rest of this paper, we discuss the
`trade-offs we made and the motivations behind these
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 3 of 10
`
`

`

`trade-offs.
`
`LI-Polymer
`BattMy
`
`Buzzer
`
`Figure 2. Hardware block diagram
`
`1.1 The Energy Challenge
`
`The holy grail, energy wise, in the design of mobile
`devices is to enable them to be self sustaining [4].
`Calculators powered by solar cells and self-winding
`mechanical watches are examples of devices that have
`attained this goal.
`Though the energy challenge is often interpreted as
`making the device run as long as possible and be as energy
`efficient as possible, a more pragmatic viewpoint is
`whether battery
`life can exceed some acceptance
`thresholds. Under normal usage patterns, cell
`phone
`batteries last about a week which appears to be an
`acceptable threshold. An analogy can be drawn from
`automobiles. Both car buyers and manufacturers do not
`agonize excessively and solely over gas mileage and pay
`attention to form and function because most people don't
`need to refuel on each trip, and when they do, it is fairly
`easy to refuel. The device is likely to get used if it
`provides services to the user that outweigh the difficulty
`for caring for it. So we have the dual challenge of
`providing useful function while minimizing the effort of
`caring for it.
`Devices that are unable to attain the goal of self
`sustenance often use primary cells and attempt to make the
`replacement of the batteries easy and less frequent. If the
`energy requirement is such that the user will need to
`replace the battery too often, rechargeable batteries have to
`be employed to minimize user aggravation and to protect
`the environment. However, while rechargeable batteries
`can be charged a several hundred times, they generally
`hold four to five times less energy than non-rechargeable
`ones for comparable size and weight (5). Devices that
`operate on rechargeable batteries must also attempt to
`reduce the time taken to charge up the battery.
`
`134
`
`If the recharging can be serendipitous, i.e., combined
`with some other benefit that is delivered to the user, such
`as receiving information from the Internet, the user may
`not perceive recharging as an inconvenience.
`
`1.2 Challenges in a wrist watch form factor
`
`In addition to the general energy challenge faced by
`other wearable computing devices,
`there are several
`additional challenges imposed by the choice of a wrist
`watch form factor.
`The simplest way to increase the battery life is to use a
`larger capacity battery which will be larger and heavier,
`but the size and weight requirements on a wrist watch
`place an upper bound on the capacity of the battery that
`it. Witb
`the best available
`can be used
`to power
`rechargeable Lithium polymer batteries
`today,
`the
`maximum capacity of a battery measuring 3cm x 2cm x
`0.5cm, that can be fit into a regular size watch, is about
`200mAh.
`Second, replacing batteries on a watch is generally
`difficult due to the small size of the watch and the need to
`make the waich water resistant. As per our discussions
`with people who repair watches, a significant number of
`customer complaints directly result from users trying to
`replace the batteries without the right tools. Traditional
`watch manufacturers attempt to make the battery last so
`long that the user is more inclined to buy a new watch
`when it is time to replace the battery.
`The third problem relates to user perception. Users are
`not accustomed to having to recharge wrist watches, but
`may be willing to recharge other devices su~h as cell
`phones and PDAs. It is important to make the user
`perceive a high function wrist wa tch as being similar to
`these other devices rather than traditional wrist watches.
`As mentioned earlier, an advantage of the wrist watch is
`that it is instantly viewable. A constraint that arises from
`this aspect is that the watch should preferably have some
`useful information on its display at all times. While saving
`energy by turning off the display is an option on many
`devices, doing so on a watch may take a significant
`advantage away unless it can be done so cleverly that the
`display is always on when the user is looking at the watch.
`In the following sections we describe the energy related
`tradeoffs associated
`to
`the device usage model,
`the
`hardware, system level software and application level
`software. We end with some suggestions for further
`improvement of battery life.
`
`2 Device usage model
`
`Wearable computing devices are generally in one of
`two modes, sl~eping or active. The device is in the active
`state typically ,when the device is doing something for the
`user; e.g., performing some computation, obtaining and
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 4 of 10
`
`

`

`displaying data etc. The rest of the time, the device sleeps.
`It transitions to the active mode in response to some action
`by the user or the environment. Depending on how long
`the device takes to come out of the sleep mode, there may
`also be states in between these two extremes, such as an
`idle mode where the device responds more guickly to the
`user than if it were in the sleep mode.
`J
`As a gross approximation one may characterize the
`device using two power metrics: The power consumed in
`the active mode (Pac,ivc) and the power consumed in the
`sleep mode (P,1eep)- One can also approximate the actual
`usage of the device using a single metric: the usage duty
`factor (D) which is the fraction of the time the device
`spends in the active mode.
`Informal surveys reveal that owners of Palm Pilots TM
`use them about ten times a day for about 30 seconds at a
`time. This adds up to about 5 minutes per day or a duty
`factor of about 0.0035 . Users of the ParcTAB [6] reported
`that their device was on· for less than I 00 seconds at a
`time. With today's hardware, the device can periodically
`go into a sleep mode, unbeknownst to the user, even when
`the user is actively interacting because of the reaction time
`of the human user and further reduce the actual duty factor.
`The exceptions to low duty factors tend to be devices that
`perform active functioQS even when the user is not
`consciously and actively using the device. An example of
`such a device is an MP3 player watch where the user only
`needs to initiate the play function and cause the watch to
`actively run and play the music requested.
`Observations of the usage patterns of wristwatches
`suggest that the amount of time the user actually spends
`interacting with the advanced features on a digital watch is
`generally an insignificant fraction of the total time. This is
`true for many high function wrist watches as well.
`Calculators probably have an even smaller duty factor.
`Based on these metrics, the average power consumed
`by the device is given by P,1ccv (1-D) + Pac,ivcD. Ifwe further
`define the ratio of active power to sleep power as the
`power factor ratio: PFR =; Pac,;vJPs1ccp, then the total power
`consumed by the device is P,1ccp (1-D) + PFR * P,1ccp D =
`P,1ecp (1-D + PFR*D).
`The PFR for our watch ranges from around 30 to I 00 as
`seen from table in section 3. In comparison, PFR for a
`PalmPilot™ ranges from 60 to 280 [7], the Psion Series
`5™ PDA ranges from 70 to 240 [8], and the Compaq Itsy
`ranges from 30 to 90 [9, 10].
`The battery
`life can be approximated as Battery
`Capacity (mWh) / Average power consumed. This is an
`approximation since battery capacity
`is not really a
`constant but is a function of the precise wave form of the
`load as opposed to just the average [5, 11 ]. Nevertheless, it
`is useful to examine the effects of the different parameters
`on the battery life.
`
`135
`
`Figure 3 below shows the interplay between the usage
`duty factor and the predicted battery life for the watch for
`different power factor ratios. The duty factor is shown on a
`logarithmic scale. The battery life is normalized to one
`when the device is in the sleep mode all of the time. Once
`is built,
`the PFR
`is
`fixed, and
`the
`the hardware
`predominant way of extending
`the battery life is to
`minimize the actual duty factor.
`the
`Ideally, both the minimum sleep current and
`maximum current consumption must both be minimized.
`But which is more important depends on the duty factor.
`At the frequently encountered low duty factors, it is
`very important to focus on minimizing the sleep power
`because reducing the PFR has less perceivable impact on
`relative battery life because we are at the left end of the
`curves in Figure 3. On the other hand, if the duty factor is
`much higher, say two hours a day, then we operate in the
`middle of the graph and it is important make the PFR
`smaller by reducing the active current consumption very
`aggressively.
`
`0.9
`o.a
`:5 0.7
`i 0.6
`CD 0.5
`-i 0 .4
`i 0.3
`
`0.2
`
`0.1
`
`0
`
`f
`
`]
`
`PF •10 -
`PFA• 20 - (cid:173)
`PFA•SO - (cid:173)
`PFA•100 ---e--(cid:173)
`PFA• 250 -------
`
`f
`]
`
`f f
`~
`
`Usage Oury Faclor
`
`!
`
`ls,
`
`{
`6i
`
`{ f
`
`;li
`
`,:.
`¾
`
`Figure 3. Relative battery life versus duty factor
`
`3 Hardware level energy trade-offs
`
`The upper and lower bounds on the battery life for a
`device are determined by hardware. The maximum battery
`life depends on the minimum sleep current that can be
`achieved. Similarly, the minimum battery life is dictated
`by the maximum current consumption when all hardware
`components are turned on.
`We attempted to keep the energy requirement as low as
`possible by
`choosing energy efficient components
`whenever possible. However, energy efficiency often
`comes with less function, e.g., task specific hardware can
`generally operate at a lower energy cost compared to a
`programmable processor, and smaller memories consume
`less power to refresh than larger ones. Compared to other
`wrist watches our design trades off energy efficiency for
`greater function , as is evident from the 32-bit processor
`and large amount of memory we incorporated.
`Fitting all of the components onto a small board,
`measuring 27.5 mm x 35.3 mm, was very challenging.
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 5 of 10
`
`

`

`Even when more energy efficient components were
`available, we were unable to use them because they were
`larger and would have increased the size of the circuit
`board. For instance we used a voltage regulator with a
`leakage current of IOOµA since it was a factor of 8 smaller
`than a voltage regulator with a leakage current of I 0µA.
`Often controls are provided to turn off portions of the
`hardware to save energy. For example, in some systems
`multiple DRAM banks are used and some banks are
`powered off when the software knows that a particular
`DRAM bank is not being used. However, we do not have
`the room on the main board to be able to fit several
`memory modules and hence cannot apply multiple DRAM
`banks to mitigate the power problem in the sleep mode.
`Likewise we did not have any room to have a battery
`backup or a capacitor to preserve DRAM contents if the
`main battery was discharged.
`In order to drive all of the components in our watch,
`except for Bluetooth™, we needed a battery that could
`supply currents as high as 50 mA. This constraint ruled out
`coin cells and led us to choose a rechargeable Lithium
`Polymer battery. A
`rechargeable battery was also
`appropriate since we did not want to open the watch to
`replace the battery often.
`Within the constraints of function and size, we designed
`the hardware to be as energy efficient as possible. The
`processor we chose, an EP72 l I chip [ 12] from Cirrus
`Logic, supported most of our required peripherals such on
`chip ROM, integrated LCD controller, IrDA controller,
`UART, serial, PCM and serial bus interfaces, etc., and we
`did not require the space or power for additional glue
`logic. The processor itself was used in bare die package
`instead of the regular package to save both space and
`power. We run the processor at 18Mhz though it can run
`up to 74 Mhz. The processor supports two power saving
`modes called Idle and Standby which are described below.
`In addition, we ensured the input devices on the watch,
`namely a touch panel, a roller wheel and a tilt sensor,
`consumed no power in the Standby mode.
`feature we
`Another
`significant
`power
`saving
`incorporated into the OLEO display was the ability rapidly
`turn the pixels on and off and to control its brightness by
`adjusting the ratio of the time that the pixels are on to the
`time they are is off. We also have a clear instruction in the
`OLEO that clears the display without requiring the CPU to
`zero all the pixels individually.
`Table
`I
`lists
`the manufacturer specified power
`consumption for various components in operational and
`sleep state in our system. (Vdd=3 .3V in most cases.)
`The range for power quoted by the specifications sheets
`for some components is rather large and the actual
`consumption is determined by the usage scenario and how
`the components are configured and interoperate, e.g., can
`the Flash and the DRAM be consuming their peak powers
`
`at the same time due to the bus architecture. Tools are
`needed to measure power accurately (13]. Also, measuring
`the power consumed by the various components is not
`straightforward in many situations because there may be
`no way to turn off a particular component explicitly, e.g.,
`the DRAM, once Linux is running. In these situations we
`devise a set of tests that are similar to power on self test,
`and measure the power consumed by various components
`before the operating system is loaded. One more issue we
`had was that the battery capacity also reduces over time
`and some of our current batteries appear to have only
`35mAh as opposed to 60mAh when they were fresh.
`
`Table 1. Power from data sheets
`Sleep
`Ooerational
`max/typ Tm W]
`max/typ [mW]
`50/-
`0.010/-
`363/-
`0.660/-
`72/-
`-/0.001
`1.8/-
`0.008/-
`86/66
`0.003/0.000
`-/3
`0.017/0.000
`27/15
`0.031
`7.2/4.8
`0.002
`0.6/0.3
`0.003/0.000
`0.6/0.3
`0.003/0.000
`198/132
`1.82
`
`Component
`CPU (EP 7211)
`DRAM (8MB)
`Flash (8MB)
`Touchpanel controller
`IrDA
`LCD
`Codec
`SPK Amp
`DC-DC 3V
`DC-DC 2.SV
`Bluetooth (early hw)
`
`Furthermore we managed to reduce the operational
`voltage to be a little lower, i.e. from 3.3V to 3.0V, by
`selecting all other components such as Flash memory,
`DRAM, touch panel controller and the IrDA module to be
`the same and this reduced the power consumption by an
`estimated 17 .3%, assuming sta_tic CMOS. The voltage
`· regulators consume 200 µA and 45 µA is consumed in bias
`circuits. Better regulators were larger and therefore not an
`option for us. Table 2 shows the current consumption in
`the watch in the different states of its usage. (Multiply by
`3.7V (nominal battery voltage)
`to get corresponding
`power.)
`In comparison, typical standby currents of the Palm
`Illx TM ( 4 MB of RAM) is 300µA, that of the Psi on Series
`5™ is 540µA (8 MB of RAM). In active modes (with the
`backlight off though), the current cons"1mption in the Palm
`Illx™ ranges from around 15 mA to 65, mA[7], and that
`for the Psion Series 5™ ranges from 35 mA to 130 mA [8] .
`The backlights on the Palm Illx add about 40 mA and on
`the Psion Series 5 add about 70 mA to the above numbers.
`How far can we go with hardware improvements? Even
`with the 200mAh in a state of the art Lithium polymer
`battery, if the system designer wants the battery to last a
`year, to a , first order approximation, ignoring battery
`non-linearitfes and dependence on battery
`recovery
`processes o~ peak current drained and the self discharge of
`the battery, the average current consumption in the device
`
`136
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 6 of 10
`
`

`

`should be 200/(365*24) = 23µA or less. This is an order of
`magnitude lower than the Palm Illx TM consumes with 4MB
`RAM. We also have to fight about a 5% self discharge rate
`per month in Lithium polymer batteries [14].
`
`I
`
`Table 2. C urrent consum~ t1on
`Total current
`LCD
`DRAM
`Processor
`Active
`15-55mA
`OPERATION On
`access
`average of
`0.3-40mA
`27.SmA
`Processor
`Refresh
`> 298µA
`Self Refresh
`298µA
`
`14mA
`IDLE
`
`348µA
`On
`
`<3744µA
`STANDBY
`
`348µA
`On
`
`4.6mA
`
`913µA
`
`22µA
`STANDBY
`
`Off
`
`348µA
`
`Self Refresh
`298µA
`
`22ttA
`
`OttA
`
`565µA
`
`4 System software level energy trade-offs
`
`As noted earlier, battery life depends to a large extent
`on the power consumed in the sleep state if the usage duty
`factor is low. So reducing the power consumed in the sleep
`state is the first issue we looked at.
`
`4.1 Kernel Optimizations
`
`the sleep mode, the task
`in
`When the system is
`scheduler in the Linux kernel sees no tasks that are ready
`to run. In this situation, the kernel switches the processor
`into the Idle mode. Also the kernel relies on a timer to
`IO ms
`interrupt it every
`in order to guarantee fair
`scheduling amongst the tasks that are ready to run. Every
`IO ms when the timer interrupt occurs the processor
`switches to the active mode for a short duration when it
`updates the kernel time variables and checks the task
`queue. If there are no tasks that are runnable, the kernel
`puts the processor back into the Idle mode. The processor
`also comes out of the Idle mode when there is an external
`interrupt as would occur if the user wanted to interact with
`the watch. Also tasks can request to be woken up at some
`point in the future. The kernel maintains a list of such tasks
`in a timer list. At each timer tick the kernel checks to see if
`sufficient time has elapsed to wake up one or more
`sleeping tasks.
`the
`the energy required by
`As observed earlier,
`processor in the Idle mode is much higher than the energy
`required in the Standby mode. So we first focused on
`trying to get the processor into the Standby mode instead
`of the Idle mode when there was no work to do.
`On the ARM processor we used in our watch the 10 ms
`timer is not available in the Standby mode. However 1/0
`interrupts work and a real
`time clock (RTC) which
`operates on a one second granularity is available. Also the
`
`137
`
`processor may take up to 250ms to get out of Standby
`mode but coming out of the Idle mode typically takes only
`a few clock cycles. From a user perspective, the 250ms
`delay is perceptible, but this may be a small price to pay
`for the improved battery life.
`In order to take advantage of the more efficient Standby
`mode, we had to rewrite the Linux timing methodology for
`our system. The basic idea is as follows: Whenever the
`scheduler found that it could put the processor in the Idle
`mode, we scan the timer list to see how much time needs
`to elapse before any of the tasks need to woken up. If this
`time-out interval is long enough we put the processor in
`Standby mode after programming the real-time clock
`(RTC) to wake us up before this time-out. When we come
`out of the Standby mode either on a RTC interrupt or some
`other interrupt, we adjust the kernel time variables so that
`the kernel knows how much time has really elapsed. If the
`nearest task is too close for the 1 second granularity of the
`RTC, we go into Idle mode instead and program the lOms
`timer to wake us up at the time-out period.
`Our changes to the Linux kernel seem to work fine with
`all of the applications we have tested so far, though we
`have not tested this code with a more demanding set of
`user workloads.
`
`while(l)
`if ( !current->need_resched &&
`!hlt_counter)
`cli ();
`if (
`lwork_candidate() ) {
`get_nearest_timeout();
`adjust_hw_timer();
`
`)
`sti ();
`if ( B_STANDBY_VALXD
`proc_stdby () ;
`else
`proc_idle();
`
`current->policy = SCHED_YIELD;
`schedule();
`
`Figure 4. Modified Linux 2.2.1 "main idle loop"
`
`Figure 4 shows a pseudo code snippet of the changes
`we made. This code is from the "main idle loop" in Linux,
`and is in the file arch/arm/kernel/process.c. The lines in
`bold are the statements we have added. The operation is as
`follows. Whenever the current task that is executing in
`Linux has run to completion, control is returned to the
`main idle loop. At the very top of the loop, the current task
`is checked to see if it needs scheduling. If there is no
`current task that needs scheduling, work_candidate ()
`is called to resolve if there are other tasks that are eligible
`for scheduling. If there are no tasks, then there's no work
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 7 of 10
`
`

`

`to be done at the current time. So to determine when there
`is work to be done, we parse the Linux timer list in
`to
`find
`the
`nearest
`get_nearest_timeout ()
`time-out. The returned time-out value is then used by
`adjust_hw_timer () to program either the RTC (if
`the time is long enough) or the 1 Oms timer to expire before
`or at that time-out value. adjust_hw_timer () also
`sets the flag B_STANDBY_VALID if the time is long
`enough to go into Standby mode.
`The selection, programming and handling of the
`hardware timers and the related issue of keeping accurate
`time is complex and we expect to publish the details in the
`future.
`
`4.2 Other Optimizations
`
`Once we have done all that we can to reduce the power
`consumption in the sleep state, the next focus from a
`system software perspective is to ensure that the duration
`spent in the active state in response to user action is
`minimized. The time spent in the active state depends to an
`extent on the libraries and APis used. It is important to
`strike a balance between widely used APis versus highly
`energy-optimized custom code. In order to reduce the time
`spent in the active state, device drivers are optimized to
`reduce the number of instructions that are executed in
`response to interrupts. Device drivers also use interrupts
`and eliminate polling or kernel timer based scheduling
`whenever possible.
`As noted above, we run the X 11 graphics library on our
`watches. The X 11 code consists of the Xserver and the
`Xlib library that applications can link to. The Xserver
`typically draws directly to a frame buffer and in our case
`would typically directly write to the LCD or the OLEO.
`Individual byte or word writes to these devices are
`expensive in terms of power consumption and can also
`result in a visible flicker. We address both of these
`problems by letting the Xserver write to a shadow display
`buffer maintained in DRAM. When we know that the
`Xserver has completed all of its drawing calls and will go
`into a mode where it waits for the next event, we copy the
`contents of the shadow display buffer to the actual display.
`
`5 Application level energy trade-offs
`
`In the default mode the application displays the time
`and calendar information. By using the touch screen and
`the roller wheel the user can navigate to other screens and
`access information such as phone lists, things to do, etc.
`The application program is written in C and uses the X 11,
`math and standard C libraries.
`While it is clear that the application code may have
`control over the energy consumed in the active mode, it is
`interesting to note that in the case of our watch it has an
`impact on the energy consumed in the sleep mode as well.
`
`Our watch shows the current time on the display when the
`watch is in the sleep mode. Both the LCD and OLEO
`displays consume less energy if fewer pixels are turned on.
`So the application code can help reduce the amount of
`energy required in the sleep mode by turning on fewer
`pixels. The impact of the number of pixels on the energy
`required is greater on the OLEO display, but on the LCD
`the difference is not as pronounced.
`On the 640x480 OLEO display, we calculated the
`number of pixels that had to be turned on to display both
`an analog clock face (with hands) and a numeric time
`display. We found that an analog clock face could be
`displayed with fewer pixels. We can reduce the number of
`pixels for a digital display by reducing the font size, but
`this impacts readability. In addition, an analog clock face
`was generally perceived to be much ·easier to read (since it
`also indicated the amount of time remaining till some point
`in the future as well as the current time) and appeared
`more elegant. The different screens we compared are
`shown in Figure 5 below in actual size.
`Eliminating the second hand increased the time the
`watch could spend in sleep state from one second to sixty
`seconds as it was sufficient for the watch to wake up once
`a minute to u date the hands rather than once a second.
`
`"
`......_,
`
`-
`
`. -
`
`Hollow minute hand
`
`On Pixel count: 3849 (1.25%)
`
`-
`
`I
`
`-----....-----
`
`-
`
`Thinner hands
`
`On Pixel count: 3280 (1.07%)
`
`Triangular hands
`
`On Pixel count: 4088 (1.33%)
`
`10:10:00
`
`120 pixel tall Verdana font
`
`On Pixel count: 13916 (4.53%)
`
`10 10·00
`
`120 pixel tall Arial font
`
`'
`
`On Pixel count: 9492 (3.09%)
`
`80 pixel tall Arial font, w/o seconds
`
`1ll 1[)
`
`On Pixel count: 2589 (0.84%)
`
`Figure 5. Representation vs. pixel count
`
`138
`
`GOOGLE EXHIBIT 1041
`GOOGLE v. NEONODE
`IPR2021-01041
`
`Page 8 of 10
`
`

`

`We examined all the different screens presented by the
`application software, making changes to convey the same
`information with fewer pixels. Fonts, icons and graphics
`shown on the LED are all designed to minimize the
`number of pixels that are turned on. On I an emissive
`display with a fine dot pitch, we find that firie lines (even
`those that are just one pixel wide), are quite visible.
`Accordingly, fonts that use thin lines are preferred to bold
`fonts. Icons are designed to use outlines instead of solid
`fills so that fewer pixels are turned on. Thin line graphics
`are used whenever possible.
`In a memory constrained device such as ours, we first
`concentrate on reducing the size (memory footprint) of the
`libraries and application executables to ensure that they fit.
`However, often there is a trade-off between size and the
`dynamic number of instructions executed to accomplish a
`certain task. For instance we· can get code to execute faster
`(i.e., fewer number of instructions) if we use macros
`instead of functions or expand functions in-line. Reducing
`the number of instructions to perform a certain t