I IIIII IIIIIIII II llllll lllll lllll lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111
`US 20060167647Al
`
`110) United States
`t12) Patent Application Publication
`rm) Pub. No.: US 2006/0167647 Al
`Jul. 27, 2006
`Krumm et al.
`(43) Pub. Date:
`
`(54) SENSING AND ANALYSIS OF AVIBIK\IT
`CO_\ITEXTU.\L SIG_\IALS FOR
`l)ISCRIMl\1-Hl\l(; 81<:'IWEEN I\IDOOR AND
`OUTDOOR LOCATIONS
`
`(75) Inventors: John C. Krumm, Rscdmund, WA (US):
`Eric .I. Hon·itz, Kirkland, WA ( US):
`Ramaswamy llariharan, !nine. CA
`(!JS)
`
`Currcspundcncc Address:
`AVIJN & TUROCY, LLP
`urn :FLOOR. NATIONAL un CENTER
`1900 K\ST NJ\ITH STREET
`CLE\'ELA\ID, Oil 44114 (US)
`
`173) Assignee: Microsoft Corporation. Redmond, WA
`(US)
`
`1)1) Appl. No.:
`
`10/994,550
`
`(22) Filed:
`
`'Ii OY. 22, 2004
`
`Publication ( 'lassitication
`
`(51 i Int. Cl.
`@IK
`(2006.01)
`J/(J(J
`(52 I U.S. Cl. ..............................................................
`702/BO
`
`ABSTRACT
`Methods and systems that determine automatically the like(cid:173)
`lihoud that a device is imide or outside L,f a st~ucture ur
`building. The svstem uses one or more sensors to detect
`ambient conditions, and make the determination. The infcr(cid:173)
`em;e can be used lo save power or suppress services from
`certain devices, which ,ire irrclevm1t. cannot be 11sed cffec(cid:173)
`livdy. or du not funcliun under rerlain circums\am:es. In
`support thereof the system includes one or more context
`sensors that measure parameters associated probabilistically
`with the context of a device. A context computing compo(cid:173)
`nent considers une ur more culllcxl sensors ,md focililalc,
`determination of ideal actions, policies. and sitrnitions 11sso(cid:173)
`ciated with the device .. \ senii.:c provided by tile subject
`invention is the inforence from one or more ,ivailable
`obscnatiuns thi: probability that the device is imi<lc versus
`outside.
`
`rtoo
`
`102~
`
`CONTEXT
`SENSOR1
`
`-~
`
`v-106
`
`/ 104
`
`~ ...
`
`CONTEXT
`COMPUTING
`COMPONENT
`
`CONTEXT
`SENSOR2
`
`..
`
`"
`
`• • •
`
`CONTEXT
`SENSORN
`
`IPR2020-01192
`Apple EX1019 Page 1
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 1 of 14
`
`US 2006/0167647 Al
`
`rtoo
`
`102~
`
`CONTEXT ,,6
`"'
`SENSOR1
`
`v-106
`
`,104
`
`..
`
`~
`
`CONTEXT
`COMPUTING
`COMPONENT
`
`CONTEXT
`SENSOR2
`
`JI
`
`....
`
`• • •
`
`CONTEXT ,,6 .... --
`
`SENSORN
`
`FIG. I
`
`IPR2020-01192
`Apple EX1019 Page 2
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 2 of 14
`
`US 2006/0167647 Al
`
`C START)
`' .
`
`DETERMINE FIRST CONTEXT ,,,,,- 20 0
`DAT A OF FIRST CONTEXT
`
`' .
`COMPUTE CONFIDENCE ESTIMATION ,,,,,-202
`FOR FIRST CONTEXT DATA
`
`' ,
`DETERMINE SECOND CONTEXT ,,,,,- 20 4
`DATA OF SECOND CONTEXT
`
`1 ,
`
`COMPUTE CONFIDENCE ESTIMATION ,..,-206
`FOR SECOND CONTEXT DA TA
`
`1 ,
`
`DETERMINE AMBIENT
`DATA VIA DEVICE
`
`,,,,,- 208
`
`1 ,.
`
`COMPUTE INFERENCE THAT v-21 0
`DEVICE IS IN SECOND CONTEXT
`
`'.
`OPERA TE DEVICE ACCORDING ,,,,,- 21 2
`TO INFERENCE
`
`1,
`
`STOP
`
`FIG. 2
`
`IPR2020-01192
`Apple EX1019 Page 3
`
`

`

`Patent Applicalion Puhlicalion .Jul. 27, 2006 Sheet 3 of 14
`
`US 2006/0167647 Al
`
`t 300
`DEVICE
`
`/ 104
`
`....
`,..-
`
`CONTEXT
`COMPUTING
`COMPONENT
`
`302 - ......... TEMPERATURE .J ...
`304 - .........
`
`LIGHT
`FREQUENCY ...
`
`.J
`
`RADIO
`FREQUENCY -...
`
`~
`
`306 - ............
`308 -" ALTITUDE
`310 - r---....
`312 -
`314 -r---.... HUMIDITY
`316 - ......... AUDIO
`
`IMAGES
`
`GPSFOR
`~ SIGNAL AND
`SPEED
`
`-....
`
`-~
`
`.J
`
`.....
`
`.J ....
`
`,
`.....
`
`• •
`•
`
`•
`• •
`
`CONTEXT
`SENSORN
`
`,
`, ....
`
`FIG. 3
`
`IPR2020-01192
`Apple EX1019 Page 4
`
`

`

`Patent Application Publication
`
`Jul. 27, 2006 Sheet 4 of 14
`
`US 2006/0167647 Al
`
`C START
`
`1,
`
`DETERMINE EXPECTED OUTSIDE ~400
`TEMPERATURE AT THAT LOCATION
`
`,
`COMPUTE PROBABILITY DISTRIBUTION FOR .,,,-402
`OUTSIDE TEMPERATURE AT THAT LOCATION
`
`1
`
`DETERMINE EXPECTED ~404
`INSIDE TEMPERATURE
`,,.
`COMPUTE PROBABILITY DISTRIBUTION FOR v- 406
`INSIDE TEMPERATURE FOR THAT LOCATION
`' .
`MEASURE AMBIENT INSIDE v-408
`TEMPERATURE WITH DEVICE
`
`1 ,
`
`COMPUTE PROBABILITY DISTRIBUTION v-41 0
`OF DEVICE ACCURACY
`
`1,
`COMPUTE PROBABILITY INFERENCE ~412
`THAT DEVICE IS INSIDE
`
`, ,
`OPERATE DEVICE ACCORDING .,,,- 414
`TO INFERENCE
`
`' ,
`
`STOP
`
`FIG. 4
`
`IPR2020-01192
`Apple EX1019 Page 5
`
`

`

`Patent Application Publication Jul. 27, 2006 Sheet 5 of 14
`
`US 2006/0167647 Al
`
`-a/2
`
`0
`
`a/2
`
`FIG. 5
`
`1/a
`
`7
`
`6.5
`
`eu 6
`...
`J.,
`5.5
`0
`J.,
`J.,
`Q,l
`
`{I) a 5
`
`J.,
`
`4.5
`
`4
`
`1
`
`1.5 2
`
`2.5 3
`
`3.5 4
`
`4.5 5
`
`exponent
`
`FIG. 6
`
`IPR2020-01192
`Apple EX1019 Page 6
`
`

`

`Patent Application Publication Jul. 27, 2006 Sheet 6 of 14
`
`US 2006/0167647 A 1
`
`0.2
`
`0.15
`
`0.1
`
`~
`C
`OJ
`:,
`C"
`
`f
`
`OJ .. ....
`OJ > ·-..-s
`
`0.05
`
`O -10
`
`0
`temperature error,°C
`
`10
`
`FIG. 7
`
`IPR2020-01192
`Apple EX1019 Page 7
`
`

`

`Patent Application Publication Jul. 27, 2006 Sheet 7 of 14
`
`US 2006/0167647 Al
`
`START
`
`COMP ARE MEASURED AMBIENT
`TEMPERATURE AND EXPECTED
`OUTSIDE TEMPERATURE
`
`800
`
`N
`
`y
`
`COMPARE MEASURED AMBIENT
`TEMPERATURE AND NORMAL
`RANGE OF INSIDE TEMPERATURES
`
`804
`
`812
`
`N
`
`HIGH LIKELIHOOD
`DEVICE IS OUTSIDE
`
`y
`
`HIGH LIKELIHOOD
`DEVICE IS INSIDE
`
`808
`
`OPERA TE DEVICE ACCORDING
`TO CONTEXT PARAMETERS
`
`810
`
`STOP
`
`FIG. 8
`
`IPR2020-01192
`Apple EX1019 Page 8
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 8 of 14
`
`US 2006/0167647 Al
`
`1
`
`0.9
`
`0.8
`
`0.7
`
`0.6
`
`}
`t...
`c. 0.5
`
`0.4
`
`0.3
`
`0.2
`
`p(out I t,J when ou
`p(ln I \ti when In
`
`Actual inside temperature oc
`23
`
`0.1 -20
`
`-10
`
`20
`0
`10
`30
`outside temperature (t0ut ), ° C
`
`40
`
`- 0.6
`-a.
`
`J:
`.5
`
`1.0
`0.9
`0.8
`0.7
`
`0.5
`0.4
`0.3
`0.2
`0.1
`0.0
`
`FIG. 9
`
`More confidence in
`Indoor temperature
`
`More confidence in
`outdoor temperature
`
`0
`
`0.5
`uncertainty ratio crin/ cr·0 ut
`
`1.5
`
`1
`
`2
`
`FIG. 10
`
`IPR2020-01192
`Apple EX1019 Page 9
`
`

`

`Patent Application Publication Jul. 27, 2006 Sheet 9 of 14
`
`US 2006/0167647 Al
`
`[J[g)[8]
`lnside/Outside
`Probability
`
`L. Infer .I
`r
`
`In
`..
`094
`
`I "
`
`Out
`0.06
`
`lif
`
`I
`
`tffl In/Out lnfer~nce
`Current Location
`Latitude: j 41,52
`Long1itud~ I ·96.73
`From Zip J
`5-digijt U.S. Zipcode: I 57105
`I Sioux ... Falls. SD
`City. St8te
`Outside Temperature ----
`rus: I
`t ee1s
`Fahrenheit: I
`Ambient Temperature ~-=--
`J Celsius: I
`· ··· Fahrenheit I
`
`Get
`
`.... u Get
`
`17.83
`64.1
`
`•
`23.44
`74.2
`
`FIG.11
`
`IPR2020-01192
`Apple EX1019 Page 10
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 10 of 14
`
`US 2006/0167647 Al
`
`START
`
`ENABLE DEVICE GPS SYSTEM
`
`TAKE LOCATION READING
`
`1200
`
`1202
`
`1214
`
`y
`
`HIGH LIKELIHOOD
`DEVICE IS OUTSIDE
`
`N
`
`MAKE SECONDARY
`MEASUREMENTS
`
`1206
`
`HIGH LIKELIHOOD
`DEVICE IS INSIDE
`
`1210
`
`OPERA TE DEVICE ACCORDING
`TO CONTEXT PARAMETERS
`
`1212
`
`STOP
`
`FIG. 12
`
`IPR2020-01192
`Apple EX1019 Page 11
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 11 of 14
`
`US 2006/0167647 Al
`
`START
`
`ENABLE DEVICE OPS SYSTEM
`
`TAKE FIRST LOCATION READING
`
`1300
`
`1302
`
`TAKE SECOND
`LOCATION READING
`
`1304
`
`COMPUTE DEVICE
`VELOCITY
`
`1306
`
`COMPARE VELOCITY
`WITII PREDETERMINED
`VELOCITY CRITERIA
`
`1308
`
`1316
`
`HIGH LIKELIHOOD
`DEVICE IS OUTSIDE
`
`N
`
`HIGH LIKELIHOOD
`DEVICE IS INSIDE
`
`1312
`
`OPERA TE DEVICE ACCORDING
`TO CONTEXT PARAMETERS
`
`1314
`
`STOP
`
`FIG.13
`
`IPR2020-01192
`Apple EX1019 Page 12
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 12 of 14
`
`US 2006/0167647 Al
`rt4oo
`
`102~
`
`CONTEXT
`SENSOR1
`
`JI ..
`
`CONTEXT
`SENSOR2
`
`4 ...
`
`...
`
`,,.-
`
`(1 04
`
`CONTEXT
`COMPUTING
`COMPONENT
`
`·~ r•
`
`402
`
`' I'
`
`• • •
`
`AI COMPONENT
`
`CONTEXT
`SENSORN
`
`FIG. 14
`
`IPR2020-01192
`Apple EX1019 Page 13
`
`

`

`Patent Ap11lication Publication Jul. 27, 2006 Sheet 13 of 14
`
`US 2006/0167647 Al
`
`r1soo
`
`1502
`
`r- 1530
`---------~--------~
`: OPERA TING SYSTEM :
`-------------------
`.--------L 1532
`1 APPLICATIONS :
`L--------------~
`f
`1534
`________
`1
`1 MODULES
`:
`L--------------"'
`,--------_r
`1536
`DATA
`:
`L--------------~
`( ______ y
`- _- --...- -
`-
`I EXTERNAL
`I
`'-
`_HDQ_ _ ..,1
`
`1
`
`~514 -
`
`1514
`
`PROCESSING
`UNIT
`
`1508
`SYSTEM
`MEMORY
`
`1504
`
`1506
`
`,__ __ __.___, 1526
`
`INTERFACE i+---+
`
`VIDEO
`ADAPTOR
`
`FDD
`
`DISK
`
`OPTICAL
`DRIVE
`DISK
`
`1518
`
`1520 .-------1--,544
`MONITOR
`
`1522
`
`1538
`
`KEYBOARD
`
`1540
`
`MOUSE
`
`1548
`
`REMOTE
`COMPUTER(S)
`
`1550
`
`MEMORY/
`STORAGE
`
`INPUT
`DEVICE
`INTERFACE ---
`
`1542 (WIRED/WIRELESS)
`1558
`
`MODEM
`
`NETWORK
`14---------+-+t
`ADAPTOR
`(WIRED/WIRELESS)
`
`FIG.15
`
`IPR2020-01192
`Apple EX1019 Page 14
`
`

`

`Patent Application Publication Jul. 27, 2006 Sheet 14 of 14
`
`US 2006/0167647 Al
`
`1602
`
`CLIENT(S)
`
`r1600
`
`1604
`
`SERVER(S)
`
`1608
`
`1610
`
`CLIENT DATA STORE(S)
`
`SERVER DATA STORE(S)
`
`FIG. 16
`
`IPR2020-01192
`Apple EX1019 Page 15
`
`

`

`US 2006/0167647 Al
`
`Jul. 27, 2006
`
`1
`
`SENSING AND ANALYSIS OF AMBIENT
`CONTEXTUAL SIGNALS FOR DISCRIMINATING
`BETWEEN INDOOR AND OUTDOOR LOCATIONS
`
`TECHNICAL FIELD
`
`[0001] This invention is related to sensing mechanisms,
`and more specifically, sensing mechanisms employed in
`portable devices to sense changes in device location.
`
`BACKGROUND OF THE INVENTION
`
`[0002] As computing moves off the desktop into the hands
`of mobile users, it is becoming more important for mobile
`devices to be aware of the user's context. Important pieces
`of context include the user's location, activities, nearby
`people and devices, and mode of transportation, if any. This
`knowledge can in tum be used by mobile devices to display
`reminders, to configure themselves for use with other
`devices, and to behave in a way that is appropriate for the
`surrounding environment ( e.g., turn off cell phone ringer) or
`subcontexts of the surrounding enviromnent such as whether
`particular states or transitions among states are occurring
`within the environment.
`
`[0003] One significant aspect of context concerns whether
`or not the user (and the device) is inside or outside of a
`building or strncture. For example, knowledge of such
`information can be used to facilitate determining the user's
`location ( e.g., in a building or strncture, in a particular
`building or strncture, or in one of a set of known buildings
`or strnctures) and the user's mode of transportation (e.g., in
`a bus, car or airplane). Such knowledge can also be used to
`conserve power on systems that do not provide useful
`services inside buildings or outside. For example, because
`GPS typically fails to operate inside because satellite signals
`are not available inside, determination of the likelihood that
`a user is inside can be used to turn off a GPS system or put
`the system into a mode where it probes for satellite signals
`periodically so as to conserve the batteries of the GPS
`system.
`
`[0004] One way to make an inside/outside determination
`would be to use a digital map of building footprints along
`with knowledge of the user's location or recent location.
`However, for most buildings such a map does not exist.
`Additionally, location data is not necessarily available, espe(cid:173)
`cially inside a strncture where GPS typically fails.
`[0005] In view of the foregoing, there is an unmet need for
`an improved technique to glean information regarding such
`inside/outside context of a device and/or an individual.
`
`SUMMARY OF THE INVENTION
`
`[0006] The following presents a simplified smnmary of the
`invention in order to provide a basic understanding of some
`aspects of the invention. This sunnnary is not an extensive
`overview of the invention. It is not intended to identify
`key/critical elements of the invention or to delineate the
`scope of the invention. Its sole purpose is to present some
`concepts of the invention in a simplified form as a prelude
`to the more detailed description that is presented later.
`[0007] The invention disclosed and claimed herein, in one
`aspect thereof~ comprises architecture for automatically
`determining and/or inferring if a device or individual is
`inside or outside is provided. The system can employ one or
`
`more sensors to detect ambient conditions, and make such
`determination and/or inference. The system can include one
`or more context sensors that measure parameters of a first
`context of a device. A context computing component inter(cid:173)
`faces to the one or more context sensors and facilitates
`determination of a change from the first context to a second
`context. Knowledge of such context transition can for
`example be used to save power in certain devices, which
`may not be used or even ftmction in certain states (e.g.,
`darkness).
`
`[0008] In another aspect of the subject invention, the
`system generates probability distributions which are math(cid:173)
`ematically combined to ultimately derive a probability infer(cid:173)
`ence that the device is inside or outside.
`
`[0009] In yet another aspect, the sensors can include
`devices suitable for measuring temperature, light frequency,
`radio frequency (e.g., 60 Hz or 50 Hz electromagnetic
`signals emitted from local power lines, systems, lights, and
`appliances, IEEE 802.11 Wi-Fi signals, AM and FM radio,
`or GPS satellite signals), capturing images or image flows,
`location data (e.g. GPS location analysis), pressure, humid(cid:173)
`ity, and audio signals.
`
`[0010] In still another aspect thereof, a machine learning
`and/or inferential component enable a probabilistic and/or
`statistical-based analysis to prognose or infer an action that
`a user desires to be automatically performed based in part on
`the likelihood that a user is indoors or outdoors.
`
`[0011] To the accomplishment of the foregoing and related
`ends, certain illustrative aspects of the invention are
`described herein in connection with the following descrip(cid:173)
`tion and the annexed drawings. These aspects are indicative,
`however, of but a few of the various ways in which the
`principles of the invention can be employed and the subject
`invention is intended to include all such aspects and their
`equivalents. Other advantages and novel features of the
`invention will become apparent from the following detailed
`description of the invention when considered in conjunction
`with the drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0012] FIG. 1 illustrates a system that determines device
`context in accordance with the subject invention.
`
`[0013] FIG. 2 illustrates a methodology of differentiating
`between inside and outside in accordance with the invention.
`
`[0014] FIG. 3 illustrates shows a device that employs one
`or more sensors and the context computing component for
`inside/outside detennination according to the invention.
`
`[0015] FIG. 4 illustrates a methodology of deriving prob(cid:173)
`ability distributions when using temperature as a means for
`determining device context in accordance with the inven(cid:173)
`tion.
`
`[0016] FIG. 5 illustrates the probability distribution for
`the temperature sensor.
`
`[0017] FIG. 6 illustrates a graph of RMS temperature
`error as a function of a weighting component that gives an
`optimal exponent.
`
`[0018] FIG. 7 illustrates shows a graph that represents a
`histogram of errors.
`
`IPR2020-01192
`Apple EX1019 Page 16
`
`

`

`US 2006/0167647 Al
`
`Jul. 27, 2006
`
`2
`
`[0019] FIG. 8 illustrates an alternative methodology of
`deriving an approximation that classifies whether the device
`is inside or outside, in accordance with the invention.
`
`[0020] FIG. 9 illustrates the behavior of the equation as
`the outside temperature varies.
`
`[0021] FIG. 10 illustrates of a graph that simulates an
`outside and inside temperature both equal to the mean inside
`temperature.
`
`[0022] FIG. 11 illustrates a user interface that facilitates
`temperature-based inside/outside determination in accor(cid:173)
`dance with the invention.
`
`[0023] FIG.12 illustrates a methodology of using GPS to
`determine the inside/outside location of a device based on
`the GPS signal, in accordance with the invention.
`
`[0024] FIG. 13 illustrates a methodology of using GPS to
`determine the inside/outside location of a device based on
`speed, in accordance with the invention.
`
`[0025] FIG.14 illustrates a context computing system that
`includes artificial intelligence for learning and automating
`features thereof in accordance with the invention.
`
`[0026] FIG. 15 illustrates a block diagram of a computer
`operable to execute the disclosed architecture.
`
`[0027] FIG.16 illustrates a schematic block diagram of an
`exemplary computing enviromnent in accordance with the
`subject invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`[0028] The invention is now described with reference to
`the drawings, wherein like reference numerals are used to
`refer to like elements throughout. In the following descrip(cid:173)
`tion, for purposes of explanation, nlllllerous specific details
`are set forth in order to provide a thorough understanding of
`the subject invention. It may be evident, however, that the
`invention can be practiced without these specific details. In
`other instances, well-known structures and devices are
`shown in block diagram form in order to facilitate describing
`the invention.
`
`[0029] As used in this application, the terms "component"
`and "system" are intended to refer to a computer-related
`entity, either hardware, a combination of hardware and
`software, software, or software in execution. For example, a
`component can be, but is not limited to being, a process
`rum1ing on a processor, a processor, an object, an executable,
`a thread of execution, a program, and/or a computer. By way
`of illustration, both an application rumung on a server and
`the server can be a component. One or more components can
`reside within a process and/or thread of execution, and a
`component can be localized on one computer and/or dis(cid:173)
`tributed between two or more computers.
`
`[0030] As used herein, the tern1 to "infer" or "inference"
`refer generally to the process of reasoning about or inferring
`states of the system, envirolllllent, and/or user from a set of
`observations as captured via events and/or data. Inference
`can be employed to identify a specific context or action, or
`can generate a probability distribution over states, for
`is, the
`example. The inference can be probabilistic-that
`computation of a probability distribution over states of
`interest based on a consideration of data and events. Infer-
`
`ence can also refer to techniques employed for composing
`higher-level events from a set of events and/or data. Such
`inference results in the construction of new events or actions
`from a set of observed events and/or stored event data,
`whether or not the events are correlated in close temporal
`proximity, and whether the events and data come from one
`or several event and data sources.
`
`[0031] While certain ways of displaying information to
`users are shown and described with respect to certain
`figures, those skilled in the relevant art will recognize that
`various other alternatives can be employed. The terms
`"screen,""web page," and "page" are generally used inter(cid:173)
`changeably herein. The pages or screens are stored and/or
`transmitted as display descriptions, as graphical user inter(cid:173)
`faces, or by other methods of depicting information on a
`screen (whether personal computer, PDA, mobile telephone,
`or other suitable device, for example) where the layout and
`information or content to be displayed on the page is stored
`in memory, database, or another storage facility.
`
`[0032] Context Classification
`
`[0033] A technique for inside/outside classification that
`exploits one or more sensor measurements on which to base
`such classification is described herein. For example, the fact
`that inside enviromnents are nornially temperature-con(cid:173)
`trolled can be exploited as one means for making this
`determination. If a mobile device can measure ambient
`temperature, and if it has knowledge of or obtains the current
`outside temperature, it can determine or infer whether or not
`it is indoors or outdoors. Outside temperature inforniation
`can be obtained from a database of worldwide outside
`temperature data maintained based on hourly updates from,
`for example, the American National Oceanic and Atmo(cid:173)
`spheric Administration's (NOAA's) National Weather Ser(cid:173)
`vice (NWS). If the device's ambient temperature is within a
`range of normal inside temperatures, and if the outside
`temperature is significantly different, then there is a high
`probability that the device is inside. If, on the other hand, the
`device's an1bient temperature is closer to the local outside
`temperature, then the device is more likely outside.
`
`[0034] One attractive characteristic of this technique is the
`simplicity of the required sensing. Temperature can be
`measured easily with a small inexpensive sensor that draws
`power from the mobile device. Knowledge of outside tem(cid:173)
`perature typically requires that the device obtain information
`as to its general location. But, typically since temperature
`varies marginally across relatively long geographic dis(cid:173)
`tances, the location estimate need not be extremely accurate.
`For example, the system can use location information
`described in terms of U.S. postal codes. In another imple(cid:173)
`mentation, the location information can be derived by GPS
`(Global Positioning System) location techniques. Such
`approximate location metrics can often provide temperature
`information suitable for making determinations described
`herein.
`
`[0035] While it is clear that determining device location
`by temperature differentiation can be difficult in locations
`where the inside and outside temperatures have little differ(cid:173)
`entiation, the disclosed technique still provides a high
`degree of correctness by reasoning mathematically about the
`temperature distributions, a probability of being inside,
`which reflects the uncertainty caused by similar inside and
`outside temperatures.
`
`IPR2020-01192
`Apple EX1019 Page 17
`
`

`

`US 2006/0167647 Al
`
`Jul. 27, 2006
`
`3
`
`[0036] The certainty of
`is
`inferences
`inside/outside
`strongly related to the certainty of three different tempera(cid:173)
`ture distributions: measured ambient temperature from the
`device, expected inside temperature, and outside tempera(cid:173)
`ture interpolated from weather stations. These distributions
`are combined mathematically to create a probability esti(cid:173)
`mate of being inside or outside.
`
`It is noted other kinds of contailllllent within indoor
`[0037]
`and outdoor locations are contemplated to enhance the
`accuracy of inferences about indoor and outdoor, and such
`considerations as the role of the sensed temperature at a
`device, and the typical indoor temperatures, and the current
`measured outdoor temperature. For example, it can be
`considered whether the device is likely to be inside an
`automobile based on the signals from accelerometers, and
`thus interpret appropriately the signals about the difference
`between typical indoor temperatures and the current outdoor
`temperature in a region. Likewise, the output of accelerom(cid:173)
`eters, the time of day and the amount of light reaching a
`photosensor on a device, as well as the differences in the
`temperatures on different facades of a device via the use of
`multiple temperature sensors are within contemplation of the
`subject invention to determine whether a device is in a
`person's pocket, thus changing the interpretation of differ(cid:173)
`ences between the measured outdoor temperature and indoor
`signals, and inferences about the overall context.
`
`[0038] FIG. 1 illustrates a system 100 that determines
`device context in accordance with the subject invention. The
`system 100 includes one or more sensors 102 (denoted
`CONTEXT SENSOR 1, CONTEXT SENSOR2 , ...
`, CON(cid:173)
`TEXT SENSORN) that measure physical differences
`between inside and outside envirolllllents. For example, one
`sensor can be a temperature sensing device. Another can be
`a pressure sensing device for sensing changes in altitude.
`The one or more sensors 102 communicate sensor data to a
`context computing component 104 over a pathway 106 for
`processing and deriving the probability estimate. The one or
`more sensors 102 and context computing component 104
`can be employed in the device such that all sensing and
`processing is performed conveniently and quickly. Note
`however, that such one or more sensors 102 need not be in
`the same enclosure as the context computing component
`104, but can be operated in wireless collllllunications there(cid:173)
`with over the pathway 106.
`
`[0039]
`In one implementation of the context architecture,
`a GPS receiver can be automatically turned off or put into a
`mode where the device only turns on periodically to see if
`it can sense GPS signals, and if it does not sense GPS, turns
`itself off after the brief periodic probes, when it is deter(cid:173)
`mined that it is likely that the device is inside a building,
`because GPS does not typically work inside, thus saving
`power. Other sensing, via ambient sources or via specialized
`embedded sensing can be used to guide policies for chang(cid:173)
`ing a power-saving policy. For example, accelerometers can
`be combined with the inference and such policies as turning
`off a GPS receiver because the absence of sensed motion
`after a system is determined to be inside means that it is
`likely that the system is still inside. Sensing motion via
`accelerometers or the sensing of sudden changes in light, or
`changes in such ambient signals as the strength of wireless
`signals ( e.g., IEEE 802.11 ), the strength of commercial AM
`or FM radio transmissions, or even changes in the amplitude
`of electromagnetic hum from nearby power systems or lines,
`
`can raise the likelihood of a context changing from inside to
`outside, and thus can be used to tum on a GPS system or
`temporarily increase the frequency of probes for GPS sig(cid:173)
`nals, in a device that had either been turned off or put into
`a low-frequency intennittent probe mode.
`
`[0040] Additionally, knowing a person's context can be
`important for invoking automatic behaviors on the device.
`As one example, if a person is scheduled for a meeting that
`may or may not be attended, but the context is that he/she is
`outside, the person is likely not in a meeting, indicating that
`the person may be less busy than he or she would have been
`if they were attending an indoor meeting that is scheduled on
`their calendar. Thus, the person may be more available for
`receiving a telephone call on that person's cell phone.
`
`[0041]
`In another example, the context architecture can be
`usefol for adding metadata to digital photos potentially
`serving as a way to filter photos in a search, and as a
`component of higher-level context inference for ubiquitous
`computing. For example, an assertion about whether a
`picture was taken indoors or outdoors or a likelihood that a
`user was indoors can be encoded in metadata, and if indoors,
`a database of potential locations based on the latest sensed
`GPS location can be used to add a single or multiple
`candidate locations to the metadata. In one conception, such
`metadata can be used to allow user's to disambiguate a
`location for a set of pictures by changing a location guess to
`an assertion for a set of images. Such metadata about indoor
`and outdoor likelihoods can be combined with image-based
`classification about whether a picture represents an indoor or
`outdoor scene.
`
`[0042] The availability of GPS signals is also a valuable
`signal about whether a user is indoors or outdoors. However,
`the lack of GPS signals can occur outside as well as inside
`because of GPS signal "shadows" caused by obstructions
`such as a building. The lack ofGPS signals can be combined
`with historical maps of GPS shadows that have been logged
`in the past to guide the interpretation of the lack of GPS
`signals in inferring whether or not a device is inside or
`outside. Such signals can be used in conjunction with other
`ambient signals.
`
`[0043] Referring now to FIG. 2, a methodology for dif(cid:173)
`ferentiating between inside and outside in accordance with
`the invention is described. While, for purposes of simplicity
`of explanation, the one or more methodologies shown
`herein, e.g., in the form of a flow chart, are depicted and
`described as a series of acts, it is to be understood and
`appreciated that the methodology(s) are not limited by the
`order of acts, as some acts may, in accordance with the
`invention, occur in a different order and/or concurrently with
`other acts from that shown and described herein. For
`example, those skilled in the art will understand and appre(cid:173)
`ciate that a methodology could alternatively be represented
`as a series of interrelated states or events, such as in a state
`diagram. Moreover, not all illustrated acts may be required
`to implement a methodology in accordance with the inven(cid:173)
`tion.
`
`[0044] At 200, context data of a first context is deter(cid:173)
`mined. At 202, a confidence estimation is computed from the
`second context data. At 204, context data of a second context
`is determined. At 206, a confidence estimation is computed
`from the second context data. At 208, the an1bient data ( e.g.,
`temperature is detem1ined via the device. At 210, a prob-
`
`IPR2020-01192
`Apple EX1019 Page 18
`
`

`

`US 2006/0167647 Al
`
`Jul. 27, 2006
`
`4
`
`ability estimation is computed which provides an infere~ce
`of whether the device is inside or outside. At 212, the device
`or its components are then operated accordingly.
`
`[0045] FIG. 3 illustrates a device 300 that employs one or
`more sensors and the context computing component 104 for
`inside/outside determination. A variety of sensors can be
`used collectively and/or as subsets thereof to facilitate
`discriminating inside from outside. At 302, a temperature
`sensor is used-inside building temperatures are nomially
`artificially maintained in a range comfortable for their
`human occupants. Outside temperatures vary with location
`and the local weather, and can be determined by consulting
`updated temperature data maintained on the Internet. The
`device measures ambient temperature and can consult a web
`service to determine the outside temperature at its location.
`
`[ 0046] Location data can be. manually ~ntered. or deter(cid:173)
`mined from GPS or other locat1011-measurmg device. If the
`measured ambient temperature is significantly different from
`the outside temperature, and if the measured ambient tem(cid:173)
`perature is in the range of normal inside tempera~res, then
`the device is likely inside. If the measured ambient tem(cid:173)
`perature is close to the local outside temperature, and if this
`temperature is significantly different .fro1? t~e normal .range,
`of inside temperatures, then the device 1s likely outside. I±
`the measured ambient and outside temperatures are in the
`range of normal inside temperatures, then the inside/outside
`determination is uncertain.
`
`[0047] At 304, light frequency can be measured. The c?lor
`of typical inside and outside light is different. Outside,
`sunlight is received directly or filtered through clouds or
`haze. Inside, sunlight is filtered through windows, and much
`inside light is generated artificially. A sensor tha~ measures
`the local illumination spectrum can serve as an mput to an
`a]aorithm that reasons about typical spectra of inside and
`light to discriminate inside from outside. At 306,
`o;side
`radio frequency information can be sensed. Buildings are
`normally bathed in 60 Hz signals or "hum" (50 Hz in so~e
`countries) that is generated inadvertently by alt~rnatmg
`current (AC) carrying wiring and devices. Except for near
`power lines, outside does not experience this hum as
`strongly. Thus, measuring the strength of 50/60 Hz hum can
`be used to discriminate inside from outside.
`
`[0048] At 308, a change in altitude can be used to de~er(cid:173)
`mine if the device is inside or outside. A pressure sensmg
`device ( e.g., a barometer) can be used as a means for testing
`altitude. If the altitude differentiation changes in a relatively
`short period of time, the device is likely in a structure. At
`310, imaging infonnation can be captured ~nd pro~esse~
`using a camera. Inside, looking up normally g!ves a v!ew o±,
`a ceiling, while outside looking up normally gives a view of
`the sky. Thus, computer vision techniques can be used to
`discriminate the color and normally structured patterns of an
`inside ceiling from the color and unstructured patterns of the
`sky based on a can1era image taken from the device.
`
`[0049] At 312, GPS information can be utilized in at least
`two ways. GPS typically fails inside