J. Sens. Actuator Netw. 2012, 1, 217-253; doi:10.3390/jsan1030217
`
`OPEN ACCESS
`Journal of Sensor
`and Actuator Networks
`ISSN 2224-2708
`www.mdpi.com/journal/jsan/
`
`Review
`Sensor Mania! The Internet of Things, Wearable Computing,
`Objective Metrics, and the Quantified Self 2.0
`Melanie Swan
`
`MS Futures Group, P.O. Box 61258, Palo Alto, CA 94306, USA; E-Mail: m@melanieswan.com;
`Tel.: +1-650-681-9482; Fax: +1-504-910-3803
`
`Received: 4 September 2012; in revised form: 31 October 2012 / Accepted: 31 October 2012 /
`Published: 8 November 2012
`
`Abstract: The number of devices on the Internet exceeded the number of people on the
`Internet in 2008, and is estimated to reach 50 billion in 2020. A wide-ranging Internet of
`Things (IOT) ecosystem is emerging to support the process of connecting real-world
`objects like buildings, roads, household appliances, and human bodies to the Internet via
`sensors and microprocessor chips that record and transmit data such as sound waves,
`temperature, movement, and other variables. The explosion in Internet-connected sensors
`means that new classes of technical capability and application are being created. More
`granular 24/7 quantified monitoring is leading to a deeper understanding of the internal and
`external worlds encountered by humans. New data literacy behaviors such as correlation
`assessment, anomaly detection, and high-frequency data processing are developing as
`humans adapt to the different kinds of data flows enabled by the IOT. The IOT ecosystem
`has four critical functional steps: data creation, information generation, meaning-making,
`and action-taking. This paper provides a comprehensive review of the current and rapidly
`emerging ecosystem of the Internet of Things (IOT).
`
`Keywords: Internet of Things; sensors; objective metrics; quantified self; personal metrics;
`high-tech hardware; integrated sensor platforms; multi-sensor platforms; information
`visualization; health Internet of Things
`
`1. Introduction: The Rapid Approach of the Internet of Things
`
`1.1. What is the Internet of Things?
`
`There are several definitions of the Internet of Things (IOT). One that is salient for how the term is
`currently in use is provided by the U.S. National Intelligence Council: “The “Internet of Things” is the
`
`Petitioner Apple Inc. – Ex. 1029, cover p. 1
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`218
`
`general idea of things, especially everyday objects, that are readable, recognizable, locatable,
`addressable, and controllable via the Internet - whether via RFID, wireless LAN, wide-area network,
`or other means [1].” A key point is that while the most familiar Internet-connected devices are
`computers such as laptops, servers, smartphones, and tablets (e.g., iPads, etc.), the IOT concept is
`much broader. In particular, everyday objects that have not previously seemed electronic at all are
`starting to be online with embedded sensors and microprocessors, communicating with each other and
`the Internet. This includes items such as food, clothing, household appliances, materials, parts,
`subassemblies, commodities, luxury items, landmarks, buildings, and roads. It is estimated that 5% of
`human-constructed objects currently have embedded microprocessors [2]. These tiny microprocessor
`chips and sensors record and transmit data such as sound waves, temperature, movement, and other
`variables. Other terms for the Internet of Things include Internet-connected devices, smart connected
`devices, wireless sensor networks, machines and devices communicating wirelessly, ubiquitous
`computing, ambient intelligence, and smart matter.
`One way of characterizing the IOT is by market segment where there are three main categories:
`monitoring and controlling the performance of homes and buildings, automotive and transportation
`applications, and health self-tracking and personal environment monitoring. Some of the basic IOT
`applications underway in the connected home and building include temperature monitoring, security,
`building automation, remote HVAC activation, management of peak and off-peak electricity usage, and
`smart power meters. Worldwide smart power meter deployment is expected to grow from 130 million in
`2011 to 1.5 billion in 2020 [3]. Some of the many automotive and transportation IOT uses include the
`Internet-connected car (syncing productivity, information, and entertainment applications), traffic
`management, direction to open parking spots, and electric vehicle charging. It is estimated that 90% of
`new vehicles sold in 2020 will have on-board connectivity platforms, as compared with 10% in
`2012 [3]. In industrial transportation, train operators like Union Pacific use IOT infrared sensors,
`ultrasound, and microphones to monitor the temperature and quality of train wheels [4]. One of the
`biggest IOT growth areas is measuring individual health metrics through self-tracking gadgets, clinical
`remote monitoring, wearable sensor patches, Wi-Fi scales, and a myriad of other biosensing
`applications. Two high-profile prizes in this area are designed to spur innovation, the $10 million
`Qualcomm Tricorder X Prize for the development of a handheld device to non-invasively monitor and
`diagnose health conditions in real-time [5], and the $2.25 million Nokia X Challenge for sensor
`technology that can bring about new ways to monitor, access, and improve consumer health [6].
`The rapid growth of the IOT is pictured in Figure 1, comparing the number of devices on the
`Internet to the number of people on the Internet. Connected devices surpassed connected people in
`2008. Cisco estimates that by 2020 there will be 50 billion connected devices, 7 times the world’s
`population [7]. Similarly, the Connected Life initiative sponsored by the GSMA (GSM Association, an
`industry-association for worldwide mobile operators) found that in 2011, there were 9 billion
`total Internet-connected devices (compared to the total human population of less than 7 billion),
`two-thirds (6 billion) of which were mobile, and estimates that in 2020, there will be 24 billion total
`Internet-connected devices, 12 billion mobile [8]. Moreover, these 24 billion Internet-connected
`devices are estimated to have an economic impact of over $4.5 trillion in 2020 [8].
`
`Petitioner Apple Inc. – Ex. 1029, p. 218
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`219
`
`Figure 1. The accelerated growth of Internet-connected devices [7].
`
`1.2. The Internet of Things Ecosystem
`
`The IOT is connecting real-world objects to the Internet with tiny sensors. A number of functional
`layers are starting to be visible in the developing ecosystem as pictured in Figure 2, starting with data
`generation, and moving to information creation, and then meaning-making and action-taking. The
`broad brush categories are the Hardware Sensor Platform layer, the Software Processing layer, the
`human-readable Information Visualization layer, and the human-usable Action-Taking layer.
`
`Figure 2. Processes and Layers in the Internet of Things Ecosystem.
`
`2. Hardware Sensor Platforms
`
`One of the biggest drivers of the IOT is the increasing number of low-cost sensors available for many
`different kinds of functionality. Some of the standard sensors include movement (via accelerometer),
`sound, light, electrical potential (via potentiometer), temperature, moisture, location (via GPS), heart rate
`and heart rate variability, and GSR (galvanic skin response or skin conductivity). Other sensors include
`ECG/EKG (electrocardiography to record the electrical activity of the heart), EMG (electromyography to
`measure the electrical activity of muscles), EEG (electroencephalography to read electrical activity along
`the scalp), and PPG (photoplethysmography to measure blood flow volume).
`These sensors are included in a variety of devices and solutions. The trend is moving towards
`multi-sensor platforms that incorporate several sensing elements. For example, the standard for the
`next-generation of personalized self-tracking products appears to be some mix of an accelerometer,
`GSR sensor, temperature sensor, and possibly heart rate sensor (from which heart rate variability may
`be calculated). Some recognized first-generation quantified tracking devices and applications include
`the Fitbit, myZeo, BodyMedia, MapMyRun, RunKeeper, MoodPanda, Nike Fuelband, The Eatery,
`
`Petitioner Apple Inc. – Ex. 1029, p. 219
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`220
`
`Luminosity’s Brain Trainer, and the NeuroSky and Emotiv brain-computer interfaces (BCI). One
`website listing of quantified tracking devices is maintained by the Quantified Self community
`(http://quantifiedself.com/guide). As of October 2012, over 500 items were listed, with keywords from
`the text descriptions visualized in Figure 3, showing that the tracking tools having a number of
`quantitative and qualitative dimensions including social, technological, and technical capabilities.
`
`Figure 3. Visualization of Keywords Used in Quantified Tracking Device Descriptions.
`
`The next generation of IOT quantified tracking devices is visible in product announcements, many
`of which fall into the category of wearable electronics and/or multi-sensor platforms. These products
`include smartwatches, wristband sensors, wearable sensor patches, artificial reality-augmented
`glasses, brain computer
`interfaces, wearable body metric
`textiles
`(such as Hexoskin
`(http://www.hexoskin.com/). Other important categories include smartphone applications and their
`enhancements, and environmental monitoring and home automation sensors.
`
`2.1. Smartwatches and Wristband Sensors
`
`Wearable computing as a category is being defined by smartwatches and wristband sensors,
`augmented eyewear such as Google’s Project Glass, and wearable textiles. A new product category,
`the smartwatch, effectively a wearable connected computer, is expected to have several product
`launches in the fall of 2012. This new generation of programmable watches includes the Pebble watch,
`the Basis watch, the Contour Watch from Wimm Labs, and the Sony SmartWatch as pictured in
`Figure 4. The Pebble watch (preorder: $150, http://getpebble.com/) has received a lot of attention as
`the developer was unable to raise funding via traditional means but then became the highest-funded
`Kickstarter project (a crowdfunding website) to date, raising $10 million from an initially-sought $100,000
`(http://www.kickstarter.com/projects/597507018/pebble-e-paper-watch-for-iphone-and-android).
`The Pebble watch provides Internet-connected applications like notification of incoming calls,
`email, and message alerts using Bluetooth to connect to smartphones. The Basis watch (preorder:
`$199, https://mybasis.com/) is a quantified self-tracking watch, a multi-sensor platform with a 3D
`accelerometer, heart rate monitor, temperature sensor, and GSR sensor. Like the Fitbit, it does not sync
`in real-time but only when connected to a computer. Wimm Labs intends to provide an open-platform
`Android operating system-based alternative to the Pebble watch. The Wimm Labs Contour Watch
`(~$200, http://www.wimm.com/) is envisioned to enable a wide range of mobile, sports,
`
`Petitioner Apple Inc. – Ex. 1029, p. 220
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`221
`
`health, fashion, finance, consumer electronics, and other applications. The Sony SmartWatch,
`offering Twitter, email, music, and weather
`information
`is currently available
`($175,
`http://www.sonymobile.com/gb/products/accessories/smartwatch/).
`In a potential extension of
`Google’s Project Glass (augmented reality eyewear), Google has patented smartwatch technology for
`an augmented reality smartwatch with two flip-up screens, a touchpad, and wireless connectivity [9].
`
`Figure 4. Smartwatches: A New Product Category of Programmable Watches: the Pebble
`Watch (from Pebble Technology Corporation), the Basis Watch, the Contour Watch from
`Wimm Labs, and the Sony SmartWatch.
`
`Wristband sensors are a predecessor to smartwatches and remain a successful product category on
`their own. One of the first examples of wristband sensors was using accelerometers to measure steps
`taken with products like the Nike Sense. Current examples continue to feature accelerometry and
`include the Nike Fuelband ($149, monitoring steps taken), the Jawbone UP wristband and iPhone app
`($99, tracking steps taken, distance, calories burned, pace, intensity level, active versus inactive time,
`and GPS), the Adidas MiCoach ($70, providing heart rate monitoring, real time digital coaching,
`interactive
`training, and post-workout analysis of pace, distance, and stride rate). Three
`next-generation products add new functionality to the standard metrics of total steps taken, distance,
`and calories. The Mio Active ($119, http://www.mioglobal.com/) adds heart rate, either with or
`without a chest strap. The LarkLife ($149, http://www.lark.com/) identifies type of activity, allows
`single-button press diet tracking, measures sleep, and uses the combined metrics to make personalized
`recommendations about changes a user can make to feel better [10]. The Amiigo ($119,
`http://www.amiigo.co/) wristband and shoe clip also measure the type of exercise, plus body
`temperature and blood oxygen levels through an infrared sensor [11]. Other sensor platforms also
`focus on fitness and athletic training, for example Somaxis (www.somaxis.com) with ECG and EMG
`muscle and heart sensors and GolfSense ($130, http://www.golfsense.me/) where users attach a
`wrist-based sensor unit to a golf glove. The unit has two accelerometers and other sensors that collect
`and transmit data wirelessly for real-time feedback. Multi-sensor wristband devices are also in
`development for clinical use, for example in epilepsy. One team created a wristband to detect
`convulsive seizures through electrodermal activity and accelerometry, a useful improvement over
`lab-based EEG methods as the device can be worn continuously [12].
`
`Petitioner Apple Inc. – Ex. 1029, p. 221
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`222
`
`2.2. Wearable Sensors and Monitoring Patches
`
`Another new product category that could quickly become commonplace is wearable sensors,
`low-cost disposable patches that are worn continuously for days at a time and then discarded. It is
`estimated that 80 million wearable sensors will be in use for health-related applications by 2017, an
`eight-fold increase over today [13]. The concept is not new, nicotine patches for smoking cessation are
`a familiar concept, but the extended on-board sensor functionality is an important innovation. The next
`generation of patches moves away from standard transdermal passive diffusion technology, and instead
`uses rich sensor technology to enable patches to transmit information wirelessly, and possibly engage
`in two-way communication for real-time adjustments. One of the most exciting potential developments
`in wearable patches is Sano Intelligence’s continuous blood chemistry monitoring patches,
`characterized as a $1 API for the bloodstream, and estimated to be available in mid-2013. The
`disposable patch (one-week use) has been demonstrated to measure blood glucose and potassium
`levels, and aims to measure a full metabolic panel, including kidney function and electrolyte balance.
`Further, there are enough probes on the wireless, battery-powered chip to continuously test up to a
`hundred different samples [14].
`
`Figure 5. Continuous Wearable Sensor Patches for Blood Chemistry and Vital Signs
`(mc10) and Cardiac Rhythm (iRhythm’s Zio Patch).
`
`A promising concept pioneered by mc10 is stretchable electronic tattoos for the continuous
`monitoring of vital signs with flexible electronics patches as shown in Figure 5. These stretchable
`electronics track and wirelessly transmit information such as heart rate, brain activity, body
`temperature, and hydration level, and may be available to athletes in the fall of 2012 [15]. Wearable
`sensor patches are also useful for heart monitoring and have again allowed an improvement over
`current methods. The Zio Patch from iRhythm (two-week use) can be worn to monitor cardiac rhythm
`and warn of arrhythmias (Figure 5). Another interesting example of new patch technology is a
`continuous blood pressure monitoring patch from Sense A/S. Instead of the cumbersome pressure cuff,
`there is a small arm patch with electrodes that sense the changing impedance of tissue around a vessel
`and convert it into a blood pressure reading via a waistband sensor unit [16].
`One of the classic use cases for wearable patches is the continuous glucose monitor (CGM) worn by
`diabetics and other self-trackers. New developments mean that the current state-of-the-art technology
`is available in several CGM solutions where an under-the-skin continuous glucose monitor uses a
`sensor and transmits glucose readings every 1–5 minutes to an external receiver or insulin pump [17].
`
`Petitioner Apple Inc. – Ex. 1029, p. 222
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`223
`
`Also promising is the idea of using the glucometer as a platform. Chemists have developed a method
`to bind short segments of DNA to a large number of potential molecules that might be present in
`blood, water, or food. The DNA segments also bind to the enzyme invertase so that glucose is
`produced if the target molecules are present, and could then be read easily with a $20 drugstore
`glucometer. So far, this glucometer-as-a-platform method has been used to detect cocaine, interferon,
`adenosine, and uranium [18,19].
`
`2.3. Continuous IOT Monitoring and Advances in Blood Testing 2.0
`
`A key expectation of IOT devices is that they allow for continuous monitoring and connected
`real-time data transmission, ideally with real-time feedback and personalized recommendations. The
`continuous remote monitoring of patients is a significant market here, estimated to be $21 billion in
`2016 as compared with $9 billion in 2011 [20]. One example of a multi-sensor remote monitoring
`platform is the FDA-cleared BodyGuardian from Preventice which integrates ECG, heart rate,
`respiration rate, and physical activity data. Another example of continuous monitoring wearable
`sensors is the FDA-cleared Visi Mobile from Sotera Wireless (Figure 6) which continually monitors
`vital signs such as ECG, heart rate, respiration, and temperature.
`Another example of the now-expected continuous monitoring and automated data transmission and
`feedback functionality is available in the AgaMatrix iBGStar blood glucose monitoring system. This
`was the first traditional glucometer to connect directly to an iPhone app (Figure 6), to allow transitory
`readings to be recorded, stored into longitudinal profiles, and shared. The Proteus digital health system
`is another example of the now-expected continuous transmission functionality, effectively defining a
`new category of medical device. Here, a biodegradable ingestible sensor is attached to a pill that
`transmits data regarding the body’s interaction with the medication to a wearable patch (Figure 6).
`
`Figure 6. Continuous Monitoring: Sotera’s Visi Mobile, AgaMatrix
`iBGStar
`Smartphone-Connected Glucometer, Proteus Digital Medicine Pill Consumption Tracking
`System, and Next-Generation Dried Blood Spot Testing from ZRT Laboratory.
`
`Blood testing is another area where sensor technology and other innovations are speeding progress.
`A key advance in user-friendly direct-to-consumer blood testing is dried blood spot technology.
`Instead of going to a lab, consumers can prick their fingers at home with a lancet, put a series of blood
`spots on a laboratory card, mail in the card for analysis, and view the results on the web (Figure 6).
`One company offering dried blood spot testing is ZRT Laboratory (http://w3.zrtlab.com/), however
`given the lack of available alternatives like the announced $1 Sano Intelligence API-for-the-bloodstream
`
`Petitioner Apple Inc. – Ex. 1029, p. 223
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`224
`
`patches, pricing is still commensurate with lab-drawn blood tests. This could change quickly as new
`market entrants have
`their eye on
`the $65 billion
`lab
`testing market
`(where
`the
`direct-to-consumer segment is growing 15%–20% per year [21]). One recently-launched consumer
`proteomics startup, Talking20 (referring to the 20 amino acids that make up the proteins in the body,
`www.Talking20.com), is offering dried blood spot testing at a significant discount, $10 per card,
`testing five markers, vitamins B1 and B9, and hormones: testosterone, estradiol, and progesterone.
`For clinical diagnosticians, there is a new point-of-care blood testing solution, the i-STAT System
`from Abbott Labs (http://www.abbottpointofcare.com/). This is a handheld blood analyzer that
`provides real-time lab-quality results in minutes and measures 25 different blood markers including
`hemoglobin, hematocrit, glucose, potassium, calcium, pH, urea nitrogen (BUN), creatinine, and
`lactate. Results can be used immediately onsite and transmitted to physicians for real-time
`consultation. Another innovative direction in blood testing focuses on developing tests for pathologies
`that were not previously measurable. One example is the newly available Ridge Diagnostics blood test
`for depression, measuring the serum levels of nine biomarkers (alpha1 antitrypsin, apolipoprotein CIII,
`brain-derived neurotrophic factor, cortisol, epidermal growth factor, myeloperoxidase, prolactin,
`resistin, and soluble tumor necrosis factor alpha receptor type II) [22]. The test is expensive, available
`on the market for $745 (http://www.ridgedx.com/). While the test is intended for depression diagnosis,
`it might be interesting to measure the same markers in a predictive progressive manner to see if
`pathologies like depression might be detected earlier and prevented. Blood markers are predictive of
`other conditions. For example, with diabetes, hemoglobin 1AC levels are predictive of condition onset
`by ten years, and could be a target for early preventive intervention [23]. Depression tendency or
`pre-onset might possibly be similarly detected and managed.
`
`2.4. Brain-Computer Interfaces (BCIs), Neuro-Sensing, and Emotion-Mapping
`
`In the coming era, it may be possible to have a much greater understanding of the brain. There
`could be numerous useful applications from this, for example mental performance optimization
`techniques and a variety of emotion reading, mapping, and management programs. An early sensor
`technology for obtaining brain data is the consumer EEG (also called the brain-computer interface
`(BCI)). Some of the first-generation consumer EEG rigs are pictured in Figure 7 and include the
`14-node EEG Emotiv ($299, http://www.emotiv.com), the single-node EEG NeuroSky ($99,
`http://www.neurosky.com/), and the sleep quality tracker myZeo, also essentially an EEG ($99,
`http://www.myzeo.com/). The Emotiv and the NeuroSky have been used for different applications
`such as improving attention and meditation, and video game performance. At least one academic study
`has validated the performance of consumer EEGs, finding that for 6 of 8 participants, the responses to
`the traditional EEG and the Emotiv were similar [24]. Emotions were mapped to a standard four
`quadrant diagram of arousal and valence (reflecting the intensity and positive or negative charge of the
`experience). An established image research library (IAPS) was employed, although the researchers
`noted that both methods were still close to baseline error rates, in other words that emotion mapping
`remains a challenging problem. Another single-node EEG, the iBrain from NeuroVigil, is available to
`academicians and claims to be better than current clinical EEG methods due to a clustering software
`
`Petitioner Apple Inc. – Ex. 1029, p. 224
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`225
`
`algorithm, SPEARS (Sleep Parametric EEG Automated Recognition System), the company has
`developed for sleep analysis (http://www.neurovigil.com/spears/).
`
`Figure 7. First- and Second-Generation Consumer EEGs: Emotiv, NeuroSky, and myZeo,
`and InteraXon and Axio.
`
`The next generation of consumer EEGs may take advantage of a variety of sensor technology
`improvements particularly in Bluetooth low-energy data transmission and battery technologies.
`This could mean that more comfortable, unobtrusive, and visually-attractive wearable electronic
`brain monitors could be available to be worn 24/7 to continuously collect data and package
`it into useful real-time applications. At least two companies have planned second-generation
`consumer EEG products as pictured in Figure 7, InteraXon (http://www.interaxon.ca/) and Axio
`(http://www.axioinc.com/). Another company, Veritas Scientific (http://www.veritasscientific.com/),
`has developed a lie-detection device, the TruthWave, based on consumer-available EEG technology. A
`standard neuroscience technique is used to detect brain activity when a person’s face is recognized,
`registering the P300 response from a type of brain activity known as event related potentials (ERPs).
`Having access to continuous neural data could significantly open up the field for the development of
`interesting applications, for example in image recognition, emotion detection and intervention, and
`flow state performance management.
`A number of companies and academic labs are working to measure emotion, also known as affect.
`This is in some sense a digital implementation and extension of work by emotion research pioneer Paul
`Ekman (http://www.paulekman.com/), who developed the FACS (Facial Action Coding System), now
`known as FACE (Facial Expression, Awareness, Compassion, Emotions), to taxonomize every human
`facial expression. Two companies, Affectiva (http://www.affectiva.com/), and Affective Interfaces
`(http://www.affectiveinterfaces.com/), use computer webcams and eye-tracking technology to read
`facial microexpressions, mainly for the purpose of neuromarketing (e.g.; determining the biophysical
`response of participants to consumer brands or entertainment products like TV shows or movie
`endings). Affectiva uses a multi-sensor wristband that captures GSR, temperature, and accelerometry
`in addition to the webcam eye-tracking technology.
`Some other interesting applications of eye-tracking, not directly related to emotion sensing, are
`from Cardiio (http://www.cardiio.com/), who calculates heart rate from the camera on a mobile phone,
`and EyeTribe (http://theeyetribe.com/), who has created portable eye-tracking—software for
`controlling mobile devices with eye movements. Even without EEGs or eye-tracking, some degree of
`emotion-mapping such as stress levels may be detected with other measures obtained from sensors like
`GSR [25]. Academic labs are also using the expanding range of neurosensing technologies to extend
`emotion research, in particular, the Interdisciplinary Affective Science Laboratory at Northeastern
`
`Petitioner Apple Inc. – Ex. 1029, p. 225
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`226
`
`University (http://www.affective-science.org/) and the Neurophysiology and Empathy research group
`at the University of Parma (http://www.unipr.it/arpa/mirror/english/staff/gallese.htm).
`
`2.5. Smartphone and Smartphone Plus Peripheral
`
`Mobile is the platform, for many activities, initially for communication, then also for computing,
`and now for quantified tracking. As of October 2012, smartphone penetration was 78% in the US,
`which lags other countries such as Singapore (92%) [26]. One reason for this is that some markets
`have leapfrogged technology roll-outs to have continuous Internet access for the first time via
`smartphone. Even just basic voice functionality coupled with automated algorithms is resulting in
`useful next-generation IOT predictive applications, for example Parkinson’s disease detection. The
`Parkinson's Voice Initiative has a phone-based voice diagnostic that they claim is 98% effective. The
`user places a call and says ‘aaaaah.’ A machine learning program analyzes different voice qualities in
`the sample such as vocal tremor, strength, breathiness, and fluctuations in the jaw, tongue, and lips to
`assess the presence and severity of Parkinson’s disease [27]. Smartphones are also being used with
`external peripherals to be part of the IOT. Many devices have been attached to smartphones for novel
`applications as illustrated in Figure 8 such as AliveCor’s electrocardiogram (ECG) recorder for heart
`monitoring (http://alivecor.com/), MobiSante’s smartphone-based ultrasound
`imaging system
`(http://www.mobisante.com/), and the CellScope (http://cellscope.com/). The CellScope has a series of
`clip-on modules for the smartphone such as an otoscope (to look into the middle ear), and a
`dermascope (to capture magnified images of the skin). Relatedly, there are many examples available
`on the Internet for how to add a lens to a smartphone camera to turn it into a microscope.
`
`Figure 8. Internet of Things (IOT) Smartphone Peripherals and Next-Generation Wearable
`Computing: Consumer ECG, Mobile Ultrasound, CellScope, and Google’s Project Glass
`(per Antonio Zugaldia).
`
`As the trends in sensor miniaturization and functionality improvement continue, it is easy to
`envision that for some applications, the next-generation of sensortech could involve the direct
`integration of sensors into the smartphone platform instead of being an alongside hardware peripheral.
`One example of this is the LifeWatch V, an Android-based smartphone with the usual suite of sensors to
`measure ECG, body fat, heart rate, stress, temperature, blood saturation, and blood glucose levels [28].
`The idea of sensors built directly into hardware is also exemplified in Google’s Project Glass,
`projected for launch in late 2012. Project Glass defines a new category of wearable computing with
`augmented reality glasses (Figure 8) where a small camera and computing node mounted on the corner
`of the glasses can search the Internet and display real-time results right in front of the eyes. In the
`
`Petitioner Apple Inc. – Ex. 1029, p. 226
`
`

`

`J. Sens. Actuator Netw. 2012, 1
`
`227
`
`future, there could be special editions of Project Glass or other similar multi-sensor platforms with
`microprocessors embedded to merge a range of functionality like EEG, eye-tracking, heart rate, GSR,
`accelerometry, temperature, GPS, and Internet search and results display into a category-defining
`human augmentation platform for real-time information, communication, and personal feedback and
`performance optimization.
`
`2.6. Environmental Monitoring and Home Automation Sensors
`
`Quantified tracking in home automation and environmental monitoring is more established as a
`sector, but here too IOT innovations continue. Some are pictured in Figure 9 such as the Sensordrone
`from Sensorcon (http://www.sensorcon.com/), a successfully-funded Kickstarter project. The
`keychain-based sensor monitors the environment and transmits data via Bluetooth to any connected
`device. Applications are envisioned such as investigating air quality, carbon monoxide levels, potential
`gas leaks, and measuring a child’s temperature. More detailed applications include using the
`capacitance sensor as a stud finder, a liquid level monitor, or a proximity monitor [29]. More
`generally, it is now possible to use environmental sensors to measure a range of concerns including air
`quality, barometric pressure, carbon monoxide, capacitance, color, gas leaks, humidity, hydrogen
`sulfide, temperature, and light. Another project (supported by Cosm) is Flexibity Internet Sensors
`(http://www.flexibity.com/), an open sensor toolkit and Internet-connected platform for wireless home
`and environmental monitoring. Each sensor has a unique IPv6 address and can be accessed with a
`standard web call or via web services like Twitter.
`the Air Quality Egg
`include
`Other new home and environmental sensing solutions
`(http://airqualityegg.wikispaces.com/AirQualityEgg), also successfully funded with Kickstarter. This
`sensing device measures the air quality in the immediate environment, and offers the now-expected
`social layer to users—the ability to share data with an on-line community in real-time. In home
`automation, there is an active development community, one example of which is a Google-sponsored
`project, openHAB (the open Home Automation Bus http://code.google.com/p/openhab/). The project
`attempts to provide a universal vendor-neutral platform for integrating multiple hardware devices, bus
`systems, and interface protocols. A related solution is the Android appliance SmartyHome
`(http://www.mysmartyhome.com/), for home auto