Exhibit 2008
`Exhibit 2008
`
`

`
`On Context Awareness for Multisensor
`Data Fusion in IoT
`
`Shilpa Gite and Himanshu Agrawal
`
`Abstract With the advances in sensor technology, data mining techniques and the
`internet, information and communication technology further motivates the devel-
`opment of smart systems such as intelligent transportation systems, smart utilities
`and smart grid. With the availability of low cost sensors, there is a growing focus on
`multi-sensor data fusion (MSDF). Internet of Things (IoT) is currently connecting
`more than 9 billion devices. IoT includes the connectivity of smart things which
`focuses more on the interactions and interoperations between things and people.
`Key problem in IoT middleware is to develop efficient decision level intelligent
`mechanisms. Therefore, we focus on IoT middleware using context-aware mech-
`anism. To get automated inferences of the surrounding environment, context -aware
`concept is adopted by computing world in combination with data fusion. We
`conduct a comprehensive review on context awareness for MSDF in IoT and
`discuss the future directions in the area of context-aware computing.
`
`Keywords Context-aware system Multisensor data fusion Dempster–Shafer
`
`theory, IoT
`
`1 Introduction
`
`The concept of the internet of things (IoT) originated in the Auto-ID Center at the
`Massachusetts Institute of Technology in 1999 [1]. Kevin Ashton had imagined a
`world in which all electronic devices are networked and every object, whether
`physical or electronic, is electronically tagged with information applicable to that
`object. The underlying aim of this concept
`is the achievement of pervasive
`
`S. Gite (&) H. Agrawal
`CS/IT Department, SIT, Symbiosis International University, Pune, India
`e-mail: shilpa.gite@sitpune.edu.in
`
`H. Agrawal
`e-mail: himanshu.agrawal@sitpune.edu.in
`
`© Springer India 2016
`S.C. Satapathy et al. (eds.), Proceedings of the Second International
`Conference on Computer and Communication Technologies, Advances
`in Intelligent Systems and Computing 381, DOI 10.1007/978-81-322-2526-3_10
`
`85
`
`

`
`86
`
`S. Gite and H. Agrawal
`
`connections between the internet and objects around us. It is perfect assimilation of
`real-world objects with logical things [2].
`Multi-sensor data fusion system is analogous to human who can sense the
`environment with the help of their sensory organs like nose, ears, skin, etc. and
`make correct inferences about their surroundings [3]. Multisensor data fusion refers
`to the comprehensive fusing of sensory data from multiple sensors and related
`information in order to provide more reliable and accurate information that could be
`achieved using a single, independent sensor [4]. Sensor fusion technology was
`primarily developed for Military surveillance research and robotics by US DoD.
`Later, it has got commercially wider acceptance in the areas, such as intelligent
`transport system, geographic information, land and ocean surveillance, robotics,
`data and information security, medical surveillance, diagnosis, etc. [5, 6]. The only
`way to gain the required amount of information with the expected intelligence is
`viable with the help of multisensors data fusion approach [7].
`Paper is organized as follows: In Sect. 2, MSDF and its various techniques are
`studied thoroughly. Section 3 deals with context-aware systems and its application
`scenario. In Sect. 4, we present Intelligent MSDF model, which takes decisions on
`user’s behalf depending on current context. In Sect. 5, we conclude and provide
`future directions.
`
`2 Overview of Multisensor Data Fusion Techniques
`
`MSDF plays a key role in providing improved probability of detection, extended
`spatial and temporal coverage, reduced ambiguity, and improved system reliability
`and robustness [7]. Being multilevel process, it helps users to make decisions in
`complicated scenarios. It can have steps like automatic detection of objects, asso-
`ciation and correlation with existing things, future estimation, and combination of
`data from single and multiple information sources [8].
`
`Multisensor data fusion can be performed [9] by using four main ways:
`
`(a) Using Kalman Filter–Kalman filter, which is named after Rudolf E. Kalman
`who proposed a linear data filtering algorithm in his famous paper in 1960
`[10]. This is considered as one of
`the most well-known methods for
`data/sensor fusion algorithms. The results of data fusion process further can be
`improved with the help Advance Kalman Filter. Kalman equation is mostly
`used for location tracking, however, it has limitation in its capacity to identify
`the variety of contextual circumstances [11, 12].
`(b) Using Neural Network Theory—The neural network deals with the mathe-
`matical modeling of nonlinear data (neurons). It can process data based on
`accumulated learning experience which is past analyzed data and it takes more
`processing time to produce results. Swift context inference and fast response to
`rapid changes can hardly be the area of the theory [13].
`
`

`
`On Context Awareness for Multisensor Data Fusion in IoT
`
`87
`
`(c) Using Fuzzy Theory—Fuzzy logic deals with only two states: True/False.
`There are some problems in the multitarget tracking using multisensor data
`association with the conventional non-Bayesian or Bayesian method. In
`addition to some specific limitations of priori condition, such an association
`could not perform well under a high clutter tracking environment [14].
`(d) Using Bayesian Theory probability theory- It has the advantage of when
`multisource information is available. Allocating belief to subsets of the uni-
`versal set and a combination rule that is able to combine multisource evidence.
`This is an exceptional virtue for making decisions (Fig. 1).
`
`Sensor Level
`
`Feature Level
`
`Decision Level
`
`Fig. 1 Overview of MSDF techniques
`
`Table 1 Comparative analysis of MSDF techniques
`
`MSDF
`technique
`Kalman Filter
`[10–12]
`
`Extended
`Kalman filter
`[9]
`Covariance
`methods [18,
`19]
`Support
`vector
`machine [16,
`20]
`Artificial
`neural
`network [13]
`
`Advantages
`
`Limitations
`
`(cid:129) Easy implementation
`(cid:129) Efficient in terms of computation
`(cid:129) Most useful
`(cid:129) Sequential method
`(cid:129) More efficient than Kalman Filter
`(cid:129) Works well in nonlinear and
`non-Gaussian uncertainty problems
`(cid:129) Easy to use and implement
`(cid:129) Result accuracy
`(cid:129) Effective for decentralized data fusion
`(cid:129) Effective for heterogeneous sensor data
`(cid:129) Better results for inconsistent sensor
`data
`
`(cid:129) Accuracy issues
`(cid:129) Too many state vector
`components result in a significant
`computational overhead
`(cid:129) High computational complexity
`(cid:129) Unstable results in small time
`durations
`(cid:129) Conflicting behavior in some
`cases
`
`(cid:129) Sensitive to noise
`(cid:129) Text categorization problems are
`linearly separable
`
`(cid:129) Handles non-linear data
`(cid:129) Efficient for heterogeneous sensor data
`(cid:129) Good for high-level inference
`
`(cid:129) High complexity
`(cid:129) Unreliable results in some cases
`(cid:129) Slow context inference
`(continued)
`
`

`
`88
`
`Table 1 (continued)
`
`S. Gite and H. Agrawal
`
`MSDF
`technique
`Clustering
`analysis [21]
`
`Bayesian
`network
`[15, 22]
`
`Fuzzy theory
`[14]
`
`Dempster–
`Shafer theory
`[20, 23]
`
`Advantages
`
`Limitations
`
`(cid:129) Computational overhead
`(cid:129) Cluster head formation is a
`difficult process
`
`(cid:129) High memory requirements
`(cid:129) Consumes longer time before
`producing a result
`
`(cid:129) Results may be doubtful.
`(cid:129) More computational efforts
`
`(cid:129) Computational complexity
`increases as no. of sensors
`increase
`(cid:129) Belief and probability may differ
`sometimes
`
`(cid:129) Can perform well under a high clutter
`tracking environment.
`(cid:129) Generates result of multitarget tracking
`using multisensor data association
`(cid:129) Works well when multi-source
`information is available
`(cid:129) Can combine multi-source evidence
`(cid:129) Appropriate methods for high-level
`inference
`(cid:129) Decision based methods
`(cid:129) Fast response to rapid changes
`(cid:129) Useful for contextual representation
`(cid:129) Decision based method
`(cid:129) Deals with statistical problems or to
`model uncertainties
`(cid:129) Can draw good inference with less
`available sensor data
`(cid:129) Adds a new flavor to safety and
`reliability modeling compared to
`probabilistic approaches
`(cid:129) Powerful tool to assign uncertainty or
`ignorance to propositions
`(cid:129) More flexible than probabilistic
`approaches
`
`Table 1 shows a comparative analysis of various MSDF techniques which are
`most frequently used and efficiently utilized in all kind of applications nowadays.
`Each technique has some pros and cons, so depending on the application area,
`particular technique is used.
`
`3 Context-Aware Systems
`
`[16].
`in 1994 by Schilit and Theimer
`The term Context was defined first
`Context-aware systems are basically self-aware systems, which have the capability
`of judgement of the surroundings, inference mechanism and accordingly take
`decisions without human intervention. Depending on the current context, an
`automated response is generated. Figure 2 shows a big picture of context which
`covers all perspectives of a context. Context covers location of the objects or
`person, identify of a person, time information, activity, environment information or
`constitution information, etc. in order to get all kind of details about users’ sur-
`roundings [17].
`
`

`
`On Context Awareness for Multisensor Data Fusion in IoT
`
`89
`
`Time information
`Schedules &
`agendas
`
`Physiological info,
`Emotional status, Persona
`
`Geographical
`& Spatial info,
`Location
`
`Time
`
`Activities
`
`Behaviour
`
`Context
`
`Environment
`
`Location
`
`Identity
`
`Current Actions,
`Ongoing Tasks,
`Physical activities
`
`Fig. 2 Big picture of context
`
`Measurements of parameters,
`Technological & Resources
`
`User profile
`info, Social
`profile
`
`Context-aware process follows a series of subprocesses like context acquisition,
`context processing, and context usage. In context acquisition, with the help of
`sensors, data like pressure, temperature, etc. would be captured and further sent to
`improvement. In the next stage of context processing, four activities are involved,
`such as Noise removal, data calibration, context interpretation, and context pre-
`diction. In noise removal process, unnecessary sensor data and noise would be
`removed out with the help of filters like Kalman or Extended Kalman filter so that
`only required and useful data would be efficiently passed further. Data calibration
`deals with the updating or correction of a device. It is basically comparison with
`standard values which are previously established. Context interpretation deals with
`understanding and analysis of contextual data which is self-aware kind of process.
`And context prediction is basically prediction process which is based on prior
`estimates. Thus with these four activities, effective context processing takes place
`[16].
`Figure 3 shows real-time (actual) working diagram of context-aware process. In
`real-time scenario, how data is captured by various sensors and then further pro-
`cesses like data processing, data fusion, and context inference take place. After
`getting a model ready for context-aware system, it is applied to various application
`areas like intelligent transportation system (ITS), road-traffic monitoring, biomed-
`ical applications, robotics, surveillance etc.
`
`

`
`90
`
`S. Gite and H. Agrawal
`
`Fig. 3 Real-time diagram of context-aware process
`
`4 Proposed Intelligent MSDF System
`
`Hereby, we are proposing a novel MSDF approach for context-aware systems. The
`concept of applying intelligence to fused sensor data in order
`to produce
`context-aware inference has the novelty in computing field. Lower level data
`processing involves with data filtering whereas at the higher level intelligence on
`gathered and processed data takes place. It is basically a five-step process, which
`includes data collection, filtering process, situation analysis, decision making and
`inference mechanism process. Figure 4 shows the stepwise process details.
`In this paper, we propose an additional intelligent inference based on Dempster–
`Shafer theory (DST) step to the MSDF approach. As per our understanding, no
`
`Fig. 4 Novel context-aware approach for MSDF
`
`

`
`On Context Awareness for Multisensor Data Fusion in IoT
`
`91
`
`previous attempt has been made in this direction. DST helps to find out uncertainty
`factor along with 0/1 or T/F state. DST is very accurate in generating results so it is
`mostly used in context-aware systems. So we are making sensor data intelligent
`with context-aware systems that in addition to DST.
`Being more analogous to our human perception-reasoning processes, the DST
`adds a new zest to safety and reliability modeling as compared with other tech-
`niques. DST can be successfully combined with other techniques like ANN or
`fuzzy logic for more realistic results.
`
`4.1 DST in Context-Aware Systems
`
`Dempster–Shafer theory as a theory of evidence has to account for the combination
`of different sources of Evidence. Rule of Combination is an essential step in pro-
`viding such a theory. In this scenario belief functions are constructed by means of
`multivalued mappings [18].
`Bel and its dual, Pl (plausibility), are special kind of lower/upper probability
`functions.
`We can see it by defining PBel = {μ: μ(U) ≥ Bel(U) for all U  W} and showing
`that Bel = (PBel)* and Pl = (PBel)*.
`(cid:129) The lower bound of the confidence interval is the belief confidence, which
`accounts all evidences Ek that support the given proposition “A”:
`P
`miðEkÞ
`Belief i(A) =
`EkA
`(cid:129) The upper bound of the confidence interval is the plausibility confidence, which
`accounts all the observations that does not rule out the given proposition:
`Plausibility i(A) = 1 P
`miðEkÞ
`Ek\A¼u
`Combination of belief and plausibility leads to a mass function which handles
`uncertainty effectively to produce more correct inferences. Thus, context-aware
`systems can draw inferences on its own and take decisions on the user’s behalf.
`
`4.2 DST Versus Bayesian Networks
`
`Being most popular inference mechanisms, we tried to compare Bayesian networks
`with DST in Table 2. Both use probabilistic approach for data fusion. DST is
`basically advancement in Bayesian networks which handles the third aspect as
`uncertainty rather than just True/False or 0/1.
`
`

`
`92
`
`S. Gite and H. Agrawal
`
`Table 2 Comparison between inference mechanisms
`
`Parameters
`High-level
`fusion
`Decision making
`Uncertainty
`management
`Tolerant of
`imprecision
`Availability of
`probabilistic
`information
`Combination
`with other
`algorithms
`Effective use in
`Limitations
`
`Bayesian networks
`Yes
`
`Dempster–Shafer theory
`Yes
`
`Possible
`Limited extent
`
`More correct
`Effectively done
`
`Limited extent
`
`Possible
`
`Works best when
`Availability of full
`probabilistic information
`Not much useful
`
`Works better than any other probabilistic
`methods when lack of full prepositional
`information
`Produces more accurate and robust results
`
`Medical diagnosis
`Difficulty to deal with
`temporal variations
`
`Robotics
`Nonintuitive results when dependent
`belief functions
`
`Depending on the availability of probabilistic information, particular technique
`is chosen in context-aware systems. Here, we found that DST turns out to be
`superior over Bayesian networks when full probabilistic information is not
`available.
`
`5 Conclusion
`
`In this paper, we have proposed context-aware framework and have reviewed
`MultiSensor Data Fusion and internet of things (IoT). In the next 5 years, i.e., by
`2020, IoT is expected to connect more than 25 Billion devices. Therefore, there will
`be lot of research rigor in IoT middleware, which will require further attention in
`terms of the development of novel solutions. Context-aware computing is going to
`be the prime focus of many researchers to address various challenges in IoT
`middleware.
`In our paper, we have stated role of DST in context-aware systems to get correct
`inferences from the environment. Also we have compared DST with Bayesian
`networks which are an alternative option for probabilistic data handling. As
`ongoing work, we are focusing on the robustness aspect using DST which would
`generate more comprehensive view about the environment. To decrease the com-
`putational overhead of DST can also be considered as a research challenge. Another
`possible extension will be in the direction of the Hidden Markov Model. We will be
`investing on HMM-based model as our future work to address the uncertainty
`aspect.
`
`

`
`On Context Awareness for Multisensor Data Fusion in IoT
`
`93
`
`References
`
`1. Ashton, K.: That ‘Internet of Things’ Thing. RFID J. 22, 97–114 (2009)
`2. The Internet of Things: ITU Internet Reports (2005)
`3. Hall, D.: An introduction to multisensor data fusion. Proc. IEEE 85(1), 6–23 (1997)
`4. Luo, R.C.: Multisensor fusion and integration: theories, applications, and its perspectives.
`IEEE Sens. J. 11(12), 3122–3138 (2011)
`5. Hall, David: Mathematical Techniques in Multisensor Data Fusion, Boston. Artech House,
`MA (1992)
`6. An, S.-h., Lee, B.-H., Shin, D.-R.: A survey of intelligent transportation systems. In: Third
`International Conference on Computational
`Intelligence, Communication Systems and
`Networks (CICSyN), IEEE (2011)
`7. Khaleghi, B., Khamis, A., Karray, F.O., Razavi, S.N.: Multisensor data fusion: a review of the
`state-of-the-art. Inf. Fus. 14(1), 28–44 (2013)
`8. Miloslavov, Adelin, Veeraraghavan, Malathi: Sensor data fusion algorithms for vehicular
`cyber-physical systems. IEEE Trans. Parallel Distrib. Syst. 23(9), 1762–1774 (2012)
`9. Ribeir, M.I.: Kalman and extended Kalman filters: concept, derivation and properties (2004)
`10. Rodger, J.A.: Toward reducing failure risk in an integrated vehicle health maintenance system:
`A fuzzy multi-sensor data fusion Kalman filter approach for IVHMS. Expert Syst. Appl. 39
`(10), 9821–9836 (2012)
`11. Kalman, R.E.: J. Basic Eng. 82(1), 35–45 (1960)
`12. Kalman, R.E.; Bucy, R.S.: New results in linear filtering and prediction theory. J. Basic Eng.
`Trans. ASME 83, 95–108 (1961)
`13. Bothe, H.-H.-H.: Multivariate sensor fusion by a neural network model
`14. Stover, J.A., Hall, D.L.: A fuzzy-logic architecture for autonomous multisensor data fusion.
`IEEE Trans. Ind. Electron. 43(3), 403–410 (1996)
`15. Friedman, N., et al.: Using Bayesian networks to analyze expression data. J. Comput. Biol. 7
`(3–4), 601–620 (2000)
`16. Schilit, B., Adams, N., Want, R.: Context-aware computing applications. In: First Work shop
`on Mobile Computing Systems and Applications, WMCSA (1994)
`17. Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on context-aware systems. Int. J. Ad Hoc
`Ubiquitous Comput. 2(4), 263–277 (2007)
`18. Shafer, G.: A Mathematical Theory of Evidence. Princeton, University Press (1976)
`19. Robust Covariance Estimation for Data Fusion From Multiple Sensors João Sequeira. IEEE
`Trans. Instrum. Meas. 60, 3833–3844 (2011)
`20. Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 66–75 (1991)
`21. Yu, S., De Moor, B., Moreau, Y.: Clustering by heterogeneous data fusion: framework and
`applications Covariance consistency methods for fault-tolerant distributed data fusion (Jeffrey
`K. Uhlmann Science Direct. Inf. Fus. 4 201—215(2003)
`22. Banerjee, T.P.: Multi-sensor data fusion using support vector machine for motor fault
`detection science direct. Inf. Sci. 217 65–77 (2012)
`23. Shafer, G.: Probability judgement in artificial intelligence. In: Kanal, L.N., Lemmer, J.F. (eds.)
`Uncertainty in Artificial Intelligence. New York, Elsevier Science (1986)
`24. Žontar, Rok: Marjan Heričko. Taxonomy of context-aware systems, Ivan Rozman (2012)