lll lll a llDl DID lll DIII DD lll DID lll 100 mil Dll lulu ll Dl lli
`
`
`US oi.~i.wi,.i
`
`Urd S~a~r,
`~,) United
`1_)Patenl Application Publication (cid:9)
`KIM et al. (cid:9)
`
`
`
`(43) Pub. Date: (cid:9)l
`
`Ju
`
`. 11,
`
`z
`
`(54) APPARATUS FOR PROCESSING A SERVICE
`AND METHOD THEREOF
`
`(71) Applicant: LG ELECTRONICS INC., Seoul (KR)
`
`(72) Inventors: Jayoen KIM, Seoul (KR); Daeyol NA,
`Seoul (KR)
`
`(73) Assignee: LG ELECTRONICS INC., Seoul (KR)
`
`(21) Appl. No.: 13/734,652
`
`(22) Filed: (cid:9)
`
`Jan. 4, 2013
`
`Related U.S. Application Data
`(60) Provisional application No. 61/583,609, filed on Jan.
`6, 2012.
`
`(30) (cid:9)
`
`Foreign Application Priority Data
`
`Mar. 6, 2012 (cid:9)
`Mar. 6, 2012 (cid:9)
`Mar. 6, 2012 (cid:9)
`
`(KR) ........................ 10-2012-0022696
`(KR) ........................ 10-2012-0022697
`(KR) ........................ 10-2012-0022698
`
`110
`
`Broadcast
`Station
`
`Computing
`Device
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`H04N 51232 (cid:9)
`(52) U.S. Cl.
`CPC ................................. H04N 5/23219 (2013.01)
`USPC ............................................................ 348/78
`
`(2006.01)
`
`(57) (cid:9)
`
`ABSTRACT
`
`A display device including a display configured to display
`multimedia contents; a wireless communication unit config-
`ured to wirelessly communicate with at least one external
`mobile terminal; a camera unit configured to sense an eye
`movement of a user holding the mobile terminal; and a con-
`troller configured to receive indication signals from the
`mobile terminal indicating at least one of a grip pattern and
`eye movement of the user holding the mobile terminal, and to
`control a streaming operation of streaming the multimedia
`contents displayed on the display to the mobile terminal.
`
`1.20
`
`1.30
`
`140
`
`150
`
`,-- — (cid:9)
`Server
`
`Eal
`e
`
`External
`Television
`
`Mobile
`Device
`
`EX. 1011
`LG Electronics, Inc. / Page 1 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 1 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 1
`
`110
`
`Broadcast (cid:9)
`Station (cid:9)
`
`100
`
`Computing
`Device
`
`120
`
`1.30
`
`140
`
`150
`
`Server
`
`External
`Device
`
`External
`Television
`
`Mobile
`Device
`
`EX. 1011
`LG Electronics, Inc. / Page 2 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 2 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 2
`
`210 (cid:9)
`
`240 (cid:9)
`
`250
`
`broadcast
`
`receiving unit
`
`I
`
`demodulating
`
`unit
`
`network
`interface unit
`
`external device
`interface unit (cid:9)
`
`220
`
`230
`
`200
`
`260
`
`video output
`
` unit
`
`270
`
`control unit
`
`audio output unit
`
`power supply unit
`
`user interface unit
`
`I— 290
`
`280
`
`remote controller (cid:9)
`
`300
`
`EX. 1011
`LG Electronics, Inc. / Page 3 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 3 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 3
`
`350
`
`351
`
`352 (cid:9)
`
`354 (cid:9)
`
`355 (cid:9)
`
`356
`
`r -
`i
`
`Idemultiplexer
`
`scaler
`
`353
`
`L----------J
`
`357
`
`EX. 1011
`LG Electronics, Inc. / Page 4 of 69
`
`

`

`ate®
`MYA s ©®®
`
`LGMagicRemote 'k
`SDAG 15:49
`
`450
`
`b
`
`exists
`
`no
`
`corrWNdingdevi (cid:9)
`
`a precious conpz lion wi tithe
`
`yes (cid:9)
`
`a history of
`tiwhether
`
`no
`
`nceisconiectible (cid:9)
`z cDWespd[o ng (cid:9)
`
`430
`
`mere 2
`>----
`
`sl>s
`
`SJ16 (cid:9)
`
`Selection device
`
`Guide HSI14
`
`}es
`
`S113
`
`no
`
`deiice is cotmeclible
`he evressponding (cid:9)
`
`Device name
`
`7V ir+ (cid:9)
`
`Blu-any
`
`STU
`
`unro-. (cid:9)
`
`home diealer
`
`Bluaay (cid:9)
`
`Stone easier (cid:9)
`
`
`
`rmn
`tl.
`
`BDP i:un (cid:9)
`
`reon (cid:9)
`
`Btu-say
`
`Device name (cid:9)
`
`STU (cid:9)
`
`co T ce
`
`W Wi-Fi nine v
`
`select connected devices O
`0^a d 15:49
`
`no
`
`FIG. 4
`
`+ 0
`
`S108 (cid:9)
`
`tLe netw'c
`devices are cons
`How Mai
`
`;l
`
`J (cid:9)
`
`connectingWi-Fi
`
`indication (Wi-Fi off) (cid:9)
`
`display a Pop-Up
`
`5104
`
`I nd (cid:9)
`
`cDnmecling
`
`Wi-Fl?
`
`yes
`
`S 10<
`
`remote
`
`an LG Magic (cid:9)
`
`activates
`
`Ver. 2.0
`
`Wi-Fi checking...
`
`(Logo image)
`
`LO Magic Remote
`
`410
`
`O O O
`
`CA
`
`460
`
`O O O
`
`O~J \J input
`
`440
`
`eOnirslteTio Con,o, z status Home scram.
`Please push a Home mites of a remote (cid:9)
`Model group or Oyer defining name'
`
`Next
`
`in a same network.
`
`No comtuctctl TV(Media device)
`
`switching Wi-Fl
`
`5106 (cid:9)
`
`first wituN LG remote. LJJ
`
`at a Mnom debt corner of TV screen.
`Please input a pattern reptestated at
`
`a:iG11:49
`
`of Home tracer to (cid:9)
`
`Media excite must be a ata!as
`
`t AQ15:49
`
`Demo
`
`Deana icon
`
`wi-Finame v
`
`select connected devices 0
`f A O 15!49
`
`EX. 1011
`LG Electronics, Inc. / Page 5 of 69
`
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`

`

`560
`
`540 (cid:9)
`
`520 (cid:9)
`
`CA
`
`in ut
`P
`
`P1~~~ KeV (cid:9)
`
`Next (cid:9)
`
`ul~e (cid:9)
`
`~l (cid:9)
`
`—~
`
`no
`
`(L oile )
`
`V( ce Lkl
`
`slvirchnaN~~F~ (cid:9)
`
`5206 (cid:9)
`
`.
`
`indication (4Vi-Fi ofd cQmec-1mg Wi-Fi (cid:9)
`
`display a Pop-Up (cid:9)
`
`no
`
`more 2
`
`' 0
`
`S208 (cid:9)
`
`f ~~o S204
`
`a merious connectioi
`ahistory of
`whether
`
`S216
`
`yes
`
`yes
`
`the isriemandine (cid:9)
`
`S214
`
`selection
`
`counzcbi~g
`
`yes
`
`9202
`
`remote
`
`an LG Magic (cid:9)
`
`activates
`
`a ble c~ r~~ld L I
`
`Device List
`
`Splash screen
`
`r--
`
`Guide
`
`no
`
`550
`
`yes
`
`S213
`
`530
`
`no (cid:9)
`
`510
`
`b
`
`FIG. 5
`
`EX. 1011
`LG Electronics, Inc. / Page 6 of 69
`
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 6 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 6
`
`Main TV
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`----- (cid:9)
`
`D1 (cid:9)
`Ch.6 (cid:9)
`
`Ch,up (cid:9)
`• (cid:9)
`
`DTV (cid:9)
`Ch.7 (cid:9)
`
`----- (cid:9)
`
`DTV
`Ch.8
`
`610
`
`it (cid:9)
`
`Ii (cid:9)
`
`it (cid:9)
`
`.I (cid:9)
`II (cid:9)
`
`Mini TV
`
`iI (cid:9)
`
`II
`
`f~
`
`II
`II (cid:9)
`
`I
`
`1
`
`Start (cid:9)
`
`----- (cid:9)
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`----- (cid:9)
`
`.DTV (cid:9)
`Ch.7 (cid:9)
`
`Ch.up (cid:9)
`
`DTV
`Ci.8
`
`620
`
`EX. 1011
`LG Electronics, Inc. / Page 7 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 7 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 7
`
`Main TV
`
`Mini TV
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`
`i (cid:9)
`r (cid:9)
`i (cid:9)
`i (cid:9)
`i (cid:9)
`
`i (cid:9)
`r (cid:9)
`t (cid:9)
`i (cid:9)
`
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r
`r (cid:9)
`
`I (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`r (cid:9)
`rr (cid:9)
`
`. (cid:9)
`
`r (cid:9)
`I (cid:9)
`r (cid:9)
`r (cid:9)
`
`Change (cid:9)
`input (cid:9)
`
`Component ----- Component (cid:9)
`
`Change
`input (cid:9)
`
`DTV
`Ch,7
`
`610
`
`A
`'
`'
`'
`~
`~
`
`r
`'
`r
`;
`
`
`
`r
`'
`'
`'
`r
`r
`
`'
`r
`r
`'
`
`620
`
`DTV
`DTV Ch.up DTV (cid:9)
`DTV (cid:9)
`Ch.7
`Ch.7 (cid:9)
`----- (cid:9)
`Ch.6 (cid:9)
`Ch.6 (cid:9)
`----- (cid:9)
`_____ H_____ _____ _____
`
`EX. 1011
`LG Electronics, Inc. / Page 8 of 69
`
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 8 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 8
`
`Main TV
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`----- (cid:9)
`
`DTV Ch.up DTV (cid:9)
`Ch.7 (cid:9)
`Ch,6 (cid:9)
`
`----- (cid:9)
`
`DTV
`Ch.7
`
`610
`
`_________F
`
`Mini TV
`
`II
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`Component ----- (cid:9)
`
`Component (cid:9)
`
`Change
`input (cid:9)
`
`DTV
`Ch.7
`
`620
`
`EX. 1011
`LG Electronics, Inc. / Page 9 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 9 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 9
`
`Main TV
`
`Mini TV
`
`DTV (cid:9)
`Ch,6 (cid:9)
`
`----- (cid:9)
`
`DTV
`Ch.6
`
`II (cid:9)
`
`f (cid:9)
`
`If (cid:9)
`
`II
`
`11
`
`DTV (cid:9)
`
`it
`
`~I
`
`II
`
`Component
`
`610
`
`620
`
`EX. 1011
`LG Electronics, Inc. / Page 10 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 10 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 10
`
`Change
`input (cid:9)
`
`DTV (cid:9)
`Ch.6
`
`pC (cid:9)
`
`----- (cid:9)
`
`PC
`
`Main TV
`
`Mini TV
`
`Start
`
`_____ (cid:9)
`
`DTV
`Ch.6
`
`610
`
`620
`
`EX. 1011
`LG Electronics, Inc. / Page 11 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 11 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 11
`
`web (cid:9)
`
`----- (cid:9)
`
`web
`
`610
`
`Main TV
`
`Mini TV
`
`620
`
`Start (cid:9)
`
`---- (cid:9)
`
`Execution is
`not possible.
`Call of a popup
`
`EX. 1011
`LG Electronics, Inc. / Page 12 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 12 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 12
`
`Main TV
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`----- (cid:9)
`
`DTV (cid:9)
`Ch.6 (cid:9)
`
`Change
`input
`
`web
`
`610
`
`II (cid:9)
`
`~~
`
`Mini TV
`
`I1
`
`I1 (cid:9)
`
`Staff (cid:9)
`
`----- (cid:9)
`
`II
`
`DIV
`Ch.6 6 (cid:9)
`
`620
`
`Execution is
`not possible.
`Call of a popup
`
`EX. 1011
`LG Electronics, Inc. / Page 13 of 69
`
`

`

`1376
`
`1370
`
`1365
`
`1360
`
`b
`
` IV'11 l NIBCIID]I inil SU~erLpttac (cid:9)
`
`11BC HD
`
`19:50
`I1 15:49
`
`1355
`
`TVA
`
`LXIT (cid:9)
`
`X 1&OO
`
`FIG. 13
`
`(b)
`
`1374
`
`Ch. List (cid:9)
`
`* , (cid:9)
`
`( N/ (cid:9)/' I u (cid:9)
`
`Tl' (cid:9)
`
`(a)
`
`1372
`
`4 (cid:9)
`
`—1335 (cid:9)
`
`1340
`
`1330 —.._
`
`1350
`
`1310
`
`jj' 1_I
`
`I
`
`1320 ----
`
`EX. 1011
`LG Electronics, Inc. / Page 14 of 69
`
`

`

`(c)
`
`(b)
`
`Icon
`
`E1 (cid:9)HIHl1
`
`/~ (cid:9)
`
`
`
`{D `II I!-J MMY lJDl
`
`a©boo
`
`(a)
`
`Ch.
`
`V (cid:9)1.
`
`9:50
`9f; d1 15:49
`
`SNS
`Vv 1
`
`TV AS OFF Sound only Cxphne (cid:9)
`
`EFfl (cid:9)
`
`0w (cid:9)
`
`bU (cid:9)
`
`wf (cid:9)
`
`X IS:OQ (cid:9)
`
`1420
`
`®tea
`
`Channels
`
`LG Magic Remote
`
`0t : n G 1549
`
`1410-.--__
`
`FIG. 14
`
`b
`
`EX. 1011
`LG Electronics, Inc. / Page 15 of 69
`
`(cid:9)
`(cid:9)
`

`

`(C)
`
`(b)
`
`(a) (cid:9)
`
`cnis
`
`j &er Fj 1 3
`
`y I^ I U * (cid:9)
`[Dh' (-1 1BC }1P] I art (cid:9)
`
`Tl' (cid:9)
`
`-152O
`
`Environment setting
`
`con
`
`TV account login
`
`icon
`
`Search
`icon (cid:9).
`in
`icci'?''
`
`Ch.
`
`11
`
`vol.
`
`--1510
`
`AMBCHD
`
`0t;i015:49
`
`TV AV OFF Sound
`
`EXIT (cid:9)
`
`x19:00 .-•,
`
`Any screen
`
`Any screen
`
`: d 1 15:49
`
`s A O 15:49
`
`b
`
`FIG. 15
`
`EX. 1011
`LG Electronics, Inc. / Page 16 of 69
`
`(cid:9)
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 16 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 16
`
`(a)
`
`(b)
`
`TV
`
`Image 2
`
`(c)
`
`(d)
`
`EX. 1011
`LG Electronics, Inc. / Page 17 of 69
`
`(cid:9)
`

`

`(d)
`
`HDMI - DVD Player
`
`r1 (I 15:49
`
`SNS
`icon
`
`(b)
`
`TV O OFF Sound only Capture (cid:9)
`icon (cid:9)
`
`icon (cid:9)
`
`icon (cid:9)
`
`1740
`
`(c)
`
`HDMI - DVD Player
`
`0
`
`EXIT (cid:9)
`
`X
`
`FQW115:49
`
`(a)
`
`1730
`
`EXIT
`X
`
`,I
`
`! H (cid:9)
`
`( (cid:9)
`
`1 ~ (cid:9)
`
`(icon
`
`* (cid:9)
`
`( (cid:9)
`
`1710 (cid:9)
`
`(III\ I1-1 \13( HD] I3i l~tiee l~;t~e3
`
`Mini TV MddN TV
`
`172 O (cid:9)
`
`c~ (cid:9)
`
`I
`
`MBC HD
`
`19^0
`Ot s n 115:49
`
`TV AV OFF Sound
`co
`....
`
`...,cot ....... (cid:9)
`
`:~........ (cid:9)
`
`......
`
`EXII (cid:9)
`
`XIF~0~1
`
`MBC HD (cid:9)
`
`) 19:50 (cid:9)
`fit^A G 15:49
`
`EXIT TVAVC
`
`-,
`
`X 19:00 (cid:9)
`
`b
`
`FIG. 17
`
`EX. 1011
`LG Electronics, Inc. / Page 18 of 69
`
`

`

`CA
`
`rr.
`
`f7
`
`b
`
`(c)
`
`(a)
`
`HDMI - DVD Player
`
`HDMI - DVD Player
`
`(cid:9)0
`
`1820
`
`0t -- d i 15:49
`
`EXT
`
`x
`
`A 6 15:49
`
`EMIT
`
`x
`
`FIG. 18
`
`1830
`
`1810
`
`EX. 1011
`LG Electronics, Inc. / Page 19 of 69
`
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 19 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 19
`
`1900
`
`1810
`
`1950
`
`EX. 1011
`LG Electronics, Inc. / Page 20 of 69
`
`

`

`CA
`
`b
`
`2030
`
`(b)
`
`(a)
`
`Ch. List
`
`icon
`
`2020
`
`(cid:9)I *
`
`H ' (cid:9)
`
`/\ (
`
`T (cid:9)
`
` _ (cid:9)
`
`/~
`
`D1 1. I,A4l'11ll]iait~rcrLp,oi 3
`
`MBCHD
`
`1950
`r1 a ]5:49
`
`SNS (cid:9)
`
`2010
`
`FIG. 20
`
`TV.4V0FF Sound only Captiu£' I (cid:9)
`
`EXIT (cid:9)
`
`:.:-:.. (cid:9)
`
`0559!
`
`
`
`X 1800 .::.. (cid:9)
`
`855Y (cid:9)
`
`EX. 1011
`LG Electronics, Inc. / Page 21 of 69
`
`(cid:9)
`

`

`(d)
`
`(C)
`
`CA
`
`— 2140
`
`MBCHD
`
`I
`
`,_sill III ill
`
`II: I (cid:9)
`
`Ii~kfigogngsson"i;or1971 E (cid:9)
`
`© LGtmIagickcconte 2.1110 ) C 10;0
`
`1 (cid:9)
`
`r, L
`
`9i A 11 15:49
`
`MACHO (cid:9)
`
`MBC ND
`
`1549
`
`,27r'in - in:
`
`Pack Jiyoung ssorb
`
`IaM
`
`Photo
`User (cid:9)
`
`(b)
`
`(a)
`
`—2120
`
`ii
`
`1
`
`1 (cid:9)
`
`+:.1. (cid:9)n~::
`
`]
`
`€:
`
`e's rIO i111 (cid:9)
`#i>E2tu (cid:9)
`
`u (cid:9)
`
`1
`
`Il (cid:9)
`
`~I (cid:9)
`
`a .~r (cid:9)Iii
`
`U aVahyLOe (cid:9)
`
`KY (cid:9)I),.l'xi (cid:9)
`Q Oio00o,GOO(U o d I IKC
`© (cid:9)42Itis really dd1imul Ir (cid:9)
`Q LgmaaPaediyaung (cid:9)
`
`e
`
`I (cid:9)
`
`. (cid:9)
`
`2130
`
`MOOED
`
`0--v A 6 15:49
`
`b
`
`Ch. Ls!
`icon
`
`(
`
`/~ \3
`
`'1 J 1i
`~ (cid:9)
`
`[1i111H IBCH.i)]] am -Pev7 §
`
`MBCHD
`
`19:50
`t Aa15:49
`
`i:r° (cid:9)
`
`TV AV Off Sodonly CapUre (cid:9)
`
`EXIT (cid:9)
`
`a
`
`a (cid:9)
`
`o>> (cid:9)
`
`X18:00 (cid:9)
`
`;> (cid:9)
`
`2110
`
`FIG. 21
`
`EX. 1011
`LG Electronics, Inc. / Page 22 of 69
`
`(cid:9)
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 22 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 22
`
`2212 (cid:9)
`
`2214
`
`O 1549
`
`t (cid:9)
`
`O
`
`— 2216
`
`cancel
`
`9a.dO15:49
`
`—2220
`
`—2230
`
`—2240
`
`-EJCllin r4s[wdaSin (cid:9)
`
`xa'1 9'JffH(,
`
`I;,[
`
`.2220
`
`nip
`
`' —2240
`
`(b)
`
`c~urcel ® (cid:9)
`
`Li ii:r~cr
`
`~~ClilaUlll Ke psd
`
`(a)
`
`2250
`
`upload
`
`ii
`cancel
`
`I (cid:9)
`
`(c)
`
`cancel ] (cid:9)
`
`O; aI1i:49
`
`0 W
`
`Beck (cid:9)
`jJ
`
`
`
`I, to (cid:9)
`
`'all (cid:9)
`
`I maSinger
`
`!;/G 15:49
`
`Mnc Hn
`
`MI (cid:9)
`
`f-]
`
`;mg.lylaaao 9! nra (cid:9) Sm,er
`(cid:9) am a Longer today
`
`®dnym We1lmamRedi (cid:9)
`IgMaa Bak liyouog's son' is (cid:9)
`
`I
`
`1 (cid:9)
`
`r- (cid:9)
`
`Q 421: is really difficulI to paean ranking
`010000 Good, good #MBt
`S sendmg has hnet, completed
`
`Uso Kirat Doeguk (cid:9)
`
`(d) (cid:9)
`
`1
`2260
`
`EX. 1011
`LG Electronics, Inc. / Page 23 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 23 of 46
`
`US 2013/0176415 Al
`
`FIG. 23
`
`c.mcel ® (cid:9)
`
`Weflmileh .tonsy.h-pi (cid:9)
`
`d (cid:9)
`
`in
`PAA(~ Iii
`
`
`
`n
`
`to
`
`Arbiloid K
`
`(1)
`
`c~mcel
`
`WtlatcuJi (cid:9)
`
`ft (cid:9)
`
`iula.,:d- -?lP.4EC (cid:9)
`
`Irnna'iuscr ...-
`
`H
`
`2310-1.
`J
`
`ancel ®
`
`O
`
`thumbnail (cid:9)
`
`thumbnnd c (cid:9)
`
`+
`
`Andioid Key p d~
`i (cid:9)
`
`I
`
`ulcd
`
`(c)
`
`2350
`
`2342 (cid:9)
`
`2344 (cid:9)
`
`2346
`
`(d)
`
`2320
`
`I59
`
`
`
`~ (cid:9)
`
`epload
`
`a rG 1549
`
`cancel
`
`cancel
`
`Well matched todap I (cid:9)
`
`I (cid:9)
`
`J I sm aSIuger
`
`humbnxit (cid:9)
`
`+
`
`udo~iJ
`
`lAiidrold Kc l)c't li
`l~ (cid:9)
`
`.I , upload
`
`(e)
`
`EX. 1011
`LG Electronics, Inc. / Page 24 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 24 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 24
`
`2410
`
`(a)
`
`MBC HD
`
`M'~ SNS Detaii O mage
`
`2420
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 25 of 69
`
`

`

`2570
`
`2564 (cid:9)
`
`2556
`
`2546
`
`2554 (cid:9)
`
`(a) (cid:9)
`
`2544 (cid:9)
`
`2542 (cid:9)
`
`prior
`
`52
`
`Lame (cid:9)
`
`con
`
`nanie
`
`amt (cid:9)
`
`icon
`
`:on (cid:9)
`
`ame name
`
`icon
`
`:on (cid:9)
`
`My Apps
`
`KBS2
`DTV 7-1* DTV 7-1* DTV 7-1*
`KBS 2
`DTV 7-1*
`KBS2
`DTV 7.1* DTV 7-l* DTV 7-1*
`KBS2
`DTV 7-1 * DTV 7-1 * DTV 7-1 *
`Chatmels
`
`KBS 2
`DTV 7-1*
`KBS2
`
`KBS 2
`DTV 7-1*
`KBS2
`
`KBS2
`
`KBS2
`
`KBS 2
`DTV 7-1*
`KBS2
`
`IBS 2
`DTV 7-1*
`KBS 2
`
`KB5 2
`DTV 7-1*
`
`Premium
`
`552
`
`P (cid:9)grain descnption(con nil
`P cu am description (content)
`P ogram description (content)
`PDDramdescrpfiDn(contenQ
`P (cid:9)gam description (content)
`Pr ecam description (content)
`Cu rently viewing program name
`DTV2-1
`
`th
`
`+ (cid:9)
`
`tin
`
`name
`thumbnail
`
`name
`
`thumbnail tin
`
`Media Files
`
`2562
`
`2530
`
`2540
`
`2510 (cid:9)
`
`2520
`
`FIG. 25
`
`b
`
`EX. 1011
`LG Electronics, Inc. / Page 26 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 26 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 26
`
`My Channel
`
`DTVcable CH, N CH. naive
`
`12t ^ rl Q 12:51
`
`Icon
`
`DIVcable Cli. K CH. name
`
`D'Rcnbi.Cn,NCH.flrnu
`
`'—aL4L.
`
`Pro
`
`PNhh:uu (cid:9)
`
`Pblhhmm
`
`II :mm (cid:9)
`
`PM hh:wu
`
`PMhh:min
`
`PMhh:mm
`
`1. list (cid:9)
`
`(a)
`
`Vol. T
`
`My Channel
`
`DTVcable CH. N CH. name
`
`'l1 , II 1 12:51
`
`Icon
`
`DTVcable CH. N CH. name
`
`DTVcable CONCH. name
`
`cj p%
`
`Pu hh: n (cid:9)
`
`PM hh:min
`
`Program name
`
`
`Prop (cid:9)
`ame
`D
`PM hh:min
`Phi hh:mm
`
`PM lib mm
`PM ilh: Mai (cid:9)
`-----------------------------------------
`CI-list
`CH07 F'HH'8
`CH10
`
`CHl3
`EBS
`
`CH1S
`CJ
`
`CHIN
`~~icul6
`&Fisheries
`
`CH06
`SBS
`
`KBS
`
`CH09
`ping KBS
`
`CHII
`skonpmg MBC
`
`I
`L-------------------------------------------
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 27 of 69
`
`

`

`Cl)
`
`b
`
`(C)
`
`(b)
`
`Copa America
`
`DTV cable CH. name
`
`Golf today
`
`DTV cable CH. name
`
`New Gisaeng episode 2
`
`DTV cable CH. name
`
`related information
`
`PM hh : mm
`
`Program name
`
`_
`
`PM hh : mm
`
`
`
`DTVcable CH. II MBC
`
`DTVcable CH. name
`
`Copa America
`
`DTVcable CH. name
`
`related information
`
`PMhh:mm
`
`PMhh:mm
`
`Program name
`
`time
`
`DTVcableCH,11 MBC
`
`New Gisaeng episode 2
`
`DTVcable CH. name
`
`Golf today
`
`DTVcable CH. name
`
`9 s rf 112:51
`
`M1QnGI 12:51
`
`(a)
`
`channel
`
`EPG Info.
`
`EPG Info,
`
`PM fib; mm
`
`PMhh:min (cid:9)
`
`Program name.
`Channel name.
`
`Program name. Program name (cid:9)
`Channel name. (cid:9)
`
`DTVcable CH. N CH. name
`
`EPG Info.
`
`mote
`real
`real time
`
`12:51
`
`FIG. 27
`
`EX. 1011
`LG Electronics, Inc. / Page 28 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 28 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 28
`
`2810
`
`X
`icon (cid:9)
`icon (cid:9)
`icon (cid:9)
`EXIT TV AV OFF Sound only Capture (cid:9)
`
`icon
`con (cid:9)
`SNS time machine (cid:9)
`
`MBCHD
`
`la1 ?II (cid:9)
`
`05:13
`
`2820
`
`I)
`
`~
`
`(cid:9) L
`
`2822 (cid:9)
`
`2824 2826 2828
`
`(a)
`
`EX. 1011
`LG Electronics, Inc. / Page 29 of 69
`
`(cid:9)
`

`

`CA
`
`b
`
`(b)
`
`(a) (cid:9)
`
`Y~
`
`1 (cid:9)
`
`6 (cid:9)
`
`)T1 I HI 1PC Hal (cid:9)
`
`Call screen
`
`Calling (cid:9)
`
`n
`
`MBCHD
`
`SNS
`
`icon I con
`
`EXIT TV AV OFF Sound only Capture (cid:9)
`X ( icon (cid:9)
`
`con (cid:9)
`
`05:13
`
`6 gold (cid:9)
`
`FIG. 29
`
`EX. 1011
`LG Electronics, Inc. / Page 30 of 69
`
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 30 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 30
`
`05:13
`d€ Yboo (cid:9)
`MBCHI)
`
`2910I
`
`lOminutes -8minutes -6minutes -4minutes -2minutes currently
`viewing image
`image
`image
`image
`image
`image
`
`H ! (cid:9) BCHDI
`
`1
`
`~urr~~i~
`(broacashn (cid:9)
`r cord ng
`1
`! (cid:9)
`2920 2930
`
`I
`
`.
`
`I (cid:9) / (cid:9)
`
`lime•, recoldmg (cid:9)
`rchine (cid:9) ~senatrn:
`list (cid:9)
`
`con
`
`Ch List
`
`2940 2950
`
`EX. 1011
`LG Electronics, Inc. / Page 31 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 31 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 31
`
`1 L1li< (cid:9)
`
`Iii ? o d (cid:9)
`Touch (cid:9)
`
`05:13
`CD
`
`3110
`
`3120
`
`3122
`
`00:30 (cid:9)
`
`O1:~0
`
`T i r
`
`3124
`
`-3130
`
`3140
`
`rj LD LJ
`
`LIII (cid:9)
`
`__
`
`
`
`Hfl
`
`EX. 1011
`LG Electronics, Inc. / Page 32 of 69
`
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 32 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 32
`
`Start
`
`activates a prescribed application
`at mobile device (cid:9)
`
`the mobile device is paired with
`a main device by the active application (cid:9)
`
`The mobile device transmits a control signal for a mini TV
`to the main device in response to a prescribed selection made by (cid:9)
`a user or a prescribed action performed by the user
`
`S310
`
`S320
`
`S330
`
`output data received from the main device to a screen (cid:9)
`
`-f --- S340
`
`End
`
`EX. 1011
`LG Electronics, Inc. / Page 33 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 33 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 33
`
`start
`
`mobile device is paired with a main device
`
`-- S410
`
`the mobile device receives an input of
`a user's selection or action
`
`—S420
`
`Based on the received input of the user's selection or action,
`the mobile device receives video data from the main device,
`then outputs the received video data to a screen and/or transmits
`a signal including video data to the main device and then controls
`the main device to output the transmitted signal including the video data
`
`-- S430
`
`End
`
`EX. 1011
`LG Electronics, Inc. / Page 34 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 34 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 34
`
`Start
`
`a prescribed application is activated in a first device (cid:9)
`
`-f ---- S5 10
`
`The first device is paired with a second device in response to
`the activation of the prescribed application (cid:9)
`
`The first device transmits a control signal for a mini TV activation
`to the second device in response to a first request for the mini TV activation (cid:9)
`
`The first device outputs a content received from the second
`device to a screen
`
`The first device stops a playback of a content output based on a second request
`for a time machine function and provides a first UI including first time machine
`information on a play point in case of the second request and second time (cid:9)
`machine information on a current point to a prescribed region of the screen
`
`The first device outputs a content to the screen in response to
`a third request for the playback of the content (cid:9)
`
`End
`
`5520
`
`5530
`
`
`
`S540
`
`S550
`
`S560
`
`EX. 1011
`LG Electronics, Inc. / Page 35 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 35 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 35
`
`Start D
`
`a first device is paired with a second device, the first device receives video data
`from the second device and then outputs the received video data to a screen,
`based on a first request for the mini TV activation or the first device transmits
`a control signal including the video data to the second device
`
`--5610
`
`The first device stops playing a content output based on a second request for
`for a time machine function and provides a first UI including first information
`on a play point in case of the second request and second information
`on a current point to a prescribed region of the screen
`
`---- S620
`
`The first device outputs a content to the screen in response to
`a third request for the playback of the content
`
`—S630
`
`End
`
`EX. 1011
`LG Electronics, Inc. / Page 36 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 36 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`0 0
`
`M
`
`Q
`
`Q
`
`Co
`CD
`
`ICI
`
`EX. 1011
`LG Electronics, Inc. / Page 37 of 69
`
`

`

`based on grip pattern
`
`Perform function
`
`yes SI000
`
`grip changed?
`
`End
`
`TV program
`
`on streaming (cid:9)
`
`I time-shift record
`
`maintain or turn I (cid:9)
`
`S950
`
`yes S940 (cid:9)
`
`no
`
`g at mobile
`eyes
`
`S930
`
`on grip pattern
`
`on eyes and grip pattern
`
`I (cid:9)
`
`mobile terminal (cid:9)
`
`Perform function based
`
`Perform specific function based
`
`I (cid:9)
`
`turn off streaming to
`
`no S980
`
`both 51010
`
`yes S920 (cid:9)
`
`(maintain)
`
`turn on streaming
`
`viewin~
`
`contents
`
`p
`i
`
`both?
`
`eyes or grip or (cid:9)
`
`user (cid:9)
`
`eyes (cid:9)
`
`TV? (cid:9)
`
`looking at (cid:9)
`
`eyes (cid:9)
`
`no (cid:9)
`
`S970
`
`S960
`
`S900
`
`S910 (cid:9)
`
`b
`
`Start
`
`FIG. 37
`
`EX. 1011
`LG Electronics, Inc. / Page 38 of 69
`
`(cid:9)
`(cid:9)
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 38 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 38
`
`3610
`
`3620
`
`01 (cid:9)
`
`0
`
`00000000000
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`0
`0 (cid:9)
`00000000000
`
`EX. 1011
`LG Electronics, Inc. / Page 39 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 39 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 39
`
`3610
`
`(a)
`
`3610
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 40 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 40 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 40
`
`3610
`
`(a)
`
`3610
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 41 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 41 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 41
`
`3610
`
`3610
`
`(a)
`
`(U)
`
`EX. 1011
`LG Electronics, Inc. / Page 42 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 42 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 42
`
`3650
`
`(a)
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 43 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 43 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 43
`
`m
`
`(a)
`
`3610
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 44 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 44 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 44
`
`3610
`
`(a)
`
`3610
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 45 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 45 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 45
`3610
`
`(a)
`
`3610
`
`(b)
`
`EX. 1011
`LG Electronics, Inc. / Page 46 of 69
`
`

`

`Patent Application Publication (cid:9)
`
`Jul. 11, 2013 Sheet 46 of 46 (cid:9)
`
`US 2013/0176415 Al
`
`FIG. 46
`
`3610
`
`(a)
`
`(b)
`
`3610
`
`EX. 1011
`LG Electronics, Inc. / Page 47 of 69
`
`

`

`US 2013/0176415 Al
`
`Jul. 11, 2013
`
`APPARATUS FOR PROCESSING A SERVICE
`AND METHOD THEREOF
`
`the spirit and scope of the invention will become apparent to
`those skilled in the art from this detailed description.
`
`[0001] This application claims the benefit of a provisional
`application No. 61/583,609, filed on Jan. 6, 2011, and the
`Korean Patent Application Nos. 10-2012-0022696, filed on
`Mar. 6, 2012, 10-2012-0022697, filed on Mar. 6, 2012, and
`10-2012-0022698, Mar. 6, 2012, which is hereby incorpo-
`rated by reference as if fully set forth herein.
`
`BACKGROUND OF THE INVENTION
`
`[0002] 1. Field of the Invention
`[0003] The present invention relates to an apparatus for
`processing a service and method thereof. Although the
`present invention is suitable for a wide scope of applications,
`it is particularly suitable for processing services among a
`plurality of devices.
`[0004] 2. Discussion of the Related Art
`[0005] Smart televisions (TVs) are now becoming popular.
`A traditional remote controller is used to operate the smart
`TV. The integration of a user's mobile terminal with the
`functions provided by the smart TV has not been sufficiently
`addressed. Further, the mobile device and TV adhere to their
`own exclusive domain and do not smoothly work together.
`
`SUMMARY OF THE INVENTION
`
`[0006] Accordingly, one object of the present invention is
`to provide an apparatus and corresponding method for pro-
`cessing a service that substantially obviate one or more prob-
`lems due to limitations and disadvantages of the related art.
`[0007] Another object of the present invention is to provide
`an interfacing method and environment, by which communi-
`cations between a mobile device and other digital devices,
`data transmissions and receptions and the like can be per-
`formed more easily and efficiently.
`[0008] Another object of the present invention is to provide
`a new service model by providing further enhanced interfac-
`ing and communication environments and processes, by
`which the limitations put on one device having a function of
`simply outputting video, audio, image and the like received
`from another digital device or a function of playing a role as
`a substitute for a remote controller are overcome.
`[0009] To achieve these objects and other advantages and in
`accordance with the purpose of the invention, as embodied
`and broadly described herein, the present invention provides
`a display device including a display configured to display
`multimedia contents; a wireless communication unit config-
`ured to wirelessly communicate with at least one external
`mobile terminal; a camera unit configured to sense an eye
`movement of a user holding the mobile terminal; and a con-
`troller configured to receive indication signals from the
`mobile terminal indicating at least one of a grip pattern and
`eye movement of the user holding the mobile terminal, and to
`control a streaming operation of streaming the multimedia
`contents displayed on the display to the mobile terminal. The
`present invention also relates to a mobile terminal interfacing
`with the display device.
`[0010] Further scope of applicability of the present inven-
`tion will become apparent from the detailed description given
`hereinafter. However, it should be understood that the
`detailed description and specific examples, while indicating
`preferred embodiments of the invention, are given by illus-
`tration only, since various changes and modifications within
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0011] The accompanying drawings, which are included to
`provide a further understanding of the invention and are
`incorporated in and constitute a part of this application, illus-
`trate embodiment(s) of the invention and together with the
`description serve to explain the principle of the invention. In
`the drawings:
`[0012] FIG. 1 is conceptional diagram illustrating one
`example of an overall system including a service processing
`apparatus according to an embodiment of the present inven-
`tion;
`[0013] FIG. 2 is a detailed block diagram illustrating one
`example of a service processing apparatus according to an
`embodiment of the present invention;
`[0014] FIG. 3 is a block diagram illustrating one example of
`a control unit according to an embodiment of the present
`invention;
`[0015] FIG. 4 and FIG. 5 are diagrams illustrating a pairing
`process between a mobile device and a main device according
`to an embodiment of the present invention;
`[0016] FIG. 6 is a diagram illustrating one example of a
`scene for a mobile device to provide a mini TV when watch-
`ing a normal screen in a main device;
`[0017] FIG. 7 is a diagram illustrating one example of a
`scene for a mobile device to provide a mini TV when chang-
`ing an input in the course of watching a broadcast screen in a
`main device;
`[0018] FIG. 8 is a diagram illustrating one example of
`operations when changing an input while a mobile device is
`providing a broadcast program on a channel received from a
`main device;
`[0019] FIG. 9 is a diagram illustrating one example of an
`operation of a mobile device while a main device provides a
`component input instead of a broadcast channel;
`[0020] FIG. 10 is a diagram illustrating one example of an
`operation of a mobile device when a main device switches to
`a PC input when providing an initial broadcast channel;
`[0021] FIG. 11 and FIG. 12 are diagrams illustrating
`examples of an operation of a mini TV when implementing a
`web browser in a main TV;
`[0022] FIG. 13 is a diagram illustrating one example of
`activating a mini TV function by activating a channel selected
`from a home screen of a mobile device;
`[0023] FIGS. 14(a) to 14(c) are diagrams describing the
`mini TV function activation;
`[0024] FIGS. 15(a) to 15(c) are diagrams illustrating one
`example of an access method for a mini TV function activa-
`tion according to an embodiment of the present invention;
`[0025] FIGS. 16(a) to 16(d) are diagrams illustrating
`another example of an access method for a mini TV function
`activation or access according to an embodiment of the
`present invention;
`[0026] FIG. 17 is a diagram illustrating attempting to
`change or switch an input while activating a mini TV function
`in a mobile device according to an embodiment of the present
`invention;
`[0027] FIG. 18 and FIG. 19 are diagrams illustrating
`describing a mini TV function according to an embodiment of
`the present invention;
`
`EX. 1011
`LG Electronics, Inc. / Page 48 of 69
`
`(cid:9)
`

`

`US 2013/0176415 Al
`
`Jul. 11, 2013
`
`2
`
`[0028] FIG. 20 is a diagram illustrating another example of
`a mini TV function according to an embodiment of the
`present invention;
`[0029] FIG. 21 is a diagram illustrating another function of
`a mini TV according to an embodiment of the present inven-
`tion;
`[0030] FIG. 22 is a diagram illustrating when a user
`attempts to compose a content using SNS instead of checking
`a related SNS content like FIG. 21;
`[0031] FIG. 23 is a diagram illustrating attaching image
`information to SNS as well as text data;
`[0032] FIG. 24 is a diagram illustrating another example for
`a mini TV access according to an embodiment of the present
`invention;
`[0033] FIGS. 25 to 27 are diagrams illustrating other
`examples for a mini TV access according to an embodiment
`of the present invention;
`[0034] FIG. 28 and FIG. 29 are diagrams illustrating a mini
`TV providing method according to an embodiment of the
`present invention;
`[0035] FIG. 30 is a diagram of a UI;
`[0036] FIG. 31 is a diagram of a mobile device;
`[0037] FIG. 32 and FIG. 33 are flowcharts illustrating a
`mini TV providing method according to an embodiment of
`the present invention;
`[0038] FIG. 34 and FIG. 35 are flowcharts illustrating a
`method of providing a time machine function in a mobile
`device;
`[0039] FIGS. 36(a), (b) and (c) illustrates an overview of a
`mobile terminal interfacing with a display device according
`to an embodiment of the present invention;
`[0040] FIG. 37 a flow chart illustrating a method of con-
`trolling a display device and mobile terminal according to an
`embodiment of the present invention;
`[0041] FIG. 38 is an overview of a mobile terminal includ-
`ing sensors for detecting different