Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 1 of 23 Page ID #:2407
`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 1 of 23 Page ID #:2407
`
`
`
`
`
`EXHIBIT 15
`
`EXHIBIT 15
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 2 of 23 Page ID #:2408
`I 1111111111111111 11111 1111111111 11111 1111111111 1111111111 111111111111111111
`US008712510B2
`
`c12) United States Patent
`Quy
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,712,510 B2
`Apr. 29, 2014
`
`(54) METHOD AND APPARATUS FOR EXERCISE
`MONITORING COMBINING EXERCISE
`MONITORING AND VISUAL DATA WITH
`WIRELESS INTERNET CONNECTIVITY
`
`(75)
`
`Inventor: Roger J. Quy, Kentfield, CA (US)
`
`(73) Assignee: Q-Tec Systems LLC, Wilmington, DE
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 379 days.
`
`(21) Appl. No.: 12/692,080
`
`(22) Filed:
`
`Jan.22,2010
`
`(65)
`
`Prior Publication Data
`
`US 2010/0120585 Al
`
`May 13, 2010
`
`Related U.S. Application Data
`
`(63) Continuation-in-part of application No. 11/649,355,
`filed on Jan. 3, 2007, now abandoned, which is a
`continuation of application No. 11/156,177, filed on
`Jun. 17, 2005, now Pat. No. 7,156,809, which is a
`continuation-in-part of application No. 10/773,501,
`filed on Feb. 6, 2004, now Pat. No. 6,976,958.
`
`(51)
`
`(2006.01)
`(2006.01)
`
`Int. Cl.
`A61B 5104
`A61B 5102
`(52) U.S. Cl.
`USPC ............................ 600/520; 600/508; 600/513
`( 58) Field of Classification Search
`USPC .................. 482/1-9, 900-902; 128/903-904;
`600/513, 520
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,282,883 A
`5,012,814 A
`5,307,263 A
`5,357,427 A
`5,410,471 A
`5,434,611 A
`5,441,047 A *
`5,474,090 A
`5,544,649 A
`5,544,661 A
`
`8/1981 Yerushalmy
`5/1991 Mills et al.
`4/1994 Brown
`10/1994 Langen et al.
`4/1995 Alyfuku et al.
`7/1995 Tamura
`8/1995 David et al. ................... 600/483
`12/1995 Begun et al.
`8/ 1996 David et al.
`8/ 1996 Davis et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`GB
`JP
`
`11/2001
`2326237
`9/1997
`9224917
`(Continued)
`OTHER PUBLICATIONS
`
`U.S. Appl. No. 60/264,739, filed Jan. 2001, Posa et al.
`
`(Continued)
`Primary Examiner - Christopher Koharski
`Assistant Examiner - Catherine Voorhees
`(74) Attorney, Agent, or Firm -Mayer & Williams PC;
`Mark D. Wieczorek
`ABSTRACT
`(57)
`Embodiments of the invention provide a method and appara(cid:173)
`tus for a wireless exercise monitoring system for interactively
`monitoring an aspect of exercise, nutrition, or fitness by con(cid:173)
`necting a wireless internet device to or with a digital camera
`and/or an exercise monitoring device. Exercise-related data
`and/or visual information from the camera is transmitted to a
`server using standard internet protocols and may be inte(cid:173)
`grated with various operating systems for mobile wireless
`devices, especially those with enhanced capabilities for hand(cid:173)
`ing images and visual data. Responses may be calculated and
`transmitted back to a user, trainer, or coach.
`23 Claims, 9 Drawing Sheets
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 3 of 23 Page ID #:2409
`
`US 8,712,510 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,549,117 A
`5,553,609 A
`5,576,952 A
`5,601,435 A
`5,626,144 A
`5,678,562 A
`5,701,904 A
`5,704,366 A
`5,724,025 A
`5,732,709 A
`5,735,285 A
`5,752,917 A
`5,772,586 A
`5,791,342 A
`5,929,782 A
`5,931,791 A
`5,933,136 A
`5,935,060 A
`5,941,829 A
`5,951,300 A
`5,959,533 A
`5,964,701 A
`5,967,975 A
`5,987,352 A
`5,987,519 A
`5,997,476 A
`6,013,007 A
`6,022,315 A
`6,024,699 A
`6,050,940 A
`6,055,506 A
`6,057,758 A
`6,059,692 A
`6,083,156 A
`6,093,146 A
`6,101,478 A
`6,144,837 A
`6,160,478 A
`6,168,563 Bl
`6,190,324 Bl
`6,266,645 Bl
`6,309,342 Bl
`6,319,199 Bl
`6,336,900 Bl
`6,353,839 Bl
`6,375,614 Bl
`6,386,882 Bl
`6,416,471 Bl
`6,418,346 Bl
`6,440,068 Bl
`6,450,955 Bl
`6,458,080 Bl
`6,478,736 Bl
`6,524,189 Bl
`6,529,771 Bl
`6,602,191 B2
`6,605,038 Bl *
`6,610,012 B2
`6,702,719 Bl *
`6,736,759 Bl
`6,790,178 Bl
`6,816,603 B2 *
`6,856,832 Bl
`6,936,007 B2
`6,976,958 B2 *
`7,156,809 B2 *
`7,602,301 Bl
`7,739,130 B2 *
`2001/0005830 Al
`2002/0016719 Al
`2002/0019584 Al
`2002/0026223 Al
`2002/0045519 Al*
`2002/0072785 Al
`2002/0082480 Al
`
`8/ 1996 Tacklind et al.
`9/ 1996 Chen et al.
`11/1996 Stutman et al.
`2/1997 Quy
`5/ 1997 Tacklind et al.
`10/ 1997 Sellers
`12/1997 Simmons et al.
`l/ 1998 Tacklind et al.
`3/ 1998 Tavori
`3/ 1998 Tacklind et al.
`4/1998 Albert et al.
`5/ 1998 Fuchs
`6/ 1998 Heinonen et al.
`8/1998 Woodard
`7 / 1999 Stark et al.
`8/ 1999 Saltzstein et al.
`8/1999 Brown
`8/ 1999 Iliff
`8/ 1999 Saltzstein et al.
`9/1999 Brown
`9/1999 Layson et al.
`10/1999 Asada et al.
`10/1999 Ridgeway
`11/1999 Klein et al.
`1111999 Peifer et al.
`12/1999 Brown
`1/2000 Root et al.
`2/2000 Iliff
`2/2000 Surwit et al.
`4/2000 Braun et al.
`4/2000 Frasca, Jr.
`5/2000 Dempsey et al.
`5/2000 Hickman
`7 /2000 Leseicki
`7/2000 Filangeri
`8/2000 Brown
`11/2000 Quy
`12/2000 Jacobsen et al.
`1/2001 Brown
`2/2001 Kieval et al.
`7/2001 Simpson
`10/2001 Blazey et al.
`11/2001 Sheehan et al.
`1/2002 Alleckson et al.
`3/2002 King et al.
`4/2002 Braun et al.
`5/2002 Linberg
`7/2002 Kumar et al.
`7/2002 Nelson et al.
`8/2002 Brown et al.
`9/2002 Brown et al.
`10/2002 Brown et al.
`11/2002 Mault
`2/2003 Rautila
`3/2003 Kieval et al.
`8/2003 Quy
`8/2003 Teller et al. ................... 600/300
`8/2003 Mault
`3/2004 Brown et al.
`5/2004 Stubbs et al.
`9/2004 Mault
`11/2004 David et al. ................... 382/107
`2/2005 Matsumura et al.
`8/2005 Quy
`12/2005 Quy .............................. 600/301
`1/2007 Quy .............................. 600/301
`10/2009 Stirling et al.
`6/2010 Surwit et al. ...................... 705/3
`6/2001 Kuroyanagi
`2/2002 Nemeth et al.
`2/2002 Schultze et al.
`2/2002 Riff et al.
`4/2002 Watterson et al. .............. 482/54
`6/2002 Nelson et al.
`6/2002 Riff et al.
`
`..................... 482/8
`
`2002/0109600 Al *
`2002/0120310 Al
`2003/0004554 Al
`2003/0036683 Al *
`2003/0065257 Al*
`2003/0072424 Al
`2003/0134714 Al*
`2003/0139785 Al
`2003/0204413 Al
`2004/0162466 Al *
`2005/0038326 Al*
`2005/0080322 Al*
`2005/0228245 Al
`2007/0033154 Al*
`2009/0093341 Al
`
`8/2002 Mault et al. ................ 340/573.1
`8/2002 Linden et al.
`1/2003 Riff et al.
`.................... 600/300
`2/2003 Kehr et al.
`4/2003 Mault et al. ................... 600/407
`4/2003 Evans et al.
`7/2003 Oishi et al ......................... 482/6
`7/2003 Riff et al.
`10/2003 Riff
`8/2004 Quy .............................. 600/300
`2/2005 Mathur ......................... 600/300
`4/2005 Korman ........................ 600/300
`10/2005 Quy
`2/2007 Trainum et al. ................... 707 /l
`4/2009 James et al.
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`JP
`KR
`KR
`KR
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`11047101
`11122369
`11259783
`2002/344660
`10-0474926 Bl
`10-2008-0028577 A
`10-2008-0098457 A
`WO 95/32480
`WO 97/28736
`WO 97/28739
`WO 98/24358
`WO 98/38909
`WO 99/04687
`WO 99/14882
`WO 99/41682
`WO 99/44494
`WO 99/46718
`WO 00/36900
`WO 00/40145
`00/54206
`WO 00/54205
`WO 00/62662
`WO 01/24038
`
`2/1999
`4/1999
`9/1999
`11/2002
`3/2005
`4/2008
`11/2008
`11/1995
`8/1997
`8/1997
`6/1998
`9/1998
`2/1999
`3/1999
`8/1999
`9/1999
`9/1999
`6/2000
`7/2000
`9/2000
`9/2000
`10/2000
`4/2001
`
`OTHER PUBLICATIONS
`
`"Bluetooth Technical Background" press release from May 26, 1998.
`Found
`at
`http://web.archive.org/web/19990427070141/www.
`bluetooth.corn/default.asp.
`Jyrki Oraskari; "Bluetooth versus WLAN IEEE 802. l lx"; Helsinki
`University of Technology (Department of Computer Science and
`Engineering) Nov. 2000.
`Jack Smith; Your Personal Health Buddy; ABCNews.com; http:/
`abcnews.go.corn/sections/tech/CuttingEdge/cuttingedge990225.
`html; 3 pages (Nov. 24, 2000).
`The Health Hero Communications Platform; The Health Hero Net(cid:173)
`work Online Services; http:/www.hhn.com/products/index.html; 2
`pages (Nov. 24, 2000).
`Painless Blood-Glucose Monitoring; Kumertrix Technology Over(cid:173)
`view; http:/www.kumertrix.com/technology.html; 2 pages; Nov. 24,
`2000.
`Technology & Clinical Results-Simple Solutions Through Tech(cid:173)
`nology-Progression of Glucose Monitoring Technology; Amira;
`http:/arnira.corn/tech/ic_tech.htm; 2 pages; Nov. 24, 2000.
`Wired for Wellness; LifeChart.com; http:/www.lifechart.com; 2
`pages; Nov. 24, 2000.
`About data Critical Corporation; Yahoo-data Critical to Provide
`Mallincrodt with Wireless Connectivity for Ventilators; http:/biz.
`yahoo.corn/pmews/00102/mo_rnallinc.html; 1 page; Nov. 24, 2000.
`Bluetooth wireless technology-bridging the gap between comput(cid:173)
`ing and communication; Bluetooth Technology; http://www.intell.
`commobile/bluetooth/; 2 pages; Nov. 28, 2000.
`Bluetooth resource center: What is Bluetooth?; palowireless.com;
`http:/www.palowireless.com/infotooth/watis.asp; 3 pages; Nov. 28,
`2000.
`Bluetooth Tutorial; palowireless.com-Bluetooth resource center;
`http://www.palowireless.com/infortooth/tutorial.asp; 4 pages; Nov.
`28, 2000.
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 4 of 23 Page ID #:2410
`
`US 8,712,510 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Nick Hunt; Bluetooth Venus 802.11; TDK Systems; http://www.
`cellular.com.za/bluetooth_versus_802.htm; 4 pages; Nov. 28, 2000.
`Bluetooth vs. Airport (802.11 Network); palowireless.com(cid:173)
`Bluetooth resource center; http://www.palowireless.com/infotooth/
`knowbase/ofthenetworks/15asp; 3 pages; Nov. 28, 2000.
`Personal Digital Assistants; A2 Anytime/anywhere-a Weekly on
`Wireless Infrastructure and Data Services; Thomas Weisel Partners
`(Merchant Banking); 5 pages; Nov. 29, 2000.
`Ashlee Vance; Ericsson and Intel Make Bluetooth Pact; Info World.
`com;
`http://www.infoworld.com/articles/hn/xml/OO/ 12/047 /
`001204hnericitel.xml?T.../printarticle.htm; 1 page; Dec. 4, 2000.
`Personal Portable Office; Nokia 9000II digital; http://www.nokiausa.
`corn/9000II; 4 pages; Dec. 7, 2000.
`Pul-Wing Tarn; Handspring Homes; Article from the Wall Street
`Journal; Section B; Nov. 2000.
`Author unknown; Articles on Phones and New Technologies; Article
`from the Wall Street Journal; Nov. 2000.
`David Pringle; Sagen to Launch hand-held computer that doubles as
`top-end mobile phone; Article from the Wall Street Journal ; Nov.
`2000.
`Svensson, Peter; "Cisco Launches WiFi Phone" Article from Austra(cid:173)
`lian IT; Apr. 29, 2003.
`"Breakthrough Devices Shown at ADA" published in Diabetes News
`for Jul. 1, 2001 at http://www.diabetesnet.com/news/news070101.
`php.
`iMetrikus published at http://www.qualcomm.com/gwbs/resource/
`resourcelib_casestdy.shtml.
`"Applications of MedStar" published on Apr. 27, 2003 by Cybernet
`Medical, 16 pages.
`"HIPP A & WiFi: Regulatory Tangles for Wireless Health Care Net(cid:173)
`works Analyzed" published at http://www.hipaadvisory.com/tech/
`wireless.htm.
`"Medtronic CareLink Network, How it Works" published at http://
`www. medtronic.corn/carelink/features.html.
`"FDA Approves Medtronic CareLink™ Monitor and Software,
`Opening a New Chapter in Patient Management Using Internet Tech(cid:173)
`nology", Medtronic News Release dated Jan. 2, 2002.
`"The MedStar System, How the MedStar System Works" brochure
`published by Cybernet Medical.
`"IMetrikus Mobile Solutions" brochure by IMetrikus, Inc.
`"Instromedix-Products" published at www.instromedix.com/
`pages/products/ products.asp 7 pages.
`
`EFI Framework Draft Version 0.8 (Jun. 3, 2000): External function(cid:173)
`ality Interface Framework; pp. 1-35.
`Internet Press Release: New York Business Wire (Sep. 25, 2000);
`MedSearch Technologies, Inc. Develops a Revolutionary Home(cid:173)
`Care Wireless Technology Utilizing PDAs-Personal Organizers-as
`Patient Monitors.
`"Cell Phone Cameras Put Doctors in the Picture", Feb. 21, 2005, 1
`page, http://news.healingwell.com/index.php?p-newsl&id-524 l l 8.
`Yan Xiao, PhD, et al. "Design and Evaluation ofa Real-Time Mobile
`Telemedicine System for Ambulance Transport", Proceedings of the
`1998 American Medical Informatics Association Annual Fall Sym(cid:173)
`posium, 1998, pp. 1102-1103.
`Yan Xiao, PhD et al. "Design and Evaluation ofa Real-Time Mobile
`Telemedicine System for Ambulance Transport", The Journal of
`High Speed Networks, 2000, vol. 9(1), pp. 47-58.
`Ky Kong et al., "Web-Based Monitoring of Real-Time ECG Data",
`Computers in Cardiology, 2000, vol. 27, pp. 189-192.
`Juha Parkka et al. "A Wireless Wellness Monitor for Personal Weight
`Management", Proc. 2000 IEEE EMBS Int'L Conf. On Info Tech.
`Applications in Biomedicine, 2000, pp. 83-88.
`Joseph Finkelstein et al. "Web-based Monitoring of Asthma Severity:
`A New Approach to Ambulatory management", Proc. 1998 IEEE
`Int'! Conf. on Info Tech. Applications in Biomedicine, 1998, pp.
`139-143.
`Gary D. Havey et al. "A Wearable Polysornnograph With an RF Link
`to a Personal Computer, Recorder, or the Internet", Proceedings of
`the 22nd Annual EMBS International Conference, Jul. 23-28, 2000, p.
`1264.
`Emil Jovanov et al. Stress Monitoring Using a Distributed Wireless
`Intelligent Sensor System:, IEEE Engineering in Medicine and Biol(cid:173)
`ogy Magazine, May/Jun. 2003, pp. 49-55.
`SP Nelwan et al. "Ubiquitous Mobile Access to Real-Time Patient
`Monitoring Data", Computers in Cardiology, 2002, vol. 29, pp. 557-
`560.
`Brent Priddy et al., "Wireless Distributed Data Acquisition System",
`2002 IEEE, pp. 463-466.
`Emil Jovanov et al. "Prolonged Telemetric Monitoring of Heart Rate
`Variability Using Wireless Intelligent Sensors and a Mobile Gate(cid:173)
`way", Proceedings of the Second Joint EMBS/BMES Conference,
`Oct. 23-26, 202, pp. 1875-1876.
`"Bluetooth Technical Background" Press releast from May 26.
`Yan Xiao et al., Design and Evaluation of a Real-Time Mobile
`Telemedicine System for Ambulance Transport, Proceedings of the
`AMIA Symposium, 1998, p. 1102.
`
`* cited by examiner
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 5 of 23 Page ID #:2411
`
`U.S. Patent
`
`Apr. 29, 2014
`
`Sheet 1 of 9
`
`US 8,712,510 B2
`
`i.n
`
`T"'"
`c.:,
`u..
`
`,-i
`r
`
`I O'):
`~ 1------------------------------------------------------
`
`00
`
`C'? I
`0 ==--------'
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 6 of 23 Page ID #:2412
`
`42-,,
`
`,.--10
`
`_____ i _________________________________ _
`
`24 1
`
`·-,
`1--l.(cid:173)
`l
`
`:
`'----
`c::::J
`
`I ◄------1-7
`
`11
`
`40
`
`I-""
`I
`
`- - - - - - - - - - -
`
`~
`r:J)_
`•
`~
`~
`~
`
`~ = ~
`
`- - -1
`I
`I
`I
`I
`I
`I
`I
`I
`
`,-44
`, - . · -----------~-i
`™
`,
`24-,
`I
`'
`,-1.-
`I
`:
`:
`HMO
`:
`_______________ L:------------, ---[ ___________________________________ J WID
`12 i
`i--____ ---------------------------------i---
`',_ 44 I
`:
`36
`:
`.--..i...-----------,
`2s:
`:
`rh ----26
`34-,,J
`t/-32
`'
`CY
`i
`i
`Mouse
`
`:
`
`1
`I
`
`I
`
`I
`
`I
`I
`I
`r---
`
`ooOoo
`
`I
`I
`
`I
`
`I
`I
`I
`---7
`
`> -e :-:
`
`N
`~\,Ci
`N
`
`0 ....
`
`.i;...
`
`rJJ =(cid:173)
`
`('D
`...,._
`('D
`N
`
`0 ....
`
`\,Ci
`
`30) --~
`Key Board
`
`·-,
`
`28 1
`:"f o~c-h:
`i Screen i
`
`I
`
`I
`
`L - - - - - - - - - - - - - -~
`
`37
`'--36
`---------
`Computer
`
`___ _.
`
`FIG. 2
`
`d r.,;_
`
`00
`~
`"""' N
`tit
`
`"""' = = N
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 7 of 23 Page ID #:2413
`
`U.S. Patent
`
`Apr. 29, 2014
`
`Sheet 3 of 9
`
`US 8,712,510 B2
`
`244
`
`246
`
`FIG. 3
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 8 of 23 Page ID #:2414
`
`U.S. Patent
`
`Apr. 29, 2014
`
`Sheet 4 of 9
`
`US 8,712,510 B2
`
`... a.,
`> ... a.,
`
`en
`
`N
`CD
`N
`N-------
`
`JI]
`
`oi:::I"
`c.:::J
`u..
`
`'--
`
`(]) > '-(cid:173)
`
`(])
`Cf)
`
`co
`
`~ ---.--
`
`-s::j""
`~
`
`C:
`0
`
`~ ......
`
`Cf)
`
`Q)
`
`en ro
`o::::i
`
`-s::j""
`N
`N
`
`co
`Cl)
`
`

`

`------------------
`
`I
`I
`
`'
`
`I
`I
`
`Application
`68---J I
`N Presentation Layer
`:
`:
`
`I
`I
`I
`
`Sensor
`Input
`
`L
`
`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 9 of 23 Page ID #:2415
`
`10,
`
`Wireless Health Monitoring Apparatus
`
`22-,
`:
`
`___ L _________________ _
`I
`
`Server
`
`-~
`
`,
`
`I
`I
`I
`
`.------------~
`I
`I
`I
`I
`
`------------
`:
`I
`
`I
`I
`I
`
`,
`
`I
`
`~
`
`I
`
`,..1
`
`I
`I
`: :,- 76
`Al
`:
`: System f'
`
`7 4--~ : External :
`: -~ Data
`:
`
`:
`
`I
`I
`I
`I
`I ~----,--------'
`I
`
`:
`I
`I
`I
`~-------.---- I
`
`72
`
`' \
`
`,..
`
`63--.J,
`: --.... , - - - r - - - ' - - - ' - -~ - -~
`:
`',, ____ !Algorithm!
`
`.. ,
`
`,
`
`I
`
`I
`
`70
`I ~
`
`Application
`
`I
`I
`
`52-...,
`
`I
`I
`L ___________ I
`
`Application
`
`I
`I
`I
`I
`I
`I
`I
`
`\ ______________________ _
`
`Manual
`Input
`
`35..J
`
`37'
`
`□
`
`Computer
`
`FIG. 5
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`> "e
`:-:
`N
`~\,Ci
`N
`
`0 ....
`
`.i;...
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`Ul
`0 ....
`
`\,Ci
`
`d r.,;_
`
`00
`~
`"'""' N
`tit
`
`"'""' = = N
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 10 of 23 Page ID #:2416
`
`24~
`Start l
`
`240
`
`User
`interacts W/WID
`
`216
`
`•\
`
`Sensor
`measures
`parameter
`'-
`218
`Sensor sends
`parameter to
`exercise machine
`220
`Exercise machine
`sends parameter
`to WID
`
`218 1
`Sensor sends
`parameter to WID
`
`222
`WID wirelessly
`communicates
`parameter to
`server application
`224
`,___ ________ _
`Server application
`processes
`parameter
`
`FIG. 6A
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`> "e
`:-:
`N
`~\,Ci
`N
`
`0 ....
`
`.i;...
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`O'I
`0 ....
`
`\,Ci
`
`d r.,;_
`
`00
`~
`"'""' N
`tit
`
`"'""' = = N
`
`225,
`, - - - .J - - - - - - - -1
`Algorithm
`l 63 employed
`------------'
`
`:
`
`1
`
`1
`
`241 1
`
`Visual
`data is captured
`241
`
`User sends
`supplemental
`data to server
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 11 of 23 Page ID #:2417
`
`D
`____ - --- - - - --~ = E~~r~~l=d-ata--L,-
`~ CB) CC
`---------,
`l
`, source em_p!_o_y~~:
`--J-' -;r; - -- - -22h
`232
`Server application
`calculates
`response
`,
`,
`,
`
`I_ - - - - - -
`. - - - - - - - - - - - -1
`~:
`Al
`:~/-234
`: system employed 1
`____________ I
`
`1,.
`
`I
`
`228
`Server application
`sends
`response to WID
`230
`
`Response
`is displayed
`
`,..__ __ _
`
`236 ,
`r - - . . l - - - - - - - -1
`:server application:
`:
`displays
`:
`:
`response on
`:
`: client computer :
`L-----T _____ ..
`238,
`: Specialist notifies~ ___ _
`: user of response :
`
`I
`. - - - . . l - -L - - - - -1
`
`228 1
`Server application
`sends response
`to alternate WID
`230 1
`
`Response
`is displayed
`
`FIG. 6B
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`> "e
`:-:
`N
`~\,Ci
`N
`
`0 ....
`
`.i;...
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`-....J
`0 ....
`
`\,Ci
`
`d r.,;_
`
`00
`~
`"'""' N
`tit
`
`"'""' = = N
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 12 of 23 Page ID #:2418
`
`U.S. Patent
`
`Apr. 29, 2014
`
`Sheet 8 of 9
`
`US 8,712,510 B2
`
`0
`
`r---------------------------------------------------------
`N
`(,0
`
`0
`(,0
`
`~ I \j
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`0
`0
`a □
`0
`0
`
`co
`(,0
`~
`
`0
`(,0
`~
`
`......
`C!J
`LL
`
`.,-------.L(cid:173)
`f, ---------
`' I
`~---J,__
`
`~ I
`I
`I
`I
`1----
`
`D
`
`□
`
`0
`~
`:::r::
`
`N
`LO~
`
`I
`I
`I
`I
`I
`I
`✓
`I
`.._ __ _J.1
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 13 of 23 Page ID #:2419
`
`U.S. Patent
`
`Apr. 29, 2014
`
`Sheet 9 of 9
`
`US 8,712,510 B2
`
`C) · - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`N
`CD
`r -
`
`C)
`CD
`
`0
`0
`a □
`0
`0
`
`00
`C!J
`u..
`
`C)
`CD
`r -
`
`Lt')
`CD
`r -
`
`□
`
`□
`
`0
`~ :r:
`
`,.-------.L(cid:173)
`f-,. ---------
`' I
`~----L
`
`r -
`
`I
`I
`I
`I
`1----
`
`N
`L t ' )~
`I
`I
`I
`I
`I
`I
`,
`~--.J/
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 14 of 23 Page ID #:2420
`
`US 8,712,510 B2
`
`1
`METHOD AND APPARATUS FOR EXERCISE
`MONITORING COMBINING EXERCISE
`MONITORING AND VISUAL DATA WITH
`WIRELESS INTERNET CONNECTIVITY
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`2
`tery space of the mobile phone and is connected to a commu(cid:173)
`nication bus of the mobile phone. Other systems have been
`proposed, but these suffer from similar deficiencies in that
`they require specially modified cellular phones to be
`5 employed.
`The deployment of the above systems also currently lacks
`employment of full back-end server functionality with which
`to provide a wide range of interactive communication with
`the patient. Instead, such systems, if internet-enabled, are
`10 often limited to mere one-way non-interactive data transfer
`via a modem.
`
`SUMMARY OF THE INVENTION
`
`This application is a continuation-in-part of U.S. patent
`application Ser. No. 11/649,355, filed Jan. 3, 2007, entitled
`"Method andApparatus For Health and Disease Management
`Combining Patient Data Monitoring With Wireless Internet
`Connectivity," now abandoned, which is a continuation of
`U.S. patent application Ser. No. 11/156,177, filed Jun. 17,
`2005, entitled "Method and Apparatus For Health and Dis- 15
`ease Management Combining Patient Data Monitoring With
`Wireless Internet Connectivity," now U.S. Pat. No. 7,156,
`809, which is a continuation-in-part ofU.S. patent application
`Ser. No. 10/773,501, filed Feb. 6, 2004, now U.S. Pat. No.
`6,976,958, the entirety of each prior application being incor- 20
`porated by reference herein.
`
`Embodiments of the present invention overcome one or
`more of the disadvantages of the prior art by providing a
`full-feature health and exercise monitoring system that may
`wirelessly connect to a back-end server application via the
`internet. The invention allows wireless access to and from a
`wide variety of present exercise or health-related instruments
`and devices, while maintaining the capability of connecting
`to future such devices.
`The advent of multimedia mobile phones and other per(cid:173)
`sonal communications devices that include a digital camera,
`25 or are able to connect with one, allows the capture and trans(cid:173)
`mission ofhealth and exercise information, including images.
`In embodiments of the present invention, multimedia mobile
`phones are used to transmit voice and images as well as data
`from an exercise monitoring device. In addition to the upload-
`30 ing of images, the display screen of a wireless internet device
`("WID"), such as a smart phone, is used to allow the display
`of images such as illustrations, diagrams or video clips which
`may be downloaded from a server as part of an interactive user
`interface ( e.g., for the purpose of describing to a user how to
`35 set up an exercise regime). The ability to include images in a
`system that is based on a WID connected to a exercise moni(cid:173)
`toring device helps facilitate the remote analysis and moni(cid:173)
`toring of exercise, fitness, nutrition, or health.
`The reviewer or provider of exercise or health information
`40 is understood to include not only the user but also a trainer,
`coach, specialist, caregiver, physician, another user, or a soft(cid:173)
`ware application or algorithm that may analyze the informa(cid:173)
`tion. The information can include data from a variety of
`monitoring devices and images relating to the condition of the
`45 user. The images could include photographs or videos of the
`user's specific situation that could aid their exercise regime.
`In particular, the invention may be embodied in several
`systems. Two complementary such systems are described
`herein, although extensions to other such systems can be
`50 envisioned. In the first embodiment, a health or lifestyle man(cid:173)
`agement plan may be implemented. Various exercise param(cid:173)
`eters, such as those relating to fitness, nutrition or exercise,
`may be received from an exercise monitoring device, and the
`same may be wirelessly communicated to a server. The exer-
`55 cise parameters may include physiological data, such as heart
`rate, blood pressure, respiration, or temperature; or data cor(cid:173)
`responding to the amount of exercise performed, such as
`number of miles traveled, how much work performed, or the
`like. In some cases, derived data, such as number of calories
`60 burned, may also be measured by an appropriate calculation.
`An application may process, store, and perform calculations
`on the exercise parameters, these may be reported back to the
`user, and a health, nutrition, or fitness specialist or other
`reviewer may optionally review the same or different derived
`65 or measured values. The second embodiment of the invention
`may be employed to manage the disease state or condition of
`a patient. In this embodiment, the patient, or a caregiver, may
`
`REFERENCE TO GOVERNMENTAL SUPPORT
`
`(none)
`
`REFERENCE TO MICROFICHE APPENDIX
`
`(none)
`
`FIELD OF THE INVENTION
`
`The present invention relates to monitoring of living sub(cid:173)
`jects, such as during exercise, and more particularly to exer(cid:173)
`cise monitoring of persons where measured or input exercise
`data is communicated by a mobile communications device to
`and from the internet.
`
`BACKGROUND OF THE INVENTION
`
`Several attempts have been made in the past to achieve
`efficient interactive communication of medical or health
`information between a subject or patient and a reviewer or
`provider of that information. However, in general, previous
`systems have limited the user to the general location in which
`the device was located. For example, in U.S. Pat. No. 5,441,
`047, images and data were transmitted by standard telephone
`lines or wireless telemetry systems.
`Even where devices are portable, as in the case of a laptop
`computer with a modem, an ordinary POTS phone line must
`be found and used. Where the user's computer employs a
`broadband connection, such as DSL or satellite, the choices
`of location are even more limited.
`Attempts have been made to remedy this deficiency. As
`noted above, some telemetry systems allow a "wireless" dis(cid:173)
`tance to be placed between a health measuring unit and a
`remote monitoring system. However, such systems are lim(cid:173)
`ited in their range.
`Other systems have used cellular telephone technology to
`increase the wireless health monitoring range. However,
`these systems have several deficiencies, such as requiring
`significant modification of the mobile phone. For example,
`U.S. Pat. No. 5,772,586, issued Jun. 30, 1998 to Heinonon et
`al., discloses a method for monitoring the health of a patient.
`This system uses a specialized connection between the
`patient health measuring unit and the cellular phone, how(cid:173)
`ever. The patient health measuring unit is located in the bat-
`
`

`

`Case 2:19-cv-06301-AB-KS Document 80-3 Filed 07/09/20 Page 15 of 23 Page ID #:2421
`
`US 8,712,510 B2
`
`3
`employ a health monitoring device ("HMD"), in particular a
`medical device, and a wireless connection to provide data
`from the medical device for processing via the internet,
`including a review by a physician or other health care profes(cid:173)
`sional if required.
`In other embodiments, the condition of subjects may be
`evaluated by collecting exercise, health, or medical data and
`providing information in response to those data by means of
`a WID designed to display interactive information through a
`connection to the Internet. The present invention may be 10
`connected to various HMDs, and may communicate informa(cid:173)
`tion via a wireless connection such as a wireless Internet
`connection. The user of the present invention may be the
`subject, or another person such as a trainer, coach, or physi(cid:173)
`cian. Wireless internet connectivity has many advantages. For 15
`example, in the first embodiment, a person interested in track(cid:173)
`ing an exercise program may take the WID to the local health
`club and attach the same to an exercise machine, send data
`output from various exercise machines over the Internet,
`which may include data relating to how much exercise or 20
`work was performed, and physiological data such as heart
`rate, and receive a personalized response from an application
`program running on a server. The individual may input caloric
`content of foods eaten, and may further input caloric content
`of exercise performed. In this way, e.g., a person in a weight- 25
`loss program may see in great detail whether they are expend(cid:173)
`ing more calories in the form of exercise than the same indi(cid:173)
`vidual is consuming in the form of food. Alternatively, in the
`second embodiment, a diabetic could connect a blood glucose
`meter to a WID away from home and download data to a 30
`Diabetes Management Company's server and, in response,
`receive guidance displayed on the screen ( or by voice) about
`choices for the next meal.
`In general, in the health management embodiment, the
`system may be employed to monitor the physiological status
`of a healthy subject while exercising, or performing other
`activities by receiving data from a variety of exercise moni(cid:173)
`tors that provide physiological data and/or data correspond(cid:173)
`ing to the amount of exercise or work performed. Exercise
`monitors that provide physiological data may include heart
`rate monitors, respiration rate monitors, blood pressure moni(cid:173)
`tors, accelerometers, pedometers, body weight scales, body
`fat gauges, biofeedback devices, physiotherapy or chiroprac-
`tic equipment, or the like, and the same may be separate
`devices or devices incorporated within an exercise machine. 45
`Exercise machines may include, e.g., treadmills, rowing
`machines, steppers, other aerobic or anaerobic exercise
`equipment, weight machines, or any other type of exercise
`machine. Exercise data corresponding to an amount of exer(cid:173)
`cise performed may be measured by such machines and sent 50
`to the WID as described. For clarity, such devices are termed
`herein "exercise monitors" or "exercise monitoring devices"
`and the same measure "exercise parameters" or "exercise
`data". Exercise parameters or data may include the above
`physiological data and/or data corresponding to an amount of 55
`exercise or work performed.
`In more detail, embodiments of the present invention pro(cid:173)
`vide a method and system for assisting subjects to maintain a
`healthy lifestyle by collecting and transmitting exercise data
`and in tum receiving information in response to those data by 60
`means of a WID designed to display interactive information
`through a connection to the Internet. The present invention
`may be connected to various HMDs, both medical and exer(cid:173)
`cise-related in nature, and may communicate information via
`a wireless connection such as a wireless Internet connection. 65
`A major advantage of embodiments of the invention is that
`the same frees the patient from the constraints of wired sys-
`
`4
`terns. The same allows users with consumer "off-the-shelf'
`wireless devices, e.g. mobile phone, to significantly extend
`the range of connectivity over that of wired computers, or
`even wireless telemetry systems.
`The WID may be a web-enabled mobile phone. Alterna(cid:173)
`tively, the WID may be a PDA, palm, handheld, tablet, net(cid:173)
`book, or laptop computer, equipped with a wireless modem.
`Besides these, the WID may be a hybrid device that combines
`the functions of a computer, PDA and telephone in some
`fashion. It should also be noted that the WID may be a web(cid:173)
`enabled mobile phone or hybrid device using a satellite com-
`munications network.
`In a separate embodiment, an adaptor is used if necessary
`to convert the output signal of the exercise monitoring device
`to a suitable input signal for the, e.g., WID. The adaptor
`allows connection of the WID to an exercise monitor, either
`separately or one that forms part of an exercise machine or
`other variety of health care equipment, and the connection
`may be made via several techniques.
`As for wired techniques, a standard parallel bus, universal
`serial bus (USB), Firewire, serial cable, or similar industry(cid:173)
`standard connection may be used if the input/output ports
`between the HMD and the WID are appropriate. Otherwise, a
`suitable separate adaptor may be employed.
`The connection may also be an input such as a memory
`device reader, a disk drive or other media input for input of
`data, a USB port or phone jack or other such wired input,
`again employing an adaptor if required.
`As for wireless techniques, infrared (IR), microwaves,
`radio frequency (RF), a variety of cellular protocols, a variety
`of 802.11 protocols, 802.15 protocols, 802.16 protocols,
`802.20 protocols, other IEEE 802 family protocols, short
`range wireless transmission, wide area network or broadband
`transmission methods, such as Bluetooth®, wireless univ