USOO945 1084B2
`
`(12) United States Patent
`Kurganov et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9.451,084 B2
`*Sep. 20, 2016
`
`(54) ROBUST VOICE BROWSER SYSTEMAND
`VOICE ACTIVATED DEVICE CONTROLLER
`
`(75) Inventors: Alexander Kurganov, Buffalo Grove,
`IL (US); Valery Zhukoff, Deerfield, IL
`(US)
`(73) Assignee: Parus Holdings, Inc., Buffalo Grove,
`IL (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 334 days.
`This patent is Subject to a terminal dis
`claimer.
`
`(*) Notice:
`
`(21) Appl. No.: 13/462,819
`(22) Filed:
`May 3, 2012
`
`(65)
`
`Prior Publication Data
`US 2012/0215545 A1
`Aug. 23, 2012
`
`Related U.S. Application Data
`(63) Continuation of application No. 12/973,475, filed on
`Dec. 20, 2010, now Pat. No. 8,185,402, which is a
`continuation of application No. 12/030,556, filed on
`Feb. 13, 2008, now Pat. No. 7,881,941, which is a
`(Continued)
`
`(2006.01)
`(2006.01)
`(2006.01)
`
`(51) Int. Cl.
`H04M 3/493
`GIOL I5/26
`GO6F 3/16
`(52) U.S. Cl.
`CPC ............ H04M 3/4938 (2013.01); G10L 15/26
`(2013.01); G06F 3/16 (2013.01); H04M
`220 1/39 (2013.01); H04M 220 1/40 (2013.01);
`Y10S 707/99942 (2013.01); Y10S 707/999.43
`(2013.01)
`
`(58) Field of Classification Search
`CPC ............................ G06F 3/167; G06F 17/3043
`USPC ............................................... 704/275, 270.1
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3, 1876 Bell
`174,465. A
`4, 1973 Kraus
`3,728,486 A
`4,058,838 A 11/1977 Crager et al.
`4,100,377 A
`7/1978 Flanagan
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CA
`EP
`
`5, 1994
`1329852
`9, 1996
`O 572 544
`(Continued)
`OTHER PUBLICATIONS
`
`“A PABX that Listens and Talks”. Speech Technology, Jan. (Feb.
`1984, pp. 74-79.
`
`(Continued)
`
`Primary Examiner — Susan McFadden
`(74) Attorney, Agent, or Firm — Patent Law Works LLP
`
`ABSTRACT
`(57)
`The present invention relates to a system for acquiring
`information from sources on a network, Such as the Internet.
`A Voice browsing system maintains a database containing a
`list of information sources, such as web sites, connected to
`a network. Each of the information sources is assigned a
`rank number which is listed in the database along with the
`record for the information Source. In response to a speech
`command received from a user, a network interface system
`accesses the information source with the highest rank num
`ber in order to retrieve information requested by the user.
`15 Claims, 4 Drawing Sheets
`
`500
`
`*
`
`SECURITY
`systE
`
`NOR
`GNG
`
`OUTDOOR
`GING
`
`
`
`
`
`WCR
`
`o & O
`
`DECEN
`
`506 -
`
`DEWICE
`BROWSNG
`SERVER
`
`DEVICE
`BROWSNG
`SERVER
`
`5 O 5
`
`CPip
`
`AABASE
`
`MEA
`SERVER
`
`MEA
`SERVER
`
`508
`
`50
`
`TLINE
`
`51 O
`
`
`
`

`

`US 9.451,084 B2
`Page 2
`
`Related U.S. Application Data
`continuation of application No. 11/409,703, filed on
`Apr. 24, 2006, now Pat. No. 7,386,455, which is a
`continuation of application No. 10/821,690, filed on
`Apr. 9, 2004, now Pat. No. 7,076,431, which is a
`ity of praise 9,996. filed on
`eb. ),
`, now Pat. No. 6,
`s
`(60) Provisional application No. 60233,068, filed on Sep.
`15, 2000, provisional application No. 60/180.344,
`filed on Feb. 4, 2000.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,158,750 A * 6/1979 Sakoe et al. .................. TO4/251
`4,313,035 A
`1, 1982 Jordan et al.
`4,327.251 A
`4, 1982 Fomenko et al.
`4,340,783. A
`7/1982 Sugiyama et al.
`4,340,797 A * 7, 1982 Takano et al. ................ 219,707
`4,340,800 A * 7/1982 Ueda et al. ................... 219,714
`
`9/1990 Morganstein et al.
`4,955,047 A
`9/1990 Grover
`4.956,835 A
`9, 1990 Cave et al.
`4.959,854 A
`4,967.288 A 10/1990 Mizutori et al.
`4.969,184 A
`1990 SR et al.
`373 A RE SE a
`4,975,941. A
`12, 1990 Morganstein et al.
`4,985,913 A
`1/1991 Shalom et al.
`4,994,926 A
`2/1991 Gordon et al.
`4,996,704 A
`2f1991 Brunson
`5,003,575 A
`3, 1991 Chamberlin et al.
`5,003,577 A 38: Mich 1
`5,008,926 A
`4/1991 Misholi
`5,020,095 A
`5/1991 Morganstein et al.
`5,027,384. A
`6/1991 Morganstein
`5,029, 196 A
`7, 1991 Morganstein
`5,036,533 A
`7/1991 Carter et al.
`5,054.054 A 10, 1991 Pessia et al.
`3. A
`36
`R. "
`WW -
`39. A
`22: S. al.
`5,099.509 A
`3/1992 Morganstein et al.
`5.09.405 A
`4, 1992 Morganstein
`5.138984. A
`7/1992 Kat
`
`4,481,574. A 1 1/1984 DeFino et al.
`4.489.438 A 12/1984 Hughes
`4,500,751 A
`2/1985 Darland et al.
`4.513,390 A
`4, 1985 Walter et al.
`4,523,055 A
`6/1985 Hohl et al.
`4,549,047 A 10, 1985 Brian et al.
`4,584,434. A
`4, 1986 Hashimoto
`4,596,900 A
`6/1986 Jackson
`4,602,129 A
`7/1986 Matthews et al.
`4,635,253 A
`1/1987 Urui et al.
`4,652,700 A
`3, 1987 Matthews et al.
`4,696,028 A
`9/1987 Morganstein et al.
`4,713,837. A 12/1987 Gordon
`4,747,127 A
`3. East el.
`SS A
`3.
`th
`- -
`4,757.525. A
`7/1988 Matthews et al.
`4,761,807 A
`8, 1988 Matthews et al.
`38. A 83. than et al.
`W-1
`27, A
`88.
`al.
`w
`4,782,517 A 1 1/1988 Bernardis et al.
`4,792.968 A 12/1988 Katz
`4,799,144. A
`1/1989 Parruck et al.
`4,809.321 A
`2/1989 Morganstein et al.
`3-6 A
`23. 8R a
`w -
`- c.
`35, A
`23. Set al
`w - W
`4,852,149 A
`7/1989 Zwick et al.
`f
`d
`4,852,170 A
`7, 1989 Bordeaux
`4.866,758. A
`9/1989 Heinzelmann
`352. A RE Sea
`4,879,743 A 1 1/1989 Burke et al.
`4,893,335 A
`1/1990 Fuller et al.
`4,903,289 A
`2f1990 Hashimoto
`4.903.291. A
`2/1990 Tsurufuji et al.
`4.905,273 A
`2/1990 Gordon et al.
`4,907,079 A 388 S. et al
`is. A 1982.
`4,922.520 A
`5/1990 Bernard etal.
`5/1990 Morganstein et al.
`4,922,526 A
`5, 1990 Ladd et al.
`4,926.462 A
`4,930.50 A
`5/1990 Kat,
`4,933,966 A
`6, 1990 Hird et al.
`4,935,955 A
`6, 1990 Neudorfer
`4,935,958 A
`6/1990 Morganstein et al.
`4.941,170 A
`7/1990 Herbst
`4.942,598. A
`7/1990 Davis
`4,953,204 A
`8/1990 Cuschleg, Jr. et al.
`
`k - J.
`7/1992 Heileman, Jr. et al.
`5,133,004 A
`9, 1992 Chevalier
`5,145.452 A
`9/1992 Pekarske
`5,146,452 A
`1 1/1992 Morganstein et al.
`5,166.974. A
`1/1993 MacMillan, Jr. et al.
`5,179,585 A
`3/1993 Jones et al.
`5,193,110 A
`3/1993 Baumgartner et al.
`5, 195,086 A
`9/1993 Sattar et al.
`5,243,643 A
`9, 1993 Bissell et al.
`5.243.645 A
`9/1993 Morganstein et al.
`5.249,219 A
`5.255.305 A 10, 1993 Sattar
`5,263,084 A 11/1993 Chaput etal
`5,276.739 A
`1/1994 Higuchi et al.
`5,287,199 A
`2/1994 Zoccolillo
`5,291.302 A
`3/1994 Gordon et al.
`5,291,479 A
`3, 1994 Vaziri et al.
`$565,39. A
`315. Morain et at
`gainstein et al.
`5.307399. A
`4, 1994 Dai et al.
`5,309,504. A
`5/1994 Morganstein et al.
`5,325,421. A
`6/1994 Hou et al.
`5,327,486 A
`7/1994 Wolff et al.
`5.327.529 A
`7, 1994 Fults et al.
`35567. A
`7.6: first al.
`5,333,266 A
`7/1994 Boaz et al.
`5,347,574. A
`9/1994 Morganstein
`5,355,403 A 10/1994 Richardson, Jr. et al.
`5,359,598 A 10/1994 Steagall et al. ............... 704/275
`5,365,524. A 1 1/1994 Hiller et al.
`5,365,574. A 1 1/1994 Hunt et al.
`5,375,161. A 12, 1994 Fuller et al.
`- -
`uller e
`5,384,771 A
`1/1995 Isidoro et al.
`5,404,231 A
`4, 1995 Bloomfield
`5,408,526 A
`4, 1995 McFarland et al.
`5,414.754. A
`5/1995 Pugh et al.
`5,416,834. A
`5/1995 Bales et al.
`5.432,845. A
`7/1995 Burd et al.
`5,436,963 A
`7/1995 Fitzpatricket al.
`5,459,584. A 10/1995 Gordon et al.
`5,463,684. A 10/1995 Morduch et al.
`5,475,791. A 12/1995 Schalk et al.
`5,479,487. A 12/1995 Hammond
`: A 5E R1
`I -
`5,499,288 A
`3/1996 Hunt et al.
`5,515,427 A
`5/1996 Carlsen et al.
`5,517,558 A
`5, 1996 Schalk
`5,526,353 A
`6/1996 Henley et al.
`5,533,115 A
`7/1996 Hollenbach et al.
`5.537,461 A
`7/1996 Bridges et al.
`5,555,100 A
`9/1996 Bloomfield et al.
`5,559,611 A
`9, 1996 Bloomfield et al.
`
`

`

`US 9.451,084 B2
`Page 3
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,559,859
`5,566,236
`5,603,031
`5,608,786
`5,610,910
`5,610,970
`5,611,031
`5,630,079
`5,652,789
`5,657,376
`5,659,597
`5,666.401
`5,675,507
`5,675,811
`5,689,669
`5,692,187
`5,699,486
`5,712,903
`5,719,921
`5,721,908
`5,724,408
`5,737,395
`5,742,596
`5,742,905
`5,752,191
`5,758,322
`5,761,294
`5,764,639
`5,764,736
`5,764,910
`5,774,860
`5,787,298
`5,793,993
`5,794,205
`5,796,791
`5,799,063
`5,799,065
`5,809,282
`5,809,481
`5,812,796
`5,819,220
`5,819,306
`5,822,727
`5,823,879
`5,832,063
`5,832,440
`5,835,570
`5,838,682
`5,867,494
`5,867.495
`5,870,550
`5,873,080
`5,881,134
`5,881,135
`5,884,032
`5,884,262
`5,884,266
`5,890,123
`5,914.951
`5,915.001
`5,917,817
`5,940,598
`5,943,399
`5,946,389
`5,953,392
`5,974,124
`5.974.413
`5.991,292
`5,995,615
`5.999,525
`5.999,611
`5.999,965
`6,012,088
`6,014,437
`6,014,626
`
`9, 1996
`10, 1996
`2, 1997
`3, 1997
`3, 1997
`3, 1997
`3, 1997
`5, 1997
`7, 1997
`8, 1997
`8, 1997
`9, 1997
`10, 1997
`10, 1997
`11, 1997
`11, 1997
`12, 1997
`1, 1998
`2, 1998
`2, 1998
`3, 1998
`4, 1998
`4, 1998
`4, 1998
`5, 1998
`5, 1998
`6, 1998
`6, 1998
`6, 1998
`6, 1998
`6, 1998
`7, 1998
`8, 1998
`8, 1998
`8, 1998
`8, 1998
`8, 1998
`9, 1998
`9, 1998
`9, 1998
`10, 1998
`10, 1998
`10, 1998
`10, 1998
`11, 1998
`11, 1998
`11, 1998
`11, 1998
`2, 1999
`2, 1999
`2, 1999
`2, 1999
`3, 1999
`3, 1999
`3, 1999
`3, 1999
`3, 1999
`3, 1999
`6, 1999
`6, 1999
`6, 1999
`8, 1999
`8, 1999
`8, 1999
`9, 1999
`10, 1999
`10, 1999
`11, 1999
`11, 1999
`12, 1999
`12, 1999
`12, 1999
`1, 2000
`1, 2000
`1, 2000
`
`Dai et al.
`MeLampy et al.
`White et al.
`Gordon
`Focsaneanu et al.
`Fuller et al.
`Hertzfeld et al.
`McLaughlin
`Miner et al.
`Espeut et al.
`Bareis et al.
`Morganstein et al.
`Bobo, II
`Broedner et al.
`Lynch et al.
`Goldman et al.
`Tullis et al.
`Bartholomew et al.
`Vysotsky et al.
`Lagarde et al.
`Morganstein
`Irribarren
`Baratz et al.
`Pepe et al.
`Fuller et al.
`Rongley
`Shaffer et al.
`Staples et al.
`Shachar et al.
`Shachar
`Bayya et al.
`Broedner et al.
`Broedner et al.
`Walters
`Polcyn
`Krane
`Junqua et al.
`Cooper et al.
`Baron et al.
`Broedner et al.
`Sarukkai et al.
`Goldman et al.
`Garberg et al.
`Goldberg et al.
`Vysotsky et al.
`Woodbridge et al.
`Wattenbarger
`Dekelbaum et al.
`Krishnaswamy et al.
`Elliott et al.
`Wesinger, Jr. et al.
`Coden et al.
`Foster et al.
`Watts et al.
`Bateman et al.
`Wise et al.
`Dvorak
`Brown et al.
`Bentley et al.
`Uppaluru
`Dunn et al.
`Strauss et al.
`Bannister et al.
`Dold
`Rhie et al.
`Schlueter et al.
`Beauregard et al.
`Focsaneanu et al.
`Miloslavsky
`Krishnaswamy et al.
`Tatchell et al.
`Kelly
`Li et al.
`Acker et al.
`Cohen
`
`6,018,710
`6,021, 181
`6,021,190
`6,031,904
`6,038,305
`6,044,107
`6,047,053
`6,052,372
`6,067,516
`6,078,580
`6,081,518
`6,081,782
`6,091,808
`6,101,472
`6,104,803
`6,115,737
`6,115,742
`6,130,933
`6,131,095
`6,137,863
`6,157,705
`6,161,128
`6,178,399
`6,185,535
`6,188,683
`6, 195,357
`6,199,076
`6,201,814
`6,201,863
`6,208.638
`6,215,858
`6,230,132
`6,233,318
`6,243,373
`6,252.944
`6,269,336
`6,285,745
`6,327,572
`6,330,538
`6,349,132
`6,353,661
`6,366,575
`6,366,578
`6.424,945
`6,430.282
`6,434,529
`6,445,694
`6,446,076
`6,456,699
`6.459,910
`6,477,240
`6,477,420
`6,490,627
`6,501,966
`6,505,163
`6,529,948
`6,532.444
`6,539,359
`6,546,393
`6,560,604
`6,584,439
`6,593.944
`6,594,348
`6,594,692
`6,618,039
`6,618,726
`6,618,763
`6,633,846
`
`6,636,831
`6,654,814
`6,665,640
`
`6,687,341
`6,704,024
`6,718,015
`6,721,705
`6,732,142
`6,763,388
`
`1, 2000
`2, 2000
`2, 2000
`2, 2000
`3, 2000
`3, 2000
`4, 2000
`4, 2000
`5/2000
`6, 2000
`6, 2000
`6, 2000
`T/2000
`8, 2000
`8, 2000
`9, 2000
`9, 2000
`10, 2000
`10, 2000
`10, 2000
`12, 2000
`12, 2000
`1, 2001
`2, 2001
`2, 2001
`2, 2001
`3, 2001
`3, 2001
`3, 2001
`3, 2001
`4, 2001
`5/2001
`5/2001
`6, 2001
`6, 2001
`T/2001
`9, 2001
`12, 2001
`12, 2001
`2, 2002
`3, 2002
`4, 2002
`4, 2002
`T/2002
`8, 2002
`8, 2002
`9, 2002
`9, 2002
`9, 2002
`10, 2002
`11/2002
`11/2002
`12, 2002
`12, 2002
`1, 2003
`3, 2003
`3, 2003
`3, 2003
`4/2003
`5/2003
`6, 2003
`T/2003
`T/2003
`T/2003
`9, 2003
`9, 2003
`9, 2003
`* 10, 2003
`
`10, 2003
`11/2003
`* 12/2003
`
`2, 2004
`3, 2004
`4, 2004
`4, 2004
`5, 2004
`T/2004
`
`Wynblatt et al.
`Miner et al.
`Fuller et al.
`An et al.
`McAllister et al.
`Gatherer et al.
`Miner et al.
`Gittins et al.
`Levay et al.
`Mandalia et al.
`Bowman-Amuah
`Rabin
`Wood et al.
`Giangarra et al.
`Weser et al.
`Ely et al.
`Franklin et al.
`Miloslavsky
`Low et al.
`Brown et al.
`Perrone
`Smyk
`Takebayashi et al.
`Hedin et al.
`Lang et al.
`Polcyn
`Logan et al.
`Greenspan
`Miloslavsky
`Rieley et al.
`Bartholomew et al.
`Class et al.
`Picard et al.
`Turock
`Hansen, II et al.
`Ladd et al.
`Bartholomew et al.
`Morton et al.
`Breen
`Wesemann et al.
`Bailey, III
`Barkan et al.
`Johnson
`Sorsa
`Bannister et al.
`Walker et al.
`Swartz
`Burkey et al.
`Burg et al.
`Houston
`Lim et al.
`Struble et al.
`Kalra et al.
`Bareis et al.
`Zhang et al.
`Bowman-Amuah
`Weber
`Ladd et al.
`Khan
`Fascenda
`Geilhufe et al.
`Nicolas et al.
`Burstrom et al.
`Reisman
`Grant et al.
`Colbath et al.
`Steinberg
`Bennett ............... GO6F 17,3043
`704/257
`
`Profit, Jr. et al.
`Britton et al.
`Bennett ............... GO6F 17,3043
`704/257
`
`Koch et al.
`Robotham et al.
`Berstis
`Kurganov et al.
`Bates et al.
`Tsimelzon
`
`

`

`US 9.451,084 B2
`Page 4
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`8, 2004 Xiao et al.
`6,771,732 B2
`8/2004 Kurganov
`6,775,264 B1
`8, 2004 Swartz
`6,785,266 B2
`10/2004 Kurganov
`6,807.257 B1
`1 1/2004 Flores et al.
`6,812,939 B1
`1 1/2004 Kredo et al.
`6,823,370 B1
`5/2005 Saylor et al.
`6,888,929 B1
`7/2005 Kuiken et al.
`6,922,733 B1
`9/2005 Loghmani et al.
`6,941,273 B1
`11/2005 Zirngiblet al.
`6,964,012 B1
`6,964,023 B2 11/2005 Maes et al.
`6,965,864 B1
`11/2005 Thrift et al.
`6,996,609 B2
`2, 2006 Hickman et al.
`6,999,804 B2
`2/2006 Engstrom et al.
`7,003,463 B1
`2/2006 Maes et al.
`7,024,464 B1
`4/2006 Lusher et al.
`7,050,977 B1 * 5/2006 Bennett ............... GO6F 17 2775
`TO4/270.1
`
`7/2006 Flores et al.
`7,075,555 B1
`7/2006 Kurganov et al.
`7,076.431 B2
`8, 2006 Zintel et al.
`7,089,307 B2
`12/2006 Elliott
`7,145,898 B1
`2/2008 Kurganov ..................... 370,352
`7,327,723 B2 *
`6/2008 Kurganov et al
`704/270.1
`7,386,455 B2 *
`7,516, 190 B2 * 4/2009 Kurganov ..................... 709/217
`7.881,941 B2* 2/2011 Kurganov et al. ............ 704/275
`8,098,600 B2 *
`1/2012 Kurganov ..........
`370,260
`8,185.402 B2 *
`5/2012 Kurganov et al. ............ 704/275
`2001/0032234 A1
`10/2001 Summers et al.
`2001/0040885 A1
`1 1/2001 Jonas et al.
`2001/0048676 A1
`12/2001 Jimenez et al.
`2002, 0006126 A1
`1/2002 Johnson et al.
`5/2002 Belanger
`2002005.9402 A1
`2002fOO64149 A1
`5, 2002 Elliott et al.
`2002/0090114 A1
`7/2002 Rhoads et al.
`2004O160913 A1
`8, 2004 Kubler et al.
`2005/0O25133 A1
`2/2005 Swartz
`2/2005 Script et al.
`2005/OO3O179 A1
`2005, 00741.04 A1
`4/2005 Swartz
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`GB
`GB
`GB
`JP
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`O794.650 A2
`2211 698
`2240 693
`2317782
`1-2.58526
`WO 91,07838
`WO 91 18466
`WO 96,09710
`97344O1
`WO 97.37481
`97.37481
`WO 98.23058
`98.23058
`
`9, 1997
`7, 1989
`8, 1991
`4f1998
`10, 1989
`5, 1991
`11, 1991
`3, 1996
`9, 1997
`9, 1997
`10, 1997
`5, 1998
`9, 1998
`
`OTHER PUBLICATIONS
`
`Amended Complaint, Parus Holdings, Inc. v. Web Telephony LLC
`& Robert Swartz, Case No. 06-cv-01146 (N.D. III.), Jul. 10, 2006,
`14 pages.
`AT&T, Press Release, "AT&T Customers Can Teach Systems to
`Listen and Respond to Voice”. Jan. 17, 1995, pp. 1-2, Basking
`Ridge, NJ., available at www.lucent.com/press/0195/950117.gbb.
`html (accessed Mar. 15, 2005).
`Bellcore Technology Licensing, “The Electronic Receptionist—A
`Knowledge-Based Approach to Personal Communications”, 1994,
`pp. 1-8.
`Brachman et al., “Fragmentation in Store-and-Forward Message
`Transfer", IEEE Communications Magazine, vol. 26(7), Jul. 1998,
`pp. 18-27.
`“Business Phone Systems for Advanced Offices', NTT Review, vol.
`2 (6), Nov. 1990, pp. 52-54.
`
`Cole et al., “An Architecture for a Mobile OSI Mail Access
`System”, IEEE Journal on Selected Areas in Communications, vol.
`7 (2), Feb. 1989, pp. 249-256.
`“Data Communications Networks: Message Handling Systems”.
`Fasciele, VIII. 7-Recommendations X.400-X.430, 42 pages, date
`unknown.
`DAX Systems, Inc., Press Release, “Speech Recognition Success in
`DAX's Grasp, Nov. 22, 1995, pp. 1-2, Pine Brook, NJ.
`Defendants Answer to the Amended Complaint and Demand for
`Jury Trial, Parus Holdings, Inc. v. Web Telephone LLC & Robert
`Swartz, Case No. 06-cv-01146 (N.D. III.), Aug. 10, 2006, 14 pages.
`“Faxpak Store and Forward Facsimile Transmission Service'. Elec
`trical Communication, vol. 54 (3), 1979, pp. 251-255.
`Garcia et al., “Issue in Multimedia Computer-Based Message Sys
`tems Design and Standardization”, NATO ASI Series, vol. 1-6,
`1984, 10 pgs.
`“GLOBECOM 85 IEEE Global Telecommunications Conference,
`New Orleans, L.A., Dec. 2-5, 1985, pp. 1295-1300.
`Hemphill et al. “Speech-Aware Multimedia.” IEEE Multimedia,
`Spring 1996, vol. 3, No. 1. pp. 74-78, IEEE.
`Hunt et al., “Long-Distance Remote Control to the Rescue”. Chi
`cago Tribune, Jun. 15, 2002, Section 4, p. 15.
`“Introducing PIC SuperFax. First PC/Fax System to Run Under
`Windows'. Pacific Image Communications, Pasadena, CA. Date
`Unknown, (received at COMDEX show, Nov. 3, 1987). 3 pgs.
`“Juggler by Pure Speech.” p. 1, available at http://members.aol.
`com/compaanda juggler.html (accessed on Dec. 8. 2006).
`Kubala et al., BYBLOS Speech Recognition Benchmark Results:,
`Workshop on Speech & Natural Language, Feb. 19-22, 1991.
`Ly, "Chatter: A Conversational Telephone Agent'. Submitted to
`Program in Media Arts, & Sciences, MIT, 1993, pp. 1-130.
`Maeda, et al., “An Intelligent Customer-Controlled Switching Sys
`tem”, IEEE Global Telecommunications Conference, Hollywood,
`Florida, Nov. 28-Dec. 1, 1988, pp. 1499-1503.
`Markowitz, J., “The Ultimate Computer Input Device May Be Right
`Under Your Nose'. Byte, Dec. 1995, pp. 1-13, available at www.
`byte.com/art/9512/sec8/art1.htm (accessed Mar. 15, 2005).
`Marx et al., “Mail Call: Message Presentation and Navigation in a
`Nonvisual Environment,' SIGCHI Conference on Human Factors
`in Computing Systems, Vancouver, B.C. Canada, Apr. 13-18, 1996.
`Marx. M.. “Toward Effective Conversational Messaging” (Thesis).
`As indicated on the cover page, the thesis was presented to the
`Department Committee on Graduate Students, Program in Media
`Arts and Sciences, School of Architecture and Planning, Massachu
`setts Institute of Technology on May 12, 1995.
`Oye, Phil., “Juggler', p. 1, available at http://www.philoye.com/
`work/juggler/index.shtml (accessed on Dec. 8, 2006).
`Oye, Phil., “Juggler', p. 1, available at http://www.philoye.com/
`work juggler 2.shtml (accessed on Dec. 8, 2006).
`Oye, Phil., “Juggler', p. 1, available at http://www.philoye.com/
`work/juggler 3.shtml (accessed on Dec. 8, 2006).
`Perdue et al., “Conversant(R) 1 Voice System: Architecture and
`Applications', Jul. 17, 1986, AT&T Technical Journal, pp. 1-14.
`Plaintiff Parus Holdings, Inc.'s Supplemental Response to Defen
`dant Web Telephony LLC's First Set of Interrogatories (Nos. 1-12),
`Parus Holdings, Inc. v. Web Telephony LLC & Robert Swartz, Case
`No. 06-cv-01146 (N.D. III.), Oct. 31, 2006, 32 pages.
`Plaintiff Parus Holdings, Inc.'s Supplemental Response to Defen
`dant Web Telephony LLC's Second Set of Interrogatories (Nos.
`13-17), Parus Holdings, Inc. v. Web Telephony LLC & Robert
`Swartz, Case No. 06-cv-01146 (N.D. III.), Oct. 31, 2006, 31 pages.
`Print outs of Internet web site, "Wildfire Communications, Inv.,',
`Nov. 5, 1997, including print outs of the following web pages:
`http://www.wildfire.com: http://www.wildfire.com/consumerhome.
`html; http://www.wildfire.com/106.html; http://www.wildfire.com/
`carrierhome.html; http://www.wildfire.com/sfandb.html; http://
`www.wildfire.com/about.html; http://www.wildfire.com/abtmgmt.
`html; http://www.wildfire.com/scoop.html; http://www.wildfire.
`com/intel.html; and http://www.wildfire.com/msft.html.
`“Proceedings of the IFIP World Computer Congress', Dublin,
`Ireland, Sep. 1-5, 1986.
`
`

`

`US 9.451,084 B2
`Page 5
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`“PureSpeech Announces Juggler PC System for First Quarter of
`1997. HighBeam Research, Sep. 19, 1996, pp. 1-3, available at
`http://www.highbeam.com/doc/1G1-186909545.html (accessed on
`Dec. 8, 2006).
`PureSpeech, “Meet the Voice of Juggler!”, the date of Nov. 18, 1996
`is shown on the top of p. 1.
`Ross, Randy, “Retrieve E-Mail from a Telephone”, Oct. 7, 1996, pp.
`1-2, available at http://resna.org/ProfessOrg?Sigs?SIGSites/sig11/
`archive/juggler.htm (accessed on Dec. 8, 2006). Printout indicates
`that the article was originally printed in PC World.
`Sartori, M., “Speech Recognition”, Apr. 1995, pp. 1-9, Mercury
`Communications, available at www.gar.co.uk/technology watch/
`speech.htm (accessed Mar. 15, 2005).
`Schmandt et al., “A Conversational Telephone Messaging Systems',
`IEEE Transactions on Consumer Electronics, 1984, vol. CE-30, No.
`3, pp. XXi-XXiv.
`Schmandt, et al., “Phone Shell: The Telephone as Computer Ter
`minal', ACM Multimedia, 1993, 11 pages.
`Schmandt, et al., “Phone Slave: A Graphical Telecommunications
`Interface”. Proceedings of the SID, 1985, vol. 26/1, pp. 79-82.
`"Secretarial Branch Exchange'. IBM Technical Disclosure Bulletin,
`vol. 26 (5), Oct. 1983, pp. 2645-2647.
`Shimamura, et al., “Review of the Electrical Communication Labo
`ratories', vol. 418, (33), No. 1, Tokyo, Japan, 1985, pp. 31-39.
`“The VMX Systems Product Reference Manual: Product Descrip
`tion Volume”, May 1994, vol. 1, release 7.1, VMX, Inc. (Octel
`Communications Corp.) San Jose, CA, USA.
`“VMXworks Product Reference Manual: vol. 3 Programmer's
`Guide', Jul. 1994, vols. 3 & 4, Release 3.1, Octel Communications
`Corp.) Milpitas, CA, USA.
`“Wildfire Communication, Inc.'', Harvard Business School, Mar.
`21, 1996, Publ. No. 9-396-305, pp. 1-22.
`“WordPerfect: New Telephony Features Boost Office”, WordPerfect
`Office TechBrief, 1994, Info-World Publishing Co., vol. 10, Issue 2,
`pp. 2-3.
`Yang, E., “INETPhone Telephone Services and Servers on the
`Internet”, Apr. 1995, University of North Texas, pp. 1-6.
`Hemphill et al., “Surfing the Web by Voice.” ACM Multimedia
`95—Electronic Proceedings, Nov. 5-9, 1995. San Francisco, CA.
`Ly, "Chatter A: A Conversational Telephone Agent.” Submitted to
`Program in Media Arts and Sciences, Massachusetts Institute of
`Technology, (1993), pp. 1-130.
`Schmandt, “Phoneshell: the Telephone as Computer Terminal.”
`Proceedings of ACM Multimedia '93 Conference (1993), pp. 373
`382.
`
`Schmandt et al., “Phone Slave: A Graphical Telecommunications
`Interface.” Proceedings of the SID (1985), vol. 26/1, pp. 79-82.
`Examples: Abstract Ideas, 2014, 20 pages.
`IBM AIX DirectTalk/6000 Version 1 Release 6 Improves Your
`Voice Processing Services to Callers and Customers, Announce
`ment No. 295-489, Nov. 28, 1995, 27 pages.
`IBM Announcement Letter No. A95-893, retrieved on Mar. 9, 2015,
`10 pages.
`IBM, AIX DirectTalk/6000 Release 6: Speech Recognition with the
`BBN Hark Recognizer, SC33-1734-00, Feb. 1996, 250 pages.
`IBM, AIX DirectTalk/6000: General Information and Planning,
`Release 6, GC33-1720-00, Dec. 1995, 162 pages.
`IBM, DirectTalkMail: Administration, Release 6, SC33-1733-00,
`Feb. 1996, 274 pages.
`“McGraw-Hill Dictionary of Scientific & Technical Terms 1101, 6th
`ed. 2003.” (Referenced in Amendment and Reply Under 37 CFR
`1.111 filed for U.S. Appl. No. 10/877,366 on Apr. 24, 2009) No copy
`is provided but please inform if a copy of this dictionary is required.
`Newton, Harry, Newtons Telecom Dictionary The Official Glos
`sary of Telecommunications and Voice Processing Terms, Dec.
`1992, 6 pages.
`Paper No. 10, Denying Institution of Covered Business Method
`Patent Review CBM2015-00109 and CBM2015-00149, Nov. 9,
`2015, 19 pages.
`Paper No. 10, Denying Institution of Covered Business Method
`Patent Review CBM2015-00110 and CBM2015-00150, Nov. 9,
`2015, 20 pages.
`Paper No. 10, Denying Institution of Covered Business Method
`Patent Review CBM2015-001 11 and CBM2015-00151, Nov. 9,
`2015, 19 pages.
`Paper No. 10, Denying Institution of Covered Business Method
`Patent Review CBM2015-001 12 and CBM2015-00152, Nov. 9,
`2015, 18 pages.
`Putz, Steve, Interactive Information Services. Using World-Wide
`Web Hypertext, First Int’l Conference on World-Wide Web (May
`25-27, 1994), 10 pages.
`Memorandum Opinion and Order, Oct. 8, 2015, 27 pages.
`Update Subject Matter Eligibility, Jul. 2015, 33 pages.
`Wikipedia Definition of “Internet', available at http://en.wikipedia.
`org/wiki/Internet Retrieved on Feb. 10, 2016, pp. 24-26.
`Opening Brief of Appellant Parus Holdings, Inc., Submitted on Mar.
`8, 2016, to the United States Court of Appeals for the Federal
`Circuit, 236 pages.
`Brief of Appellees, submitted on Jun. 20, 2016, to the United States
`Court of Appeals for the Federal Circuit, 53 pages.
`* cited by examiner
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep. 20, 2016
`
`Sheet 1 of 4
`
`US 9.451,084 B2
`US 9,451,084 B2
`
`~_
`
`_.___w
`
`_
`
`mmmc
`
`mm0.
`
`~we
`
`_me
`
`0252/95mm?
`
`mm>mmw
`
`E
`
`0000
`
`NmEmmm>>vammm;4Nv:
`
`
`
`
`
`
`
`Oz~w§0mmmm>>
`
`mw>mmw
`
`.8
`
`wHs82'_mméfié_-__H__M_
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 20, 2016
`
`Sheet 2 of 4
`
`US 9.451,084 B2
`
`200
`
`
`
`
`
`RANK
`NUMBER
`
`EXTRACTION AGENT
`COMMAND
`
`TIMESTAMP
`
`202
`
`204
`
`206
`
`208
`
`FIG.
`2
`
`300
`
`302
`
`304
`
`306
`
`308
`
`
`
`
`
`
`
`
`
`
`
`
`
`SPEECH RECOGNITION
`ENGINE
`
`SPEECH SYNTHESIS
`ENGINE
`
`VRAPPLICATION
`
`CALL PROCESSING SYSTEM
`
`TELEPHONY AND VOICE
`HARDWARE
`
`
`
`MEDIASERVER
`
`FIG. 3
`
`

`

`U.S. Patent
`
`Sep. 20, 2016
`
`Sheet 3 of 4
`
`US 9.451,084 B2
`
`
`
`102
`
`CONTENTEXTRACTION
`AGENT
`
`CONTENT FETCHER
`
`400
`
`402
`
`POLLING AND RANKINGAGENT
`
`4.04
`
`CONTENT DESCRIPTOR FILES
`
`A06
`
`WEB BROWSNG SERVER
`
`FIG. 4
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep. 20, 2016
`Sep. 20, 2016
`
`Sheet 4 of 4
`Sheet 4 of 4
`
`US 9.451,084 B2
`US 9,451,084 B2
`
`_
`
`gm@
`
`mg8_can9
`sows.__mméfié~_-__52mmmmsmwm_wanozagomm 1llllllllllllllllllI»lIiiJ”Sm2m.8m_mm>mmw”mama__«am:
`
`
`
`3m
`
`
`
`ozaaomm#w83mg85%gm.
`
`
`
`Now
`
`mOOQSO
`
`$22.10:
`
`
`
`wzFIw:5.55
`
`
`
`10002.EEDOmm
`
`4wO8
`
`
`
`

`

`1.
`ROBUST VOICE BROWSER SYSTEMAND
`VOICE ACTIVATED DEVICE CONTROLLER
`
`US 9,451,084 B2
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of application Ser. No.
`12/973,475, entitled “Robust Voice Browser System And
`Voice Activated Device Controller, filed Dec. 20, 2010,
`which is a continuation of application Ser. No. 12/030,556,
`entitled "Robust Voice Browser System And Voice Activated
`Device Controller, filed Feb. 13, 2008, now U.S. Pat. No.
`7.881,941, which is a continuation application of application
`Ser. No. 11/409,703, entitled “Robust Voice Browser System
`And Voice Activated Device Controller.” filed Apr. 24, 2006,
`now U.S. Pat. No. 7,386,455, which is a continuation
`application of application Ser. No. 10/821,690, entitled
`“Robust Voice Browser System And Voice Activated Device
`Controller.” filed Apr. 9, 2004, now U.S. Pat. No. 7,076.431,
`which is a continuation application of application Ser. No.
`09/776,996, entitled “Robust Voice Browser System And
`Voice Activated Device Controller, filed Feb. 5, 2001, now
`U.S. Pat. No. 6,721,705, which claims priority to U.S.
`Provisional Application Ser. No. 60/180,344, entitled “Voice
`Activated Information Retrieval System,” filed Feb. 4, 2000
`and U.S. Provisional Application No. 60/233,068, filed Sep.
`15, 2000, entitled “Robust Voice Browser System and Voice
`Activated Device Controller, all assigned to the assignee of
`the present application. The subject matter in the above
`identified co-pending and commonly owned applications is
`incorporated herein by reference.
`
`10
`
`15
`
`25
`
`30
`
`FIELD OF THE INVENTION
`
`The present invention relates to a robust and highly
`reliable system that allows users to browse web sites and
`retrieve information by using conversational Voice com
`mands. Additionally, the present invention allows users to
`control and monitor other systems and devices that are
`connected the Internet or any other network by using voice
`commands.
`
`35
`
`40
`
`BACKGROUND OF THE INVENTION
`
`Currently, three options exist for a user who wishes to
`gather information from a web site accessible over the
`Internet. The first option is to use a desktop or a laptop
`computer connected to a telephone line via a modem or
`connected to a network with Internet access. The second
`option is to use a Personal Digital Assistant (PDA) that has
`the capability of connecting to the Internet either through a
`modem or a wireless connection. The third option is to use
`one of the newly designed web-phones or web-pagers that
`are now being offered on the market. Although each of these
`options provide a user with access to the Internet to browse
`web sites, each of them have their own drawbacks.
`Desktop computers are very large and bulky and are
`difficult to transport. Laptop computers solve this inconve
`nience, but many are still quite heavy and are inconvenient
`to carry. Further, laptop computers cannot be carried and
`used everywhere a user travels. For instance, if a user wishes
`to obtain information from a remote location where no
`electricity or communication lines are installed, it would be
`nearly impossible to use a laptop computer. Oftentimes,
`information is needed on an immediate basis where a
`computer is not accessible. Furthermore, the use of laptop or
`desktop computers to access the Internet requires either a
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`direct or a dial-up connection tan an Internet Service Pro
`vider (ISP). Oftentimes, such connections are not available
`when a user desires to connect to the Internet to acquire
`information.
`The second option for remotely accessing web sites is the
`use of PDAs. These devices also have their own set of
`drawbacks. First, PDAs with the ability to connect to the
`Internet and access web sites are not readily available. As a
`result, these PDAs tend to be very expensive. Furthermore,
`users are usually required to pay a special service fee to
`enable the web browsing feature of the PDA. A further
`disadvantage of these PDAS is that web sites must be
`specifically designed to allow these devices to access infor
`mation on the web site. Therefore, a limited number of web
`sites are available that are accessible by these web-enabled
`PDAs. Finally, it is very common today for users to carry
`cell phones, however, users must also carry a separate PDA
`if they require the ability to gather information from various
`web sites. Users are therefore subjected to added expenses
`since they must pay for both cellular telephone service and
`also for the web-enabling service for the PDA. This results
`in a very expensive alternative for the consumer.
`The third alternative mentioned above is the use of
`web-phones or web-pagers. These devices suffer many of
`the same drawbacks as PDAs. First, these devices are
`expensive to purchase. Further, the number of web sites
`accessible to these devices is limited since web sites must be
`specifically designed to allow access by these devices.
`Furthermore, users are often required to pay an additional
`fee in order to gain wireless web access. Again, this service
`is expensive. Another drawback of these web-phones or
`web-pagers is that as technology develops, the methods used
`by the various web sites to allow access by these devices
`may change. These changes may require users to purchase
`new web-phones or web-pagers or have the current device
`serviced in order to upgrade the firmware or operating
`system stored within the device. At the least, this would be
`inconvenient to users and may actually be quite expensive.
`Therefore, a need exists for a system that allows users to
`easily access and browse the Internet from any location.
`Such a system would only require users to have access to
`any type of telephone and would not require users to
`subscribe to multiple services.
`In the rapidly changing area of Internet applications, web
`sites change frequently. The design of the web site may
`change, the information required by the web site in order to
`perform searches may change, and the method of reporting
`search results may change. Web browsing applications that
`Submit search requests and interpret responses from these
`web sites based upon expected formats will produce errors
`and useless responses when Such changes occur. Therefore,
`a need exists for a system that can detect modifications to
`web sites and adapt to Such changes in order to quickly and
`accurately provide the information requested by a user
`through a voice enabled device. Such as a telephone.
`When users access web sites using devices such as
`personal computers, delays in receiving responses are tol
`erated and are even expected, however, such delays are not
`expected when a user