UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`GOOGLE LLC, SAMSUNG ELECTRONICS CO., LTD., SAMSUNG
`ELECTRONICS AMERICA, INC., LG ELECTRONICS INC., and LG
`ELECTRONICS U.S.A., INC.,
`Petitioners,
`
`v.
`
`PARUS HOLDINGS, INC.,
`Patent Owner.
`
`Case No. IPR2020-00846, IPR2020-00847
`U.S. Patent No. 7,076,431, U.S. Patent No. 9,451,084
`
`DECLARATION OF ALEXANDER KURGANOV IN SUPPORT OF
`PATENT OWNER’S RESPONSE TO INTER PARTES REVIEW OF U.S.
`PATENT NO. 7,076,431 and 9,451,084
`
`Parus Exhibit 2020
`Google, et al. v. Parus Holdings, Inc.
`IPR2020-00846
`Page 1 of 69
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`GOOGLE LLC, SAMSUNG ELECTRONICS CO., LTD., SAMSUNG
`ELECTRONICS AMERICA, INC., LG ELECTRONICS INC., and LG
`ELECTRONICS U.S.A., INC.,
`Petitioners,
`
`v.
`
`PARUS HOLDINGS, INC.,
`Patent Owner.
`
`Case No. IPR2020-00846, IPR2020-00847
`U.S. Patent No. 7,076,431, U.S. Patent No. 9,451,084
`
`DECLARATION OF ALEXANDER KURGANOV IN SUPPORT OF
`PATENT OWNER’S RESPONSE TO INTER PARTES REVIEW OF U.S.
`PATENT NO. 7,076,431 and 9,451,084
`
`Parus Exhibit 2020
`Google, et al. v. Parus Holdings, Inc.
`IPR2020-00846
`Page 1 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`TABLE OF CONTENTS
`
`PAGE
`BACKGROUND ............................................................................................. 1
`CONCEPTION ................................................................................................ 7
`
`I.
`II.
`
`i
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`corresponding to a speech command; said speech
`command comprising an information request selectable
`by the user; ................................................................................ 24
`a.
`Evidence of the early use of the voice browsing
`weather grammar on or before 12/31/1999 .......... 27
`
`ii
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`iii
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`iv
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`the plurality of web sites, the computing device
`configured to access a first web site of the plurality of
`web sites and, if the information to be retrieved is not
`found at the first web site, the computer configured to
`access the plurality of web sites remaining in an order
`defined for accessing the listing of web sites until the
`information to be retrieved is found in at least one of the
`plurality of web sites or until the plurality of web sites
`have been accessed; .................................................................. 50
`
`v
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`associated with each web site address, the pre-defined
`order of access indicating the order in which the plurality
`of web sites are accessed. .......................................................... 51
`
`vi
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`I, Alexander Kurganov, being of legal age, hereby declare, affirm, and state
`
`the following:
`
`I.
`
`BACKGROUND
`1. I am a highly skilled professional software engineer and a technical leader
`
`with 40+ years of continuous work experience in the IT industry. I graduated from
`
`a high school with a special concentration in math and physics. Continuing my
`
`education, I received MS in Electrical Engineering (with concentration in Control
`
`Systems) from the Belorussian Technical University, Minsk, former USSR. I also
`
`completed most of the Ph.D. program in Computer Science at the Institute of
`
`Technical Cybernetics, Minsk, former USSR, before I and my family left the USSR
`
`in 1989 as refugees (without citizenship) headed to the US. We were still coming
`
`out from behind of the Iron Curtain before it fell later that year.
`
`2. Starting
`
`in 1990, I worked as a software engineer in
`
`the US
`
`telecommunications industry for such companies as Dytel (the inventor of Auto-
`
`Attendant), Interactive Unix (a Kodak company, developer of the original network
`
`file system technology, NFS), Motorola, Ardis (IBM and Motorola wireless joint
`
`venture). In 1994 I became an employee #2 of a startup Vail Systems where, working
`
`as VP Engineering, I was the sole designer and developer of their network-based
`
`integrated voice and web platform that had been deployed in production and
`
`successfully hosted high volume voice and web applications for major companies,
`
`1
`
`Parus Exhibit 2020, Page 8 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`such as Allstate, US West, and others. Before 1996, while still working for Vail
`
`Systems, I conceived and started the design and development of the innovative
`
`computer, internet, and telecommunications system integrating voice and web with
`
`a natural speech interface. In 1996, I conceived the Webley Virtual Assistant
`
`application to run on this system. Vail Systems spun off Webley Systems in 1997
`
`and transferred the above-mentioned system technology, as well as Webley
`
`Assistant technology to Webley Systems.
`
`3. From 1997 to 2010, I worked at Webley Systems (“Webley”) as a Chief
`
`Technology Officer and a Chief Scientist acting as a lead software architect and
`
`software engineer. Webley is now Parus Holdings, Inc. From 2010 till the present
`
`time, I have been working for Webley as a consultant in the Chief Scientist role. I
`
`am a co-founder of Webley that in 1997 was a startup on a mission to develop the
`
`innovative, computer, internet, and telecommunications based network system for
`
`delivering unified messaging, real-time communications and information services
`
`with speaker independent, natural speech recognition and web user interfaces. The
`
`Webley computer, internet, and telecommunications based network system and the
`
`first version of the Webley Assistant (“WA I”) was launched in 1997. It allowed
`
`cross platform communication flow with real time communication from all types of
`
`devices and networks. Users could manage phone calls, access voicemails and
`
`emails from anywhere using Webley’s website or by phone using speech commands.
`
`2
`
`Parus Exhibit 2020, Page 9 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`4. To further improve the WA I, we were always brainstorming ways that we
`
`could extend its functionality to make it more robust. In October of 1998, in an
`
`email to my colleagues, I proposed creating a system for voice web browsing from
`
`any voice-enabled mobile device. Ex. 2021. I believed that we could expand the
`
`functionalities of WA I. In my initial email, I suggested building a prototype for
`
`voice web browsing. Id. To allow users an intuitive way to access worldwide web-
`
`based information over any voice-enabled mobile device, I proposed building a
`
`natural speech browsing and searching command interface with text-to-speech
`
`playback of the web information of interest to users. Id. The initial prototype
`
`implementation I suggested involved a www access API running on the FreeBSD
`
`computer and delivering filtered content to the voice servers using the available
`
`service via User Datagram Protocol (UDP). Id.
`
`5. In early 1999, I memorialized a list of features and products that I wanted
`
`Webley to focus on. Ex. 2022. The list included the phone browsing enhancements
`
`that I suggested in late 1998. Specifically, I wanted to incorporate “natural sounding
`
`text-to-speech with barge-in capability”, “[s]tock, weather, traffic, travel, [and] news
`
`information services under Webley with [a] natural speech interface.” and “Major
`
`portal access. Natural speech interface to portals like: Yahoo, Netscape, AOL, etc.”
`
`Id.
`
`3
`
`Parus Exhibit 2020, Page 10 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`6. I involved Valery Zukoff and together we worked on the voice web
`
`browsing project. The first version of the Webley Assistant, WA I, already included
`
`speech recognition and text-to-speech capabilities. For this expanded project, we
`
`needed to capture from the web and deliver over the telephone in the audio form the
`
`information desired by the users. Valery worked on the web browsing server that
`
`would extract and fetch the desired information content (according to a given content
`
`descriptor that we would define), such as stocks or weather, from the web, and I
`
`worked on all other aspects of the web browsing project including: natural speech
`
`grammar development, speech recognition and
`
`text-to-speech functionality
`
`implementation including handling of user speech commands and conversion of the
`
`web captured text information into a proper audio form for telephone-based delivery
`
`to the users, extension and integration of the Webley Assistant main C-code module
`
`with the Web Browsing Server and other functions.
`
`7. Similar attempts by others to browse the web by phone suffered from
`
`several drawbacks. For example, unlike browsing the web by a computer where a
`
`user will tolerate and even expect delays, users expected immediate responses when
`
`using a telephone. This meant that any telephone-based web browsing system would
`
`need to provide rapid responses. In order to provide these rapid responses these
`
`early systems could only typically map a single web site to a single resource. So,
`
`the “news” resource might be mapped to “www.somenews.com.” A user would utter
`
`4
`
`Parus Exhibit 2020, Page 11 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`the term “news” through the phone and the system would map that to the
`
`somenews.com website and retrieve the news. Depending on the availability of the
`
`website this simple one-to-one mapping may or may not provide fast voice responses
`
`(or even may not provide any responses at all for some time if the site is down or
`
`taken out for maintentance) required by users using phones, and it severely limited
`
`the system in other ways.
`
`8. For example,
`
`these systems were not
`
`fault
`
`tolerant.
`
`
`
`If
`
`the
`
`www.somenews.com web site was down when the user requested the news, there
`
`would be no way to get the news. Another drawback of those other implementations
`
`was that such websites, like “www.somenews.com” in the above example, would
`
`have to be adapted by the web site developers for voice browsing in special ways.
`
`Further, the user was limited to the web site that the resource was mapped to.
`
`Perhaps the user wanted the news from betternews.com instead of news.com. The
`
`user would have no way of specifying this preference. Finally, by mapping a single
`
`web site to the resource, the system was limited to generic information, and was not
`
`able to search multiple web sites to find specific information.
`
`9. The voice web browsing project that Valery and I worked on solved all
`
`these drawbacks. For example, our voice web browsing system used an instruction
`
`set for retrieving the requested information from any of a plurality of websites, and
`
`our implementation didn’t require any adaptation of the web sites by web site
`
`5
`
`Parus Exhibit 2020, Page 12 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`developers for voice browsing. Providing and using multiple potential websites for
`
`delivering the same requested information allows our system to adjust for
`
`unavailable web sites and provide more responsive and accurate information to more
`
`specific requests, thereby solving problems that other attempts were grappling with
`
`and had not addressed, such as lack of fault tolerance, limited and requiring
`
`adaptation web site resources, and generic search options and results.
`
`10.
`
`I am one of the named inventors of U.S. Patent No. 7,076,431 (“the
`
`’431 patent”) and U.S. Patent No. 9,451,084 (“the ’084 patent”). The other named
`
`inventor is Valery Zhukoff. I conceived of voice access to information on the
`
`internet as part of the computer, internet and telecommunications based system
`
`including the first version of the Webley assistant, WA I, before March 3, 1997,
`
`when I was employed with Vail Systems, as evidenced in U.S. Provisional
`
`Application – filed on March 3, 1997, with which I provided the complete source
`
`code of some 250 pages. After my move to Webley, to further develop features for
`
`the next, expanded version of the Webley Assistant, (“WA II”), Valery Zhukoff and
`
`I collectively conceived of the robust voice browser system claimed in the ’431 and
`
`’084 patents on or before October 13, 1998 Ex. 2021. A team of my colleagues and
`
`I worked on the voice web browsing project for WA II nearly every business day
`
`from July 12, 1999 or earlier to December 14, 2000 as can be confirmed by
`
`reviewing the relevant software file commits into the Webley CVS source control
`
`6
`
`Parus Exhibit 2020, Page 13 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`system. No later than December 31, 1999, we had a working speech activated web
`
`browsing system (as part of the Webley Assistant) that included all the features
`
`claimed in the ’431 and ’084 patents.
`
`11. Our system was able to overcome the problems of delays in responses
`
`from accessing not one but multiple web sites within acceptable thresholds allotted
`
`for real-time voice communications, by incorporating a novel algorithm to find the
`
`information fast. The instruction set in our system included information to access a
`
`database of pre-selected web sites that were continuously ranked both for speed and
`
`the usefulness of the information stored thereon, so that the web sites may be
`
`sequentially accessed in their ranked order until the requested information is
`
`retrieved. This ranked order, sequential access of pre-selected web sites allowed our
`
`system to provide the speed and audio response required by a voice enabled system,
`
`while still utilizing the plurality of web sites that improve upon prior phone-based
`
`web browsing systems.
`
`II.
`
`CONCEPTION
`12. No later than at least October 13, 1998, Valery Zhukov and I
`
`collectively conceived of the novel robust voice web browsing system disclosed in
`
`the ’431 patent. Since that time, we each contributed into different parts of this
`
`system. Valery worked on the web browsing server that included software agents
`
`that would extract and fetch textual content (according to content descriptors we
`
`7
`
`Parus Exhibit 2020, Page 14 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`developed) from specific web sites for weather, stocks, news etc., and I worked on
`
`all other aspects of the web browsing project including: natural speech grammar
`
`development, speech recognition and text-to-speech functionality implementation
`
`including handling of user speech commands and conversion of the web captured
`
`text information into a proper audio form for telephone-based delivery to the users,
`
`extension and integration of the Webley Assistant main C-code module with the
`
`Web Browsing Server and other functions.
`
`13.
`
`July 12, 1999 is the latest documented date when the novel feature of
`
`accessing a plurality of pre-selected web sites until an answer is found or there are
`
`no more web sites left to access was conceived. This can be seen in www.pl PERL
`
`script, which is dated 7/12/1999, in the subroutine “weather_zip” that accesses five
`
`pre-selected web sites in order to attempt to get the weather. Ex. 2023. The
`
`subroutine continues to access web sites to get the weather until it is found. Id.
`
`8
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`15. By 1999, when Valery and I began our voice web browsing
`
`enhancements for WA II, the WA I was already a mature software product. It was
`
`a fully-featured virtual assistant that allowed users to manage their calls, conference
`
`calls, voice messages, faxes, emails contact lists and more by phone using natural
`
`speech commands or from Webley’s web site. The WA I licensed the Automatic
`
`Speech Recognition (“ASR”) Engine software from Nuance. The ASR software
`
`allowed WA I to understand naturally spoken commands. I created grammar files
`
`that would allow WA I to recognize natural speech commands.
`
`16.
`
`For example, the menuNew.grammar file defined the main menu
`
`options for the WA I. The menuNew.grammar file, which was dated October 21,
`
`1999,
`
`included
`
`the main menu for
`
`this Webley Assistant.
`
` Ex. 2026
`
`menuNew.grammar. The following excerpt from the menuNew.grammar shows
`
`some of the functionality of the WA I:
`
`9
`
`Parus Exhibit 2020, Page 16 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2026.
`
`17.
`
`From the excerpt above, it can be seen that a user could listen to
`
`messages, check faxes, send pages, and a host of other options. The grammar file
`
`was compiled and loaded into the ASR software at runtime. A user would call into
`
`this Webley Assistant and the Assistant would ask the user what it could help with.
`
`The user would utter a command such as “Check my email” and the ASR Core
`
`Engine software would recognize the command based on the grammar and pass the
`
`corresponding code (<mcall 80_P> in this case as seen in the grammar excerpt
`
`above) into this Webley Assistant Application which would trigger the appropriate
`
`function from the mc_vm.c file and carry out the request. Once this Webley
`
`10
`
`Parus Exhibit 2020, Page 17 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`Assistant has carried out the request, the corresponding audio response will be
`
`generated for the user. Such audio response may include a concatenation of
`
`prerecorded and dynamically generated audio files. This Webley Assistant would
`
`use speech synthesis software or text-to-speech (“TTS”) software to convert the
`
`textual part of the response that is not covered by the pre-recorded audio into the
`
`audio form dynamically and would deliver it back to the user in real-time. We
`
`licensed Lernout & Hauspies’ RealSpeak TTS software to support this functionality
`
`in this Webley Assistant.
`
`11
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`the information in a manner that was fast, efficient, and did not incur needless delays
`
`that would not be tolerated over a telephone.
`
`19. After designing and developing these novel enhancements which
`
`became integrated into the WA II, Valery and I worked to implement it. This work
`
`will become obvious by reviewing at least the following two types of documents:
`
`(1) My colleagues and I continuously developed, tested, and debugged the source
`
`code and other components that supported and interacted with the system and this
`
`can be seen by looking at the creation and modification dates on the source code
`
`files; and (2) emails between me, my colleagues, and others show the progress and
`
`choices we made to develop the system. I will discuss the proposed enhancements,
`
`demonstrate that they were reduced to practice in the source code for the WA II, and
`
`demonstrate that we continuously worked on these enhancements until the complete
`
`weather functionality was available in the WA II.
`
`12
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`13
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2025 mc_vm.c at lines 13,607-13,641.
`
`23.
`
`The webget.pl PERL script was last modified on December 13, 1999
`
`by Valery Zhukoff.
`
`Ex. 2032 webget.pl at lines 1-10.
`
`14
`
`Parus Exhibit 2020, Page 21 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`24. At lines 24-31 of webget.pl, the service is determined by the parameter
`
`supplied to the webget.pl, in this case weather. The webget.pl script then reads in
`
`the weather.ini file which was a content descriptor file:
`
`Ex. 2032 weget.pl at lines 24-31.
`
`25.
`
`The weather.ini file included web sites URLs from which the Webley
`
`Assistant retrieved weather information, such as cnn, lycos, weather, snap, and
`
`infospace:
`
`Ex. 2033 weather.ini at lines 1-4, 63-68, 121-124, 183-186, and 241-245.
`
`15
`
`Parus Exhibit 2020, Page 22 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`26. A user would utter speech commands into the telephone, which is a
`
`voice enabled device. The ASR system, licensed from Nuance, would use the
`
`Webley Assistant grammar to recognize the commands, pass the corresponding code
`
`to the Webley Assistant application which would trigger the appropriate functions
`
`that are coded in mc_vm.c. The ASR system from Nuance was version 6.11 as
`
`evidenced from the following code snippet from mc_vm.c:
`
`Ex. 2025 mc_vm.c at lines 68-71.
`
`27. After uttering the commands, such as “weather,” into the phone, the
`
`Webley Assistant would retrieve the information from a pre-selected web site and
`
`then convert the textual information into audio by using the Lernout & Hauspie TTS
`
`software. Evidence demonstrating that Lernout & Hauspie TTS was being used can
`
`be found in the mc_vm.c file which includes the “app_lnhtts.h” which was a Lernout
`
`& Hauspie file that included internal TTS definitions:
`
`Ex. 2025 mc_vm.c at lines 43-50.
`
`16
`
`Parus Exhibit 2020, Page 23 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`The Webley Assistant would convert the retrieved textual information
`
`28.
`
`into audio form and would provide the information to the user via the telephone.
`
`17
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`Further, the user used the telephone to send speech commands to the
`
`32.
`
`Webley Assistant. For example, as described on its web page from 1997, “[y]ou
`
`speak to Webley [,and] Webley speaks to you – then carries out your commands.”
`
`Id.
`
`18
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2036 mc_vr.c at lines 1-18.
`
`34.
`
`The mc_vr.c includes the code that works with the Nuance ASR:
`
`Ex. 2036 at mc_vr.c at lines 69-85.
`35. Nuance 6 was a speaker-independent speech recognition device. Ex.
`
`2035. A user would speak commands into the telephone, which would be connected
`
`to said computer via a voice telephone call. The Nuance 6 ASR software engine,
`
`being also connected to said computer, would receive and recognize the voice
`
`commands, pass the appropriate code to the Webley Assistant application which
`
`would then fulfill the commands.
`
`19
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`1999. Ex. 2037. The RealSpeak TTS allowed the Webley Assistant to read emails
`
`and other text information aloud in a natural sounding voice Id.
`
`37. Review of the source code shows the Webley Assistant used the
`
`Lernout and Hauspie software for speech synthesis. For example, in the mc_vm.c
`
`file, the “app_lnhtts.h” is included at line 50. Ex. 2025, mc_vm.c at line 50. Further,
`
`the stringToSpeech() function in the same file accepts a text buffer as a parameter
`
`and uses the Lernout and Hauspie library function to convert that text buffer into an
`
`audio file in the proper format, and then plays back the file to the user. Ex. 2025,
`
`mc_vm.c at lines 12365-12410.
`
`38.
`
`The TTS software engine was operatively connected to said computer
`
`through the TTS library software component integrated into the Webley Assistant
`
`application running on said computer which enabled text to audio form conversion
`
`and playback to the user. The TTS software engine was further operatively
`
`connected to said voice enabled device because the audio files generated by that TTS
`
`software engine were then played back to the user by the Webley Assistant
`
`application.
`
`20
`
`Parus Exhibit 2020, Page 27 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`21
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2025 mc_vm.c at lines 13,607-13,641.
`
`41.
`
`The webget.pl PERL module, which was modified on 12/13/1999,
`
`assigns its argument value, for example “weather”, to variable $service which is then
`
`passed to a function reading in the weather.ini content descriptor file:
`
`22
`
`Parus Exhibit 2020, Page 29 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`2032 at lines 1-10 and 24-31.
`
`42.
`
`The weather.ini file includes a plurality of pre-selected web site
`
`addresses and other data, each web site address identifying a web site URL
`
`containing weather information to be retrieved. For example, the weather.ini file
`
`included URLs, or web site addresses, containing weather information on cnn.com,
`
`lycos.com, weather.com, snap.com, and infospace.com:
`
`Ex. 2033 at lines 1-4, 63-68, 121-124, 183-186, and 241-245.
`
`23
`
`Parus Exhibit 2020, Page 30 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`The getWeather() function, the webget.pl PERL module, and the
`
`43.
`
`weather.ini content descriptor file are an instruction set for identifying weather
`
`information to be retrieved, said instruction set being associated with said computer
`
`because it runs on the computer, and the instruction set comprises a plurality of pre-
`
`selected web site addresses and other data, which are included in the weather.ini
`
`content descriptor file, and each web site address identifies a web site URL
`
`containing weather information to be retrieved.
`
`24
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2040, menuProto.grammar at lines 201-215.
`
`45.
`
`The weather grammar allowed a user to request the weather information
`
`from the Webley Assistant. When a user utters the “weather in zip code 60015”
`
`command, the Nuance ASR Engine would recognize that command based on the
`
`match found in the weather sub-grammar within the grammar section “MENU90”.
`
`The Nuance ASR engine then would pass code 194 plus the “60015” parameter for
`
`the zip code to the Webley Assistant application. Once the Webley Assistant
`
`application receives the command code and the zip parameter from the Nuance ASR
`
`Engine, it would call the appropriate function(s) to retrieve the weather information
`
`in the user desired location.
`
`46.
`
`Specifically, the login_vmail() function, found in the Webley Assistant
`
`application software file mc_vm.c, is passed the command code returned from the
`
`Nuance ASR Engine. The user weather command is then processed in the command
`
`code matching section of the long switch statement:
`
`25
`
`Parus Exhibit 2020, Page 32 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2025 mc_vm.c at lines 1816-1817.
`
`Ex. 2025 mc_vm.c at lines 2226-2227.
`
`47. At line 2766, the weather command code passed from the Nuance ASR
`
`Engine is matched to case “MD_WEATHER,” and the getWeather() function or
`
`instruction set is called.
`
`Ex. 2025 mc_vm.c at lines 2766-2775.
`
`48.
`
`In the mcall.h file we see that the constant “MD_WEATHER”
`
`corresponds to integer 194, which is the command code returned by the Nuance ASR
`
`Engine upon matching the user utterance to the proper weather sub-grammar:
`
`Ex. 2041 mcall.h at 861-869.
`
`26
`
`Parus Exhibit 2020, Page 33 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`49. Although the getWeather() function is commented out in this code, it
`
`demonstrates that it was working as of the date of the mc_vm.c file which was
`
`modified 12/31/1999.
`
`a.
`
`Evidence of the early use of the voice browsing
`weather grammar on or before 12/31/1999
`The Webley Assistant used recognition grammars for recognizing
`
`50.
`
`speech commands. The menuProto.grammar was a prototype grammar used for
`
`initial implementation of the voice browsing portal, as part of the Webley Assistant,
`
`that included weather information retrieval related functionality. That grammar was
`
`first submitted to the Webley CVS source control system on January 8, 2000. Here
`
`is the header of this grammar file showing the CVS commit date and my (Alex
`
`Kurganov) alias “alex” as the developer of that grammar:
`
`Ex. 2040 menuProto.grammar.
`
`51. Despite the fact the menuProto.grammar file was first submitted to CVS
`
`source control on 1/8/2000, we have the following evidence showing that this
`
`27
`
`Parus Exhibit 2020, Page 34 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`grammar and specifically its voice portal weather related part was used earlier in
`
`1999.
`
`52.
`
`For example, file mc_vm.c, submitted to CVS source control system on
`
`12/31/1999 (see below), is the main Webley Assistant application C-code module.
`
`Ex. 2025 mc_vm.c.
`
`53.
`
`This earlier file includes voice portal weather related functionality
`
`which is objectively linked to the weather grammar menuProto.grammar as is
`
`demonstrated below. Specifically, the menuProto.grammar included several sub-
`
`grammars that correspond to instruction sets and to speech commands including the
`
`weather sub-grammar, “MENU90”, that was used in the initial voice browsing portal
`
`prototype.
`
`28
`
`Parus Exhibit 2020, Page 35 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`Ex. 2040 menuProto.grammar at lines 201-215.
`
`54. When a user utters the “weather” or “what is the weather in area code
`
`312” command the Nuance ASR speech engine would determine the command is
`
`weather related. At that point, the engine would return code 194 plus perhaps the
`
`optional area code parameter into the main voice application module (see code 194
`
`in the above grammar fragment).
`
`55. At line 2766 of the main voice application module mc_vm.c, the
`
`weather command is matched to case “MD_WEATHER.” Although the
`
`getWeather() function is commented out in this code, it demonstrates that it was
`
`working as of the date of the mc_vm.c file which was last modified 12/31/1999.
`
`Ex. 2025 mc_vm.c at lines 2766-2775.
`
`29
`
`Parus Exhibit 2020, Page 36 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`Further, in the main voice application include file mcall.h, also
`
`56.
`
`submitted to CVS on 12/31/1999 we can see that “MD_WEATHER” is a command
`
`equal to 194, which is what the MENU90 sub-grammar would return.
`
`Ex. 2041 mcall.h.
`
`Ex. 2041 mcall.h at 861-869.
`
`57.
`
`Therefore, the above evidence based on the source code files submitted
`
`to the CVS source control system on 12/31/1999 (the CVS system and the files are
`
`still available today for verification) demonstrates that the weather related grammar
`
`was in fact used on or before 12/31/1999 before it was first submitted to the CVS
`
`source control system on 1/8/2000.
`
`30
`
`Parus Exhibit 2020, Page 37 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`31
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`zip parameter in our example. The login_vmail() function, found in the Webley
`
`Assistant application software file mc_vm.c, is passed the command code returned
`
`from the Nuance ASR Engine. The user weather command is then processed in the
`
`section of the long switch statement matching the command code:
`
`Ex. 2025 at lines 2226-2227.
`60. At line 2766, the weather command code is matched to case
`
`“MD_WEATHER,” which we know is 194 as discussed previously.
`
`Ex. 2025 at lines 2766-2775.
`
`61. Once the weather command is matched, the weather instructions set is
`
`triggered by the getWeather() call.
`
`32
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`in the Webley Assistant application software file mc_vm.c, is passed the command
`
`code returned from the Nuance ASR Engine. The user weather command is then
`
`processed in the section of the long switch statement matching the command code:
`
`Ex. 2025, mc_vm.c at lines 1816-1818.
`
`Ex. 2025, mc_vm.c at lines 2226-2227.
`
`63. At line 2766, the weather command code is matched to case
`
`“MD_WEATHER.” Although the getWeather() function is commented out in this
`
`code, it demonstrates that it was working as of the date of the mc_vm.c file which
`
`was last modified 12/31/1999.
`
`Ex. 2025, mc_vm.c at lines 2766-2775.
`
`64. Once the weather command is matched, the weather instructions set is
`
`triggered with the getWeather() call.
`
`33
`
`Parus Exhibit 2020, Page 40 of 69
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`
`34
`
`
`
`Case Nos. IPR2020-00846, IPR2020-00847
`U.S. Patent Nos. 7,076,431, 9,451,084
`weather information to be retrieved, the webget.pl PERL script calls the
`
`process_section subroutine, which accesses at least one of said plurality of web sites.
`
`Ex. 2032, webget.pl at lines 96-1
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
