Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 1 of 46 PageID #: 1
`
`IN THE UNITED STATES DISTRICT COURT
`FOR THE DISTRICT OF DELAWARE
`
`C.A. No.
`
`JURY TRIAL DEMANDED
`
`)
`)
`)
`)
`)
`)
`)
`)
`
`))
`
`
`)
`)
`)
`)
`
`VB Assets, LLC,
`
`Plaintiff,
`
`v.
`
`Amazon.com, Inc.; Amazon.com LLC; Amazon
`Web Services, Inc.; A2Z Development Center,
`Inc. d/b/a Lab126; Rawles LLC; AMZN Mobile
`LLC; AMZN Mobile 2 LLC; Amazon.com
`Services, Inc. f/k/a Amazon Fulfillment Services,
`Inc.; and Amazon Digital Services LLC,
`
`Defendants.
`
`COMPLAINT FOR PATENT INFRINGEMENT
`
`VB Assets, LLC (“Plaintiff” or “VoiceBox”) hereby alleges as follows for its complaint
`
`against Defendants Amazon.com, Inc.; Amazon.com LLC; Amazon Web Services, Inc.; A2Z
`
`Development Center, Inc. d/b/a Lab126; Rawles LLC; AMZN Mobile LLC; AMZN Mobile 2
`
`LLC; Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc.; and Amazon Digital
`
`Services LLC (collectively, “Defendants” or “Amazon”):
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 2 of 46 PageID #: 2
`
`NATURE OF THE ACTION
`
`1.
`
`VoiceBox, through its predecessor companies VoiceBox Technologies
`
`Corporation and VoiceBox Technologies, Inc. (collectively “VoiceBox Technologies”), pioneered
`
`voice-based search and commerce technology. It invented what Amazon itself has described as
`
`“Echo-like” products long before Amazon.
`
`2.
`
`In recognition of its many innovations, the U.S. Patent & Trademark Office
`
`awarded and issued the VoiceBox Patents.1 The innovations in these patents were fundamental to
`
`the development of voice commerce technology.
`
`3.
`
`VoiceBox Technologies’ opportunities to promote and build a business based on
`
`these patents were crushed when Amazon introduced the infringing Echo and Alexa Products2
`
`and used its enormous size and clout to poach dozens of VoiceBox Technologies’ engineers and
`
`scientists.
`
`4.
`
`VoiceBox has brought this case to hold Amazon accountable for its infringement
`
`of VoiceBox’s patent rights.
`
`1 “VoiceBox Patents” collectively refers to U.S. Patent Nos. 8,073,681 (“the ’681 patent”);
`9,015,049 (“the ’049 patent”); 9,626,703 (“the ’703 patent”); 7,818,176 (“the ’176 patent”);
`8,886,536 (“the ’536 patent”); and 9,269,097 (“the ’097 patent”).
`2 “Alexa Products” collectively refers to Amazon’s Alexa virtual assistant and offerings that
`include Alexa, including the Echo product line (such as Echo 1st Gen., Echo 2nd Gen., Echo Dot
`1st Gen., Echo Dot 2nd Gen., Echo Dot 3rd Gen., Echo Dot Kids Edition, Echo Show 1st Gen.,
`Echo Show 2nd Gen., Echo Show 5, Echo Spot, Echo Plus 1st Gen., Echo Plus 2nd Gen., Echo
`Auto, and Echo Look), Amazon’s Alexa apps, Music apps, and Shopping apps on a smartphone
`or other mobile device, Amazon’s Alexa cloud, Alexa Voice Services, and Amazon.com
`website, and any other device, app, or instrumentality that includes, provides access to, or works
`with Alexa (such as Amazon Tap, Amazon Dash Wand, Echo Wall Clock, AmazonBasics
`Microwave, Amazon SmartPlug, Amazon Fire TV Sticks, Amazon Fire TVs, Amazon Fire TV
`Cubes, and Amazon Fire and Fire HD tablets) as well as software, hardware, and cloud
`infrastructure associated with any of the foregoing.
`
`2
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 3 of 46 PageID #: 3
`
`THE PARTIES
`
`5.
`
`VB Assets, LLC, VoiceBox, is a limited liability company organized under the
`
`laws of Delaware and has its principal place of business at 1229A 120th Ave. NE, Bellevue, WA
`
`98005.
`
`6.
`
`On information and belief, Amazon.com, Inc. is a corporation incorporated in
`
`Delaware and has a principal place of business at 410 Terry Avenue North, Seattle, WA, 98109.
`
`On information and belief, Amazon.com, Inc. is the ultimate parent company of the other
`
`companies that make up Amazon, and is responsible for making, using, selling, offering for sale
`
`and/or importing Alexa Products.
`
`7.
`
`On information and belief, Amazon.com LLC is a limited liability company
`
`organized under the laws of Delaware and has a principal place of business at 410 Terry Avenue
`
`North, Seattle, WA, 98109. On information and belief, Amazon.com LLC includes as members
`
`various operating companies, which make, use, sell, offer for sale, and/or import Alexa Products.
`
`8.
`
`On information and belief, Amazon Web Services, Inc. is a corporation
`
`incorporated in Delaware and has a principal place of business at 410 Terry Avenue North,
`
`Seattle, WA, 98109. On information and belief, Amazon Web Services, Inc. provides Alexa cloud
`
`computing platforms, that are Alexa Products or are for use with one or more Alexa Products.
`
`9.
`
`On information and belief, A2Z Development Center, Inc. d/b/a Lab126 is a
`
`corporation incorporated in Delaware and has its principal place of business at 1120 Enterprise
`
`Way Sunnyvale, CA 94089. On information and belief, A2Z Development Center, Inc. d/b/a
`
`Lab126 performed research and development, including for one or more Alexa Products.
`
`10.
`
`On information and belief, Rawles LLC is a limited liability company formed
`
`under the laws of Delaware and has its principal place of business at 103 Foulk Road, Suite 100,
`
`3
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 4 of 46 PageID #: 4
`
`Wilmington, DE 19803. On information and belief, Rawles LLC performed research and
`
`development, including for one or more Alexa Products.
`
`11.
`
`On information and belief, AMZN Mobile LLC is a limited liability company
`
`organized under the laws of Delaware and has a principal place of business near Seattle, WA. On
`
`information and belief, AMZN Mobile LLC provides mobile apps that are Alexa Products or are
`
`for use with Alexa Products.
`
`12.
`
`On information and belief, AMZN Mobile 2 LLC is a limited liability company
`
`organized under the laws of Delaware and has a principal place of business near Seattle, WA. On
`
`information and belief, AMZN Mobile 2 LLC provides mobile apps that are Alexa Products or
`
`are for use with Alexa Products.
`
`13.
`
`On information and belief, Amazon.com Services, Inc. f/k/a Amazon Fulfillment
`
`Services, Inc. is a corporation incorporated in Delaware and has a principal place of business at
`
`410 Terry Ave. N. Seattle, WA 98109. On information and belief, Amazon.com Services, Inc.
`
`f/k/a Amazon Fulfillment Services, Inc. participated in the sale or offer for sale of one or more
`
`Alexa Products.
`
`14.
`
`On information and belief, Amazon Digital Services LLC is a limited liability
`
`company organized under the laws of Delaware and has a principal place of business at 410
`
`Terry Ave. N, Seattle, WA 98109. On information and belief, Amazon Digital Services LLC has
`
`participated in the sale or offer for sale of one or more Alexa Products and/or has provided digital
`
`services and content for use by Alexa Products.
`
`JURISDICTION AND VENUE
`
`15.
`
`This Court has original jurisdiction over the subject matter of this action under 28
`
`U.S.C. §§ 1331 and 1338(a) because the action arises under the patent laws of the United States.
`
`4
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 5 of 46 PageID #: 5
`
`16.
`
`Personal jurisdiction over each Defendant is proper in this District based on one
`
`or more of the following: its presence in this judicial district; it has availed itself of the rights and
`
`benefits of the laws of Delaware; or it has derived substantial revenue from sales of Alexa
`
`Products in Delaware and it has systematic and continuous business contacts with Delaware.
`
`Each Defendant was incorporated in Delaware and/or formed under the laws of Delaware and
`
`Amazon designs Alexa Products, which are advertised, offered for sale, sold, and used in
`
`Delaware.
`
`17.
`
`Venue is proper in this district under 28 U.S.C. § 1400(b) and 28 U.S.C. §§
`
`1391(b)(1), (b)(2). For purposes of § 1400(b), each Defendant was incorporated in Delaware
`
`and/or formed under the laws of Delaware and therefore resides within this District. For purposes
`
`of § 1391(b)(1), (b)(2), Amazon resides in the District of Delaware by virtue of being
`
`incorporated in Delaware and/or formed under the laws of Delaware.
`
`FACTUAL BACKGROUND
`
`A.
`
`18.
`
`VoiceBox Technologies Invents Groundbreaking Voice Technology
`
`In 2001, three brothers, Mike, Rich, and Bob Kennewick founded VoiceBox
`
`Technologies to bring natural language understanding (“NLU”) to a wide array of computer
`
`applications. They recognized that the typical computer speech-recognition systems forced
`
`human operators to adhere to a limited number of rigid speech prompts. These rigid prompts
`
`limited how systems were used and inhibited the widespread adoption of speech-recognition
`
`systems. The brothers believed that VoiceBox Technologies could become the first company to
`
`enable people to naturally and effectively interact with computer speech systems.
`
`19.
`
`From its inception, VoiceBox Technologies engaged in intense research efforts to
`
`develop its NLU technology. As part of these efforts, VoiceBox Technologies achieved a
`
`5
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 6 of 46 PageID #: 6
`
`significant milestone when it developed an early prototype called “Cybermind.” As demonstrated
`
`on the King5 news, Cybermind was a voice-controlled speaker that could provide weather,
`
`recipes, sports scores, calendar updates, or play a song.3
`
`Figure 1: Cybermind Prototype
`
`20.
`
`VoiceBox Technologies’ groundbreaking work did not go unrecognized. After
`
`learning about VoiceBox Technologies’ technology, Toyota hired it to build a sophisticated NLU
`
`speech interface for its Lexus automobiles. VoiceBox Technologies built the voice and NLU
`
`capability for Toyota’s award-winning Entune multimedia system.
`
`21.
`
`As part of the development effort of an NLU interface for Lexus, VoiceBox
`
`Technologies demonstrated a personal assistant called “Alexus” that showcased the power of its
`
`Conversational Voice technology. On information and belief, the VoiceBox Technologies
`
`“Alexus” concept was introduced to the public more than six months before Amazon announced
`
`“Alexa.”
`
`3https://www.youtube.com/watch?v=DDcRyPnvWhw
`
`6
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 7 of 46 PageID #: 7
`
`Figure 2: “Alexus” Demonstration
`
`22.
`
`Throughout its research and development efforts, VoiceBox Technologies realized
`
`that its technology could be deployed in a wide range of applications from connected home to
`
`mobile personal assistants.
`
`Figure 3: Connected Home
`
`Figure 4: Mobile
`
`23.
`
`By January 2012, VoiceBox Technologies was a leader in NLU and
`
`conversational voice technology. Leading companies throughout the world, including Toyota,
`
`Lexus, TomTom, Pioneer, Chrysler, Dodge, and Magellan used VoiceBox Technologies’ award-
`
`winning and patented contextual speech technology. VoiceBox Technologies had software
`
`7
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 8 of 46 PageID #: 8
`
`applications that ran on smart speakers, in-car systems, smartphones, smart TVs, computers,
`
`tablets, e-readers, and personal navigation devices.
`
`24.
`
`In 2013, the Institute of Electrical and Electronics Engineers (“IEEE”) ranked
`
`VoiceBox Technologies number 13 in patent power for the computer software industry.
`
`VoiceBox Technologies had become the leader in conversational Artificial Intelligence (“AI”),
`
`including Voice Recognition (VR), NLU, and AI services.
`
`B.
`
`25.
`
`Amazon Takes VoiceBox’s Technology
`
`In 2011, VoiceBox Technologies contacted Amazon to explore a potential
`
`business relationship where VoiceBox Technologies would provide core NLU services to
`
`Amazon. Amazon’s corporate development department expressed interest and asked for
`
`“company and/or product overview slides” to facilitate an October 7, 2011 teleconference. In
`
`response, VoiceBox Technologies provided Amazon with a presentation that described its award-
`
`winning patented technology and explicitly referred to VoiceBox Technologies’ “patented
`
`Contextual Speech Technology.” Slides from that presentation are reproduced below:
`
`Figure 5: 2011 Slides
`
`26.
`
`Amazon was so impressed by the technology VoiceBox Technologies presented
`
`on October 7, 2011 that its representative emailed VoiceBox Technologies the next business day
`
`8
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 9 of 46 PageID #: 9
`
`to invite VoiceBox to visit Amazon’s offices on October 19, 2011. That meeting was with
`
`Douglas Booms, Amazon’s Vice President of Worldwide Corporate Development, as well as
`
`engineers and product/business development members of Amazon’s devices and digital teams.
`
`The email from Amazon stated that this was “the right audience to discuss [VoiceBox
`
`Technologies’] personal digital assistant and underlying conversational voice technology.” On
`
`information and belief, in addition to Mr. Booms, the following Amazon personnel attended the
`
`meeting: Nick Komorous (Director, Corporate Development), Ian Freed (VP, Amazon Devices),
`
`Greg Hart (VP, Digital), Al Lindsay (Director, Software Engineer), Frederic Deramat (Senior
`
`Principal Engineer), and John Thimsen (Software Developer). On information and belief, another
`
`Amazon engineer from Cupertino attended by teleconference.
`
`27.
`
`Two days after the meeting at Amazon, on Friday October 21, 2011, Amazon’s
`
`Mr. Komorous emailed VoiceBox Technologies and asked to visit the company’s office for a
`
`“deeper dive.” Mr. Komorous requested that this meeting occur the very next week. VoiceBox
`
`Technologies agreed to host Amazon’s personnel for a meeting at VoiceBox Technologies’ office
`
`on October 26, 2011. On information and belief, from Amazon, Marcello Typrin (Product
`
`Management / Business Development), Frederic Deramat (Senior Principal Engineer), John
`
`Thimsen (Software Developer), and Sean Fitz (Software Developer) joined Mr. Komorous at the
`
`meeting.
`
`28.
`
`In advance of the meeting, Mr. Komorous sent a detailed set of technical
`
`questions that would help Amazon’s “tech team understand the scope and [r]ange of things
`
`[Amazon] can try to tackle with VoiceBox[] as a partner.” Mr. Komorous also indicated that
`
`Amazon’s culture was “engineering heavy” and asked that VoiceBox Technologies “have
`
`engineering and speech representation at the meeting.”
`
`9
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 10 of 46 PageID #: 10
`
`29.
`
`The October 26, 2011 meeting at VoiceBox Technologies’ office ran from 10 am
`
`until 12 noon with some Amazon engineers staying even later until around 2:30 pm. The meeting
`
`included a tour with additional demos, a review of the technical architecture for VoiceBox
`
`Technologies’ server software, and plans for next generation products.
`
`30.
`
`During the meeting, VoiceBox Technologies presented a deck of 42 slides to the
`
`visiting Amazon personnel. The slides provided even more detail about VoiceBox Technologies’
`
`patented technology and informed Amazon that VoiceBox Technologies had 12 patents at the
`
`time with an additional 14 pending applications. The slide deck included the following slide
`
`regarding the ’176 patent, which is asserted in this lawsuit.
`
`Figure 6: Slide from 2011 Presentation
`
`10
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 11 of 46 PageID #: 11
`
`31.
`
`The slide deck VoiceBox Technologies presented to Amazon at the October 26,
`
`2011 meeting also proposed a business arrangement where VoiceBox Technologies would
`
`provide “Voice Services” to Amazon. The VoiceBox Technologies Voice Services from the
`
`presentation are shown below:
`
`Figure 7: Slide from 2011 Presentation
`
`32.
`
`On information and belief, some of the Amazon personnel involved in the 2011
`
`meetings became technical leaders for Amazon’s Alexa Products while others became high-level
`
`executives with close working relationships with the senior leadership of Amazon. On
`
`information and belief, one of Mr. Freed’s past positions was Technical Advisor to CEO, a
`
`position that has been described as CEO Jeff Bezos’s “shadow” advisor. On information and
`
`11
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 12 of 46 PageID #: 12
`
`belief, Mr. Freed then went on to become Vice President, Amazon Devices where he led a team
`
`of engineers working on Amazon’s Alexa Products. On information and belief, Mr. Hart has also
`
`been Mr. Bezos’s “shadow” advisor—he held the Technical Advisor to CEO position around
`
`2011. On information and belief, around 2011, Al Lindsey was promoted to Vice President
`
`managing the Alexa Engine Software team. On information and belief, around 2011, Mr.
`
`Deramat was promoted to the position Vice President & Distinguished Engineer for Amazon
`
`Alexa. On information and belief, around 2011, Mr. Thimsen was promoted to the position
`
`Director of Engineering for Amazon Echo. As for Mr. Typrin, he states on his LinkedIn page that
`
`he is “[o]ne of the founding members of the team that shaped the vision and direction for
`
`Amazon's Echo and Alexa Voice Services.”
`
`33.
`
`A couple days after the last meeting in 2011, VoiceBox Technologies sent an
`
`email to Amazon asking for Amazon’s feedback. Mr. Komorous from Amazon replied that
`
`Amazon was “still discussing internally how contextual speech / cybermind 2012 could play a
`
`part in [Amazon’s] future.” Amazon did not provide the results of these discussions and did not
`
`pursue a business relationship with VoiceBox Technologies. Instead, on information and belief,
`
`Amazon decided to build its Alexa Products—without telling VoiceBox Technologies or asking
`
`permission to use VoiceBox Technologies’ patented technology.
`
`34.
`
`In 2014, Amazon announced the launch of Alexa, a virtual assistant, along with
`
`the first-generation Echo product, a smart speaker. Amazon’s Alexa and first-generation Echo
`
`product were strikingly similar to the patented technology that VoiceBox Technologies showed
`
`Amazon in 2011.
`
`35.
`
`In 2016, Amazon abruptly hired Philippe Di Cristo, who was VoiceBox
`
`Technologies’ Chief Scientist. While at VoiceBox Technologies, Mr. Di Cristo gained knowledge
`
`12
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 13 of 46 PageID #: 13
`
`of the company’s voice technology and had full access to VoiceBox Technologies’ intellectual
`
`property. As Mr. Di Cristo explains on his LinkedIn Page, he had worked on an “Amazon Echo-
`
`like system” while at VoiceBox Technologies. Exhibit A at 3. On information and belief, Mr. Di
`
`Cristo has helped design and implement VoiceBox Technologies’ patented technology into
`
`Amazon’s Echo and other Alexa Products.
`
`36.
`
`On information and belief, Mr. Di Cristo played an active role in soliciting
`
`additional VoiceBox employees to join Amazon. For example, shortly after Mr. Di Cristo joined
`
`Amazon from VoiceBox Technologies, Amazon ramped-up its efforts to recruit VoiceBox
`
`Technologies employees.
`
`37.
`
`On January 10, 2017, Amazon hosted an “Evening with the Leadership of
`
`Amazon Voice & Advanced Shopping,” which Amazon expressly described as an “invite only
`
`networking event for Voice Box employees . . . to talk . . . about opportunities at Amazon.” The
`
`director of Amazon’s Voice & Advanced NUI Shopping group— on information and belief, the
`
`group Mr. Di Cristo joined—sent an invite for the event to a large number of VoiceBox
`
`Technologies employees and indicated that Amazon was “building a world-class speech & NLU
`
`engineering team” and “[y]our profile looks quite relevant and we’d love to talk to you and see if
`
`there’s a fit.” For this event, Amazon rented out Seastar, the premier seafood restaurant near
`
`VoiceBox Technologies’ office.
`
`38.
`
`On information and belief, Mr. Di Cristo was originally scheduled to be a speaker
`
`at the event targeting VoiceBox Technologies employees—but abruptly withdrew. The event
`
`emphasized Amazon’s need for VoiceBox Technologies’ NLU technology. On information and
`
`belief, around this time, Amazon faced numerous NLU challenges and had only completed a
`
`13
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 14 of 46 PageID #: 14
`
`small portion of work toward Amazon’s goal to build conversational dialog interaction into
`
`Alexa Products.
`
`39.
`
`On January 17, 2017, after Amazon’s rampant poaching of VoiceBox
`
`Technologies employees came to light, Mike Kennewick, CEO of VoiceBox Technologies, sent a
`
`letter to Jeff Bezos, CEO of Amazon, to propose a business solution. A true and correct copy of
`
`that letter is attached as Exhibit B. The letter explains that, at the time, VoiceBox Technologies
`
`had “a deep portfolio of technology and IP, including a large number of significant patents not
`
`only in NLU but also in Voice[]Commerce, running on over 200 million devices.”
`
`40.
`
`On information and belief, Mr. Booms, Amazon’s Vice President of Worldwide
`
`Corporate Development, responded to VoiceBox Technologies’ January 17, 2017 letter on behalf
`
`of Mr. Bezos. In Amazon’s response, Mr. Booms requested a meeting at VoiceBox Technologies’
`
`offices for purposes of “go[ing] fairly deep on the technology, data, customer relationships.”
`
`41.
`
`On February 2, 2017, the parties met at VoiceBox Technologies’ office. Amazon
`
`came with a team of technologists from its Alexa program, including on information and belief,
`
`Manoj Sindhwani (Director Alexa), Karthik Ramakrishnan (Senior Manager, Alexa software),
`
`Nikko Strom (Scientist, Alexa), and Deepesh Mohnani (Alexa Voice Services Product
`
`Management) joined by Mr. Booms. VoiceBox Technologies provided a detailed technical
`
`presentation, which included information about patents and pending applications then owned by
`
`VoiceBox Technologies. Slides in the presentation listed all VoiceBox Technologies patents and
`
`published applications at the time, including the ’681 patent, the ’049 patent, the ’176 patent, the
`
`’536 patent, and the ’097 patent. The February 2, 2017 slides also reproduced a highlighted claim
`
`from the ’681 patent and a highlighted claim from the ’176 patent.
`
`14
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 15 of 46 PageID #: 15
`
`42.
`
`Following the meeting, Mr. Booms emailed VoiceBox Technologies to request
`
`even more technical details. Then, on February 20, 2017, Amazon specifically asked VoiceBox
`
`Technologies for a list of all patents owned by the company.
`
`43.
`
`In March of 2017, VoiceBox Technologies hosted another meeting with Amazon
`
`personnel at VoiceBox Technologies’ office. The Amazon attendees included senior executives
`
`and managers responsible for Alexa. VoiceBox Technologies informed Amazon by email after
`
`the meeting that VoiceBox Technologies had “[p]atents that could be useful as the market goes
`
`mainstream.” Shortly, thereafter, Mr. Komorous from Amazon emailed VoiceBox Technologies,
`
`writing that Amazon had been “[poring] through the material” VoiceBox Technologies provided
`
`and that Amazon had created yet another, even more detailed, list of requests.
`
`44.
`
`In April of 2017, VoiceBox Technologies shared a written summary of VoiceBox
`
`Technologies’ patent portfolio with Amazon. The written summary explained that VoiceBox
`
`Technologies’ patents covered “core & fundamental areas.” The summary showcased the ’176
`
`patent and the ’703 patent, and identified the ’536 patent and ’097 patent by number. The
`
`summary further explained that “[v]oice advertising & conversational e-commerce are essential
`
`to the core strategies and future revenue streams of many of the world’s leading technology
`
`companies, including: . . . Amazon . . . .”
`
`OVERVIEW OF VOICEBOX’S PATENTS-IN-SUIT
`
`45.
`
`United States Patent Number 8,073,681 (“the ’681 patent”), entitled “System and
`
`Method for a Cooperative Conversational Voice User Interface,” was duly and legally issued on
`
`December 6, 2011, and names Larry Baldwin, Tom Freeman, Michael Tjalve, Blane Ebersold,
`
`and Chris Weider as the inventors. Attached as Exhibit C is a true and correct copy of the ’681
`
`patent.
`
`15
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 16 of 46 PageID #: 16
`
`46.
`
`The ’681 patent claims, among other things, a system for providing a cooperative
`
`conversational voice user interface, comprising: a voice input device configured to receive an
`
`utterance during a current conversation with a user; and a conversational speech engine, wherein
`
`the conversational speech engine includes one or more processors configured to: accumulate
`
`short-term shared knowledge about the current conversation, wherein the short-term shared
`
`knowledge includes knowledge about the utterance received during the current conversation;
`
`accumulate long-term shared knowledge about the user, wherein the long-term shared knowledge
`
`includes knowledge about one or more past conversations with the user; identify a context
`
`associated with the utterance from the short-term shared knowledge and the long-term shared
`
`knowledge; infer additional information about the utterance from the short-term shared
`
`knowledge and the long-term shared knowledge in response to determining that the utterance
`
`contains insufficient information to complete a request in the identified context; establish an
`
`intended meaning for the utterance within the identified identify a context based on the
`
`additional information inferred about the utterance; and generate a response to the utterance
`
`based on the intended meaning established within the identified context.
`
`47.
`
`48.
`
`VoiceBox is the assignee of the entire right, title, and interest in the ’681 patent.
`
`United States Patent Number 9,015,049 (“the ’049 patent”), entitled “System and
`
`Method for a Cooperative Conversational Voice User Interface,” was duly and legally issued on
`
`April 21, 2015, and names Larry Baldwin, Tom Freeman, Michael Tjalve, Blane Ebersold, and
`
`Chris Weider as the inventors. Attached as Exhibit E is a true and correct copy of the ’049 patent.
`
`49.
`
`The ’049 patent claims, among other things, a system for facilitating
`
`conversation-based responses, the system comprising: one or more physical processors
`
`programmed with one or more computer program instructions such that, when executed, the one
`
`16
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 17 of 46 PageID #: 17
`
`or more computer program instructions cause the one or more physical processors to: receive a
`
`natural language utterance during a conversation between a user and the system; identify a first
`
`model that includes short-term knowledge about the conversation, wherein the short-term
`
`knowledge is based on one or more prior natural language utterances received during the
`
`conversation; identify, based on the short-term knowledge, context information for the natural
`
`language utterance; determine, based on the context information, an interpretation of the natural
`
`language utterance; and generate, based on the interpretation of the natural language utterance, a
`
`response to the natural language utterance.
`
`50.
`
`51.
`
`VoiceBox is the assignee of the entire right, title, and interest in the ’049 patent.
`
`United States Patent Number 9,626,703 (“the ’703 patent”), entitled “Voice
`
`Commerce,” was duly and legally issued on April 18, 2017, and names Michael R. Kennewick,
`
`Sr. as the inventor. Attached as Exhibit G is a true and correct copy of the ’703 patent.
`
`52.
`
`The ’703 patent claims, among other things, a system for providing voice
`
`commerce, the system comprising: one or more physical processors programmed with computer
`
`program instructions which, when executed, cause the one or more physical processors to:
`
`receive a user input comprising a natural language utterance; provide, without further user input
`
`after the receipt of the user input, the natural language utterance as an input to a speech
`
`recognition engine; obtain, without further user input after the receipt of the user input, one or
`
`more words or phrases recognized from the natural language utterance as an output of the speech
`
`recognition engine; identify, without further user input after the receipt of the user input, a
`
`context based at least on the one or more words or phrases; determine, without further user input
`
`identifying a product or service after the receipt of the user input, a first product or service to be
`
`purchased on behalf of a user based at least on the determined context; obtain, without further
`
`17
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 18 of 46 PageID #: 18
`
`user input identifying payment information after the receipt of the user input, first payment
`
`information with which to pay for the product or service; obtain, without further user input
`
`identifying shipping information after the receipt of the user input, first shipping information
`
`with which to deliver the product or service, wherein the first shipping information specifies a
`
`name or address of a recipient to which the product or service is to be delivered after the product
`
`or service is purchased; and complete, without further user input identifying a product or service,
`
`payment information, or shipping information after the receipt of the user input, a purchase
`
`transaction for the first product or service based on the first payment information and the first
`
`shipping information.
`
`53.
`
`54.
`
`VoiceBox is the assignee of the entire right, title, and interest in the ’703 patent.
`
`United States Patent Number 7,818,176 (“the ’176 patent”), entitled “System and
`
`Method for Selecting and Presenting Advertisements Based on Natural Language Processing of
`
`Voice-based Input,” was duly and legally issued on October 19, 2010, and names Tom Freeman
`
`and Mike Kennewick as the inventors. Attached as Exhibit I is a true and correct copy of the
`
`’176 patent.
`
`55.
`
`The ’176 patent claims, among other things, a system for selecting and presenting
`
`advertisements in response to processing natural language utterances, comprising: an input
`
`device that receives a natural language utterance containing at least one request at an input
`
`device; a speech recognition engine coupled to the input device, wherein the speech recognition
`
`engine recognizes one or more words or phrases in the natural language utterance, wherein to
`
`recognize the words or phrases in the natural language utterance, the speech recognition engine
`
`is configured to: map a stream of phonemes contained in the natural language utterance to one or
`
`more syllables that are phonemically represented in an acoustic grammar; and generate a
`
`18
`
`

`

`Case 1:19-cv-01410-MN Document 1 Filed 07/29/19 Page 19 of 46 PageID #: 19
`
`preliminary interpretation for the natural language utterance from the one or more syllables,
`
`wherein the preliminary interpretation generated from the one or more syllables includes the
`
`recognized words or phrases; a conversational language processor coupled to the speech
`
`recognition engine, wherein the conversational language processor is configured to: interpret the
`
`recognized words or phrases, wherein interpreting the recognized words or phrases includes
`
`establishing a context for the natural language utterance; select an advertisement in the context
`
`established for the natural language utterance; and present the selected advertisement via an
`
`output device.
`
`56.
`
`57.
`
`VoiceBox is the assignee of the entire right, title, and interest in the ’176 patent.
`
`United States Patent Number 8,886,536 (“the ’536 patent”), entitled “System and
`
`Method for Delivering Targeted Advertisements and Tracking Advertisement Interactions in
`
`Voice Recognition Contexts,” was duly and legally issued on November 11, 2014, and names
`
`Tom Freeman and Mike Kenn[e]wick as the inventors. Attached as Exhibit K is a true and
`
`correct copy of the ’536 patent.
`
`58.
`
`The ’536 patent claims, among other things, a computer-implemented method of
`
`providing promotional content related to one or more natural language utterances and/or
`
`responses, the method being implemented by a computer system that includes one or more
`
`physical processors executing one or more computer program instructions which, when
`
`executed, perform the method, the method comprising: receiving, at the one or more physical
`
`processors, a first natural language utterance; providing, by the one or more physical processors,
`
`a response to the first natural language utterance; receiving, at the one or more physical
`
`processors, a second natural language utterance relating to the first natural language utterance;
`
`identifying, by the one or more physical processors, requests associated with the second natural
`
`19
`
`

`