`FOR THE DISTRICT OF DELAWARE
`
`Civil Action No. 10-389-LPS
`
`CONSOLIDATED
`
`JURY DEMANDED
`
`))))))))))))))))))))))))))
`
`SOFTVIEW LLC,
`
`v.
`
`Plaintiff,
`
`APPLE INC.; AT&T MOBILITY LLC;
`DELL INC.; HTC CORP.; HTC
`AMERICA, INC.; HUAWEI
`TECHNOLOGIES CO., LTD.;
`FUTUREWEI TECHNOLOGIES, INC.;
`KYOCERA CORP.; KYOCERA
`WIRELESS CORP.; LG ELECTRONICS,
`INC.; LG ELECTRONICS USA, INC.;
`LG ELECTRONICS MOBILECOMM
`U.S.A., INC.; MOTOROLA MOBILITY
`INC.; SAMSUNG ELECTRONICS CO.,
`LTD.; SAMSUNG ELECTRONICS
`AMERICA, INC.; SAMSUNG
`TELECOMMUNICATIONS AMERICA,
`LLC; and SONY ERICSSON MOBILE
`COMMUNICATIONS (USA) INC.,
`
`Defendants.
`
`PLAINTIFF SOFTVIEW LLC'S TECHNOLOGY TUTORIAL
`
`2700769
`
`Motorola PX 1025_1
`
`
`
`Plaintiff SoftView LLC respectfully submits the enclosed technology tutorial related
`
`to the patents-in-suit.
`
`
`
`Dated: September 21, 2012
`
`
`
`
`
`BLANK ROME LLP
`
`/s/ Steven L. Caponi
`By: ______________________________
`Steven L. Caponi (I.D. No. 3484)
`1201 Market Street, Suite 800
`Wilmington, DE 19801
`(302) 425-6400
`caponi@blankrome.com
`
`Attorneys for SoftView LLC
`
`
`
`
`
`
`
`
`
`
`
`OF COUNSEL:
`
`Morgan Chu
`Samuel K. Lu
`Erin McCracken
`Dorian S. Berger
`Craig Johnson
`IRELL & MANELLA LLP
`1800 Avenue of the Stars, Suite 900
`Los Angeles, CA 90067-4276
`(310) 277-1010
`mchu@irell.com; slu@irell.com;
`emccracken@irell.com; dberger@irell.com;
`cjohnson@irell.com
`
`September 21, 2012
`
`Motorola PX 1025_2
`
`
`
`
`
`
`This is SoftView’s technology tutorial for the SoftView LLC v. Apple
`
`Incorporated. et al. case.
`
`
`
`
`Motorola PX 1025_3
`
`
`
`
`
`
`
`
`Hello, my name is Gary Rohrabaugh. I am one of the two named
`
`inventors of the '353 and '926 patents that are at issue in this litigation.
`
`
`In this tutorial, I will explain some of the basic concepts involved in
`displaying web pages on mobile devices. I will also describe the technology
`disclosed by the patents-in-suit.
`
`Motorola PX 1025_4
`
`
`
`
`
`
`
`
`
`Before I begin, let me provide a brief summary of my educational
`background and my relevant work history.
`
`
`I have a bachelor of science in management from Purdue University, where
`I took a number of computer science classes. From 1979 to 1982, I worked for
`Texas Instruments in its computer systems division.
`
`After leaving Texas Instruments, I started a number of computer software
`companies that specialize in displaying graphical information on video displays. I
`started SoftSource in 1982.
`
`Motorola PX 1025_5
`
`
`
`
`
`
`
`
`In 2000, SoftSource spun off its mobile web browser business as a new
`
`company called Catarra. In addition to being one of the founders of Catarra, I
`was also its Chief Technical Officer. Unfortunately, the Internet bubble burst a
`year later. Catarra shut its doors when it ran through the remainder of its venture
`capital financing.
`
`In 2004, I acquired all of the intellectual property assets of Catarra. And in
`2005, I founded SoftView, the plaintiff in this case.
`
`
`Motorola PX 1025_6
`
`
`
`
`
`
`
`
`In this technology tutorial, I will first provide some background about
`
`Internet browsers for mobile devices. Then, I will discuss SoftSource's novel
`technology, which is disclosed in the two patents-in-suit. Finally, I will describe
`the concepts behind the implementation of this technology.
`
`Motorola PX 1025_7
`
`
`
`
`
`
`
`
`
`One of SoftSource's core areas of expertise was software for creating and
`displaying computer-aided design drawings. You will hear me refer to this
`software as "CAD software." Before the introduction of CAD software,
`companies designing products such as automobiles or houses used paper
`blueprints. In fact, the term "blueprint" refers to the background color of the
`printed drawings.
`
`Motorola PX 1025_8
`
`
`
`
`
`
`
`With CAD software, drawings are created on a computer and stored
`electronically. Changes can be easily made to the drawings. And the drawings can
`be shared by engineers, even if they work in different locations.
`
`
`
`
`
`Motorola PX 1025_9
`
`
`
`
`
`
`
`
`
`CAD software also allows an engineer to view a drawing on a computer
`
`display rather than as a paper copy. Engineering drawings can be large and
`complex. Imagine a detailed drawing of a house as an example.
`
`CAD software also enables an engineer to zoom in on a portion of the
`
`drawing being displayed, as well as to move the relevant portion of the drawing
`into view on the display.
`
`Motorola PX 1025_10
`
`
`
`
`
`
`
`
`
`When the Internet took off in the mid-1990's, SoftSource began exploring
`
`whether its expertise in CAD software could be used in this new area of
`technology.
`
`CAD drawings, like web pages, are viewed by multiple users who may
`operate computers having different size and resolution displays. One user might
`view a drawing on a large and very expensive high resolution display associated
`with a top of the line computer workstation, while another user might view the
`same drawing on a much smaller, low resolution notebook computer.
`
`
`The insights that SoftSource acquired while working in the CAD field
`formed the basis for the web browser technologies that we later developed. In
`particular, SoftSource addressed the challenge of how to view web pages on the
`small displays found on mobile devices. Those web pages were intended for the
`much larger displays found on desktop and notebook computers.
`
`
`Motorola PX 1025_11
`
`
`
`
`
`
`
`
`
`The Court is no doubt familiar with computer programs called web
`browsers that are used to access and display web pages. These web browsers
`include Internet Explorer, Safari and Chrome.
`
`
`
`Motorola PX 1025_12
`
`
`
`
`
`
`
`
`
`
`Using a web browser, a user can surf the web and access a web page such
`as the New York Times.
`
`
`
`
`Motorola PX 1025_13
`
`
`
`
`
`
`
`
`
`
`In the late 1990's, web pages were almost uniformly accessed by users from
`desktop or notebook computers having large, easily viewed screens. Cellular
`phones were used primarily to make and to receive phone calls. And personal
`digital assistants were used primarily as electronic Rolodexes and calendars.
`
`However, it became apparent to those of us at SoftSource that, as mobile
`devices became more prevalent, users would want to access these web pages
`through their mobile devices. In fact, engineers were already requesting the ability
`to access CAD drawings on their mobile devices for use on construction sites and
`factory floors.
`
`
`
`
`Motorola PX 1025_14
`
`
`
`
`
`
`
`
`SoftSource's prediction came true. Today, web browsing is a central
`function of smart phones, tablets, and other mobile devices. In fact, cell phone
`carriers now charge by the amount of data downloaded rather than for the
`number of minutes of phone usage.
`
`
`
`Motorola PX 1025_15
`
`
`
`
`
`
`
`
`
`
`Let's go back to the late 1990's. At the time that SoftSource began working
`on its web browser, some companies and individuals had already begun
`developing technologies to enable the viewing of web pages on mobile devices.
`But these technologies were very limited.
`
`
`Motorola PX 1025_16
`
`
`
`
`
`
`
`
`
`Technically, it was possible to display an entire web page on the small
`
`screen of a mobile device. However, as you can see here, doing so was not
`particularly useful. The text and the images are too small to be read. The display
`of a mobile device had far fewer pixels than the display of a desktop or notebook
`computer.
`
`
`
`Motorola PX 1025_17
`
`
`
`
`
`
`
`
`
`
`Because of the difficulties encountered in displaying web pages on mobile
`devices, one approach was to strip out much of the information on a web page
`and reformat the remaining information so that it could be viewed on a mobile
`device. For example, this could involve removing all of the images on a web page
`and keeping only the text. This approach could also involve re-organizing the
`layout of the web page to make it easier to read on a mobile device’s small, low
`resolution display.
`
`One drawback to this approach is that much of the information on a web
`page might be contained in images. Another drawback to this approach is that re-
`organizing the layout of the web page means that the web page on the mobile
`device looks nothing like the web page as it would appear on a desktop or a
`notebook computer. This makes it difficult for a mobile device user to find the
`information he or she is looking for.
`
`
`Motorola PX 1025_18
`
`
`
`
`
`
`
`
`Now that I have provided some background about web browsing on
`early mobile devices, I will turn to SoftSource’s novel technology and the patents
`in suit.
`
`
`
`Motorola PX 1025_19
`
`
`
`
`
`
`
`
`
`
`SoftSource had experience in designing software for making CAD drawings
`viewable on a variety of devices. Thus, SoftSource was in a unique position to
`begin developing software for making web pages viewable on mobile devices.
`
`
`
`Motorola PX 1025_20
`
`
`
`
`
`SoftSource sought to design a web browser that was fully functional, even
`though the web page was being viewed on the small screen of a mobile device.
`SoftSource focused on a number of key features that would enable this
`functionality. I'll discuss the following four features in greater detail in a few
`moments, but they are:
`
`Preserving the original page layout of the web page;
`Zooming the web page in and out;
`Panning the web page; and
`Context-based zooming of the web page.
`
` •
`
`
`•
`•
`•
`
`
`
`
`
`
`
`
`Motorola PX 1025_21
`
`
`
`
`
`
`
`
`
`SoftSource realized that the user should be able to view the web page as it was
`intended to be viewed by the web page designer. In other words, the web page should
`not be reformatted to strip out information such as images. The layout of the web page
`should not be re-formatted from the original page layout.
`
`Thus, SoftSource pursued an approach in which the web page displayed on a
`mobile device would look the same as if it were being displayed on a desktop computer
`– just smaller. SoftSource believed that users would benefit from using a web page they
`were already familiar with, rather than a reformatted web page with information, such as
`images, stripped out.
`
`Here, we see an example of what SoftSource's invention taught. To the far left
`we can see the New York Times web site displayed on a desktop computer. In the
`center is a web page with the same page being displayed on a mobile device. Two things
`jump out. First, information such as photos have not been stripped out of the web page
`when it is displayed on the mobile device. Second, the web page displayed on the
`mobile device in the center has the same page layout as the web page displayed on the
`desktop computer, on the left.
`
`Motorola PX 1025_22
`
`
`
`
`
`
`
`
`Here, we have a video of what preserving the original page layout looked
`like on SoftSource's mobile browser, which was developed in 2001.
`
`
`
`Motorola PX 1025_23
`
`
`
`
`
`
`
`
`
`
`Second, SoftSource recognized that users should be able to zoom in and
`out on a web page. Zooming is vital on mobile devices where the reduced display
`makes it necessary to allow a user to zoom in on a section of the web page which
`is relevant to their inquiry. Zooming allows a viewer to read text and to see
`images that would otherwise be too small if the full web page were displayed on
`the small screen of the mobile device.
`
`Here, we have a video of what zooming in and out looks like on the
`SoftSource's Clearview mobile browser.
`
`Motorola PX 1025_24
`
`
`
`
`
`
`
`
`
`
`Third, SoftSource realized that the user should be able to pan the web
`page. In other words, once the user has zoomed in on a portion of the web page,
`the user should be able to move the web page up and down, as well as right and
`left, to view portions of the web page that are not currently being displayed.
`
`Here, we have a video of panning on SoftSource's ClearView
`mobile browser.
`
`
`
`
`Motorola PX 1025_25
`
`
`
`
`
`
`
`
`
`
`Fourth, SoftSource realized that users should be able to zoom in on a
`specific image, a paragraph, or a column of the web page simply by tapping on it.
`Context zooming avoids the need for a user to manually zoom in and pan to the
`relevant area of interest. Instead, tapping on an image zooms in on that image.
`Tapping on a column would zoom in on that column. And the same would be
`true of a paragraph.
`
`Here, we have a video showing context zooming on SoftSource's
`ClearView mobile browser. Notice that the image which is selected is zoomed
`through the user tapping on the image.
`
`Motorola PX 1025_26
`
`
`
`
`
`
`
`
`In this part of the technology tutorial, I will briefly describe the basic
`concepts behind the implementation of SoftSource's novel technology.
`
`
`
`
`
`Motorola PX 1025_27
`
`
`
`
`
`
`
`
`
`
`I will assume that the Court has some familiarity with the global system of
`interconnected computer networks that we call the Internet. A user's client
`computer or mobile device, shown in blue, can access the Internet to
`communicate with a web server, shown in orange. This connection can be over a
`wireless connection such as a cellular network or a WiFi network.
`
`
`Motorola PX 1025_28
`
`
`
`
`
`
`
`
`
`
`Let's discuss how web pages are requested and transferred to a user's
`mobile device.
`
`First, a user requests a web page through the browser on their mobile
`device. For example, a user opens a browser and requests the New York Times’s
`web page.
`
`Second, the mobile device sends a message through the cellular network
`that is then routed via the Internet to the web server. This is indicated with the
`red highlighted lines on the diagram.
`
`Motorola PX 1025_29
`
`
`
`
`
`
`
`
`
`
`Third, the web server responds to the request by providing the mobile
`device with the information needed to display the web page. This information is
`provided to the web browser on the mobile device.
`
`This information may include text, images, sounds, and hyperlinks to other
`web pages. This information may also include computer markup language that
`defines the appearance and layout of the text, images, and other information on
`the web page.
`
`Motorola PX 1025_30
`
`
`
`
`
`
`
`
`
`
`You will hear me refer to Hyper Text Markup Language or HTML, which
`is the most commonly used computer markup language for displaying web pages.
`You will also hear me refer to Cascading Style Sheet Format, another commonly
`used computer markup language for displaying web pages. HTML and Cascading
`Style Sheets are web page formats which describe the layout of a web page,
`including the content such as images, links, and text, and the location and
`relationship of that content. Each web page is comprised of the HTML or
`Cascading Style Sheet file and then all the images and other elements on the
`web page.
`
`
`Motorola PX 1025_31
`
`
`
`
`
`
`
`
`
`
`All web browsers contain what is called a rendering engine. A rendering
`engine is a part of the web browser program that determines how to lay out a web
`page, including the text, images, and other data contained on the web page.
`
`
`
`Motorola PX 1025_32
`
`
`
`
`
`
`
`
`
`
`The first thing that the disclosed rendering engine does is to parse or
`analyze the language that defines the appearance and layout of the web page. This
`enables the rendering engine to understand all of the elements in a web page and
`the relationship of those elements to one another. When an HTML or Cascading
`Style Sheet file is parsed by the rendering engine, the rendering engine creates a
`tree structure identifying the text, images, and other elements on a web page, and
`creating a hierarchy between them.
`
`
`
`Motorola PX 1025_33
`
`
`
`
`
`
`
`
`
`
`The next thing that the disclosed rendering engine does is translate layout
`information received from the server into content that can be displayed in a
`resolution independent manner. This means that the size of the screen won't
`matter when the content is displayed.
`
`Here we can see objects in the HTML or Cascading Style Sheet Format—
`such as the text element "Ancient Egyptian Life" and the image
`"Hieroglyphics"—are laid out on an X Y coordinate by the rendering engine in
`the web browser.
`
`It is important to realize that the web page information or content received
`by the client from the server is in HTML language or Cascading Style Sheet
`Format language. A web page is designed for display on desktop and notebook
`computer displays, not mobile devices.
`
`
`Motorola PX 1025_34
`
`
`
`
`
`
`
`
`
`
`The purpose of the translation step is to process HTML and Cascading
`Style Sheet data into resolution independent content that can be displayed on any
`size display. In other words, web pages that are designed to be displayed on a
`desktop or notebook computer display are translated so that they can be easily
`displayed on a much smaller mobile device screen. When a rendering engine lays
`out information on an X Y grid independent of screen resolution, the web page
`can be displayed not only on a desktop computer, but also on the reduced screen
`size of a mobile device display.
`
`The patents teach that the processed content takes the form of "scalable
`content" or "vector based content." The Court should be aware that the terms
`"scalable content" and "vector-based content" are disputed claim terms.
`However, it would be difficult to discuss the patents-in-suit without touching on
`these concepts, so I will endeavor to do so in a non-argumentative manner.
`
`Motorola PX 1025_35
`
`
`
`
`
` A
`
`
`
`
`
`
`Let's talk about scaling first. Like a lot of kids, I built scale models when I
`was younger.
`
` scale model is a physical copy of a real object, such as an airplane. In the
`case of an airplane scale model, the model is smaller than the actual object being
`represented. But a scale model could also be larger than the actual object being
`represented if the object is small. For example, a scale model of a human cell is
`much larger than an actual human cell.
`
`The idea of a scale model is to maintain the relative proportions of the
`physical size of the original object. This relative proportion is called the scale
`factor.
`
`Motorola PX 1025_36
`
`
`
`
`
`
`
`
`Here, we have an image of a box that contains a model airplane. You'll
`notice the number one-colon-seventy-two on the box. That means the model
`airplane is one-seventy-second the size of the actual, real life airplane. One-to-
`seventy-two is what is referred to as the scale factor.
`
`
`
`
`Motorola PX 1025_37
`
`
`
`
`
`
`
`
`
`
`What do I mean by scale factor? If you were to take a model airplane and
`increase its size by 72 times, which is the scale factor, the model should have the
`same dimensions as the actual airplane being represented.
`
`So imagine that the 1/72nd scale model airplane is a mere 10 inches long.
`This would mean that the airplane in real life would be 72 times the size of that,
`which is 60 feet long, or 720 inches.
`
`
`Motorola PX 1025_38
`
`
`
`
`
`
`
`
`Conversely, if you were to take the actual airplane and shrink it to 1/72nd
`of its size, the actual airplane should have the same dimensions as the model
`airplane. So, for example, if a real life airplane were 60 feet or 720 inches long,
`the model airplane would be 1/72nd of that or 10 inches long.
`
`
`
`
`Motorola PX 1025_39
`
`
`
`
`
`
`
`
`The same idea applies to scalable content for a web page. Again, "scalable
`content" is a disputed claim term. But the patents-in-suit teach converting the
`HTML and Cascading Style Sheet Format content into scalable content.
`
`
`
`Motorola PX 1025_40
`
`
`
`
`
`
`
`
`Using that scalable content, the web page can be quickly drawn at various
`sizes or zoom levels.
`
`
`
`Motorola PX 1025_41
`
`
`
`
`
`
`
`
`
`
`By way of example, let's imagine that this web page has been translated into
`scalable content. The browser can zoom in and zoom out the web page by
`multiplying the scalable content by a scale factor. A scale factor of 2 would
`correspond to a view of the web page that is twice the size. A scale factor of 4
`would correspond to a view of the web page that is four times the size. And a
`scale factor of ½ or .5 would correspond to a view of the web page that is half its
`size.
`
`Motorola PX 1025_42
`
`
`
`
`
`
`
`
`
`
`This takes us to vectors, which is a disputed claim term. A vector is a type
`of scalable content. In computer graphics, vectors are mathematical equations
`used to represent information, such as shapes. Vector information can take
`several forms such as images, technical drawings, and/or fonts. Vectors allow
`information to be scaled up or down.
`
`Here you can see two images. Although they appear at first glance to be
`the same, one is what is called a raster graphics image, which means that each
`pixel in the image is defined. When we scale the image up, the image becomes
`rough edged or pixelated. In contrast, the second image is a vector image where
`individual pixels are not defined. Instead, mathematical formulas define the data
`used to display and scale the image. As you can see, this image can be scaled
`without degradation.
`
`Motorola PX 1025_43
`
`
`
`
`
`
`
`
`
`
`I realized that layout information for a web page could be converted into
`vectors. By defining layout information as vectors, the layout of a web page can
`be preserved, and a user can easily zoom in and out of relevant sections. These
`vectors define how objects in a web page are laid out.
`
`Let's say you have an image in a web page. Instead of saying the image is
`going to be laid out 30 pixels down and 40 pixels to the right, image layout
`information could be described using vectors. Here we can see the vector based
`layout information indicated in blue. For a web page such as the New York
`Times that means that an image is laid out not based on the number of pixels the
`image should be, but instead, by its relationship to the X Y coordinates.
`
`Motorola PX 1025_44
`
`
`
`
`
`
`
`The math surrounding vectors can be quite complicated, so I won't delve into it
`too deeply in this tutorial. However, I would like to discuss the basic concept of a
`vector. You cannot touch or feel a vector. It is simply a mathematical expression. Like
`the equation 2+2. As we discussed above, vectors are used in a variety of contexts such
`as images, fonts and layout information.
`
`
`
` A
`
` A
`
` vector is a connection between two points in an X Y coordinate. In this case,
`we'll be talking about two-dimensional space because a web page on a computer display
`is drawn in two dimensions. Thus, we will use an X Y coordinate system.
`
`Then, we need to define an origin point for the vector. In this case, we will use
`(0,0). However, the decision to use an origin point of (0,0) is an arbitrary one. We
`could choose any origin point we desire, so long as we consistently use that origin point.
`
` vector represents both a length and a direction in two-dimensional space. In
`an x-y coordinate system, we can represent a vector by the mathematical expression (10,
`10). This expression signifies a length, which is the distance from the origin to the point
`(10,10). This expression also signifies a direction relative to the origin.
`
`Motorola PX 1025_45
`
`
`
`
`
`
`The vector can be scaled up by multiplying it by a scale factor. Here, we
`multiply the vector (10,10) by a scale factor of 2 to get (20,20). This would
`produce a vector that is twice as long as the original.
`
`
`
`Motorola PX 1025_46
`
`
`
`
`
`
`
`
`The vector can also be scaled down by multiplying it by a scale factor.
`Here, we multiply the vector (10,10) by a scale factor of ½ or .5 to get (5,5). This
`would produce a vector that is half the length of the original.
`
`Vectors can be manipulated in many other ways, but as I previously
`mentioned, a discussion of the math surrounding vectors is beyond the scope of
`this brief tutorial.
`
`Motorola PX 1025_47
`
`
`
`
`
`
`
`
`As I’ve explained, elements such as text and images can be laid out using
`vectors. Here, we can see a web page on the left as a user of a mobile device sees
`a web page displayed. On the right, we can see how the mobile device thinks
`about the same web page. Notice that a stock chart, shown in green, is viewed by
`the mobile device as being positioned based on vectors which define the layout
`information of the element. Here, the upper left corner of the green box is
`identified at 250C. The arrow from the origin to the point is the vector.
`
`This goes to the core of SoftSource's groundbreaking web browser, which
`displays web pages on mobile devices by breaking down a web page into elements
`and rendering the elements of a web page in a resolution independent space. This
`allows the web page to be displayed as originally intended and to be zoomed and
`panned by a user.
`
`Motorola PX 1025_48
`
`
`
`
`
`
`
` I
`
` hope that my introduction to mobile web browsers, as well as the key
`concepts disclosed in the '353 and '926 patents, has been useful to the Court.
`
`Thank you.
`
`Motorola PX 1025_49
`
`