IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Ezra Eddie Bakhash
`In re Patent of:
`9,696,868
`U.S. Patent No.:
`July 4, 2017
`Issue Date:
`Appl. Serial No.:—14/614,708
`Filing Date:
`February 5, 2015
`Title:
`SYSTEM AND METHOD FOR PROVIDING THREE-
`DIMENSIONAL GRAPHICAL USER INTERFACE
`
`Attorney Docket No.: 50095-0111 0I1PI
`
`DE.
`
`JCHS
`
`I declare that all statements made herein on my own knowledge are true and
`
`that all statements made on information and belief are believed to be true, and
`
`further, that these statements were made with the knowledge that willful false
`
`statements and the like so made are punishable under Section 1001 of Title 18 of
`
`the United States Code,
`
`B
`
`Henry Fuchs, Ph.D.
`
`Date:
`
`2oOrwme.(V (9
`
`1
`
`APPLE 1003
`
`1
`
`APPLE 1003
`
`

`

`Table of Contents
`INTRODUCTION ........................................................................................... 4 
`I. 
`QUALIFICATIONS ........................................................................................ 4 
`II. 
`III.  LEGAL STANDARDS ................................................................................. 10 
`A.  Person of Ordinary Skill in the Art ......................................................... 11 
`B.  Obviousness ............................................................................................ 11 
`C.  Claim Interpretation ................................................................................ 13 
`IV.  PERSON OF ORDINARY SKILL IN THE ART ........................................ 14 
`V.  MATERIALS CONSIDERED ...................................................................... 15 
`VI.  OVERVIEW OF CONCLUSIONS FORMED ............................................. 17 
`VII.  OVERVIEW OF THE ’868 PATENT .......................................................... 18 
`VIII.  CLAIM CONSTRUCTION .......................................................................... 23 
`IX.  OVERVIEW AND COMBINATIONS OF PRIOR ART REFERENCES .. 23 
`A.  Overview of Anthony ............................................................................. 23 
`B.  Overview of Hanggie .............................................................................. 28 
`C.  Combination of Anthony and Hanggie ................................................... 36 
`D.  Combination of Hanggie and Anthony ................................................... 42 
`E.  Overview of Matthews ............................................................................ 46 
`F.  Combination of Matthews with HAC ..................................................... 48 
`G.  Overview of Robertson ........................................................................... 49 
`H.  Combination of Robertson with HAC and Matthews ............................ 51 
`X.  MANNER IN WHICH THE PRIOR ART REFERENCES RENDER THE
`’868 CLAIMS UNPATENTABLE ............................................................... 53 
`A.  Claim 1 .................................................................................................... 53 
`B.  Claim 2 .................................................................................................... 78 
`C.  Claim 3 .................................................................................................... 79 
`D.  Claim 4 .................................................................................................... 80 
`E.  Claim 5 .................................................................................................... 85 
`F.  Claim 6 .................................................................................................... 87 
`G.  Claim 7 .................................................................................................... 91 
`H.  Claim 8 .................................................................................................... 93 
`I.  Claim 9 .................................................................................................... 97 
`J.  Claim 10 .................................................................................................. 98 
`K.  Claim 11 ................................................................................................105 
`L.  Claim 12 ................................................................................................106 
`M.  Claim 13 ................................................................................................107 
`
`2
`
`

`

`N.  Claim 14 ................................................................................................107 
`O.  Claim 15 ................................................................................................108 
`P.  Claim 16 ................................................................................................108 
`Q.  Claim 17 ................................................................................................108 
`R.  Claim 18 ................................................................................................109 
`S.  Claim 19 ................................................................................................109 
`T.  Claim 20 ................................................................................................112 
`XI.  CONCLUSION ............................................................................................112 
`
`
`
`
`
`
`3
`
`

`

`I.
`1.
`
`INTRODUCTION
`I have been retained as a technical expert by counsel on behalf of Apple Inc.
`
`(“Apple”/“Petitioner”). I understand that Apple is requesting that the Patent Trial
`
`and Appeal Board institute an inter partes review (“IPR”) proceeding with respect
`
`to U.S. Patent No. 9,696,868 (“the ’868 Patent”) (APPLE1001).
`
`2.
`
`I have been asked to provide my independent analysis of the ’868 Patent in
`
`light of certain prior art publications, and I have done so based on my personal
`
`knowledge and experience.
`
`3.
`
`I am not, and never have been, an employee of Apple. I received no
`
`compensation for this Declaration beyond my normal hourly compensation based
`
`on my time actually spent analyzing the ’868 Patent, the prior art publications cited
`
`below, and various issues related thereto. I have no financial interest in the
`
`outcome of this proceeding, and will not receive any added compensation based on
`
`the outcome of any IPR or other proceeding involving the ’868 Patent.
`
`
`
`II. QUALIFICATIONS
`4.
`In formulating my opinions, I have relied on my knowledge, training, and
`
`experience in the relevant field. My education and experience are described more
`
`fully in the attached curriculum vitae (Appendix A). For ease of reference, I have
`
`highlighted certain information below.
`
`4
`
`

`

`5.
`
`I am the Federico Gil Distinguished Professor of Computer Science and an
`
`Adjunct Professor of Biomedical Engineering at the University of North Carolina
`
`at Chapel Hill. I head the UNC Graphics and Virtual Reality research group,
`
`supervising research scientists, engineers, PhD, MS, and undergraduate students.
`
`Students I have advised have gone on to senior faculty positions at leading
`
`universities such as MIT, Stanford, and Georgia Tech, and leading research labs
`
`such as Google, Intel, Meta (fka Facebook), and Microsoft, as well as to smaller
`
`companies and startups around the world.
`
`6.
`
`I received a Bachelor of Arts degree in Information and Computer Science
`
`from the University of California at Santa Cruz in 1970, and a Ph.D. from the
`
`University of Utah in 1975.
`
`7.
`
`I have over 50 years of experience working in the field of computer
`
`graphics. I have worked in the Imaging Lab of Caltech’s NASA Jet Propulsion
`
`Laboratory, and have consulted for numerous organizations, including General
`
`Electric, the RAND Corporation, Mitsubishi Electric Research Laboratory, and
`
`Xerox Palo Alto Research Center. I have held visiting professorships at ETH
`
`Zurich and at TU Wien (the Vienna University of Technology).
`
`8. My research in computer graphics has been supported by, among others,
`
`Cisco, DARPA, Google, Intel, Meta (fka Facebook), Microsoft, ONR, NIH, NSF,
`
`NVIDIA, U.S. Air Force, and Xerox. I have published over 250 papers, including
`
`5
`
`

`

`some in the top journals, such as ACM SIGGRAPH, SIGGRAPH Asia, ACM
`
`Transactions on Graphics, and IEEE Computer Vision and Pattern Recognition
`
`(CVPR).
`
`9.
`
`I have served on the program and papers committees of some of the top
`
`conferences in the field, such as ACM SIGGRAPH, ACM SIGGRAPH Asia, and
`
`Eurographics. I was instrumental in starting the symposium series ACM Interactive
`
`3D Graphics and Games, organizing the first meeting in 1986. I helped with the
`
`founding of the ACM Transactions on Graphics, the top journal in computer
`
`graphics, by serving as the guest editor of its inaugural issue in 1982.
`
`10. A number of ideas from my publications have been widely adopted in
`
`commercial products. For example, my introduction of Binary Space Partitioning
`
`Trees (“BSP Trees”) in 1980 enabled real-time 3D games such as Doom and
`
`Quake to run on modest-sized PCs, and BSP Trees is now a standard rendering
`
`technique taught in major textbooks in computer graphics. The Pixel-Planes
`
`hardware graphics engines, which were developed in my UNC research group,
`
`pioneered several rendering techniques, for example, “tiled rendering”, a method
`
`now adopted world-wide in the Graphics Processing Units (GPUs) in computers of
`
`all sizes. The Pixel-Planes 5 project validated the tiled approach, a method of
`
`parallel rendering of an image in order of rectangular tiles, and introduced many of
`
`the techniques now standard for tiled renderers.
`
`6
`
`

`

`11.
`
`I have taught courses on computer graphics and related topics at UNC
`
`Chapel Hill, at University of Texas at Dallas, and at TU Wien. These courses often
`
`covered subjects such as stereoscopic displays and other 3D display technologies
`
`and applications. I am a named inventor on 20 patents and several pending patent
`
`applications, several of which involve stereoscopic display, autostereoscopic
`
`displays, and other 3D display technologies and applications. Examples of a few
`
`such patents are provided below.
`
`12. US Patent No. 9,361,727 (issued June 7, 2016) to inventors H. Fuchs, L.
`
`McMillan, and A. Nashel, titled “Methods, systems, and computer readable media
`
`for generating autostereo three-dimensional views of a scene for a plurality of
`
`viewpoints using a pseudo-random hole barrier,” describes a system for generating
`
`stereoscopic display for multiple users without the need for any special eyewear.
`
`13. US Patent No. 8,896,655 (issued November 25, 2014) to inventors J.
`
`Mauchly, M. Marathe, H. Fuchs, and J. Frahm, titled “System and method for
`
`providing depth adaptive video conferencing,” is about creating a combined image
`
`from panoramic image data of a conferencing room captured by a first camera and
`
`close-up image data of one or more conference participants captured through a
`
`second camera.
`
`14. US Patent No. 5,870,136 (issued February 9, 1999) to inventors H. Fuchs,
`
`M. Livingston, T. Bishop, and G. Welch, titled “Dynamic generation of
`
`7
`
`

`

`imperceptible structured light for tracking and acquisition of three dimensional
`
`scene geometry and surface characteristics in interactive three dimensional
`
`computer graphics applications,” solves registration and occlusion problems in
`
`augmented reality systems.
`
`15. U.S. Patent 4,607,255 (issued Aug. 19, 1986) to inventors H. Fuchs and S.
`
`Pizer, titled “Three dimensional display using a varifocal mirror,” teaches a
`
`method for creating a true 3-D display which presents stereoscopic views to
`
`multiple users simultaneously using a vibrating varifocal mirror, a point-plotting
`
`CRT, and a conventional 2D color video system.
`
`16.
`
`I have been honored with multiple awards, including the ACM SIGGRAPH
`
`Steven A. Coons Award, considered the highest award in the field of computer
`
`graphics. I am a member of the National Academy of Engineering, one of about
`
`20 computer graphics specialists who are members. I am also a fellow of the
`
`American Academy of Arts and Sciences, a fellow of the ACM and IEEE. I
`
`received an honorary doctorate from TU Wien, the Vienna University of
`
`Technology, in 2018, one of two conferred that year.
`
`17.
`
`I believe that I am qualified to opine as a technical expert in this proceeding.
`
`The ’868 Patent is about three-dimensional graphical user interfaces (“3D-GUIs”),
`
`and I have obtained ample knowledge and experience in this area over the years.
`
`Indeed, a portion of my work in the 1970s and 1980s focused on the computer
`
`8
`
`

`

`graphics hardware improvements necessary to transform consumer-level 3D-GUIs
`
`from a theoretical concept to an obtainable reality. Since that time, I’ve led
`
`extensive research and development efforts on 3D-GUIs, generally and with a
`
`particular focus on virtual reality and augmented reality applications. Like 3D
`
`environments portrayed on conventional 2D screens, such as the ’868 Patent
`
`describes, augmented and virtual reality 3D-GUIs leverage depth cues (e.g.,
`
`perspective, size, and occlusion) to create the illusion that virtual spaces and
`
`objects have real-world depth. Given this similarity, there is significant overlap
`
`between the various types of the 3D-GUIs, including from an underlying
`
`technology perspective and a design perspective.
`
`18.
`
`I therefore believe that I have a detailed understanding of the state of the art
`
`during the relevant period, as well as a sound basis for opining how persons of skill
`
`in the art at that time would understand the technical issues in this case.
`
`19. Additional details about my employment history, fields of expertise, and
`
`publications are included in my curriculum vitae (attached as Appendix A).
`
`20. Counsel (Fish & Richardson) has informed me that I should consider these
`
`materials through the lens of one of ordinary skill in the art related to the ’868
`
`Patent at the time of the earliest possible priority date of the ’868 Patent, and I have
`
`done so during my review of these materials. I have been informed by Counsel to
`
`use the filing date of the ’868 Patent, February 5, 2015, as the “Priority Date” in
`
`9
`
`

`

`my analysis below. My analysis would not change even if Patent Owner asserted
`
`an earlier Priority Date such as September 13, 2005, the filing date of US
`
`Provisional Application Serial No. 60/717,019, from which the ’868 Patent claims
`
`priority from.
`
`21.
`
`In writing this declaration, I have considered the following: my own
`
`knowledge and experience, including my work experience in the fields of
`
`computer science and computer graphics and imaging; my experience in teaching
`
`related subjects; and my experience in working with others involved in those
`
`fields. In addition, I have analyzed various publications and materials, in addition
`
`to other materials I cite in my declaration.
`
`22. My opinions, as explained below, are based on my education, experience,
`
`and expertise in the fields relating to the ’868 Patent. Unless otherwise stated, my
`
`testimony below refers to the knowledge of one of ordinary skill in the art as of the
`
`earliest possible priority date. Any figures that appear within this document have
`
`been prepared with the assistance of Counsel and reflect my understanding of the
`
`’868 Patent and the prior art discussed below.
`
`
`
`III. LEGAL STANDARDS
`23.
`In conducting my analysis and providing my statements in this Declaration, I
`
`have applied the legal principles described to me by counsel for the Petitioner.
`
`10
`
`

`

`A. Person of Ordinary Skill in the Art
`I understand that the factors considered in determining the ordinary level of
`
`24.
`
`skill in a field of art include: the level of education and experience of persons
`
`working in the field; the types of problems encountered in the field; the teachings
`
`of the prior art regarding solutions to such problems; and the sophistication of the
`
`technology at the time of the alleged invention. I understand that a person of
`
`ordinary skill in the art (“POSITA”) is not a specific real individual, but rather is a
`
`hypothetical individual having the qualities reflected by the factors above and
`
`knowledge of all relevant prior art references, including the references I cite below.
`
`B. Obviousness
`I have been informed that a patent claim is invalid as “obvious” in light of
`
`25.
`
`one or more prior art references if it would have been obvious to a POSITA at the
`
`time of the alleged invention, taking into account (1) the scope and content of the
`
`prior art, (2) the differences between the prior art and the claims, (3) the level of
`
`ordinary skill in the art, and (4) any so called “secondary considerations” of non-
`
`obviousness, which include: (i) “long-felt but unresolved need” for the claimed
`
`invention, (ii) commercial success attributable to the claimed invention, (iii)
`
`unexpected results of the claimed invention, and (iv) “copying” of the claimed
`
`invention by others.
`
`11
`
`

`

`26. While I do not know the exact date that the alleged invention of the ’868
`
`Patent was made, I do know that the ’868 Patent’s earliest claimed priority date is
`
`September 13, 2005. For purposes of my obviousness analysis, I have applied a
`
`date of September 13, 2005 as the date of the alleged invention (“Critical Date”),
`
`although in many cases the same analysis would hold true even if the date of the
`
`alleged invention were earlier or later.1
`
`27.
`
`I have been informed that a claim can be obvious in light of a single prior art
`
`reference or multiple prior art references. To be obvious in light of a single prior
`
`art reference or multiple prior art references, there must be a reason that would
`
`have prompted a POSITA to modify or supplement the single prior art reference,
`
`or to combine two or more references, in a manner that provides the elements of
`
`the claimed invention. This reason may come from a teaching, suggestion, or
`
`motivation to combine, or may come from the reference(s) themselves, the
`
`knowledge or “common sense” of a POSITA, or from the nature of the problem to
`
`be solved, and this reason may be explicit or implicit from the prior art as a whole.
`
`I have been informed that, under the law, the combination of familiar elements
`
`according to known methods is likely to be obvious when it does no more than
`
`
`
`1 Note, however, that I do not concede that the ’868 Patent is entitled to its earliest
`
`claimed priority date.
`
`12
`
`

`

`yield predictable results. I also understand it is improper to rely on hindsight in
`
`making the obviousness determination.
`
`28.
`
`I understand that an obviousness determination also requires that a POSITA
`
`would have had a reasonable expectation of success. This concept has been
`
`explained to me as relating to the relative likelihood or predictability—from the
`
`perspective of a POSITA—of successfully modifying the prior art in a manner that
`
`would meet the claimed limitations of the patent being challenged (here the ’868
`
`Patent). I understand that the expectation of success need only be “reasonable”
`
`and, thus, does not require the absolute certainty gleaned from physically creating
`
`the proposed prior art modification.
`
`C. Claim Interpretation
`I understand that, for purposes of my analysis in this inter partes review
`
`29.
`
`proceeding, the terms appearing in the patent claims should be interpreted
`
`according to their “ordinary and customary meaning.” In determining the ordinary
`
`and customary meaning, I understand that the words of a claim are first given their
`
`plain meaning that those words would have had to a person of ordinary skill in the
`
`art at the time of the alleged invention. I also understand that the structure of the
`
`claims, the specification and file history may be used to better construe a claim.
`
`Moreover, I understand that even treatises and dictionaries may be used, albeit
`
`13
`
`

`

`under limited circumstances, to determine the meaning attributed by a person of
`
`ordinary skill in the art to a claim term.
`
`30.
`
`I have been informed by counsel for Petitioner that the parties proposed
`
`constructions for certain claim terms in the ’868 Patent in the co-pending litigation.
`
`I am not involved in the co-pending litigation and do not have direct knowledge of
`
`the events in that proceeding. Nor do I provide an opinion on claim construction in
`
`this proceeding.
`
`
`
`IV. PERSON OF ORDINARY SKILL IN THE ART
`31. Based on my knowledge and experience in the field and my review of the
`
`’868 Patent and its file history, I believe that a POSITA would have included a
`
`Bachelor’s of Science degree in computer science or a comparable field and at
`
`least two years of professional experience working with 2D and 3D graphical user
`
`interfaces. Such experience could be obtained through research and study in a
`
`graduate program or through comparable exposure through industry employment,
`
`and additional years of experience could substitute for the advanced-level degree.
`
`32. Accordingly, my analysis and conclusions expressed in this Declaration are
`
`based on the perspective of a POSITA having this level of knowledge and skill.
`
`My education, technical expertise and personal knowledge discussed in Section II
`
`shows that I meet the qualifications of a POSITA.
`
`14
`
`

`

`V. MATERIALS CONSIDERED
`33. My analysis in this Declaration is based on my knowledge and experience.
`
`Based on my above-described qualifications in Section II, the Patent Trial and
`
`Appeal Board should consider me to be an expert in the field. Also, based on my
`
`experiences, I understand and know of the capabilities of persons of ordinary skill
`
`in this field.
`
`34. As part of my independent analysis for this Declaration, I have considered
`
`the following: the ’868 Patent (APPLE1001) and its prosecution history
`
`(APPLE1002); the background knowledge/technologies that were commonly
`
`known to persons of ordinary skill; my own knowledge and experience gained
`
`from my work in the field; my experience in teaching and advising others in this
`
`field; and my experience working with others involved in this field. In addition, I
`
`have analyzed the following publications and materials:
`
`APPLE1005
`
`Apple Opening Claim Construction Brief, SpaceTime3D, Inc. v
`
`Apple Inc., 6-22-cv-00149, (W.D. Tex.), Sept. 1, 2022
`
`APPLE1006
`
`US Patent Pub. No. 2005/0088447 (“Hanggie”)
`
`APPLE1007 US Patent Pub. No. 2005/0091596 (“Anthony”)
`
`APPLE1008 US Patent Pub. No. 2006/0107229 (“Matthews”)
`
`APPLE1009
`
`US Patent No. 6,414,677 (“Robertson”)
`
`15
`
`

`

`APPLE1010
`
`Dictionary of Electrical & Computer Engineering, McGraw-Hill,
`
`2004.
`
`APPLE1011
`
`US Patent Pub. No. 2012/0053926 (“Satpute”)
`
`APPLE1012
`
`US Patent No. 5,664,896 (“Blumberg”)
`
`APPLE1013
`
`US Patent No. 7,536,676 (“Baker”)
`
`
`
`35. Note that my citations to non-patent literature throughout this Declaration
`
`reference the absolute page number added to the exhibit (as opposed to the original
`
`pagination of the document). For patent literature, I’ve used the column/line
`
`numbers or paragraph numbers. Note also that, unless I’ve indicated otherwise, all
`
`emphasis (bold/italics/underline) in any quoted text are ones I have added.
`
`36. Although I cite and quote to selected portions of various references in this
`
`Declaration, the reader should understand that these citations are representative
`
`examples. A POSITA would have viewed each reference in its entirety and in
`
`combination with other references. Accordingly, I intend the references identified
`
`in this Declaration to be viewed as incorporated in their entireties.
`
`
`
`
`
`
`
`16
`
`

`

`VI. OVERVIEW OF CONCLUSIONS FORMED
`37. This declaration explains my review of the ’868 Patent, materials
`
`considered, and the conclusions that I have formed based on my knowledge and
`
`experience. Based on my review, I have concluded that:
`
` Claims 1-5, 9-14, and 18-20 are obvious over Anthony in combination
`
`with Hanggie;
`
` Claims 6-8 and 15-17 are obvious over Anthony in combination with
`
`Hanggie and Matthew;
`
` Claim 7 is obvious over Anthony in combination with Hanggie,
`
`Matthews, and Robertson;
`
` Claims 1-5, 9-14, and 18-20 are obvious over Hanggie in combination
`
`with Anthony;
`
` Claims 6-8 and 15-17 are obvious over Hanggie in combination with
`
`Anthony and Matthews; and
`
` Claim 7 is obvious over Hanggie in combination with Anthony,
`
`Matthews, and Robertson.
`
`38.
`
`In support of these conclusions, I provide an overview of the references in
`
`Section IX and more detailed comments regarding the obviousness of claims 1-20
`
`(“the Challenged Claims”) of the ’868 Patent in Section X.
`
`
`
`17
`
`

`

`VII. OVERVIEW OF THE ’868 PATENT
`39. The ’868 Patent is directed to a three-dimensional (3D) interactive
`
`computing interface “that allows the user to efficiently navigate though [sic] a
`
`virtual space wherein groups of windows can be easily organized, stored, and
`
`retrieved.” APPLE1001, 1:30-39, 2:41-50. The ’868 Patent’s graphical user
`
`interface (GUI) “uses the two-dimensional (2D) display of a user's computer to
`
`display three-dimensional (3D) objects in a simulated real-time 3D immersive
`
`Cartesian space.” APPLE1001, 2:57-62. “Because the 3D GUI creates the illusion
`
`of infinite space in 3D, it can create a visual history of the user's computing
`
`session, whereby the user can visit past visual computing events (or a snapshot in
`
`time) by simply navigating to previously recorded states or viewpoints.”
`
`APPLE1001, 5:17-32.
`
`40.
`
`In particular, the 3D GUI program “simulates a 3-D space within a 2-D
`
`window by redrawing objects in the space relative to one another as determined by
`
`their perceived distance from the viewer. Objects that are supposed to be further
`
`away are smaller whereas objects that are supposed to be closer are larger.”
`
`APPLE1001, 8:32-38. “The program runs within web browsers (e.g., Internet
`
`Explorer and Mozilla Firefox) or as a stand-alone application compatible with the
`
`local operating system.” APPLE1001, 8:14-18.
`
`18
`
`

`

`41. Examples of the ’868 Patent’s GUI are shown in the following figures. FIG.
`
`9 (reproduced below) depicts “a bird's eye view of four 3D stacks 302, 304, 306,
`
`308 drawn in a virtual space 300.” APPLE1001, 18:4-27. 3D icons (342, 344,
`
`346, 348) represent four different searches that resulted in the creation of stacks
`
`302, 304, 306, 308. APPLE1001, 29:38-50. Navigator 320 allows a user to
`
`navigate through the items (e.g., pages) in each stack. APPLE1001, 29:51-30:7.
`
`APPLE1001 (’868 Patent), FIG. 9.2
`
`
`
`
`
`2 Annotations to the figures throughout this Petition are shown in color.
`
`19
`
`

`

`In another embodiment, shown in FIG. 15, content output into the
`
`3D GUI application’s virtual space results from a search initiated
`
`with the helper application 522 (e.g., Yahoo! Image Search). This
`
`helper application shows the search term van gogh, having been
`
`input into this helper application’s text input field 530. The resulting
`
`first four image outputs (552, 554, 556, 558), generated by helper
`
`application 522 …, created the 3D output of images and information
`
`from Yahoo! webservice, as shown in the 3D virtual space.
`
`APPLE1001, 33:37-58.
`
`
`
`APPLE1001 (’868 Patent), FIG. 15.
`
`20
`
`

`

`42.
`
`“[E]ach new Yahoo! Image Search results in a new 3D stack created for that
`
`search result plotted in the 3D virtual space with its corresponding 3D icon drawn
`
`on the timeline. The search for information could be images, video or any other
`
`content available from a webservice.” APPLE1001, 33:58-34:4. For example, as
`
`shown in FIG. 16A, when a second search is conducted, a second stack 570 is
`
`created and a second icon 572 representing stack 570 is added to the timeline to the
`
`right of a previous icon in the timeline. APPLE1001, 13:4-28. In some cases, the
`
`timeline “includes an icon for each object presented within the virtual space,
`
`wherein the icons are organized in linear chronological order according to when
`
`the objects were presented within the virtual space.” APPLE1001, 3:11-24.
`
`21
`
`

`

`
`
`APPLE1001 (’868 Patent), FIG. 16A.
`
`43. A user of the 3D GUI “can toggle or switch between 2D and 3D for any
`
`selectively captured computing output and information (webpages, applications,
`
`documents, desktops or anything that can be visualized on a computer) that was
`
`drawn within a 3D virtual space at will by” “clicking an icon or bottom (analogous
`
`to minimize in windows operating system).” APPLE1001, 22:7-30.
`
`
`
`
`
`
`
`22
`
`

`

`VIII. CLAIM CONSTRUCTION
`44.
`I was informed by counsel that in pending district court proceedings, the
`
`construction of two terms has been agreed upon by the Patent Owner and the
`
`Petitioner.
`
` 3D space (claims 1, 10, 29): a virtual space defined by a three-dimensional
`
`coordinate system; and
`
` 2D space (claims 1, 10, 29): a finite graphical area defined by a two-
`
`dimensional coordinate system.
`
`APPLE1005, 6.
`
`45.
`
`It is my understanding that no other terms are at issue. I have applied the
`
`above-noted constructions for purposes of my analysis.
`
`
`
`IX. OVERVIEW AND COMBINATIONS OF PRIOR ART REFERENCES
`A. Overview of Anthony3
`46. Anthony discloses “[a] three-dimensional (3D) view of a data collection
`
`based on an attribute.” APPLE1007, Abstract. As shown below in Anthony’s
`
`
`
`3 General descriptions provided for this and other references and combinations
`
`thereof are incorporated into each subsection and mapping of the claims that
`
`includes citations to these references.
`
`23
`
`

`

`FIG. 4, the 3D view can take the form of a timeline, which when displayed, depicts
`
`a plurality of items in focal groups according to a logical order. APPLE1007,
`
`Abstract, ¶¶[0042], [0043], [0011]. The items can be “a visual representation of a
`
`file, folder, virtual folder, or any other data object that may be stored in an
`
`operating system and/or file system, for example, icons, thumbnails, and the like.”
`
`APPLE1007, ¶[0042]. The order of the items “may be based on time, thus
`
`arranging the items chronologically, e.g., by using a date of creation or date of edit
`
`attribute as the ordering attribute.” APPLE1007, ¶[0042].
`
`APPLE1007 (Anthony), FIG. 4.
`
`
`
`47. Anthony’s method is implemented in a computing device that includes
`
`computer 110 connected to a monitor 191. APPLE1007, FIG. 1, ¶¶[0034]-[0039].
`
`Monitor 191 has a screen with a fixed resolution, and is used to display the
`
`24
`
`

`

`timeline. APPLE1007, Abstract, ¶[0050]. Examples of resolutions include a 96
`
`dpi screen resolution. APPLE1007, ¶[0050]. As shown below in FIG. 1,
`
`Anthony’s computer system includes “application programs 135[/145], other
`
`program modules 136[/146], and program data 137[/147].” APPLE1007,
`
`¶¶[0036], [0038]. In the context of viewing images, Anthony’s computer system
`
`can display images using various application programs, such as Digital Image Pro,
`
`preinstalled or installed thereafter on the operating system. APPLE1007, ¶[0003].
`
`Anthony also gives an example of excel spreadsheets, indicating that Anthony’s
`
`computer system can also execute the Microsoft Excel application. APPLE1007,
`
`¶[0049] (“For example, if the ordering attribute were file type, the focal group
`
`header 602a might contain a label that simply says, ‘Excel Spreadsheets’ or some
`
`other file type as is known in the art”).
`
`25
`
`

`

`APPLE1007 (Anthony), FIG. 1.
`
`
`
`48. Anthony’s dynamic timeline can be viewed in a 3D environment, as shown
`
`above in FIG. 4. APPLE1007, ¶[0043]. A user viewing the timeline can “freely
`
`navigate documents, files, or other data objects in a chronological manner, and”
`
`can “change the point of focus to an arbitrary location on a timeline.”
`
`APPLE1007, ¶[0044]. For example, as shown below in FIG. 5:
`
`[a] timeline 500 has a start date 502 and an end date 504. The timeline
`
`500 may include various time intervals (e.g., days, months, years, etc.,
`
`not called out in FIG. 5). According to an aspect of the invention, the
`
`point of view of the GUI may be based on a conceptual camera 506
`
`26
`
`

`

`pointed at the time interval containing the files or objects on which the
`
`user wishes to focus. This time range is called the focal point 508 or
`
`focal point interval. The focal point 508 may represent a point in time
`
`or an interval of time on the timeline 500. Items t