IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`In re Patent of: W. Daniel Hillis, et al.
`U.S. Patent No.:
`8,665,239 Attorney Docket No.: 39521-0048IP1
`Issue Date:
`March 4, 2014
`
`Appl. Serial No.: 13/686,692
`
`Filing Date:
`November 27, 2012
`
`Title:
`METHOD AND APPARATUS CONTINUING ACTION OF
`USER GESTURES PERFORMED UPON A TOUCH SENSITIVE
`INTERACTIVE DISPLAY IN SIMULATION OF INERTIA
`
`DECLARATION OF DR. BRAD A. MYERS
`
`
`
`
`
`
`
`1
`
`APPLE 1003
`
`
`
`
`In re Patent of: W. Daniel Hillis, et al.
`U.S. Patent No.:
`8,665,239 Attorney Docket No.: 39521-0048IP1
`Issue Date:
`March 4, 2014
`
`Appl. Serial No.: 13/686,692
`
`Filing Date:
`November 27, 2012
`
`Title:
`METHOD AND APPARATUS CONTINUING ACTION OF
`USER GESTURES PERFORMED UPON A TOUCH SENSITIVE
`INTERACTIVE DISPLAY IN SIMULATION OF INERTIA
`
`DECLARATION OF DR. BRAD A. MYERS
`
`
`
`
`
`
`
`1
`
`APPLE 1003
`
`
`
`Table of Contents
`Introduction ..................................................................................................................... 4
`
`Brief Overview of the ’239 Patent ............................................................................. 8
`
`I.
`
`II.
`
`III. Legal Standards for Prior Art ...................................................................................... 9
`
`A.
`
`B.
`
`Legal Standards for Anticipation .......................................................10
`
`Legal Standards for Obviousness ......................................................11
`
`IV. U.S. Patent Application Publication No. 2003/0156145 to Hullender, et al.
`
`(“Hullender”) ......................................................................................................... 15
`
`V.
`
`P.E. Renaud, Introduction to Client/Server Systems: A Practical Guide for
`
`Systems Professionals (1996) (“Renaud”) ..................................................... 29
`
`VI. Combination of Hullender and Renaud .................................................................. 29
`
`VII. U.S. Patent No. 6,249,606 to Kiraly, et al. (“Kiraly”) ........................................ 34
`
`VIII. U.S. Patent No. 5,347,295 to Agulnick, et al. (“Agulnick”) ............................. 42
`
`IX. Combination of Kiraly and Agulnick ...................................................................... 49
`
`X.
`
`Conclusion ..................................................................................................................... 54
`
`
`
`
`
`
`
`2
`
`
`
`EXHIBITS
`
`APPLE-1001
`
`U.S. Patent No. 8,665,239 to Hillis, et al. (“the ’239 Patent”)
`
`APPLE-1002
`
`Excerpts from the Prosecution History of the ’239 Patent (“the
`Prosecution History”)
`
`APPLE-1003
`
`Declaration of Dr. Brad Myers
`
`APPLE-1004
`
`Curriculum Vitae of Dr. Brad Myers
`
`APPLE-1005
`
`U.S. Patent Application Publication No. 2003/0156145 to
`Hullender, et al. (“Hullender”)
`
`APPLE-1006
`
`P.E. Renaud, Introduction to Client/Server Systems: A Practical
`Guide for Systems Professionals (1996) (“Renaud”)
`
`APPLE-1007
`
`U.S. Patent No. 6,249,606 to Kiraly, et al. (“Kiraly”)
`
`APPLE-1008
`
`U.S. Patent No. 5,347,295 to Agulnick, et al. (“Agulnick”)
`
`APPLE-1011
`
`Dean Rubine, Specifying Gestures by Example, 25 Computer
`Graphics 329-337 (1991).
`
`
`
`3
`
`
`
`I.
`
`Introduction
`1. My name is Dr. Brad A. Myers. My curriculum vitae is provided as
`
`Exhibit 1004.
`
`2.
`
`I am presently Professor at the Human Computer Interaction Institute
`
`in the School of Computer Science at Carnegie Mellon University in Pittsburgh,
`
`Pennsylvania.
`
`3.
`
`I received a Master of Science in Computer Science and a Bachelor of
`
`Science in Computer Science and Engineering in 1980 from the Massachusetts
`
`Institute of Technology. I was awarded a Ph.D. in Computer Science from the
`
`University of Toronto in Toronto, Ontario, Canada in May 1987, where I was also
`
`a Teaching Assistant for various courses. From 1987 to 1992, I worked as a
`
`Research Computer Scientist at the School of Computer Science at Carnegie
`
`Mellon University. From 1992 to 1995, I was a Senior Research Computer
`
`Scientist at the School of Computer Science at Carnegie Mellon University. From
`
`1995 to 2003 I was a Senior Research Scientist in the Human Computer Interaction
`
`Institute in the Department of Computer Science at Carnegie Mellon University.
`
`From 2003 to 2004, I was an Associate Research Professor, and in 2004, became a
`
`full Professor in the Human Computer Interaction Institute in the Department of
`
`Computer Science at Carnegie Mellon University, which position I still hold.
`
`4
`
`
`
`4.
`
`I have been working in the area called “human-computer interaction”
`
`or HCI, also called “user interfaces,” for over 40 years. Over the course of my
`
`career, I have authored over 475 articles relating to, among other subjects, mobile
`
`computing, gestural interfaces, user interface software, window management,
`
`visualization, and novel interaction techniques. In recognition of my contributions
`
`to research, I was chosen to receive the ACM Special Interest Group on Computer-
`
`Human Interaction (“SIGCHI”) Lifetime Achievement Award in Research in 2017,
`
`for outstanding fundamental and influential research contributions to the study of
`
`human-computer interaction. I was selected as a Fellow of the Association for
`
`Computing Machinery (“ACM”) in 2005, a Fellow of the Institute of Electrical and
`
`Electronics Engineers (“IEEE”) in 2013, and elected to the “CHI Academy” by
`
`SIGCHI in April 2004, as one of the top 25 “principal leaders of the field” of HCI.
`
`I have also received a number of “best paper” awards with my students, and have
`
`been invited to give 22 keynote addresses at conferences.
`
`5.
`
`One focus of my research and teaching has been on various ways to
`
`interact with graphical objects on screens. For example, in 1990, I created a video
`
`called “All the Widgets” showing the history of interaction techniques such as
`
`scroll bars, buttons, text editing, and graphical object manipulation. I also taught a
`
`class on this topic in 2014 and 2016, and am currently writing a book on
`
`Interaction Techniques.
`
`5
`
`
`
`6.
`
`I have worked on gestures as an interaction technique since at least
`
`1991, when I was on the PhD dissertation committee for Dean Rubine. His
`
`dissertation included various ways to use gestures to manipulate graphics,
`
`including probably the first demonstration of two finger “pinch-to-zoom” on a
`
`screen. (Rubine. Ex. 1011 is an article he wrote based on his thesis.) My student
`
`James Landay and I used Dr. Rubine’s algorithms in a system we created called
`
`“Agate,” which supported using gestures to manipulate graphics, which we
`
`published in 1993 [James A. Landay and Brad A. Myers. "Extending an Existing
`
`User Interface Toolkit to Support Gesture Recognition," Adjunct Proceedings of
`
`INTERCHI'93. Amsterdam, The Netherlands, April 24-29, 1993. pp. 91-92].
`
`7.
`
`A later student and I explored a gestural technique for text entry on
`
`touch-sensitive interactive displays, which resulted in, among other publications, a
`
`patent issued June 1, 2010 [Jacob O. Wobbrock and Brad A. Myers. "Using Edges
`
`and Corners for Character Input." US Patent No. 7,729,542, Filed March 29, 2004,
`
`claiming priority to provisional filed April 4, 2003.]
`
`8.
`
`I have been retained on behalf of Apple Inc. to offer technical
`
`opinions relating to U.S. Patent No. 8,665,239 (“the ’239 Patent” or Ex. 1001), and
`
`prior art references relating to its subject matter. I have reviewed the ’239 Patent
`
`and relevant excerpts of the prosecution history of the ’239 Patent (Ex. 1002). I
`
`have also reviewed the following:
`
`6
`
`
`
`• U.S. Patent Application Publication No. 2003/0156145 to Hullender,
`
`et al. (Ex. 1005);
`
`• P.E. Renaud, Introduction to Client/Server Systems: A Practical
`
`Guide for Systems Professionals (1996) (Ex. 1006);
`
`• U.S. Patent No. 6,249,606 to Kiraly, et al. (Ex. 1007); and
`
`• U.S. Patent No. 5,347,295 to Agulnick, et al. (Ex. 1008).
`
`9.
`
`Counsel has informed me that I should consider these materials
`
`through the lens of a person of ordinary skill in the art related to the ’239 Patent at
`
`the time of the earliest purported priority date of the ’239 Patent, and I have done
`
`so during my review of these materials. I understand that the ’239 Patent claims
`
`priority to U.S. Patent Application Serial No. 13/458,915, which was filed April
`
`27, 2012; U.S. Patent Application Serial No. 12/862,564, which was filed on
`
`August 24, 2010; U.S. Patent Application Serial No. 11/188,186, which was filed
`
`on July 22, 2005; and U.S. Patent Application Serial No. 10/913,105, which was
`
`filed on August 6, 2004. It is therefore my understanding that the filing date of the
`
`earliest of these applications, i.e., August 6, 2004 (hereinafter the “Critical Date”),
`
`represents the earliest possible priority date to which the ’239 Patent is entitled.
`
`10. A person of ordinary skill in the art (hereinafter a POSITA) as of the
`
`Critical Date would have had a Bachelor of Science Degree in an academic area
`
`emphasizing electrical engineering, computer engineering, computer science, or an
`
`7
`
`
`
`equivalent field, and two or more years of experience in touch sensitive computer
`
`systems or gesture-based control of computer systems. Additional education in a
`
`relevant field or industry experience may compensate for a deficit in one or more
`
`of the aspects of these requirements.
`
`11.
`
`I am familiar with the knowledge and capabilities of a POSITA as
`
`noted above. Specifically, my experience working with industry, undergraduate
`
`and post-graduate students, colleagues from academia, and designers and engineers
`
`practicing in industry has allowed me to become directly and personally familiar
`
`with the level of skill of individuals and the general state of the art.
`
`12.
`
`I have no financial interest in either party or in the outcome of this
`
`proceeding. I am being compensated for my work as an expert on an hourly basis,
`
`for all tasks involved. My compensation is not dependent in any manner on the
`
`outcome of these proceedings or on the content of my opinions.
`
`13. My opinions, as explained below, are based on my education,
`
`experience, and background in the fields discussed above. Unless otherwise stated,
`
`my testimony below refers to the knowledge of a POSITA in the fields as of the
`
`Critical Date, which I understand to be August 6, 2004.
`
`
`
`II.
`
`Brief Overview of the ’239 Patent
`14. The ’239 Patent describes an approach to “identifying user gestures to
`
`8
`
`
`
`control an interactive display.” Ex. 1001 (’239 Patent), Abstract. “Each user
`
`gesture corresponds to at least one predetermined action for updating imagery
`
`presented by the display.” Ex. 1001 (’239 Patent), 2:23-25.
`
`A. Terminology
`
`15.
`
`I have been informed by petitioner’s counsel that claim terms should
`
`be given their plain meaning unless, for example, the intrinsic record, including the
`
`patent specification, or the knowledge of a person of ordinary skill in the art,
`
`requires otherwise. I have followed this approach in my analysis and for all of the
`
`claim terms considered in this Declaration, I have applied the plain and ordinary
`
`meaning of those terms unless noted otherwise below. I reserve the right to update
`
`my opinions should the parties agree on a construction of any claim terms, or
`
`should the Board adopt a particular construction.
`
`III.
`
`Legal Standards for Prior Art
`In view that I am not an attorney, my understanding of the legal
`16.
`
`standards throughout this section are based on discussion with petitioner’s counsel
`
`and experience in prior patent cases.
`
`17.
`
`I understand that a patent or other publication must first qualify as
`
`prior art before it can be used to invalidate a patent claim.
`
`18.
`
`I understand that a U.S. or foreign patent qualifies as prior art to an
`
`asserted patent if the date of issuance of the patent is prior to the invention of the
`
`9
`
`
`
`asserted patent. I further understand from petitioner's counsel that a printed
`
`publication, such as an article published in a journal, magazine or trade
`
`publication, qualifies as prior art to an asserted patent if the date of publication is
`
`prior to the invention of the asserted patent.
`
`19.
`
`I understand that a U.S. or foreign patent also qualifies as prior art to
`
`an asserted patent if the date of issuance of the patent is more than one year
`
`before the filing date of the asserted patent. I further understand that a printed
`
`publication, such as an article published in a journal, magazine or trade
`
`publication, qualifies as prior art to an asserted patent if the publication occurs
`
`more than one year before the filing date of the asserted patent.
`
`20.
`
`I understand that a U.S. patent qualifies as prior art to the asserted
`
`patent if the application for that patent was filed in the United States before the
`
`invention of the asserted patent.
`
`A. Legal Standards for Anticipation
`I understand that patents or printed publications that qualify as prior
`21.
`
`art can be used to invalidate a patent claim as anticipated or as obvious.
`
`22.
`
`I understand that, once the claims of a patent have been properly
`
`construed, the second step in determining anticipation of a patent claim requires a
`
`comparison of the properly construed claim language to the prior art on a
`
`limitation-by-limitation basis.
`
`10
`
`
`
`23.
`
`I understand that a prior art reference “anticipates” an asserted claim,
`
`and thus renders the claim invalid, if all limitations of the claim are disclosed in
`
`that prior art reference, either explicitly or inherently (i.e., necessarily present).
`
`24.
`
`I understand that anticipation in an inter partes review must be proven
`
`by a preponderance of the evidence.
`
`B.
`25.
`
`Legal Standards for Obviousness
`I understand that even if a patent is not anticipated, it is still invalid if
`
`the differences between the claimed subject matter and the prior art are such that
`
`the subject matter as a whole would have been obvious at the time the invention
`
`was made to a POSITA.
`
`26.
`
`I understand that a POSITA provides a reference point from which the
`
`prior art and claimed invention should be viewed. This reference point is applied
`
`instead of someone using his or her own insight or hindsight in deciding whether a
`
`claim is obvious.
`
`27.
`
`I also understand that an obviousness determination includes the
`
`consideration of various factors such as: (1) the scope and content of the prior art,
`
`(2) the differences between the prior art and the asserted claims, (3) the level of
`
`ordinary skill in the pertinent art, and (4) the existence of secondary considerations
`
`such as commercial success, long-felt but unresolved needs, failure of others, etc.
`
`28.
`
`I understand that an obviousness evaluation can be based on a
`
`11
`
`
`
`combination of multiple prior art references. I understand that the prior art
`
`references themselves may provide a suggestion, motivation, or reason to combine,
`
`but other times the linkage between two or more prior art references is common
`
`sense. I further understand that obviousness analysis recognizes that market
`
`demand, rather than scientific literature, often drives innovation, and that a
`
`motivation to combine references may be supplied by the direction of the
`
`marketplace.
`
`29.
`
`I understand that if a technique has been used to improve one device,
`
`and a POSITA would recognize that it would improve similar devices in the same
`
`way, using the technique is obvious unless its actual application is beyond his or
`
`her skill.
`
`30.
`
`I also understand that practical and common sense considerations
`
`should guide a proper obviousness analysis, because familiar items may have
`
`obvious uses beyond their primary purposes. I further understand that a POSITA
`
`looking to overcome a problem through invention will often be able to fit together
`
`the teachings of multiple publications. I understand that obviousness analysis
`
`therefore takes into account the inferences and creative steps that a POSITA would
`
`employ under the circumstances.
`
`31.
`
`I understand that a particular combination may be proven obvious
`
`merely by showing that it was obvious to try the combination. For example, when
`
`12
`
`
`
`there is a design need or market pressure to solve a problem and there are a finite
`
`number of identified, predictable solutions, a POSITA has good reason to pursue
`
`the known options within his or her technical grasp because the result is likely the
`
`product not of innovation but of ordinary skill and common sense.
`
`32. The combination of familiar elements according to known methods is
`
`likely to be obvious when it does no more than yield predictable results. When a
`
`work is available in one field of endeavor, design incentives and other market
`
`forces can prompt variations of it, either in the same field or a different one. If a
`
`POSITA can implement a predictable variation, the patent claim is likely obvious.
`
`33.
`
`It is further my understanding that a proper obviousness analysis
`
`focuses on what was known or obvious to a POSITA, not just the patentee.
`
`Accordingly, I understand that any need or problem known to those of ordinary
`
`skill in the field of endeavor at the time of invention and addressed by the patent
`
`can provide a reason for combining the elements in the manner claimed.
`
`34.
`
`I understand that a claim can be obvious in light of a single reference,
`
`without the need to combine references, if the elements of the claim that are not
`
`found explicitly or inherently in the reference can be supplied by the common
`
`sense of one of ordinary skill in the art.
`
`35.
`
`I understand that secondary indicia of non-obviousness may include
`
`(1) a long felt but unmet need in the prior art that was satisfied by the invention of
`
`13
`
`
`
`the patent; (2) commercial success of processes covered by the patent; (3)
`
`unexpected results achieved by the invention; (4) praise of the invention by others
`
`skilled in the art; (5) taking of licenses under the patent by others; (6) deliberate
`
`copying of the invention; (7) failure of others to find a solution to the long felt
`
`need; and (8) skepticism by experts. I understand that evidence of secondary
`
`indicia of non-obviousness, if available, should be considered as part of the
`
`obviousness analysis.
`
`36.
`
`I also understand that there must be a relationship between any such
`
`secondary considerations and the invention. I further understand that
`
`contemporaneous and independent invention by others is a secondary consideration
`
`supporting an obviousness determination.
`
`37.
`
`In sum, my understanding is that prior art teachings are properly
`
`combined where a POSITA having the understanding and knowledge reflected in
`
`the prior art and motivated by the general problem facing the inventor, would have
`
`been led to make the combination of elements recited in the claims. Under this
`
`analysis, the prior art references themselves, or any need or problem known in the
`
`field of endeavor at the time of the invention, can provide a reason for combining
`
`the elements of multiple prior art references in the claimed manner.
`
`38.
`
`I understand that obviousness in an inter partes review must be
`
`proven by a preponderance of the evidence.
`
`14
`
`
`
`IV.
`
`U.S. Patent Application Publication No. 2003/0156145 to Hullender,
`et al. (“Hullender”)
`39. Hullender describes a system in which a “user may use gestures to
`
`quickly command a pen-based input device to perform operations.” Ex. 1005
`
`(Hullender), Abstract. A gesture “may be made on a display surface,” for instance,
`
`using a stylus or the user’s own finger. Ex. 1005 (Hullender), [0043], [0037].
`
`Hullender’s system “recogniz[es] gestures made by a user” and “perform[s]
`
`functions related to the gestures.” Ex. 1005 (Hullender), [0002].
`
`System Implementation
`
`40. Hullender’s system is implemented in the context of “a stylus-based
`
`computer processing system (also referred to as a tablet PC).” Ex. 1005
`
`(Hullender), [0036]. The tablet PC includes “a large display surface 202” such that
`
`“a user can select, highlight, and write on the digitizing display area.” Ex. 1005
`
`(Hullender), [0036]. That is, the display surface 202 (shown in Fig. 2, reproduced
`
`below) is a touch-sensitive display surface.
`
`15
`
`
`
`Figure 2 of Ex. 1005 (Hullender)
`
`
`
`41. Hullender’s tablet PC also includes “a processing unit 110” and digital
`
`data storage, including “a hard disk drive 170 for reading from and writing to a
`
`hard disk (not shown), a magnetic disk drive 180 for reading from or writing to a
`
`removable magnetic disk 190, and an optical disk drive 191 for reading from or
`
`writing to a removable optical disk 192 such as a CD ROM or other optical
`
`media.” Ex. 1005 (Hullender), [0030], [0031]. As shown in Fig. 1, reproduced
`
`below, the processing unit and the data storage are coupled via a system bus 130.
`
`16
`
`
`
`Figure 1 of Ex. 1005 (Hullender)
`
`
`
`
`
`Overview of Hullender’s Gestures
`
`42. Hullender defines a gesture as a “drawing or other ink that may be
`
`interpreted as a command.” Ex. 1005 (Hullender), [0028]. Ink is a “sequence or set
`
`of one or more strokes,” where a stroke is a “sequence or set of captured points.”
`
`Ex. 1005 (Hullender), [0021], [0022]. For instance, a stroke can be the sequence or
`
`set of points on the display surface that are touched by a stylus or finger drawing a
`
`gesture on the display surface, starting from the point of first contact and ending at
`
`the point at which the stylus or finger is removed from contact. Consequently, the
`
`strokes of Hullender correspond to the claimed one or more touches experienced
`
`17
`
`
`
`by the display surface.
`
`43. Some gestures can be formed of a single stroke, such as the Circle 604
`
`shown in Fig. 6, reproduced below (see also the Table at [0088]). The single stroke
`
`of the Circle gesture is the set of points on the display surface that were touched as
`
`the user drew the circle. Some gestures can be formed of multiple strokes, such as
`
`the “Circle-Tap” 606, also shown in Fig. 6 and listed in the Table at [0088].
`
`“Circle-Tap” is a gesture composed of two strokes: the first stroke is the set of
`
`points on the display surface that were touched as the user drew the circle, and the
`
`second stroke is the set of points that were touched as the user made the tap in the
`
`center of the circle.
`
`
`
`Fig. 6 of Ex. 1005 (Hullender)
`
`
`
`44. Hullender identifies several properties of a stroke, including “stroke
`
`18
`
`
`
`windings, duration of the stroke, aspect ratio of the stroke, maximum distance of
`
`any point from a segment connecting endpoints, points of inflection.” Ex. 1005
`
`(Hullender), [0075]. These properties are, or make obvious, a magnitude of the
`
`stroke. For instance, the duration of a stroke is a magnitude, as is apparent from the
`
`algebraic expression for duration (tP-1 – t0) given in Rubine. Ex. 1011, p. 333.
`
`Similarly, Rubine computes stroke windings as “the total angle traversed (f9)”. Id.
`
`The duration of the stroke in Hullender is also a force history, i.e., the duration is
`
`based on past contact with the display surface.
`
`45. Hullender also identifies several properties of a gesture as a whole.
`
`The size of a gesture is one property identified by Hullender: “A gesture may have
`
`another characteristic based on the size of the gesture.” Ex. 1005 (Hullender),
`
`[0057]. The area of a gesture is another property identified by Hullender. See Ex.
`
`1005 (Hullender), [0052]. Hullender also identifies “the speed of writing the
`
`gesture, the pressure of the stylus on the surface of the digitizer, the angle of the
`
`stylus …, [and] the angle of the gesture” as properties of a gesture. Ex. 1005
`
`(Hullender), [0053]. Each of these properties (that is, the size, area, speed of
`
`writing, stylus pressure, stylus angle, and gesture angle) is a magnitude of the one
`
`or more touches of the gesture.
`
`46. For a gesture composed of a single stroke, a property of the gesture is
`
`also a property of the single stroke that forms the gesture. For instance, for the
`
`19
`
`
`
`Circle gesture, the size and area of the Circle gesture and the speed of writing the
`
`Circle gesture are also the size, area, and speed of writing the single, circle-shaped
`
`stroke that forms the Circle gesture. Thus, for a single-stroke gesture, the
`
`magnitude of the gesture (e.g., the size, area, speed of writing, stylus pressure and
`
`angle, or gesture angle) is also a magnitude of the single stroke that forms the
`
`gesture. For instance, the size (e.g., length) and area of a stroke are a current length
`
`and current area of the stroke, i.e., the touches, respectively, in that the stroke size
`
`and area do not take into account past sizes or areas. The stylus pressure of a stroke
`
`is a current force of the stroke, i.e., the touches, in that stylus pressure does not
`
`take into account past stylus pressures. The writing speed of a stroke is a length
`
`history of the stroke, i.e., touches, in that speed is dependent on past positions (i.e.,
`
`lengths) of the touches.
`
`Recognition of a Stroke or Strokes as a Gesture
`
`47. Hullender describes a process for “recognizing certain strokes as
`
`gestures.” Ex. 1005 (Hullender), [0062]. As shown in Fig. 4, reproduced below,
`
`“[i]n step 401, a stroke is received. In step 403, the system determines whether a
`
`gesture was received. This determination may include scaling, reorientation, and/or
`
`other operations to determine if a received stroke is a gesture.” Ex. 1005
`
`(Hullender), [0063]. Recognition of strokes as gestures can be based on properties
`
`of the strokes.
`
`20
`
`
`
`Fig. 4 of Ex. 1005 (Hullender)
`
`
`
`
`
`48. The shape of the strokes is a property that can be relied on for gesture
`
`recognition. For instance, as shown in Fig. 6 (above), the identity of a gesture can
`
`be based on the shape of the gesture; a chevron (gesture 603), a triangle (gesture
`
`615), and an arrow (gesture 621) are distinguished by their shapes. See, e.g., Ex.
`
`1005 (Hullender), [0060].
`
`49. The number of strokes in the gesture is another property that can be
`
`relied on for gesture recognition. “The system may also look at recent strokes from
`
`step 404 as some strokes may be both gestures when taken alone as well as in
`
`combination with other strokes. For example, a tap on a paragraph may act as a
`
`selection of a word under the tap. Two taps … may select a sentence or paragraph
`
`that contains the word under the original tap.” Id., [0063]. The number of strokes
`
`21
`
`
`
`can be considered a magnitude of the strokes. In particular, the number of strokes
`
`is an intensity history of the strokes, i.e., touches, because the number of strokes is
`
`determined by tracking the touches that make up the stroke over time.
`
`50. Other properties can also be used to recognize a stroke or strokes as a
`
`gesture, including stroke coordinates, time scale, windings, duration, aspect ratio,
`
`distance, or points of inflection. Hullender describes a process “for recognizing
`
`gestures” in which “a stroke is … received” and processed. Ex. 1005 (Hullender),
`
`[0071]. The processing includes “any one or more of the following: … 2. Scale the
`
`X and Y coordinate to a predetermined size … the scaling information may be
`
`retained for future use. 3. Scale the time entry of points … the time scale
`
`information may be retained for future use. … Additional items including stroke
`
`windings, duration of the stroke, aspect ratio of the stroke, maximum distance of
`
`any point from a segment connecting endpoints, [and] points of inflection … may
`
`be computed.” Ex. 1005 (Hullender), [0071]-[0075]. These computed properties
`
`are used to “[c]ompute Chebychev polynomials,” which are “input to one or more
`
`Bayes net.” Ex. 1005 (Hullender), [0076], [0077]. The stroke is then scored based
`
`on the Bayes net and “[a]ny net that exceeds a threshold is recognized” as a
`
`gesture. Ex. 1005 (Hullender), [0078]. That is, in this process, a stroke is
`
`recognized as a gesture based on the processing of one or more properties of the
`
`stroke, including stroke windings, duration, aspect ratio, distance, and points of
`
`22
`
`
`
`inflection, each of which is an example of a magnitude of the stroke. See Paragraph
`
`44, above.
`
`Identification and Execution of Actions
`
`51. Each gesture “may have one or more default actions associated with
`
`it.” Ex. 1005 (Hullender), [0046]. A default action, which “may be referred to as a
`
`control method or control property of a gesture,” is “what the gesture does or
`
`affects when executed.” Ex. 1005 (Hullender), [0047]. For instance, the execution
`
`of a Double Tap gesture invokes the action “Select Word” and the execution of a
`
`Right-flick gesture invokes the action “Scroll left until action area is at right edge
`
`of window.” See, e.g., the table at [0089] of Hullender. Many of the actions that
`
`can be invoked by execution of a gesture result in the modification of subject
`
`matter that is displayed on the display surface, including at least the “Select”
`
`actions, the “Scroll” actions, the actions that change rendering of displayed text
`
`(e.g., “Make Target Bold”), the “Delete Target” action, and the “Paste” actions.
`
`See, e.g., the tables at [0088] and [0089].
`
`52. Hullender describes that “[o]nce a gesture is created (and/or
`
`recognized) … it may be executed,” meaning that the default action associated
`
`with the gesture is carried out. Ex. 1005 (Hullender), [0041]. If it is decided that
`
`Hullender’s disclosure of executing an action associated with a gesture does not
`
`constitute an explicit disclosure of identifying that action, then it is my opinion that
`
`23
`
`
`
`a POSITA would still have found it obvious for the action to be identified so that
`
`the action could then be carried out.
`
`53. The execution of an action corresponding to a gesture can be affected
`
`by properties of the gesture (or by properties of the stroke that makes up the
`
`gesture, at least for a single-stroke gesture). Generally, Hullender describes that
`
`“[e]xecution of a gesture may depend on what an application is supposed to do
`
`with a gesture (given the size, shape, and position of the gesture).” Ex. 1005
`
`(Hullender), [0090]. Specific properties of a gesture or stroke that affect the
`
`execution of an action corresponding to a gesture include size, area, writing speed,
`
`stylus pressure, stylus angle, and gesture angle, which are types of magnitudes of a
`
`stroke. See Ex. 1005 (Hullender), [0052], [0053], [0057]; see also Paragraph 45,
`
`above.
`
`54. Gesture size (or stroke size, at least for a single-stroke gesture) can
`
`affect execution of the action associated with the gesture: gesture size “permits
`
`gestures to have varying impact or extent on what they are modifying.” Ex. 1005
`
`(Hullender), [0057]. For instance, Hullender’s Left-Bracket gesture corresponds to
`
`the action “Set left edge of selection.” Ex. 1005 (Hullender), Table at [0089]. The
`
`execution of the Left-Bracket gesture depends on the height (i.e., a size) of the
`
`bracket: “a left bracket may be one line tall or may be multiple lines tall. With a
`
`three line high left bracket, more area is intended to be selected than a two line
`
`24
`
`
`
`high left bracket.” Ex. 1005 (Hullender), [0057]. That is, the manner in which an
`
`action (e.g., the extent of the content selected) is carried out is impacted by the size
`
`of a gesture (or the size of a stroke), such as the height of the bracket gesture.
`
`55. Gesture area (or stroke area, at least for a single-stroke gesture) can
`
`affect execution of the action associated with the gesture. “The action area of a
`
`gesture … may be based on one or more attributes of the gesture. For instance, an
`
`X placed on a single word may have an action associated with the gesture to delete
`
`the word … Alternatively, a large X over a paragraph may expand the scope of the
`
`action area to encompass the entire paragraph.” Ex. 1005 (Hullender), [0052]. That
`
`is, the manner in which an action (e.g., the extent of the content deleted) is carried
`
`out is impacted by the area of a gesture (or the area of a stroke).
`
`56. Other gesture properties (or stroke properties, at least for a single-
`
`stroke gesture) can also affect execution of the action associated with the gesture.
`
`In particular, the “scaling of action areas may be adjusted by … the speed of
`
`writing the gesture, the pressure of the stylus on the surface of the digitizer, the
`
`angle of the stylus …, the angle of the gesture, and the like.” Ex. 1005 (Hullender),
`
`[0053]. The action area of a gesture is “the area that the gesture is to affect.” Ex.
`
`1005 (Hullender), [0049]. That is, the area that is affected by execution of an
`
`action can be impacted by the speed of writing a gesture or stroke, stylus pressure
`
`or angle when writing a gesture or stroke, or the angle of a gesture or stroke.
`
`25
`
`
`
`57. For propert
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
