0 PI C
`
`·FICI DI LA PROPRlhf.
`
`!CTUILLI DU CANADA
`
`\ \
`
`Cl PO
`CulADLUI INTIUICT\JAL
`Paouan o.,,c,
`
`<•2><•9><CA> Demande-Application
`
`(2l)(AI) 2,237,939
`(22) 1998/06/29
`(43)1998/08/28
`
`(72) MANN, Steve,
`CA
`(71)MANN, Steve,
`CA
`6 H04N 7/18, H04N 5/77, G06F 3/()Z 003B 29/00
`(SI) Int.CI.
`(S4) SYSTEME D'IMAGERIE
`PERSONNEL A VEC VISIONNEUSE
`ET
`DISPOSITIF
`DE PRISE DE NOTES
`(54)PERSONAL IMAGING SYSTEM WITH VIEWFINDER AND
`ANNOTATION
`MEANS
`
`110
`
`120
`
`140
`
`___________
`---------------------
`
`1'10
`
`/
`___ } ______
`_
`
`IIATIEllY
`PACK
`
`t-----t
`
`C0M.'lt.
`
`SYSTEM
`
`(.57) L 'invention
`est constitute
`par une nouvelle
`m�
`et un nouvel appareil
`de photographic et de videograpbie
`de documents personnels.
`Elle porte particulieremcnt sur
`une camera personnelle avec visionneusc et sur un
`systeme video personnel de prise de notes. Cette camm
`pennet de tenir un journal personnel manuscrit 01: un
`
`(S7J A novel means and apparatus for personal
`documcnlary
`photography and videographyis
`described.
`In pmicu lar. a personal camera with viewfinder
`means
`and a persona) video annotation system is introduced.
`1be camera system integrates
`the process of making a
`personal
`handwritten
`diary or the like, with the c:--'."'ture of
`
`IPR2021-00923
`Apple EX1004 Page 1
`
`

`

`0 PI C
`
`CI PO
`
`0PPICI DI IA PROPRII.Tf
`
`
`
`'liLLICTU!LLI DU UNADA
`
`\
`\
`
`UNADIAN INTILLICTUAL
`
`Paonan o,.,c,
`
`(2)) (AJ) 2,237,939
`1998/06/29
`(22)
`1998/08/28
`(43)
`
`video, from an optimal point of vantage and camera
`
`document similaire avec saisie video a optimisation de
`
`
`
`angle. This gives rise to a new form of personal diary, as
`
`
`
`l'angle de prise de vues. Ceci donne une nouvelle forme
`
`de journal personnel, ainsi qu'un nouveau type
`
`well as a new genre of documentary video. Video of a
`
`
`
`d'enregistrement video de documents. Un client d'un
`
`subject such ac; an official behind a counter may be
`
`
`etablissement peut effectuer un enregistrement vi�
`
`captured by a customer or patron of an establishment, in
`
`
`
`
`d'un prepose instaJJe derriere un comptoir de telle �
`
`
`such a manner that the official cannot readily determine
`
`
`
`
`que ce prepose ne puisse facilement determiner si un
`whether or not video is being captured with the
`
`enregistrement video est effectue en mbne temps que
`handwritten notes or annotations.
`
`l'enregistrement des notes manuscrites .
`
`....
`
`IPR2021-00923
`Apple EX1004 Page 2
`
`

`

`ABSTRACT: PERSONAL IMAGING SYSTEM WITH
`
`VIEWFINDER AND ANNOTATION MEANS
`
`
`
`
`
`A novel means and apparatus for persona! documentary photography and videog­
`
`
`
`
`
`raphy is described. In particular, a personal camera with viewfinder means a.nd a
`
`
`
`
`
`
`
`
`
`personal v1deo annotation system is int.roduced. The camera system integrates the
`
`
`
`
`
`process
`
`
`
`
`
`
`of making a personal handwritten diary or the like, with the capture of video,
`
`
`
`from an optimal point of \'antage and camer-c1 angle. This gives rise to a new form
`
`
`
`
`
`of personal diary, as well as a new genre of documentary video. Video of a subjed
`
`
`
`
`
`
`
`
`
`such as an official behind a counter may be captured by a customer or patron of an
`
`
`
`
`
`
`
`
`
`establishment, in such a manner tha.t the official cannot readily determine whether
`
`
`
`
`
`or not video is being captured with the handwritten notes or annotations.
`
`
`
`IPR2021-00923
`Apple EX1004 Page 3
`
`

`

`Patent Application
`
`of
`
`Steve �fann
`
`for
`
`PERSONAL IMAGING SYSTEM WITH VIEWFINDER AND
`
`ANNOTATION MEANS
`
`
`
`of which the following is a specification:
`
`FIELD OF THE INVENTION
`
`
`
`The present invention pertains generally to a new photographic or video means and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`apparatus typically cor.1prising a hand-held portable electronic camera sy"Jtem with
`
`
`
`
`
`
`
`
`
`viewfinder mean,; and electronic pen-based annotation means.
`
`BACKGROUND OF THE INVENTION
`
`
`
`In photography (and in movie and video production). it is often desirable to cap­
`
`
`
`
`
`
`
`
`
`ture events in a natural manner with minimal intervention or disturbance. In the
`
`
`
`present invention, the scenario to be considered is that of face-to-face conversation
`
`
`
`
`
`
`
`
`
`between two individuals, where one of the individuals wish� to make a.n annotated
`
`
`
`
`
`video diary of the conversation without disrupting the natural Oow of the conver­
`
`
`
`
`
`o;ation. In this context. it is desirabl� to create a personal video diary or personal
`
`
`
`
`
`documentary, or to have some kind of personal photographic or videographic memory
`
`
`
`
`
`
`
`
`
`aid that forms the visual cquiva.lent of what the electronir organizers and personal
`
`
`
`
`
`digital a.qgistants do to help us remember textual or syntactic information.
`
`
`
`
`
`
`
`Current state-of-the-ut photographic or video apparatus er • ..;o ... visual dis-
`
`
`
`
`
`turbance to others and a!trads considerable attention vn ar JUDt of the gesture of
`
`
`
`
`
`bringiug th� ca.mt>ra up to the eye. Even if the size of the � �ra could be reduced
`
`
`
`to the poiut of being negligible (e.g. no bigger than th,. , · ,-, of a typical cam-
`
`
`
`era viewfinder, for example), the very gestt;re of hringirg a device up to the eye is
`
`
`
`
`
`IPR2021-00923
`Apple EX1004 Page 4
`
`

`

`
`
`-·· .., __ ..,, .,..,., • .,.,u "'"' • .,
`
`
`
`
`
`
`
`
`
`
`
`unnatural and attracts considerable attention, especially in establishments such as
`
`
`
`
`
`
`
`department stores where photography is often prohibited. However it is in these very
`
`
`
`
`
`
`
`
`
`
`
`establishments in which a customer may wish, for example, tu have a video record of
`
`
`
`
`
`
`
`
`
`the clerk's statement of the refund policy or the term.ci of a sale. Just as department
`
`
`
`
`
`
`
`stores often keep a video recording
`
`
`of all transactions (and often even a video record­
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing of all activity within the establishment, sometimes including a video recording of
`
`
`
`
`
`
`
`
`
`
`
`customers in the fitting rooms), the goal of the present invention is to assist, for ex­
`
`
`
`ample, a customer who may wish to keep a video record of a transaction, interaction
`
`
`
`
`
`
`
`
`
`
`
`with a clerk. manager, refund cY.p)anatioo, or the like.
`
`
`Although
`
`
`
`beneath there exist a variety of covert cameras such a camera concealed
`
`
`
`
`
`the jewel of a necktie clip, cameras concealed in baseball caps, and cameras concealed
`
`
`
`
`
`
`
`
`
`
`
`in eyeglasses, these c3Illeras 1eud to produce inferior images, not just because of the
`
`
`
`
`
`
`
`technical limitations imposed by their small si�e, but, more importantly be•_:\use they
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`lack a viewfinder means (e.g. means of viewing the image to adjust ...;,mera a.ngle for
`
`
`
`best composition). Because llf the lack of viewfinder means, the subJect matter iis
`
`
`
`
`
`
`
`not necessarily centered well in the viewfinder, or even captured by the camera., and
`
`
`
`
`
`
`
`
`
`or for use in documentary thus these covert cameras are not well suited to personal
`
`
`
`
`
`
`
`
`
`memory assistant. a personal photographic/videographic
`
`
`
`A wearable camera system (called WearCam) has been proposed in, for example,
`
`
`
`
`
`IEEE Computer, Vol 30, No 2, or equivalently http://wearc:am.org/ieeecomputer.html
`
`
`
`
`
`
`
`
`
`(see also http://wearcomp.org/wearpubs.htm)). Unfortunately embodiments of the
`
`
`
`
`
`clothing, and such as special WearCa.m invention typically require special preparation
`
`
`
`
`
`
`
`
`
`
`
`
`
`special wiring of the body a-priori. Such systems will work for those dedicated to be­
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing constantly wired with various computational a.nd sensory apparatus incorporated
`
`
`
`into special clothing, but there are a good many individual!! who may wish to obtain
`
`
`
`
`
`
`
`
`
`the same convenience without a change of lifesty)L-.
`
`
`
`Ac.corciingly, what is proposed is a camera and viewfinder means for unobtrusiveiy
`
`
`
`
`
`capturing v:deo of exceptionally high compositional qualily an<l possibly even artistic
`
`
`
`
`
`
`
`
`
`2
`
`IPR2021-00923
`Apple EX1004 Page 5
`
`

`

`t..A U22J7939 1991·06·29
`
`
`
`merit, and some embodiments of this invention that are hand-held rather than body-
`
`
`
`
`
`
`
`worn.
`
`
`
`can be used to create versions of the apparatus In addition to the fact that covert
`
`
`
`
`
`
`
`
`
`
`
`
`
`the device need not investigative documentary videos having very good compositi'>n:
`
`
`
`
`
`
`
`
`
`necessarily be covert. Instead, it may merely be designed so that the subject of the
`
`
`
`picture or video cannot readily determine whether or not the apparatus is in use
`
`
`
`
`
`
`
`
`
`
`
`recording. Just as department stores and the like often have dark domes on their
`
`
`
`
`
`ceilings, so that customers do not know whethf!r or not there are cameras in the
`
`
`
`
`
`
`
`domes ( or which ones have cameras and even which way the cameras are pointed if
`
`
`
`
`
`there are cameras in the domes), the proposed invention creates a situation in which a
`
`
`
`
`
`
`
`department store clerk or the like wiJJ not know whether or not a customer's personal
`
`
`
`
`
`
`
`memory assistant is recording video. This uncertainty is created by having the camera
`
`
`
`
`
`
`
`
`
`
`
`
`
`posit�oned so that it will typically he pointed at a person at aJI times, whether or not
`
`
`
`
`
`
`
`it is actual} v being used.
`
`BACKGROUND OF THE INVENTION WITH RESPECT TO PRIOR
`
`
`
`ARI': The invention is related to the Personal Digital Assistantf (PDA) such as the
`
`
`
`
`
`
`
`Palm Pilot. the so-called .. Paper-Based AudioNotebook" (Lisa J. Stifelman Aug­
`
`
`
`
`
`
`
`menting Real-World Objects: A Paper-Based Audio Notebook CHl'96 Conference
`
`
`
`
`
`
`
`
`
`
`
`comput­and the general ideas of pen-based Companion, pp. 199-200, April 1996),
`
`
`
`
`
`mg.
`
`
`
`
`
`There is also an important connection to portable computers, mobile computing,
`
`
`
`
`
`
`
`
`
`Vol 30 No 2). IEEE Computer, and wearable computers (See for example,
`
`
`
`
`
`DESCRIPTION OF THE INVENTION
`
`
`
`
`
`A typical embodimenL of the invent:on comprises a hand-held pen-based computer
`
`
`
`
`
`
`
`
`
`
`
`or a combination clipboard and pen-based computer input device, hereafter referred
`
`
`
`
`
`
`
`to .sa "clipboard" (whether it be an electronic organi�r, PUA, or a real pen and
`
`
`
`
`
`
`
`paper clipboard into which some electronic devices have been added).
`
`
`
`3
`
`IPR2021-00923
`Apple EX1004 Page 6
`
`

`

`CA 02237939 l991·06·Z9
`
`
`
`
`
`A camera is built into the clipboard with the optical axis of the lens facing the
`
`
`
`
`
`<lirection from bottom to top of the clipboard, so that during normal face-to-face
`
`
`
`
`
`conversation, the person holding the clipboard will tend to point the camera at the
`
`
`
`
`
`
`
`
`
`other person while taking written notes of the conversation. In this manner, the
`
`
`
`
`
`
`
`
`
`
`
`intentionality ( whether or not the person taking written notes is intending to point
`
`
`
`
`
`the camera at the other person) is masked by the fact that the camera will aJways
`
`
`
`Thus the in the clipboard. of its placement be pointed at the other person by virtue
`
`
`
`
`
`
`
`
`
`
`
`camera lens opening need not necessarily be covert, and in fact, could be deliberately
`
`
`
`
`
`accentuateci (e.g. made more visible) if desired. To understand why it might be
`
`
`
`
`
`
`
`
`
`<lesirable to make it more visible, one can look to the earner� in department stores.
`
`
`
`
`
`
`
`
`
`which are often placed in hrge dark smoked plexiglass domes. In thi::i way they arP
`
`
`
`
`
`
`
`neither hidden nor visible, but rather, they serve as an uncertain deterrent b criminal
`
`
`
`
`
`
`
`
`
`
`
`conduct. \\ihile they could easily be hidden inside smoke cfetectors, ventillation slots,
`
`
`
`
`
`
`
`
`
`or small openings, the goal of the dome is to make the camera conceptually visible
`
`
`
`
`
`on the clipboard manner, a large lens opening yet completely bidden. In a similar
`
`
`
`
`
`
`
`
`
`may, at times, be desirable, so that the subject will be reminded that there could be a
`
`
`
`
`
`
`
`
`
`
`
`recording, but will be uncertain as to whether or not such a recording is actually taking
`
`
`
`
`
`place. Alternatively, a large dark shiny plexiglass strip. made from darkly smoked
`
`
`
`
`
`
`
`
`
`plexig]ass (typically 1cm high and 22cm across) is installed across the top of the
`
`
`
`clipboard, as a very subtle yet visible deterrent to criminai behaviour and disrespect.
`
`
`
`
`
`
`
`
`
`
`
`One or more miniature cameras are then installed behind the dark plexigla.ss, looking
`
`
`
`
`
`
`
`forward through it. In other embodiments, a camera is installed in a PDA, and then
`
`
`
`
`
`
`
`th� top of the PDA is covered with dark smoky plexiglass.
`
`
`
`The clipboard is fitted with an el�ctronir. rlisplay system which includes the capa­
`
`
`
`
`
`
`
`
`
`bility of di5playing the imag" fro,, the camera, so that it can serve as a viewfinder
`
`
`
`
`
`
`
`
`
`constructed for aiming the camera at th,-:mhject. Moreover, the display is typically
`
`
`
`
`
`so that it is visibJ,-only to the user of the clipboard, or, at the very lea.st, so that the
`
`
`
`
`
`
`
`imhjcct ot the picture cannot readily see the display. Such concealment of the display
`
`
`
`
`
`
`
`1
`
`IPR2021-00923
`Apple EX1004 Page 7
`
`

`

`
`
`filter placed over the display. may be accomplishecf through the use of a houeycomb
`
`
`
`
`
`
`
`Such honeycomb filters are common in phvtography, where they are placed over lights
`
`
`
`
`
`
`
`
`
`to make the light sourc�s behave more directionally. They a.re a.lso sometimes placed
`
`
`
`
`
`
`
`over traffic :ights where there is a wye intersection, so Lbat the lights can only be seen
`
`
`
`
`
`
`
`drivers on do not confuse from one direction, for example, so that the traffic lights
`
`
`
`
`
`
`
`
`
`
`
`another branch of a wye intersection that faces almost the same way. Alternatively,
`
`
`
`ua.rrow field of view, or other to provide the display may be designed an inherently
`
`
`
`
`
`
`
`
`
`barriers may be constructed to prevent the subject from seeing the screen.
`
`
`
`
`
`
`
`The function of thf> camera is integrated with the clipboard so that textual in­
`
`
`
`
`
`
`
`
`
`
`
`
`
`with system, together in a computer may be stored formation, ,:i.q well as drawings,
`
`
`
`
`
`
`pictures or videocli
`
`still pictures
`
`ps (herafter and segments of video will both be re­
`
`
`
`
`
`
`
`
`
`ferrr.d to as videoclips, with the understanding that a still picture is just a video
`
`
`
`st>quence which is one frame in length).
`
`Since videoclips are stored in the computer together with other information, these
`
`
`
`
`
`
`
`
`
`
`
`
`
`together with that videodips may be recalled by an associative memory working
`
`
`
`
`
`other information. Tr.us Lools Jikt· the UNIX "grep" command may be applied to
`
`
`
`
`
`
`
`
`
`resides i\.'I a information which typically videodips by virtue of the associated textual
`
`
`
`
`
`
`
`
`vidrographic header. way, for example,
`one could grep for the word "meijer",
`
`In this
`
`
`
`
`
`and may find various videoclips taken during conversations with clerks in the Meijer
`
`
`
`
`
`department store. Thus such a videographic memory system may give rise to a
`
`
`
`
`
`
`
`memory re<"all of prPvious videoclips taken during previous visits to this particular
`
`
`
`
`
`
`
`
`
`department store! provided that one has been diligent enough to write down ( e.g.
`
`
`
`
`
`
`
`enter tP-xtually) the name of the department store upon each visit.
`
`
`
`Videoc)ips are typically time-stamped (e.g. there exist file creation date11) and
`
`
`
`
`
`
`
`
`
`
`
`CPS-st.amped (e.g. there ,�xists global positioning system headers from last valid
`
`
`
`
`
`
`
`
`readout) so that one can search on setting (time+pla.-e).
`
`Thus the video clipboard may be progrunmed so that the act of ::.:mply taking
`
`rP.latt>d
`notes causf'S previous
`
`
`vidroclips to play bat:k automatically in a separate
`
`5
`
`IPR2021-00923
`Apple EX1004 Page 8
`
`

`

`�A 02237939 )991·06·29
`
`
`
`
`
`window (in addition to the viewfinder window which should always remain active for
`
`
`
`
`
`continued proper aiming of the camera). Such a video clipboard may, for example,
`
`
`
`
`
`
`
`assist in a refund explanatioD by providing the customer with an index into previous
`
`
`
`
`
`
`
`
`
`visual information to accompany previous notes taken during a purchase. This systP.m
`
`
`
`
`
`
`
`
`
`
`
`
`
`who do department store representatives is especially beneficial when encountering
`
`
`
`
`
`
`
`
`
`
`
`not wear name tags and who refuse to identify themselves by name (as is often the
`
`
`
`
`
`
`
`case when t:1ey know they k.ve done something wrong such as disrespect a customer).
`
`
`
`
`
`SUMMARY OF THE INVENTION: OBJECTS AND ADVANTAGES
`
`It is an object of this invention to provide a method of posjtioning a camera to take
`
`
`
`
`
`
`
`
`
`a picture or video of a subject without the subject's knowledge or at least without
`
`
`
`the subject ·s being certain as lo whether or not said picture or video is being taken.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`It is a further object of this invention to provide a means of recording a picture or
`
`
`
`without video when· the spatial extent (fiel<l of view} of the image may be ascertained
`
`
`
`
`
`
`
`ha\·ing to h-:>ld any device up to the eye.
`
`
`
`It is a further object of this invention to provide such a method of taking a picture
`
`
`
`
`
`
`
`
`
`
`
`or vidro where the tonal cl1Macteristics of the picture may be ascertained without
`
`
`
`
`
`
`
`having to h:>ld any device up to the eye.
`
`
`
`It is a furLher object of this invention to provide such a method of taking a picture
`
`
`
`
`
`
`
`
`
`
`
`or video wlwre no apparent diference in body movement or gesture between when a
`
`
`
`
`
`
`
`
`
`picture is bP.ing taken and when no picture is being taken is detectable by others.
`
`
`
`
`
`
`
`ft is a further object of this invention to provide the user witl, a means of deter­
`
`
`
`
`
`
`
`
`
`
`
`
`
`mining the composition of the picture from a display device that is located such that
`
`
`
`
`
`only the user can see the display device, and so that the user can ascertain the com­
`
`
`
`
`
`
`
`
`
`p0ti!lion of .-i pictur� or take a picture or video ;md transmir. image(s) to one or mort>
`
`
`
`
`
`
`
`
`
`
`
`rt>mote lot::ations without tht-knowledgr. of others in the immediate environment, or
`
`
`
`
`
`
`
`
`
`
`
`without
`
`
`
`
`
`other, ir. the environmPnt heing certain whether or not such picture or
`video
`
`is being captured.
`
`6
`
`IPR2021-00923
`Apple EX1004 Page 9
`
`

`

`CA 02137939 1991·06·19
`
`
`
`
`
`
`
`It is a further object of this invention to provide a means and 2pparatus for a user
`
`
`
`to capture a. plurality of images of the same scene or objects, in a natural process
`
`
`
`
`
`
`
`
`
`
`
`of simply sli<ling a small object around on a. countertop, and then have these images
`
`
`
`
`
`
`
`
`
`combined together into a single image of increased spatial extent, spatial resolution,
`
`
`
`
`
`
`
`
`
`
`
`dynamic ra.'.lge, or tonal fidelity.
`
`
`
`
`
`It is a further object of this invention to provide
`a camera with viewfinder mea.,s
`
`
`
`for collaboration between the user of the apparatus and one or more other persons
`
`
`
`
`
`at remote locations through the manipulation of virtual objects such as cursors, or
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`computer graphics renderings displayed upon the camera viewfinder.
`
`It is a further object of this invention
`
`
`to allow multiple users of the invention,
`
`
`
`whether at remote locations or side-by-side, or in the same rooG"\ withjn ea.ch other�s
`
`
`
`
`
`
`
`
`
`
`
`capabilities of the coJla.borative field of view, to interact with one another through
`
`
`
`
`
`the apparatus.
`
`
`
`
`
`It is a further object of this invention to allow multiple users of the invention, at
`
`
`
`
`
`
`
`
`
`remote locatioM, to collaborate in such a way that a virtual environment is shared in
`
`
`
`
`
`
`
`
`
`of video and of each user r�ults in acquisition which camera-based camera-tracking
`
`
`
`
`
`
`
`
`
`subseq uent generation of virtual information being made available to the other(s).
`
`
`
`
`
`
`
`It is a further object of this invention to allow multiple users of the invention,
`
`
`
`
`
`
`
`
`
`at the same location. to collaborate in such a way tha.t multiple camera viewpoints
`
`
`
`
`
`
`
`may be shared among the u�r!I RO that the:· can advise each other on matters such
`
`
`
`
`
`as composition, or so that one or more viewers at remote locations can advise one or
`
`
`
`
`
`
`
`more of the u'3ers on mattr.rs such as composition or camera angle.
`
`
`
`
`
`
`
`It is a further object of this invention to allow multiple users of the invention, at
`
`
`
`
`
`
`
`
`
`to different locations, to collaborate on a project or endeavour that may not pertain
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`photography or virlrography directly, but a project or endeavour nevertheless that is
`
`
`
`r.nhancr.d by the ability for ea.ch persou to experience the viewpoint of another.
`
`
`
`
`
`
`
`a.t to allow one or more ttmote participants It is a. further object of this invention
`
`
`
`
`
`rnnventional desktop oomputers or the like to interact with one or more users of th�
`
`
`
`
`
`7
`
`IPR2021-00923
`Apple EX1004 Page 10
`
`

`

`
`
`invention, at one or more other locations, to collaborate on a project or endeavo:>ur that
`
`
`
`
`
`
`
`
`
`
`
`
`
`may not pe=tain to photography or videogr apby directly, hut a project or endeavour
`
`
`
`
`
`
`
`
`
`nevertheless that is enhanced by the ability for one or more users of the invention to
`
`
`
`
`
`
`
`either provide or obtain advice from or to another individual at a remote locat;on.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`It is a further object of this invention to facilitate a very close-up camera angle,
`
`
`
`
`
`
`
`
`
`from slightly below the 3ubjcct of the picture or video, most notahly so that the
`
`
`
`camera can be brought closer to the subject than would be acceptable for the body of
`
`
`
`
`
`the camera operator, so that the personal space of the subject can be violated more
`
`
`
`
`
`
`
`effectively than with a body-worn camera such as that of the \VearCam invention.
`
`
`
`
`
`
`
`It is a fort.her object of this invention t.o facilita�e the creation of a database
`
`
`
`
`
`comprising video records linked to associated band-written notes.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`It is a fmther object of this invention to facilitate the creation of video records
`
`
`
`
`
`
`
`
`
`linked to a5$0Ciated capture of hau<l writing a.nd \)ther information in and around the
`
`
`
`
`
`immediate environmem..
`
`
`
`
`
`SUMMARY OF THE INVENTION: Informal review of what the new
`
`inveiJtion does
`
`
`
`One aspect of the invention a.Jlows the user tn take note:; with pen and paper ( or
`
`
`
`pen and screen) and continuously record video together with the written notes. Even
`
`
`
`
`
`
`
`
`
`if there is insufficient memory lo capture a continuous video recording, the invention
`
`
`
`
`
`
`
`
`
`to produce a picture can be desigue<l so the user will always end up with the ability
`
`
`
`
`
`from something that wa.s seen a couple of minutes ago. This may be useful to everyone
`
`
`
`
`
`in the sense that we may not want to miss a great photo opportunity, a.nd often great
`
`
`
`photo opportunities only become known to us after we have had time to think ahout
`
`
`
`something we previously saw. At the very least, if for example. a departr�1ent store
`
`
`
`
`
`
`
`the customer may owner or manager becomes angry and imulting to the customer,
`
`
`
`
`
`
`retroactively
`
`
`
`record the event, by opening a circular buffer.
`
`
`
`
`
`there are a safety. Although Such an apparatus might also he of use in personal
`
`
`
`�owing number of vidro surveillance camf!ras installed in the environment, a.Jlege<lly
`
`IPR2021-00923
`Apple EX1004 Page 11
`
`

`

`
`for "public safety", there have been recent
`
`
`
`
`
`
`
`questions as to the true benefit of such
`
`
`
`
`
`centralized surveil1ance infrastructures. Most notably there have been several exam­
`
`
`
`
`
`
`
`
`
`ples in which such centralized infrastructure has been abused by the owners of it ( as
`
`
`
`
`
`
`
`
`
`
`
`in roundups and detainment of peaceful demoostrators). Moreover, "public safety"
`
`
`
`
`
`systems may fail to protect individuals against crimes committed by members of thf"
`
`
`
`
`
`
`
`
`
`
`
`organizations that installed the systems. Accordingly, embodiments of the invention
`
`
`
`
`
`
`
`
`
`often implement the storage and retrieval of images hy transmitting and recording
`
`
`
`
`
`
`
`images at one or more remote locations. In one embodiment of the invention, images
`
`
`
`
`
`
`
`
`
`were transmitted and recor,fod in different countries, so that they would be difficult
`
`
`
`
`
`
`
`
`
`
`
`to destroy, in the event that the perpetrator of a crime or other misconduct might
`
`wish to do so.
`
`
`
`
`
`
`
`
`
`
`
`The invc�ntion allows images to be captured iu a natural manner, without giving
`
`
`
`
`
`an unusual appearance to others (such as m abusive official).
`
`
`
`
`
`
`
`
`
`
`
`Moreover� a.s an artistic toot of personal expression, the apparatus allows the user
`
`
`
`
`
`
`
`to record, f,.om a new perspective, experiences that. have b'---eh <lifccnlt to so record
`
`
`
`
`
`
`
`
`
`in the past. For example, a customer might be tt.ble lo record an argument with a
`
`
`
`
`
`
`
`fraudulent business owner from a very close camera angle. This is possible because
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a cliphoard may be extended outward Loward the person without violating personal
`
`
`
`space in th,� same way as might be necessary to do the same with a camera. hid­
`
`
`
`den in a tit. clip, baseball cap, or sungfasses. Since a clipboard may extend outward
`
`
`
`
`
`
`
`from the body, it may be placed closer lo the subject than the normal eye viewpoint
`
`
`
`
`
`
`
`in normal face-to-face conversation. As a result, the camera can capture a very
`
`
`
`
`
`
`
`close-up vit!W of the subject. Such a view is often more desirable than the view ol>­
`
`
`
`
`
`tainable fro:-i1 a telephoto lens located in eyeglasses (e.g. te)eph'lto embodiment o! th�
`
`
`
`
`
`
`
`
`
`Wea.rCam i:wention) becauSf" the close-up view also provldes a dramatic and power­
`
`
`
`
`
`ful compoeition, from underneath the subject's face, and is therefore most suitable.
`
`
`
`
`
`
`
`
`
`cinematographically, for captcring the menacing nature of a corrupt or disrespectfu 1
`
`
`
`
`
`official.
`
`9
`
`IPR2021-00923
`Apple EX1004 Page 12
`
`

`

`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`The invention will now he described in more detail, by wa.y of ex?.mples whicl1
`
`
`
`
`
`
`
`
`
`in no way are meant to limit the scope of the invention, but, rather, these examples
`
`
`
`
`
`
`
`
`
`
`
`
`
`wiJl serve to iliustrate the invention with reference to the accompanying drawings, in
`
`which:
`
`
`
`
`in which there is of the invention FIG. t .s a diagram of a simple embodiment
`
`
`
`
`which has a. separate display a camera borne by a personal digilal assistant (PDA)
`
`
`
`
`attached to it to function as a. view.finder Co: the camera.
`
`
`
`
`embodiment of the camera sy�tem in which a FIG. 2 is a diagram of au alternate
`
`
`
`pen is concealed under a pad of paper and an electronic graphics t ahlet is concealed
`
`
`
`
`and on the paper is raptured inside an ordinary ink pen so that all of the writing
`
`
`recorded electronically t?gether with video from the subject.
`
`
`
`in which a. camera of the invention FIG. 3 llustrates a wristwatch embodiment
`
`
`
`
`
`system which computer concealed inside the watch sends video to a covert body-worn
`
`
`tram;rnits f1e video to a remote site where it is processed by one or more remote
`
`
`
`experts and a<lvice from the remote expert(s) is displayed on a computer screen
`
`
`
`
`
`
`
`
`
`on the face of the watch together with time, date, and video from the camera (for
`
`
`
`viewfinder purposes).
`
`system, which faci1itates the display of an "xclock" together with other material such
`
`
`
`
`
`
`to be display� which happens view of the wristwatch FIG. 4 depicts a close-up
`
`the display of a body-worn computer
`on top of the Linux operating
`running XFree86
`
`
`
`
`
`
`
`
`menu items suitable for =t. wristwatch videotelephone.
`
`
`
`and variou� as a viev1rfindn), as vid� frcm the camera (so the display can function
`
`
`
`
`
`
`
`While the invention shall now be described with reference to the preferred em­
`
`
`
`
`
`
`
`
`
`is not that the intention bodiments shown in the drawings, it should be w1cJerstood
`
`
`
`
`
`
`
`
`
`to limit the invention only to the particular embodi1mm1s shown but rather to cover
`
`
`
`
`
`�ltPTations, modific�.t:ons equivalent arrangements possible within the scope
`
`
`
`
`all
`and
`
`of appended claims.
`
`10
`
`IPR2021-00923
`Apple EX1004 Page 13
`
`

`

`
`
`
`
`
`
`
`
`
`
`In all aspects of the prt>sent invrntion, referencf'S to "camera" mean any device or
`
`
`
`
`
`coHection of devices capable of simultaneousl; determir:
`
`ing a quantity of light arriving
`
`
`
`
`
`
`
`
`
`from a plurality of directions and or at a plurality of locations, or determining some
`
`
`
`
`
`
`
`
`
`
`
`
`
`other attribute of light arriving from a plurality of directions anri or at a plurality
`
`
`
`
`
`of locations. Similarly references to "display" sha.ll not be limited to just television
`
`
`
`
`
`
`
`monitors or traditional televisions used for the ill�lay uf video from a camera near or
`
`
`
`distant, but shall a.lso indudc computer data display means, computer data monitors.
`
`
`
`
`
`other video di9play devices, still picture dispb.:, devices, ASCII text display devices
`
`and the lik,-.
`References to .. video" shall inr:lude any signal that i11clu<les one or more streams
`
`
`
`
`
`
`
`from O!le or m�re mi­with one or more audio channels of visible pictures, together
`
`
`
`crophones. Thus the term "'video" may, for example, refer to a signal recorded from
`
`
`
`two camera5 and several microphones arranged in an array, to be later processed for
`
`beamformi11g purposes or to be processed in such a way as to project a nullspace
`
`
`
`
`
`
`
`
`toward unwanted audiblt: interference such as the sound of a vacuum cleaner in the
`
`
`
`
`
`
`
`
`
`
`
`y
`
`
`
`
`
`appliances 5ection of a Sean, department store during the shooting of a documenatar
`
`video there. The term '"video .. shall also be taken to include intermediate representa­
`
`
`
`
`
`
`
`
`tions of signals derived from cameras, microphones, and associated auxiliary sensors.
`
`Examples of such intermediate representations shcll include. but not be limited to.
`
`
`
`
`
`recording of some number M of audio channels from some number N of microphones.
`
`where M may be less than �v. Examples of "'video" may also include image represen­
`
`
`
`
`tations from multiple electromagnetic sensors represented as a single signal.
`
`DETAILED DESCRIPTION OF THE INVENTION WITH
`�FERENCE TO DRAWINGS
`
`Fig 1 shows an embodiment of the invention built into a PDA 100. A video camera
`
`
`
`the us.,r of the PDA and displays
`
`
`110 captures a view of a person standing in front of
`
`
`
`tte image on an auxiliary screen 120. which may be "'&sily concealed by the usf'r's
`
`
`
`
`
`II
`
`IPR2021-00923
`Apple EX1004 Page 14
`
`

`

`
`
`hand while the user is writing or pertending to write on the PDA 's screen 130 with
`
`
`
`
`
`
`
`
`
`
`
`pen 140. In commercial embodiments of the invention the auxiliary screen 120 may
`
`
`
`not be nen·ssary, as it may be implemented :.S a window displa.yi-r.g the camera's
`
`
`
`
`
`
`
`view on a portion of the majn screen 130. Annotations made on 9creen J 30 with pen
`
`
`
`
`
`
`
`140 may als() be captured and stored together with videodips from camera l 10� so
`
`
`
`
`
`are linked with that there is a unified database in which the notf's &-id annotations
`
`
`
`
`
`the video .. <\n optional second camera 150 may be used if the user wishes to make
`
`
`
`
`
`
`
`
`
`
`
`a video recc-r<ling of himself /herself while-recording another person with camera 110.
`
`
`
`
`
`In this way. hoth sides of the c.onver11ation may he simultaneously recorded by the
`
`
`
`
`two cameras, so that the rcsultin�
`
`
`
`
`
`rerordings could be edited later, so that there
`
`for exampl", be a rnt back and forth between tht: two CcL-neras to follow tbe
`could,
`
`
`
`natural flow of the conversation. Such a recording might, for ,•,'{ample. �e used for au
`
`
`
`
`
`
`
`invrstigativ,� journalism story on corrupt store owners who illegally chaiu
`
`
`
`
`department
`
`
`
`their fire exits shut. In the ea.rly protor.ypeti, an additional wire 160 was run up the
`
`
`
`
`
`sleeve of th£-user into a separate body worn pa.ck 170 powered by its own battery pack
`
`
`
`
`
`172.The body worn pack typically contains a c.omputer system 174 which houses
`
`
`
`
`
`
`
`vi<loo capture ha.rdwa.re and is connected to a communication!: system 176 comprising
`
`
`
`
`
`p:tcket radk, t�rminal node cuutroiler (high level data link controller with modem) and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`radio. which typicaJly establishe; a.n Internet connection by way of antenna 178. The
`
`
`
`l>ody-worn pack 170 would likely disappear in t.hc final embodiment of the invention
`
`
`
`
`
`
`
`
`
`
`
`devi<:e the hand-held which would <:onta.in the functionality of the pack 170 inside
`
`
`
`
`
`JOO.
`
`
`
`
`
`Fig 2 <le ,i<:t� an alternate embodiment of the invention depicted in Fig 1 in which
`
`
`
`
`
`
`
`
`
`
`
`on
`
`the inventicn is built into ar1 ordinary clipboard
`200. Video camera 210 displays
`
`
`
`
`
`ii minature srreen 220 mounted to the clipboard. A folded back piece of paper 230
`
`
`
`
`
`
`
`conce..als th,-screen 220. The rest of the sheets of pciper 231 are placed slightly below
`
`
`
`
`
`
`
`th• top sh,. .. t 230, so that th<! u11cr ran write on them in a natural fashion. From
`
`
`
`
`
`
`
`
`
`tb� p�npec.iw of �omeonP. facing the user (the subjed), the clivLoard will haw the
`
`
`
`
`
`12
`
`IPR2021-00923
`Apple EX1004 Page 15
`
`

`

`
`
`to be part of in which the top sheet 230 appears apprara.Tice of a normal cliphoard
`
`
`
`
`
`
`
`
`
`the stack 2:n. Pen 240 is a combined electronic pen and real pen, so that the user
`
`
`
`can simultaneously write on the paper with real ink, as well as make an electronic
`
`
`
`annotation by virtue of a graphics tablet below the �tack of paper 231, provided that
`
`
`
`
`
`
`
`
`
`
`
`
`
`the stack is not excessively thick. In this way therr. is a computer database )inking thr
`
`
`
`
`
`
`
`
`
`real physical paper with its pen stroke11 a.nd the video recorded of the subject. From
`
`
`
`
`
`
`
`a legal point of view, real physical pen strokes may have some forensic value that the
`
`
`
`
`
`electronic
`
`
`
`material may not ( e.g. i