United States Patent
`US 7,092,735 B2
`(10) Patent No.:
`(12)
`Osann, Jr.
`(45) Date of Patent:
`Aug. 15, 2006
`
`
`US007092735B2
`
`(54) VIDEO-VOICEMAIL SOLUTION FOR
`WIRELESS COMMUNICATION DEVICES
`
`(76)
`
`Inventor: Robert Osann, Jr., 328 Costello Ct.,
`Los Altos, CA (US) 94024
`:
`:
`:
`:
`Subjectto any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 551 days.
`
`.
`(*) Notice:
`
`21)
`(21)
`(22)
`
`(65)
`
`Appl. No.: 10/104,934
`Appl.
`No
`,
`Filed:
`Mar. 22, 2002
`
`Prior Publication Data
`US 2004/0203608 Al
`Oct. 14, 2004
`
`(51)
`
`Int. Cl.
`(2006.01)
`HOAN 7/14
`(2006.01)
`HO4M 1/00
`(2006.01)
`HO4B 1/38
`(52) US. Ch wee 455/556.1; 455/557; 455/566;
`348/14.02
`(58) Field of Classification Search ................see None
`See application file for complete search history.
`.
`References Cited
`U.S. PATENT DOCUMENTS
`
`(56)
`
`5,963,245 A * 10/1999 McDonald ............... 348/14.01
`5/2000 Suso et al. ue 348/14.02
`6,069,648 A *
`
`...........
`wee 348/23 1.99
`6,380,975 B1*
`4/2002 Suzuki
`
`............. 455/566
`6,424,843 B1*
`7/2002 Reitmaaet al.
`6/2004 Goyal et al. ......0... 455/556.1
`6,751,473 B1*
`6,812,954 B1* 11/2004 Priestman et al.
`....... 348/14.01
`6,882,864 B1*
`4/2005 Miyake oe. 455/556.1
`
`6,906,741 BL*
`6/2005 Canova et al... 348/14.08
`2001/0032335 Al* 10/2001 Jones wo... eee 725/105
`
`2001/0050977 Al* 12/2001 Gerszberetal. ..
`379/88.13
`
`
`. ett
`soosoraboee Al" ae Sishimura seen
`.........0...
`.. 348/211
`2002/0130956 Al
`9/2002 Suzuki
`
`2002/0147661 A1* 10/2002 Hatakamaet al... 705/26
`
`... 345/700
`2002/0171673 Al* 11/2002 Brownetal. .....
`1/2004 Vnnen «see
`1 455/413
`2004/0014456 AL*
`
`............ 361/681
`2005/0083642 Al*
`4/2005 Senpukuet al.
`
`* cited by examiner
`Primary Examiner—Fan Tsang
`Assistant Examiner—Lisa Hashem
`
`ABSTRACT
`(57)
`An enhanced communication and voicemail solution for
`mobile phones is described wherestill images and/or video
`clips are injected into the voice stream creating a “video-
`voice” call. When a receiving party is not available to take
`a video-voicecall, this combined stream ofvoice and image
`information is stored at the mobile service provider in a
`mannersimilar to voice mail today. Then, stored video-
`voicemails may beretrieved at a later time by the receiving
`party. Also,
`realtime video-voice conversations may be
`recorded for later retrieval in order to document the con-
`versation or because a party in the conversation is not able
`to view the imagesrealtime.
`
`While the sending party may use a normalsize mobile phone
`containing a miniature digital camera, the receiving party
`may view video-voicemail images on a variety of devices
`including a wireless mobile phone or PDA,oralternately a
`conventional PC connected to the World Wide Web.
`
`11 Claims, 7 Drawing Sheets
`
`Video-Voicemail Information Flow
`
`2
`
`—
`
` Service Provider
`
`
`Realtime
`
`
` Video-Voice
`Receiving party receives realtime
`
`Comunication
`voice and imagesdirectly via
`wireless phone device, or via
`wired or wireless Web access and
`
`provides realtime voice response.
` Video-Voicemail
`
`Message —~__|
`
`
`Receiving party retrieves
`
`
`stored message and
`
`imagesdirectly via
`
`
`wireless phone device
`
`
`
`Receiving party retrieves
`
`
`stored message and
`
`
`imagesvia wired or
`
`wireless web access
`
`1
`
`~ Calling party transmits
`voice and
`
`Images/VideoClips
`
`Calling Party
`
`47
`
`APPLE 1020
`
`APPLE 1020
`
`1
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 1 of 7
`
`US 7,092,735 B2
`
`
`
`"aSUOdSa!BIIOAOUIN|Bd1SApPIAOId
`
`
`
`
`
`
`
`
`
`owl}ea1SeAigoe)AjiedBulaisoay
`
`
`
`
`
`BIAAj}DauIpSaBew!pue3d10A
`
`
`
`
`
`BIA10‘aDdIAapsuOoUdSsajauIM
`
`
`
`puessao0egaffSSd]2JIM10palim
`
`MO|-4{UONEWUOJU][IEWSDIOA-OaPIA
`
`
`€
`
`
`
`JOPIAOIdBIIAIBS
`
`a
`|eseqej}egdijg0api,pue‘abeuy‘ad10A
`
`
`
`
`
`SdAdl}o1AyedBulala.ay
`paio}s9Lte1AAoaupsabeu|\pureebessouw
`
`
`
`
`soaolije1AedBulaiaoay
`
`
`
`pueobessowpaiojs
`
`JOpailMBIASabeu!
`
`\
`
`S
`
`SS999PGamSSOIOJIM
`
`AyiedgBularazay
`
`[ons
`
`sdijpo0apipssobew|
`
`
`
`pukeOd10A
`
`
`
`AyiedBuryjed
`
`SdIANpauOYdssajadIM
`
`
`syiwsuelAjiedBuryes~~
`
`
`
`uonesiunwoy
`
`BIIOA-O9PIA
`
`awnjeoy
`
`
`
`/—_aBessayy
`
`JJEWI9D10/A-0ap!A
`
`I
`
`2
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 2 of 7
`
`US 7,092,735 B2
`
`
`
`MalJeeyMAlAJeay
`
`
`
`SUuOUJeINI}9DOU!payesba}u]esowey
`
`
`
`
`
`(
`
`
`
`
`
`
`
`pasodxyeiawuesi"(pa19sA0yeiowes)
`
`
`
`MalJUOI-J
`
`o7oINBIyQZoInBI,J
`
`BTOINSIY
`
`3
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 3 of 7
`
`US 7,092,735 B2
`
`
`
`J&paso}sJO9UIJea)Pawajsues
`
`
`
`
`
`Ssdi]jD0apiApue‘abew]‘a01I0A,
`
`[JEWADIOA-OaPIABIAs]le}0q
`
`
`
`SHSUol}ONIYSUOyDBuimai,As}owsay
`
`
`
`JO}JBPIAOIdBOIAIISJENIN
`
`
`
`
`
`
`
`JEADII}OI1932]
`
`€MNS
`
`4
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 4 of 7
`
`US 7,092,735 B2
`
`JTEWISIIOA-OSPIA
`
`
`
`
`
`
`
`JO}WeRdI}S9DIOAUlBulUOnISOgobew]
`
`
`
`JOUSIIHS
`
`
`
`JOUSINS
`
`0¢
`
`81
`
`61J0USIIS
`
`yeahee
`
`PEEoly,eeyeabetnSapPeieSORESET
`falvei
`
`esDEEStewe
`
`Pr
`
`
`
`gaaasaat‘3
` itoleESY'BMigBanepesteAeNyhaeOe
`
`aaahare:
`Peepers
`
`segPORELION”Ge
`aeSheeegeeei
`eRe
`
`pwnsiy
`
`
`
`WESI]SBDIOA
`
`LI
`
`5
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`
`
`JIEWIDDIOA-OSPIAJO}WedI}]S9910UlBulUOTISOgebew|
`
`
`
`
`
`
`
`Aug. 15, 2006
`
`Sheet 5 of 7
`
`US 7,092,735 B2
`
`YBIH
`
`uoI}]Njosey
`
`dn-asojd
`
`
`
`JOUSIIHS
`
`dio
`
`
`
`ue,JO}
`
`¢MNBL
`
`LI
`
`6
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 6 of 7
`
`US 7,092,735 B2
`
`
`
`SddIAVGUdIOJJIGUOsabessapy|IEWIDDIOA-OapPIAHUIMdIA
`
`
`
`
`
`
`
`KienSU88I9SHUIMAIAJOSOIjesJOOdsy
`
`
`
`
`
`
`
`
`
`JUuaJayipusamjoqAyjueoyiubis
`
`
`
`VddSS2]a4IMUOpeMmadlAsobew]
`
`
`
`WSsejnqpIeJepa10WeplAold
`
`
`
`Ja6ie]YMpazedwodpoyiwi|
`
`"qdJPEUOIJUSAUODBUOUDDIOS
`
`9OINSL]
`
`ra
`
`©anonnea
`
`
`
`"SODIAOPUOIJEOIUNWIWIOD
`
`ve
`
`
`
`aquedsabew]
`
`UOPOMIIA
`
`
`
`‘auoudajiIqow
`
`SI[!eJepynq
`
`0}enppoy}ul]
`
`
`
`"Ud9IOS[JEWS
`
`7
`
`
`
`

`

`U.S. Patent
`
`US 7,092,735 B2
`
`=4YBIYYIM
`
` \oS—+NAeb“U89JOSUOINIOSSI
`ve=Dni
`
`ACR,PAREDERennaAYrsetomNEforenameeSOoOpBSIEPm.,,,ageseencoracpemngPa
`
`
`SeetasehS|]}IeJapJSa}eS1s)
`
`
`
`fo]~—Ye
`
`~Sm
`
`SC
`
`Lomnsty
`
`
`
`
`
` |FEWISDIOA-OSPIABIASjle}9q9}ISUOHONAYSUODHBulmoiAajoway
`
`OdUOa/qejieAe
`
`8
`
`

`

`US 7,092,735 B2
`
`1
`VIDEO-VOICEMAIL SOLUTION FOR
`WIRELESS COMMUNICATION DEVICES
`
`FIELD OF THE INVENTION
`
`This invention relates to the operation of mobile phone
`communication systems such as those including cellular
`phones or any form of mobile wireless communication
`device capable of voice communication, and in particular,
`enhancements to conventional realtime voice communica-
`
`tions and voicemail storage andretrieval systems for mobile
`phones allowing the integration of still images and video
`clips.
`
`BACKGROUND
`
`Today, voicemail for mobile phonesis simplythat, storing
`voice messages whenthe receiving party is not available, to
`beretrievedat a later time. Essentially, voice mail for mobile
`phones operates in a very similar manner to voicemail for
`conventional office phones. To date,
`there has been no
`attempt to integrate images or video with voicemail for
`mobile phones or conventional office phones.
`Communications between two parties where both voice
`and video are utilized is well known and is commonly
`referred to as video conferencing or teleconferencing. Some-
`timesthis capability is also known as a “Net Meeting”. Here,
`a group of individuals, each having a computing device
`including microphone, speaker, video camera, and a con-
`nection to the World Wide Web, are able to communicate in
`a real-time manner through both video and voice mediums.
`The ultimate goal of a video conferencing system is to
`transmit a continuous stream of audio and video to and from
`
`in the meeting or conversation, and to
`each participant
`emulate as muchaspossible the interaction that would occur
`if all participants were in the same room. To dothis, there
`will be compromises and limitations for many years to come
`relative to the desired functionality for video conferencing
`due to bandwidth limitations. In contrast to this goal and
`these compromises,
`the invention described herein uses
`existing bandwidth capabilities to selectively integrate
`images and video with voice communication in order to
`solve very specific and valuable problems. The manner in
`which this integration occurs has not been offered before.
`The term “videomail” is often used in the industry, but in
`contrast to voice mail, does not refer to the storage of
`messages for later retrieval. Instead,
`it refers to attaching
`video clips to e-mails and a similar manner to the common
`practice of attaching files containing digital photographs to
`e-mails.
`
`Cameras have, at times, been incorporated into mobile
`phones for surveillance purposes. Instead of broadcasting
`digital video via a data link with the mobile/cellular service
`provider, these phones broadcast an RF signal to a viewing
`receiver in the same manneras any other covert surveillance
`camera system.
`Digital cameras are available as attachments to some
`PDAs(Personal Digital Assistants), some of which also are
`available with wireless connection to a service provider
`allowing accessibility to the World Wide Web. Some PDAs
`with wireless Web-interface capability can also function as
`mobile or cellular phones. Some mobile phones have added
`PDAcapabilities, again with wireless Web-interface capa-
`bility. Also, some mobile phones now have digital cameras
`available as options to allow pictures to be captured and
`attached to emails. However, none such devices offer the
`integration of still
`images or video with cellular voice
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`communication or with voicemail for cellular phones fol-
`lowing the existing paradigm of phonecalls and voicemail
`messages.
`A very successful capability offered by one cellular ser-
`vice provider (the Nextel Direct Connect® digital two-way
`radio service), allows frequent communications among a
`group of individuals who work closely together to be more
`convenient and less costly. As a result, this capability has
`been adopted by the majority of businesses that require
`frequent communications with individuals working at dif-
`ferent locations in a local area, for the mostpart, businesses
`in the construction industry. This allows a manager, fore-
`man, or responsible person to more easily keep track of the
`progress at a variety of locations, and more readily commu-
`nicate to affect swift problem resolution. Unfortunately,
`these communications rely on the ability of the individuals
`involved to clearly describe situations and problems they
`observe in terms that the responsible person can understand
`in order to best make decisions and guide the remote
`workers. The ability for the responsible person to see the
`subject or problem area would significantly enhance the
`value of these communications.
`A solution is needed that, given the bandwidth limitations
`of current and next generation cellular data transmission
`capability, provides an easy way for persons to communicate
`image and video information, while maintaining a commu-
`nication paradigm thatis familiar, basically the paradigm of
`phonecalls and voicemails. Such a system would allow high
`resolution images to be transmitted when a high level of
`detailed is required, and alternately, video clips (which may
`be at lower resolutions) where spatial relationships and or
`motion information is required.
`
`SUMMARY
`
`An enhanced communication and voicemail solution for
`mobile phones is described wherestill images and/or video
`clips are injected into the voice stream creating a “video-
`voice” call. When a receiving party is not available to take
`a video-voicecall, this combined stream of voice and image
`information is stored at the mobile service provider in a
`manner similar to voice mail today. Then, stored video-
`voicemails may beretrieved at a later time by the receiving
`party. While the sending party may use a normal size mobile
`phone containing a miniature digital camera, the receiving
`party may view video-voicemail
`images on a variety of
`devices including a wireless mobile phone or PDA, or
`alternately a conventional PC connected to the World Wide
`Web.
`
`Compared with continuous, full motion video, occasion-
`ally injecting a still image or video clip into the voice stream
`allows much higher resolution images to be sent given
`bandwidth limitations, allowing the receiving party to view
`a subject or situation in much greater detail. For use in
`business applications, conveniently viewing this more
`detailed information, synchronized with voice explanations,
`enables better decisions thereby saving time and money.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is described with respect to par-
`ticular exemplary embodiments thereof and reference is
`accordingly made to the drawings in which:
`FIG. 1 showsa flow chart for voice and image informa-
`tion within the scope of the present invention.
`FIG. 2 shows a cellular phone incorporating a digital
`camera.
`
`9
`
`

`

`US 7,092,735 B2
`
`3
`FIG. 3 shows a camera-enabled cellular phone in action
`with a high-resolution image being viewed remotely on a
`cellular phone/PDA combination device.
`FIG. 4 shows a video-voice message where high resolu-
`tion still photos have been injected into the voice stream.
`FIG. 5 shows a video-voice message where a video clip
`and a high-resolution still photo have been injected into the
`voice stream.
`FIG. 6 shows a variety of wireless communication
`devices having digital communications capability for dis-
`playing the images from video-voice messages, with empha-
`sis on the variation in aspect ratio of the displays.
`FIG. 7 shows a conventional PC, in this case a notebook
`computer, having a large high-resolution display, and
`capable of receiving the video-voice messages through
`conventional Web access.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`A better way to communicate between persons operating
`at remote locations and a manager, advisor, or person in
`authority, would include interspersing images and/or video
`clips within the voice communication stream, whetheror not
`it is real time or a voicemail messageleft for future retrieval.
`Such a solution utilizes video clips where they are most
`effective (even with reduced resolution)—conveying motion
`information, or alternately conveying spatial information of
`the subject area by way of a “Pan” motion with the camera.
`In addition, high resolution still images can be injected into
`the voicestream to allow a very detailed view of a particular
`subject or problem area, synchronized with a verbal descrip-
`tion and other related discussion.
`
`Manyapplications will benefit from this new capability,
`including the construction industry, the medical and care
`industry, field service and repair, building inspection, insur-
`ance adjusters, or any application where people need a
`convenient way to documentsituations at remote locations
`and make this information available to others. Another
`
`specific application that will benefit from video-voice com-
`munications is that of emergency situations. When someone
`calling 911 to report an emergency can also provide video
`clips and high resolution still images, a dispatcher or para-
`medic receiving the call can much better understand the
`situation and even instruct the caller in a way that may save
`lives.
`In general, integrating this capability with the familiar
`paradigm of the mobile phone call and voicemail is most
`convenient and useful. As discussed earlier, it is known to
`attach digital photos and digital video clips to emails. Emails
`are inherently digital, so this attachment is natural. Emails
`are also not a realtime communications medium. However,
`it is not knownto attach digital photos anddigital video clips
`to voice communications, whether realtime or as stored
`messages.
`Note that, throughoutthis specification, the terms “mobile
`phone”, “cellular phone”, and “wireless phone” are synony-
`mous and refer to any mobile communications device
`capable of bi-directional voice communication.
`FIG. 1 shows a flow chart for voice and image informa-
`tion within the scope of the present invention. Here, calling
`party 1 transmits a stream of voice information with still
`images and/or video clips interspersed throughout. These
`maybepart of a real-time conversation with receiving party
`4 where the receiving party interactively communicates
`(voice) with the calling party while receiving imagesthat are
`displayed on the receiving device. Scenario 7, describing the
`
`4
`receiving party’s communication during a realtime conver-
`sation, indicates that the receiving party may communicate
`via some form of wireless phone device with digital capa-
`bility communicating directly via the service provider, or
`alternately may communicate through the World Wide Web
`using a conventional PC with multimedia and voice com-
`munication capability (i.e., speaker and microphone or some
`form of headset).
`If the receiving party is not available for realtime com-
`munications, a message may bestored at service provider 2
`in database 3 designed to store voice, image, and video clip
`information while retaining the time relationships between
`the three. Per scenario 6,
`the retrieving party accesses
`previously stored voice, image, and video clip information
`directly from the service provider via some form of wireless
`phone device. Alternately, per scenario 5, the receiving party
`mayaccess stored voice, image, and video clip information
`via the World Wide Web through either wired or wireless
`Webaccess. Although FIG. 1 only shows one mobile service
`provider, it is possible that there is more than one service
`providerin the link shown between the calling party and the
`receiving party. For simplicity, only one is shown here.
`Throughout this specification and the attached claims, the
`term “mobile service provider” will refer to one or more
`service providers who support mobile (or cellular or wire-
`less) communication.
`For cellular voice communications today, real-time con-
`versations are never recorded. Only voicemail messages are
`recorded. As relates to video-voice conversations,
`it
`is
`however useful to record real-time communications that
`
`contain either still images or video clips injected into the
`voice stream. This may be desired in order to documentthe
`visual information being conveyedas part of the conversa-
`tion for later retrieval. It may also be desired when the
`receiving party answers a call and, as part of the conversa-
`tion, subsequently realizes that images or video clips are
`being transferred, but can’t properly or safely view them.
`With the ability to have voicemail or video-voicemail
`messagesstored at the mobile service providerretrieved via
`the World Wide Web, users can archive conversations,
`voicemail messages, and video-voicemail conversations and
`messages on their personal or business computer system.
`Today, there is not a convenient way to archive mobile
`voicemail messages.
`If the receiving party is not in a position to view these
`images real-time, such as when driving a car, having the
`conversation recorded including all images will allow them
`to review the visual and audio information by retrieving the
`recorded messageat a later time. Alternately, if the receiving
`party is driving or otherwise in a situation where viewing is
`inconvenient or impossible, it may be useful to have the
`ability to transfer a real-time conversation into video-voice-
`mail, if the conversation reaches a point where it is mean-
`ingless to continue real-time without the receiving party
`being able to view the imagesor video clips. This capability
`may be implemented by always recording realtime video-
`voice conversations at
`the mobile service provider, and
`discarding the information at the end of the conversation if
`the receiving (or calling) party has not taken some action
`(like pressing some button) to initiate the saving of the
`video-voice conversation. Alternately, the user could set the
`default mode to be that of automatically saving conversa-
`tions, deleting them later if not needed or deleting them after
`they have been downloaded via the Web and archived. A
`variation on this these would include automatically saving
`conversations from a particular calling party, and deleting
`them later if not needed or deleting them after they have
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`10
`
`10
`
`

`

`US 7,092,735 B2
`
`6
`5
`been downloaded via the Web and archived. The accumu-
`to enable a video clip to be transmitted during the duration
`of time for which the button is held. In the second method,
`lation of information resulting from these recording sce-
`
`narios would require a much larger amount of memory for button 8 maybepartially depressed to captureastill image
`storing messages at
`the service provider, but if this is
`or fully depressed to capture a video clip. Alternately for
`valuable, it is a service that users would be willing to pay
`each of the above methods, which button action capturesstill
`extra for.
`images and which captures video clips may be reversed. In
`The scenarios just described can also be applied to
`addition to the methodsjust described for determining when
`recording realtime voice conversations forlater retrieval and
`video-clips are captured as opposedto still images, a more
`archive. Even archiving voice alone can be a powerful tool
`conventional method can always be utilized where the
`within business applications. For recording realtime conver-
`desired mode of capture is first selected through a key or
`sations, the issue of permissions and privacy arises. One
`combination of key presses on the phone’s keypad, followed
`easy to handle the granting of “permission to be recorded”
`by pressing a “shutter” button.
`would be that permission is deemed to be given to record
`Althoughthis specification refers to transmitting bothstill
`conversations from or to a particular phone number by
`images and video clips, an implementation may only deal
`with one of the two. If the bandwidth limitations are severe,
`calling from that particular phone number and taking a
`prescribed action which could include entering a specified
`occasionally injecting high resolution still images into the
`code. It may instead be desired to record only video-voice
`voicestream is probably more valuable than video clips.
`conversations and not voice-only conversations, again with
`Eventually, when the available bandwidthis at a level where
`permissions having been given.
`high resolution video clips can be easily sent via mobile
`Although the essence and value of a video-voice conver-
`communications, sending only video clips will be appropri-
`sation as described herein is bi-directional for voice com-
`ate. In the interim, the combination of high resolutionstill
`images and lower resolution video clips may be the best
`overall compromise.
`FIG. 26 showsa rear view of the cell phone revealing
`battery cover 9 and an integral sliding protective cover 10
`that protects the lens for the integral digital camera. The
`sliding protective cover allows the phone to have a normal
`conformation when the cover is closed and provides maxi-
`mum protection for the integral camera. Alternately, an
`integral protective cover might be hinged at one end and
`“flip-open” to expose the camera lens. In any case, for a
`robust solution for business use, an integral protective cover
`must be always attached to the main body of the phone so
`that it can be easily restored to its protective position after
`the useris finished using the camera feature, and so that the
`cover will never be misplacedorlost.
`FIG. 2c showsanotherrear view of the same phone where
`protective cover 10 has been withdrawn to reveal camera
`lens 11. Note that, as for conventional digital cameras and
`video cameras, an optical zoom capability may be added.
`FIG. 3 shows an application example where a construc-
`tion worker (the sending party), in this case a mason,
`is
`communicating with his supervisor regarding a problem
`with a brick column that has just been constructed. As shown
`in image 12 and enlargement13, the sending party is holding
`the phone in front of them like a camera in order to capture
`image and/or video information.In this mode, the display on
`the phone should temporarily act like the viewfinder display
`on a digital camera. This mode can beactivated by a button
`on the keypad, a push ofthe shutter button 8, or some other
`mechanism. Whenbutton 8 is released, normal phonedis-
`play information usually consisting of digits and icons can
`be optionally superimposed over the camera display, in a
`black or white (reversed) format.
`Since the user of the phone/camera will be holding the
`device in front of them, they will not be able to talk directly
`into a normal microphone. Hence, it is necessary to have
`either a speaker phone capability, or some form of wired or
`wireless headset to allow bi-directional voice communica-
`
`munication, but mostly or solely unidirectional with regard
`to the transmission of video clips andstill images,
`the
`methods described may in certain circumstances, be valu-
`able in a bi-directional manner. For instance, a worker at a
`jobsite may send imagesto a supervisorat a remotelocation,
`and the supervisor may, in return, send an imageof a portion
`of a blueprint while pointing out some specific details to
`resolve certain issues. Of course,
`this requires that
`the
`worker have a device with a display capable of presenting
`the blueprint
`image with enough resolution to properly
`discriminate the necessary information.
`Another purpose for recording video-voice messages
`involves a reverse scenario where the supervisor/foreman/
`responsible party may wish to record information and
`instructions concerning a particular job site, such that the
`information and instructions may be viewed by workers at a
`later time before commencing work or as part of problem
`resolution during the job.
`FIG. 2 shows onevariation of a camera integrated accord-
`ing to this invention with a conventional cellular phone.
`While cameras are sometimes offered as external options to
`mobile phones, such solutions may not be rugged enough for
`business use, especially in the construction industry. Incor-
`porating the camera within the phoneis a simpler and more
`rugged solution. While the variation shown in FIG. 2 does
`not allow the user to photograph themselves while viewing
`the display, such as would be required for a mobile video
`conference or net meeting, this capability is not required
`wherethe purpose of the video-voice conversation is remote
`viewing. FIG. 2a is a front view of the same phone.
`Notice in FIG. 2a that button 8 located on left side is
`
`positioned such that the index finger of the right hand may
`easily press this button to activate the “shutter” for the
`integral digital camera. Alternately, having the button on the
`right side would allow operation by the index finger if the
`user’s right hand covers the keypad. These alignments
`would be reversed for a left-handed person. Either way,
`having the “shutter button” on the side allows a more firm
`grip on the camera body, allowing a steadier picture, when
`pushing the “shutter” button.
`If the integral camera and associated electronics are
`designed to allow either still images or video clips to be
`captured and transmitted, there are at least two methods of
`utilizing shutter button 8 to easily support both. In the first
`method, the button may be pressed briefly and released to
`recorda still image, or pressed and held for a longer duration
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`11
`
`tion while the camera function is being utilized.
`Looking again at FIG. 3, voice, image, and video clip
`information is either transferred in real time by way of the
`service provider 15 to the receiving party, or alternately is
`stored as a video-voicemail message at the mobile service
`provider for retrieval at a later time. In addition, as men-
`tioned previously, a real-time conversation with image and
`video clips added mayalso bestoredat the service provider
`
`11
`
`

`

`US 7,092,735 B2
`
`7
`to further documenta situation for later retrieval. Note that
`during any conversation where image and video clip infor-
`mation is being transferred, bi-directional voice communi-
`cation will occur for real-time conversations.
`In FIG.3, the camera is incorporated into a conventional-
`looking cell phone 14 andis observed by the receiving party
`on a cell phone/PDA combination unit 16 allowing a larger,
`more detailed view of images enabling the receiving party to
`make better decisions.
`
`FIG. 4 shows how imagesare positioned in voice stream
`17 of a video voicemail conversation with the initial position
`in time of the images relative to the voicestream being
`maintained for all such information transmitted. Here three
`high-resolution still images, image 18, image, and image 20,
`are injected into the voice stream at different times, a voice
`description typically coordinated with each image to explain
`any issues. To allow this discussion to continue while the
`receiving party is viewing a related image, the last still
`image transmitted or the last frame of a video clip will
`typically be maintained on the screen of the receiving device
`until superseded by another image or video clip, or until
`otherwise terminated by an action of the receiving party.
`FIG. 5 shows both a video clip sequence 21 and a high
`resolution still image 22 injected into the voice stream 17,
`such that the initial position in time of images and video
`clips relative to the voicestream are maintained for all such
`information transmitted. Notice that the video clip is able to
`convey spatial orientation by sweeping (panning) from left
`to right, thereby positioning a specific focal point properly
`within its surrounding environment. In this case, a construc-
`tion crew has uncovereda pipe andin the process has broken
`it in one specific place. The pan video clip sequence sweeps
`along the length of the pipe, ending the sequence at the
`specific location of interest, where high resolution still
`image 22 provides a close-up detail of the break itself.
`Where, according to this invention, video clips or high-
`resolutionstill images are injected into the voice stream, the
`last image to be captured according to the action of shutter
`button 8 will typically be maintained on the display of the
`capturing phone for a predeterminedtime period after button
`8 is released, or until otherwise terminated by a subsequent
`action of the sending party. Also, when button 8 is released,
`normal phone display information may optionally again be
`superimposed on the displayed image if desired.
`Since it is desired that the party capturing and sending
`images can clearly and easily observe what imagesare being
`captured, it may be necessary to add some form of sun-shade
`to allow clear observation of an LCD “viewfinder” display
`on the phone/camera. Alternately, or in addition, it may be
`useful to add a polarizing filter over an LCD display for
`better viewing in bright sunlight. Other display technologies,
`more easily viewed in direct sunlight, may be utilized. It
`mayalso be useful to add an optical viewfinder such as those
`found in many conventional digital cameras.
`types of
`FIG. 6 shows some examples of additional
`video-voice enabled viewing devices, with emphasis on the
`variation of aspect ratios among them.Previously, unit 16 in
`FIG. 3 showed a cellular phone device that opens into a
`wireless data enabled PDA, having a very wide format
`screen. Shown additionally in FIG. 6 are a conventional
`flip-phone 23 having a relatively standard aspect ratio
`screen, and a wireless enabled Palm Pilot PDA 24, having an
`aspect ratio that is unusually tall in the vertical direction.
`FIG. 7 shows a conventional PC 25,
`in this case a
`notebook computer with a large high-resolution screen,
`having a fairly typical aspect ratio. The large variation in
`aspect ratio between the screens as shownin FIGS. 3, 6, and
`
`8
`7 provides an opportunity to pre-process video clip and
`imagedata at the service provider to better match the aspect
`ratio of a receiving device, before that information is sent to
`the particular receiving device. Essentially, when matching
`the aspect ratio of a particular receiving device, there is
`information that will not be displayed anyway, and removing
`(cropping) this extraneous information before transmitting,
`can reduce the amountof time acquired to transmit video
`clip and image information to a particular receiving device.
`Therefore, a methods and apparatus for implementing a
`combination video/voicemail system especially useful in the
`construction industry and other industries requiring remote
`viewing with guidance and supervision, has been described
`It should be understood that the particular embodiments
`described above are only illustrative of the principles of the
`present invention, and various modifications could be made
`by those skilled in the art without departing from the scope
`and spirit of the invention. Thus, the scope of the present
`invention is limited only by the claimsthat follow.
`
`Whatis claimedis:
`
`1. A methodfor transferring video, voice, andstill image
`information during a realtime conversation, wherein video
`clips and still images ar