(12)
`
`United States Patent
`
`US 7,092,735 B2
`(10) Patent No.:
`Osann, Jr.
`(45) Date of Patent:
`Aug. 15, 2006
`
`
`USOO7092735B2
`
`(54) VIDEO-VOICEMAIL SOLUTION FOR
`WIRELESS COMMUNICATION DEVICES
`
`6/2005 Canova et al.
`6,906,741 B1*
`........... 348/14.08
`2001/0032335 A1* 10/2001 Jones ...................... 725/105
`
`..
`379/8813
`2001/0050977 A1* 12/2001 Gerszber et al.
`
`
`uz
`l
`.............
`:88:/8?§é;§é :1:
`~~ 3355423
`3388; gisfimum ~~~~~
`
`2002/0147661 A1* 10/2002 Hatakama et al.
`..... 705/26
`
`2002/0171673 A1* 11/2002 Brown et al.
`.....
`345/700
`
`1/2004 Vnnen ..............
`2004/0014456 A1*
`455/413
`2005/0083642 A1*
`4/2005 Senpuku et al.
`............ 361/681
`* cited by examiner
`Primary ExamineriFan Tsang
`Assistant ExamineriLisa Hashem
`
`Inventor: Robert Osann, Jr., 328 Costello Ct.,
`L05 Altos’ CA (Us) 94024
`.
`.
`.
`.
`SHbJeCt.t° any (gimme; the 33““?th
`131215613 1155?:an 55°; 3 J“Ste
`“11 er 35
`.
`.
`.
`y
`ays.
`A l. N .: 10/104 934
`pp
`0
`3
`Filed:
`Mar. 223 2002
`
`.
`.
`(*) Nome“
`
`(76)
`
`21
`
`)
`(
`(22)
`
`(65)
`
`Prior Publication Data
`
`(57)
`
`ABSTRACT
`
`US 2004/0203608 A1
`
`Oct. 14, 2004
`
`(51)
`
`Int. Cl'
`(200601)
`H04N 7/14
`(200601)
`H04M 1/00
`(200601)
`H043 ”38
`(52) US. Cl.
`................... 455/556.1; 455/557; 455/566;
`348/1402
`None
`(58) Field ofClassification Search ................
`See app11cat1on file for complete search hIStOYY
`_
`References Clted
`U.S. PATENT DOCUMENTS
`
`(56)
`
`5,963,245 A * 10/1999 McDonald ............... 348/14.01
`..... 348/14.02
`6,069,648 A *
`5/2000 Suso etal.
`
`...........
`6,380,975 B1*
`4/2002 Suzuki
`348/231.99
`
`6,424,843 B1*
`7/2002 Reitmaa et al.
`.
`..... 455/566
`............. 455/556.1
`6,751,473 B1*
`6/2004 Goyal et al.
`6,812,954 B1* 11/2004 Priestman etal.
`....... 348/14.01
`6,882,864 B1*
`4/2005 Miyake ................... 455/556.1
`
`An enhanced communication and voicemail solution for
`mobile phones is described where still images and/or Video
`clips are injected into the voice stream creating a “video-
`voice” call. When a receiving party is not available to take
`a video-voice call, this combined stream of voice and image
`information is stored at the mobile service provider in a
`manner similar to voice mail today. Then, stored video-
`voicemails may be retrieved at a later time by the receiving
`party A150;
`realtime video—voice conversations may be
`recorded for later retrieval in order to document the con-
`versation or because a party in the conversation is not able
`to view the images realtime.
`
`While the sending party may use a normal size mobile phone
`containing a miniature digital camera, the receiving party
`may view video-voicemail images on a variety of devices
`including a wireless mobile phone or PDA, or alternately a
`conventional PC connected to the World Wide Web.
`
`11 Claims, 7 Drawing Sheets
`
`Video-Voicemail information Flow
`
`2
`
`\
`
` Service Provider
`
`Video-Voicemail
`
`1
`
`
`Realtime
`
`
` Video-Voice
`Receiving party receives realtime
`Comunication
`
`voice and images directly via
`wireless phone device, or via
`wired or wireless Web access and
`
`provides realtime voice response.
`
`
`Message \
`
`
`Receiving party retrieves
`stored message and
`
`
`images directly via
`
`
`wireless phone device
`\ Calling party transmits
`
`voice and
`
`
`Receiving party retrieves
`
`lmagesNideoClips
`
`stored message and
`images via wired or
`
`
`wireless web access
`
`
`Calling Party
`
`4 /
`
`APPLE 1020
`
`APPLE 1020
`
`1
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 1 of 7
`
`US 7,092,735 B2
`
`
`
`
`
`.wmcoame88>9:3.me$23.5
`
`
`
`$630.5momtwm
`
`
`
`25:9883825:3m:_>_¢owm
`
`
`
`«3288%mmmwc:.2586>
`
`
`
`«3._o603%9.0.5322:5
`
`tammmwoom325322:5.569:3
`
`
`
`26E:o_...mE.oE___mEoo_o>-omE>
`
`m
`
`I$8.38350295cam63::.oo_o>
`
`
`
`[/N
`
`:ozmoEsEoo
`
`mu_o>-ooU_>
`
`323”.
`
`__ano_o>-8u_>
`
`$5523.3m:_>_wowm
`
`
`
`.25$3on6805
`
`/
`
`m
`
`3305am;329:?359:3«.3mwmmg
`
`
`AnnimEEwuom
`
`floSwE
`
`\EmmEsau
`
`mQ__oowu_2mwmmE_
`
`.2538>
`
`35:3.3.8m=_>_mowm
`83299.0%.329:3mm_>232%89::/mucm$530.:
`
`.9205onm
`
`mtEmcm:3.5am:_=mo/
`
`H
`
`/09$me
`
`2
`
`
`
`
`
`
`
`
`
`

`

`U S. Patent
`
`Aug. 15, 2006
`
`Sheet 2 of 7
`
`US 7,092,735 B2
`
`A
`
`32>Rom32>.53.
`
`
`
`
`
`935..5260BEvoymhmBE9958
`
`
`
`
`
`ummoaxmEwe—Wu:89830«5:88
`
`32>E8".
`
`onBaaLNoEwE
`
`«NuSwE
`
`3
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 3 of 7
`
`US 7,092,735 B2
`
`
`
`“a.9205.025:3.30:05:55
`
`
`
`350035uca.mmmc:.oo_o>
`
`__~Eoo_o>-o%_>m3258
`
`
`355303.5509.56530:51
`
`
`
`
`
`.8335.5oo_>._wm.5330
`
`
`
`$552.32
`
`mohswE
`
`4
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 4 of 7
`
`US 7,092,735 B2
`
`__NEOO_O>IO@_U_>
`
`
`
`
`
`.8E525oo_o>=_mchEmommam—E
`
`om
`
`‘ "7';
`
`.9
`
`wA
`C11,35
`
`
`
`3
`2w
`
`uocm:Sm
`
`“05m==w
`
`M:
`
`saw55
`
`Fania
`
`n,4.4m,4c
`
`mm
`
`v0.53.»
`
`Emmbw
`¢u_o>
`
`S
`
`5
`
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 5 of 7
`
`US 7,092,735 B2
`
`
`
`
`
`__mE¢o_o>-ow_o_>3..Emobw88>EmchEwon—mmmE.
`
`NN
`
`:9:
`
`:o_u:_owwm
`
`94-820
`
`“05m55
`
`82>
`
`3.0
`
`
`
`can..5”—
`
`mohswfi
`
`S
`
`6
`
`
`

`

`U.S. Patent
`
`Aug. 15, 2006
`
`Sheet 6 of 7
`
`US 7,092,735 B2
`
`
`
`3032.“55:5:0wommwmos...mEmo.o>-oo.u_>m:_>>o.>
`
`
`
`
`
`
`
`
`
`35>mcwwbm9.353”.0mozm.Homam<
`
`
`
`
`
`229:5:wozawn>=cmoE=9m
`
`.32m.:.,
`
`.miii?i§.§iisfi§§ifl
`
`633%cosmoEsEEoo
`
`
`
`<9.822.3:0.3303$9.5.
`
`
`
`.Ew2m:3330.920:.$30.8.
`
`
`
`.6925.332ng8REE:
`
`
`
`.on..mcoscgcoom:o:ooBm
`
`o85mm”.
`
`.VN
`
`on:8mmmmE.
`
`:o.8263
`
`
`
`.9550.2.90:.
`
`
`
`m.:335..
`
`
`
` .combm...mEmBcan53.2...
`
`7
`
`
`
`

`

`U.S. Patent
`
`2u,
`
`6
`
`7f07mh
`
`US 7,092,735 B2
`
`o_0.
`SH
`:25?meM:9;5:3
`g.5wa
`
`
`5sinmgfiwifififiwfigg0n—:0m_nm:m>w
`
`3.3$2..3a;m___wquwwwumwgmv
`
`.59.:Eyvfi‘mafl
`
`
`my2.,......,éu.$41.31;?
`$3M.Egg
`
`In,.1533.552.\L
`..§cx:~§._1nwm
`
`
`
`423.95:2,#53353%.3.3.avWu“mm7.6%
`
`5853m
`
`
`
`
`
`
`
`Magma«9‘9
`
`
`x51:5:it)!51‘.x112.2_15,1214;31$”.
`.1.3::
`
`.
`
`,in
`
`
`
`$2afiéééfifixxiii...
`
`a\,3.fl
`
`mm
`
` » =mEmo_o>-om_o_>w_>mEBwo62m:ozosbwcoo9:265muoEmm
`
`8
`
`

`

`US 7,092,735 B2
`
`1
`VIDEO-VOICEMAIL SOLUTION FOR
`WIRELESS COMMUNICATION DEVICES
`
`FIELD OF THE INVENTION
`
`This invention relates to the operation of mobile phone
`communication systems such as those including cellular
`phones or any form of mobile wireless communication
`device capable of voice communication, and in particular,
`enhancements to conventional realtime voice communica-
`
`tions and voicemail storage and retrieval systems for mobile
`phones allowing the integration of still images and video
`clips.
`
`BACKGROUND
`
`Today, voicemail for mobile phones is simply that, storing
`voice messages when the receiving party is not available, to
`be retrieved at a later time. Essentially, voice mail for mobile
`phones operates in a very similar manner to voicemail for
`conventional office phones. To date,
`there has been no
`attempt to integrate images or video with voicemail for
`mobile phones or conventional office phones.
`Communications between two parties where both voice
`and video are utilized is well known and is commonly
`referred to as video conferencing or teleconferencing. Some-
`times this capability is also known as a “Net Meeting”. Here,
`a group of individuals, each having a computing device
`including microphone, speaker, video camera, and a con-
`nection to the World Wide Web, are able to communicate in
`a real-time manner through both video and voice mediums.
`The ultimate goal of a video conferencing system is to
`transmit a continuous stream of audio and video to and from
`
`in the meeting or conversation, and to
`each participant
`emulate as much as possible the interaction that would occur
`if all participants were in the same room. To do this, there
`will be compromises and limitations for many years to come
`relative to the desired functionality for video conferencing
`due to bandwidth limitations. In contrast to this goal and
`these compromises,
`the invention described herein uses
`existing bandwidth capabilities to selectively integrate
`images and video with voice communication in order to
`solve very specific and valuable problems. The manner in
`which this integration occurs has not been offered before.
`The term “videomail” is often used in the industry, but in
`contrast to voice mail, does not refer to the storage of
`messages for later retrieval. Instead,
`it refers to attaching
`video clips to e-mails and a similar manner to the common
`practice of attaching files containing digital photographs to
`e-mails.
`
`Cameras have, at times, been incorporated into mobile
`phones for surveillance purposes. Instead of broadcasting
`digital video via a data link with the mobile/cellular service
`provider, these phones broadcast an RF signal to a viewing
`receiver in the same manner as any other covert surveillance
`camera system.
`Digital cameras are available as attachments to some
`PDAs (Personal Digital Assistants), some of which also are
`available with wireless connection to a service provider
`allowing accessibility to the World Wide Web. Some PDAs
`with wireless Web-interface capability can also function as
`mobile or cellular phones. Some mobile phones have added
`PDA capabilities, again with wireless Web-interface capa-
`bility. Also, some mobile phones now have digital cameras
`available as options to allow pictures to be captured and
`attached to emails. However, none such devices offer the
`integration of still
`images or video with cellular voice
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`communication or with voicemail for cellular phones fol-
`lowing the existing paradigm of phone calls and voicemail
`messages.
`A very successful capability offered by one cellular ser-
`vice provider (the Nextel Direct Connect® digital two-way
`radio service), allows frequent communications among a
`group of individuals who work closely together to be more
`convenient and less costly. As a result, this capability has
`been adopted by the majority of businesses that require
`frequent communications with individuals working at dif-
`ferent locations in a local area, for the most part, businesses
`in the construction industry. This allows a manager, fore-
`man, or responsible person to more easily keep track of the
`progress at a variety of locations, and more readily commu-
`nicate to affect swift problem resolution. Unfortunately,
`these communications rely on the ability of the individuals
`involved to clearly describe situations and problems they
`observe in terms that the responsible person can understand
`in order to best make decisions and guide the remote
`workers. The ability for the responsible person to see the
`subject or problem area would significantly enhance the
`value of these communications.
`
`A solution is needed that, given the bandwidth limitations
`of current and next generation cellular data transmission
`capability, provides an easy way for persons to communicate
`image and video information, while maintaining a commu-
`nication paradigm that is familiar, basically the paradigm of
`phone calls and voicemails. Such a system would allow high
`resolution images to be transmitted when a high level of
`detailed is required, and alternately, video clips (which may
`be at lower resolutions) where spatial relationships and or
`motion information is required.
`
`SUMMARY
`
`An enhanced communication and voicemail solution for
`
`mobile phones is described where still images and/or video
`clips are injected into the voice stream creating a “video-
`voice” call. When a receiving party is not available to take
`a video-voice call, this combined stream of voice and image
`information is stored at the mobile service provider in a
`manner similar to voice mail today. Then, stored video-
`voicemails may be retrieved at a later time by the receiving
`party. While the sending party may use a normal size mobile
`phone containing a miniature digital camera, the receiving
`party may view video-voicemail
`images on a variety of
`devices including a wireless mobile phone or PDA, or
`alternately a conventional PC connected to the World Wide
`Web.
`
`Compared with continuous, full motion video, occasion-
`ally injecting a still image or video clip into the voice stream
`allows much higher resolution images to be sent given
`bandwidth limitations, allowing the receiving party to view
`a subject or situation in much greater detail. For use in
`business applications, conveniently viewing this more
`detailed information, synchronized with voice explanations,
`enables better decisions thereby saving time and money.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is described with respect to par-
`ticular exemplary embodiments thereof and reference is
`accordingly made to the drawings in which:
`FIG. 1 shows a flow chart for voice and image informa-
`tion within the scope of the present invention.
`FIG. 2 shows a cellular phone incorporating a digital
`camera.
`
`9
`
`

`

`US 7,092,735 B2
`
`3
`FIG. 3 shows a camera-enabled cellular phone in action
`with a high-resolution image being viewed remotely on a
`cellular phone/PDA combination device.
`FIG. 4 shows a video-voice message where high resolu-
`tion still photos have been injected into the voice stream.
`FIG. 5 shows a video-voice message where a Video clip
`and a high-resolution still photo have been injected into the
`voice stream.
`
`FIG. 6 shows a variety of wireless communication
`devices having digital communications capability for dis-
`playing the images from video-voice messages, with empha-
`sis on the variation in aspect ratio of the displays.
`FIG. 7 shows a conventional PC, in this case a notebook
`computer, having a large high-resolution display, and
`capable of receiving the video-voice messages through
`conventional Web access.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`A better way to communicate between persons operating
`at remote locations and a manager, advisor, or person in
`authority, would include interspersing images and/or video
`clips within the voice communication stream, whether or not
`it is real time or a voicemail message left for future retrieval.
`Such a solution utilizes video clips where they are most
`effective (even with reduced resolution)4conveying motion
`information, or alternately conveying spatial information of
`the subject area by way of a “Pan” motion with the camera.
`In addition, high resolution still images can be injected into
`the voicestream to allow a very detailed view of a particular
`subject or problem area, synchronized with a verbal descrip-
`tion and other related discussion.
`
`Many applications will benefit from this new capability,
`including the construction industry, the medical and care
`industry, field service and repair, building inspection, insur-
`ance adjusters, or any application where people need a
`convenient way to document situations at remote locations
`and make this information available to others. Another
`
`specific application that will benefit from video-voice com-
`munications is that of emergency situations. When someone
`calling 911 to report an emergency can also provide video
`clips and high resolution still images, a dispatcher or para-
`medic receiving the call can much better understand the
`situation and even instruct the caller in a way that may save
`lives.
`
`In general, integrating this capability with the familiar
`paradigm of the mobile phone call and voicemail is most
`convenient and useful. As discussed earlier, it is known to
`attach digital photos and digital video clips to emails. Emails
`are inherently digital, so this attachment is natural. Emails
`are also not a realtime communications medium. However,
`it is not known to attach digital photos and digital video clips
`to voice communications, whether realtime or as stored
`messages.
`Note that, throughout this specification, the terms “mobile
`phone”, “cellular phone”, and “wireless phone” are synony-
`mous and refer to any mobile communications device
`capable of bi-directional voice communication.
`FIG. 1 shows a flow chart for voice and image informa-
`tion within the scope of the present invention. Here, calling
`party 1 transmits a stream of voice information with still
`images and/or video clips interspersed throughout. These
`may be part of a real-time conversation with receiving party
`4 where the receiving party interactively communicates
`(voice) with the calling party while receiving images that are
`displayed on the receiving device. Scenario 7, describing the
`
`4
`
`receiving party’s communication during a realtime conver-
`sation, indicates that the receiving party may communicate
`via some form of wireless phone device with digital capa-
`bility communicating directly via the service provider, or
`alternately may communicate through the World Wide Web
`using a conventional PC with multimedia and voice com-
`munication capability (i.e., speaker and microphone or some
`form of headset).
`If the receiving party is not available for realtime com-
`munications, a message may be stored at service provider 2
`in database 3 designed to store voice, image, and video clip
`information while retaining the time relationships between
`the three. Per scenario 6,
`the retrieving party accesses
`previously stored voice, image, and video clip information
`directly from the service provider via some form of wireless
`phone device. Altemately, per scenario 5, the receiving party
`may access stored voice, image, and video clip information
`via the World Wide Web through either wired or wireless
`Web access. Although FIG. 1 only shows one mobile service
`provider, it is possible that there is more than one service
`provider in the link shown between the calling party and the
`receiving party. For simplicity, only one is shown here.
`Throughout this specification and the attached claims, the
`term “mobile service provider” will refer to one or more
`service providers who support mobile (or cellular or wire-
`less) communication.
`For cellular voice communications today, real-time con-
`versations are never recorded. Only voicemail messages are
`recorded. As relates to video-voice conversations,
`it
`is
`however useful to record real-time communications that
`
`contain either still images or video clips injected into the
`voice stream. This may be desired in order to document the
`visual information being conveyed as part of the conversa-
`tion for later retrieval. It may also be desired when the
`receiving party answers a call and, as part of the conversa-
`tion, subsequently realizes that images or video clips are
`being transferred, but can’t properly or safely view them.
`With the ability to have voicemail or video-voicemail
`messages stored at the mobile service provider retrieved via
`the World Wide Web, users can archive conversations,
`voicemail messages, and video-voicemail conversations and
`messages on their personal or business computer system.
`Today, there is not a convenient way to archive mobile
`voicemail messages.
`If the receiving party is not in a position to view these
`images real-time, such as when driving a car, having the
`conversation recorded including all images will allow them
`to review the visual and audio information by retrieving the
`recorded message at a later time. Altemately, if the receiving
`party is driving or otherwise in a situation where viewing is
`inconvenient or impossible, it may be useful to have the
`ability to transfer a real-time conversation into video-voice-
`mail, if the conversation reaches a point where it is mean-
`ingless to continue real-time without the receiving party
`being able to view the images or video clips. This capability
`may be implemented by always recording realtime video-
`voice conversations at
`the mobile service provider, and
`discarding the information at the end of the conversation if
`the receiving (or calling) party has not taken some action
`(like pressing some button) to initiate the saving of the
`video-voice conversation. Alternately, the user could set the
`default mode to be that of automatically saving conversa-
`tions, deleting them later if not needed or deleting them after
`they have been downloaded via the Web and archived. A
`variation on this these would include automatically saving
`conversations from a particular calling party, and deleting
`them later if not needed or deleting them after they have
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`10
`
`10
`
`

`

`US 7,092,735 B2
`
`5
`been downloaded Via the Web and archived. The accumu-
`
`lation of information resulting from these recording sce-
`narios would require a much larger amount of memory for
`storing messages at
`the service provider, but if this is
`valuable, it is a service that users would be willing to pay
`extra for.
`
`The scenarios just described can also be applied to
`recording realtime voice conversations for later retrieval and
`archive. Even archiving voice alone can be a powerful tool
`within business applications. For recording realtime conver-
`sations, the issue of permissions and privacy arises. One
`easy to handle the granting of “permission to be recorded”
`would be that permission is deemed to be given to record
`conversations from or to a particular phone number by
`calling from that particular phone number and taking a
`prescribed action which could include entering a specified
`code. It may instead be desired to record only video-voice
`conversations and not voice-only conversations, again with
`permissions having been given.
`Although the essence and value of a video-voice conver-
`sation as described herein is bi-directional for voice com-
`
`munication, but mostly or solely unidirectional with regard
`to the transmission of video clips and still
`images,
`the
`methods described may in certain circumstances, be valu-
`able in a bi-directional manner. For instance, a worker at a
`jobsite may send images to a supervisor at a remote location,
`and the supervisor may, in return, send an image of a portion
`of a blueprint while pointing out some specific details to
`resolve certain issues. Of course,
`this requires that
`the
`worker have a device with a display capable of presenting
`the blueprint
`image with enough resolution to properly
`discriminate the necessary information.
`Another purpose for recording video-voice messages
`involves a reverse scenario where the supervisor/foreman/
`responsible party may wish to record information and
`instructions concerning a particular job site, such that the
`information and instructions may be viewed by workers at a
`later time before commencing work or as part of problem
`resolution during the job.
`FIG. 2 shows one variation of a camera integrated accord-
`ing to this invention with a conventional cellular phone.
`While cameras are sometimes offered as external options to
`mobile phones, such solutions may not be rugged enough for
`business use, especially in the construction industry. Incor-
`porating the camera within the phone is a simpler and more
`rugged solution. While the variation shown in FIG. 2 does
`not allow the user to photograph themselves while viewing
`the display, such as would be required for a mobile video
`conference or net meeting, this capability is not required
`where the purpose of the video-voice conversation is remote
`viewing. FIG. 2a is a front view of the same phone.
`Notice in FIG. 2a that button 8 located on left side is
`
`positioned such that the index finger of the right hand may
`easily press this button to activate the “shutter” for the
`integral digital camera. Altemately, having the button on the
`right side would allow operation by the index finger if the
`user’s right hand covers the keypad. These alignments
`would be reversed for a left-handed person. Either way,
`having the “shutter button” on the side allows a more firm
`grip on the camera body, allowing a steadier picture, when
`pushing the “shutter” button.
`If the integral camera and associated electronics are
`designed to allow either still images or video clips to be
`captured and transmitted, there are at least two methods of
`utilizing shutter button 8 to easily support both. In the first
`method, the button may be pressed briefly and released to
`record a still image, or pressed and held for a longer duration
`
`6
`to enable a video clip to be transmitted during the duration
`of time for which the button is held. In the second method,
`button 8 may be partially depressed to capture a still image
`or fully depressed to capture a video clip. Altemately for
`each of the above methods, which button action captures still
`images and which captures video clips may be reversed. In
`addition to the methods just described for determining when
`video-clips are captured as opposed to still images, a more
`conventional method can always be utilized where the
`desired mode of capture is first selected through a key or
`combination of key presses on the phone’s keypad, followed
`by pressing a “shutter” button.
`Although this specification refers to transmitting both still
`images and video clips, an implementation may only deal
`with one of the two. If the bandwidth limitations are severe,
`occasionally injecting high resolution still images into the
`voicestream is probably more valuable than video clips.
`Eventually, when the available bandwidth is at a level where
`high resolution video clips can be easily sent via mobile
`communications, sending only video clips will be appropri-
`ate. In the interim, the combination of high resolution still
`images and lower resolution video clips may be the best
`overall compromise.
`FIG. 2b shows a rear view of the cell phone revealing
`battery cover 9 and an integral sliding protective cover 10
`that protects the lens for the integral digital camera. The
`sliding protective cover allows the phone to have a normal
`conformation when the cover is closed and provides maxi-
`mum protection for the integral camera. Alternately, an
`integral protective cover might be hinged at one end and
`“flip-open” to expose the camera lens. In any case, for a
`robust solution for business use, an integral protective cover
`must be always attached to the main body of the phone so
`that it can be easily restored to its protective position after
`the user is finished using the camera feature, and so that the
`cover will never be misplaced or lost.
`FIG. 20 shows another rear view of the same phone where
`protective cover 10 has been withdrawn to reveal camera
`lens 11. Note that, as for conventional digital cameras and
`video cameras, an optical zoom capability may be added.
`FIG. 3 shows an application example where a construc-
`tion worker (the sending party), in this case a mason,
`is
`communicating with his supervisor regarding a problem
`with a brick column that has just been constructed. As shown
`in image 12 and enlargement 13, the sending party is holding
`the phone in front of them like a camera in order to capture
`image and/or video information. In this mode, the display on
`the phone should temporarily act like the viewfinder display
`on a digital camera. This mode can be activated by a button
`on the keypad, a push of the shutter button 8, or some other
`mechanism. When button 8 is released, normal phone dis-
`play information usually consisting of digits and icons can
`be optionally superimposed over the camera display, in a
`black or white (reversed) format.
`Since the user of the phone/camera will be holding the
`device in front of them, they will not be able to talk directly
`into a normal microphone. Hence, it is necessary to have
`either a speaker phone capability, or some form of wired or
`wireless headset to allow bi-directional voice communica-
`
`tion while the camera function is being utilized.
`Looking again at FIG. 3, voice, image, and video clip
`information is either transferred in real time by way of the
`service provider 15 to the receiving party, or alternately is
`stored as a video-voicemail message at the mobile service
`provider for retrieval at a later time. In addition, as men-
`tioned previously, a real-time conversation with image and
`video clips added may also be stored at the service provider
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`11
`
`11
`
`

`

`US 7,092,735 B2
`
`7
`to further document a situation for later retrieval. Note that
`
`during any conversation where image and Video clip infor-
`mation is being transferred, bi-directional voice communi-
`cation will occur for real-time conversations.
`
`In FIG. 3, the camera is incorporated into a conventional-
`looking cell phone 14 and is observed by the receiving party
`on a cell phone/PDA combination unit 16 allowing a larger,
`more detailed view of images enabling the receiving party to
`make better decisions.
`
`FIG. 4 shows how images are positioned in voice stream
`17 of a video voicemail conversation with the initial position
`in time of the images relative to the voicestream being
`maintained for all such information transmitted. Here three
`
`high-resolution still images, image 18, image, and image 20,
`are injected into the voice stream at different times, a voice
`description typically coordinated with each image to explain
`any issues. To allow this discussion to continue while the
`receiving party is viewing a related image, the last still
`image transmitted or the last frame of a video clip will
`typically be maintained on the screen of the receiving device
`until superseded by another image or video clip, or until
`otherwise terminated by an action of the receiving party.
`FIG. 5 shows both a video clip sequence 21 and a high
`resolution still image 22 injected into the voice stream 17,
`such that the initial position in time of images and video
`clips relative to the voicestream are maintained for all such
`information transmitted. Notice that the video clip is able to
`convey spatial orientation by sweeping (panning) from left
`to right, thereby positioning a specific focal point properly
`within its surrounding environment. In this case, a construc-
`tion crew has uncovered a pipe and in the process has broken
`it in one specific place. The pan video clip sequence sweeps
`along the length of the pipe, ending the sequence at the
`specific location of interest, where high resolution still
`image 22 provides a close-up detail of the break itself.
`Where, according to this invention, video clips or high-
`resolution still images are injected into the voice stream, the
`last image to be captured according to the action of shutter
`button 8 will typically be maintained on the display of the
`capturing phone for a predetermined time period after button
`8 is released, or until otherwise terminated by a subsequent
`action of the sending party. Also, when button 8 is released,
`normal phone display information may optionally again be
`superimposed on the displayed image if desired.
`Since it is desired that the party capturing and sending
`images can clearly and easily observe what images are being
`captured, it may be necessary to add some form of sun-shade
`to allow clear observation of an LCD “viewfinder” display
`on the phone/camera. Alternately, or in addition, it may be
`useful to add a polarizing filter over an LCD display for
`better viewing in bright sunlight. Other display technologies,
`more easily viewed in direct sunlight, may be utilized. It
`may also be useful to add an optical viewfinder such as those
`found in many conventional digital cameras.
`types of
`FIG. 6 shows some examples of additional
`video-voice enabled viewing devices, with emphasis on the
`variation of aspect ratios among them. Previously, unit 16 in
`FIG. 3 showed a cellular phone device that opens into a
`wireless data enabled PDA, having a very wide format
`screen. Shown additionally in FIG. 6 are a conventional
`flip-phone 23 having a relatively standard aspect ratio
`screen, and a wireless enabled Palm Pilot PDA 24, having an
`aspect ratio that is unusually tall in the vertical direction.
`FIG. 7 shows a conventional PC 25,
`in this case a
`notebook computer with a large high-resolution screen,
`having a fairly typical aspect ratio. The large variation in
`aspect ratio between the screens as shown in FIGS. 3, 6, and
`
`8
`7 provides an opportunity to pre-process video clip and
`image data at the service provider to better match the aspect
`ratio of a receiving device, before that information is sent to
`the particular receiving device. Essentially, when matching
`the aspect ratio of a particular receiving device, there is
`information that will not be displayed anyway, and removing
`(cropping) this extraneous information before transmitting,
`can reduce the amount of time acquired to transmit video
`clip and image information to a particular receiving device.
`Therefore, a methods and apparatus for implementing a
`combination video/voicemail system especially useful in the
`construction industry and other industries requiring remote
`viewing with guidance and supervision, has been described
`It should be understood that the particular embodiments
`described above are only illustrative of the principles of the
`present invention, and various modifications could be made
`by those skilled in the art without departing from the scope
`and spirit of the invention. Thus, the scope of the present
`invention is limited only by the claims that follow.
`
`What is claimed is:
`
`1. A method for transferring video, voice, and still image
`information during a realtime conversation, wherein video
`clips and still images are captured by a calling party on a
`capturing device, said video clips and still images being
`interspersed in time and combined with