WWWWWWMM
`
`IMWWMWWWMWW
`
`S 2U(}5U'l43136/\1
`
`(19) United States
`(12) Patent Application Publication (10) Pub. N0.: US 2005/0143136 Al
`Lev et al.
`(43) Pub. Date:
`Jun. 30, 2005
`
`(54) MMS SYSTEM AND METHOD WITH
`PROTOCOL CONVERSION SUITABLE FOR
`MOBII.E,fPOR'l’/\B[.F. HAN DSE1‘ DISPLAY
`
`(76)
`
`Inventors: Tvsi Lev, 'lel Aviv (I[_); Ran
`Avnilllelech, Ramat (iflfl {I[.)
`
`Correspondence /\d(|re:~‘£.:
`BLAKILLY SOKOI.()l“[“ TAYLOR & ZAF MAN
`[2400 WILSHIRE BOULEVARII
`SEVENTH FLOOR
`l.()S ANGl£I.ICS, CA 90025-1030 ([15)
`
`(21) Appl. No;
`
`l|]{482,566
`
`(22)
`
`l"iled:
`
`Feb. 18, 2005
`
`Related U.S. Application Data
`
`(60)
`
`Provisional application No. fill,-299,745, filed on Jun.
`23, 2001.
`
`l’uI)|ic:tlion Classification
`
`(51)
`
`Int.Cl.’
`
`H041} U38
`4553566
`
`(57)
`
`ABSTRACT
`
`An MMS communication system for displaying images on
`it display terminal of a mobile or portable L2(lI'I'tl1'I1.Jl'IiL'.'Jli()i'l
`device. the systeln comprising: an input adaptecl to receive
`pre-source information; 21 Irzimimitter adapted to transmit lhe
`pre-source inforrnnliolt;
`:1 server adnplecl
`to receive the
`lramsmiltecl pre—source information and further adapted to
`eonverl the pre—:'-source information to source information
`suitable for clisplay on the (lisplay terminal, and a source
`lransmilter udapletl lo lransmil [he source information lo the
`display terminal.
`
`PANORAMA
`
`1
`
`AT&T - Exhibit 1019
`
`AT&T — Exhibit 1019
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 1 of 37
`
`Us 2005/0143135 A1
`
`2 QI
`
`-0L
`
`
`
`L!
`on
`n:
`
`OO
`
`HISTOGRAM
`
`
`
`
`
`°"'“E”T“T'°”ENHANCEMENTDITHERING
`
`
`
`2
`
`

`
`Patent Application Publication Jun. 30, 2005
`
`Sheet 2 of 37
`
`US 2005/0143136 Al
`
`<s_<moz<n_
`
`N..O_n_
`
`3
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 3 of 37
`
`M(DBm:0:5W
`
`4
`
`
`
` 2.%.mu:u.._n¢¢.1..r
`
`
`
`
`
`uéSue:uxo._.:_uE_m_..._..
`
`
`
`l.U.—au...4...Fu:.i..fi.
`
`
`
`:o:mEmU<._..W
`
`t...In‘:
`
`‘nan-Ki...53_—d1:.P£u
`
`
`
`
`
`coiuoZuoouwlommmE.8:..oU.
`
`
`
`
`
`m.o_n_mczsfiunew$6...2m:_mmmoo:.___.:®._wt_Q
`
`mc_uoowcE.r
`
`9,._.m.z
`
`
`
`:o_..mE._oE__m>o_$590:9
`
`
`
`a_u__omms.:_mE_ms_
`
`Hmcootmo
`
`
`
`m>E92$.Hova.:L.m¥.._.”...._M.n.3,’|||I.u.a-.19.35;...__.x.55.h...w_u........u
`
`
`
`
`
`3%.....gm.£ou.»oo¢_3...,
`-_.wuzmaafixmmafir
`
`
`..ww...__...
`
`\.....|:¢.:1A.-Pz
`
`
`
`
`
`
`
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 4 of 37
`
`US 2005/0143136 A1
`
`.
`
`I
`
`....,..—.r..I...
`9an-
`
`
`
`m:__ooomcm_.._.........
`
`m>._.m.z-
`
`
`
`b_o__omm:__:_mE_m_2
`
`8_aa%<.?4a..W.rl,m
`
`
`_2.__ao.....1I3I'll
`Enma:_m>omvmE...fix.2.2%...
`
`
`
`
`=a_o_E.m§..._i
`
`...
`
`
`
`IIJ._m
`
`5
`
`
`
`:._mEuw..E._.«_._m.wt_n_
`
`
`
`US...hon.
`
`v._0_u_
`
`
`
`:o_m.8>:oUwomwmE350
`
`
`
`”m”_.._.m._._Oxoofiw
`
`
`
`
`
`co=mE._o.E_m:o:_tmn_:mm>oEmm
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 5 of 3'?
`
`US 2005/0143136 A1
`
`.6>.5m3«£020«EH-OU:UD
`
`
`
`uomufimaBnowflmmnmvwmfi:.
`
`
`
`.wEUoumEw£.Eamon.
`
`
`
`newnofimmmmaooEEDQO.
`
`uomcmmn
`
`
`
`...—C®fiH®UC.mrd.HmOwoflm.-
`
`
`
`
`
`wfimmmoflmUwmmnEBHQO.
`
`wmmmgBodqouamooum.
`
`bbmmEm.8oufifioa:.
`
`.muo.mfio:mm<
`
`6
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 5 of 37
`
`1.34.1M.M02SU
`
`M
`
`
`
`xomo...humaohm
`
`
`
`mmflsomEmucoo
`
`msm9.3%.ms_s_
`
`xomxomxom
`
`/.
`
`you._Vq_.9
`
`7
`
`m._nmn_moEm_EEoom_._m$$s_mucosa255._
`mucosawwcozmmucosa-
`
`
`
`
`
`
`
`
`
`
`
`
`=aE.m_mamE..=o_n_mucosa.n_<..5
`
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 7 of 37
`
`US 2005/0143136 A1
`
`
`
`E33._3_.__::
`
`
`
`m.=_m$$s_mc_mammm_.a
`
`.
`
`n_=n_U._.
`
`>§>w..mOEE,
`
`8
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 8 of 37
`
`US 2005/0143136 A1
`
`
`
`WEB Portal
`Photo Sharing Silt
`
`E-Mail
`Module
`
`
`
`
`WEB
`PAGES
`
`9
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 9 of 37
`
`US 2005/0143136 A1
`
`To E-MAIL SERVER
`
`To SMS GATEWAY
`
`SMTP I POP3
`
`MODULE
`
`SMS I PM I EMS
`
`PM I EMS
`
`SERVLETS
`
`SERVLETS
`
`FIG.9
`
`10
`
`10
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 10 of 37
`
`US 2005/0143136 A1
`
`SMS GATEWAY
`
`SMS
`
`RECEIVER i i SENDER
`
`SERVLET
`
`SERVLET
`
`EM3;pM DECQDER
`
`EMS I PM ENCODER
`
`\ M
`
`AS UTILITY SERVLETS
`EMS to/from PM
`
`EMS+PM tolfrom WAP+WEB
`
`EMS+PM to/from E-MAI L
`
`FIGJO
`
`11
`
`11
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 11 of 37
`
`US 2005/0143136 A1
`
`E-MAIL SERVER
`
`ENCODER
`
`DECODER
`
`SERVLET
`
`SERVLET
`
`MAS UTILITY SERVLETS
`
`E-MAIL to/fromPM+EMS
`
`E-MAIL to/from WAP
`
`FIG.1l
`
`12
`
`12
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 12 of 37
`
`US 2005/0143136 A1
`
`Enter Phone
`Number
`
`First time users or peolple
`that forget their password
`can get a new password
`by SMS.
`- Enter this page
`- Enter your phone number
`- System creates new user
`- System sends SMS.
`- Re-enter the login page.
`- User you new password
`
`Enter Password
`
`E"$rl_:::r“°
`
`- Hang up.
`- Enter logln page.
`- User password.
`
`Auihenticetion
`OK 7
`
`h1aln..EAae
`- Incoming Messages
`- Archives
`
`Refer to ‘Main Page‘
`flowchart
`
`- Seleci ‘Try Again"
`'- Go back to Iogin page
`- Get new password it
`you fail again.
`
`Return to Logln
`P39“
`
`(‘ [ L, -
`
`I 1.
`
`13
`
`13
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 13 of 37
`
`US 2005/0143136 A1
`
`Afchive Main Page
`- get list of all archives
`- display link (or each
`
`W15 Main Page
`
`"Messages"
`selected
`
`Messages "am Page
`- get list of messages
`- display link {or each
`
`Message Page
`- get message text
`- get message knage
`- gel phone type.
`- lransoode image
`- ueate wmi or html based on client
`
`'5B|'|d'' 58196135
`
`Refer to ‘Send’ flowchart
`
`14
`
`14
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 14 of 37
`
`US 2005/0143136 A1
`
`Send Message
`
`Edit your text
`
`Enter
`
`Recipient's
`Number
`
`"Send as“
`selected
`
`(21 (I\ ' \“’\
`
`"Se nd" selected
`
`Refer to "Message Transfer" flowchart
`
`Select message
`W139
`
`15
`
`15
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 15 of 37
`
`US 2005/0143136 A1
`
`WAP to WAP
`
`- Verify recipient's operator has this service.
`- Register recipient in user table
`- Register message in message table
`
`16
`
`16
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 16 of 37
`
`US 2005/0143136 A1
`
`WAP to EMS or PM
`
`- Verify recipients operator has this service.
`- Verify recipient can View this message.
`- If unknown. assume he can.
`- Register recipient in user table
`- Add this EMS I PM to his capabilities with low confidence.
`- Transcode to correct format.
`
`- Create EMS or PM string.
`- Send to SMS GW
`
`— Register message in message table
`
`gwxt.
`
`17
`
`17
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 17 of 37
`
`US 2005/0143136 AI
`
`WAP to E-Mail
`¥
`- Take original and lransoode to about 5|-(
`— Check that you have reclpienfs e-mail.
`- Create the e-mail message and send
`- Register the message
`
`18
`
`18
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 18 of 37
`
`US 2005/0143136 A1
`
`<-‘Yes
`
`r-
`
`,
`
`IIII
`
`.
`
`Select
`
`'
`
`Locate, cut
`face
`
`_
`
`Emphasize
`features
`
`FIG.13
`
`19
`
`19
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 19 of 37
`
`US 2005/0143136 A1
`
`Output
`
`
`
` Ignore grid.
`background
`
`
`
`Isolate sufficient
`
`range lntormation
`Ruize['a'_|
`[rnirunax values).
`Reeize if possible
`
`j
`ocn rn
`
`
`
`
`
`
`
`Resize graph jg
`
`Determine
`graph size
`
`
`
`
`
`
`
`Separate
`graphics by
`contenttype
`
`
`
`Redraw boundaries
`
`Relocate label
`
`
`
`FIG.19
`
`20
`
`20
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 20 of 37
`
`US 2005/0143136 A1
`
`
`
`Split line art
`and text
`
`
`
`FIG.20
`
`21
`
`21
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 21 of 37
`
`US 2005/0143136 A1
`
`Input
`
`
`
`Separate
`strip to
`blocks
`
`No. I'm sure the
`
`problem is on your
`en
`
`
`
`Split line art
`and text
`
`
`
`FIG.21
`
`esuze me
`
`22
`
`22
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 22 of 37
`
`US 2005/0143136 A1
`
`
`
`23
`
`23
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 23 of 37
`
`US 2005/0143136 A1
`
`24v
`
`m
`
`wig:
`
`
`
`Showomen:ODGDD
`
`
`
`e.:5ooz:o..<Emummm
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 24 of 37
`
`US 2005/0143136 A1
`
`25m
`
`m
`
`
`
`nx_:mmmEH=unOwmoH=m&fl.ummEHumuwm=nuhmmEwv
`
`
`
`ax:uxms=auo«mun.uxu.uxmvgunwm..Uhmuxuuu
`
`
`
`n\=uEm.u:=omumumm=anunowusmv
`
`
`
`Ax=GmmEH=uno«mwH=mmw-ummEHU:oumm=uuummfimv
`
`
`
`Ax.uxms.unommon=uxu.uxuHunouum=aUnmuxuuu
`
`Ax=nEm.unSomunouum=1ohmDwtflmv
`
`
`
`A\.wmmEH=|flDmmflh=mQh.flmflBHflHH&H=nUnamemu
`
`
`
`ax=»xmH,uuoflmuu.uxu.uxueuuH;H=uuumuxmuv
`
`A\=HEd.flfl5OmfiHH§F:uuumDfiflfiflv
`
`n=mv=anunnmmv
`
`nhflfixv
`
`n_.m...._..nannmmu
`
`nhmmxv
`
`n=mm=uHawunmu
`
`aumunxu
`
`AMUODV
`
`Aummav
`
`Ahfloaxv
`
`aflfieuxu
`
`
`
`
`
`A..ausohmHxvnlmv
`
`
`
`.a--uumnum=ufiwHuuom=mmmmanexxnuaommav--mu
`
`
`
`
`
`Ausoxmaxv
`
`
`
`
`
`a--nx=wMm=.unmHun=unn=.guvfi3u:amaH-uoonv--_v
`
`
`
`
`
`
`
`
`
`?-3...9..nou..o..-uuua..3:__u£mwmn_.mS._u533__mmuE....3Gowmuhv13.
`
`
`
`
`
`
`
`
`
`
`
`3:9...mo»_.m3.._...._muH__¢.§...unmfiuz..m...H._..fi3scowmmuv
`flx.:D..lQOu._D..IU%w.m_..VVfi:flu_.~W..nW.£..Wn..H-.I£|.—.U._.._3._HDfiWUHV
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ax=vva=uu:mwu:=mmm=nzuvH3u5ommH-uoouv
`
`
`
`
`
`nuuhuwuwcmwnmvhumgwmmuwflnmngcamH:B-:.vnusommav
`
`
`
`ax=guHEm:non=.uawuaau.Hozu=m.umEmnfiuMEv
`
`
`
`
`
`
`
`
`
`ax=mEE=uuuwucau:mMuflu=uuEm:mumeu
`
`nflmunv
`
`nafifimv
`
`
`
`
`
`
`
`n--n\=u¢H.unon=o=uuumfismrgaunmfiun=whH¢usuUw3=uxmH=:uHnoumwnvnlflV
`
`
`
`
`
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 25 of 37
`
`US 2005/0143136 A1
`
`26
`
`om
`
`
`
`
`
`.wEoEm_mEm._mt_um5.6coamoam>_..d_2
`
`
`
`w£m.__9a€3%_,...8a.nm$w_n_m_vanE5cmoEflcoo9:.E9w=_uEm_wc_E..2.m>_ooo..ucw
`
`
`
`
`
`
`
`
`
`.2mc_m_._o9:wem>m_n_m_u9:.._-“ma.mczmcm2zmsocmm_aE_manum:Emmcmcoxm.=_2w.Eo.$._mE..
`
`
`
`
`
`
`
`
`
`._m:__E£m_.__>_moa._an..6..€_%__o9:BEE3:mean>_aE_m30.6._m:_m_.o9:E5mémmoa
`
`
`
`
`
`
`
`
`
`
`
`.6mocfiwmdamofiScamco.amE._oE._.o._.__mEoomama.:_2mm.m.._._mE_o_:o....mEmmmB.w2._=2w
`
`
`
`
`
`
`
`
`
`on.m.:0Eficoo«commie.9gm:m_._:2w.._m.._>>.>w_%_nm.comEmEe_mw_UOC.__=3F_t._m._mE_U
`
`
`
`
`
`
`
`
`
`.=_2w9:«o.EmEm_m50.5.ms.3umccwuw_mamamongumcmao.....__.sou_.__3m>__m.Eo:_cmm.om
`
`
`
`
`
`
`
`Emcm.___.scamzom9:comama.___>_wmp:.08§_$&m9.:gm:£5E.E.§%_u2.2mama
`
`
`
`
`
`
`
`.._ufimmaum;._o£:m.onemmEflcoum£.6:o=mN_cmm._o05ztmxm
`
`
`
`
`
`.6mumon..m_m:_E._2m__noEcommmmmmmEm_uoE_.._:Ehecozflcmmmams..2new:m_._=2w:o_.=_S
`
`
`
`
`
`
`
`
`
`m_._._.._..,m_am_u_mc__.E99:homocmawmuamnew:o_S_omE9:E_...2_E__x_w._m>mmm_ao_...:_a9:
`
`
`
`
`
`
`
`
`
`__m..sm_u_:5.._2m:_m_.ome..5u2mm._ommEflcoum£flcmmmam.mmmmmmem_umE.£:Em.5.35
`
`.:o_«mEom¢._n_ms?
`
`

`
`27
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 27 of 37
`
`US 2005/0143136 A1
`
`
`
`EOEQOHQQcommmuhsasom:o._.‘533O«Emwmv
`
`
`
`
`
`id;
`
`
`.oz_~_mmmzozbmmmoo
`
`s_<~_wo.8_:uz_mmz._.Epzm_._,_moz¢_._zmzoE§zm_mo
`
`
`
`
`
`28
`
`28
`
`
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 28 of 37
`
`US 2{}0Sf0143136 A1
`
`‘Vr--a
`
`J L
`
`}
`
`
`detection+1:ranscoding Face
`Face
`Detection
`
`
`
`Na'I'veTranscoding
`
`
`
`29
`
`29
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 29 of 37
`
`US 2005/0143136 A1
`
`30o
`
`m
`
`
`
`D._:o.aS:umP£F\~_J~
`
`_w.5ud.Hom..:ooouE»P
`
`._.8.EmEo_m
`
`HofimmE_.n.m
`
`zouflmnmuh.
`
`
`
`=om£_.:._.=EEummfim
`
`
`
`..mo>_u.qE1.Huaua
`
`ummb
`
`HUQE
`
`=o_mmmnqm§...H.
`
`vommxmufiai-mo_E._.mu_wm.Em.
`
`
`
`
`
`Saouaflm»wm_o:moo.HOmwmum
`
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 30 of 37
`
`US 2005/0143136 A1
`
`
`
`
`
`Pixeis in a group that should be flipped:
`
`- Flipping different pixels causes different
`amount of disruption.
`
`- Choose pixel with min. cost for flipping.
`
`‘ -iii‘;-II [limiting
`l'D-W
`
`.
`
`—-i
`
`i
`
`fi
`F F ? I I I
`'
`l
`I‘.
`
`Morphological rule: Flipping cost
`depends on neighbors
`
`I
`
`I
`
`I
`
`E
`
`I
`
`Highest
`
`Lowest
`
`Flipping cost by morphology {neighbors}
`
`UCnGO
`
`Figure XX: Watermarking aigorithm for BIW images (picture messages, WBMP)
`
`Multilayer Input
`
`
`
`sized 4GL Reduced
`
`
`
`2,, '-
`
`"". ...-:3
`
`Reduced (8 la_ve1's:)
`
`Figure XX: Layer reduction and its effect on the message appearance
`
`31
`
`31
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 31 of 37
`
`US 2005/0143136 A1
`
`I'-
`it «tr.
`u
`
`5‘
`
`i
`
`Content Based Conversion
`
`Stock Charts:
`
`Remove superfluous information
`
`Maintain line solidity
`
`i~la'I've
`_
`'_- ‘-_Transcoding
`_
`.
`s
`\,
`_-\'A._"
`
`: ' I"!
`
`
`
`
`Eflgu Valu 0:
`are Sufllcient
`
`Differ ant Tre at nm-.I'
`F '2 r T e ‘.1
`
`Figure XX: content based conversion for charts (color Line-art)
`Content Based Conversion
`
`Cartoons:
`
`Different processing for text and drawings
`
`Ignore gray level information
`Naive
`Maintain line solidity
`
`Transcoding
`
`
`
`.-'. nu: '. ma Irvi"
`wruau-run~'.a_. -
`_ -ac-.-mt
`5-tn '
`It -7».
`
`_
`£111-tlxf EVIL HA. nuxcroa
`. .AND I HAVE FIVE
` .
`
`‘YEARS EJGERIENCE
`A5 A DOT-COM -‘
`
`
`PP-£5!.DENT.
`
`
`Maintain Text
`
`Figure XX: content based conversion for cartoons (B&W Line-art)
`
`32
`
`32
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 32 of 37
`
`US 2005/0143136 A1
`
`[1 31 Ca men:
`Provide!
`
`
`
`Idilb-5:09‘ III
`
`ul Imus!
`p E II 1'
`
` [1 1)] Original color
`image [ram ctunrra
`
`
`
`ljgnG()
`
`#333.
`
`Figure: cases where caching and smart-forwarding are required.
`
`33
`
`33
`
`I nut uvt
`wnnu. l2:f{II(NCl
`| as n out-can
`Ifll I'|.Dl I11‘.
`
`'|
`
`‘ uwuo-A I ma
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 33 of 37
`
`US 2005/0143136 A1
`
`,_.--
`
`iLI'cr-
`J
`
`Specifications
`
`Implementation: The caching and smart forwarding system is based on a web-caching
`mechanism — convened and original media objects are stored in a speciaily configured
`set of wcb—caches, and are retrieved by URI. The basic operation scenario is depicted
`in the figures below.
`
`Figure: Sequence of operations for a cached transaction
`
`Algorithmic Cache Seguence
`
`- For incoming object A (after SMF) + filtered Target Parameters
`calculate MD5 “C" plus target MIME type
`
`o Perform HTTP HEAD request to check if in cache. request filename is
`" D".
`
`0 if not in cache. perform transcode. perform HTTP PUT for transcode
`result in cache.
`
`- Return URI.
`
`Figure: sequence of operations for caching
`
`34
`
`34
`
`

`
`Patent Application Publication Jun. 30,2005 Sheet 34 of 37
`
`US 2005/0143136 A1
`
`Algorithmic SMF Seguence
`
`- For incoming object A calculate MD5. extract internal hash MIME
`type "D" from SQL database (indexed query).
`
`- If new object, put in database as original content. perform HTTP PUT
`for original object in cache.
`
`- If existing object, perform HTTP HEAD request to check if in cache.
`Object filename is
`Update last requested counter.
`
`- If in cache. return URI.
`
`- If not in cache, put the new input object A as file with the original
`content hash value "C" as the file name.
`
`Figure: sequence of operations for smart-forwarding
`
`Hash Database
`
`« Any SQL database can be used, e.g. MySQL.
`
`- Single indexed table (index on key value)! double indexed table (last
`requested update).
`
`- Periodic cleanup required of least accessed objects.
`
`- Table can reside in RAM — eg. for 10 Million cached messages. less
`than 4Gb of RAM required.
`-
`
`«» Can be implemented as an indexed linked list with no database if
`database performance issues arise.
`
`- Database scaling is simple based on hash value partitioning (e.g.
`server number 1 handles hash values less than TBD etc. ).
`
`Figure: Hash database connecting transcoded objects and original objects
`
`35
`
`35
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 35 of 37
`
`US 2005/0143136 A1
`
`{,3 35'
`
`MIVIS PROXY
`{ (3)m—seml.|'eq
`
`I\[ess:1ge 3 Iowa
`
`MCS
`' convert
`
`(5)
`
`m-semhconf
`
`(1) m-semI.req
`
`‘W.-KP Gateway
`
`
`
`MCS
`
`(2) Convert
`
`
`
`(I) MM 7-suIm1it.REQ
`
`I:H:|
`
`(5) M|\-‘I?-sulm1it.RE-S
`
`Figure XX: MM7 message interface to the UCnGO MCS
`
`36
`
`36
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 36 of 37
`
`US 2005/0143136 A1
`
`ifnj-3‘°
`
`«:wait for connection on TCP port 25>
`<open connection to server:
`22B mmsc.ucngomcs.net Simple Mail Transfer Service Ready
`BI-[LO cr1n.corn
`
`ZSEI mmse.uengomcs.net
`25!] SIZE 150000
`
`MAIL FROM: <:,_;my§;;;;;> SIZE=$DlJEIU
`ZSEI OK
`
`RCPT TO: <+§§Q§fiUl 234-5t57r'@_m_1I1sc opera§_qr.net>
`250 OK
`DATA
`
`354 Start mail input,‘ end with <CR:>«:LF>.< CR>«:LF>
`Date: 01 November 2001 [10:00 EST
`
`from: gm. com
`to: +
`U
`2
`
`56?‘
`
`mo erator.n
`
`subject: Newscast with text and animated gjf
`Come nt-typ e: multipartlmiaze d
`X-LgLg;;eMess'age-Class: personal
`x.;g1;;;;.Expiq:: 85 November 2E|E|2 IlJ:C|lJ EST
`X—;g1g;g5_—delivery4irne: 05 November 2002 USEUU EST
`X-15153;,-Priority: Normal
`—=_Pa.rt__lJ_9840S22.99??34?9?363
`Conce nt.Ty-pe: text.’pIai.n; s,na,c§,g§gu,§;5:,g;,i,'g
`Unisys launches MMS video services in Portugal
`_...=_Part_fl_984—£IS22. 99? 3' 3479? 3 I53
`Content-T399: W3s¢f ’
`Content-Disposition: attachment;
`—-—=__Part_fl_984D522.99??34?9?3fi3
`Content.Type: imagelfill’
`Cont: nt,-Transfer-Enco cling: b aseo-4
`jICYMTAyMQCNkIkagDElJNy4xMTcuMjASLjk3LlRZUEU9SVB2NACPgJ E3Lj
`
`bmcgdGHLIC.IQcmlvcm10eSIgZmLL‘oGQuIELOIGLzlHBldCBhcyAiTG93Ii4NC
`g==
`——=__Pa.rt_fl_984EIS22.99??-"34?9?3d3
`<CR><LF>.< CR?-<LI’>
`250 OK
`
`QUIT
`221 mmsc.ucngomcs.net Service closing transmission channel
`
`C;
`C:
`C:
`S:
`
`C:
`S:
`
`Figure: Sample MMS submission for transcoding from a VAS through MM?
`
`37
`
`37
`
`

`
`Patent Application Publication Jun. 30, 2005 Sheet 37 of 37
`
`US 2005/0143136 Al
`
`F.I..l.uD.I
`M.nN».n
`PMGGBJ
`
`JFEG JR
`MS
`Nalcua SM
`[FF
`MP3
`win:
`SM Ar-1R
`GSM A-.!»1R VH3
`MA
`
`f‘1EC‘{
`okna Rmamn
`urdel-ocl
`mmatad GIF
`
`4PEG4
`263
`!.I\'MV
`DF PSD
`S WORD
`
`4
`
`
`
`. ;,~M%u--.----.m%.Ililllllnllmfiflflj
`»;.1.2
`..WMNEAMMIH.
`
`
`
`_..__n._.cm
`
`Esnfizam
`
`_1m_
`
`EIII..IIIIIlIllIIIlE.
`
`
`
`
`EIIII.;IIIIllIllIIIIE.
`
`_....IIII-__IIIIIIIIlIIIlfl..
`
`
`.__
`
`
`
`......
`
`
`
`Forrnai Conversion
`
`Eros-5!-1e-dia
`
`38
`
`38
`
`

`
`US 2005/0143136 A1
`
`Jun. 30, 2005
`
`MMS SYSTEM AND Ml*'.'I‘HOD WITH PROTOCOL
`CONVERSION SUITABLE FOR
`MOBILEJPORTABLE HANDSET DISPLAY
`
`Rl_iI_A’I'l_7.D APP1.I(.‘A'I'[()NS
`
`[0001] This Application claims priority from co-pending
`U.S. Provisional Application Ser. No. 60i’299,745 filed Jun.
`22, 2001, which is incorporated in its entirety by reference.
`
`FII3.I..I)
`
`[0002] This disclosure teaches techniques related to an
`MMS {Multimedia Messaging System), including images,
`graphics, numerics, and text, suitable for display on the
`display of a mobile or portable communication handset
`terminal.
`
`BACKGROUND
`
`[0003] Glossary of Technical Terms
`
`To understand the disclosure better, the following
`[0004]
`definitions for technical
`terms used in this disclosure is
`
`provided:
`
`1. EMS: Extended Messaging System, a source
`[0005]
`protocol used for EMS handsets, designed to encode mul-
`timedia messages, Including images, graphics, numerics,
`animations, audio and formatted text.
`
`2. MMS: A pre-source (multimedia+formatting
`[0006]
`information) protocol used to encode types of messages,
`including images, graphics, numerics,
`and text,
`and
`transcoded for the display phone speaker on various display
`terminals. Used in most Nokia handsets.
`
`3. PM: Picture Messaging protocol, a graphic for-
`[0007]
`mat (source protocol) used to display SEW Images in Nokia
`handsets supporting the NSM per source format.
`
`In this application,
`4. Pre—source information:
`[0008]
`information, which may be a full multimedia message or
`some part thereof, which appears in a non-source format and
`is not coded in a source protocol. Pre-source information
`refers to “packaged" multimedia content in a "raw" format
`such as:
`
`a. A set of TCPIIP packets composing a MIME
`[0009]
`rnultipart message {could be an email message or an MMS
`MM1 message), where some parts of the message are media
`objects which need to be convertedftranscoded, and some
`other parts (eg. SMIL attachment) are presentation layer
`Information relating to how the information has to be
`arranged and displayed Figa. 24 and 25 illustrate this con-
`cept,.
`
`b. Ablock of SMS messages that together comprise
`[0010]
`an EMS or a Nokia Smart Messaging (NSM) message and
`contain multiple media objects—pictures. ringtones, etc.
`These SMS messages are further encapsulated into the
`SMSC protocol which can be SMPP, UCP, CIMD etc.
`
`5. Smart Messaging: A source protocol being
`[0011]
`developed by Nokia for Nodia handsets. This refers to
`everything defined in the NSM pre-source protocol and adds
`functionality for calendar events (vCalendar), electronic
`business cards (v(_‘ard) etc.
`
`infor-
`In this application,
`6. Source information:
`[0012]
`mation, which may be a full multimedia message or some
`part thereof, which appears in a source format and which is
`coded in a source protocol. Typical source protocols are
`WI3MP, EMS, and PM, but new protocols are being devel-
`oped on an ongoing basis. Source protocols enable the
`display of messages on terminals with limit memory, pro-
`cessing, and display capabilities, such as those of mobile and
`portable radio communication handsets (i.e. cellular tele-
`phones,
`land mobile radios, Instant Messaging terrninals.
`radio enabled PDAS, and the like). On the other hand,
`source information constitutes media objects in some media
`format, eg. a JPEG picture. an MP3 audio file, an AV] video
`CIC.
`
`7. Transcoding: To perform protocol conversion,
`[0013]
`either from one source protocol
`to another. or from a
`pre-source protocol into a source protocol.
`
`8. WAP, “Wireless Application Protocol”, one pro-
`[0014]
`tocol used for what have been called 2.5G cellular systems.
`In the cellular world, 1G was the original set of analog
`cellular systems lG has been mainly displaced by 2G
`systems, which are low-speed digital systems, which a
`typical raw data rate of 9.5 kbps. Operators are currently
`deploying what are known as 2.5G systems, which are
`higher speed digital systems, expected to operate up 384
`kbps 2.56 systems are expected to be replaced by 3(i
`systems, which are higher speed digital systems promising
`speeds up to 2 Mbps, WAP is one of the chief manifestations
`of 2.56 systems WAP Is a pre-source protocol.
`
`9. WBMP: The display protocol for handsets in the
`[0015]
`WAP system.
`
`[0016]
`
`Introduction
`
`[0017] The process of transcoding is not a new idea.
`Indeed, in forms the basis of communication systems. Even
`the conversion from analog to digital, or vice verse, is a form
`of transeoding. With the proliferation of higher speed digital
`cellular systems, the challenge and the problem oftranscod-
`ing have become much greater. There is, as yet, no standard
`display protocol for higher speed communication terminals.
`Therefore, transcoding from one display protocol to another
`is required to insure that the receiving tenninal will be able
`to display the transmitted. There is, however, no method or
`system to do this in such a way that the integrity and quality
`of the transmitted message will he maintained in the display
`tcnninal. Further, there is no method or system for transcod-
`ing non-source information into a source protocol suitable
`for display on a communication tennina], while maintaining
`the integrity and quality ol’ the irtforrnation which originally
`appeared in the non-source format. There are transcoding
`systems and methods, to be sure, but they are primitive, and
`lose much of the quality of the source or pre-source infor-
`mation, even to the point where in some cases the displayed
`information in the display terminal is not recognizable. What
`is required is algorithms, a method, and a system, that will
`allow identification of the specific display characteristics of
`the target display terminal, and will also allow the source or
`pre-source information to be displayed on the target display
`tenninal with the maximum amount of integrity and quality
`in comparison to the pre-coded information.
`
`SUMMARY
`
`[0013] The disclosed teachings provide for:
`
`1. Conversion of source information coded in a
`[0019]
`source format into a protocoi suitable for transmission to and
`
`39
`
`39
`
`

`
`US 2005/0143136 A1
`
`Jun. 30, 2005
`
`display on the terminal A variety of new processing tech-
`niques are disclosed, Source information is typically coded
`in protocols such as WBMP (the protocol
`for Wireless
`Application Protocol, or “W/\P", systems), EMS, and PM.
`This infonrtation must be transcoded for display on dilferertt
`terminals, also using source protocols, but where the pro-
`tocols and variations of the protocols are typically different
`between the input source and the display terminal.
`
`is,
`that
`2. Conversion of pre-source information,
`[0020]
`information which is coded but not in a source protocol, into
`a source protocol. For example, an ordinary digital picture
`will be transeoded into the source protocol WBMP. It will be
`appreciated that information in source protocol will then be
`transcoded again into the
`target
`source protocol,
`as
`explained in introductory point 1 immediately above.
`
`[0021] An MMS communication system for displaying
`images on a display terminal of a mobile or portable
`communication device,
`the system comprising: an input
`adapted to receive pre-source information; a transmitter
`adapted to transmit
`the pre-source information;
`a server
`adapted to receive the transmitted pre-source information
`and further adapted to convert the pre-source information to
`source information suitable for display on the display ter-
`minal; and a source transmitter adapted to transmit
`the
`source information to the display terminal
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0022] The above advantages of the disclosed teachings
`will become more apparent by describing in detail preferred
`embodiments thereof with reference to the attached draw-
`
`ings in which:
`
`[0023] FIGS. 1-3? show various features of the disclosed
`teachings as described in the rest of this document.
`
`DETAILED DESCRIPTION
`
`[0024] Overall Architecture
`
`[0025] An implementation of the disclosed teachings is
`shown in FIG. 5. The structure includes the input devices
`5.11-5.13 on the left, the server 52 represented by the block
`in the middle, and the display devices 5.31-5.34 shown on
`the right information may be in source format, as is the case
`for the cellular telephone 5.12 (picture of two people) and
`the digital camera attached to a cellular telephone 5.13
`(picture of the automobile) Or information may be in a
`pre—source format, such as the cartoon of the man 5.11. At
`the server, the source information or pre-source information
`is processed by a variety of components, adapted to imple-
`ment a variety of algorithms or techniques, which form an
`integral part of the disclosed teachings.
`
`[0026] Some of the components mentioned above perform
`at least the following tasks:
`
`transcoding (from the pre—sourec
`1. Format
`[0027]
`information into source information, or from source
`information into other source information suitable
`
`for display on the display tenninal);
`
`2. Image adaptation, to adapt the image to the
`[0028]
`particular display screen on the display terminal,
`{This is discussed in further detail in section IVG);
`
`original information. The information must be com-
`pressed, both for transmission and for display.
`
`4. Photo enhancement: Specific sections of a
`[0030]
`photograph may be cutout and enhanced. (This is
`discussed in further detail in section IVG);
`
`5. Content based processing. by which differ-
`[0031]
`ent aspects of a multimedia message are identified
`and processed dilferently. (This is discussed in fur-
`ther detail in section IVG);
`
`6. Recognition front images: This is allied to
`[0032]
`the photo enhancement algorithm. Diflcrcnt portions
`of an image
`are
`recognized and "cutout” for
`enhancement. (This is discussed in further detail in
`section IVG);
`
`7. Interfaces to 3rd Party Applications: Third
`[0033]
`party applications may be processed separately and
`sent to the display terminals, or may be added to the
`original information. In addition, if there are soft-
`ware packages with additional algorithms for addi-
`tional processing of the source information, these
`may ahe accessed and applied to the original infor-
`mation for eventual display on the display tenninals.
`An example of an interface to a third party applica-
`tion is an XML-based interface over TCPIIP.
`Another example of such an interface would be and
`API in C++ or Java. By third party application we are
`referring to both external SW modules, and to
`V/-\S—value added services—e.g. a news service, a
`gaming platform for cellular phones (for example
`www,wirelessgames.eom, www.cash-u.com),
`a
`photo album service. These applications wish to send
`MMS content and also in certain cases to perforrrt
`special processing on such content prior to sending
`it. For a review of the special functionality please
`refer to section IV. I-I.
`
`[0034] Another implementation of the disclosed teachings
`is shown in FIG. 6. Examples of input sources, called
`“Content Sources”, 6.11-6.14, appear in the column at the
`right. However,
`the input sources are much broader than
`these pictures. At
`the bottom of the ligure are various
`information devices 6.21-6.24 which can serve as both
`sources of information to the server 63, and also receivers
`of information processed by the server. These information
`SDt.ll'(.‘t:S can be WAP or i-Mode Phones, called “MMS Box"
`in the slide. (i-Mode phones are those that operate on i-Mode
`2.5G cellular systems currently functioning in Japan. i-Mode
`phones will also operate on 3G systems expected to be
`introduced in Japan in late 2001 and in 2002.) Picture
`messaging phones, operating with the PM protocol, are
`portrayed in the “Picture Box". Similarly, EMS phones,
`operating with the EMS protocol, are portrayed in the “EMS
`Box”. Finally, Email enabled phones are portrayed in the
`“E-mail Box“. In this implementation, information may be
`coded out of or coded into, any of the protocols show,
`including WAP phones (WBMP protocol), I-Mode Phones
`(the Japanese version of WAP), Picture Messeging Phones
`(PM protocol), EMS Compliant Phones (EMS protocol),
`and E-mail Capable Phones (POP3. SMTP, and IMAP4
`protocols).
`
`3. Optimal compression for handset. The dis-
`[0029]
`play tenninal cannot display all of the bits of the
`
`[0035] The server will receive, transcode, and optimize for
`display on a specific communication terminal, at least any of
`
`40
`
`40
`
`

`
`US 2005/0143136 Al
`
`Jun. 30, 2005
`
`the following protocols: ll’, SMPP, TCPIPT POP3iSMTPt’
`IMAP-HXML This is illustrated in FIG. 7.
`
`[0036] Using the transcoding several tasks are accom-
`plished, for example:
`
`1. The conversion from and into any of the various
`[0037]
`formats WBMP, PM, and EMS.
`
`2. The conversion of formats on the lly when the
`[0038]
`user logs on to his or her MMS box {so that even if the user
`were to switch terminal devices, he or she would get the
`correct content, properly formatted).
`
`3. The conversion taking into account the exact
`[0039]
`parameters {such as, for example, screen size, pixel dimen-
`sions, etc.) of the panicular terminal and terminal display.
`
`4. Transcoding of source information into other
`[0040]
`sou ree information suitable for display on a target terminal,
`maintaining integrity and quality of the original message,
`examples would be:
`
`a. A .|PE(i image is converted to a (ill-'-' image so
`[0041]
`that a phone with a WAP browser than can display GIF
`Images will be able to view it.
`
`b. A Nokia ringtone is converted to an EMS
`[0042]
`iMelody ringlone so that a non-Nokia phone can play it.
`
`is convened to
`c. A video in MPEG1 format
`[0043]
`Ml-’l'_'l(i4 Simple Visual profile so that an MMS compliant
`phone can display it, or to an animated GIF sequence so that
`a non-MMS compliant (legacy) phone can view it.
`
`d. Formatted text in the EMS format can be con-
`[0044]
`verted to an image or to I-lTMI_.+tex1 to preserve the for-
`matting (underline, bold, letter size etc.).
`
`It should be noted that both the EMS and NSM
`[0045]
`formats include not only images but also ringtones, anima-
`tions and formatted text. This fact is well documented in the
`EMSINMS standards documents for the last few years.
`
`5. Transcoding of non-source information into
`[0046]
`other source information, for later transcoding into source
`information for display on a specific target terminal, where
`all transcoding maintains the integrity and quality of the
`original message.
`
`6. Recognizing the specific display characteristics
`[0047]
`of a specific target display terminal, to enable the display of
`high quality messages, whatever the characteristics of the
`terminal.
`
`[0043] An Example Implementation: Overview
`
`[0049] FIG. 8 shows another example implementation of
`the overall system. This implementation is called MMR. The
`MMR system allows users to send and receive messages
`containing text and images at least in the following formatsr’
`protocols: WAP; PM; EMS; MMS; E-MAIL; WEB; SMS.
`
`[0050] While the tenn message and information has been
`used in discussing the implementations in detail, it should be
`clear to a skilled artisan as to which mcssageiinformation
`constitutes pre-source information and which constitutes
`source information according to the definitions for the same
`provided in the background.
`
`[0051] User WAP Pages
`
`[0052] The MMR provides a WAP based messaging appli-
`cation, allowing users to logjn to their personal messaging
`page. From this page users can view and send messages in
`
`variety of formats. The MMR sends the WAP recipient an
`SMS notification with a link to the newly received message.
`Alternatively, the MMR can send a WAP push message with
`the same link. MMS recipients receive a notification to the
`new MMS. PM and EMS recipients receive the message
`directly. Email recipients receive an e-mail with an image
`attachment to their regular e-mail address.
`
`It should be noted that WAP pages can contain
`[0053]
`multimedia for immediate integrated dis