as) United States
`a2) Patent Application Publication co) Pub. No.: US 2007/0094609 Al
`(43) Pub. Date: Apr. 26, 2007
`
`Gilboa et al.
`
`US 20070094609A 1
`
`(54)
`
`EXECUTABLE AND DECLARATIVE
`SPECIFICATION FOR GRAPHICAL USER
`INTERFACES
`
`(75)
`
`Inventors: Yuval Gilboa, Pardesiya (IL); Rinat
`Gilboa, Pardesiya (IL)
`
`Correspondence Address:
`TOWNSEND AND TOWNSEND AND CREW,
`LLP
`TWO EMBARCADERO CENTER
`EIGHTH FLOOR
`SAN FRANCISCO,CA 94111 (US)
`
`(73)
`
`Assignee: SAP Portals Israel Ltd., Ra’anana (IL)
`
`(21)
`
`Appl. No.:
`
`11/324,155
`
`(22)
`
`Filed:
`
`Dec. 29, 2005
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/722,888, filed on Sep.
`30, 2005.
`
`Publication Classification
`
`(51)
`
`Int. CL
`(2006.01)
`GO6F 3/00
`(2006.01)
`GO6F
`7/00
`(2006.01)
`GO6F 9/44
`(52) US. Che
`caccecssssssssensen 715/762; 707/102: 717/104
`
`(57)
`
`ABSTRACT
`
`Techniques for decoupling models created during design-
`time from the runtime environment. A declarative and
`executable representation for GUIs for applications is pro-
`vided that is independent of any particular runtime platform,
`GUI framework, device, or programming language and
`whichacts as an interface between the design environment
`and the runtime environment.
`
`EXAMPLE #1: BANKING APPLICATION GUI (FOR DHTML RUNTIME PLATFORM )
`Bank Address
`
`
`
`{|Search|} |]Clear_]]
`
`1 Pobk Curac:
`[] Bank No:
`[] AdarNo:
`[[Bank Brench:|Main Branch
`BanksList
`[ Bank Group:
`I
`[| Bank Name:
`BANK.CTRY
`Low.
`us
`[ Region:
`US
`
`[acy
`[ Hox Rows
`
`Celate
`4
`
`BANKKEY
`
`3
`
`&
`BANKNAME
`First Union Bank & Trust
`Citibank
`
`[ Bank Name: Chase Manhattan 84
`
`123445678
`
`Chase Manhattan Ba
`ouYou Oy
`
`[ Switt Code:
`[] Post Bank:
`
`ABCD3891
`
`Chicago
`New York
`
`
`
`12345678
`Commerce Bank
`Philadelphia
`PNG
`123456780
`Martton
`Chase Manahattan
`123456784
`New York
`Mellon Bank
`123456788
`Philadelphia
`123456789
`us
`Fidelity Bank
`Los Angeles
`
`Us
`
`
`
`AEE HEE)
`
`12
`
`[| Bank Cty:
`
`[ Bank Key:
`
`] city:
`
`Page 1 of 40
`
`GOOGLEEXHIBIT 1016
`
`Page 1 of 40
`
`GOOGLE EXHIBIT 1016
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 1 of 16
`
`US 2007/0094609 Al
`
`DATA
`PROCESSING
`SYSTEM
`102
`
`DESIGN-TIME
`ENVIRONMENT
`
`MODELING
`TOOL
`
`
`
`ABSTRACT REP. GENERATOR
`XGL GENERATOR)
`
`ABSTRACT REP.
`(XGL REP.)
`
`100
`
`x
`
`4104
`
`106
`
`408
`
`DEVICE AND PLATFORM
`SPECIFIC RUNTIME TOOLS
`
`j-----
`
`en ene
`
`
`
`J||
`
`
`
`
`
`XGL > DHTML
`XGL > FLASH
`COMPILER
`INTERPRETER
`
`i XGL > JAVA
`112-1 | COMPILER
`
`114
`
`JAVA CODE
`
`112-2
`
`FLASH CODE
`
`120
`
`112-3
`
`116
`
`{22
`
`1
`
`26
`
`JAVA
`RUNTIME
`
`FLASH
`RUNTIME
`
`DHTML
`RUNTIME
`
`118
`
`124
`
`128
`
`GUI ON JAVA
`PLATFORM
`
`GUION FLASH
`PLATFORM
`
`GUI on DHTML
`PLATFORM
`
`J
`
`ENVIRONMENT
`
`FIG. 1
`
`Page 2 of 40
`
`Page 2 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 2 of 16
`
`US 2007/0094609 Al
`
`
`
` MODEL REPRESENTATION
`
` 106
`(COMPRISING MODEL GU! COMPONENTS)
`
`
`
`USING AN ABSTRACT REP.
`GENERATOR (XGL REP.
`
` GENERATOR)
`
`
`
`ABSTRACT REPRESENTATION
`
` 110
`
`(COMPRISING ABSTRACT COMPONENTS
`AND ASSOCIATED PROPERTIES)
`
`(XGL REPRESENTATION)
`
`
`
`
` IN RUNTIME
`
`ENVIRONMENT RUNTIME
`
`RUNTIME
`REPRESENTATION
`(TARGET DEVICE
`SPECIFIC)
`
`202
`
`REPRESENTATION
`(TARGET DEVICE
`SPECIFIC)
`
`FIG. 2
`
`202
`
`Page 3 of 40
`
`Page 3 of 40
`
`

`

`Patent Application Publication Apr. 26, 2007 Sheet 3 of 16
`
`300
`
`RECEIVE A SIGNAL TO IDENTIFYING A MODEL
`REPRESENTATION
`
`302
`
`GENERATE AN ABSTRACT REPRESENTATION (XGL
`REPRESENTATION) FOR THE MODEL
`REPRESENTATION
`
`304
`
`US 2007/0094609 Al —
`
`PERFORM RUNTIME ENVIRONMENT PROCESSING
`USING THE ABSTRACT REPRESENTATION
`GENERATED IN 304
`
`306
`
`FIG. 3
`
`Page 4 of 40
`
`Page 4 of 40
`
`

`

`Patent Application Publication Apr. 26, 2007 Sheet 4 of 16
`
`US 2007/0094609 Al
`
`EXAMPLE#1: BANKING APPLICATION GUI
`
`MODEL REP.
`(VISUAL REP. DEPICTEDIN
`
`402 Fic.48)
`
`XGL GENERATOR
`
`404
`
`XGL Rep.
`
`406
`
`XGL > JAVA
`
`XGL > FLASH
`
`XGL > DHTML
`
`Raa
`
`412
`
`FLASH
`
`420
`
`DHTML
`
`RUNTIME
`
`426
`
`GUI on JAVA
`PLATFORM
`
`414
`
`422 (FIG. 4C)
`
`GUI ON FLASH
`PLATFORM
`(FIG. 4D)
`
`(FIG. 4E)
`
`FIG. 4A
`
`Page 5 of 40
`
`Page 5 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 5 of 16
`
`US 2007/0094609 Al
`
`
`
`
`
`TACONOILVONTddVONDINVG
`
`
`
`siiejaqyueg
`
`a?‘Sld
`
`sseippyYUE1S]syuegUo
`
`
`
`lEJaqYued>SSalppyyuegjndujdSoeInoJ
`
`
`WO4yee
`
`{SIPYUeg
`
`
`
`yoeagsyueg
`
`Page 6 of 40
`
`Page 6 of 40
`
`
`
`
`
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 6 of 16
`
`US 2007/0094609 Al
`
`QLOSHECL
`
`
`
`aein9400¢|]
`
`onveg|]
`
`
`
` yueg|OnJppy|] yueg|dnoigyueg|young
`
`yuegueyeyueyyeseyo
`
`
`AIDHOAMEN
`yangueW
`Leseqogy
`
`yueg180d|9po9YING|veg|uo}éeu|Avo|awen
`
`yoees (WYOSLV1d
`
`
`SWILLNAYYAPYO)INDNOLVONddONIMNVG‘4SIdWWXA
`
` euepewa|[__wegeamuueo]Sao[___azaserez|ee)aaa[occa]Lo]
`oo1ee
`
` SaHO)MENyuegVEYeyUeW]BseyD
`
`zi,abe=)=D11(=)
`AYO0),MANuegueyeyueWaseydS29GPbE71[sal
`
`
`
`
`oP‘Dld
`
`
`
`Ap|awenyueg|fayyueg|]Angyueg|
`
`
`
`Silejeqyueg
`
`as
`
`Isn»Angyuegf
`
`
`
`1S]Syueg
`
`
`
`sseippyyueg
`
`Wwo4
`
`Page 7 of 40
`
`Page 7 of 40
`
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 7 of 16
`
`US 2007/0094609 Al
`
`egueeyueWyaseu
`
`
`‘AWENyUeg
`
`Al0,MAN‘AIO
`
`suolBey
`
`
`
`“youelgyueg
`
`
`
`‘drossyueg
`
`‘ONJppy
`
`B96PrEZI
`
`‘ONyueg
`
`
`
`‘oBINDyqOd
`
`
`
`ssaippyyueg
`
`
`
`‘queg}s0g
`
`syueg (WHOSLV1dSWILNNY
`
`
`
`HSV14YO)INSNOLLWOlIddySNINNVG:1#SIdWVX9
`
`yangueJeans
`
`
`AYOOAMANyuegVeneyueEyaseyp
`
`
`legseqdav‘8po9WINS
`
`
`eIydyapelygyuegaaawwod
`
`WO,MONued
`
`yen'pyuegUOlUr)sly
`
`
`
`4S!syueg
`
`—S—_——__—>
`rrerrrnnaSMOYXE
`
`“ANQyueg
`
`youeas
`
`Page 8 of 40
`
`
`
`WO,MAN
`
`uoILEW
`
`BIydjapenyg
`
`
`
`sajabuyso7
`
`UO}sno}y
`
`
`
`uepeyeueyaseyd
`
`UOISNOHJoyUeg
`
`
`
`yuegLOyayy
`
`yuegAaply
`
`ONd
`
`
`
`Ob‘Sld
`
`ApHO,san
`
`
`
`bgueyeyueWasey9
`
`
`
`AID‘aweyUeg
`
`S2aSPree
`
`shayyueg
`
`Page 8 of 40
`
`
`

`

`“Agyueg|] (WHOSLVTdSNINAY
`
`
`
`
`TWLHCGYO)INDNOLLVOlddlSNDINVG‘14JIdWWXA
`
`gOzgcrezt|weamon|ubyeyeueyyvarasvezt|sseyud|
`
`
`sejebuyso76BL95RE7Z1
`eiydjaperygyuegUOIa\
`
`
`
`
`B29S7hE7}
`
`
`
`“22iN9400d|
`
`“ONs9py|ONUeg|]
`
`
`
`ssolppyyueg
`
`
`
`
`
` uleyyyeanslN-worbau|] bégcaoay:9poyYIMSljeans
`
`
`
`aqueg30d|]
`
`dP‘Sls
`
`
`
`ANDoAManegUENeYUeWaseyd
`
`m9|]cawienued|]
`
`yuegAyjapi4
`
`SL9Sprect
`
`:foyyueg[]
`
`
`
`sirejegyueg
`
`
`
`ENE)N=N=1
`
`wnIwn
`
`D>
`
`Patent Application Publication Apr. 26,2007 Sheet 8 of 16
`
`US 2007/0094609 Al
`
`
`
`sOWweN
`
`
`
`yueg|] yueg]‘dnayueg|“youesg
`
`ANDYO,MONAND|egueHeyeyWeseuy
`
`Les|
`
`[|
`
`or
`
`swoyx2[]
`
`
`
`3s]syueg
`
`w3ALON!
`
`D>]
`
`WOina
`a]>3
`
`Page 9 of 40
`
`Page 9 of 40
`
`
`

`

`Patent Application Publication Apr. 26, 2007 Sheet 9 of 16
`
`US 2007/0094609 Al
`
`EXAMPLE#2: ANALYTICS DASHBOARD GUI
`
`
`
`MODEL REP.
`(VISUAL REP. DEPICTED IN
`FIG. 5B)
`
`502
`
`XGL GENERATOR
`
`504
`
`XGL Rep.
`
`506
`
`XGL > JAVA
`
`XGL > FLASH
`
`XGL > DHTML
`
`Ryne
`
`512
`
`FLASH
`
`920
`
`DHTML
`
`RUNTIME
`
`926
`
`(FIG. 5E)
`
`GUI ON JAVA
`PLATFORM
`(FIG. 5C)
`
`514
`
`GUI ON FLASH
`PLATFORM
`(FIG. 5D)
`
`522
`
`GUI on DHTML
`PLATFORM
`
`528
`
`FIG. 5A
`
`Page 10 of 40
`
`Page 10 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 10 of 16
`
`US 2007/0094609 Al
`
`HEYSaur]
`
`
`
`
`
`TACOP]GYVOEHSVSOILATVNY
`
`
`
`suey)ueiseyey
`
`
`gige|SIapiCsajesa
`
`ISUBPICsajeSynduj>
`
`©WINIOY
`
`
`
`}siJ}99Jopsosajes
`
`LB@
`
`
`
`Salesyoleas
`
`Page 11 of 40
`
`Page 11 of 40
`
`
`
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 11 of 16
`
`yoreas (WHOL1dSWILNAYWAWPHO4)
`
`00000"!PeAueuny[2Gyorg[SpHrenvon|Senenon|Seougien[SAobey
`
`[una|000007|ao'érr600'6rr6
`Pana|ovo00:|Pana|00000"|wna
`
`COl-d=60010d_——ODI-dcow
`
`
`INDGUYVOSHSVGSOILATVNY:2#JIdWYX3
`
`fb0hodd=OPELc0r-d
`
`0000002=—O0000S!—d000001
`
`og'eB0ges09'P9682
`
`00°000100:00)
`}seuegco¢saulagca
`|sauegom¢salesoO
`026292000'9¢|owsive|o00'8e
`
`
`—0002'60'20|narAung|cow]~—nka000]+=aepg0n00D0]Oo||__c00z'60'z0|weakuung|cow]
`
` 1002'S0'82p>Mona}wervous|2evan|S“onun]2sopst|aiqe,eco]?
`
`
`
`ogo000|—_assson0000}o|Q3udaduingoloo00|@er9000000[oO|ueysaduing0rdovoo0e|—_-6ssg00c000/oO|O3ud
`
`
`
`€Ob-d60010d000}-dcowLhd20h”
`
`COb-d=000dODI-dcowbOl-d20b-d
`601-dEL0bOd0=—sOFEL-20rdPOld==10ddVININ
`
`
`
`
`60l-d=CLOLDdd=OLY20b-dv0l-d
`
`
`zsauag«|SaagaESalaswa
`
`
`
`
`O3udadung20b-doroo00|6ss9000000adwngbOl-d020000|—_669000000
`
`
`qseuegoS=|Sellaga
`
`
`VWUZ|10ddVININ0100008869000000
`[0004JBigsores]
`eee
`CLP
`
`US 2007/0094609 Al
`
`qsauagES
`
`sauobaye9
`
`60l-d
`
`Old
`
`20l-d_
`
`Old
`
`LO”dlINIW
`
`souoBayep
`
`¢saulegoa
`
`os
`old
`
`pftaBeg
`
`Og"BB0ges
`
`Oc'erecel
`
`Og'B80ges
`
`O¢'08900¢
`
`00°0001
`
`oO2e0¢
`
`06bP6
`
`0002'b42)
`
`000e'HhAL
`
`0002"Zt
`
`O002'bLAt
`
`2wo°=aa
`
`ved
`
`yeyseq
`
`7salag&O
`
`SOn|eA
`
`squobayeg
`
`ooogose
`
`000005
`
`10ddVININ
`
`yey)aut
`
`weysuunjo5
`
`ONJowojsn5]]
`
`0001000000
`
`sales
`
`
`
`siapigsajes
`
`Page 12 of 40
`
`Page 12 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 12 of 16
`
`‘24J1dWYXS
`
`
`8969000000|paung|SyaxgpHeAwN|ameAyen|amaINMobay|ADAGfaeaood|meruous|eweew|ONUA[20aps|“Lo0004]
`
`
`
` wlnaltyocoroo;[CTC“JOSE|HOddeMoeWuy|ddVINIW010000
`
`
`
`|-9'ga09es"LE|una|+igeeoces|cece|26192ovSpPoozLuna]st|9'@809¢5powe|66|BEpoozbh2}[POLedwng|una]stT-9°eg09esg'00zE}NOISIOSYdtz)|OLNOISIOSudSduing
`
`
`
`
`
`WeewreWeWotWeb008OOF6DedDDDPIDDDD|NEONUNaaSSaNLGN
`gyoegs|eee[|e|KOOEAL
` |uns{ss:
`
`SD>DDBUBVBUB_MIDBBV3dDLBVPwWWS
`
`eSeee|una{ek|Gb
`
`
`
`(WHOSLY1dJNLLNNYHSY14YO)INDGUVOSHSVGSOILATWNY
`
`
`eseuegm> zseveses|sevegcoEsayess3
`
`Zsauagsa=|Saasa€s9l0gSI
` \
`
`Py&3BdNID
`SESSS88
`
`
`
`SHHKGPagasGFGus
`
`201NOISIDSYdBOWNd
`
`
` 659000000|Obed=f0F0000_6559000000_||POld959000000_||cOkd|OL0000_bSS9c00000_|
`[020000
`Leow_|€73000000[|€ow|Q10000_Be?3000000
`
`ddVeddd.d-
`SKgrigisissye”
`0rd00000
`
`£O-W0100006€¥9000000
`*6001.000000
`
`US 2007/0094609 Al
`
`as‘Old
`
`Zsoyegma
`}savesZa
`
`\
`
`£savags3
`
`ZSaaswa
`|soyes2a
`
`HEYSULIN|OD
`
`s
`
`GOsees
`
`
`
`syeysdueisaye)
`
`[ONJewojsn4
`
`‘SeesYeas
`
`Page 13 of 40
`
`Page 13 of 40
`
`

`

`US 2007/0094609 Al
`
`2LOddvINiW
`
`yey)aur]
`
`Conooeez
`
`o'onoo02
`
`0'0000261
`
`4S‘Sid
`
`Patent Application Publication Apr. 26,2007 Sheet 13 of 16
`
`PONJaut0}sn5[] SayeS
`
`
`INNOfroHOXaHWAaN[|3MTWAI3NpidaNloosefuvecog[SsxalyonspaiwwBaNANWL!|3oogas|.81921fuoana
`
`
`
`
`
`
`"0|una [oocoor|ogeeoccs|—os'oozecs|orsire|ovoee|o00eehooziii2)23udeduing9200006969000000
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`|_¥na|ood:|osrezp|OSbezyTosrre|ons[0fooweouo[tLums|OW(ot00009€r9000000|__una|00000700'6h+605br6coon;|0_-foowsozo=|=Aung|OWL0006tr9000000|una|00000702'099002OPSaie00's|00096$99000000|una|ooo00}|oge909es6g5900000050j5:O2'erzezh026292000°9rO0'SrpoogbiztO3udedwing¢000009'88096SpooeteLuesaduindop000000°66
`
`Cd=6Ol-dOdd0L0d0Md7COW20rdWhdwldZOldVININOld=GDFddOdd0Ida
`
`
`
`
`(WHOSLV1dSWILNAYTWLHGYO)INDGYVOSHSYGSOILATYNY‘2#STdWVXy
`]
`
`
`
`10000260'0000rbL07000096000008i.SOW2rdLOldPol=201-dVININ
`peysregyeyealy
`
`
`Lees}=(_)DogLOOoou0
`rhigsaesf)
`
`
`
`S39P1OSa/eS
`
`Youeag
`
`AAV.
`
`EEE
`
`Page 14 of 40
`
`Page 14 of 40
`
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 14 of 16
`
`US 2007/0094609 Al
`
`>
`
`.
`<?xml version="1.0"?>
`<xg:Componenturn namespace-decl___
`many
`<xg:Window...>
`contanerdeested
`interactor-dec! an
`control-deciy
`
`</xg:Window>
`<xg:Data>
`infosets-section
`infoactors-section
`operators-section
`enumerations-section
`</xg:Data>
`<xg:Controller>
`actions-section
`relays-section
`execution-plans-section
`</xg:Controller>
`|
`<xg:Resources>
`images-section
`palettes-section
`translations-section
`systems-section
`imports-section
`</xg:Resources>
`</xg:Component>
`
`FIG. 6
`
`Page 15 of 40
`
`Page 15 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 15 of 16
`
`US 2007/0094609 Al
`
`<?xml version="1.0"?>
`<xg:Component urn="AAD6BU"xmins:xg="http:/www.sap.com/visualcomposer/2005/xg!"
`xmins:ep="hitp:/www.sap.com/visualcomposer/2005/xgl/portal">
`<xg:Window windowTitle="Wizard1" windowType="top" preferredWidth="760"
`preferredHeight="480"frameStyle="primary">
`<xg:Canvas enableScrolling="true" scaleMode="fix">
`<xg:Wizard defaultChild="ABA6BV" roadmapStyle="full" frameStyle="none">
`<xg:PanelframeStyle="primary" w="320" h="160" x="344" y="16"
`showTitlebar="true">
`<xg:FormView id="ACA6CL"infoset="AAA6CL" title="Form5" scaleMode="fix"
`formLayout="absolute" formWiath="184" formHeight="64"isEditable="true">
`<xg:Controls>
`</xg:FormView>
`</xg:Panel>
`</xg:Canvas>
`</xg:Window>
`<xg:Data>
`<xg:Infosets>
`<xg:Infoactors>
`- <xg:Operators/>
`<xg:Enumerations>
`</xg:Data>
`<xg:Controller>
`<xg:Actions>
`- <xg:Relays/>
`<xg:ExecutionPlans>
`</xg:Controller>
`<xg:Resources>
`- <xg:Images/>
`- <xg:Palettes/>
`- <xg:Translations/>
`<xg:Systems>
`- <xg:Imports/>
`</xg:Resources>
`</xg:Component>
`
`FIG. 7
`
`Page 16 of 40
`
`Page 16 of 40
`
`

`

`Patent Application Publication Apr. 26,2007 Sheet 16 of 16
`
`US 2007/0094609 Al
`
`809
`
`COMPUTER
`SYSTEM
`
`MEMORY SUBSYSTEM
`
`FILE STORAGE
`SUBSYSTEM
`
`USER INTERFACE
`INPUT DEVICES
`
`818
`
`Bus SUBSYSTEM
`
`800 STORAGE SUBSYSTEM
`
`p
`
`ROCESSOR(S)
`
`NETWORK
`INTERFACE
`
`USER INTERFACE
`OUTPUT DEVICES
`
`NETWORKS, SYSTEMS, DEVICES
`
`FIG. 8
`
`Page 17 of 40
`
`Page 17 of 40
`
`

`

`US 2007/0094609 Al
`
`Apr. 26, 2007
`
`EXECUTABLE AND DECLARATIVE
`SPECIFICATION FOR GRAPHICAL USER
`INTERFACES
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`‘The present application is a non-provisional appli-
`[0001]
`cation claiming benefit under 35 U.S.C. 119(e) of U.S.
`Provisional Patent Application No. 60/722,888 (Attorney
`Docket No. 017900-005900US)
`titled EXECUTABLE
`
`SPECIFICATION OF GRAPHICAL USER INTERFACES
`(XGL)filed Sep. 30, 2005, the entire contents (including all
`the appendices) of which are herein incorporated byrefer-
`ence for all purposes.
`BACKGROUND OF THE INVENTION
`
`invention relates to graphical user
`[0002] The present
`interfaces, and more particularly to techniques for facilitat-
`ing creation of graphical user interfaces for different runtime
`environments.
`
`information is com-
`In the computing domain,
`[0003]
`monly stored in the form of models. For example, during the
`design phase, a designer may use a modeling tool to design
`an application. As part of the application design, the mod-
`eling tool may generate a model representation that stores
`information for the application including information about
`graphical user interfaces (GUIs) associated with the appli-
`cation. The model maystore information related to various
`entities or components of the application,
`information
`describing inputs or outputs of the components, information
`describing dependencies between the various components
`and their input and outputs, a component’s response to
`signals, the runtime environment, and other information for
`the application. The model that is used for storing applica-
`tion information may be configured by the designer after
`taking into consideration various factors such as the domain
`in whichthe application is to be used, the memoryresources
`available for
`storing the information,
`the computing,
`resources available for processing the model, tools to be
`used for manipulating the model, and other factors. A model
`representation (also referred to as canonic model represen-
`tation) thus provides a convenient way Lo persist information
`for an application in a machine-readable representation.
`Various different types of model representations may be
`used. The type of model that is used may depend upon the
`domain in which the application is to be used, the design-
`time modeling environment used by the designer, available
`computing and memoryresources, andthelike,
`
`[0004] During runtime, a design-time model (i.e., a model
`generated during design-time) is typically transformed to a
`form that can then be executed by a target device. The
`transformation from a model representation to an executable
`form is usually performed by a runtime framework which
`may comprise tools such as source code generators, com-
`pilers,
`interpreters,
`etc. This
`transformation generally
`depends onthe characteristics of the runtime environment
`such as the runtime platform of the target device on which
`the application is to run (e.g., the target device that is used
`to output or display the GUI), characteristics of the target
`device such as GUI capabilities of the target device, and
`other factors that are specific to the runtime environment.
`
`transformations from a design-
`[0005] Conventionally,
`time model representation to a runtime form are performed
`
`by specialized runtime tools (c.g., compilers), cach tool
`programmed to convert a particular type of model to a
`particular target device-specific and runtime environment-
`specific executable form. ‘The model representation is thus
`tightly coupled to the runtime environment. Dueto this tight
`coupling, many times, it becomes very difficult to evolve
`model representations without affecting the runtime envi-
`ronment and the designeris forced to take into consideration
`runtime environment dependencies during the design phase.
`
`
`
`BRIEF SUMMARY OF THE INVENTION
`
`invention provide
`[0006] Embodiments of the present
`techniques for decoupling models created during design-
`time from the runtime environment. A declarative and
`executable representation for GUIs for applications is pro-
`vided that is independent ofany particular runtime platform,
`GUI framework, device, and programming language.
`
`[0007] According to an embodimentof the present inven-
`tion,
`techniques are provided for generating an abstract
`representation for a model representation. Information is
`received identifying a model representation for an applica-
`tion,
`the model
`representation comprising information
`describing a graphical user interface (GUI) for the applica-
`tion. An abstract representation is generated for the GUI
`based upon the model representation, wherein the abstract
`representation is independent of a runtime environment
`platform for executing the application.
`
`[0008] According to an embodimentof the present inven-
`tion, a first GUI for a first runtime environment platform
`using the abstract representation. In one embodiment, the
`first GUI is generated by generating first source code based
`upon the abstract representation, wherein the first source
`code is in a language supported bythe first runtime envi-
`ronmentplatform, and executing thefirst source code using
`the first runtime environmentplatform to generate the first
`GUI. According to another embodiment, the first GUI is
`generated by generating machine executable statements spe-
`cific to the first runtime environment platform based upon
`the abstract
`representation, and executing the machine
`executable statements using the first runtime environment
`platform to generate the first GUI. The first runtime envi-
`ronment platform may be a Java platform, a Flash platform,
`Extensible Application Markup Language (XAML)plat-
`form, a dynamic HTML (DHTML)platform, or some other
`platform.
`
`[0009] According to an embodimentofthe present inven-
`tion, a second GUI is generated for a second runtime
`environment platform using the abstract representation,
`wherein the second runtime environment platform is differ-
`ent from the first runtime environment platform.
`
`In one embodiment, generating the abstract repre-
`[0010]
`sentation comprises using a first set of mapping rules for
`generating the abstract representation from the model rep-
`resentation, gencrating the first GUI comprises using a
`second set of mapping rules for generating the first GUI
`from the abstract representation, and generating the second
`GUI comprises using a third set of mapping rules for
`generating the second GUIfrom the abstract representation.
`
`[0011] According to an embodimentof the present inven-
`tion,
`the abstract representation is not dependent on a
`GUlI-framework of the runtime environment platform.
`
`Page 18 of 40
`
`Page 18 of 40
`
`

`

`US 2007/0094609 Al
`
`Apr. 26, 2007
`
`According to another embodimentof the present invention,
`the abstract representation is not dependent on a language of
`the runtime environment platform.
`
`[0012] According to an embodimentofthe present inven-
`tion,
`techniques are provided for generating an abstract
`representation for a model representation. Information is
`received identifying a model representation for an applica-
`tion,
`the model
`representation comprising information
`describing a graphical user interface (GUID) for the applica-
`tion. An abstract representation is generated for the GUI
`based upon the model representation, wherein the abstract
`representation is declarative and executable.
`
`[0021] FIG. 4D depicts a screenshot generated GUIfor a
`Flash platform using an XGL representation generated for
`Example #1 according to an embodiment of the present
`invention;
`
`[0022] FIG. 4E depicts a screenshot generated GUI for a
`DHTMI, platform using an XGI. representation generated
`for Example #1 according to an embodiment of the present
`invention;
`
`[0023] FIG. 5A depicts a processing flow applicable to
`Example #2 according to an embodiment of the present
`invention;
`
`[0024] FIG. 5B depictsa visual representation of a model
`representation for a bank data-entry application GUI used in
`Example #2 according to an embodiment ofthe present
`invention;
`
`[0025] FIG. 5C depicts a screenshot generated GUI for a
`Java platform using an XGL representation generated for
`[Example #2 according to an embodiment of the present
`invention;
`
`[0026] FIG. 5D depicts a screenshot generated GUI for a
`Flash platform using an XGL representation generated for
`Example #2 according to an embodiment of the present
`invention;
`
`[0013] According to an embodimentofthe present inven-
`tion, a generic, declarative, and executable GUI language
`(XGL)is defined that is independent of any GUI framework
`or runtime platform, not dependent on characteristics of a
`target device on which the graphic user interface is to be
`displayed, and is also independent of any programming
`language. XGL is used to generate a generic representation
`CXGLrepresentation) for a design-time model representa-
`tion. The XGL representation is declarative and therefore
`does not depend on any GUI framework, runtime platform,
`or device, or programming language. An XGL representa-
`tion is thus a device-independent representation of a GUI.
`The XGLrepresentation is executable implying that for a
`model representation unambiguously encapsulates execu-
`tion semantics for the GUI described by the model repre-
`sentation. The XGL representation that is created from a
`model representation may then be processed in the runtime
`environment. The XGL representation may be used to gen-
`erate device-specific
`and runtime environment-specific
`GUIs and runtimerepresentations for various target devices
`and runtime platforms. An XGL representation thus serves
`as the common-ground or interface between design-time
`[0029] FIG. 7 depicts a sample listing showing the skel-
`user interface modeling tools andaplurality of user interface
`eton of a sample XGL component according to an embodi-
`runtime frameworks.
`ment of the present invention; and
`
`
`
`[0027] FIG. 5E depicts a screenshot generated GUI for a
`DHTMLplatform using an XGL representation generated
`for Example #2 according to an embodimentof the present
`invention;
`
`[0028] FIG. 6 depicts a sample XGL componentdeclara-
`tion structure according to an embodiment of the present
`invention;
`
`features,
`together with other
`foregoing,
`[0014] The
`embodiments, and advantages of the present invention, will
`become more apparent when referring to the following
`specification, claims, and accompanying drawings.
`BRIEVY DESCRIPTION OF THLE DRAWINGS
`
`[0015] FIG. 1 depicts a simplified environment in which
`an embodiment of the present invention may be used;
`
`[0016] FIG. 2 depicts a simplified process for mapping a
`model representation to a runtime representation according
`to an embodiment of the present invention;
`
`[0017] FIG. 3 depicts a simplified high-level flowchart
`showing processing performed according to an embodiment
`of the present invention;
`
`[0018] FIG. 4A depicts a processing flow applicable to
`Example #1 according to an embodiment of the present
`invention;
`
`[0019] FIG. 4B depicts a visual representation of a model
`representation for a bank data-entry application GUI used in
`Example #1 according to an embodiment ofthe present
`invention;
`
`[0020] FIG. 4C depicts a screenshot generated GUIfor a
`Java platform using an XGL representation generated for
`Example #1 according to an embodiment of the present
`invention;
`
`[0030] FIG. 8 is a simplified block diagram of a computer
`system that may be used to perform processing according to
`an embodiment of the present invention.
`
`DETAILED DESCRIPTION OF‘THE
`INVENTION
`
`In the following description, for the purposes of
`[0031]
`explanation, specific details are set forth in order to provide
`a thorough understanding of the invention. However,it will
`be apparent that the invention may be practiced without
`these specific details.
`
`invention provide
`[0032] Embodiments of the present
`techniques for decoupling models created during design-
`time from the runtime environment. Model representations
`for GUIs created in a design time environment are
`decoupled from the runtime environment in which the GUIs
`are executed. A declarative and executable representation for
`GUIs for applications is provided that is independent of any
`particular runtime platform, GUI framework, device, or
`programming language.
`
`[0033] According to an embodimentof the present inven-
`tion, a generic, declarative, and executable GUI language
`(XGL) is provided. XGL is independent of any particular
`GUI framework or runtime platform. XGL is also not
`dependent oncharacteristics of a target device on whichthe
`
`Page 19 of 40
`
`Page 19 of 40
`
`

`

`US 2007/0094609 Al
`
`Apr. 26, 2007
`
`graphic uscr interface is to be displayed. XGL is also
`independent of any programming language. XGL is used to
`generate a generic representation (referred to as the XGL
`representation or XGL-compliant
`representation)
`for a
`design-time model representation. The XGLrepresentation
`is declarative which implies that the representation does not
`depend on any GUIframework, runtime platform, or device,
`or programming language. An XGL representation is thus a
`device-independent representation of a GUI. The XGLrep-
`resentation is executable and therefore unambiguously
`encapsulates execution semantics for the GUI described by
`a model representation. Models of different types can be
`transformed to XGL representations.
`
`is created from a
`[0034] An XGL representation that
`model representation may then be used for processing in the
`runtime environment. For example, the XGI. representation
`may be used to generate a machine-executable runtime GUI
`(or someother runtime representation) that may be executed
`by a target device. As part of the runtime processing, an
`XGLrepresentation maybe transformed into one or more
`runtime representations (e.g., source code in a particular
`programming language, machine-executable code for a spe-
`cific runtime environment, executable GUI, etc.) that may be
`generated for specific runtime environments and devices.
`Since the XGLrepresentation, rather than the design-time
`model representation, is used by the runtime environment,
`the design-time model representation is decoupled from the
`runtime environment. An XGLrepresentation thus serves as
`the common-ground orinterface between design-time user
`interface modeling tools and a plurality of user interface
`runtime frameworks. It provides a self-contained, closed,
`and deterministic definition of all aspects of a graphical user
`interface in a devicc-independent and programming-lan-
`guage independent manner. In an embodiment,
`the XGL
`semantics may be rigorously defined using, an abstract model
`of a GUI framework. A specific embodiment of XGL is
`described below under the section titled “SAMPLE XGL
`SPECIFICATION (example embodiment)’.
`
`In the runtime environment, various different GUI
`[0035]
`framework-specific and platform-specific tools (e.g., code
`generators,
`interpreters, compilers, etc.) may be used to
`transform an XGL representation into GUIs running on
`target devices. These tools may comply with XGL specifi-
`cation. For example, XGL-compliant code generators may
`conform to semantics of XGL, as described below. XGL-
`compliant code gencrators may ensure that the appearance
`and behavior of the generated user interfaces is preserved
`across a plurality of target GUI frameworks, while accom-
`modating the differences in the intrinsic characteristics of
`each and also accommodating the different levels of capa-
`bility of target devices.
`
`[0036] XGL maybe used for generating representations of
`various different GUIs and supports various GUI features
`including full windowing and componentization support,
`rich data visualizations and animations, rich modes of data
`entry and userinteractions, and flexible connectivity to any
`complex application data services. XGL is fully declarative
`and therefore does not rely on anyparticular programming
`language.
`
`[0037] FIG. 1 depicts a simplified environment 100 in
`which an embodimentof the present invention maybe used.
`As depicted in FIG. 1, a modeling tool 104 running ona data
`
`processing system 102 may be used by a GUI designer
`during the application design phase to create a model
`representation 106 for a GUI application. Model represen-
`tation 106 may be a machine-readable representation of an
`application or a domain specific model. Model representa-
`tion 106 may encapsulate various design parameters related
`to the GUI such as GUI components, dependencies between
`the GUI components, inputs and outputs,andthe like. Model
`representation 106 provides a form in which a model(s) is
`persisted and transported, and possibly handled by various
`tools such as code generators, runtime interpreters, analysis
`and validation tools, merge tools, and the like. In one
`embodiment, model representation 106 may be a collection
`of XML documents with a well-formed syntax. Various
`ditferent modeling tools 104 may be used to generate model
`representation 106. These tools include butare not restricted
`to Visual Composer™ provided by SAP AG of Germany,
`Rational Rose™, Borland Together™, Microsoft ViSio™,
`and others. Modeling tool 104 and modelrepresentation 106
`may be considered to be part of a design time environment.
`
`[0038] According to an embodimentof the present inven-
`tion, an abstract representation generator (or XGL generator)
`108 is provided that generates an abstract representation
`(XGLrepresentation or XGL-compliant representation) 110
`based upon model representation 106. Abstract representa-
`tion generator 108 takes model representation 106 as input
`and outputs abstract representation 110 for the model rep-
`resentation. A specific embodiment of XGL is described
`below. Various other types of XGLs may also be used in
`alternative embodiments.
`
`[0039] Model representation 106 may be of various forms
`or types depending on the tool/language used for the mod-
`eling. These various different model representations may be
`mapped to XGL representation 110. Different mapping rules
`may be provided for mapping a model representation to an
`XGLrepresentation. Different types of model representa-
`tions may be transformed or mapped to XGL representa-
`tions. For each type of model representation, mapping rules
`may be provided for mapping the model representation to an
`XGLrepresentation.
`
`[0040] As previously indicated, XGL is independent of
`any GUI framework or runtime platform and is also inde-
`pendent of any programming language or target device
`characteristic. Further, XGL is capable of unambiguously
`encapsulating execution semantics for the GUI model rep-
`resentation. Accordingly, XGL representation 110 generated
`for a model representation 106 is declarative and execut-
`able—XGT. representation 110 thus provides a representa-
`tion of the GUI of model 106 that is not dependent on any
`device or runtime platform, is not dependent on anypro-
`gramming language,
`and unambiguously encapsulates
`execution semantics for the GUI. The execution semantics
`
`may include for example, identification of various compo-
`nents of the GUI, interpretation of connections between the
`various GUI components, information identifying the order
`of sequencing of events, rules governing dynamic behavior
`of the GUI, rules governing handling of values by the GUI,
`and the like. The XGL representation is also not GUI
`runtime-platform specific. The XGL representation provides
`a self-contained, closed, and deterministic definition of all
`aspects of a graphical user interface that is device indepen-
`dent and language independent. XGL generator 108 may be
`
`Page 20 of 40
`
`Page 20 of 40
`
`

`

`US 2007/0094609 Al
`
`Apr. 26, 2007
`
`configured to gencrate XGL representations for modcls of
`different types, which may be created using different mod-
`eling tools 104.
`
`[0041] XGL or abstract representation 110 provides an
`interface between the design time environment and the
`runtime environment. As depicted in FIG. 1, abstract rep-
`resentation 110 may be used by runtime processing. As part
`of runtime processing, various runtime tools 112 may gen-
`erate different types of runtime representations based upon
`abstract representation 110. Examples of runtime represen-
`tations include device or language dependent source code,
`runtime platform-specific machine-readable code, GUIs for
`a particular target device, and the like. Runtime tools may
`include compilers, interpreters, source code generators, and
`other such tools that are configured to generate runtime
`platform-specific or target device-specific runtime represen-
`tations of abstract representation 110.
`
`[0042] A runtime tool may generate