ollglIIIIIIIIIIIIIIIIIIII mi
`11111111111111111111111114911)p
`
`USOO861
`
`(12) United States Patent
`(12) United States Patent
`Wysopal et al.
`Wysopal et al.
`
`(10) Patent No.:
`(10) Patent No.:
`(45) Date of Patent:
`(45) Date of Patent:
`
`US 8,613,080 B2
`US 8,613,080 B2
`Dec. 17, 2013
`Dec. 17, 2013
`
`(54) ASSESSMENT AND ANALYSIS OF
`(54) ASSESSMENT AND ANALYSIS OF
`SOFTWARE SECURITY FLAWS IN VIRTUAL
`SOFTWARE SECURITY FLAWS IN VIRTUAL
`MACHINES
`MACHINES
`
`(56)
`(56)
`
`References Cited
`References Cited
`
`U.S. PATENT DOCUMENTS
`U.S. PATENT DOCUMENTS
`
`(73)
`(73)
`(*)
`*)
`
`Filed:
`Filed:
`
`Jun. 7, 2011
`Jun. 7, 2011
`
`(75) Inventors: Christopher J. Wysopal, Concord, MA
`Inventors: Christopher J. Wysopal, Concord, MA
`(75)
`(US); Matthew P. Moynahan,
`(US); Matthew P. Moynahan,
`Gloucester, MA (US); Jon R. Stevenson,
`Gloucester, MA (US); Jon R. Stevenson,
`Sudbury, MA (US)
`Sudbury, MA (US)
`Assignee: Veracode, Inc., Burlington, MA (US)
`Assignee: Veracode, Inc., Burlington, MA (US)
`Notice:
`Subject to any disclaimer, the term of this
`Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 60 days.
`U.S.C. 154(b) by 60 days.
`Appl. No.: 13/154,576
`(21)
`(21) Appl. No.: 13/154,576
`(22)
`(22)
`(65)
`(65)
`
`Prior Publication Data
`Prior Publication Data
`US 2012/OO72968 A1
`Mar. 22, 2012
`US 2012/0072968 Al
`Mar. 22, 2012
`
`Related U.S. Application Data
`Related U.S. Application Data
`(63) Continuation-in-part of application No. 12/884,544,
`(63) Continuation-in-part of application No. 12/884,544,
`filed on Sep. 17, 2010, which is a continuation-in-part
`filed on Sep. 17, 2010, which is a continuation-in-part
`of application No. 12/819,627, filed on Jun. 21, 2010,
`of application No. 12/819,627, filed on Jun. 21, 2010,
`which is a continuation-in-part of application No.
`which is a continuation-in-part of application No.
`12/031,918, filed on Feb. 15, 2008, now Pat. No.
`12/031,918, filed on Feb. 15, 2008, now Pat. No.
`8,499,353.
`8,499,353.
`Provisional application No. 61/243,735, filed on Sep.
`(60)
`(60) Provisional application No. 61/243,735, filed on Sep.
`18, 2009, provisional application No. 61/352,079,
`18, 2009, provisional application No. 61/352,079,
`filed on Jun. 7, 2010, provisional application No.
`filed on Jun. 7, 2010, provisional application No.
`60/901,874, filed on Feb. 16, 2007.
`60/901,874, filed on Feb. 16, 2007.
`
`(2006.01)
`(2006.01)
`
`(51)
`(51) Int. Cl.
`Int. C.
`H04L 29/06
`H04L 29/06
`(52)
`(52) U.S. Cl.
`U.S. C.
`USPC ............................................... 726/19; 726/22
` 726/19; 726/22
`USPC
`(58)
`(58) Field of Classification Search
`Field of Classification Search
`USPC ...................................................... 726/19, 22
` 726/19, 22
`USPC
`See application file for complete search history.
`See application file for complete search history.
`
`4,527,237 A
`4,527,237 A
`4,533,997 A
`4,533,997 A
`4,931,928 A
`4,931,928 A
`5,263,162 A
`5,263,162 A
`5,325,531 A
`5,325,531 A
`
`7/1985 Frieder et al.
`7, 1985
`Frieder et al.
`Furgerson
`8/1985 Furgerson
`8, 1985
`6/1990 Greenfeld
`6, 1990
`Greenfeld
`Lundeby
`11/1993 Lundeby
`11, 1993
`6/1994 McKeeman et al.
`6, 1994
`McKeeman et al.
`(Continued)
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`FOREIGN PATENT DOCUMENTS
`
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`WO-0186427 A2
`WO-0186427 A2
`WO-2004003706 A2
`WO-2004003706 A2
`WO-2008103286 A2
`WO-2008103286 A2
`WO-2009097610 Al
`WO-200909761.0 A1
`
`11/2001
`11, 2001
`1/2004
`1, 2004
`8/2008
`8, 2008
`8/2009
`8, 2009
`
`OTHER PUBLICATIONS
`OTHER PUBLICATIONS
`
`Ahpah Software, Inc. SourceAain and Java Decompilation Up
`Ahpah Software, Inc. SourceAain and Java Decompilation—Up-
`dated Dec. 9, 2001. White Paper: SourceAgain and Java Decompila
`dated Dec. 9, 2001. White Paper: SourceAgain and Java Decompila-
`tion retrieved from http://www.ahpah.com/whitepaper.html on Dec.
`tion retrieved from http://www.ahpah.com/whitepaper.html on Dec.
`4, 2002.
`4, 2002.
`
`(Continued)
`(Continued)
`
`Primary Examiner — Jeffrey Pwu
`Primary Examiner — Jeffrey Pwu
`Truong
`Assistant Examiner — Thong
`Assistant Examiner — Thong Truong
`(74) Attorney, Agent, or Firm
`- Goodwin Procter LLP
`(74) Attorney, Agent, or Firm — Goodwin Procter LLP
`
`(57)
`ABSTRACT
`(57)
`ABSTRACT
`Security analysis and Vulnerability testing results are "pack
`Security analysis and vulnerability testing results are "pack-
`aged' or “bound to the actual software it describes. By
`aged" or "bound to" the actual software it describes. By
`linking the results to the software itself, downstream users of
`linking the results to the software itself, downstream users of
`the software can access information about the software, make
`the software can access information about the software, make
`informed decisions about implementation of the software,
`informed decisions about implementation of the software,
`and analyze the security risk across an entire system by
`and analyze the security risk across an entire system by
`accessing all (or most) of the reports associated with the
`accessing all (or most) of the reports associated with the
`executables running on the system and Summarizing the risks
`executables running on the system and summarizing the risks
`identified in the reports.
`identified in the reports.
`
`16 Claims, 12 Drawing Sheets
`16 Claims, 12 Drawing Sheets
`
`APPLICATION
`APPLICATION
`0.
`1111
`
`EXTERNAL
`EXTERNAL
`THREATS
`THREATs
`SOURCES5
`SOURCES ja
`
`councAtNS
`COMMON CATIONS
`SERWER
`2.
`
`ANALYSIS ENGINE
`ANALYSSENGINE
`125
`12,5
`
`
`
`DYNAMI
`%NT:
`TSTING
`ENGINE3
`ENGINE112
`
`STATIC
`STAT&
`TESTING
`TESTING
`ENGINESS
`ENGINE 145
`
`PEN
`PEN
`TESTING
`TESTING
`ENGINE14
`ENGINE 192
`
`ANUAL CE
`MAVE‘gAr
`REWEW
`MoDULEas
`MODULE ja
`
`
`
`RTY
`THREAT
`DATABASE
`5
`
`
`
`T
`SSESS
`ASSESSMENT
`RESULTS
`RESULTS
`DATABASE
`DATABASE
`155
`
`sECURITY AssEssMENTPLATFoRMiGs
`SECURITY ASSESSMENT PLATFORM M
`
`WIZ, Inc. EXHIBIT - 1036
`WIZ, Inc. v. Orca Security LTD.
`
`WIZ, Inc. EXHIBIT - 1036
`WIZ, Inc. v. Orca Security LTD.
`
`

`

`US 8,613,080 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,432,942 A
`5,481,708 A
`5,586,328 A
`5,590,330 A
`5,715,403 A
`5,793,374 A
`5,812,851 A
`5,819,097 A
`5,854,924 A
`5,854,929 A
`5,862,382 A
`5,864,871 A
`5,875,334 A
`5,881,290 A
`5,892,900 A
`5,918,035 A
`5,933,635 A
`5,937,190 A
`5,937,192 A
`6,009,256 A
`6,014,518 A
`6,026,485 A
`6,064,819 A
`6,071,317 A
`6,078,745 A
`6,125,439 A
`6,151,701 A
`6,151,706 A
`6,154,876 A
`6,175,948 B1
`6,240,376 B1
`6,240,547 B1
`6,243,848 B1
`6,249,910 B1
`6,311,327 B1
`6,336,087 B2
`6,381,738 B1
`6,412,106 B1
`6,457,172 B1
`6,594,761 B1
`6,601,235 B1
`6,631,473 B2
`6,668,325 B1
`6,766,481 B2
`6,779,114 B1
`6,820,256 B2
`6,892,303 B2
`6,925,638 B1
`6,928,638 B2
`6,961,925 B2
`6,980,927 B2
`7,051,322 B2
`7,089,590 B2
`7,140,008 B2
`7,155,708 B2
`7,162,473 B2
`7,171,655 B2
`7,185,323 B2
`7,266,813 B2
`7,284,274 B1
`7,315,903 B1
`7,376,939 B1
`7,389,208 B1
`7,430,670 B1
`7,437,764 B1 *
`7,458,067 B1
`7,548,946 B1
`7,594,269 B2
`7,707,566 B2
`7,743,336 B2
`7,752,609 B2
`7,779,394 B2
`7,779,472 B1
`7,840,845 B2
`7,840,951 B1
`
`7/1995 Trainer
`1/1996 Kukol
`12/1996 Caron et al.
`12/1996 Coskun et al.
`2/1998 Stefik
`8/1998 Guenter et al.
`9/1998 Levy et al.
`10/1998 Brooks et al.
`12/1998 Rickel et al.
`12/1998 Van Praet et al.
`1/1999 Kataoka
`1/1999 Kitain et al.
`2/1999 Chow et al.
`3/1999 Ansari et al.
`4/1999 Ginter et al.
`6/1999 Van Praet et al.
`8/1999 Holzle et al.
`8/1999 Gregory et al.
`8/1999 Martin
`12/1999 Tseng et al.
`1/2000 Steensgaard
`2/2000 O'Connor et al.
`5/2000 Franssen et al.
`6/2000 Nagel
`6/2000 De Greef et al.
`9/2000 Tremblay et al.
`11/2000 Humphreys et al.
`11/2000 Lo et al.
`11/2000 Haley et al.
`1/2001 Miller et al.
`5/2001 Raynaud et al.
`5/2001 Holzle et al.
`6/2001 Guignet et al.
`6/2001 Ju et al.
`10/2001 O'Brien et al.
`1/2002 Burgun et al.
`4/2002 Choi et al.
`6/2002 Leask et al.
`9/2002 Carmichael et al.
`7/2003 Chow et al.
`7/2003 Holzle et al.
`10/2003 Townsend
`12/2003 Collberg et al.
`7/2004 Estep et al.
`8/2004 Chow et al.
`11/2004 Fleehart et al.
`5/2005 Le Pennec et al.
`8/2005 Koved et al.
`8/2005 Parvathala et al.
`11/2005
`Callahan, II et al.
`12/2005
`Tracy et al.
`5/2006
`Rioux
`8/2006
`Judge et al.
`11/2006 Chilimbi et al.
`12/2006 Hammes et al.
`1/2007 Dumais et al.
`1/2007 Gordon et al.
`2/2007 Nair et al.
`9/2007 Nistler et al.
`10/2007 Walls et al.
`1/2008 Bowden
`5/2008 Nayak et al.
`6/2008 Solinsky
`9/2008
`Horning et al.
`10/2008
`Sobel et al.
`11/2008
`Tirumalai et al.
`6/2009 Saulpaugh et al.
`9/2009 Durham et al.
`4/2010 Grover et al.
`6/2010 Louch et al.
`7/2010 Rioux
`8/2010 Homing et al.
`8/2010 Lou
`11/2010 Doddapaneni et al.
`11/2010 Wright et al.
`
` 726/22
`
` 713/164
`
`7,856,624 B2
`7,874,001 B2
`7,891,003 B2
`7,930,753 B2
`7,937,693 B2
`8,069,487 B2
`8,087,067 B2
`8,108,933 B2
`8,136,104 B2
`8,161,464 B2
`8,161,548 B1 *
`8,171,553 B2
`8,181,251 B2
`8,225,409 B2
`8,272,058 B2
`8,281,290 B2
`8,290,245 B2
`8,332,944 B2
`8,347,386 B2
`8,365,155 B2
`2001/0020272 Al
`2004/0073445 Al
`2004/0102923 Al
`2004/0267700 Al
`2005/0108037 Al
`2005/0138426 Al
`2006/0021055 Al
`2006/0095967 Al
`2006/0136424 Al
`2006/0190769 Al
`2006/0218639 Al
`2006/0277607 Al
`2007/0022287 Al *
`2007/0101433 Al
`2007/0180490 Al
`2007/0240218 Al
`2007/0261061 Al
`2007/0294766 Al
`2008/0005782 Al
`2008/0204763 Al
`2008/0209567 Al
`2009/0165135 Al
`2010/0031353 Al
`2010/0058474 Al
`2010/0058475 Al
`2010/0281248 Al
`2011/0047594 Al
`2011/0145920 Al
`2011/0173693 Al
`2012/0072968 Al
`2012/0117650 Al
`2012/0174224 Al
`2013/0097706 Al
`
`12/2010 Plum
`1/2011 Beck et al.
`2/2011 Mir et al.
`4/2011 Mellinger et al.
`5/2011 Victorov
`11/2011 Fanton et al.
`12/2011 Mahaffey et al.
`1/2012 Mahaffey
`3/2012 Papakipos et al.
`4/2012 Archambault et al.
`4/2012 Wan
`5/2012 Aziz et al.
`5/2012 Kennedy
`7/2012 Newman et al.
`9/2012 Brennan
`10/2012 Thompson
`10/2012 Turbell et al.
`12/2012 Rozenberg et al.
`1/2013 Mahaffey et al.
`1/2013 Rioux
`9/2001 Le Pennec et al.
`4/2004 Mellinger et al.
`5/2004 Tracy et al.
`12/2004 Dumais et al.
`5/2005 Bhimani et al.
`6/2005 Styslinger
`1/2006 Judge et al.
`5/2006 Durham et al.
`6/2006 Nuggehalli et al.
`8/2006 Doddapaneni et al.
`9/2006 Newman et al.
`12/2006 Chung
`1/2007 Beck et al.
`5/2007 Louch et al.
`8/2007 Renzi et al.
`10/2007 Tuvell et al.
`11/2007 Staniford et al.
`12/2007 Mir et al.
`1/2008 Aziz
`8/2008 Turbell et al.
`8/2008 Lockhart et al.
`6/2009 Lomont et al.
`2/2010 Thomas et al.
`3/2010 Hicks
`3/2010 Thummalapenta et al.
`11/2010 Lockhart et al.
`2/2011 Mahaffey et al.
`6/2011 Mahaffey et al.
`7/2011 Wysopal et al.
`3/2012 Wysopal et al.
`5/2012 Nachenberg
`7/2012 Thomas et al.
`4/2013 Titonis et al.
`
`OTHER PUBLICATIONS
`
`Ahpah Software, Inc. SourceAgain PC Professional, Ahpah Soft-
`ware, Inc. / SourceAgain PC Professional retrieved fromhttp://www.
`ahpah.com/sourceagain/sourceagain.sub.--professional.html
`on
`Dec. 4, 2002.
`Backer Street Software. REC—Reverse Engineering Compiler. REC
`Decompiler Home Page retrieved from http://www.backerstreet.
`com/rec/rec.htm on Dec. 3, 2002.
`Blume, W. & Eigenmann, R., "Demand-driven Symbolic Range
`Propagation", University of Illinois Urbana-Champaign, 1-15
`(1995).
`Blume, W.J., "Symbolic Analysis Techniques for Effective Auto-
`matic Parallelization", University of Illinois at Urbana-Champaign
`(1995).
`Bohme, Rainer "A Comparison of Market Approaches to Software
`Vulnerability Disclosure", Springer-Verlag Berlin Heidelberg, 2006
`(14 pages).
`Breuer et al, "Decompilation: The enumeration of types and gram-
`mars", ACM Trans. on Prog. Lang. and Sys. vo. 16, No. 5, pp.
`1613-1647, 1994.
`Breuer, P.T. and Bowen, J.P., (1992d), "Generating Decompilers",
`Draft, Oxford University Computing Laboratory. Submitted for Pub-
`lication.
`
`726/22
`
`

`

`US 8,613,080 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Burke, "An interval based approach to exaustive and incremental
`interprocedural data flow analysis", ACM Trans. on Prog. Language
`and Systems, vol. 12, No. 3, pp. 341-395, 1990.
`Business Wire, "The Kernel Group Unveils Java Support for
`AutoTracel Serviceability Solution Expands Capabilities; Provides
`Java Support and Key Features that Extend its Power and Flexibility",
`Jun. 4, 2001.
`Choi, J.D. & Ferrante, J., "Static Slicing in the Presence of GOTO
`Statements", IBM T.J. Watson Research Center.
`Cifuentes, C., "An Environment for the Reverse Engineering of
`Executable Programs", Proceedings of the Asia-Pacific Software
`Engineering Conference (APSEC), IEEE Computer Society Press,
`Brisbane, Australia, pp. 41-419, Dec. 1995.
`Cifuentes, C., "Partial Automation of an Integrated Reverse Engi-
`neering Environment of Binaiy Code", Proceedings Third Working
`Conference on Reverse Engineering, Monterey, CA, IEEE-CS
`Press, pp. 50-56, Nov. 8-10, 1996.
`Cifuentes, C., "Reverse Compilation Techniques", Queensland Uni-
`versity of Technology (Jul. 1994).
`Cifuentes, C. et al., "Assembly to High-Level Language Translation",
`University of Queensland, Brisbane, Australia, Dept. of Comp. Sci.
`& Elec. Eng., Technical Report 439, Aug. 1993.
`Cifuentes, C. et al., "The Design of a Resourceable and Retargetable
`Binary Translator", University of Queensland, Dept. Comp. Sci. &
`Elec. Eng. & Ramsey, N., University of Virginia, Dept. Comp. Sci.
`Cifuentes, C. & Fraboulet, A., "Intraprocedural Static Slicing of
`Binary Executables", University of Queensland, Dept. Comp. Sci.,
`Centre for Software Maintenance.
`Cifuentes, C. & Gough, K.J., "Decompilation of Binaiy Programs",
`Software—Practice and Experience, vol. 25, pp. 811-829, Jul. 1995.
`Cifuentes, C. & Sendall, S., "Specifying the Semantics of Machine
`Instructions", University of Queensland, Dept. Comp. Sci. & Elec.
`Eng., Technical Report 442, Dec. 1997.
`Cifuentes, C. & Simon, D., "Precedural Abstration Recovery from
`Binary Code", University of Queensland, Dept. Comp. Sci. & Elec.
`Eng., Technical Report 448, Sep. 1999.
`Cifuentes, C. & Van Emmerik, M., "Recovery of Jump Table Case
`Statements from Binary Code", University of Queensland, Dept.
`Comp. Sci. & Elec. Eng., Technical Report 444, Dec. 1998.
`Cytron, R. et al., "Efficiently Computing Static Single Assignment
`Form and the Control Dependence Graph", IBM Research Division,
`Mar. 7, 1991.
`Dejean et al, "A definition optimization technique used in a code
`translation algorithm", Comm.. of the ACM, vol. 32, No. 1, pp.
`94-105, 1989.
`Di Lucca Guiseppe A., et al. "Testing Web-based applications: The
`state of the art and future trends" Information and Software Technol-
`ogy, 2006 (pp. 1172-1186).
`Duesterwald et al, "A demand driven analyzer for data flow testing at
`the integration level", IEEE ICSE, pp. 575-584, 1996.
`Duesterwald, E. et al., "A Practical Framework for Demand-Driven
`Interprocedural Data Flow Analysis", ACM Transactions on Pro-
`gramming Languages and Systems 19, pp. 992-1030, Nov. 1997.
`Dyer, D. Java decompilers compared ; Our detailed examples of how
`3 top decompilers handle an extensive test suite will help you deter-
`mine which, if any, meet your needs. JavaWorld (Jul. 1997).
`Gough, I., Queensland University of Technology & Klaeren, H.,
`University of Tubingen, "Eliminating Range Checks using Static
`Single Assignment Form", Dec. 19, 1994.
`Gupta, R., University of Pittsburgh, "Optimizing Array Bound
`Checks Using Flow Analysis", ACM SIGPLAN Conference on Pro-
`gramming Language Design and Implementation, White Plains, NY,
`Preliminary Version (1995).
`
`Harrold, M.J. & Soffa, M.L., "Efficient Computation of
`Interprocedural Definition-Use Chains", ACM Transactions on Pro-
`gramming Language and Systems 16, 175-204 (Mar. 1994).
`Hollingum, J., "Arithmetic robot modules control robotic produc-
`tion", Industrial Robot 22, 32-35 (1995).
`International Search Report for PCT/US2008/002025, mailing date
`Sep. 2, 2008 ( 4 pages).
`for TurboC": http://www.
`Kumar, S., "Disc Decompiler
`debugmode.com/dcompile/disc.htm modified Oct. 22, 2001.
`Liang, D. & Harrold, M.J., "Efficient Computation of Parameterized
`Pointer Information for Interprocedural Analyses", Georgia Institute
`of Technology, Tech Report GIT-CC-00-35, 1-17 (Dec. 2000).
`Liang, D. & Harrold, M.J., "Light-Weight Context Recovery for
`Efficient and Accurate Program Analyses", Proceedings of the 22nd
`International Conference on Software Engineering 1-10 (Jun. 2000).
`MacUser, Programming & systems; Software Review; software for
`the Apple Macintosh computer, Evaluation, vol. 8, 237 (Jan. 1993).
`Mittal et al, "Automatic translation of software binaries onto
`FPGAs", ACM DAC, pp. 389-394, 2004.
`Mycroft, A., "Type-Based Decompilation", Cambridge University,
`Computer Laboratory.
`Myreen et al, "Machine code verification for multiple architectures"
`IEEE, pp. 1-8, 2008.
`Orso, A. et al., "Effects of Pointers on Data Dependences", Georgia
`Institute of Technology, College of Computing, Technical Report
`GIT-VV-00-33, 1-17 (Dec. 2000).
`Partial International Search Report for EP12184590.3 dated May 10,
`2013, 7 pages.
`Patterson, J.R.C., "Accurate Static Branch Prediction by Value Range
`Propagation", Proc. ACM SIGPLAN Conference on Programming
`Language and Design Implementation, La Jolla, San Diego, 67-78
`(Jun. 1995).
`Pingali, K. & Bilardi G., "Optimal Control Dependence Computation
`and the Roman-Chariots Problem", ACM Transactions on Program-
`ming Languages and Systems 19, 1-30 (May 1997).
`Reilly, D., "Decompilers—friend or foe?", Java Coffee Break updaed
`Jun. 2, 2001 retrieved from http://www.javacoffeebreak.com/ar-
`ticles/decompilers.sub.--friend.sub.--or- .sub.--foe.html.
`Sagiv, M. et al. Precise Interprocedure Dataflow Analysis with Appli-
`cations to Constant Propagation. University of Wisconsin-Madison,
`Computer Sciences Dept.
`Saul, J. M. Hardware/Software Codesign fir FPGA-Based Systems.
`Proceedings of the 32.sup.nd Hawaii International Conference on
`System Sciences (1999).
`Sinha, S. & Harold, M.J., "Analysis and Testing of Programs with
`Exception-Handling Constructs", IEEE Transactions on Software
`Eng. 26, 1-24 (Sep. 2000).
`Stitt et al, "New decompilation techniques for binary level co-pro-
`cessor generation", IEEE, pp. 546-553, 2005.
`University of Queensland, Comp. Sci & Elec. Eng., The dcc
`Decompiler, updated on May 4, 2002, retrieved from http://itee.uq.
`edu.au/.about.cristina/dcc.html on Dec. 4, 2002.
`Ural et al, "Modeing software for acurate data flow representaion",
`IEEE, pp. 277-286, 1993.
`Van Emmerik, M. Signatures for Library Functions in Executable
`Files. Queensland University of Technology, Information Security
`Research Centre.
`Van Tyle, S., "Engineering software tools meet demands", Electronic
`Design 42, 71 (Jun. 27, 1994).
`Written Opinion of the International Searching Authority for PCT/
`US2008/002025 mailing date Sep. 2, 2008 (6 pages).
`Xu et al, "Dynamic purity analysis for Java programs", ACM Paste,
`pp. 75-82, 2007.
`Yardimci et al, "Mostly static program partitioning of binary
`executables", ACM Trans. on Prog.Lang. and Sys. vo. 31, No. 5,
`article 7, pp. 1-46, 2009.
`
`* cited by examiner
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 1 of 12
`
`US 8,613,080 B2
`
`APPLICATION
`110
`
`EXTERNAL
`THREATS
`SOURCES 115
`
`COMMUNICATIONS
`SERVER
`120
`
`AL
`
`ANALYSIS ENGINE
`125
`
`DYNAMIC
`TESTING
`ENGINE 130
`
`STATIC
`TESTING
`ENGINE 135
`
`PEN
`TESTING
`ENGINE 140
`
`MANUAL CODE
`REVIEW
`MODULE 145
`
`--,
`..-
`
`SECURITY
`THREAT
`DATABASE
`150
`
`.._.,
`
`ASSESSMENT
`RESULTS
`DATABASE
`155
`
`_,.."
`
`•........
`
`---/
`
`SECURITY ASSESSMENT PLATFORM 105
`
`FIG. 1
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 2 of 12
`
`US 8,613,080 B2
`
`ASSURANCE
`RECOMMENDATION
`ENGINE 205
`
`WORKFLOW
`CONSTRUCTOR
`210
`
`STIMULUS ANALYSIS
`EVALUATION
`ENGINE 215
`
`WORKFLOW
`ENGINE 220
`
`TESTING
`ENGINES
`130-145
`
`ASSESSMENT
`CORRELATION
`ENGINE 225
`
`I-
`
`4,
`
`BENCHMARK
`MODULE 235
`
`4,
`
`ANONYMIZER
`240
`
`V_
`FLAW
`VIEWER
`245
`
`GRADING AND
`REPORTING
`MODULE 230
`
`"I
`
`DRM
`PACKAGER
`250
`
`I_
`
`DRM
`ENGINE
`255
`
`ANALYSIS ENGINE 125
`
`APPLICATION
`110
`
`FIG. 2
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 3 of 12
`
`US 8,613,080 B2
`
`REPORTER
`270
`
`Ak
`
`VALIDATOR
`275
`
`ANALYSIS ENGINE
`
`125 1 FILE
`
`EXTRACTOR
`265
`
`LOAD I EXECUTE
`MODULE
`260
`
`I_
`
`TESTING
`ENGINES
`130 - 145
`
`Alk
`
`VIRTUAL MACHINE
`IMAGE FILE
`F
`
`APPLICATION
`IDENTIFIER
`280
`
`I-D
`
`INTERACTION
`ANALYZER
`285
`
`FIG. 2a
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 4 of 12
`
`US 8,613,080 B2
`
`tart Meat Proms
`
` V
`
`
`
`305
`
`:Cored task iipp.ffoaticr;
`pm%
`fmaloaioff latof
`**Jolt,
`
`Stan Optional. woo
`
`7
`
` 315
`
`310
`
`Applkation
`Nona
`
`tlFipadiWaiffallM itPeiSsi
`
`notailod opfaloatfori
`proffle
`(laziness cooled.)
`
`320
`
`Yot
`
`.
`
`.
`SpaOled?
`
`No
`
`2 325
`
`CvtandowRefm.rarwevANI
`Mumma Le
`
`Prowl anfilaria,
`scomsnerfdations a!A
`options
`
`330
`
`_.335
`
`&sired mammon.
`ifwal
`
`340
`
`Aatiafaotle
`Flocasmoodatior,
`Ef vim 205
`
`.f3tall Analysis Workflow
`
`Analysis Watkikwe
`Comtrotor
`
`210
`
`Anafysis
`ork€low
`
`tan Anaty
`WWsk€t
`.v
`
`Aninysiis. WonsVny
`Eneirst
`
`220
`
`FIG. 3
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 4 of 12
`
`US 8,613,080 B2
`
`
`
`Shah Maa Pr
`
`
`
`a
`
`
`
`
`
`Sarl Ontianal geasess
`
`
`
`
`
`
`
`
`
`
`
`
`
`as Lage)
`
`
`
`
`
`
`
`
`
`
`
`124
`
`
`
`
`
`
`
`
`¥
`
`
`¥¥
`
`【 有
`
`
`
`
`
`
`
`
`
` 和
`
`330
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ae)
`
`
`
`
`
`
`
`
`
`
`Bull An
`
`Worktiow
`
`aw
`
`+
`
`
`
`
`
`
`
`
`
`
`
`a
`
`
`
`a
`
`ae
`
`
`
`
`
`
`
` Wiskivar
`SESS
`
`220
`
`
`
`
`
`
`
`网
`”
`
`FIG. 3
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 5 of 12
`
`US 8,613,080 B2
`
`&mg Main Proom
`
`405
`
`Nmiletalys I - Type
`(
`qty).
`
`.410
`
`Angilys 2 Type 2
`tksinekratk5i1TestirV
`
`415
`Age
`
`tdsi?iAnaDigt 3
`firtlilni.athybrid)
`
`d
`
`COrreinte Resuks
`
`420
`
`425
`
`R „
`
`s
`
`ssels.lierit Corre%tior
`EgrAne
`
`225
`
`Unified
`Resoitz.‘
`
`Gefierate Anstysis
`5unrentlfy
`
`Analniz &Tiopnciin9
`'''t** 230
`
`FIG. 4
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 5 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`¥
`
`
`
`二
`
`
`
`¥
`
`
`
`U 7
`
`
`
`
`
`
`
`
`
`
`ai
`
`i
`
`
`
`
`
`
`420
`
`+=
`
`Gorrekis Re
`
`
`
`
`
`
`
`
`
`425
`
`
`
`
`
`
`
`
`
`
`
`
`
`Agyication Grading,
`
`
`
`
`RPRSY
`
`
`
`
`
`
`
`FIG. 4
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 6 of 12
`
`US 8,613,080 B2
`
`carte
`
`Po-&
`
`505
`
`510
`
`Sp
`
`y fir l conditioz3
`
`515
`
`—
`
`P
`
`Up&ad kdicaVon
`
`Applicatior,
`
`FIG. 5
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 6 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`
`
`
`
`
`
`Spacky mscan conditions
`
`
`
`
`
`
`
`
`
`} 一 515
`
`
`1 Ago’
`
`ne
`
`FIG. 5
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 7 of 12
`
`US 8,613,080 B2
`
`Suers M3In Process
`Newt Three)
`
`Slott !Arlin Prowas
`(NM Spas: Copobi§iy)
`
`605
`
`nomt. T
`Meiedm
`
`610
`
`Sosr M
`NietsKtat.
`of
`
`Alpert new gocutyittrent
`
`Mel new
`
`11-;mr,
`150
`
`150
`
`615
`
`Create enern.# skitiius
`efxsord
`
`Ilk
`
`Exxerta3 1i15
`Arakmis
`£timatw
`
`620
`
`Neld.conailtsm anairance
`application list
`
`625
`
`_
`
`Oenoniin N tippnAtijoil
`shoetd rde :mined
`
`mild soplitakii
`84 onNxet'fi
`
`630
`
`„. 635
`
`Bisk eneViis watidit-iw
`
`640
`
`645
`
`Q.0(1.t?aWiateial
`snob/gm
`
`Vas
`&list?
`
`AfOrf5i,i,
`Evaklistivi En .m
`' 215
`
`At prrookm P
`PrMilo.
`
`wormw
`4,1on$1..mw
`210
`
`Anaipi*
`WtNitIkw
`
`quo
`
`FIG. 6
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 7 of 12
`
`US 8,613,080 B2
`
`
`
`PT
`
`
`
`art BS
`iad Sean
`
`
`
`
`
`SER 2
`
`trent
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`210
`
`WAS
`
`¥
`
`【
`
`Bint ana
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 6
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.R. Patent
`
`Dec. 17, 2013
`
`Sheet 8 of 12
`
`US 8,613,080 B2
`
`Stan Main Pe
`
`705
`
`appiicako.
`anat*i)
`
`a
`
`Loral
`Form 2
`(binary)
`
`f-teMao
`Form 2 —
`(binary)
`
`moo.
`
`715
`
`720
`
`RJR ub
`
`Vmsi Flew detaI
`
`725
`
`%Awe More Nam'?
`
`ma
`
`Erigine
`135
`
`• Regnao R
`Analysis
`Revd%
`
`•
`
`_ye
`
`245
`
`LecA
`Farm I
`(OW*
`
`FIG. 7
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 8 of 12
`
`US 8,613,080 B2
`
`
`
`Stat Main Presses:
`
`}
`
`
`
`
` 705
`
`
`
`
`
`
`
`710
`
`
`
`
`
`
`
`Rinavy Anaiyets Erigits
`135
`
`
`
`
`
`
`
`
`
`
`
`【
`
`【
`
`
`
`
`
`
`
`
`
`715
`
`
`
`APRHYZS® BER
`
`
`
`Review Reesslis:
`
`
`
`
`
`
`
`
`
`720
`
`
`
`
`
`
`
`
`by
`ant
`A
`
`AS Fie deal
`
`
`
`
`
`
`
`
`
`
`|
`oem
` 245
`{serra}
`
`Flaw Viewer
`
`
`
`
`
`
`
`725
`
`slow More FRean? |
`
`FIG, 7
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 9 of 12
`
`US 8,613,080 B2
`
`(
`
`-'
`Start Main Pfacast
`m'O 0 ,0 10 04W
`msedit db)
`
`805
`
`7
`
`Spe0ify Appficaw Pn0f0a
`
`AppUcaNw
`p
`€'r€fle
`
`.710
`
`Analyze applIcat0n
`
`Ares Ernes-%,
`125
`
`Analysis
`R lts
`
`815
`
`7
`
`kiantirate A€alysis
`Durnawy
`
`AI)* c6::-.D &Mina.
`Analysis& Faiwrang
`no 230
`
`820
`
`Update Areiwalize4
`Results DB
`
`Remit Anony!ri,2
`240
`
`Amortyrp&ed
`Appication
`A110.lyiV DB
`
`ace
`
`FIG. 8
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 9 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`
`
`
`
`
`
`710
`a
`
`
`
`
`
`
`Anal sor
`
`
`
`
`
`te
`
`-
`
`
`
`
`
`STRESS
`
`125
`
`
`
`
`
`者
`
`ra
`
`815
`
`
`
`SA
`
`
`
`
`
`oe
`
`
`
`
`
`
`
`
`
`
`
`
`820
`
`'
`~~
`¥
`*
`¥
`
`
`
`
`Ubdat
`ge
`Re
`圖
`圖
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 8
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 10 of 12
`
`US 8,613,080 B2
`
`( Start Main Ft-ixess
`
`(Aivige ;v:d berg:iv-Ws
`
`805
`
`,4iecify Application NDlik,?
`
`Application
`Peofile
`
`710
`
`Analyze a alicakion
`
`Aralysis
`
`125
`
`nsi$
`Itti•
`
`815
`../".
`
`Gionklrge .MIDI .ski
`Sovirtaty
`
`Peftr BeIFICNINAKR
`
`9°5
`
`910
`
`priwiN1 SSttdirF a.
`Ar
`-a 8, Retper
`EviOrka
`230
`
`Appkatm
`
`Aro ia
`Eien;Tory
`
`—
`
`Arolpis St fIchrmi
`rEe
`
`235
`
`Manrizad Y
`Appliattios
`Analysis Da
`
`iliewtay bandirnaeg
`
`dvrta
`Restitis
`
`—
`
`FIG. 9
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 10 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`
`
`Speavy Ape
`
`
`
`
`
`
`
`
`
`
`
`
`
`:
`
`
`
`ig ROSAS
`RSS
`125
`
`
`
`
`
`党
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 9
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 11 of 12
`
`US 8,613,080 B2
`
`'alart Main Pr000ss
`
`.7 805
`
`SpeOfv.Appkarioo Profile
`
`1005
`
`.V
`
`applkurtion riga fix
`e..rabr4teksil
`
`1010
`
`Jits for
`submission
`
`1015
`
`sanScatice ansiys:i.3
`data
`
`1020
`
`11 I 11 1111
`
`11
`
`Appriastion
`P Profile
`
`
`
` 1.1:11.1•11
`
`D
`II —
`
`DRkil Pac.-ki:qw
`250
`
`:rad
`suteilwasin
`data )loth)
`
`inue iireliegJ use IMM
`fOr anzdysis *VI.*
`
`DEM Erigim
`255
`
`Areal
`amine flmited
`:MS IDE:MS
`
`aPim
`
`255
`
`Em*:e
`125
`
`Anestyg
`
`(esrnMe)
`
`710
`
`7'
`
`Ausiy2e applkation
`
`Dor
`
`FIG. 10
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 11 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`
`
`
`
`SA Apy
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1015
`
`
`
`.
`
`
`
`
`
`
`it
`
`a
`
`
`
`
`
`
`
`SRS
`250
`
`
`
`
`
`
`
`
`
`a ace’
`OWES
`prtecind
`
`SUESTHE AEH
`TBE BSS
`lor
`
`lg
`
`press
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`¥ i 0
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tH
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 10
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 12 of 12
`
`US 8,613,080 B2
`
`art bur: Pn)tx40.
`
`1105
`../".
`
`Select Kipfic,135011 to VieW
`Mid:5
`
`rel'ika&
`Ft*? r
`tertIO
`
`1110
`
`1115
`
`pate‘10
`a:n*4s results
`
`1020
`
`/
`
`:RP;
`issue Nrnitad
`Ikerise tc 16foof analysis
`Fesuite
`
`1120
`
`Maysis, R
`Resists
`(gernot)
`
`rAM
`
`250
`
`wotasota
`sodyas molts
`Omuta)
`
` w/DRM ptiotectel
`
`allgyFAS results
`(ima4
`
`
`
`:Limited me
`€Imlo
`view 8ortlysis
`results
`
`255
`
`3KiesaN,
`
`aoalysi3
`Eta
`
`[AM Eng •-:o
`255
`
`FIG. 11
`
`U.S. Patent
`
`Dec. 17, 2013
`
`Sheet 12 of 12
`
`US 8,613,080 B2
`
`
`
`
`
`RN
`
`
`
`
`
`1105
`
`a
`
`
`Gatect ay
`
`to ey
`
`
`
`4
`Pa 1110
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1115
`a
`
`
`
`
`
`
`
`一 1020
`
`
`
`
`
`
`
`【
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS PREY
`SER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 11
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`1
`ASSESSMENT AND ANALYSIS OF
`SOFTWARE SECURITY FLAWS IN VIRTUAL
`MACHINES
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation-in-part of and claims
`priority to and the benefits of U.S. patent application Ser. No.
`12/884,554, filed on Sep. 17, 2010, which claims priority to
`and the benefit of U.S. provisional patent application Ser. No.
`61/243,735, filed on Sep. 18, 2009, and is a continuation-in-
`part of and claims priority to U.S. patent application Ser. No.
`12/819,627, filed on Jun. 21, 2010, which itself is a continu-
`ation-in-part of and claims priority to U.S. patent application
`Ser. No. 12/031,918, filed on Feb. 15, 2008, which claims
`priority to and the benefits of U.S. provisional patent appli-
`cation Ser. No. 60/901,874, filed on Feb. 16, 2007. This
`application also claims priority to and the benefit of U.S.
`provisional patent application Ser. No. 61/352,079, filed on
`Jun. 7, 2010.
`
`FIELD OF THE INVENTION
`
`The invention relates generally to systems and techniques
`for analyzing computer code, and more particularly to deter-
`mine whether the computer code representing a virtual
`machine contains security flaws or is vulnerable to known
`security threats.
`
`BACKGROUND
`
`In recent years, many companies and government agencies
`have been exposed to negative press and legal proceedings
`due to high-profile security breaches in which sensitive data
`has been either inadvertently disclosed or stolen. While many
`of these incidents were the result of human error, a significant
`percentage was traced back to poorly designed software
`architecture and/or applications. Conventional techniques for
`testing software applications can identify many vulnerabili-
`ties, but no one methodology is failsafe. Furthermore,
`although many security-analysis techniques require signifi-
`cant time and resources to administer, not every application
`necessitates the same level or degree of analysis.
`As a result, companies face a difficult trade-off between the
`desire to test software and limitations on available resources
`and time. Moreover, many companies do not have the exper-
`tise to apply some of the more intricate and complex security
`assessment techniques, and thus look to industry experts for
`such services. This creates yet another challenge, in that often
`what is being tested is highly sensitive, proprietary software.
`There are a myriad of testing and assessment techniques
`for validating various properties of software applications and
`network implementations. However, one of the most critical
`processes for ensuring that the deployment of software does
`not expose an organization to unacceptable risks is security
`and vulnerability testing. Some of the conventional tech-
`niques used to perform such testing includes static analysis
`(automated code review), dynamic analysis (automated pen-
`etration testing) and manual analyses such as code review,
`design review, and manual penetration testing. All of these
`analysis techniques are aimed at finding security weaknesses
`and vulnerabilities in an application and typically provided in
`report format to the programmers, product managers and
`quality assurance (QA) staff. The report can provide detailed
`results (e.g., program names, line numbers, variable names,
`
`US 8,613,080 B2
`
`2
`data connections, etc.) as well as a summary of the results.
`The report may be a conventional document such as a text file
`or a structured XML file.
`However, once the report is run and reviewed by a QA
`5 engineer or product manager, it is typically no longer refer-
`enced or used. Furthermore, as an executable or application is
`implemented and/or provided to a customer, the report is
`forever decoupled from the software that was tested. In fact,
`an individual or organization using software has no knowl-
`10 edge that a report was ever created or used to analyze the
`software they are now using. As such, valuable information
`about what aspects of the application were tested, how secure
`certain features or functions may be and what testing meth-
`odologies were used are unknown to those that value such
`15 information.
`Another trend in systems engineering is the use of so-
`called "virtual machines." Generally, a virtual machine (or
`"VM") refers to completely isolated operating system instal-
`lation within a normal operating system, which may be imple-
`20 mented either using software emulation