USOO8370583B2
`
`(12) Unlted States Patent
`(10) Patent No.:
`US 8,370,583 B2
`
`Hughes
`(45) Date of Patent:
`Feb. 5, 2013
`
`(54) NETWORK MEMORY ARCHITECTURE FOR
`PROVIDING DATA BASED ON LOCAL
`
`(75)
`
`ACCESSIBILITY
`.
`_
`Inventor: Dav1d Anthony Hughes, Mounta1n
`View, CA (US)
`
`.
`( * ) Not1ce:
`
`(73) Assignee: Silver Peak Systems, Inc., Santa Clara,
`CA (US)
`.
`.
`.
`.
`Subject to any d1scla1mer, the term of th15
`patcnt is cxtcndcd or adjustcd under 35
`U.S.C. 154(b) by 1301 days.
`
`(21) App1.N0.: 11/202,697
`
`(22) Filed:
`
`Aug. 12, 2005
`
`(65)
`
`Prior Publication Data
`
`US 2007/0050475 A1
`
`Mar. 1: 2007
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 12/00
`(2006.01)
`G06F 13/00
`(2006.01)
`G06F 13/28
`(2006.01)
`G06F 15/16
`(52) US. Cl.
`........ 711/147; 711/100; 711/111; 711/112;
`711/113; 711/117; 711/118; 709/217; 709/218;
`709/219
`
`(58) Field of Classification Search ........................ None
`See application file for complete search history.
`
`(56)
`
`References Clted
`U.S. PATENT DOCUMENTS
`
`5,307,413 A
`5,359,720 A
`5,483,556 A
`5,592,613 A
`5,611,049 A *
`5,675,587 A
`5,754,774 A
`
`4/1994 Denzer
`10/1994 Tamura et a1.
`1/1996 Pillan etal.
`1/1997 Miyazawaetal.
`3/1997 Pitts .................................. 707/8
`10/1997 Okuyama et al.
`5/1998 Bittinger et al.
`
`5,802,106 A
`5,883,891 A
`6,000,053 A
`6,003,087 A
`6,081,883 A *
`6,295,541 B1
`6,374,266 B1
`6,434,662 B1
`
`9/1998 Packer
`3/1999 Williams et al.
`12/1999 Levine et al.
`12/1999 Housel et al.
`6/2000 Popelka et al.
`9/2001 Bodnar ct 31.
`4/2002 Shnelvar
`8/2002 Greene et al.
`
`................. 712/28
`
`6,438,664 B1
`6,452,915 B1
`6,587,985 B1
`6,618,397 B1
`6,633,953 132
`6,643,259 B1
`
`8/2002 MCGmth et 31'
`9/2002 Jorgensen
`7/2003 Fukush1ma et al.
`9/2003 Huang
`10/2003 Stark
`11/2003 Borella et a1.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`”2005
`1507353
`OTHER PUBLICATIONS
`
`Muthitacharoen, Athicha et al., “A Low-bandwidth Network File
`System,” 2001, in Proc. of the 18th ACM Symposium on Operating
`Systems Principles, Banff, Canada, pp. 174-187.
`.
`(Cont1nued)
`
`Primary Examiner 7 Tuan Thai
`Assistant Examiner 7 Marwan Ayash
`(74) Attorney, Agent, or Firm 7 Carr & Ferrell LLP
`
`(57)
`
`ABSTRACT
`
`A network memory system comprises a first appliance and a
`second appliance. The first appliance receives data and deter-
`mines whether a portion ofthe data is locally accessible to the
`second appliance. The first appliance generates an instruction
`based on the determination and transfers the instruction to the
`
`second appliance over a communication network. The second
`appliance receives the instruction from the first appliance
`over the communication network and processes the instruc-
`tion to obtain the data. The second appliance then transfers the
`data to a computer.
`
`27 Claims, 12 Drawing Sheets
`
`m
`
`COMQETER
`
`CENTRAL
`CENTRAL
`BRANCH
`APPLIANCE
`SERVERS
`m
`APPLIANCEm
`DATA REQUEST m
`
`DATA REQUEST 91E
` DATA REQUEST m
`
`PROCESS DATA
`REQUEST
`
`
`GENERATE
`
`
`RESPONSE DATA
`
`RESPONSE DATA 525
`
`
`PROCESS RESPONSE DATA TO DETERMINE
`
` WHETHER PORTION OF RESPONSE DATA IS
`
`LOCALLY ACCESSIBLE TD BRANCH APP
`
`
`IF RESPONSE DATA IS NOT LOCALLV
`ACCESSIBLE TO BRANCH APP, GENERATE
`STORE INST TO STORE RESPONSE DATA AT
`INDEX WITHIN A DATABASE
`
`
`
`RESPONSE DATA WITH STORE
`
`INSTRUCTION 510
`
`PROCESS RESPONSE DATA WITH STORE INST
`STORE RESPONSE DATA AT INDEX WITHIN
`
`DATABASE
`
`FORWARD RESPONSE DATA
`RESPONSE DATA $.25
`
`
`
`
`
`445
`
`45°
`455
`
`RIV-1001 / Page 1 of 26
`
`

`

`US 8,370,583 B2
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`ggigagg‘: 3 13,4388: gollev emf
`1
`6738379 B1
`”004 1331531113551}
`617695048 B2
`7/2004 Gala?“ e i1
`«
`,
`0
`erg eta~
`6,791,945 B1
`9/2004 Levenson et al.
`6,859,842 B1
`2/2005 Nakamichi et a1.
`6,910,106 B2
`6/2005 Sechrest et al.
`6978 384 B1
`”/2005 Milliken
`7:007:044 B1
`2,2006 Rafert e, 31.
`7,020,750 B2
`3/2006 Thiyagaranjan et 31.
`7,035,214 B1
`4/2006 Seddigh et 31.
`7,069,342 131
`6/2006 Biederman
`7,113,962 B1
`9/2006 Kee et a1.
`7,120,666 B2
`10/2006 McCanne et a1.
`7,145,889 B1
`12/2006 Zhang et a1.
`7,200,847 B2*
`4/2007 Straube etal.
`7,215,667 B1
`5/2007 Davis
`7,242,681 B1
`7/2007 Van Bokkelen etal.
`
`................ 719/313
`
`72665645 BZ
`73185100 32
`7,366,829 B1
`7,380,006 B2
`7,383,329 B2
`7,383,348 B2
`
`”007 Garg etal
`“2008 Demeretal
`4/2008 Luttrellet a1.
`5/2008 Srinivas et al.
`6/2008 Erickson
`6/2008 Sekiet a1.
`
`7389357 B2
`7,389,393 B1
`7,417,991 B1
`7,420,992 B1
`7,428,573 B2
`7,451,237 B2
`7,453,379 B2
`7,457,315 B1
`7.471.629 B2
`75325134 132
`323:3”; 3%
`«
`,
`716195545 BZ
`7,620,870 B2
`7,639,700 B1
`7,676,554 B1
`7714 747 B2
`7:746:781 B1
`7849134 B2
`7:853:699 32
`7,953,869 132
`2001/0054084 A1
`2002/0007413 A1
`2002/0040475 A1
`2002/0078242 A1
`2002/0101822 A1
`2002/0107988 A1
`2002/0116424 A1
`2002/0131434 A1
`”OZ/0163911 A1
`2002/0169818 A1
`2002/0181494 A1
`2002/0188871 A1
`2002/0194324 A1
`2003/0009558 A1
`2003/0123481 A1
`2003/0142658 A1
`2003/0149661 A1
`2003/0233431 A1
`2004/0008711 A1
`2004/0047308 A1
`2004/0083299 A1
`2004/0114569 A1
`2004/0117571 A1 *
`2004/0123139 A1
`2004/0199771 A1
`2004/0202110 A1
`2004/0203820 A1
`2004/0205332 A1
`2004/0243571 A1
`2004/0255048 A1
`2005/0010653 A1
`
`@2008 Duffie em.
`6/2008 Karr et 31.
`8/2008 Crawford et 31,
`9/2008 Fang etal.
`9/2008 McCanne etal.
`11/2008 Takekawa eta1.
`11/2008 Plamondon
`11/2008 Smith
`12/2008 Melpignano
`50009 Samuelsetal
`$883 $113231: al~
`~
`“/2009 samuels et 31'
`11/2009 Srinivasan et a1.
`12/2009 Nabhan et a1.
`3/2010 Malmskog et a1.
`5,2010 Fallon
`6,2010 Xiang
`12,2010 McCanne e, 31.
`12/2010 Wu eta1.
`5/2011 Demmer etal.
`12/2001 Kosmynin
`1/2002 Garcia-Luna-Aceves et al.
`4/2002 Yap eta1.
`6/2002 Viswanath
`8/2002 Ayyagarietal.
`8/2002 Jordan
`8/2002 Rfidemlacheretal
`”002 VUkOVIC etal
`“/2002 Wee
`11/2002 Stewart et al.
`12/2002 Rhee
`12/2002 Noehring et al.
`12/2002 G h
`1,2003 angehezkel
`7/2003 Neale eta1.
`7/2003 Ofuji et 31.
`8/2003 Mitchell et a1.
`12/2003 Reddy et a1.
`1/2004 Lahti et a1.
`3/2004 Kavanagh et a1.
`4/2004 D1etz et 31'
`6/2004 Naden et a1.
`6/2004 Chang et a1.
`6/2004 Aiello et 31.
`10/2004 Morten et 31~
`10/2004 Kim
`10/2004 Billhartz
`10/2004 Bouchard et al.
`12/2004 Judd
`12/2004 Lev Ran et 31,
`1/2005 McCanne
`
`.................. 711/162
`
`2005/0044270 A1
`2005/0053094 A1
`2005/0091234 A1
`2005/0111460 A1
`2005/0131939 A1
`2005/0132252 A1
`2005/0141425 A1
`2005/0171937 A1
`2005/0177603 A1
`2005/0190694 A1
`2005/0210151 A1
`2005/0220019 A1
`2005/0235119 A1
`2005/0256972 A1
`2005/0278459 A1
`2005/0286526 A1
`2006/0013210 A1
`2006/0031936 A1
`2006/0036901 A1
`
`2006/0059171 A1
`2006/0059173 A1
`2006/0117385 A1
`2006/0143497 A1
`2006/0195547 A1
`
`2006/0212426 A1
`2006/0218390 A1
`2006/0227717 A1
`2006/0250965 A1
`2006/0280205 A1
`2007/0002804 A1
`2007/0011424 A1
`2007/0110046 A1
`2007/0115812 A1
`2007/0127372 A1
`2007/0140129 A1
`2007/0174428 A1
`2007/0195702 A1
`2007/0198523 A1
`2007/0226320 A1
`2007/0244987 A1
`2007/0245079 A1
`2007/0258468 A1
`2007/0263554 A1
`2007/0276983 A1
`2008/0005156 A1
`2008/0013532 A1
`2008/0133536 A1
`2008/0184081 A1
`2008/0229137 A1
`2008/0243992 A1
`2008/0313318 A1
`2008/0320151 A1
`2009/0060198 A1
`2009/0100483 A1
`2009/0158417 A1
`2009/0175172 A1
`2009/0234966 A1
`2010/0011125 A1
`2010/0115137 A1
`2010/0290364 A1
`
`2/2005 Grove etal.
`3/2005 Cain eta1.
`4/2005 Hsu et al.
`5/2005 Sahita
`6/2005 Douglis eta1.
`6/2005 Fifer eta1.
`6/2005 Foulds
`8/2005 Hughesetal.
`8/2005 Shavit
`9/2005 Ben-Nun eta1.
`9/2005 Abdo et a1.
`10/2005 Melpignano
`10/2005 Sechrestetal.
`11/2005 Cochran et al.
`12/2005 Boucher et al.
`12/2005 Soodetal.
`1/2006 Bordognaet 31,
`2/2006 Nelsonetal.
`2/2006 Yang etal.
`
`3/2006 Borthakuretal.
`3/2006 Hirsch etal.
`6/2006 Mester et al.
`6/2006 Zohar et al.
`8/2006 Sundarrajan et al.
`
`9/2006 Shakara eta1.
`9/2006 Loughran et al.
`10/2006 van den Berg et a1.
`11/2006 Irwin
`12/2006 Cho
`1/2007 Xiong etal.
`1/2007 Sharmaetal.
`5/2007 Farrell et al.
`5/2007 Hughes
`6/2007 Khan etal.
`6/2007 Bauer et al.
`7/2007 LevRanetal.
`8/2007 Yuen eta1.
`8/2007 Hayim
`9/2007 Hager et al.
`10/2007 Pedersen eta1.
`10/2007 Bhattacharjee eta1.
`11/2007 Bennett
`11/2007 Finn
`11/2007 Zohar et al.
`1/2008 Edwards eta1.
`1/2008 Garner et al.
`6/2008 Bjorneretal.
`7/2008 Hama eta1.
`9/2008 Samuels eta1.
`10/2008 Jardetzky eta1.
`12/2008 Vermeulen eta1.
`12/2008 McCanne et a1.
`3/2009 Little
`4/2009 McDowell
`6/2009 Khanna et al.
`7/2009 Prytz et a1.
`9/2009 Samuels eta1.
`“2010 Yang etal
`5/2010 K1m et a1.
`11/2010 Black
`
`OTHER PUBLICATIONS
`Zhao et al., “Analysis and Improvement on IPSEC Anti-Replay Win-
`d
`P
`1,, 2003. IEEE’
`553 558
`4
`PP~
`0W “”000 ,
`,
`'
`Singh et al. ,“Future of Internet SecurityilPSEC”; 2005; pp. 1-8.
`“Shared LAN Cache Datasheet”, 1996, http://www.lancache.com/
`s1cdata.htm.
`.
`.
`.
`.
`.
`“
`.
`Sprlng et al., A protocol-1ndependent techn1que for e11m1nat1ng
`redundant network traffic,” ACM SIGCOMM Computer Communi-
`cation Review, vol. 30, Issue 4 (Oct. 2000) pp. 87-95, Year of Publi-
`cation: 2000.
`
`RlV-1001 / Page 2 of 26
`
`

`

`US 8,370,583 B2
`Page 3
`
`B. Hong, D. Plantenberg, D. D. E. Long, and M. Sivan-Zimet.
`“Duplicate data elimination in a SAN file system,” InProceedings of
`the 21 st Symposium on Mass Storage Systems (MSS ’04), Goddard,
`MD, Apr. 2004. IEEE.
`You,L.L.andKaramanolis, C. 2004. Evaluationofeflicientarchival
`storage techniques, In Proceedings of the 21st IEEE Symposium on
`Mass Storage Systems and Technologies (MSST).
`Fred Douglis andArun lyengar, Application specific Delta-encoding
`Via Resemblance Detection, Published in the 2003 USENIXAnnual
`Technical Conference.
`
`You L. L. et a1., “Deep Store an Archival Storage System Architec-
`ture,” Data Engineering, 2005, ICDE 2005, Proceedings. 21st.inti
`Conf on Tokyo, Japan, Apr. 5-8, 2005, pp. 12.
`t
`S
`F'l
`Ud'M b .“F' d'
`S'
`'1 Fl
`'
`L
`1e ys em,
`1
`an er
`in mg
`1m1ar 1es1na arge
`93:33 OCt' 1994’ Department Of Computer Science,Univers1ty (Ff
`Arizona. http://Webg11mpse.net/pubs/TR93-33.pdf. Also appears in
`the 1994 Winter USENIX Technical Conference.
`
`” TR
`
`* Cited by examiner
`
`RIV-1001 / Page 3 of 26
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet],of12
`
`US 8,370,583 B2
`
`
`
`
`
`
`oow
`
`MOELOJ<KHZMO
`
`dflfl
`
`ZOP<QZDE§OO
`
`meZfimz
`
`ddfl
`
`mmHDOm
`
`ddfl
`
`wmmFDQEOO
`
`dmfl
`
`
`
`l_H__
`
` IE
` nH1-
`
`
`
` 5E
`ID!
`
`
`
`
`
`
`
`
` ED
`LE-,
`
`
` IE
`
`
`
`
`
`
`
`
`
`MOEuOIOZ<mm
`
`
`
`
`
`
`
`
`
`
`
`
`mmm>mmm2&sz
`
`dufl
`
`meDOm
`
`dMfi
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mmm>mmm4<mhzmo
`
`dmfl
`
`mKMFDQEOO
`
`dwfl
`
`._.m<mOEn.F.OE
`
`RIV-1001 / Page 4 of 26
`
`

`

`US. Patent
`
`Feb. 5, 2013
`
`Sheet20f12
`
`US 8,370,583 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`awmw>mwm4<mkzwo
`
`%KMHDOM
`
`
`
`
`
`
`
`
`mmw>mmm4<mkzwo
`
`olnlm
`
`onmOA<mkzmo
`
`oom
`
`moEuOIOZ<mm
`
`2929232200
`
`VEO>>E2
`
`fl
`
`KMHDOK
`
`fl
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%wmm>mmmIOZ<mm
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Hm<mOEn.N.GE
`
`wKMFDQEOO
`
`dwlm
`
`RIV-1001 / Page 5 of 26
`
`
`
`

`

`US. Patent
`
`Feb. 5, 2013
`
`Sheet3of12
`
`US 8,370,583 B2
`
`
`
`wOTEO4<m._.Zm_o
`
`
`
`
`
`
`
`
`wmm>mmm46$:sz
`
`de
`
`mmHDOm
`
`dam
`
`4<mkzmo
`
`moz<3aa<
`
`awn
`
`ZO_._.<O_ZD_>=>_OO
`
`me>>th
`
`dam
`
`
`
`
`
`
`
`
`Ioz<mm
`
`mmmHDQEOO
`
`dwm
`
`Ioz<mm
`
`moz<jaa<
`
`awm
`
`mmzbom
`
`dam
`
`m.OE
`
`RIV-1001 / Page 6 of 26
`
`
`
`
`
`_-7$1".
`
`
`
` E
`
`oom
`
`NOTED
`
`dfim
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet4of12
`
`US 8,370,583 B2
`
`
`
`n_n_<Ioz<mmO._.m._m_wwmoo<>._._<QO._
`
`
`
`
`
`
`
`
`
`>I_I_<OO._._.02w_<H<QwaOmwmmu:
`
`
`
`
`
`m_._.<mm_Zm_O.n_n_<Ioz<mmOHmjm_wwm_00<
`
`
`
`
`
`H<<._.<QmeOmemmeHmO._.Hw2_mmOFm
`
`mm<m<k<o<Z_I._._>>Xtz_
`
`
`
`
`
`
`
`m_Z__>_mm.erO._.<._.<QmeOme—mmwMOOma
`
`
`
`w.<F<QmmZOawmmnoZOFEOQmMIka>>
`
`
`
`<._.<Dmmqumn.
`
`PmMDOmm
`
`wh<mmzw0
`
`
`
`<._.<DmmZOawmm
`
`
`
`fl<._,<DmmZOawwm
`
`a536mm<20l
`
`2“H838”.<29
`
`
`
`3.erDOmm<._.<D
`
`4<NFZMO
`
`4/»:sz
`
`Ioz<mm
`
`aa%wmw>mwwovm
`
`moz<_._n_m<
`
`moz<_._n_n*<
`
`mm._.Dn=>_OO
`
`
`
`mmOPwI._._>><._.<n_mmZOQWm—m
`
`
`
`
`
`mmZOFODmeZ
`
`
`
`
`
`kmz_mePmI._._>><H<DmszmmmmwwMOOmm
`
`
`
`Z_I._._>>meZ_._.<<._.<n_waOQmmmMKOHm
`
`
`
`
`
`v.0.“—
`
`mm
`
`
`
`<._.<DmeOQmm—m
`
`ww<m<k<o
`
`
`
`
`
`<._.<n_meOmwmmDm<>>m0m
`
`RIV-1001 / Page 7 of 26
`
`
`
`
`
`
`
`

`

`US. Patent
`
`Feb. 5, 2013
`
`Sheet 5 of 12
`
`US 8,370,583 B2
`
`m.O<n_
`
`
`
`mm0<n_>mO._.w_I>>O._u_
`
`
`
`m_._.<._.mmO<n_
`
`flZO_._.<_>_N_OH_Z_
`
`m_Z_u_
`
`
`
`mmDF<ZO_wm_Z_n_
`
`
`
`m_._m<._.Iw<I
`
`amm.._..__n_
`
`mwm<OO
`
`
`
`Im<ImEDF<ZO_m
`
`am_._m<._.
`
`
`
`mmm%m0<n_XOMIO
`
`
`
` Qmm_._..__u_mwm<OO
`
`
`
`OZ_._._Om_._._m_-wNH
`
`mmIm<I
`
`RIV-1001 / Page 8 of 26
`
`

`

`US. Patent
`
`Feb. 5, 2013
`
`Sheet 6 of 12
`
`US 8,370,583 B2
`
`aa
`
`a
`
`
`
`mzzzmmzmoO._.<._.<DmmZOmmwmwwwoomm
`
`
`
`
`
`
`
`
`
`m_<._.<Dwszawmm”.0ZOFKOQmMIFmI>>
`
`
`
`
`
`n_n_<Ioz<mmOHm._m_wwm_oo<>j<OO._
`
`
`
`O._.m._m:wmm_oo<>._._<004w_<._.<DmeOammmu:
`
`
`
`
`
`
`
`<H<QwaOamm—m
`
`
`
`g<._.<n_mszmmmm
`
`(Had
`
`FwwDme
`
`wk<mmzm0
`
`wmmOOmm
`
`aSwamm<20l
`
`BSwami<55
`
`
`
`%FmMDOmm<._.<D
`
`.2??sz
`
`4,11%sz
`
`woz<_..n_n_<
`
`Ioz<mm
`
`moz<3nE<
`
`
`
` %mflw>mmmmmFDQEOO
`
`
`
`
`
`
`
`
`
`OHHmz_m_>m__m._.wmm._.<mwzm0_n_n_<Ioz<mm
`
`
`
`
`
`
`
` mm<m<h<0<Z_I.:>>Xm_02_H<<H<DmmZOmwmmm_>m:m.rm_w_
`
`
`
`aZOFODmHmZw>m=m._.wm
`
`
`
`
`
`._.wZ_m>m=m:.m_mwmmooma
`
`
`
`Z_I._._>>XMDZ_H<<H<DmmZOQmmmm>m_m.rmw_
`
`
`
`
`
`o.®_n_
`
`
`
`mm<H<QwaOQmmm
`
`wm<m<._.<D
`
`
`
`
`
`<._.<DmeOawmmQm<>>mOn_
`
`RIV-1001 / Page 9 of 26
`
`
`
`
`
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet7of12
`
`US 8,370,583 B2
`
`4<m._.2m_0
`
`4/??sz
`
`moz<3aa<
`
`%aa
`
`
` %mmm>mmmmmSagoo
`
`IOZ<Mm
`
`mUZ<_._n_n_<
`
`
`
`<._.<Dmszmwmm
`
`
`
`mlNM<._.<DmeOmwmm
`
`meaOm—m
`
`mF<mm2m0
`
`
`
`<._.<DmmeOm—n.
`
`
`
`EFwwDOm—m<._.<n_
`
`
`
`aHmMDOmm<._.<n_
`
` n_n_<
`
`
`
`w.<.,_.<DmmZOmmm—m“.0ZO_._.mOn_KMIHMI>>
`
`
`m_Z__>_mmFm0OF<._.<DmszmwmmwwmoOmm
`
`
`Ioz<mmOp.m3m_mmm_oo<>.S<OO._
`
`
`
`>j<004m_<H<omszammm“.0ZOFKOn.n:
`
`
`
`mkémeO.Qn_<Ioz<mmO._.m._m_mmm_00<
`
`
`
`
`
`
`
`m_._m_mwm_00<>j<OO:_m>m:w:.mmOF._.wZ_m_>m_~:.mm
`
`
`
`
`
`map—bOHHmz_mmOHmwhémzmwQz<mm<m<._.<o<Z_I._._>>
`
`
`XmQZ_h<<F<Dmszame”.0ZOFKOn.
`
`
`
`
`
`nozOFmOn.m._m_mwmoo<>._._<OO._-ZOZ
`
`
`
`mm<m<H<DZ_I.:>>xmoz_F<<H<Dmszmwmm
`
`
`
`
`
`
`
`In.._.wm_DOME<H<Q
`
`
`
`
`
`<n.OE
`
`
`
`m_._m:wwm_oo<>._._<OO._-ZOZ
`
`
`
`<._.<n_mszammmn_OZOFKOQ
`
`
`
`mmOHwQZ<m>m=w:.m_mIt>>
`
`%sz_._.ODm_.rmZ_
`
`RIV-1001 / Page 10 of 26
`
`
`
`
`
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet80f12
`
`US 8,370,583 B2
`
`4<mFZmo
`
`awmm>mmw
`
`mn.OE
`
`Z
`
`._<m._.Zm_O
`
`%MOZ<_._n.n_<
`
`Ioz<mm
`._.mz_meFwDZ<m>w_wz.mmI._._>><._.<DmeOQmmm
`
`
`
`n_OZOFmOQm_._m__mmm_oo<>j<OO+ZOZmmwOOmn.
`
`
`
`
`%moz<_._n_n_<
`
`
`
`
`
`
`
`“.0ZOFmOamjmzmwmoo<>._._<OO._m_>m:m.rmm
`
`
`
`
`
`mw<m<._.<n_2_I._._>>XmDZ_._.<<._.<DmmZOmem
`
`
`
`
`
`mvn
`
`omw
`
`mmw
`
`own
`
`mow
`
`mmFDQEOO
`
`%.
`
`
`
`
`
`OPmOn.Qm>m_m._.m_m20m“.<._.<QmmZOQmmmZEFmO
`
`
`
`
`
`
`
`
`
`
`
`.202QmmmmumzxxmhDz<<._.<n_mszmwmm”.0
`
`
`
`<._.<omszmmmm”.0ZO_._.w_On_m._m_mmm_oo<>._._<OO..
`
`
`
`
`
`
`
`m3m__mmm_oo<>4._<OO..-ZOZDmmmwumz/«E.mmOHw
`
`
`
`
`
`
`
`Z_I._._>>Xm_02_._.<<._.<QmeOmmmm“.0ZOFmOm
`
`
`
`
`
`mw<m<k<o
`
`
`
`
`
`<._.<DmszmwmmQm<>>m0m
`
`
`
`Nu<._.<Dmszmwmm
`
`RIV-1001 / Page 11 of 26
`
`
`
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet9of12
`
`US 8,370,583 B2
`
`omw
`
`oww
`
`EnXuZd$>
`
`%mo<mKMFZ_
`
`EOOZ<4
`
`%mo<mKMHZ_
`
` %moz<3mm<Ioz<mm
`
`amOmwmoomm
`
`>mOEwE
`
`dmm
`
`w.OE gmw<m<h<o
`
`RIV-1001 / Page 12 of 26
`
`
`
`

`

`US. Patent
`
`b.eF
`
`01
`
`1
`
`00S
`
`073,
`
`2B
`
`Ul
`
`mm<m<k<o
`
`08
`
`3.mmmo:
`
`mg8SEE?
`
`So20025
`
`mImova
`
`5,mowwmoomn.
`
`2qmw.
`
`,mE052
`
`awo<ummkz_
`
`
`
`8a.200z<>>I.
`
`moz<311<4<mkzmo
`
`own
`
`RIV-1001 / Page 13 of 26
`
`
`

`

`US. Patent
`
`Feb. 5, 2013
`
`Sheet 11 0f 12
`
`US 8,370,583 B2
`
`>mO_>_m=>_
`
`amoz<_._n_n_<
`
`
`
`me>>kmzmlrolr
`
`
`
`melmMHDQEOO
`
`crow
`
`
`
`mo_n_n_OPmeooow
`
`
`
`
`
`
`
`MOVEDemf...MOPED0200mm
`
`omowomor
`
`or.OE
`
`¥m0>>HmZ%mm>mmmDm_I._..DZOOmwmmEbQEOO
`
`aaw02<_._n_n_<m02<_._n_n_<
`
`>mO_>_m=>_>mO_>_m=>_
`me>>._.m_Z
`
`
`
`
`
`
`
`
`RIV-1001 / Page 14 of 26
`
`
`
`
`
`

`

`US. Patent
`
`Feb.5,2013
`
`Sheet120f12
`
`US 8,370,583 B2
`
`<_>_ZDEE.Z_(budmehw
`
`oneomorONE
`
`
`
`<_>_Z0200mm2—<F<DmmOFm
`
`ZOFODNFMZmePmIt>><._.<n_
`
`
`
`
`
`ZOFODmHmZmmOFwIt>><H<D
`
`
`
`(2201:2.(.220200mm<_>_Z._.wN__n_
`
`
`
`
`
`
`
`
`
`>mm>00m5EmOmmma>mm>00w5SEOnEmE>mm>00w5EmOumwm
`
`
`
`
`
`
`
`
`
`
`
`ZO:.<_.__OZOOmmDZ<ZOC.<_.__OZOOM_MDZ<ZO_._.<_.:OZOOm_mDZ<
`
`
`
`
`
`m_._.<n_n50253402.mH<On5OZ_QD._OZ_m_._.<n_n502.03402.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`n.<_>_>mO_>_m=>_._<D._.m_>n_<_>_>mO_>_m_>_._<ka_>n_<_>_>mO_>_m=>_._<ka_>
`
`
`
`
`
`
`
`
`
`
`
`
`
`<22OzoowmOF<._.<n_wozmmmmFDQEOO
`
`
`
`
`
`<._.<D“.029.55“”.EMIHMI>>m_Z__>_mmFm_D
`
`
`
`<_>_zDEE.OFm._m=wwm_oo<>4._<OO._w—
`
`
`
`
`
`DIE...Z_<._.<Dm>m=mkmmO._..52.m>m:m._.m_mwhammzmmu
`
`
`
`
`
`
`
`wm<m<H<QZ_I._._>>waZ_._.<<H<Dm>m=w:.m_mmm:
`
`
`
`
`
`
`
`sz_w>m_w:.m_mwwwOOmmom:
`
`
`
`ZOFODmeZm>mEHmm
`
`
`
`
`
`mm<m<k<o<Z_I.:>>XmDZ_._.<<22
`
`<20om<>>m0m8:
`
`:.O_n_
`
`RIV-1001 / Page 15 of 26
`
`
`
`
`

`

`US 8,370,583 B2
`
`1
`NETWORK MEMORY ARCHITECTURE FOR
`PROVIDING DATA BASED ON LOCAL
`ACCESSIBILITY
`
`BACKGROUND
`
`5
`
`1. Technical Field
`
`10
`
`The present invention relates generally to network memory
`and more particularly to a network memory architecture.
`2. Description of Related Art
`To allow remote employees access to an enterprise’s infor-
`mation systems, organizations typically choose between two
`networking approaches: centralized servers or distributed
`servers. Centralized server implementations have the advan-
`tage of simplicity since an information technology (IT) pro-
`fessional centrally manages, maintains, and enforces policies
`for the organization’s data.
`FIG. 1 illustrates a centralized server system 100 in the
`prior art. The centralized server system 100 includes a branch
`office 110 and a central office 120 coupled by a communica- 20
`tion network 130. The communication network 130 forms a
`WAN between the branch office 110 and the central office
`120.
`
`15
`
`Typically, the central servers 160 in the central office 120
`store the organization’s data. Computers 140 make requests 25
`for the data from the central servers 160 over the communi-
`cation network 130. The central servers 160 then return the
`
`data to the computers 140 over the communication network
`130.
`
`The communication network 130 typically comprises a 30
`private network (e.g., a leased line network) or a public net-
`work (e.g., the Internet). The connections to the communica-
`tion network 130 from the branch office 110 and the central
`
`office 120 typically cause a bandwidth bottleneck for
`exchanging the data over the communication network 130.
`The exchange of the data between the branch office 110 and
`the central office 120, in the aggregate, will usually be limited
`to the bandwidth of the slowest link in the communication
`network 130.
`
`35
`
`For example, the router 150 connects to the communica- 40
`tion network 130 by a T1 line, which provides a bandwidth of
`approximately 1.544 Megabits/second (Mbps). The router
`170 connects to the communication network 130 by a T3 line,
`which provides a bandwidth of approximately 45 Megabits/
`second (Mbps). Even though the communication network 45
`130 may provide an internal bandwidth greater than 1.544
`Mbps or 45 Mbps,
`the available bandwidth between the
`branch office 110 and the central office 120 is limited to the
`bandwidth of 1.544 Mbps (i.e., the T1 connection). Connec-
`tions with higher bandwidth to relieve the bandwidth bottle- 50
`neck across the communication network 130 are available,
`but are generally expensive and have limited availability.
`Moreover, many applications do not perform well over the
`communication network 130 due to the limited available
`
`bandwidth. Developers generally optimize the applications 55
`for performance over a local area network (LAN) which
`typically provides a bandwidth between 10 Mbps to Gigabit/
`second (Gbps) speeds. The developers of the applications
`assume small latency and high bandwidth across the LAN
`between the applications and the data. However, the latency 60
`across the communication network 130 typically will be 100
`times that across the LAN, and the bandwidth of the commu-
`nication network 130 will be 1/1ooth of the LAN.
`Alternatively, many organizations select the distributed
`server implementation to mitigate some of the problems with 65
`the centralized server implementation. FIG. 2 illustrates a
`distributed server system 200 in the prior art. The distributed
`
`2
`
`server system 200 includes a branch office 210, a central
`office 220, and a communication network 230. The commu-
`nication network 230 forms a WAN between the branch office
`210 and the central office 220.
`
`In the distributed server system 200, the branch servers 240
`(e.g., email servers, file servers and databases) are placed
`locally in the branch office 210, rather than solely in the
`central office 220. The branch servers 240 typically store all
`or part of the organization’s data. The branch servers 240
`generally provide improved application performance and
`data access. The branch servers 240 respond to a request for
`the organization’s data from the local data. For each request
`for the data, the central servers 270 potentially do not need to
`transfer the data over the communication network 130 (i.e.,
`the WAN). Synchronization and backup procedures are
`implemented to maintain the coherency between the local
`data in the branch office 210 and the data in the central office
`220.
`
`Unfortunately, managing the distributed server system 200
`is complex and costly. From a physical point of view, the
`distributed server system 200 with one hundred branch
`offices requires an order of one hundred times more equip-
`ment than the centralized server approach. Each piece of the
`equipment not only needs to be purchased, but also installed,
`managed, and repaired driving significant life cycle costs.
`The branch office 210 may need additional local IT personnel
`to perform operations because of this “Server Sprawl”. Fur-
`thermore, the multiplication of managed devices means addi-
`tional license costs, security vulnerabilities, and patching
`activities.
`
`In distributed server implementations (e.g., the distributed
`server system 200), the data, including the “golden copy” or
`most up-to-date version of mission critical data,
`is often
`stored (at least temporarily) only on the branch servers 240 in
`the branch office 21 0. Organizations implement complex pro-
`tocols and procedures for replication and synchronization to
`ensure that the mission critical data is backed up and kept
`in-sync across the WAN with the central servers 270.
`Furthermore, although FIG. 1 and FIG. 2 illustrate a single
`branch office and a single central office, multiple branch
`offices and multiple central offices exacerbate the previously
`discussed problems. For example,
`in a centralized server
`implementation having multiple branches, computers in each
`ofthe multiple branch offices make requests over the WAN to
`central servers for the organization’s data. The data transmit-
`ted by the central servers in response to the requests quickly
`saturate the available bandwidth of the central office’s con-
`nection to the communication network, further decreasing
`application performance and data access at the multiple
`branch offices. In a distributed server implementation having
`multiple branches, the cost to provide branch servers in each
`of the multiple branch offices increases, as well as the prob-
`lems oflicensing, security vulnerabilities, patching activities,
`and data replication and synchronization. Moreover, different
`branches may simultaneously attempt to modify the same
`piece of information. Maintaining coherency in a distributed
`implementation requires complex and error prone protocols.
`As well as implementing centralized servers or distributed
`servers, organizations also implement mechanisms for cach-
`ing to improve application performance and data access. A
`cache is generally used to reduce the latency of the commu-
`nication network (e.g., communication network 23 0) forming
`the WAN (i.e., because the request is satisfied from the local
`cache) and to reduce network traffic over the WAN (i.e.,
`because responses are local, the amount of bandwidth used is
`reduced).
`
`RIV-1001 / Page 16 of 26
`
`

`

`US 8,370,583 B2
`
`3
`Web caching, for example, is the caching of web docu-
`ments (i.e., HTML pages, images, etc.) in order to reduce web
`site access times and bandwidthusage. Web caching typically
`stores local copies of the requested web documents. The web
`cache satisfies subsequent requests for the web documents if
`the requests meet certain predetermined conditions.
`One problem with web caching is that the web cache is
`typically only effective for rarely modified static web docu-
`ments. For dynamic documents, there is a difficult tradeoff
`between minimizing network trafiic and the risk of the web
`cache serving up stale data. The web cache may serve stale
`data because the web cache responds to requests without
`consulting the server.
`Another problem is that the web cache does not recognize
`that two otherwise identical documents are the same if they
`have different Uniform Resource Locator (URL). The web
`cache does not consider the content or context of the docu-
`
`ments. Thus, the web cache caches the documents by URL or
`filename without a determination of the content or context of
`
`the document. Moreover, the web cache stores entire objects
`(such as documents) and cache-hits are binary: either a per-
`fect match or a miss. Even where only small changes are made
`to the documents, the web cache does not use the cached copy
`of the documents to reduce network trafiic.
`
`SUMMARY OF THE INVENTION
`
`The invention addresses the above problems by providing
`a network memory system and method implemented by the
`system. A network memory system comprises a first appli-
`ance and a second appliance. The first appliance receives data
`and determines whether a portion of the data is locally acces-
`sible to the second appliance. The first appliance generates an
`instruction based on the determination and transfers the
`
`instruction to the second appliance over a communication
`network. The second appliance receives the instruction from
`the first appliance over the communication network and pro-
`cesses the instruction to obtain the data. The second appliance
`then transfers the data to a computer.
`Advantageously, the first appliance may not transfer the
`data over the communication network if the data is locally
`accessible to the second appliance. Based on the instruction,
`the second appliance obtains the data and transfers the data to
`the computer. In one embodiment, the communication net-
`work may comprise a wide area network (WAN). The net-
`work memory system effectively reduces latency over the
`communication network, and reduces network trafiic by
`minimizing the amount of data sent over the communication
`network. In a further advantage, the first appliance may
`assume the role of the second appliance, and the second
`appliance may assume the role of the first appliance, thereby
`reducing network trafiic and latency for uni-directional and
`bi-directional communication over the communication net-
`
`work. The second appliance receives the data and determines
`whether a portion of the data is locally accessible to the first
`appliance. The second appliance generates another instruc-
`tion based on the determination and transfer the another
`
`instruction to the first appliance over the communication net-
`work. The first appliance receives the another instruction
`from the second appliance over the communication network
`and process the another instruction to obtain the data. The first
`appliance then transfers the data to another computer.
`In some embodiments, the first appliance generates the
`instruction indicating to store the data in a database. The
`instruction may also indicate to retrieve the data from the
`database. In some embodiments, the first appliance may gen-
`erate a plurality of instructions. The plurality of instructions
`
`5
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`may indicate a plurality of indexes for retrieving the data. The
`instruction may also indicate an index within the database for
`the data. Accordingly, the second appliance may store the data
`in the database based on the instruction. The second appliance
`may retrieve the data in the database based on the instruction.
`The first appliance may transfer to the second appliance a
`portion of the data that is not locally accessible. The second
`appliance receives the transferred portion of the data,
`retrieves the portion ofthe data that is locally accessible to the
`second appliance, and processes the portions to obtain the
`data. The first appliance may receive the data from a computer
`server. The second appliance may also transfer the data to the
`computer based on a request for the data over the communi-
`cation network by the computer. The network memory sys-
`tem therefore provides the advantages and simplicity of cen-
`tralized data storage, with the ability to quickly retrieve
`locally accessible data.
`Further, in some embodiments, the first appliance further
`determines whether the data is locally accessible to a third
`appliance. The first appliance then generates another instruc-
`tion to the third appliance based on the determination and
`transfers the another instruction to the third appliance over the
`communication network. The third appliance receives the
`another instruction from the first appliance over the commu-
`nication network and processes the another instructions to
`obtain the data. The third appliance then transfers the data to
`another computer. Furthermore, the network memory system
`may localize a copy of the data in multiple appliance/node
`implementations without the license fees and expenses, main-
`tenance, and security costs of distributed server hardware.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 illustrates a centralized server system in the prior
`art;
`FIG. 2 illustrates a distributed server system in the prior art;
`FIG. 3 illustrates a network memory system, in an exem-
`plary implementation of the invention;
`FIG. 4 illustrates a message sequence chart for the network
`memory system where a response to a data request is not
`locally accessible to a branch appliance, in an exemplary
`implementation of the invention;
`FIG. 5 illustrates data structures for the network memory
`system to determine whether a portion of the data is locally
`accessible to the branch appliance, in an exemplary imple-
`mentation of the invention;
`FIG. 6 illustrates a message sequence chart for the network
`memory system where the response to the data request is
`locally accessible to the branch appliance, in an exemplary
`implementation of the invention;
`FIG. 7A and FIG. 7B illustrate a message sequence chart
`for the network memory system where a portion of the
`response to the data request is locally accessible to the branch
`appliance, in an exemplary implementation of the invention;
`FIG. 8 illustrates a block diagram of the branch appliance,
`in an exemplary implementation of the invention;
`FIG. 9 illustrates a block diagram of a central appliance, in
`an exemplary implementation of the invention;
`FIG. 10 illustrates a network memory system between a
`first ofiice, a second ofiice, and a third office, in an exemplary
`implementation of the invention; and
`FIG. 11 illustrates a message sequence chart for the net-
`work memory system for discovery and reconciliation, in an
`exemplary implementation of the invention.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`The embodiments discussed herein are illustrative of one
`
`example of the present invention. As these embodiments of
`
`RIV-1001 / Page 17 of 26
`
`

`

`US 8,370,583 B2
`
`5
`the present invention are described with reference to illustra-
`tions, various modifications or adaptations of the methods
`and/or specific structures described may become apparent to
`those skilled in the art. All such modifications, adaptations, or
`variations that rely upon the teachings of the present inven-
`tion, and through which these teachings have advanced the
`art, are considered to be within the scope of the present
`invention. Hence, these descriptions and drawings should not
`be considered in a limiting sense, as it is understood that the
`present invention is in no way limited to only the embodi-
`ments illustrated.
`
`To provide improved application performance and data
`access, the network memory system generally comprises a
`first appliance and a second appliance. The first appliance
`receives data and determines whether a portion of the data is
`locally accessible to the second appliance. The first appliance
`generates an instruction based on the determination and trans-
`fers the instruction to the second appliance through the com-
`munication network.
`
`The network memory system provides that the second
`appliance processes the instruction to obtain the data and
`transfers the data to a computer. The data may be locally
`accessible to the second appliance, and the transfer to the
`computer may occur faster than transferring the data over the
`commlmication network. Accordingly, the second appliance
`transfers the data to computer without the first appliance
`transferring the data over the communication network that
`may have a high latency and low bandwidth. Thus, the net-
`work memory system operates to reduce latency and network
`traffic over the communication network.
`
`FIG. 3 illustrates a network memory system 300, in an
`exemplary implementation of the invention. The network
`memory system 300 includes a branch office 310, a central
`office 320, and a communication network 330. The branch
`office 310 includes computers 340, a branch appliance 350,
`and a router 360. The central office 320 includes central
`
`servers 370, a central appliance 380, and a router 390.
`In the branch office 310, the computers 340 are linked to
`the branch appliance 350. The branch appliance 350 is linked
`