US010621228B2
`
`(12) United States Patent
`US 10,621,228 B2
`(10) Patent No.:
`*Apr. 14, 2020
`(45) Date of Patent:
`Desmond etal.
`
`(54)
`
`(71)
`
`(72)
`
`METHOD AND APPARATUS FOR
`MANAGING DIGITAL FILES
`
`Applicant: NCM IP Holdings, LLC, Glen Ellyn,
`IL (US)
`
`Inventors: Christopher J. Desmond, Glen Ellyn,
`IL, (US); Naney L. Desmond, Glen
`Ellyn, IL (US); L. Michael Taylor,
`Chicago, IL (US)
`
`(73)
`
`Assignee: NCM IP Holdings, LLC, Glen Ellyn,
`IL (US)
`
`ny
`
`Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21)
`
`Appl. No.: 16/578,238
`
`(22)
`
`Filed:
`
`Sep. 20, 2019
`
`(65)
`
`(63)
`
`Prior Publication Data
`
`US 2020/0026727 Al
`
`Jan. 23, 2020
`
`Related U.S. Application Data
`
`Continuation of application No. 16/536,300, filed on
`Aug. 8, 2019, whichis a continuation of application
`(Continued)
`
`(51)
`
`Int. Cl.
`
`GO6F 16/58
`GO6F 16/901
`
`U.S. CL.
`
`(52)
`
`(2019.01)
`(2019.01)
`(Continued)
`
`CPE sisccues GO06F 16/5866 (2019.01); GO6F 16/51
`(2019.01); G06F 16/901 (2019.01); GO6F
`16/907 (2019.01); GO6F 3/0481 (2013.01)
`Field of Classification Search
`
`(58)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,694,514 A
`6,629,104 BL
`
`12/1997 Evans
`9/2003 Parulski
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CN
`EP
`
`102591922 B
`2466869 A3
`
`LO/2015
`2/2017
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Kustanowitz et al., “Motivating Annotation for Personal Digital
`Photo Libraries: Lowering Barriers while Raising Incentives,” Tech.
`Report HCIL-2004-18, U. Maryland, 2005 (10 pages).
`(Continued)
`
`Primary Examiner — Loc Tran
`(74) Attorney, Agent, or Firm — Nixon Peabody LLP
`
`(57)
`
`ABSTRACT
`
`A computer-implemented method of associating digital tags
`with digital files comprises storing a plurality of digital files
`having embedded therein content data and metadata includ-
`ing tags; receiving, via a user interface device ofa client
`device, a first tag label containing alphanumeric text created
`and inputted by a user of the client device; modifying, using
`a controller device, a selected first one ofthe tags of the
`metadata in a first of the digital files to include the first tag
`label; receiving, via the user interface device or another user
`interface device, an instructionto searchforall of the digital
`files having at least the first tag label; responsive to receiving
`the instruction, automatically searching for all of the digital
`files having at least the first tag label; and displaying, on a
`video display device associated with the client device,a first
`indicationof the first tag label.
`
`CPC ... GO6F 16/5866; GOG6F 16/907; GOG6F 16/901
`19 Claims, 50 Drawing Sheets
`See application file for complete search history.
`
`
`Select None
`Select All
`Delete Selected
`
`
`
`
`1 Fide(s) Selected
`File Attributes:
`Date:
`September 2, 2010 change
`Yellowstone, MT, USA change
`Location:
`‘Chrstepher Desmond, Nancy Desmond
`People:
`HyFishing in Yellewstene
`Event:
`Add
`Comments:
`Add
`Keywords:
`Add
`Alburn:
`ocoee
`Star Ranking:
`Sharing Rights: Add
`Recipe:
`Add
`Document Type: Add
`File Name:
`IMG 123 change
`
`EXHIBIT
`EXHIBIT
`2021
`2021
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`(51)
`
`Int. Cl.
`
`GO6F 16/51
`GO6F 16/907
`GO6F 3/0481
`
`(2019.01)
`(2019.01)
`(2013.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`US 10,621,228 B2
`
`Page 2
`
`Related U.S. Application Data
`
`No. 15/375,927, filed on Dec. 12, 2016, now Pat. No.
`10,423,658, which is a continuation of application
`No. 14/193,426, filed on Feb. 28, 2014, now Pat. No.
`9,552,376, which is a continuation-in-part of appli-
`cation No. 13/157.214, filed on Jun. 9, 2011, now Pat.
`No. 9,098,531.
`
`11/2013
`B2
`8,583,620
`Govindachetty
`11/2013
`B2
`8,583,668
`Higgins
`Bisdikian
` 1L/2013
`B2
`8,589,389
`Martinez
`11/2013
`B2
`8.589.486
`11/2013
`B2
`Naaman
`8,594,702
`Conwell
`12/2013
`B2
`8,606,021
`Xie
`1/2014
`B2
`8,626,699
`Davis
`3/2014
`B2
`8.671.154
`Brucher
`3/2014
`B2
`8.676.001
`Kalaboukis
`4/2014
`B2
`8,706,406
`Martinez
`6/2014
`B2
`8,745,133
`6/2014
`B2
`Davis
`8,762,285
`T2014
`5
`Pasceri
`D708, 196
`7/2014
`s
`Pasceri
`D708, 197
`Pasceri
`7/2014
`S
`D708, 198
`Kalaboukis
`7/2014
`B2
`8,769,099
`7/2014
`BL
`8,769,393
`Abhyanker
`Davis
`8/2014
`B2
`8,799,371
`Luo
`8/2014
`B2
`8,805,165
`8/2014
`B2
`8,806,365
`Stapleton
`8/2014
`B2
`8,810,597
`Akiya
`5/2008 Rhoad
`7,372,976 B2
`8/2014
`B2
`8,813,107
`Higgins
`9/2014
`B2
`1/2009 Toyama
`7,475,060 B2
`8,825,472
`Raghuveer
`1/2009 Lo
`7,480,669 B2
`Gao
`9/2014
`B2
`8.831.352
`4/2010 Gusmorino
`7,694,236 B2
`Kakarla
`9/2014
`B2
`8,849,854
`10/2010 Svendsen
`7,822,746 B2
`V2014
`B2
`Farmer
`8,849,909
`12/2010 Takahashi
`7,860,846 B2
`Pasceri
`Lo/2014
`5
`D715,819
`8/2011 Chen
`7,991,283 B2
`11/2014
`B2
`8,880,568
`Perezynski
`8/2011 Svendsen
`8,001,124 B2
`Lee
`11/2014
`B2
`8,890,888
`11/2014
`B2
`9/2011 Zheng
`8,015,144 B2
`8.892.495
`Hoffberg
`9/2011 Nair
`$.024,317 B2
`Kalaboukis
`12/2014
`B2
`8,914,342
`10/2011 Martinez
`8.032.508 B2
`Svendsen
`12/2014
`B2
`8,923,889
`Lim
`1/2015
`B2
`1/2011 Higgins
`8,055,675 B2
`8,930,848
`11/2011 Nair
`8,060,492 B2
`2/2015
`Bl
`8,949,212
`Dhandapani
`11/2011 Davis
`8,069,142 B2
`Xiao
`J2015
`B2
`8,954,425
`12/2011 Naaman
`§.086,048 B2
`2/2015
`B2
`8,966,121
`Josefsberg
`1/2012 Athsani
`8,108,778 B2
`3/2015
`B2
`8,972,177
`Zheng
`4/2012 Redstone
`§.150,844 B2
`/2015
`Bl
`8,998,422
`Snavely
`4/2012 King
`8,150,967 B2
`4/2015
`B2
`9,009,177
`Zheng
`Brucher
`4/2015
`B2
`9,014,511
`4/2012 Higgins
`8,166,016 B2
`4/2015
`B2
`4/2012 Hayashi
`8,166,168 B2
`9,015,617
`Stapleton
`§/2012 Zaltzman
`8,171,388 B2
`Takamura
`/2015
`B2
`9,015,633
`Adam
`4/2015
`B2
`7/2012 Dugan
`§,230,338 B2
`9,020,247
`Rathnavelu
`5/2015
`B2
`/2012 Govindachetty
`8,255,379 B2
`9,031,953
`9/2012 Karimoto
`8,264,570 B2
`Crawford
`5/2015
`B2
`9,032,320
`9/2012 Martinez
`8,271,506 B2
`Evans
`6/2015
`B2
`9,055,037
`10/2012 Martinez
`8,281,027 B2
`6/2015
`B2
`9,063,226
`Zheng
`10/2012) Amer-Yahia
`8,285,483 B2
`Hourie
`T2015
`B2
`9,076,259
`11/2012 Davis
`8,307,029 B2
`7/2015
`B2
`9,092,409
`Charaniya
`8/2015
`B2
`11/2012 Zheng
`8,315,959 B2
`9,098,545
`Abhyanker
`1/2013 Svendsen
`8,359,314 B2
`8/2015
`B2
`9,104,729
`Dong
`Conwell
`8/2015
`B2
`1/2013 Tendjoukian
`8,364,611 B2
`9,104,915
`2/2013 Luo
`8,380,039 B2
`Martinez
`8/2015
`B2
`9,110,903
`(2013 Martinez
`8,386,506 B2
`Amer-Yahia
`10/2015
`B2
`9,151,618
`3/2013 Bhatt
`8,390,702 B2
` LO/2015
`B2
`9,158,794
`Higgins
`3/2013) Krumm
`8.401.771 B2
`Svendsen
`10/2015
`B2
`9,160,802
`3/2013 Martinez
`$402,356 B2
`10/2015
`B2
`Murdock
`9,172,666
`4/2013 Buchmueller
`§.429,156 B2
`Stibel
`B2=12/2015
`9,202,200
`8,447,120 B2
`5/2013 Ji
`B2
`12/2015
`9,218,328
`Stapleton
`Churchill
`B2
`12/2015
`8.452.855 B2
`5/2013 Higgins
`9,224,172
`8,458,115 B2
`6/2013 Cai
`B2
`1/2016
`Li
`9,235,766
`8,463,931 B2
`6/2013 Evans
`B2
`1/2016
`Liu
`9,239,848
`$484,223 B2
`7/2013 Ota
`B2
`1/2016
`Pilskalns
`9,245,041
`$8,489,115 B2
`7/2013 Rodriguez
`B2
`2/2016
`9,261,376
`Zheng
`8,490,011 B2
`7/2013 Stapleton
`s
`3/2016
`Pasceri
`D751,597
`8,493,495 B2
`7/2013 D'Souza
`B2
`/2016
`Meadow
`9,311,396
`8,503,791 B2
`8/2013 Conwell
`Bhatt
`B2
`5/2016
`9,336,240
`8,504,073 B2
`8/2013 Svendsen
`B2
`6/2016
`9,372,931
`Capt
`8,520,979 B2
`8/2013 Conwell
`B2
`7/2016
`Thomee
`9,390,104
`D689,079 §
`9/2013 Edwards
`B2
`8/2016
`Li
`9,405,981
`D689,080 S
`(2013 Edwards
`B2
`8/2016
`9,418,485
`Lindberg
`D689,083 S
`9/2013 Pasceri
`B2
`8/2016
`Svendsen
`9,424,595
`D689,084 8
`9/2013 Pasceri
`B2
`10/2016
`Pilskalns
`9,460,116
`D689,085 $
`9/2013 Pasceri
`B2
`10/2016
`Poletto
`9,462,054
`8,538,811 B2
`9/2013 Higgins
`B2
`10/2016
`Sims
`9,465,513
`8,538,813 B2
`9/2013 Kakarla
`Bl
`10/2016
`9,471,834
`Filip
`8,542,294 B2
`9/2013 Bhatt
`B2
`11/2016
`Brucher
`9,483,500
`8.554.623 B2
`10/2013 Higgins
`B2
`11/2016
`9,501,577
`Zheng
`8.560.390 B2
`10/2013 Higgins
`B2
`11/2016
`Jaffe
`9,507,778
`8,560,517 B2
`10/2013 Yang
`
`MemoryWebEx. 2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`US 10,621,228 B2
`
`Page 3
`
`9,519,682 Bl
`9,535,563 B2
`9.536.146 B2
`9,552,376 B2
`9,557,162 B2
`9,576,253 B2
`9,582,546 B2
`9,593,957 B2
`9.600.484 B2
`9,626,685 B2
`9,646,025 B2
`9,654,570 B2
`9,674,650 B2
`9.679.456 B2
`9,680,929 B2
`9.683.858 B?
`9.691.073 B2
`9.706.345 B2
`9,710,961 B2
`9,721,188 B2
`9.754.226 B2
`9,772,745 B2
`9.787.799 B2
`9,805,123 B2
`9.811.879 B2
`9.858348 Bl
`9,870,572 B2
`9.881.179 B2
`9.882.994 B2
`9,942,121 B2
`10,019,850 B2
`10.037.327 B2
`10,068,178 B2
`10,074,093 B2
`10,083,533 B2
`10,110.541 B2
`10,187,543 B2
`10.223.701 B2
`10,230,803 B2
`10.235.444 B2
`10,242,051 B2
`10,282,752 B2
`10,288,433 B2
`10,289,643 B2
`10.303.975 B2
`10,311,611 B2
`10,318,110 B2
`10,331,863 B2
`10,360,352 B2
`10,430,452 B2
`10,445,346 B2
`2005/0060299 Al
`2006/0165380 AL*
`
`2007/0118508 Al
`2007/0282908 Al
`2008/0040034 Al
`2008/0051994 Al
`I
`5
`Oe aL.
`2008/0250398 Al
`2009/0013041 Al
`2009/0019085 Al
`2009/0049408 Al
`2009/010670S Al
`2009/0113350 AL*®
`
`5/2007 Svendsen
`12/2007 Van Der Meulen
`2/2008 Kanno
`2/2008. Fisher
`/
`Teams
`Senna Bee
`10/2008 Takahashi
`1/2009 Farmer
`1/2009 Abhyanker
`2/2009 Naaman
`4/2009 Takamura
`4/2009 Hibino ...ceceseee eee GOOF 16/41
`715/853
`
`12/2016 Pujara
`1/2017 Hoffberg
`1/2017 Zheng
`1/2017 Desmond
`1/2017 Rodriguez
`2/2017 Zaltzman
`2/2017 Hartford
`3/2017 Zheng
`3/2017 Davis
`/2017 Martinez
`5/2017 Boyns
`5/2017 Bisdikian
`6/2017 Hartford
`6/2017 East
`6/2017 Tseng
`6/2017 Zheng
`6/2017 Tseng
`7/2017 Davis
`9/2017 Setlur
`8/2017 Adam
`9/2017 Zheng
`9/2017 Hasenei
`10/2017 Gme
`10/2017 Nair
`11/2017 Miller
`1/2018 Higgins
`1/2018 Chapin
`1/2018 Patton
`1/2018 Bisdikian
`4/2018 Poletto
`7/2018 Lindberg
`7/2018 Thomee
`9/2018 van Zwol
`9/2018 Higgins
`9/2018 Bhatt
`10/2018 Li
`1/2019 Laheanski
`3/2019 King
`3/2019 Higgins
`3/2019 Poletto
`3/2019 Shinn
`5/2019 Athsani
`5/2019 Zheng
`5/2019 Brucher
`5/2019 Adam
`6/2019 Stoop
`6/2019 Naaman
`6/2019 Patton
`7/2019 Patton
`10/2019 Ross
`10/2019 Govindachetty
`3/2005 Filley
`7/2006 Tanaka oo. GLB 27/34
`386/297
`
`11/2009 Stapleton
`2009/0288005 AL
`References Cited
`(56)
`2009/0290812 Al=11/2009 Naaman
`U.S. PATENT DOCUMENTS
`2009/0307618 AL
`12/2009 Lawler
`2009/0325602 Al
`12/2009 Higgins
`2010/0041419 Al
`2/2010 Svendsen
`2010/0046842 Al
`2/2010 Conwell
`2010/0053371 Al
`3/2010 Karimoto
`2010/0064239 Al
`3/2010 Crawford
`2010/0082427 Al
`4/2010 Burgener
`2010/0107125 AL
`4/2010 Ockene
`2010/0153348 Al
`6/2010 Perezynski
`2010/0162411 Al
`6/2010 Chang
`201L0/OL71763 Al
`7/2010 Bhatt
`2010/0182341 Al
`7/2010 Lee
`2010/0185509 Al
`7/2010 Higgins
`2010/0241689 Al
`9/2010 Davis
`2010/0241944 Al
`9/2010 Athsani
`2010/0268717 AL
`10/2010 Pilskalns
`2010/0268766 Al
`10/2010 Bouget
`2010/0280913 AL
`11/2010 O'Sullivan
`2010/0293035 Al
`11/2010 Athsani
`2010/0293193 AL
`11/2010 Harrison
`2011/0040779 Al
`2/2011 Svendsen
`2011/0093458 Al
`4/2011 Zheng
`2011/0109769 Al
`5/2011 Bhatt
`2011/0113064 Al
`5/2011 Govindachetty
`2011/0145258 Al
`6/2011 Kankainen
`2011/0191014 Al
`8/2011 Feng
`2011/0191253 Al
`8/2011 Pilskalns
`2011/0202267 Al
`8/2011 Amer-Yahia
`2011/0208426 Al
`8/201L Zheng
`2011/0289031 Al
`LL/2011 Zheng
`2011/0301832 Al
`12/2011 Zheng
`2011/0314016 AL
`12/2011 Svendsen
`2012/0114249 Al
`/2012 Conwell
`2012/0158755 Al
`6/2012 Gammill
`2012/0218150 Al
`8/2012 Oyabu
`2012/0220311 Al
`8/2012 Rodriguez
`2012/0251011 Al
`10/2012 Gao
`2012/0266090 Al
`10/2012 Nealer
`2012/0278171 Al
`11/2012 Tang
`2012/0278767 Al
`11/2012 Stibel
`2012/0329441 Al
`12/2012 Tseng
`2012/0331091 Al
`12/2012 Tseng
`2013/0018881 AL
`1/2013" Bhatt
`2013/0036165 Al
`2/2013 Tseng
`2013/0063613 Al
`3/2013 Conwell
`2013/0073202 Al
`3/2013 Zheng
`2013/0101157 Al
`4/2013) Li
`2013/0138685 Al
`5/2013 Brucher
`2013/0141612 Al
`6/2013 Bhatt
`2013/0151597 Al
`6/2013" Akiya
`2013/0202198 Al
`8/2013 Adam
`2013/0275536 Al
`10/2013 Murdock
`2013/0339440 Al
`12/2013 Balassanian
`2014/0059477 Al
`/2014 Wong
`2014/0071272 Al
`3/2014 Rodriguez
`2014/0089811 Al
`3/2014 Meadow
`2014/0101531 Al
`4/2014 Capt
`SeIGOL t= ele dane
`2014/0143247 Al
`§/2014 Rathnavelu
`enone a UG ee
`2014/018L089 Al
`/2014
`Desmon
`2014/0188880 Al
`7/2014. Abhyanker
`2014/0193087 Al
`7/2014 Conwell
`2014/0354628 Al
`12/2014 Lindberg
`2015/0039630 Al
`2/2015 Thomee
`2015/0070165 Al
`3/2015 East
`2015/0070397 Al
`3/2015 Miller
`2015/0116540 AL
`4/2015 Gilman
`201S/0117713 Al
`4/2015 Zheng
`2015/0156247 Al
`6/2015 Hensel
`2015/0186389 AL
`7/2015 Zheng
`2015/0213057 Al
`7/2015 Brucher
`2015/0213329 Al
`7/2015 Adam
`2015/0244794 Al
`8/2015 Poletto
`2015/0244833 Al
`8/2015 Grue
`2015/0358224 AL
`12/2015 Poletto
`2016/0092741 Al
`3/2016 Li
`
`2009/0132689 Al
`2009/0132941 AL
`2009/0135438 Al
`2009/0216704 Al
`2009/0222302 Al
`2009/0254867 Al
`2009/0265631 Al
`2009/0279794 Al
`
`5/2009 Zaltzman
`5/2009. Pilskalns
`5/2009 Chopra
`8/2009 Zheng
`9/2009 Higgins
`10/2009 Farouki
`10/2009 Sigurbjornsson
`11/2009 Brucher
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`US 10,621,228 B2
`
`Page 4
`
`(56)
`
`References Cited
`
`U.S, PATENT DOCUMENTS
`
`2016/0162512 Al
`2016/0247307 Al
`2016/0253358 Al
`2016/0314187 Al
`2016/0321269 Al
`2016/0328444 Al
`2016/0344888 Al
`2017/0024415 Al
`2017/01L03081 Al
`
`6/2016 Battistini
`8/2016 Stoop
`9/2016 Bhatt
`10/2016 Poletto
`11/2016 Thomee
`11/2016 Shinn
`11/2016 Lahcanski
`1/2017 Brucher
`4/2017 Jones
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`WO
`wo
`Wo
`
`2410414 Bl
`WO 2011/070225 Al
`WO 2013/019376 AL
`WO 2013/099704 Al
`
`10/2019
`6/2011
`2/2013
`7/2013
`
`OTHER PUBLICATIONS
`
`Miller et al., “Give and take: a study of consumer photo-sharing
`culture and practice.” CHI *07 Proceedings of the SIGCHI Confer-
`ence on Human Factors in Computing Systems, pp. 347-356, 2007
`(10 pages).
`Ameset al., “Why we tag: motivations for annotation in mobile and
`online media,” CHI °07 Proceedings of the SIGCHI Conference on
`Human Factors in Computing Systems, pp. 971-980, ACM, 2007
`(10 pages)10083533.
`Yeeet al., “Faceted Metadata for Image Search and Browsing”, CHI
`2003, pp. 401-408, 2003, ACM.
`Ferre, “CAMELIS: Organizing and Browsing a Personal Photo
`Collection with a Logical Information System”, Int. Conf. Concept
`Lattices and Their Applications, pp. 112-123, 2007, HAL.
`Tomasson et al., “PhotoCube: Effective and Efficient Multi-
`Dimensional Browsing of Personal Photo Collections”, ICMR * 11,
`2011, ACM.
`Bartolini et al., “Integrating Semantic and Visual Facets for Brows-
`ing Digital Photo Collections”, SBED, pp. 65-72, 2009.
`Trattner et al., “Evaluating Tag-Based Information Access in Image
`Collections”, Proceedings of the 23°? ACM Conference on Hyper-
`text and Social Media, pp. 113-122, 2012 ACM.
`Kang et al., “Capture, Annotate, Browse, Find, Share: Novel
`Interfaces for Personal Photo Management”, International Journal
`of Human-Computer Interaction, 23(3), pp. 315-337, 2007, Law-
`rence Eribaum Associates, Inc.
`Jaffe et al., “Generating Summaries and Visualization for Large
`Collections of GeoReferenced Photographs”, MIR’06, pp. 89-98,
`2006 ACM.
`Torniai et al., “Sharing, Discovering and Browsing Geotagged
`Pictures on the Web”, 2007, Hewlett-Packard Development Com-
`pany, L.P., pp. 1-18 (19 pages).
`Snavely et al., “Photo Tourism: Exploring Photo Collection in 3D”,
`SIGGRAPH °06 ACM Transactions on Graphics, vol. 25, Issue 3,
`pp. 835-846, 2006 ACM.
`
`Kisilevich et al, “Event-based analysis of People’s Activities and
`Behavior Using Flickr and Panoramic Geotagged Photo Collec-
`tions”, 14International Conference Information Visualization, pp.
`289-296, 2010 IEEE.
`
`Ahern et al., “World Explorer: Visualizing Aggregate Data From
`Unstructured Text in Geo-Referenced Collections”, JCDL’07, pp.
`1-10, 2007, ACM,
`Kopf et al., “Deep photo: model-based photograph enhancement
`and viewing”, ACM Transactions on Graphics, vol. 27, No. 5,
`Article 116, Dec. 2008, ACM (10 pages).
`Amundsen, J.; “Using the Geographical Location of Photos in
`Mobile Phones”; Master of Science in Computer Science submis-
`sion; Norwegian University of Science and Technology; Jul. 2008
`(112 pages).
`Gentile, L; “Using Flickr Geotags to Find Similar Tourism Desti-
`nations”; master thesis, 2011; Politecnico di Milano, Dept. of
`Computer Engineering (96 pages).
`Hollenstein, L.; “Capturing Vernacular Geography from Georeferenced
`Tags”; Master Thesis; Institute of Geography, University of Zurich;
`Nov. 2008 (139 pages).
`Nutanong, S. et al; “An Efficient Layout Method for a Large
`Collection of Geographic Data Entries”; Center for Automation
`Research, Institute for Advanced Computer Studies, Dept. of Com-
`puter Science, University of Maryland: pp. 717-720 (4 pages).
`Slingsby, A. et al: “Interactive tag maps and tag clouds for the
`multiscale exploration of large spatio-temporal datasets”; Informa-
`tion Visualization, pp. 497-504; 2007,
`[TV °07. 11" International
`Conference. ISSN: 1550-6037 (9 pages).
`Hoffman, A.; “Create Great iPhone Photos: Apps, Tips, ‘Tricks, and
`Effects”; copyright 2011; ISBN-10: 1-59327-285-5, ISBN-13: 978-
`1-59327-285-2 (216 pages).
`Chen, Y-F, et al.; “GeoTracker: Geospatial and Temporal RSS
`Navigation”; AT&T Labs—Research, Florham Park, NJ; WWW
`2007 / Track: Browsers and User Interfaces, Session: Smarter
`Browsing, pp. 41-50 (10 pages).
`Goodman, E.; “Destination Services: Tourist media and networked
`places”; School of Information, UC Berkeley; Mar. 2, 2007 (11
`pages).
`Rattenbury, T. et al.; “Towards Automatic Extraction of Event and
`Place Semantics from Flickr Tags”; SIGIR °07, Jul. 23-27, 2007;
`Amsterdam, The Netherlands; ACM 978-1-59593-597-7/07/0007
`(8 pages).
`Kadar, B. et al.; “Where Do Tourists Go? Visualizing and Analysing
`the Spatial Distribution of Geotagged Photography”; Cartographica
`48:2, pp. 78-88; 2013; University of Toronto Press; doi: 10.3138/
`carto.48.2.1839 (LL pages).
`Kennedy, L.. et al.: “How Flickr Helps Us Make Sense ofthe World:
`Context and Content
`in Community-Contributed Media Collec-
`tions”; MM '07, Sep. 23-28, 2007; Augsburg, Bavaria, Germany;
`Copyright 2007; ACM 978-1-59593-701-8/07/0009 (10 pages).
`Richard, G.et al.; “Geotagging Photographs for Distribution on the
`Web”; Mineral Physics Institute, Earth and Space Sciences Build-
`ing, Stony Brook University, Stony Brook. NY; date unknown (9
`pages).
`
`* cited by examiner
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 1 of 50
`
`US 10,621,228 B2
`
`FIG. 1
`
`Select None
`Select All
`Delete Selected
`
`IMG 123 change
`
`1 File(s) Selected
`File Attributes:
`September 2, 2010 change
`Date:
`Yellowstone, MT, USA change
`Location:
`Christopher Desmond, Nancy Desmond
`People:
`Fly Fishing in Yellowstone
`Event:
`Add
`Comments:
`Add
`Keywords:
`Add
`Alburm:
`eeoee
`Star Ranking:
`Sharing Rights: Add
`Recipe:
`Add
`Document Type: Add
`File Name:
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 2 of 50
`
`US 10,621,228 B2
`
`FIG. 2
`
`People:
`Jack Wong
`Cc] Wong
`Mary Firestone
`Zoe Peika
`Nick Persons
`
`Event: Suzanne & Anthony’s Wedding Reception 2010
`
`Camera Details: more
`
`Comments:
`Suzanne and Anthony's Wedding Party where the cousins posed
`for a photo in the grass. Note, Jack with the lollipop and the
`photographer with his shoe in the photo
`
`Location:
`Historical Society
`Lisle, IL 60532
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 3 of 50
`
`US 10,621,228 B2
`
`FIG. 3
`
`Thumbnail | Table
`
`Mary Bader and Clint Firestone
`Wedding October 11,1945
`
`Capt. Clint Firestone
`1952
`
`Germany Trip 1955
`
`Father's Day 1966
`
`Christopher 1*
`Year 1972
`
`
`
`College Break 1992
`
`
`
`EIU CollegeGraduation 1994
`
`Cairo, Egypt 1996
`
`Europe
`Backpacking
`
`Nancy James and
`Christopher Desmond
`Wedding October 17, 1998.
`
`Amazon Fishing Trip
`November 2009
`
`Cousins Party 2010
`
`Fishing in Yellowstone
`Montana September 2010
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 4 of 50
`
`US 10,621,228 B2
`
`FIG. 4
`
`Captions On | Off
`
`Christopher Desmond and Nancy Desmond Wedding — October 17, 1998
`
`Event &§
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 5 of 50
`
`US 10,621,228 B2
`
`
`
`Fue
`
`2
`
`
`
`
`
`gh
`
`
`MemoryWebEx. 2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 6 of 50
`
`US 10,621,228 B2
`
`FIG. 6
`
`Thumbnail | Table
`
`
`
`Steve Fire
`
`Suzanne Lucky
`
`Steve Fwah
`
`Ben Sherman
`
`Joe Schmo
`
`Mary Virginia
`
`Clint Dewitt
`
`Charlotte Killian
`
`Steve Fire
`
`Suzanne Lucky
`
`Steve Fwah
`
`BenSherman
`
`Joe Schmo
`
`MaryVirginia
`
`Clint Dewitt
`
`Charlotte Killian
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 7 of 50
`
`US 10,621,228 B2
`
`FIG. 7
`
`Clinton Dewitt Firestone IV
`
`July 12, 1896
`Birth:
`April 29, 1971
`Death:
`Parents: Clinton Dewitt Firestone Ill and Viola Miller
`
`Recipes
`
`Comments: He was a WWII U.S.Air force pilot and POW in WWII and veteran
`honorably discharged in December of 1947. He worked for 44 years for the
`Firestone Tire and Rubber Companyin retail, wholesale and original equipment
`sales, marketing and management. He was born in Akron, OH andis buried in
`Columbiana, OH.
`
`Edit bio
`
`Family Tree
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 8 of 50
`
`US 10,621,228 B2
`
`FIG. 8
`
`Tree | Table
`
`Descendants of Charlotte Killian Halter
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 9 of 50
`
`US 10,621,228 B2
`
`FIG. 9
`
`ssae
`
`
`
`Cousins Party 2010
`
`Fishing in Yellowstone
`Montana September 2010
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 10 of 50
`
`US 10,621,228 B2
`
`Original Handwritten Recipe
`
`Take thick part of coconut milk out into pan (about 4
`tablespoons), Curry paste, 1 spoon of lemon grass, 1
`spoon of ginger and garlic, Heat over high with boil and
`THENstir for 1 minute. Add meat (uncooked) and fry
`7
`until cooked over high heat
`“1
`is
`Add milk, brown sugar andsalt. Bring back to slight boil
`and constantly stir. Add veggies and soy sauce. Cook for
`about 10-14 minutes COVERED until veggies are cooked.
`Serve witha smile.
`
`Desmond's Yellow Thai Chicken Curry
`
`Curry Mix
`Coconutmilk (400 ml) - DO NOT SHAKE IT UP
`800 gram of chicken (4 chicken breast)
`Fish sauce (Nam Pla) Thai Bamboo Garden — Bottle
`Garlic (2 claves)
`<
`Broccoli { 2cups chopped)
`2 Peppers (chopped)
`2 Carrots (chopped)
`
`Chef: Barry Desmond
`
`1 Zucchini (chopped)
`Thai Basil (8 leaves)
`Lernon Grass(in jar) 1 teaspoon
`Chinese Ginger Root(in jar) 1 teaspoon
`
`Rice
`* Thai Rice (something that only takes 2 cups of water)
`*
`Dice chicken in bowl and add two tablespoons offish
`sauce. Let marinate for 20 minutes.
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 11 of 50
`
`US 10,621,228 B2
`
`FIG. 11
`
`
`Thumbnail | Table
`
`|Albumytvent___—|date|tocation_|# Photos
`
`ack Monk's Arrival
`29-Dec-2003
`Chicago,IL
`
`Mike Testy's 2nd Birthda 13-Sep-1983_|Minneapolis, MN
`
`Cubs Beat Cards 1998
`
`5-Aug-1998
`
`Chicago,
`
`IL
`
`13-Sep-1983|Minneapolis, MN
`
`Mike Testy's 3rd Birthda
`
`
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 12 of 50
`
`US 10,621,228 B2
`
`FIG. 12
`
`
`Thumbnail | Table
`
`26-Dec-2003
`
`Chicago,
`
`IL
`
`IL
`
`Albumn/Event
`ack Monk's Arrival
`
`Mike Testy's First Birthda
`Cubs Beat Cards 1998
`
`13-Sep-1983 Minneapolis, MN
`5-Aug-1998
`Chicago,
`
`
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 13 of 50
`
`US 10,621,228 B2
`
`FIG. 13
`
`
`Thumbnail | Table
`
`pe
`et+ Bade
`
`IIDIDEDPIDIDELDIDIDELEIDIDEDEIDIDIDEIDIDEDEIDIDELEIDIDEEDDD,2 788
`BRACRAREREDELODEILIDEDEDELEDIOLLIDEDEDEDDIDOLLIEDEL
`
`
`
`
`+ Bacon
`+ Bates
`+ Boone
`+ Danas
`+ Danes
`
`Sy Monk,CJ
`Rs
`x Monk,Jack
`
`b2~pu~d
`
`NhS
`ay2
`
`he3
`
`
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 14 of 50
`
`US 10,621,228 B2
`
`
`
`FIG. 14
`
`Thumbnail | Table
`
`7 5 7 9
`
`[a|
`
`
`|||oo|oo|
`if
`
`
`[Kilianjack|Son|0||_| a
`
`
`|os[|oo|o|
`
`|os[|oo|o|
`Le|oo
`illi
`
`[Kiian.jonn|GreatNephew|6||2
`
`|5|a|a|
`|2|Es
`|ia
`9
`|
`00
`
`LE|GreatGrandson__|199|
`199
`
`2 1 4
`2 1
`
`|Moore,Bertha|Greatiece|a|6|3|
`[siythe,Sarah_|sister|op|to
`
`
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 15 of 50
`
`US 10,621,228 B2
`
`FIG. 15
`
`
`
`Count
`German
`
`# Photos
`Ww
`
`Lucila &Roberto|| Montalcino|
`898 West St
`USA
`wn
`USA
`college 545 Market|on_|
`ed
`in
`Amazon Trip|| Manus_|
`USA
`
`cabin__|__999 Pine
`IL
`USA
`|u|
`rad School
`903 Plymouth
`USA
`
`uo Thumbnail | Table
`nmfelewnimooad)hh
`
`
`
`LA Equestrian Ctr]
`
`568 Horse Dr
`
`Glendale
`
`rary
`
`USA
`USA
`USA
`L
`USA
`49
`co
`Home|aaaMain|Anywhere |__|
`IL
`USA
`10,987
`USA
`un
`GAGrilParty|321silver_| Macon|GA_|
`USA
`
`singapore |_|
`
`
`
`MemoryWebEx. 2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 16 of 50
`
`US 10,621,228 B2
`
`
`
`FIG. 16
`
`Category | Card | Table
`
`|cher|ate|category_|
`
`SkinnyGermans_|Gerda|29Dec2003|Breakast_|
`
`Cookies|Mrsfields|2asuiy1978_|Dessert_|
`
`nt
`
`Mrs.Fields
`[dinner_|
`|VitoSpadavecchio|29Dec2003_|dinner_|
`|charlesaso|13Sept1988_|dinner_|
`
`Mudpie|NancyMonk|5.Aug1998_|Dessert_|
`
`&i
`
`Sy
`2 March 2011
`Daddio Pancakes
`
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`3 ;33333333333333333333333333
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 17 of 50
`
`US 10,621,228 B2
`
`FIG. 17
`
`Close [X}
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 18 of 50
`
`US 10,621,228 B2
`
`FIG. 18
`
`Advanced Search Filter
`
`Keywords:
`Date:
`Location:
`
`Add
`Add
`
`Document Type: Add
`
`Chicago, IL USA
`Yellowstone, MT USA
`Cologne, Germany
`
`Mike Cubbie
`Mary Larnb
`Christopher Monk
`Nancy Monk
`Dwight Schrut
`
`Jack’s 1* Birthday
`Fly Fishing in Yellowstone
`Raking Leaves
`Christmas 2010
`Thanksgiving 2010
`July 4'* Parade
`Add
`Album:
`Add
`Star Ranking:
`Sharing Rights: Add
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 19 of 50
`
`US 10,621,228 B2
`
`FIG. 19
`
`Captain Phil’s Memory-Webb
`
`Welcome, Captain Phil
`Last Login:
`11.18.2010
`
`Media
`# Photos
`# Videos
`@ Documents:
`
`count
`1,342
`75
`7t73
`
`i
`
`count
`80% complete
`61% complete
`
`License renewal due 1,15.2011 Cotte|
`
`My recent memories:
`123 Photos uploaded on 11.07.10
`2 alburns created 11.17.10
`12 visitors since last login date
`PeopleStats:
`
`Falburnscrestediii720. ‘lastname|#peopie|photos|videos|#docs_|2 albums created 11.17.10
`it People
`it Photos
`# Videos
`reeeee
`
`My recent Webb views:
`* Captain Phil 2010 (photo album)
`+
`Chicken Pot Pie (recipe)
`* Captain Phil (Timeline)
`
`Updates and Alerts:
`
`Event Stats:
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 20 of 50
`
`US 10,621,228 B2
`
`FIG. 20
`
`6050
`
`401
`
`Original Digital
`file(s) upload to
`MemoryWeb
`
`
`
`;
`.
`This PatyPeel
`Recognition
`Provider
`
`Third Party Facial
`Recognition
`‘System
`
`0501
`
`1101
`Third Party
`Geographical
`Third Party Media Provider
`Mapping Provider
`
`
`
`
`_...Qutside MWSystem_
`Inside MW System
`
`Third Party Media
`System (Data
`Exchange}
`
`Share to Social
`Network Provider
`System
`
`1201
`
`Individual /
`Group of
`People
`
`@xported from
`MemoryWeb to Storage
`stem
`
`1200
`
`
`
`1300
`
`Application Export System
`
`
`
`
`
`
`
`
` Third Party Geographical
`
`
`
`Relationship Table Phase
`
`
`
`
`System Reading
`Phase
`
`System Interpreting and
`Adding Data to
`
`100
`
`0200
`
`
`
`
`
`Location Mapping
`User Relationship
`
`System
`Table
`
`System Indexing
`"5
`‘Share to Individual
`DigitalFiles and
`System
`Tag Sata Blocks
`for a Digital File
`within User
`Relationship Table
`
`Application Digital
`Tag Organizer
`tem
`0600
`
`Continuous Link
`of Application
`Dot-Tags System
`o7oo
`
`Advanced Filters
`Syetern
`=
`0200
`
`Keyword Fast
`Search System
`i"
`0900
`
`MemoryWebEx. 2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr. 14, 2020
`
`Sheet 21 of 50
`
`US 10,621,228 B2
`
`FIG. 21
`
`0200
`
`System Reading Phase
`
`
`Data to Relationship Table Phase
`
`
`
`
`
`‘Original Digitalfile begins to
`upload ta MemoryWeb
`
`
`
`
`
`Recognition Provider
`
`Outside MW System
`utside MW System
`Inside MW System
`0400 Inside MW System
`
`
`
`Third Party Facial Recognition
`System
`
`Upload original
`file into MW
`
`System Indexing Digital Files and
`Tag Data Blocks for a DigitalFile
`within User Relationship Table
`0300
`
`User Relationship Table
`1900
`
`
`
` Third Party Facial
`
`
` ge
`
`Duplicate
`File?
`
`File not
`uploaded
`
`Locate EXIF
`image File
`Directories in
`Digital File
`
`Read and iterate through
`each EXIF tag item
`
`(Create duplicates
`of originalfile in
`different file sizes
`
`Save EXIF data to
`User Relationship
`Table for Digital
`
`
`
`
`Create new
`MW tag
`data
`
`book up MW
`Tag data
`;
`tog data
`exists?
`Windows Tag.
`Data Blocks!
`MemoryWeb
`Tag (Data Blocks)
`MemoryWeb
`Person Tag (Data
`Blocks}
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr. 14, 2020
`
`Sheet 22 of 50
`
`US 10,621,228
`
`B2
`
`FIG. 22
`
`EXIF Tags version 2.3 Image File Directories i
`0322
`(Data Blocks)
`o321
`
`Tag Labels
`
`EXIF Family
`Group Name
`
`0320
`
`‘0323
`
`0324
`
`0325
`
`Description Title
`
`eeeee
`peseriptonSubject
`+.
`
`OxO10e
`
`MemoryWebTag (Data Blocks)
`
`0360
`
`
`
`LemumemeiaAsset.Caption
`
`MediaAsset.StarRanking.
`
`o361
`
`362
`
`0363
`
`(acingee
`prginawhorsoodseng
`0334
`0335
`0364
`fediaAsset.DateCreated
`pindaetaien ete" fvo003 —
`
`af
`
`rnincopwighetoss
`
`Jmogesimagei.Dimensionswtheightey|Iain|
`0327
`0365
`fseso
`en)
`0328
`0366
`peeess
`
`
`
`kamera(GomeraMaher,ComeroModeler|p|
`
`
`avoncedPhotonsMake,lensModelet)|tte
`his is used to Inject Information that do not currently have
`og29
`EXIF standardized tags including Collection, People, Location
`0367
`ipe Name, Person Tag Data Blocks (0380), etc.
`0368.
`
`
`
`Description Rating
`
`0330
`0331
`0332
`0333
`
`ExifiFD
`
`SE
`
`Medansseuiccauoniongude
`
`a
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 23 of 50
`
`US 10,621,228 B2
`
`FIG. 23
`
`0340
`
`0360
`
`0361
`
`mate
`
`0373
`
`é
`MemoryWeb Tag (Data Blocks)
`
`MemoryWeb Tag
`
`MediaAsset. Filename
`
`:
`
`ediaAsset.UploadBatch.BatchName
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`WindowsImaging Component (WIC) Metadata
`(Data Blocks)
`
`0341
`
`Owner, Computer, etc’
`
`0342
`
`343
`
`ag ike
`
`File Item Type
`
`File Folder Path
`
`| |
`
`File (Offline availability, Offline status, Shared with,
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 24 of 50
`
`US 10,621,228 B2
`
`FIG. 24
`
`0380
`
`A
`
`MemoryWebPerson Tag (Data Blocks)
`
`MemoryWebEx.2021
`Samsung v. MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 25 of 50
`
`US 10,621,228 B2
`
`FIG. 25
`
` 406
`
`System request for
`thumbnails of afl
`f
`DigitalFiles that have
`unconfirmed faces
`
`System sends
`Digital Files to
`.
`Third Party
`Facial
`Recognition
`Provider
`
`-
`System receives
`response from
`Third Party Facial
`Recognition
`Provider
`
`‘System parses:
`identified faces
`and cerrespanding
`Facial Recognition
`data for each
`Digital Fite
`
`0400
`
`Outside MW System
`
`Inside MW System
`
`System records and associates specific
`face for Digital File with a MemoryWeb
`is Face Recognition
`Existing (Teained) Face 1D
`
`Person ID and sends to MemoryWeb
`above score
`
`
`in User Relationship
`Relationship
`Person Tag (Data Blocks)in User
`
`
`threshold?
`Tabhe?
`Table
`Retrieves
`
`Relationship Table
`
`Thumbnails ofall
`OigitalFiles that
`have uncorfirmed
`faces
`
`
`
`System generates virtual
`System generates facial
`
`
`unconfirmed Facial
`recognition recerd for unknown
`
`
`recognition record for person
`
`person and then sends
`and then sends information to
`
`
`information to MemoryWeb
`
`
`MemoryWeb Person Tag
`Person Tag (Data Blocks)in User
`
`
`
`(Data Blocks) in User
`Relationship Table
`Relationship Table
`
`
`
`
`
`Samsungv.
`
`MemoryWebEx. 2021
`MemoryWeb — IPR2022-00221
`
`MemoryWeb Ex. 2021
`Samsung v. MemoryWeb – IPR2022-00221
`
`

`

`U.S. Patent
`
`Apr.14, 2020
`
`Sheet 26 of 50
`
`US 10,621,228 B2
`
`FIG. 26
`
`O500
`
`0604
`
`Third Party Media Provider
`
`Outside MW System
`
`System receives contact names
`and IDs from Third Party Media
`
`fe Persoa
`Match,
`No
`
`Yes
`
`0510
`
`‘System add new person to Tag
`Data Blockof Relationship
`Table for Digital File
`
`System associates
`existing Person to