( 12 ) United States Patent
`Desmond et al .
`
`US 11,163,823 B2
`( 10 ) Patent No .:
`( 45 ) Date of Patent :
`* Nov . 2 , 2021
`
`US011163823B2
`
`( * ) Notice :
`
`( 54 ) METHOD AND APPARATUS FOR
`MANAGING DIGITAL FILES
`( 71 ) Applicant : Memoryweb , LLC , Glen Ellyn , IL
`( US )
`( 72 ) Inventors : Christopher J. Desmond , Glen Ellyn ,
`IL ( US ) ; Nancy L. Desmond , Glen
`Ellyn , IL ( US ) ; L. Michael Taylor ,
`Chicago , IL ( US )
`( 73 ) Assignee : Memoryweb , LLC , Glen Ellyn , IL
`( US )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U.S.C. 154 ( b ) by 226 days .
`This patent is subject to a terminal dis
`claimer .
`( 21 ) Appl . No .: 16 / 536,300
`Aug. 8 , 2019
`( 22 ) Filed :
`( 65 )
`Prior Publication Data
`Dec. 5 , 2019
`US 2019/0370285 A1
`Related U.S. Application Data
`( 63 ) Continuation of application No. 15 / 375,927 , filed on
`Dec. 12 , 2016 , now Pat . No. 10,423,658 , which is a
`( Continued )
`
`( 51 ) Int . Ci .
`G06F 16/58
`G06F 16/51
`
`( 2019.01 )
`( 2019.01 )
`( Continued )
`
`( 52 ) U.S. Cl .
`CPC
`
`GO6F 16/5866 ( 2019.01 ) ; GO6F 16/51
`( 2019.01 ) ; G06F 16/901 ( 2019.01 ) ; G06F
`16/907 ( 2019.01 ) ; G06F 3/0481 ( 2013.01 )
`( 58 ) Field of Classification Search
`CPC ... GOOF 16/5866 ; G06F 16/907 ; GO6F 16/901
`See application file for complete search history .
`
`( 56 )
`
`CN
`EP
`
`References Cited
`U.S. PATENT DOCUMENTS
`5,450,504 A
`5,694,514 A
`
`9/1995 Calia
`12/1997 Evans
`( Continued )
`FOREIGN PATENT DOCUMENTS
`
`10/2015
`102591922 B
`2/2017
`2466869 A3
`( Continued )
`
`OTHER PUBLICATIONS
`Marco et al . , “ Location privacy and public metadata in social media
`platforms : attitudes , behaviors and opinions ” , Springer Science +
`Business Media New York 2014 , 31 pages . *
`( Continued )
`
`Primary Examiner Loc Tran
`( 74 ) Attorney , Agent , or Firm — Nixon Peabody LLP
`
`( 57 )
`ABSTRACT
`A computer - implemented method of associating digital tags
`with digital files comprises storing a plurality of digital files
`having embedded therein content data and metadata includ
`ing tags ; receiving , via a user interface device of a client
`a
`device , a first tag label containing alphanumeric text created
`and inputted by a user of the client device ; modifying , using
`a controller device , a selected first one of the tags of the
`metadata in a first of the digital files to include the first tag
`label ; receiving , via the user interface device or another user
`interface device , an instruction to search for all of the digital
`files having at least the first tag label ; responsive to receiving
`the instruction , automatically searching for all of the digital
`files having at least the first tag label ; and displaying , on a
`video display device associated with the client device , a first
`indication of the first tag label .
`
`a
`
`a
`
`34 Claims , 50 Drawing Sheets
`
`Select None
`Select All
`Delete Selected
`
`1 File ( s ) Selected
`File Attributes :
`September 2 , 2010 change
`Date :
`Yellowstone , MT , USA change
`Location :
`People :
`Christopher Desmond , Nancy Desmond
`Fly Fishing in Yellowstone
`Event :
`Add
`Comments :
`Add
`Keywords :
`Add
`Album :
`Star Ranking :
`Add
`Sharing Rights :
`Recipe :
`Add
`Document Type : Add
`IMG 123 change
`File Name :
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`US 11,163,823 B2
`Page 2
`
`Related U.S. Application Data
`continuation of application No. 14 / 193,426 , filed on
`Feb. 28 , 2014 , now Pat . No. 9,552,376 , which is a
`continuation - in - part of application No. 13 / 157,214 ,
`filed on Jun . 9 , 2011 , now Pat . No. 9,098,531 .
`( 51 ) Int . Ci .
`GO6F 16/901
`G06F 16/907
`G06F 3/0481
`
`( 2019.01 )
`( 2019.01 )
`( 2013.01 )
`References Cited
`U.S. PATENT DOCUMENTS
`
`( 56 )
`
`5,835,616 A
`5,850,470 A
`5,982,912 A
`6,002,401 A
`6,134,339 A
`6,246,779 B1
`6,301,370 B1
`6,629,104 B1
`6,697,502 B2
`6,714,215 B1
`6,728,401 B1
`6,940,545 B1
`6,965,693 B1
`7,003,135 B2
`7,190,829 B2
`7,372,976 B2
`7,461,099 B1
`7,474,317 B2
`7,475,060 B2
`7,480,669 B2
`7,522,701 B2
`7,587,671 B2
`7,634,662 B2
`7,646,895 B2
`7,694,236 B2
`7,702,185 B2
`7,804,982 B2
`7,822,746 B2
`7,840,344 B2
`7,840,892 B2
`7,860,846 B2
`7,916,905 B2
`7,929,771 B2
`7,948,502 B2
`7,962,467 B2
`7,965,908 B2
`7,970,240 B1
`7,991,283 B2
`8,001,124 B2
`8,015,144 B2
`8,024,317 B2
`8,032,508 B2
`8,055,675 B2
`8,060,492 B2
`8,069,142 B2
`8,073,461 B2
`8,086,048 B2
`8,086,867 B2
`8,108,778 B2
`8,121,408 B2
`8,144,232 B2
`8,150,844 B2
`8,150,967 B2
`8,165,352 B1
`8,166,016 B2
`8,166,168 B2
`8,171,388 B2
`8,175,340 B2
`8,230,338 B2
`8,239,784 B2
`8,255,379 B2
`8,264,570 B2
`
`11/1998 Lobo
`12/1998 Kung
`11/1999 Fukui
`12/1999 Baker
`10/2000 Luo
`6/2001 Fukui
`10/2001 Steffens
`9/2003 Parulski
`2/2004 Luo
`3/2004 Flora
`4/2004 Hardeberg
`9/2005 Ray
`11/2005 Kondo
`2/2006 Hsieh
`3/2007 Zhang
`5/2008 Rhoad
`12/2008 Sharpe
`1/2009 Dolph
`1/2009 Toyama
`1/2009 Lo
`4/2009 Jensen
`9/2009 Saft et al .
`12/2009 Monroe
`1/2010 Haupt
`4/2010 Gusmorino
`4/2010 Keating
`9/2010 Howard
`10/2010 Svendsen
`11/2010 Sloo
`11/2010 Pyhalammi
`12/2010 Takahashi
`3/2011 Yen
`4/2011 Ko
`5/2011 Stanton
`6/2011 Howard
`6/2011 Hayashi
`6/2011 Chao
`8/2011 Chen
`8/2011 Svendsen
`9/2011 Zheng
`9/2011 Nair
`10/2011 Martinez
`11/2011 Higgins
`11/2011 Nair
`11/2011 Davis
`12/2011 Altman
`12/2011 Naaman
`12/2011 Freeman
`1/2012 Athsani
`2/2012 Omori
`3/2012 Larson
`4/2012 Redstone
`4/2012 King
`4/2012 Mohanty
`4/2012 Higgins
`4/2012 Hayashi
`5/2012 Zaltzman
`5/2012 Tsuitsui
`7/2012 Dugan
`8/2012 Hotelling
`8/2012 Govindachetty
`9/2012 Karimoto
`
`8,271,506 B2
`8,281,027 B2
`8,285,483 B2
`8,307,029 B2
`8,315,959 B2
`8,326,000 B2
`8,332,402 B2
`8,358,811 B2
`8,359,314 B2
`8,364,611 B2
`8,380,039 B2
`8,386,506 B2
`8,390,702 B2
`8,401,771 B2
`8,402,356 B2
`8,416,312 B2
`8,429,156 B2
`8,447,120 B2
`8,458,115 B2
`8,463,931 B2
`8,484,223 B2
`8,489,115 B2
`8,490,011 B2
`8,493,495 B2
`8,503,791 B2
`8,504,073 B2
`8,520,979 B2
`D689,079 S
`D689,080 S
`D689,083 S
`D689,084 S
`D689,085 S
`8,538,811 B2
`8,538,813 B2
`8,542,294 B2
`8,554,623 B2
`8,560,390 B2
`8,560,517 B2
`8,583,620 B2
`8,583,668 B2
`8,584,015 B2
`8,589,389 B2
`8,589,486 B2
`8,594,702 B2
`8,606,021 B2
`8,626,699 B2
`8,649,604 B2
`8,660,358 B1
`8,671,154 B2
`8,676,001 B2
`8,698,762 B2
`8,706,406 B2
`8,712,192 B2
`8,743,411 B2
`8,745,133 B2
`8,750,574 B2
`8,762,285 B2
`D708,196 S
`D708,197 S
`D708,198 S
`8,769,099 B2
`8,769,393 B1
`8,799,371 B2
`8,805,165 B2
`8,806,365 B2
`8,810,597 B2
`8,811,775 B1
`8,813,107 B2
`8,825,472 B2
`8,831,352 B2
`8,845,855 B2
`8,849,854 B2
`8,849,909 B2
`D715,819 S
`8,880,535 B1
`8,880,568 B2
`8,890,888 B2
`8,892,495 B2
`8,914,342 B2
`
`9/2012 Martinez
`10/2012 Martinez
`10/2012 Amer - Yahia
`11/2012 Davis
`11/2012 Zheng
`12/2012 Jung
`12/2012 Forstall
`1/2013 Adams
`1/2013 Svendsen
`1/2013 Tendjoukian
`2/2013 Luo
`2/2013 Martinez
`3/2013 Bhatt
`3/2013 Krumm
`3/2013 Martinez
`4/2013 Matsunaga
`4/2013 Buchmueller
`5/2013 Ji
`6/2013 Cai
`6/2013 Evans
`7/2013 Ota
`7/2013 Rodriguez
`7/2013 Stapleton
`7/2013 D'Souza
`8/2013 Conwell
`8/2013 Svendsen
`8/2013 Conwell
`9/2013 Edwards
`9/2013 Edwards
`9/2013 Pasceri
`9/2013 Pasceri
`9/2013 Pasceri
`9/2013 Higgins
`9/2013 Kakarla
`9/2013 Bhatt
`10/2013 Higgins
`10/2013 Higgins
`10/2013 Yang
`11/2013 Govindachetty
`11/2013 Higgins
`11/2013 Osten
`11/2013 Bisdikian
`11/2013 Martinez
`11/2013 Naaman
`12/2013 Conwell
`1/2014 Xie
`2/2014 Steinberg
`2/2014 Bergboer
`3/2014 Davis
`3/2014 Brucher
`4/2014 Wagner
`4/2014 Kalaboukis
`4/2014 Thota
`6/2014 Bachman
`6/2014 Martinez
`6/2014 Ganong
`6/2014 Davis
`7/2014 Pasceri
`7/2014 Pasceri
`7/2014 Pasceri
`7/2014 Kalaboukis
`7/2014 Abhyanker
`8/2014 Davis
`8/2014 Luo
`8/2014 Stapleton
`8/2014 Akiya
`8/2014 Chao
`8/2014 Higgins
`9/2014 Raghuveer
`9/2014 Gao
`9/2014 Hubacek et al .
`9/2014 Kakarla
`9/2014 Farmer
`10/2014 Pasceri
`11/2014 Agarwal
`11/2014 Perczynski
`11/2014 Yu - Cheng
`11/2014 Hoffberg
`12/2014 Kalaboukis
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`( 56 )
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`8,923,889 B2
`8,930,848 B2
`8,949,212 B1
`8,954,425 B2
`8,966,121 B2
`8,972,177 B2
`8,998,422 B1
`9,009,177 B2
`9,014,511 B2
`9,015,617 B2
`9,015,633 B2
`9,020,247 B2
`9,031,953 B2
`9,032,320 B2
`9,047,847 B2
`9,055,037 B2
`9,063,226 B2
`9,076,259 B2
`9,092,409 B2
`9,098,545 B2
`9,104,729 B2
`9,104,915 B2
`9,110,903 B2
`9,135,751 B2
`9,151,618 B2
`9,152,849 B2
`9,158,794 B2
`9,160,802 B2
`9,172,666 B2
`D742,405 S
`9,202,200 B2
`9,218,328 B2
`9,224,172 B2
`9,235,766 B2
`9,239,848 B2
`9,245,041 B2
`9,261,376 B2
`D751,597 S
`9,311,396 B2
`9,323,855 B2
`9,336,240 B2
`9,372,931 B2
`9,390,104 B2
`9,405,981 B2
`9,418,485 B2
`9,424,595 B2
`9,436,374 B2
`9,460,116 B2
`9,462,054 B2
`9,465,513 B2
`9,471,200 B2
`9,471,834 B1
`9,483,500 B2
`9,495,583 B2
`9,501,577 B2
`9,507,778 B2
`9,419,682 B2
`9,535,563 B2
`9,536,146 B2
`9,552,376 B2
`9,557,162 B2
`9,576,253 B2
`9,582,546 B2
`9,593,957 B2
`9,600,484 B2
`9,606,668 B2
`9,626,685 B2
`9,639,740 B2
`9,646,025 B2
`9,654,570 B2
`9,674,650 B2
`9,677,886 B2
`9,679,456 B2
`9,680,929 B2
`9,683,858 B2
`
`12/2014 Svendsen
`1/2015 Lim
`2/2015 Dhandapani
`2/2015 Xiao
`2/2015 Josefsberg
`3/2015 Zheng
`4/2015 Snavely
`4/2015 Zheng
`4/2015 Brucher
`4/2015 Stapleton
`4/2015 Takamura
`4/2015 Adam
`5/2015 Rathnavelu
`5/2015 Crawford
`6/2015 Hochmuth
`6/2015 Evans
`6/2015 Zheng
`7/2015 Hourie
`7/2015 Charaniya
`8/2015 Abhyanker
`8/2015 Dong
`8/2015 Conwell
`8/2015 Martinez
`9/2015 Moore
`10/2015 Amer - Yahia
`10/2015 Ganong
`10/2015 Higgins
`10/2015 Svendsen
`10/2015 Murdock
`11/2015 Choi
`12/2015 Stibel
`12/2015 Stapleton
`12/2015 Churchill
`1/2016 Yi
`1/2016 Liu
`1/2016 Pilskalns
`2/2016 Zheng
`3/2016 Pasceri
`4/2016 Meadow
`4/2016 Hochmuth
`5/2016 Bhatt
`6/2016 Capt
`7/2016 Thomee
`8/2016 Li
`8/2016 Lindberg
`8/2016 Svendsen
`9/2016 Marr
`10/2016 Pilskalns
`10/2016 Poletto
`10/2016 sims
`10/2016 Dellinger
`10/2016 Filip
`11/2016 Brucher
`11/2016 Gilley
`11/2016 Zheng
`11/2016 Jaffe
`12/2016 Pujara
`1/2017 Hoffberg
`1/2017 Zheng
`1/2017 Desmond
`1/2017 Rodriguez
`2/2017 Zaltzman
`2/2017 Hartford
`3/2017 Zheng
`3/2017 Davis
`3/2017 Hotelling
`4/2017 Martinez
`5/2017 Ganong
`5/2017 Boyns
`5/2017 Bisdikian
`6/2017 Hartford
`6/2017 Didjusto
`6/2017 East
`6/2017 Tseng
`6/2017 Zheng
`
`US 11,163,823 B2
`Page 3
`
`9,691,073 B2
`9,703,873 B2
`9,706,345 B2
`9,710,961 B2
`9,715,366 B2
`9,721,188 B2
`9,754,226 B2
`9,772,745 B2
`9,787,799 B2
`9,805,123 B2
`9,811,879 B2
`9,836,183 B1
`9,857,941 B2
`9,858,348 B1
`9,870,572 B2
`9,881,179 B2
`9,882,994 B2
`9,916,075 B2
`9,942,121 B2
`10,001,917 B2
`10,019,850 B2
`10,037,327 B2
`10,068,178 B2
`10,073,584 B2
`10,074,093 B2
`10,083,533 B2
`10,110,541 B2
`10,120,947 B2
`10,139,989 B2
`10,140,743 B2
`10,145,704 B2
`10,147,215 B2
`10,187,543 B2
`10,223,701 B2
`10,230,803 B2
`10,235,444 B2
`10,242,051 B2
`10,282,752 B2
`10,288,433 B2
`10,289,643 B2
`10,303,975 B2
`10,311,611 B2
`10,318,110 B2
`10,324,973 B2
`10,331,863 B2
`10,360,352 B2
`10,430,452 B2
`10,445,346 B2
`10,489,980 B1
`10,540,668 B2
`10,606,449 B2
`10,628,463 B2
`10,643,263 B2
`10,650,039 B2
`10,650,475 B2
`2001/0030667 A1
`2001/0043727 Al
`2002/0019224 Al
`2002/0075322 A1
`2002/0097894 A1
`2002/0112237 A1
`2002/0191818 A1
`2003/0033296 Al
`2003/0039380 A1
`2003/0053663 Al
`2003/0063669 Al
`2003/0103652 A1
`2003/0122787 Al
`2003/0133599 Al
`2003/0179911 A1
`2003/0198368 A1
`2003/0210808 A1
`2004/0081338 Al
`2004/0109584 A1
`2004/0125991 A1
`2004/0135797 Al
`2004/0190758 Al
`2004/0205504 A1
`2004/0218894 A1
`
`6/2017 Tseng
`7/2017 Fakeih
`7/2017 Davis
`7/2017 Setlur
`7/2017 Bostick
`8/2017 Adam
`9/2017 Zheng
`9/2017 Hasenei
`10/2017 Grue
`10/2017 Nair
`11/2017 Miller
`12/2017 Love
`1/2018 Wagner
`1/2018 Higgins
`1/2018 Chapin
`1/2018 Patton
`1/2018 Bisdikian
`3/2018 Chen
`4/2018 Poletto
`6/2018 Kim
`7/2018 Lindberg
`7/2018 Thomee
`9/2018 van Zwol
`9/2018 Miura
`9/2018 Higgins
`9/2018 Bhatt
`10/2018 Li
`11/2018 Kritt
`11/2018 Shiroor
`11/2018 Hochmuth
`12/2018 Lanza
`12/2018 Lanza
`1/2019 Lahcanski
`3/2019 King
`3/2019 Higgins
`3/2019 Poletto
`3/2019 Shinn
`5/2019 Athsani
`5/2019 Zheng
`5/2019 Brucher
`5/2019 Adam
`6/2019 Stoop
`6/2019 Naaman
`6/2019 Circlaeys
`6/2019 Patton
`7/2019 Patton
`10/2019 Ross
`10/2019 Govindachetty
`11/2019 Canavor
`1/2020 Hoertz
`3/2020 Canavor
`4/2020 Purumala
`5/2020 Wormhoudt
`5/2020 Mariner
`5/2020 Berg
`10/2001 Kelts
`11/2001 Cooper
`2/2002 Meyers
`6/2002 Rosenzweig
`7/2002 Staas
`8/2002 Kelts
`12/2002 Matsuo
`2/2003 Rothmuller
`2/2003 Sukegawa
`3/2003 Chen
`4/2003 Lee
`6/2003 Lee
`7/2003 Zimmerman
`7/2003 Tian
`9/2003 Ho
`10/2003 Kee
`11/2003 Chen
`4/2004 Takenaka
`6/2004 Lestideau
`7/2004 Yokoi
`7/2004 Meier
`9/2004 Doi
`10/2004 Phillips
`11/2004 Harville
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`( 56 )
`
`References Cited
`U.S. PATENT DOCUMENTS
`11/2004 Toyama
`12/2004 Zhang
`12/2004 Taugher
`2/2005 Hwang
`3/2005 Filley
`5/2005 Sung
`5/2005 Li
`5/2005 Nagaoka
`6/2005 Yonaha
`6/2005 Nagahashi
`8/2005 Yang
`8/2005 Amano
`10/2005 Enomoto
`11/2005 Takikawa
`12/2005 Porter
`1/2006 Chiu
`1/2006 Saito
`22006 Kim
`3/2006 Adam
`4/2006 Kim
`6/2006 Li
`7/2006 Tanaka
`9/2006 Steinberg
`10/2006 Jung
`10/2006 Kitamura
`10/2006 McIntyre
`5/2007 Corcoran
`5/2007 Svendsen
`7/2007 Ording
`9/2007 Shinkai
`9/2007 Aoki
`10/2007 Song
`11/2007 Jaffe
`12/2007 Van Der Meulen
`2/2008 Kanno
`2/2008 Fisher
`3/2008 Matas
`3/2008 Okada
`4/2008 Schneiderman
`5/2008 Zhang
`6/2008 Naaman
`7/2008 Lamiraux
`7/2008 Blumenberg
`8/2008 Kimchi
`9/2008 Gao
`9/2008 Torii
`9/2008 Steinberg
`9/2008 Noda
`10/2008 Takahashi
`12/2008 Wen
`12/2008 Westerman
`12/2008 Steinberg
`1/2009 Farmer
`1/2009 Abhyanker
`2/2009 Naaman
`4/2009 Takamura
`4/2009 Hibino
`
`2004/0225635 A1
`2004/0264780 A1
`2004/0264810 A1
`2005/0031173 A1
`2005/0060299 A1
`2005/0094849 Al
`2005/0100195 A1
`2005/0105806 Al
`2005/0117802 A1
`2005/0141766 Al
`2005/0180627 A1
`2005/0183026 A1
`2005/0220347 A1
`2005/0251015 A1
`2005/0265603 A1
`2006/0001652 A1
`2006/0021027 Al
`2006/0029265 A1
`2006/0050933 A1
`2006/0078201 A1
`2006/0133672 A1
`2006/0165380 A1 *
`
`2006/0204034 A1
`2006/0222215 Al
`2006/0222217 A1
`2006/0224738 A1
`2007/0110305 Al
`2007/0118508 A1
`2007/0152984 Al
`2007/0206834 A1
`2007/0211925 A1
`2007/0239764 A1
`2007/0271297 Al
`2007/0282908 Al
`2008/0040034 A1
`2008/0051994 Al
`2008/0052945 Al
`2008/0056580 Al
`2008/0080743 Al
`2008/0122944 A1
`2008/0148175 Al
`2008/0168349 Al
`2008/0168402 A1
`2008/0201302 Al
`2008/0212849 A1
`2008/0212879 Al
`2008/0220750 A1
`2008/0232695 A1
`2008/0250398 Al
`2008/0298766 A1
`2008/0309632 A1
`2008/0317379 A1
`2009/0013041 A1
`2009/0019085 Al
`2009/0049408 Al
`2009/0106705 Al
`2009/0113350 A1 *
`
`2009/0132689 Al
`2009/0132941 A1
`2009/0135438 A1
`2009/0185784 Al
`2009/0216704 A1
`2009/0222302 A1
`2009/0254867 Al
`2009/0265631 Al
`2009/0278806 Al
`2009/0279794 A1
`2009/0288005 A1
`2009/0290812 A1
`2009/0307618 Al
`2009/0324018 A1
`2009/0325602 Al
`2010/0030578 A1
`
`5/2009 Zaltzman
`5/2009 Pilskalns
`5/2009 Chopra
`7/2009 Hiroike
`8/2009 Zheng
`9/2009 Higgins
`10/2009 Farouki
`10/2009 Sigurbjornsson
`11/2009 Duarte
`11/2009 Brucher
`11/2009 Stapleton
`11/2009 Naaman
`12/2009 Lawler
`12/2009 Tell
`12/2009 Higgins
`2/2010 Siddique
`
`US 11,163,823 B2
`Page 4
`
`2010/0041419 A1
`2010/0046842 A1
`2010/0053371 A1
`2010/0061631 Al
`2010/0064239 Al
`2010/0082427 Al
`2010/0083173 Al
`2010/0088641 Al
`2010/0107125 Al
`2010/0153348 Al
`2010/0162411 Al
`2010/0171763 Al
`2010/0172550 A1
`2010/0182341 A1
`2010/0185509 Al
`2010/0231537 A1
`2010/0241689 Al
`2010/0241944 A1
`2010/0245614 Al
`2010/0268717 A1
`2010/0268766 A1
`2010/0272363 Al
`2010/0280913 Al
`2010/0283743 Al
`2010/0287053 A1
`2010/0293035 Al
`2010/0293193 Al
`2010/0302179 A1
`2010/0312596 Al
`2011/0040779 Al
`2011/0063108 A1
`2011/0093458 A1
`2011/0109769 Al
`2011/0113064 A1
`2011/0122153 A1
`2011/0145258 Al
`2011/0188713 Al
`2011/0191014 A1
`2011/0191253 Al
`2011/0202267 A1
`2011/0208426 A1
`2011/0289031 A1
`2011/0301832 Al
`2011/0307836 A1
`2011/0314016 A1
`2012/0047457 A1
`2012/0096361 Al
`2012/0113475 Al
`2012/0114249 A1
`2012/0158755 Al
`2012/0162249 A1
`2012/0192110 A1
`2012/0204101 Al
`2012/02 10200 A1
`2012/0218150 A1
`2012/0220311 A1
`2012/0251011 A1
`2012/0266090 A1
`2012/0278171 Al
`2012/0278767 A1
`2012/0329441 A1
`2012/0331091 Al
`2013/0018881 A1
`2013/0036165 A1
`2013/0063613 A1
`2013/0073202 Al
`2013/0101157 A1
`2013/0138685 A1
`2013/0141612 A1
`2013/0151597 A1
`2013/0185676 A1
`2013/0202198 A1
`2013/0275536 A1
`2013/0339440 A1
`2014/0040774 Al
`2014/0059477 Al
`2014/0059492 Al
`2014/0071272 A1
`2014/0088861 Al
`
`2/2010 Svendsen
`2/2010 Conwell
`3/2010 Karimoto
`3/2010 Omori
`3/2010 Crawford
`4/2010 Burgener
`4/2010 Germann
`4/2010 Choi
`4/2010 Ockene
`6/2010 Perczynski
`6/2010 Chang
`7/2010 Bhatt
`7/2010 Gilley
`7/2010 Lee
`7/2010 Higgins
`9/2010 Pisula
`9/2010 Davis
`9/2010 Athsani
`9/2010 Matsunaga
`10/2010 Pilskalns
`10/2010 Bouget
`10/2010 Steinberg
`11/2010 O'Sullivan
`11/2010 Coddington
`11/2010 Ganong
`11/2010 Athsani
`11/2010 Harrison
`12/2010 Ahn
`12/2010 Saffari
`2/2011 Svendsen
`3/2011 Aonuma
`4/2011 Zheng
`5/2011 Bhatt
`5/2011 Govindachetty
`5/2011 Okamura
`6/2011 Kankainen
`8/2011 Chin
`8/2011 Feng
`8/2011 Pilskalns
`8/2011 Svendsen
`8/2011 Zheng
`11/2011 Zheng
`12/2011 Zheng
`12/2011 Cho
`12/2011 Svendsen
`2/2012 Park
`4/2012 Osten
`5/2012 Sugiyama
`5/2012 Conwell
`6/2012 Gammill
`6/2012 Tsuda
`7/2012 Wu
`8/2012 Yoshida
`8/2012 Berger
`8/2012 Oyabu
`8/2012 Rodriguez
`10/2012 Gao
`10/2012 Nealer
`11/2012 Tang
`11/2012 Stibel
`12/2012 Tseng
`12/2012 Tseng
`1/2013 Bhatt
`2/2013 Tseng
`3/2013 Conwell
`3/2013 Zheng
`4/2013 Li
`5/2013 Brucher
`6/2013 Bhatt
`6/2013 Akiya
`7/2013 Cao
`8/2013 Adam
`10/2013 Murdock
`12/2013 Balassanian
`2/2014 Chartyoniuk
`2/2014 Wong
`2/2014 Hashida
`3/2014 Rodriguez
`3/2014 Nash
`
`HO4N 5/772
`386/227
`
`G06F 16/41
`715/853
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`US 11,163,823 B2
`Page 5
`
`( 56 )
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`2014/0089811 A1
`2014/0101531 Al
`2014/0101601 A1
`2014/0112553 A1
`2014/0143247 A1
`2014/0149036 A1
`2014/0161326 A1
`2014/0181089 Al
`2014/0188880 A1
`2014/0193087 A1
`2014/0207444 Al
`2014/0258850 A1
`2014/0354628 A1
`2015/0039630 A1
`2015/0066919 Al
`2015/0070165 Al
`2015/0070397 Al
`2015/0116540 A1
`2015/0117713 Al
`2015/0131872 A1
`2015/0156247 A1
`2015/0186389 Al
`2015/02 13057 A1
`2015/0213329 A1
`2015/0244794 A1
`2015/0244833 A1
`2015/0358224 A1
`2016/0048279 Al
`2016/0092741 Al
`2016/0162512 Al
`2016/0247307 A1
`2016/0253358 Al
`2016/0314187 Al
`2016/0321269 Al
`2016/0328444 A1
`2016/0344888 A1
`2016/0357822 Al
`2017/0024415 A1
`2017/0046565 A1
`2017/0069123 Al
`2017/0103081 A1
`2017/0192645 A1
`2017/0357672 A1
`2018/0181281 Al
`2018/0364872 A1
`
`3/2014 Meadow
`4/2014 Capt
`4/2014 Tang
`4/2014 Yamaguchi
`5/2014 Rathnavelu
`5/2014 Amer - Yahia
`6/2014 Ganong
`6/2014 Desmond
`7/2014 Abhyanker
`7/2014 Conwell
`7/2014 Heiman
`9/2014 Carey
`12/2014 Lindberg
`2/2015 Thomee
`3/2015 Park
`3/2015 East
`3/2015 Miller
`4/2015 Gilman
`4/2015 Zheng
`5/2015 Ganong
`6/2015 Hensel
`7/2015 Zheng
`7/2015 Brucher
`7/2015 Adam
`8/2015 Poletto
`8/2015 Gm
`12/2015 Poletto
`2/2016 Han
`3/2016 Li
`6/2016 Battistini
`8/2016 Stoop
`9/2016 Bhatt
`10/2016 Poletto
`11/2016 Thomee
`11/2016 Shinn
`11/2016 Lahcanski
`12/2016 Woods
`1/2017 Brucher
`2/2017 Gilley
`3/2017 Hochmuth
`4/2017 Jones
`7/2017 Murray
`12/2017 Circlaeys
`6/2018 Suki
`12/2018 Miura
`
`FOREIGN PATENT DOCUMENTS
`2410414 B1
`10/2019
`WO 2011/070225 A1
`6/2011
`2/2013
`WO 2013/019376 A1
`7/2013
`WO 2013/099704 Al
`
`EP
`WO
`WO
`WO
`
`OTHER PUBLICATIONS
`Pogue , D. & Biersdorfer , J. D. , “ iPhoto '09 : The Missing Manual , "
`O'Reilly ledia , Inc. ( 2009 ) .
`Amundsen , J .;
`“ Using the Geographical Location of Photos in
`Mobile Phones ” ; Master of Science in Computer Science submis
`sion ; Norwegian University of Science and Technology ; Jul . 2008
`( 112 pages ) .
`Gentile ; L ; “ Using Flickr Geotags to Find Similar Tourism Desti
`nations ” ; master thesis , 2011 ; Politecnico di Milano ; Dept. of
`Computer Engineering ( 96 pages ) .
`Hollenstein , L. , “ Capturing Vernacular Geography from Georeferenced
`Tags ” ; Master Thesis ; Institute of Geography , University of Zurich ;
`Nov. 2008 ( 139 pages ) .
`Nutanong , S. et al .;
`" An Efficient Layout Method for a Large
`Collection of Geographic Data Entries ” ; Center for Automation
`Research , Institute for Advanced Computer Studies , Dept. of Com
`puter Science , University of Maryland , pp . 717-720 ( 4 pages )
`Slingsby , A. et al .;
`“ Interactive tag maps and tag clouds for the
`multiscale exploration of large spatio - temporal datasets ” ; Informa
`
`tion Visualization , pp . 497-504 ; 2007 , IV '07 . 11 ° h International
`Conference . ISSN : 1550-6037 ( 9 pages )
`Hoffman , A. , “ Create Great iPhone Photos : Apps , Tips , Tricks , and
`Effects ” , copyright 2011 , ISBN - 10 : 1-59327-285-5 , ISBN - 13 : 978
`1-59327-285-2 ( 216 pages ) .
`Chen , Y - F . et al .;
`“ GeoTracker : Geospatial and Temporal RSS
`Navigation ” ; AT &
`T Labs - Research , Florham Park , NJ , WWW
`2007 / Track : Browsers and User Interfaces , Session : Smarter
`Browsing , pp . 41-50 ( 10 pages ) .
`Goodman , E .; “ Destination Services : Tourist media and networked
`places ” ; School of Information , UC Berkeley ; March 2 , 2007 ( 11
`pages ) .
`Rattenbury , T. et al .; " Towards Automatic Extraction of Event and
`Place Semantics from Flickr Tags ” ; SIGIR '07 ; Jul . 23-27 , 2007 ;
`Amsterdam , The Netherlands ; ACM 978-1-59593-597-7 / 07 / 0007
`( 8 pages ) .
`Kadar , B. et al . , “ Where Do Tourists Go ? Visualizing and Analysing
`the Spatial Distribution of Geotagged Photography ” ; Cartographica
`48 : 2 , pp . 78-88 ; 2013 ; University of Toronto Press ; doi : 10.3138 /
`carto.48.2.1839 ( 11 pages ) .
`Kennedy , L. et al .; “ How Flickr Helps Us Make Sense of the World :
`Context and Content in Community - Contributed Media Collec
`tions ” ; MM '07 , September 23-28 , 2007 , Augsburg , Bavaria , Ger
`many , Copyright 2007 , ACM978-1-59593-701-8 / 07 / 0009 ( 10 pages ) .
`Richard , G. et al .; “ Geotagging Photographs for Distribution on the
`Web " ; Mineral Physics Institute , Earth and Space Sciences Build
`ing , Stony Brook University , Stony Brook . NY , date unknown ( 9
`pages ) .
`Kustanowitz et al . ,
`“ Motivating Annotation for Personal Digital
`Photo Libraries : Lowering Barriers while Raising Incentives , ” Tech .
`Report HCIL - 2004-18 , U. Maryland , 2005 ( 10 pages ) .
`Miller et al . , “ Give and take : a study of consumer photo - sharing
`culture and practice , ” CHI '07 Proceedings of the SIGCHI Confer
`ence on Human Factors in Computing Systems , pp . 347-356 , 2007
`( 10 pages ) .
`Ames et al . , “ Why we tag : motivations for annotation in mobile and
`online media , ” CHI '07 Proceedings of the SIGCHI Conference on
`Human Factors in Computing Systems , pp . 971-980 , ACM , 2007
`( 10 pages ) .
`Yee et al . , “ Faceted Metadata for Image Search and Browsing ” , CHI
`2003 , pp . 401-408 , 2003 , ACM .
`Ferre , “ CAMELIS : Organizing and Browsing a Personal Photo
`Collection with a Logical Information System ” , Int . Conf . Concept
`Lattices and Their Applications , pp . 112-123 , 2007 , HAL .
`Tomasson et al . ,
`“ PhotoCube : Effective and Efficient Multi
`Dimensional Browsing of Personal Photo Collections ” , ICMR ’11 ,
`2011 , ACM .
`Baitolini et al . , “ Integrating Semantic and Visual Facets for Brows
`ing Digital Photo Collections ” , SBED , pp . 65-72 , 2009 .
`Trattner et al . , “ Evaluating Tag - Based Information Access in Image
`Collections ” , Proceedings of the 23rd ACM Conference on Hyper
`text and Social Media , pp . 113-122 , 2012 ACM .
`Kang et al . ,
`“ Capture , Annotate , Browse , Find , Share : Novel
`Interfaces for Personal Photo Management ” , International Journal
`of Human - Computer Interaction , 23 ( 3 ) , pp . 315-337 , 2007 , Law
`rence Enbaum Associates , Inc.
`Jaffe et al . ,
`“ Generating Summaries and Visualization for Large
`Collections of GeoReferenced Photographs ” , MIR'06 , pp . 89-98 ,
`2006 ACM .
`Tornial et al . ,
`“ Sharing , Discovering and Browsing Geotagged
`Pictures on the Web ” , 2007 , Hewlett - Packard Development Com
`pany , L.P. , pp . 1-18 .
`Snavely et al . , “ Photo Tourism : Exploring Photo Collection in 3D ” ,
`SIGGRAPH '06 ACM Transactions on Graphics , vol . 25 , Issue 3 ,
`pp . 835-846 , 2006 ACM .
`Kisilevich et al . ,
`“ Event - based analysis of People's Activities and
`Behavior Using Flickr and Panoramio Geotagged Photo Collec
`tions ” , 14'h International Conference Information Visualization , pp .
`289-296 , 2010 IEEE .
`Ahern et al . ,
`“ World Explorer : Visualizing Aggregate Data From
`Unstructured Text in Geo - Referenced Collections ” , JCDL'07 , pp .
`1-10 , 2007 , ACM .
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`US 11,163,823 B2
`Page 6
`
`( 56 )
`
`References Cited
`OTHER PUBLICATIONS
`“ Deep photo : model - based photograph enhancement
`Kopf et al . ,
`and Viewing ” , ACM Transactions on Graphics , vol . 27 , No. 5 ,
`Article 116 , Dec. 2008 , ACM ( 10 pages ) .
`Petition for Inter Partes Review of U.S. Pat . No. 10,621,228 by
`Unified Patents , filed Sep. 3 , 2021 ( 104 pages ) .
`Declaration of Dr. Benjamin B. Bederson supporting the IPR
`Petition of Unified Patents , LLC Sep. 2 , 2021 ( 251 pages ) .
`Declaration of Kevin Jakel supporting the IRP Petition of Unified
`Patents , LLC Sep. 2 , 2021 ( 7 pages ) .
`Bederson , B ,
`“ Photo Mesa : A Zoomable Image Browser Using
`Quantum Treemaps and Bubblemaps , ” Proceedings of the 14th
`Annual ACM Aymposium on User Interface Software and Tech
`nology , pp . 71-80 , Nov. 11-14 , 2001 ( 10 pages ) .
`Suh , B. et al . , “ Automatic Thumbnail Cropping and its Effective
`ness , ” Proceedings of the 16th Annual ACM Symposium on User
`Interface and Technology , pp . 95-104 , 2003 ( 10 pages ) .
`
`>
`
`Shneiderman , B. et al . , “ Find That Photo ! Interface Strategies to
`Annotate , Browse , and Share , ” Communications of the ACM ,
`49 ( 4 ) : 69-71 , Apr. 2006 ( 3 pages ) .
`National Aeronautics and Space Administration , “ Space Station
`Assembly - Baikonur Cosmosphere , ” Retrieved from the Internet
`on Sep. 1 , 2021 , https://web.archive.org/web/20060928234623/http://
`www.nasa.gov/mission_pages/station structure / elements / baikonur .
`html / ; alleged by Unified Patents to have been published on Sep. 28 ,
`2006 ( 3 pages ) .
`Shneiderman , B. & Plasiant , C .. “ Designing the User Interface :
`Strategies for Effective Human - Computer Interaction ( 5th ed . ) . "
`Addison - Wesley Publishing Company , USA , pp . vii - xviii , 456-457 ,
`alleged by Unifed Patent to have been published in 2009 ( 28 pages ) .
`Perez , L. ,
`“ iPhoto 09 Basics , " Florida Center for Instructional
`Technology , Retrieved from the Internet on Apr. 10 , 2021 , https : //
`etc.usf.edu/te_mac/movies/pdf/iphoto09.pdf/ ; alleged by Unified Pat
`ents ot have been published on Aug. 9 , 2009 ( 15 pages ) .
`* cited by examiner
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 1 of 50
`
`US 11,163,823 B2
`
`Select None
`Select All
`Delete Selected
`
`FIG . 1
`
`September 2 , 2010 change
`Yellowstone , MT , USA change
`Christopher Desmond , Nancy Desmond
`Fly Fishing in Yellowstone
`Add
`Add
`Add
`
`1 File ( s ) Selected
`File Attributes :
`Date :
`Location :
`People :
`Event :
`Comments :
`Keywords :
`Album :
`Star Ranking :
`Add
`Sharing Rights :
`Recipe :
`Add
`Document Type : Add
`File Name :
`IMG 123 change
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 2 of 50
`
`US 11,163,823 B2
`
`FIG . 2
`
`Comments :
`Suzanne and Anthony's Wedding Party where the cousins posed
`for a photo in the grass . Note , Jack with the lollipop and the
`photographer with his shoe in the photo
`People :
`Jack Wong
`CJ Wong
`Mary Firestone
`Zoe Peika
`Nick Persons
`Event : Suzanne & Anthony's Wedding Reception 2010
`
`Camera Details : more
`
`2
`
`Location :
`Historical Society
`Lisle , IL 60532
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 3 of 50
`
`US 11,163,823 B2
`
`FIG . 3
`
`Thumbnail Table
`
`Mary Bader and Clint Firestone
`Wedding October 11 , 1945
`
`Capt . Clint Firestone
`1952
`
`Germany Trip 1955
`
`Father's Day 1966
`
`Christopher 1st
`Year 1972
`
`College Break 1992
`
`LA Trip 1993
`
`EIU College
`Graduation 1994
`
`Cairo , Egypt 1996
`
`Europe
`Backpacking
`1996
`
`Nancy James and
`Christopher Desmond
`Wedding October 17 , 1998
`
`Amazon Fishing Trip
`November 2009
`
`Cousins Party 2010
`
`Fishing in Yellowstone
`Montana September 2010
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 4 of 50
`
`US 11,163,823 B2
`
`FIG . 4
`
`Captions On | Off
`Christopher Desmond and Nancy Desmond Wedding - October 17 , 1998
`
`-
`
`Event
`
`M
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 5 of 50
`
`US 11,163,823 B2
`
`FIG . 5
`
`30
`
`AY
`
`xde
`
`2
`
`Svo
`
`{
`
`;
`Srts
`
`X
`
`3 : 353
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 6 of 50
`
`US 11,163,823 B2
`
`FIG . 6
`
`Thumbnail | Table
`
`Clint Firestone
`
`Tim Dog
`
`Steve Fire
`
`Suzanne Lucky
`
`Steve Fwah
`
`Ben Sherman
`
`Joe Schmo
`
`Mary Virginia
`
`Clint Dewitt
`
`Charlotte Killian
`
`Chris Jeep
`
`Peter LongHorn
`
`Tim Dog
`
`Steve Fire
`
`Suzanne Lucky
`
`Steve Fwah
`
`Ben Sherman
`
`Joe Schmo
`
`Mary Virginia
`
`Clint Dewitt
`
`Charlotte Killian
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 7 of 50
`
`US 11,163,823 B2
`
`FIG . 7
`
`Clinton Dewitt Firestone IV
`
`July 12 , 1896
`Birth :
`April 29 , 1971
`Death :
`Clinton Dewitt Firestone III and Viola Miller
`Parents :
`Comments : He was a WWII U.S. Air force pilot and POW in WWII and veteran
`honorably discharged in December of 1947. He worked for 44 years for the
`Firestone Tire and Rubber Company in retail , wholesale and original equipment
`sales , marketing and management . He was born in Akron , OH and is buried in
`Columbiana , OH .
`
`Edit bio
`
`Locations
`
`Timeline
`
`Family Tree
`
`Recipes
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 8 of 50
`
`US 11,163,823 B2
`
`FIG . 8
`
`Descendants of Charlotte Killian Halter
`
`Tree | Table
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 9 of 50
`
`US 11,163,823 B2
`
`FIG . 9
`
`Year
`Month
`Day
`
`on
`
`2 5
`
`Fishing in Yellowstone
`Montana September 2010
`
`Cousins Party 2010
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 10 of 50
`
`US 11,163,823 B2
`
`FIG . 10
`
`1 Zucchini ( chopped )
`Thai Basil ( 8 leaves )
`Lemon Grass ( in jar ) 1 teaspoon
`Chinese Ginger Root ( in jar ) 1 teaspoon
`
`Rice
`Thai Rice ( something that only takes 2 cups of water )
`Dice chicken in bowl and add two tablespoons of fish
`sauce . Let marinate for 20 minutes .
`Take thick part of coconut milk out into pan ( about 4
`tablespoons ) , Curry paste , 1 spoon of lemon grass , 1
`1
`spoon of ginger and garlic . Heat over high with boil and
`THEN stir for 1 minute . Add meat ( uncooked ) and fry
`until cooked over high heat
`Add milk , brown sugar and salt . Bring back to slight boil
`and constantly stir . Add veggies and soy sauce . Cook for
`about 10-14 minutes COVERED until veggies are cooked .
`Serve with a smile .
`
`Desmond's Yellow Thai Chicken Curry
`
`Curry Mix
`Coconut milk ( 400 ml ) – DO NOT SHAKE IT UP
`800 gram of chicken ( 4 chicken breast )
`Fish sauce ( Nam Pla ) Thai Bamboo Garden - Bottle
`Garlic ( 2 cloves )
`Broccoli ( 2 cups chopped )
`2 Peppers ( chopped )
`2 Carrots ( chopped )
`
`.
`
`•
`
`Chef : Barry Desmond
`
`Video on How to Make It
`
`Original Handwritten Recipe
`
`H
`
`::
`
`...
`
`MemoryWeb Ex. 2006
`Unified Patents v. MemoryWeb – IPR2021-01413
`
`

`

`U.S. Patent
`
`Nov. 2. 2021
`
`Sheet 11 of 50
`
`US 11,163,823 B2
`
`FIG . 11
`
`Thumbnail Table
`
`Album / Event
`Jack Monk's Arrival
`Mike Testy's First Birthday
`Cubs Beat Cards 1998
`Jack Monk's Arrival
`Mike Testy's 2nd Birthday
`Cubs Beat Cards 1998
`Jack Monk's Arrival
`Mike Testy's 3rd Birthday
`Cubs Beat Cards 1998
`Jack Wrigley Monk's Arrival
`Mike Testy's 4th Birthday
`Cubs Beat Cards 1998
`Nancy Learns How to Ride a Bike
`
`Date
`26 - Dec - 2003
`13 - Sep - 1983
`5 - Au