USOO9323442B2
`
`(12) United States Patent
`Ledet et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,323.442 B2
`Apr. 26, 2016
`
`(54) MANAGING ITEMS IN A USER INTERFACE
`(75) Inventors: Raleigh Joseph Ledet, Sunnyvale, CA
`(US); Jeffrey Traer Bernstein, San
`Francisco, CA (US); B. Michael Victor,
`Menlo Park, CA (US); Avi E.
`Cieplinksi, San Francisco, CA (US);
`Kristin Forster, Los Altos Hills, CA
`(US); Craig Federighi, Los Altos Hills,
`CA (US)
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 834 days.
`(21) Appl. No.: 12/895,770
`(22) Filed:
`Sep. 30, 2010
`
`(65)
`
`Prior Publication Data
`US 2012/OO84689 A1
`Apr. 5, 2012
`
`(2013.01)
`(2013.01)
`
`(51) Int. Cl.
`G06F 3/048
`G06F 3/0486
`(52) U.S. Cl.
`CPC .................................... G06F 3/0486 (2013.01)
`(58) Field of Classification Search
`CPC ............................... G06F 3/0486; G06F 3/048
`USPC .......................................................... 71.5/769
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`1/1996 Yasutake
`5,483.261 A
`1/1996 Mead et al.
`5.488.204 A
`4, 1998 Kung et al.
`5,742,286 A
`5,825,352 A 10, 1998 Bisset et al.
`5,835,079 A 11, 1998 Shieh
`
`5,880,411 A
`6,188,391 B1
`6,310,610 B1
`6,323,846 B1
`
`3/1999 Gillespie et al.
`2/2001 Seely et al.
`10/2001 Beaton et al.
`1 1/2001 Westerman et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`
`6, 2000
`2000-163031 A
`2002-342033. A 11, 2002
`(Continued)
`OTHER PUBLICATIONS
`
`Nayuko Watanabe et al., “Bubble Clusters: An Interface for Manipu
`lating Spatial Aggregation of Graphical Objects'. UIST'07 Proceed
`ings of the 20th Annual ACM Symposium on User InterfaceSoftware
`and Technology, Oct. 7-10, 2007, Newport, Rhode Island, pp. 173
`182, XP-002664449.
`
`(Continued)
`
`Primary Examiner — Angie Badawi
`(74) Attorney, Agent, or Firm — Morrison & Foerster LLP
`(57)
`ABSTRACT
`User interface changes related to moving items in a user
`interface are disclosed. An operation (e.g., a drag operation)
`can be initiated on selected items by moving a cursor or
`pointing device in the user interface, and an animation can be
`presented illustrating representations of the selected items
`moving from their respective original locations toward a cur
`rent location of the cursor or pointing device and forming a
`cluster in proximity to the current location of the cursor or
`pointing device. As the cluster of items is moved over a
`container object in the user interface, the representations of
`the items can adopt the appearance style defined by that
`container object. The representations of the items can also be
`shown to depart from the cluster and move toward anticipated
`locations of the items in the container object as a preview of
`a drop operation into the container object.
`
`68 Claims, 25 Drawing Sheets
`
`
`
`File File Edit Wey WoW He
`
`rii:55 Aki
`
`122a
`Adoc -
`his 122
`
`APPL-1015
`APPLE INC. / Page 1 of 47
`
`

`

`US 9,323.442 B2
`Page 2
`
`(56)
`
`References Cited
`
`FOREIGN PATENT DOCUMENTS
`
`U.S. PATENT DOCUMENTS
`
`8, 2003
`2003233446 A
`JP
`WO2007 142256 A1 12/2007
`WO
`WO WO-2012/044779 A1
`4/2012
`
`OTHER PUBLICATIONS
`
`PCT International Search Report and Written Opinion for Applica
`tion No. PCT/US2011/053901 dated Dec. 9, 2011, 8 pages.
`Dutch Search Report and Written Opinion for Application No.
`
`2, 2004 Zimmerman et al.
`6,690,387 B2
`3, 2006 Morohoshi
`7,015,894 B2
`7,072,450 B1* 7/2006 Pinard ........................ 379.88.11
`7,184,064 B2
`2/2007 Zimmerman et al.
`7,337.409 B2
`2/2008 Doblmayr et al.
`7,370.281 B2
`5 2008 Weber.
`7,650,575 B2
`1, 2010 Cummins et al.
`7,663,607 B2
`2/2010 Hotelling et al.
`Lee, S.K. et al. (Apr. 1985). "A Multi-Touch Three Dimensional
`8,479.2 B2
`7/2013 Hotelling et al.
`Touch-Sensitive Tablet.” Proceedings of CHI: ACM Conference On
`2005/0283742 A1* 12/2005 Gusmorino et al. .......... 71.5/839
`Human Factors in Computing Systems, pp. 21-25.
`2006, 0026521 A1
`2/2006 Hotelling et al.
`Rubine, D.H. (Dec. 1991). “The Automatic Recognition of Ges
`2006/01 12354 A1* 5, 2006 Park et al. ..................... 715.835
`tures. CMU-CS-91-202, Submitted in Partial Fulfillment of the
`2006/0129586 A1
`6/2006 Arrouye et al.
`Requirements for the Degree of Doctor of Philosophy in Computer
`2006.0161868 A1* 7, 2006 Van Dok et al. .............. 715.835
`Science at Carnegie Mellon Universitv. 285
`2006/O197753 A1
`9/2006 Hotelling
`gie Mellon University, 285 pages.
`2006/0256084 A1* 11/2006 Hey et al. .
`345,157
`Rubine, D.H. (May 1992). “Combining Gestures and DirectManipu
`2007,0192749 A1* 8, 2007 Baudisch
`715,863
`lation.” CHI '92, pp. 659-660.
`2008/0298766 A1* 12, 2008 Wen et al.
`... 386/46
`Westerman, W. (Spring 1999). “Hand Tracking, Finger Identifica
`2009/0164439 A1* 6/2009 Nevins .............................. 707/3
`tion, and Chordic Manipulation on a Multi-Touch Surface.” A Dis
`2009/0228820 A1* 9, 2009 Kim et al.
`715,769
`sertation Submitted to the Faculty of the University of Delaware in
`2010/0122195 A1* 5/2010 Hwang ................. G06F 3.0488
`2010/02751 22 A1* 10, 2010 Buxton et al. ................ 22: Partial Fulfillment of the Requirements for the Degree of Doctor of
`2011/0161852 A1* 6, 2011 Vainio et al. .................. 715,769
`Philosophy in Electrical Engineering, 364 pages.
`2011/0296336 A1* 12/2011 Law et al. ..................... 71.5/777
`2012/0066144 A1
`3/2012 Berkvens et al. ............. 705.345
`
`7 752 566 B1 ck
`
`T/2010 Nelson .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. 715,769
`
`2007.490 dated Dec. 14, 2011, 8 pageS.
`
`
`
`* cited by examiner
`
`APPL-1015
`APPLE INC. / Page 2 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 1 of 25
`
`US 9,323.442 B2
`
`
`
`LOET
`
`APPL-1015
`APPLE INC. / Page 3 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 2 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 4 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 3 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 5 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 4 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 6 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 5 Of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 7 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 6 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 8 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 7 Of 25
`
`US 9,323.442 B2
`
`{}{},
`
`
`
`APPL-1015
`APPLE INC. / Page 9 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 8 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 10 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 9 Of 25
`
`US 9,323.442 B2
`
`
`
`/
`
`----- r ----
`
`APPL-1015
`APPLE INC. / Page 11 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 10 of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 12 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 11 of 25
`
`US 9,323.442 B2
`
`{}{}}.
`
`
`
`APPL-1015
`APPLE INC. / Page 13 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 12 of 25
`
`US 9,323.442 B2
`
`c
`
`relan
`
`
`
`APPL-1015
`APPLE INC. / Page 14 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 13 Of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 15 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 14 of 25
`
`US 9,323.442 B2
`
`{}{}}
`
`
`
`§ 3,
`
`APPL-1015
`APPLE INC. / Page 16 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 15 Of 25
`
`US 9,323.442 B2
`
`
`
`©
`
`APPL-1015
`APPLE INC. / Page 17 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 16 of 25
`
`US 9,323.442 B2
`
`{}{}},
`
`
`
`APPL-1015
`APPLE INC. / Page 18 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 17 Of 25
`
`US 9,323.442 B2
`
`{}{}},
`
`
`
`No. ?
`
`*
`
`APPL-1015
`APPLE INC. / Page 19 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 18 of 25
`
`US 9,323.442 B2
`
`c
`star
`
`
`
`APPL-1015
`APPLE INC. / Page 20 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 19 Of 25
`
`US 9,323.442 B2
`
`
`
`APPL-1015
`APPLE INC. / Page 21 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 20 of 25
`
`US 9,323.442 B2
`
`2002
`
`Receiving input
`
`in response to the input, howing espective representations of the one or
`more sciected it cris from thc represcitations respective original locations in
`the riser interface toward a current location of a pointer in the user interface
`and forming a custer in proximity to the current location of the pointer
`
`Causing a count indicator to be presented in proximity to the citaster, the
`count indicator showing a count of item representations in the cluster
`26
`
`
`
`F. A
`
`|Detecting continued novenient of the pointer citagging the one or nore
`selected items in the user interface
`22
`
`Animatig the caster of the item representations to follow the continuesi
`moveneit if pointer
`
`F.G. E.
`
`APPL-1015
`APPLE INC. / Page 22 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 21 of 25
`
`US 9,323.442 B2
`
`300 &
`
`Detecting the current scation of the pointer within a drop Zoe Qf a second
`container in the laser interface
`302
`
`White the current iocation of the pointer remains within the irop zone of the
`Second container, causing the representations of the Selected iters to adopt
`respective second appearances
`30:
`
`Detecting aii input requesting a preview of the icias represcitations being
`dropped in the second containe
`.
`
`a.
`
`in response to detecting the inpui, aniinating the representations of the
`seected items to depart from the cister and move toward their respective
`anticipated locations in the second container
`4.
`
`
`
`FG. A
`
`etceting an input releasing the One or more sciected iteras into the drop
`Zone of the second container
`412
`
`it response to the 'eleasing input, causing the one or lore selected items to
`be inserted at the respective anticipated locations of the selected items in the
`scoid containci
`44.
`
`FG. B.
`
`APPL-1015
`APPLE INC. / Page 23 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 22 of 25
`
`US 9,323.442 B2
`
`Detecting a movement of the pointer exiting the second container
`
`O2
`
`in response to the ovement, animating respective representatiots of the
`selected items to move from the items respective anticipated locations in the
`second contaiiar toward the current iocatio of the pointer and feiiii the
`case in proximity is the currett jocation of the pointer
`504
`
`60?
`
`
`
`F.G. 5,
`
`Monitoring a motion characticristic of the pointer for dragging the
`represcitations of the oiac or Roic sciecici iicits in the user interface
`8.
`
`Causing the rcprescitations of thc siccici items to adopt the icspective
`sec{iai appearances upon deterination that the notion characteristic of the
`pointer over the drop zone of the second container satisfies a defined
`criterion
`64
`
`Receiving an input draggig respective representatio is {}i sciected iteins
`from a first container into a drop zone of a second container, the respective
`representations of each item having a respective first appearance associated
`with the first contair
`702
`
`While the respective representatios of the selected items are at least
`partially within the drop zone of the second container, causing the
`rcprescitations of the sciccici items to adopt rcs.cctive sccord appearances
`F04.
`
`APPL-1015
`APPLE INC. / Page 24 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 23 Of 25
`
`US 9,323.442 B2
`
`or, 8.2
`
`
`
`Applications
`
`fag and Drop ij victification Egine
`
`Servicesiliaries
`
`{OfS
`
`adwaj'e
`W3
`
`F. 8A
`
`8.
`
`88
`
`3.
`
`APPL-1015
`APPLE INC. / Page 25 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 24 of 25
`
`US 9,323.442 B2
`
`8: C.
`
`AP-Cailing Component(s)
`826
`
`A Cals,
`Parameters
`
`
`
`Retir
`Waiues,
`aareters
`
`
`
`
`
`Application Programming interface
`824
`APi-implementing Component(s)
`822
`
`
`
`F.G. 8
`
`t",
`
`
`
`
`
`
`
`Application A840
`
`Applicatio; B 844
`
`
`
`
`
`Sewice AA
`
`Service A 832
`
`
`
`
`
`Service
`Service B
`A. A 838 A 3842
`Service 3 834
`
`
`
`
`
`OS AP B 848
`OS APA 848
`Operating System (OS) 836
`
`
`
`FG. 8C
`
`APPL-1015
`APPLE INC. / Page 26 of 47
`
`

`

`U.S. Patent
`
`Apr. 26, 2016
`
`Sheet 25 Of 25
`
`US 9,323.442 B2
`
`3.
`
`i.
`ACCeleforeier
`
`(C3of f(CeSSof
`
`Magaetoilet
`agnetoileef
`
`iotic SensOf
`
`light Se?sor
`
`Proximity Sensor
`
`917
`
`35
`
`98
`
`3.
`
`9.
`
`9A.
`
`92
`
`Operating Systern risiructions
`Colini Picatio; instructiis
`is fiftisfs
`G is fuctifs
`Seso? Processing structions
`w; tra
`Pione strictions
`Electronic lessaging instructions
`Web Browsing instructions
`Media Processing strictions
`Fairs
`r
`GPS Navigation instructions
`Caera iristictors
`Othef iristictions
`
`952
`95.
`958
`958
`
`960
`r
`982
`964
`988
`
`970
`fy
`972
`
`^,
`
`
`
`Meritory interface
`
`3O2
`
`''
`
`350
`
`ax
`36
`
`94.
`
`Caier8
`Subsystem
`
`
`
`Wireless
`- CO: AT:Lilication
`Subsysterns)
`
`Peripherals
`interface
`
`922
`
`9.28
`
`93.
`
`i... Audio Subsystern
`
`
`
`|O Subsystem
`
`OC-SCfeel Confroiief
`
`Other input Controller(s)
`
`
`
`948
`
`otic Screer
`
`Other input Control
`evices
`
`948
`
`FG, 9
`
`APPL-1015
`APPLE INC. / Page 27 of 47
`
`

`

`1.
`MANAGING TEMIS IN AUSER INTERFACE
`
`US 9,323,442 B2
`
`TECHNICAL FIELD
`
`This disclosure relates generally to managing information
`items on a computing device.
`
`5
`
`BACKGROUND
`
`Modern computing devices often include file systems that
`store information items in various directories or Subdirecto
`ries (e.g., folders) in a file system hierarchy. In a graphical
`user interface (GUI), representations of the information items
`(e.g., icons) can be shown in respective container objects
`(e.g., the desktop, folder windows) associated with a parent
`item of the information items to indicate the locations of the
`information item in the file system hierarchy.
`Each container object can be associated with an appear
`ance style (e.g., a large icon view, a small icon View, a list
`view, a column view, etc.). Representations of the informa
`tion items contained in the container object can take on
`respective appearances according to the appearance style
`associated with the container object. When an item is moved
`from one container to another container having a different
`appearance style, or when the appearance style of a current
`container is modified, the appearance of the items represen
`tation can be changed according to the current appearance
`style of the items current container.
`Many operating systems enable a drag and drop operation
`to be performed on items that are currently selected in a GUI.
`In the drag and drop operation, representations of the selected
`items can be moved (or "dragged') in the user interface from
`one container object to another container object following the
`movement of a pointer (e.g., a mouse cursor or pointing
`device on a touch-sensitive surface). When the items are
`released (or “dropped') over the drop Zone of a desired target
`container, the selected items become the content items of the
`target container and the representations of the items appearin
`the target container according to the appearance style associ
`ated with the target container.
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`SUMMARY
`
`A user or application can select one or more item repre
`sentations in a user interface and initiate an operation (e.g., a
`drag operation) on the selected item representations by a
`movement of a cursor (e.g., a mouse cursor or trackball cur
`sor) or pointing device (e.g., a finger or stylus) in the user
`interface. When the operation is initiated, an animation can be
`presented illustrating the item representations moving toward
`a current location of the cursor or pointing device in the user
`interface and forming a cluster in proximity to the current
`location of the cursor or pointing device. As the cursor or
`pointing device moves in the user interface, the cluster of item
`representations can move with the cursor or pointing device
`in the user interface. Each item representation in the cluster
`can move in the user interface along a different path and at a
`different speed or acceleration than other item representa
`tions in the cluster.
`When the item representations are initially selected, the
`item representations can have respective first appearances
`according to the appearance style associated with the con
`tainer object from which the item representations are
`selected. As the cluster of item representations is moved over
`a drop Zone of a container object in the user interface, the item
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`representations in the cluster can take on respective appear
`ances based on the appearance style associated with that
`container object.
`In some implementations, a change in appearance is not
`presented if a user is moving the cluster over a container
`object in passing to reach another destination in the user
`interface, and the change in appearance is presented upon
`determination that the user is moving the cluster over the
`container object with the intent to place the cluster in the
`container object. The users intent can be estimated based on
`motion characteristics such as the speed or acceleration of the
`cursor or pointing device in the user interface as the cluster is
`moved over the container object.
`In some implementations, when the cluster of item repre
`sentations is moved over an area of an ineligible drop target,
`the item representations in the cluster can take on respective
`appearances that are different from their original appearances
`in the Source container from which the item representations
`are selected. The new appearances can be based on an appear
`ance style that is defined by the source container for the
`ineligible drop target. In some implementations, the Source
`container can define any number of different appearance
`styles that can be applied to the item representations based on
`different trigger events (e.g., hovering of the item represen
`tations over an ineligible drop target, passing of the item
`representations over particular regions within the source con
`tainer, etc.).
`In Some implementations, when the pointer or the cluster is
`hovered over a potential target container for more than a
`defined time period, an animation can be presented to show
`the item representations departing from the cluster and mov
`ing toward the representations anticipated locations in the
`target container object as a preview of a drop operation into
`the target container object. The item representations can
`appear in a preview state (e.g., as semi-transparent overlays)
`at or in proximity to the item representations anticipated
`locations and adopt the appearance style associated with the
`potential target container object. If the anticipated locations
`of Some item representations are in an area of the potential
`target container object that is not currently visible in the user
`interface, those item representations can be shown to move
`from the cluster into the invisible area along respective paths
`in the user interface.
`In some implementations, if the cursor orpointing device is
`moved away from the potential target container without drop
`ping the selected items, the representations of the selected
`items can be animated to dislodge from their respective loca
`tions at or near the anticipated locations and rejoin the cluster
`in proximity to the current location of the cursor or pointing
`device. Alternatively, if the selected items are released into
`the potential target container, the representations of the
`selected items can be animated to transition from a first state
`(e.g., a semi-transparent state) to a second state (e.g., an
`opaque state) and settle at the items anticipated locations in
`the potential target container.
`Some implementations include one or more application
`programming interfaces (APIs) in an environment with call
`ing program code interacting with other program code being
`called through the one or more interfaces. Various function
`calls, messages, or other types of invocations, which further
`may include various kinds of parameters, can be transferred
`via the APIs between the calling program and the code being
`called. In addition, an API may provide the calling program
`code the ability to use data types or classes defined in the API
`and implemented in the called program code.
`At least certain implementations include an environment
`with a calling software component interacting with a called
`
`APPL-1015
`APPLE INC. / Page 28 of 47
`
`

`

`US 9,323,442 B2
`
`3
`Software component through an API. A method for operating
`through an API in this environment includes transferring one
`or more function calls, messages, and other types of invoca
`tions or parameters via the API.
`In some implementations, the item representations can
`have appearances that are generated by the operating system
`based on various appearance styles defined by the Source
`container object, and various potential drop target containers.
`Each appearance style can be defined through the API by a
`corresponding container. Each appearance style can specify
`one or more layers for the item representation, and each layer
`can be associated with different properties and rules that
`control the appearance of the layer, the animation style of the
`layer, and so on. Different animations can be performed on
`the different layers independently of one another.
`In addition to the appearance style of the individual item
`representations, each container object can also define respec
`tive formation styles for the multiple items (e.g., all selected
`items) collectively as the item representations are in their
`various appearances. The formation styles can also be defined
`by their corresponding container objects through the API.
`Examples of the formation styles can include the layout of the
`item representations in their source container, the cluster
`formation near the pointer, and the preview formation in a
`potential drop target container as described herein. Other
`formation styles can be defined. Different animation styles
`that can be applied to illustrate the respective transitions
`between appearances and formations can also be specified by
`the source container or the potential drop target containers
`through the API.
`The details of one or more implementations of managing
`items in a user interface are set forth in the accompanying
`drawings and the description below. Other features, aspects,
`and advantages will become apparent from the description,
`the drawings, and the claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`4
`FIGS. 1J-1K illustrate exemplary user interface changes
`showing the cluster of item representations being dragged
`into a second potential target container associated with a
`second appearance style and the cluster of item representa
`tions changing their appearances according to the appearance
`style of the second potential target container.
`FIG.1L illustrates exemplary user interface changes show
`ing the representations of the selected items departing from
`the cluster and moving toward their respective anticipated
`locations in the second potential target container as a preview
`of the container's configuration if the selected items were to
`be dropped into the second potential target container.
`FIG. 1M illustrates exemplary user interface changes
`showing the representations of the selected items settling into
`their respective anticipated locations when the items are
`dropped into the second potential target container.
`FIGS. 1N-1P illustrate exemplary user interface changes
`showing selection and dragging of one or more items from a
`first container to a second container that is in a free-arranged
`State.
`FIGS. 1C-1R illustrate exemplary user interface changes
`showing a preview of the item representations being dis
`played at their anticipated locations in the second container
`and being moved by the pointer in the second container object
`during the preview.
`FIG.1S illustrates exemplary user interface changes show
`ing the selected items settling into their respective locations
`as shown in the preview after the items are released into the
`second container.
`FIG. 2A is a flow diagram of an exemplary process for
`causing representations of selected items to form a cluster in
`proximity to the current location of a pointer in response to
`the initiation of a defined operation (e.g., a drag operation) on
`the selected items.
`FIG. 2B is a flow diagram of an exemplary process for
`causing representations of the selected items to follow the
`continued movement of the pointer in the cluster.
`FIG. 3 is a flow diagram of an exemplary process for
`changing the appearances of the item representations when
`the cluster of item representations are moved to a container
`associated with a different appearance style.
`FIG. 4A is a flow diagram of an exemplary process for
`showing a preview of the item representations at their respec
`tive anticipated locations in a potential target container.
`FIG. 4B is a flow diagram of an exemplary process for
`dropping the selected items in the potential target container.
`FIG. 5 is a flow diagram of an exemplary process for
`moving away from the potential target container after the
`preview without dropping the selected items into the potential
`target container.
`FIG. 6 is a flow diagram of an exemplary process for
`determining whether to present the preview of the item rep
`resentations in their anticipated locations in the potential
`target container.
`FIG. 7 is a flow diagram of an exemplary process for
`changing the appearances of item representations when the
`items are being dragged from one container to another con
`tainer.
`FIGS. 8A-8C illustrate exemplary software architecture
`for implementing the menu presentation processes described
`in reference to FIGS. 1A-1S.
`FIG.9 is a block diagram of exemplary hardware architec
`ture for implementing the user interfaces and processes
`described in reference to FIGS 1-8C.
`
`FIGS. 1A-1C illustrate exemplary user interface changes
`showing representations of multiple selected items moving
`toward a current location of a pointer and forming a cluster in
`proximity to the current location of the pointer in response to
`the initiation of a defined operation (e.g., a drag operation) on
`the selected items.
`FIG.1D illustrates exemplary user interface changes show
`45
`ing movement of the cluster of item representations following
`the movement of the pointer in the user interface.
`FIGS. 1E-1F illustrate exemplary user interface changes
`showing the item representations in the cluster changing their
`appearances according to the appearance style of a potential
`target container while the clusteris within the drop Zone of the
`potential target container.
`FIG.1G illustrates exemplary user interface changes show
`ing the item representations departing from the cluster and
`moving toward their respective anticipated locations in the
`potential target container as a preview of the container's
`configuration if the selected items were to be dropped into the
`potential target container.
`FIG. 1H illustrates exemplary user interface changes show
`ing the representations of the selected items settling into their
`respective anticipated locations when the selected items are
`released or dropped into the potential target container.
`FIG. 1I illustrates exemplary user interface changes show
`ing the cluster of item representations departing from the
`potential target container following the movement of the
`pointer (e.g., the user decided not to drop the selected items
`into the potential target container).
`
`50
`
`55
`
`60
`
`65
`
`APPL-1015
`APPLE INC. / Page 29 of 47
`
`

`

`US 9,323,442 B2
`
`5
`Like reference symbols in the various drawings indicate
`like elements.
`
`DETAILED DESCRIPTION
`
`Exemplary User Interfaces for Dragging and
`Dropping Items in a User Interface
`
`6
`sented illustrating the representations (e.g., icons) of the
`selected items moving from their original locations toward a
`current location of the cursor or pointing device in the user
`interface and forming a cluster in proximity to the current
`location of the cursor or pointing device. Hereinafter, the term
`"pointer will be used to describe cursors (e.g., mouse or
`trackball cursors) and pointing devices (e.g., finger or stylus
`on a touch-sensitive display).
`In some implementations, as the user continues to move the
`pointer during the drag operation, the representations of the
`selected items can move as a cluster in the user interface
`following the pointer (e.g., a cursor in the user interface or the
`touch point of a finger or stylus on a touch-sensitive display).
`The individual movement of each item representation in the
`cluster can be independently represented in terms of path,
`speed, and acceleration relative to other item representations
`in the cluster. In some implementations, the item representa
`tions can trail behind the current location of the pointer at
`varying distances within the cluster.
`Furthermore, in Some implementations, the container
`objects displayed in the user interface can be associated with
`different appearance styles, and representations of items dis
`played within each container object can take on characteristic
`appearances according to the appearance style associated
`with the container object. Common appearance styles can
`include a large icon view, a small icon view, a list view, a
`column view, or a picture view, for example. Other appear
`ance styles associated with application windows can be
`defined by the application programs providing the application
`windows, for example, through an API associated with the
`operating system. In addition to the appearance styles for
`individual item representations, each container object can
`also define a formation style for the selected item represen
`tations collectively. Example formation styles include a free
`arranged layout, a keep-arranged layout, a cluster, etc.
`In some implementations, when items are initially
`selected, the representations of the selected items can have
`respective first appearances according to the appearance style
`associated with the items’ respective source containers. As
`the representations of the selected items form a cluster and are
`moved over the drop Zones of different container objects in
`the user interface, the representations of the selected items in
`the cluster can take on different appearances depending on the
`appearance style associated with the container object over
`which the cluster of item representations is currently located.
`In some implementations, when the selected items in the
`cluster are over areas of ineligible drop target containers, the
`representations of the selected items in the cluster can take on
`appearances based on other appearance styles that are defined
`by the source container object for the ineligible drop target
`containers. The formation style of the selected items can also
`change according to a formation style defined by the source
`container for the ineligible drop target containers.
`FIGS. 1A-1C illustrate exemplary user interface changes
`showing representations of multiple selected items moving
`toward a current location of a pointer and forming a cluster
`near the current location of the pointer in response to the
`initiation of a defined operation (e.g., a drag operation) on the
`selected items by the pointer.
`In FIG. 1A, an exemplary user interface 100 is presented.
`The exemplary user interface 100 can be a desktop of an
`operating system. A number of container objects (e.g., a
`folder window 104, a folder window 106, and an application
`window 108) are displayed on the user interface 100, for
`example, within a display area 102 of the desktop.
`As shown in FIG. 1A, the folder window 104 contains
`representations of items (e.g., item representations 110, 112,
`
`In a file system, information items (e.g., files and folders)
`can be organized in a file system hierarchy. Each information
`item (e.g., files and folders) can be contained in a parent item
`(e.g., a parent folder) in the file system hierarchy. In an appli
`cation program (e.g., an html editor, a word processor, an
`email editor, etc.) for editing a document, content items (e.g.,
`image, videos, data objects, etc.) can be embedded at particu
`lar locations in the document. In addition, information items
`(e.g., address book entries, data objects) and user interface
`elements (e.g., buttons) can be moved from and inserted at
`defined locations in an application window.
`In a graphical user interface (GUI), container objects, such
`as the desktop, folder windows and/or application windows,
`can be used to represent folders in the file system hierarch