CiM Ex. 1027 Page 1
`
`

`
`About the Author
`
`an Cooper» the “Father ofVisu I B3
`
`’
`asic " is an aw
`
`Since 1975 Ala
`’
`
`n Cooper has desi
`
`
`
`"
`
`
`
`nterFace Design
`
`Published by
`Hungry Minds, Inc.
`909 Third Avenue
`New York, NY 10022
`www.hungryrninds.com
`Library of Congress Catalog Card No.
`ISBN 1-56334-322-4
`Printed in the United States of America
`16 1514 131211 10
`Limited in the United
`Minds, inc.
`e United States by Hungry
`G Books Canada inc. For Canada; by Transworld Publishers
`Books for Norway; by IDG Sweden Boolts For Sweden; yPublishers Pte Ltd. for
`Publishing Corporat
`Australia and New
`' Thailand, Indonesia, and Hong Kong; by G
`'
`al
`'
`lle
`
`: 95-75055
`
`_
`
`'
`
`our Sub—R'ights Customer C
`
`REE
`
`.
`.
`
`
`
`-
`
`.
`
`.
`
`.
`
`
`
` rx......m-Minds“ is a tradernarlt of Hungry Minds, inc.
`
`'
`Publishing for Germany,
`.
`.
`.
`International For Brazil; by Galileo Libros for Chile; by Edi
`'
`'
`' by Contemporanea
`Distributors for the Caribbean and
`ter Publishing Corporati
`.
`de CV. for Mexico; by Edi
`Cornpu
`ucts and services please contact our Customer Care
`ax317-572-4002
`Venezuela; by Express Computer
`Distributor, Inc. For Micronesia; by Chips Computadoras S.A.
`Panama S.A. for Panama; by American Bookshops for Finland.
`For general information on Hungry Minds’ prod
`hin the 1.1.3. at 800-762-2974, outside the U.S. at 317-um and bulk quantity sales, an
`Department wit
`For sales inquiries an
`'
`including discounts, prerni
`languagetranslations,
`CareDepartmentat800-434-3422,fax317-572-4002,
`or write to Hungry Minds, Inc., Attn: Customer Care Department, 10475 Crosspoint Boulevard,
`'
`Shed and developeel software '
`-;.
`l-
`me Ll-ding Su_g>erProj.-_=;-
`Indianapolis, IN 46256.
`
`
`For information on licensing foreign or domestic rights, please contact
`rEducationalSalesDepartmentat800-434-2086orfax317-572-4005.
`_';_(C0mpu:e,Associates) M‘
`ForinformationonusingHungryMinds’ productsandservicesintheclassroomorfororderingexarniiia-
`_
`2-884-5000
`--
`_
`»
`ICTOP/JoneUfa!‘ lifindaw: (Softvv V
`eriture"s)" and
`_
`are
`h
`-
`t e visual
`In I976 h
`Departmentat
`2-884-5000.
`’
`'
`'
`E ‘C0’-“'1 éd Stru
`3.2.
`Ctured
`
` Ystems Group which Fr '5
`.
`5
`cl erger and swaine in
`Ei1eI1'.lJ00kF1'?‘eih
`J
`884-5163 for press review copies or 21
`Q COIIEHCC DU.
`..
`r/ac Lpyyyey’ Credited
`tion copies, pleas
`c Relations Department at 212-
`
`with pr0duC- ..mg Pfirhaps the first seriou b
`
`.
`s usiness sof
`.
`..
`.
`d other publicity information or fax 212-884-5400.
`fiva
`:
`for a In.
`..
`lcrocom
`Please contact our Publi
`
`'
`'
`'ty information,
`For author interviews an
`fax 317-572-4168. or educational use, please contact Copyright
`For press review copie
`tment at
`Relations depar
`For authorization to photocopy items for corporate, personal,
`or fax 978-750-4470.
`Clearance Center, 222 Rosewood
`TY THE AUTHOR AND PUBLISHER OF
`0
`THE PUBLISH-
`CT TO
`
`ard-Winning use ‘r interface Con-
`Sultant
`baflcbfoftware designer. His comp
`C
`with
`PEI Interact;
`.
`ge of clients to 1
`_
`O0
`an)?’
`{'03
`ran
`.
`at
`On Design has
`Inprove their
`products and he]
`.
`!
`worked
`.
`.
`cm Create '7-‘XC1t1ng
`p
`and successful new sofEware products. His ex
`.periences in i
`'
`.
`H1 [fin-1
`'-
`P mung his unique
`aPpr03Ci1 to C1‘
`‘
`.
`eating better software thmugh g0al—direct d de
`esign led him to W .
`book.
`
`“:6 this ‘. -_
`
`
`‘
`
`
`
`rh
`
`
`
`CiM Ex. 1027 Page 2
`
`

`
`PART IV: THE INTERACTION
`
`cursor makes as it merely passes over something on the screenlfi-_§:_'et1rsog
`Once the captive phase has begun, I call changes to the cursoripmajglvj
`i9“..r.s9r'l.iinfiI!.1§-
`Microsoft Word uses the clever free cursor hint of reversing the angle of the
`arrow when the cursor is to the left of text to indicate that selection will be line-
`by-line or paragraph-by-paragraph instead of character-by—character as it nor-
`mally is within the text itself. Many other programs use a hand—shaped cursor
`to indicate that the document itself, rather than the information in it, is drag-
`
`Microsoft is using captive cursor hinting more and more as they discover its
`usefulness. Dragging—and-dropping text in Word or cells in Excel are accompa-
`nied by cursor changes indicating precisely what the action is and whether the
`objects are being moved or copied. In Windows 95, when you drag a file in the
`Explorer, you actually drag the text of the name of the file from one place to
`
`When something is dragged, the cursor must drag either the thing or some sim-
`ulacrum of that thing. In a drawing program, for example, when you drag a
`complex visual element from one position to another, it may be too difficult for
`the program to actually drag the image (clue to the computer’s performance
`limitations), so it often just drags an outline of the object. If you are holding
`down the CTRL key during the drag to drag away a copy of the object instead
`of the object itself, the cursor may change from an arrow to an arrow with a lit-
`tle plus sign over it to indicate that the operation is a copy rather than a move.
`This is a clear example of captive cursor hinting.
`
`
`
`of all the direct—manipulation idioms characteristic of
`
`the GUI, nothing defines it more than the drag-and—drop
`
`°P91'3d0n. clicking and holding the button while moving
`some obyect across the screen. Surprisingly, drag—and-drop
`isn’t used as widely as we imagine, and it certainly hasn’t
`
`lived up to its fi.ill potential.
`
`Whither drag—and-drop?
`Any mouse action is very efficient because it combines two
`lcomniand components in a single user action: a geographical
`téfiatlon and a specific function. Drag-and-drop is doubly
`c
`cient because, in a single, smooth action, it adds a second
`geographical location. Although drag—and—drop was accepted
`
`r
`
`is
`it
`Immeiiately as a cornerstone of the modern GUI,
`.
`»
`Omar able that drag-and—drop IS found so rarely outside of
`Programs
`that
`specialize
`in
`drawing
`and
`painting
`
`CiM Ex. 1027 Page 3
`
`

`
`PART IV: THE INTERACTION
`
`
`
`_
`
`CHAPTER 18: Dims-AND-Dnop
`
`249‘
`
`.
`specification. To get a communi
`FIY Of third- at
`d
`[
`P W W: opus to adopt 3 tech‘
`"0108?! there is somethin
`b
`_
`1
`3 solid Set of library mutigneeyegatcgrleliiafii::iI:iI:i1ving a defined standard: having
`applications without having to invent th
`[:0 implement features in their
`libraries have ever been made available in E ti; nology th.cmsclvcs' N0 such
`is so large and frustratingly complex th
`:16 'lnd0W5 environment. OLE 2_()
`drop standard will become
`.
`at t ere is some peril that the dra —and-
`.
`either lost
`b
`'
`-
`.
`implementations. This unfortunate bifld :3‘ mliardized in various proprietary
`vendor encapsulates the fiinctionality F d 011 Ydbe resolved when some clever
`°
`'“’="“‘“ '°"°P in =1 P0Werfiil
`flexible
`and easy—to-program pack
`11
`-
`-
`developers.
`336. t en makes it widely available to applications
`
`I find it amusing that th M‘
`-
`It makes it Sou“
`_
`e
`icrosoft style guide treats drag-and-dro so li htl
`Y.
`d like a simple and commonly known process
`hp
`h S
`1 3-5 E Ollg
`it was
`describin g how to put on your shoes '
`-
`m the '“°"“"1E- SONY, It‘s just not that
`easy.
`
`There are several variations of drag-and-drop, and they are only a subset of the
`many forms of direct manipulation. The characteristics of drag-and-drop are
`fuzzy and difficult to define exactly. We might define it as “clicking on some
`object and moving it elsewhere,” although that is a pretty good description of
`repositioning, too. A more accurate description of drag-and-drop is “clicking
`on some object and moving it to imply a transformation.”
`The Macintosh was the first successful system to offer drag-and-drop. A lot of
`expectations were raised with the Mac’s drag-and-drop that were never truly
`realized for two simple reasons:
`1. Drag-and-drop wasn’t a system-wide facility, but rather an artifact of the
`Finder, a single program.
`the concept of drag-and-drop between
`2. As a single-tasking computer,
`applications didn’t surface as an issue for many years.
`To Apple’s credit, they described drag-and-drop in their first user interface stan-
`dards guide. On the other side of the fence, Microsoft not only didn’t put drag-
`and-drop aids in their system, but it wasn't described in their programmer docu-
`mentation. Nor was it implemented in their Finder equivalent, the notoriously
`brain-dead MSDOS.ExE, the first Windows shell. The only drag-and-drop anywhere
`in Windows was in the simple paint utility distributed with the system. Yet again,
`Microsoft shipped an operating system—a standard-defining tool—but abdicated
`their responsibility for adequately defining collateral standards. I’m not ungrateful,
`as Windows was still by far the best thing around on the PC platform. Still, had
`Microsoft defined even some rudimentary standards, the drag-and-drop world
`would have evolved stronger and more rapidly.
`It wasn’t until Windows 3.0 that any drag-and-drop outside of MSPAINT.EXE
`appeared. The new File Manager and Program Manager programs supported a
`rudimentary form of drag-and-drop. You could drag icons around in the
`Program Manager and files and directories around in the File Manager. Wonder
`of wonders, you could also drag an EXE file‘ from the File Manager into the
`Program Manager and create an icon, although few users knew this. This dis-
`appointing lack of design leadership has resulted in an industry-wide sluggish-
`ness to embrace drag-and-drop, much to our software’s detriment.
`After ten years, though, Windows is finally getting a drag-and-drop standard.
`
`
`
`g-and-dr
`
`Dragging where?
`Fundamentally you can dra
`-
`v
`op something from one place to another
`inside your program oi-
`-
`d-d
`-
`.
`,
`YOU can dr
`:
`ag an mp 5°m°tl“"g 5'0!!! inside your
`program into some other program I can
`._
`th
`-
`i...
`:.
`_____ ___ ___"_u
`.
`EH :ind.‘.’x.‘.5_°£“.?E' _‘.i..1I.‘!,E;and_-Q1-l:°£i, respectiveugse variants E_1fl|I!:de_-E9.‘-q._§13E:2_-r‘l"‘g:j
`-.1...
`Interior drag—and—droP C311 be made pret
`s‘
`1
`d
`W Imp e, both from a conce tual
`from a coding
`.
`,
`an
`cl
`_
`_
`. P
`point of view. Exte '
`more sophisticated support because Eel:-ly MB and drop demands significantly
`°°“°°P'55. and they must be im lem
`dp-mgmms rlmst subscribe t° the Same
`about the exterior variant ill’-tCI'FV)VC gli::II:tai:loli1kc-,“:l?fi3‘1t'.i,hli;: ways. Wsvll talk more
`e
`asics o drag-and-drop,
`
`Iclassified rc
`'
`'
`'
`'
`-
`.
`.
`positioning as a dlrect-
`'
`d '
`m3mPl1l3tl0n Idiom and disc
`last cha
`U556 “m thc
`pter. Now we will discuss
`'
`the remai
`’
`_
`.
`.
`.
`Pnmanllfi there are two: master-and-tar et
`cl mng dnlg ancbdmp “Hams-
`3
`an tool manipulation.
`
`Master-and-target
`When the use
`-
`r clicks on a discrete ob‘
`‘
`-
`'
`_5S It to another discrete b‘
`111 order to
`.
`J?“ in
`perform a function, I call it §i_i;;_;s£e,,_..
`0 Ject
`1"“ Obicct with‘
`-
`
`d dra
`
`CiM Ex. 1027 Page 4
`
`

`
`Paar IV: THE INTERACTION
`
`CHAPTER 18: DRAG-AND-DROP
`
`251
`
`
`
`fi
`
`.
`
`to be‘ M05‘-[Ya W5 a matter of d
`'
`dragged to an object, all the targefotlzcgt Eltcrface stan“dards. When data is
`drop” to the mast" Oblect. The two 0131-
`as to say '5 yes’ I can 3‘3°‘3Pt I116
`because it is unreasonable to expect eve]°Ctsbtl‘lcn must negotiate over formats,
`I' 0
`-
`Programk Pr°P"i°t‘“'Y formats. If the mas}t’er 11:9: to. accept dam "1 °"'¢"Y Othcr
`offer the data in its internal format Anoth
`1?I‘]cCt is Excel’ say’ it may initially
`to decipher this format but a Bméd X
`icroszft Program may know how
`ecii'
`-
`.
`’
`pro uct mig I: not. So th B
`get object polit I d
`_
`C
`rand X tar-
`contents. ExceL°3;1c°r:1a::':r :lr(1J1:5ltIOtllililfi droptlfbut to the format of the dropwa
`3°“°"° f°'“m5= SYLK, csv ASCII rhieifigci tile. dam ”' ‘“°°°““""'Y ‘"0"
`SYLK or CSV but b
`’
`.
`, '
`C 0 Jcct can turn up its nose at
`9
`y conventio
`_
`.
`.
`mon denominator format on all algccpt ASCII, it is the lowest com-
`‘“‘“"“a"Y accept ASCII, siri1Ple bitma 5'
`‘fiery °’"°“°' ““P°b'° Object must
`functions. Objects that hope to becorl: a Pointer;[to files and, as we.” see,
`in the 0 C
`.
`“WP! many more formats than th
`[3
`P n market W!“
`with '3"°l'Ythif1g. Even an audio file afor elilathclc four gliarantcc °°"‘P“ibi1ity
`simple pointer to a disk file. I call exterior ZIP 6, can ultimately be passed as a
`port this type ofhaggling over formats §%‘§: i“_ls that sup.
`
`e success
`
`'
`'
`|.-
`-n;....._.._....._
`- _3‘an -
`o .
`
`I call protocols like thos
`
`‘
`
`h
`
`'
`
`.
`Manager, which don’t ncgzfiraitilz: ff)rl(1‘1f;I:Sfl?g5-§’|vIF:|‘?E3l'c Mamlggy Ellid Program
`|.-.._..n.........__
`_
`’ -
`°"==a=»;;..?=‘Ea.-s:.a_r:i:._,4ii;=iig.
`
`Dragging fimctions to data
`
`.
`
`is the window. When the user ultimately releases the mouse button, whatever
`was dragged is dropped on some'tar-,g_et' object‘;
`The main purpose of the term “master-and-target” is to differentiate this oper-
`ation from the kind of drag-and-drop operations we find in drawing and paint-
`where tools and graphical objects are dragged around on an
`ing programs,
`open canvas. Master-and-target
`is a more function-oriented idiom, where
`manipulating logical objects represents some behind-the-scenes processes. The
`and-target drag-and-drop is rearranging icons in
`most familiar form of master-
`the Program Manager or in the Macintosh Finder.
`
`Dragging data to functions
`Instead ofdragging a file or folder to another folder, you can drag it to a gizino
`function. This idiom is arguably the most famous expression
`that represents a
`trashcan. Windows
`of direct manipulation because of the Macintosh’s familiar
`95 copies this familiar idiom with its “recycle bin.” Someday, as we build soft-
`ware with better object-orientation, we’ll be able to drag-and-drop objects
`senting functions other than just delete. Imagine targets rep-
`onto gizmos repre
`a contents-indexer.
`resenting a cloner, an archiver, a file compressor, a fairer or
`Notice that all of the idioms in the above paragraph involve exterior drag-and-
`drop, because the target objects are separate programs. Within a single pro-
`the code knows what objects are draggable——usually one type—and any
`n will easily handle it. In an exterior drop,
`function gizmo that it gets dropped 0
`the master object can come from any program, and the target gizmo may well
`not have any direct knowledge of the originating program or the dropped
`object. The target must be able to handle the unknown object in some reason-
`able way without necessarily understanding what it is or what is in it. The
`Program Manager, for example, can do this because it knows that it will only
`be handed files. What would it do if it were handed a paragraph of dragged text
`from a word processor, for example? If it can’: handle the text, it isn’t truly
`exterior capable. To Microsoft‘s credit, the Recycle Bin in Windows 95 can
`actually accept paragraphs of text dragged from Word or cells dragged from
`Excel. I have not yet been able to determine whether these are generic opera-
`tions or just code specific to Microsoft applications.
`To be truly exterior capable, an object must be able to accept a drop of any-
`
`Prop" ncgmiatcd exterior dra
`.’
`’
`-
`d-d
`'
`'
`-
`.
`d"°PD1ng functions onto data as Ewell“as d::p Iapablhty mchides dmggmg'a"d‘
`tions. Defining the scope of such actions gg"bg—and-dropping data onto fimc‘
`concrete data, but it can still be generall:1:;
`':problf:1:_lI]na£lC when working in
`could click on the italic buttcon
`m 6 use
`' 0' °’‘amPl° 3 11561‘
`th
`-
`’
`a spreadsheet. Clearly, the use;~=s0i:ten: l3i::Otllll);ra:r'i:ii:l drag It down onto a cell in
`that can to italic Part of th
`on is to turn the content in
`_
`-
`e lo
`'
`'
`-
`.
`mm a fimcfion as a valid droPrr‘1’1;tJ:iegCoct:uon includes being able to recog,
`bttween the function “delete” and the-fu
`‘ccptsgalh-I’ fiber: is hm: difference
`Effirn deletes its internal copy of the dagcnog hltal:llC.. In one, the target pm-
`other the target
`an
`an s it to the master. In th
`’
`Program hands a
`f
`.
`.
`_
`C
`°°“"°'t5 the text to italic and hahcdgyif btllicdirfii-t° "‘° mil“: fim~=t1on,which
`Exce1’s window can be dragged onto text §n.W lfpway the italic buttcon in
`
`
`
`CiM Ex. 1027 Page 5
`
`

`
`Paar IV: THE INTERACTION
`
`For now, there is no standar
`purports to offer one. Certainly,
`col.‘ A given protocol may allow fo
`mileage may vary.
`
`d exterior drag-and-drop protocol, although OLE
`there is no negotiated drag—and-drop proto-
`rmat negotiation or function dragging. Your
`
`How master-and-target works
`isually hint at its pliancy, either statically in
`A well-designed master object will v
`imating as the cursor passes over it.
`the way it is drawn, or actively, by an
`The idea that an object is dragable is easily learned idiomatically. It is difficult
`to forget that an icon, selected text or other distinct object is directly manipu-
`lable, once the user has been shown this. He may forget the details of the
`action, so other feedback forms are very important after the user clicks on the
`object, but the fact of direct-manipulation pliancy itself is easy to remember.
`The first-timer or very infrequent user will probably require some additional
`help. This help will come either through additional training programs or by
`advice built right into the interface. In general, a program with a forgiving
`interaction encourages users to try direct manipulation on various objects in
`the program.
`
`use button over an object, that object
`As soon as the user presses the mo
`tion of the drag-and-drop. On the other
`becomes the master object for the dura
`hand, there is no corresponding target object because the mouse-up point hasn‘t
`yet been determined:
`it could be on another object or in the open space
`between objects. However, as the user moves the mouse around with the but-
`ton held down-—remember, this is called the captive phase—-the cursor may
`f objects inside or outside the master object’s program. If
`pass over a variety 0
`possible targets, and I call
`these objects are drag-and-drop compliant, they are
`them l§'9§ =.*'.3_1+3.'.i."'.i!’E'.=§-
`get in a drag, but there may be many
`There can only be one master and one tar
`and-drop protocol, the drop candi-
`drop candidates. Depending on the drag-
`lar dropped value, it just has to
`date may not know how to accept the particu
`uire
`know how to accept the offered drop protocol. Other protocols may req
`that the drop candidate recognize immediately whether it can do anything use-
`ful with the offered master object. The latter method is slower but offers much
`better feedback to the user. Remember, this operation is under direct human
`
`'
`'
`- ROP
`CHAPTER 13. DRAG AND D
`
`253
`
`extensive convcl-sin
`.
`-
`3 between
`""°“"°“°“ fa“ bc sluggish
`thI:'[llasm ohm‘ 3”“ “Ch drop Candidate th
`, at w ic point it isn’i; w
`’
`
`Ol'Ll‘l
`
`lIl'lC gamc
`
`C
`
`Visual indications
`
`The ‘mil? task of each dm[3 candidate is to '
`-
`-
`[1
`visua y indicate that the hotspot of
`the captive curso '
`-
`r is over it, meanin
`.
`B that it will acccPt ‘hf d"°P—0r at least
`°°mPrehend it—-if the user releases the mo“se button Such ani
`-
`ndi
`'
`-
`.
`its rlatur
`-
`-
`.
`.
`e, active visual hinting,
`cation is, by
`""—' -A.-.-—_ -..
`_
`.
`.'~.--~
`_
`P
`-lm1iJ.i‘a'.;i'g_ri= tip: Theth-.
`'
`.
`dropibiclfilgandldaue must mS“‘3lY'i11dicate its L
`
`The weakest way to offer the visual indicati
`.
`.
`.
`ropability is by changing the
`on o
`cursor. It is the job of the c
`- h .
`'ursor to represent wh
`H .
`.
`_
`indications ofdrop candidacy to the drop Cand‘i::l:te ft];-leglffllraggcd and have
`a
`pays.-;,;i.;;,,:i~.1t,,;£......--..,.eW - ,_,
`an
`I
`_ p]?J.PqF' 8
`sor must visually indicate the masterl
`
`fd
`
`It is important that these two 'visual fiinctions not be confiised Unfortun
`ately,
`'
`Microsoft seemS to have done so in
`.
`.
`both W d
`pcct this decision was m
`In ows 3.x and Wind
`ade more for the C
`ows 95. I sus-
`ase of co
`ding than for a
`d -
`ge the cursor than it is to ll1:lrvc:1s:En
`~
`uch e
`'
`considerations It is m
`Candidates highlight to showa:ll:r' “:1 Chan
`bil'
`. Th
`.
`P
`eir
`ropa
`: sllguld lac om: c"“°’ ‘5 ‘° rep-
`b
`’
`d
`'
`resent the master the dra
`drop candida”;
`S89 0 leer. I
`e used to represent the
`
`As if that w
`1
`'
`eren t bad enou
`Eh. Microsoft perform
`.
`-
`-
`-—s._c£1_ri(f"_‘l}inting with the
`detestable circle with bend sinister which I c
`’
`3“ 3 El-_|_1ister-cirrcI_g_
`
`The sinist
`'Cr-circle is not a
`I
`-
`-
`P casant 1dl0m because it tells u
`do. It is ne
`'
`sers what they carp:
`a
`gative feedback. The sinister circle '
`.d
`user can easil
`‘
`IS an 1
`iom for “cl
`-
`’
`'
`Y construe its
`'
`0n t do It ” and
`Yousu do some irreversible dmeanirlg. to be ‘salon’: let go of the mouse nbw
`nothing will in
`amagc
`Instead of “go ahead
`d
`°'
`ppen.” Adding the Sims
`_
`an let go now and
`ter—circle to c
`-
`-
`.
`I-lrsor hinting is a sad com-
`blnation of “V0
`
`CiM Ex. 1027 Page 6
`
`

`
`
`
`AND-DROP
`
`255
`
`PART IV: THE INTERACTION
`
`Once the user finally releases the mouse button, the current drop candidate
`becomes the target.
`If the user releases the mouse button in the interstice
`between valid drop candidates, or over an invalid drop candidate, there is no
`target and the drag-and-drop operation ends with no action. Silence, or visual
`inactivity, is a good way to indicate this termination. It isn’t a cancellation,
`exactly, so there is no need to show a cancel stamp.
`
`Indicating drag pliancy
`Active cursor hinting to indicate drag pliancy is a problematic solution. In an
`increasingly object-oriented world, more things can be dragged than not. A
`cursor flicking and changing rapidly can be more of a visual distraction than a
`help. One solution is to just assume that things can be dragged and let the user
`experiment. This method is reasonably successfiil in the Program Manager, the
`File Manager and the Explorer. Without cursor hinting, drag pliancy can be a
`hard-to-discover idiom, so you might consider building some other indication
`into the interface, maybe a textual hint or a ToolTip-style popup.
`
`Once the master object is picked up and the drag operation begins, there must
`be some visual indication of this. The most visually rich method is to fully ani-
`mate the drag operation, showing the entire master object moving in real-time.
`This method is hard to implement, can be annoyingly slow and very probably
`isn’t the proper solution. The problem is that a master-and-target operation
`requires a pretty precise pointer. For example, the master object may be 6 cen-
`timeters square, but it must be dropped on a target that is 1 centimeter square.
`The master object must not obscure the target, and, because the master object
`is big enough to span multiple drop candidates, we need to use a cursor hotspot
`to precisely indicate which candidate it will be dropped on. What this means is
`that, in master-and-target, dragging a transparent outline of the object may be
`much better than actually dragging a fully animated, exact image of the master
`object. It also means that the dragged object can’t obscure the normal arrow
`cursor either. The tip of the arrow is needed to indicate the exact hotspot.
`
`Dragging an outline also is appropriate For most repositioning, as the outline
`can be moved relative to the master object, which is still visible in its original
`
`CiM Ex. 1027 Page 7
`
`

`
`PART IV: THE INTERACTION
`
`CHAPTER 18: DRAG-AND-Daop
`
`257
`
`
`
`to use some other tool on the ra h‘
`box and Select the eraser too] EH11: ic. like anEraser, he must return to the tool.
`
`'
`
`.
`
`program [ cu enters “eras
`
`C1’ mode” and can
`
`'
`
`The implication of this more-detailed conversation is that the transfer may fail.
`That is okay. It is better to show dropability and choke on the actual drop than
`it is to not indicate dropability. (If minimum common format standards are
`adhered to, after all, there should never be a physical failure.) If the drag-and-
`drop is negotiated, the format of the transfer remains to be resolved. If infor-
`mation is transferred, the master-and-target may wish to negotiate whether the
`transfer will be in some proprietary format known to both, or whether the data
`will have to be reduced in resolution to some weaker but more common for-
`mat, like ASCII text.
`
`Visual indication of completion
`If the target and the master can agree, the appropriate operation then takes
`place. A vital step at this point is the visual indication that the operation has
`occurred. If the operation is a transfer, the master object must disappear from
`its source and reappear in the target. If the target represents a function rather
`than a container (such as a print icon), the icon must visually hint that it
`received the drop and is now printing. It can do this with an animation, or by
`changing its visual state.
`A richly visual master-and-target drag-and-drop operation is one of the most
`powerful operations in the GUI designer’s bag of tricks. I know that if this
`idiom is better supported by tool vendors, it will grow in popularity with appli-
`cation developers. Users will be the beneficiaries.
`
`Tool-manipulation drag—and—drop
`In drawing and painting programs, the user manipulates tools with drag-and-
`drop, where a tool or shape is dragged onto a canvas and used as a drawing tool.
`There are two basic variants of this that I call modal tool and charged cursor.
`
`Modal tool
`
`the user selects a tool from a list, usually called a toolbox or
`palette. The program is now completely in the mode of that tool: it will only
`do that one tool's job. The cursor usually changes to indicate the active tool.
`
`..._
`an eraser tool and
`
`.
`
`Modal tool works for both tools th
`-‘ —"*—"
`eraser—or for
`that can be d at Pei-lfprm ______
`.._p
`fawn-—i C c '
`.
`-
`erase anything previously cmcred
`tool and draw any number ofnew Cllips¢s_
`Modal tool is not botherso
`'
`me in a program like Paint wh
`1
`drawing tools is vu-Y small. I
`.
`Adobe Illustrator, however thcnmzcrliipre ‘advanced drawing program such as
`3
`3
`a ity is very disruptive because as uh
`gets more facile with the cursor and th
`motion devoted to selecting and d
`1
`F tools, the percentage of time and
`ese
`.
`.
`ma“°3“Y- Modal tools are excellent .d-cam: tOO]S‘th° °XClse—-increases dra-
`l 101115 orintroducin3 Users to the range
`of features of such 3 P1'°gi'am but the d
`’
`users of more sophisticated P;Ogmms'Y on t usually scale well for experienced
`
`'
`
`, or it can become an Clllpsc
`
`.
`
`ere the number of
`
`e user
`
`2 Products like Adobe
`5. The SHIFT key is com-
`monly used for constr 'a
`mctmkcys and uses thcmiped drags. but Illustrator a
`dd‘ many Hon-standard
`_
`non standard ways, For cm
`mples holding down the
`-
`ALT key whil d
`.
`.
`e
`tagging an 0b]CClZ drags away a copy of
`_‘h‘“ °bl¢Ct, but the ALT
`
`n
`
`CiM Ex. 1027 Page 8
`
`

`
`PART IV: THE INTERACTION
`
`CHAPTER 18: DRAG~AND-DROP
`
`259
`
`
`
`must press the ALT key, point to the object, press and hold the mouse button
`without moving the mouse, release the ALT key, then drag the object to the
`desired position! What were these people thinking?
`
`Admittedly, the possible combinations are powerful, but they are very hard to
`learn, hard to remember and hard to use. If you are a graphic arts professional
`working with Illustrator for eight hours a day, you can turn these shortcomings
`into benefits in the same way that a race car driver can turn the cantankerous
`behavior of a car into an asset on the track. The casual user of Illustrator, how-
`ever, is like the average driver behind the wheel of an Indy car: way out of his
`depth with a temperamental and unsuitable tool.
`
`Adobe Illustrator is firmly rooted in the Macintosh world. One of the major
`errors that Adobe made in their Windows interface design was a refusal to take
`advantage of the benefits of the two-button mouse, something that comes
`cheap or free with Windows. Illustrator doesn’t use the right mouse button at
`all. I suspect that someone in the company felt that interoperability with the
`Mac was more important—a bad notion, as I’ve discussed before. Adobe could
`have put all selection tools on the left button and all drawing tools on the right
`button, just for an example. Users could then go back and forth between draw-
`ing things and manipulating them just by deciding which mouse button to use,
`and, even better, each button would then have available to it three meta-keys:
`ALT, CTRL and si-III-'1'. Not taking advantage of the right mouse button was an
`error on their part.
`
`Charged cursor
`The second tool—manipulation drag-and-drop technique is what I call
`
`Man)’ common drawing to rams
`'
`-
`-
`m hi
`d_
`_
`_
`. work this way, but it is also very popular for
`gf P dc mxt mampulanon idioms in Programs that aren’t normally thou ht
`o as
`rawing programs A
`.
`.
`_
`E
`-
`E0011 example is Visual Basic Wh
`h
`'
`on one of thc gizmos on the mo]
`.
`en t e user clicks
`palette, the cursor becomes char
`_
`cl
`'
`.
`h that
`'
`gizmo. The user then clicks a
`gc wit
`-
`.
`gain to create a sin le inst
`f‘
`Bo,.],md,s Del hi uses ch
`ance 0 it on a form.
`‘
`3
`P
`urged cursor moi but 1fY0l1 SI-IIFT—click on a gizmo in
`the palette, you
`-
`get a modal tool instead for c
`'
`'
`-
`gizmo. Nice touch.
`mating mulnp1: instances of 3
`
`In many charged curso
`
`-
`
`.
`
`tlge user cannot always
`deposit the object with :1 Eiilc-:E:':1':ll9IiCll:l{t: I’owerP(cjiint,
`dcmminc the Size of the dc
`I
`‘u must
`rag a ounding rectangle to
`Posited object. Some
`ro rams l'k
`'
`-
`allow cither method A sin
`_
`P 8
`,
`I e Visual Basic,
`815 Click of a charged cursor cr
`°
`'
`'
`of-the object in 3 standard Size Th
`eates a single instance
`.
`.
`_
`.
`b
`-
`so it is surrounded by grapples andzlaccyy :0: :11-cared in ‘a state ofselection,
`-
`-
`.
`j
`e iate precision resha '
`d
`'°5'z'“g' Th“ ‘"-""“'"‘°d‘=» allowing either a single click for a dcsaiiiiaiid
`Obi."-‘Ct or dragging a rectan
`-
`_
`gle for a custom—s
`d
`‘
`‘
`-
`flexible and discoverable and will satisfy rnustlzc object is certainly the most
`=
`users.
`
`I have seen char ed cur
`
`the cursor. For cfamplC,S:(l)l:hPc:'l‘J3§l:il‘I;‘1i:J-Ezgffi to change the appearance of
`when it’s charged, Delphi doesn’t change it at aly ‘1;i‘_"5 ‘the cursor to crosshairs
`501' has assumed :1 modal bch,wiO,._if dickin .
`«
`is is really -silly: if the cur.
`g it somewhere will create some-
`thing-it is imperative that it v’
`'
`'
`isuall
`'
`absolutely demands ood c
`| ‘d.
`Y Indicate lhls state’ Charged Cur5°r 315°
`1;
`ance l
`iorns; otherwise,
`charge the cursor?
`how do you harrnlessly dis-
`
`Bomb sighting
`
`-
`'
`h dro cand'd
`.
`.
`.
`eac
`_
`'
`_
`_
`d
`.
`p
`i are visu
`any changes as lt
`is pointe to, which indicates its ability to accept the drop. In
`can i
`t
`'
`S°"‘° P"°g1'='-"15, the master ob'ect
`'
`J
`"5 cad be d"_°PP€d in the spaces between
`Oth" °bl¢CtS I call this variant of dra
`" "
`'
`'
`g—a d—d
`'
`-
`-
`-
`.
`'
`is 3 bombardicr nopcr::p0[:tJmb11'd1¢fi. Dragging text
`as are most rearranging
`
`m Word, for example,
`°P€1'ations.
`
`The viral visual feedback of bomb
`'ardier drag-and—dro
`is sho '
`P
`master
`f
`»
`wlng WIICTC thfl
`all if the user releases the mouse button. In master-
`0 ject wi
`
`
`
`With charged cursor, the user again selects a tool or shape from a palette, but
`this time the cursor, rather than becoming an object of the selected type,
`becomes loaded—or charged—with a single instance of the selected object.
`When the user clicks once on the drawing surface, an instance of the object is
`created—dropped, if you will—on the surface at the mouse-up point. Charged
`cursor doesn’t work too well for tools, but it is nicely suited for graphic objects.
`PowerPoint, for example, uses it extensively. The user selects a rectangle from
`the graphics palette and the cursor then becomes a modal rectangle tool
`
`CiM Ex. 1027 Page 9
`
`

`
`P.-iar IV: THE INTERACTION
`
`CHAPTER 18: DRAG-AND-DROP
`
`251
`
`
`
`not-currently-
`___ _
`hutuscsg
`
`visible location in [I]:
`
`same document. I call such scrolling
`
`-
`Autoscroll is a very im
`Poffaflt adjunct to drag-and-dro
`03¢, you will likely have to im I
`1;
`Possibly be scrolled offscreenPthl:;:J;r::r:ti::1ruil-Zhcrcvcr thlldmp target can
`’
`
`»
`lr1TJesigu_ti.-A‘.
`.
`_
`-
`I
`1‘,
`.3;Zfi,_or'°n'-ab1e-'drag"?nd*15l‘0p target -must auu_o_
` ii. —-i..._._._._ _.____ __
`
`-
`p. Where you implement
`
`s autoscro .
`
`“L
`
`In early implementations autoscro .
`,
`,
`3
`llin
`k d '
`.
`apphcmows window. This had two fataffljig eh if You ‘dragged outside the
`was maximized, how could you get the cm-so . t ouCgh.:ii-st, if the application
`you want to drag the object to an th
`ference between that and the desii: t(:ral:.)ll:())E(lZ-:(I)]l]l,?how can the app ten the dif.
`
`’°“‘5' Ct eapp? And second if
`
`Microsoft develo
`
`-
`'
`ped a very intelli ent sol
`
`t’
`
`‘
`
`.
`
`f
`
`I’
`
`ll’
`
`‘
`
`begin autoscrolling just inside thf appiic;lltil;O.1l1=,u.:$,S pliable"; Bgslcauyi th°Y
`the border As the dm
`er instea o just outside
`g cursor approaches the b d
`-
`d0w__bm is sti" ins.
`.
`Or ers of the scrollable win-
`ide it—a scroll in rh
`.
`.
`'
`‘
`the drag cursor comes within three or fDu:;;;l1lill.i‘:.Qt"3i~, of Eh: drag is initiated. If
`window, Word be '
`.
`“'"¢t¢1'5 0 t c bottom ofthe text
`81115 to scroll the window’
`50' mmcs C9‘-“IUY close to the top edge oftlfecfntenls upward‘ Ifthe drag WT’
`Unfortunate]
`’
`-
`.
`ext wmdowv Word SCFOIIS down,
`h
`y‘ Word 5 lmpkmcntfltlon doesn't take into acco
`t h
`t e microprocessor and th
`'
`un t 6 Power °f
`C action occurs t
`,
`‘J
`slow) 486/66. Be