52 command history
`
`~,orkspace ! Currerlt ~rector~/~
`
`P1ATLAB’s command history, shown in the lower left
`
`As the user performs actions, keep a visible record
`
`of what was done, to what, and when.
`
`Users perform long and complex sequences of ac-
`
`tions, either with a GUI or a command line. Host
`users are fairly experienced, or if not, they at least
`want an efficient interface that’s supportive of
`long-term and recurring work. Graphical editors
`and programming environments usually are good
`candidates for this pattern.
`
`¯ Repeat a sequence of operations, originally
`done to one object, on a different object
`
`¯ Keep a log of his actions, for legal or
`security reasons
`
`¯ Convert an interactive series of commands
`into a script or macro (see the Macros
`pattern in this chapter)
`
`Computers are good at keeping an accurate re-
`
`cord of steps taken; people aren’t. Take advantage
`of that.
`
`Sometimes a user needs to remember or review
`what he did in the course of working with the soft-
`ware. For instance, he may want to do any of these
`things:
`
`¯ Repeat an action or command done earlier,
`one he doesn’t remember well
`¯ Recall the order in which some actions were
`done
`
`Keep a running list of the actions taken by the user.
`
`If the interface is driven from a command line, then
`you have it easy--just record everything typed
`there. See Figure 5-22. If you can, keep track of the
`history across sessions, so the user can see what
`was done even a week ago or longer.
`
`156
`
`I)OING THI NC~S
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`If it’s a graphic interface, or a combination of
`
`graphic and command-line interfaces, then things
`
`get a little more complicated. Find a way to ex-
`
`press each action in one consistent, concise way,
`
`usually with words (:though there’s no reason why
`
`it can’t be done visually). Make sure you define
`
`these with the right granularity--if one action is
`
`done en masse to seventeen objects, record it as
`
`one action, not seventeen.
`
`What commands should you record, and which
`ones shouldn’t you? See the Multi-Level Undo pat-
`tern for a thorough discussion of what commands
`
`should "count." Ifa command is undoable, it should
`be recorded in the history.
`
`Finally, display the history to the user. That display
`should be optional in most software, since it will
`almost certainly play a supporting role in the user’s
`work, not a starring role. Lists of commands--
`oldest to newest--tend to work well. If you’d like,
`you could timestamp the history display somehow.
`MATLAB, shown in Figure 5-22, puts a date and
`time into the history whenever the program
`restarts.
`
`swing-shift:/var/log/httpd> history
`pwd
`1 23:55
`2 23:55
`pushd /var/Iog/httpd/
`Is -{ access_log
`3 23:55
`tat[ -1888 access_[og I grep "index"
`4 23:56
`
`taiL -1880 access_log I grep "tndex"l wc
`tail -1888 access_log I grep "Dtagona["l wc
`
`is -[
`cd jtt~weltnet/
`tail -1888 access_Log ] grep "index"l wc
`
`tat[ -2Be access_Log I more
`tail -288 access_log I grep "goggle"
`
`5 23:56
`6 23:57
`? 23:5?
`8 23:5?
`9 23:5?
`18 23:58
`&l 23:58
`12 23:58 cd.,
`tat[ -$088 access_log I grep "goggle"
`13 23:58
`14 23:59
`:15 23:59
`
`tail -1888 access_Log I grep "goog[ebot"
`
`swing-shtft:/var/log/httpd> ~
`
`history
`
`Unix and its many variants use shell programs, such as tcsh and
`
`bash, that keep track of their own command histories in files. The user can call it up
`with the "history" command, as shown here. The history also is accessible through
`
`various command-line constructs, like "]!" (reuse the last command), "!3" (reuse the
`command issued three commands ago), and Control-P, which you can issue
`
`repeatedly to show the previous commands one at a time.
`
`Photoshop’s undo stack, also seen in the Multi-Level Undo pattern, is effectively a
`
`command history. You can use it to undo what you did, but you don’t have to; you can just look at it and
`
`scroll through it, ~eviewing what you did. It uses icons to identify different classes of actions, which is
`
`unusual, but nice to use.
`
`CC) MMAND II I S’I’() RY
`
`157
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`53 macros
`
`Photoshop’s actions list
`
`Macros are single actions composed of other,
`smaller actions. Users can create them by putting
`together sequences of actions.
`
`Users want to repeat Ion9 sequences of actions or
`commands. They might want to loop over lists of
`files, images, database records, or other objects,
`for instance, doing the same things to each object.
`You might already have implemented Multi-Level
`Undo or Command History.
`
`No one wants to perform the same set of repeti-
`tive interactive tasks over, and over, and over again!
`This is exactly what computers are supposed to be
`good at. Chapter ] described a user-behavior pat-
`tern called Streamlined Repetition; macros are
`precisely the kind of mechanism that can support
`that well.
`
`Macros obviously help users work faster. However,
`by reducing the number of commands or gestures
`needed to get something done, they also reduce
`the possibility of finger-slips, oversights, and simi-
`lar mistakes.
`
`You might also recall the concept of "flow," also
`discussed in Chapter 1. When a user can compress
`
`a long sequence of actions down to a single com-
`mand or keyboard shortcut, the experience of flow
`is enhanced--the user can accomplish more with
`less effort and time, and she can keep her larger
`goals in sight without getting bogged down in
`details.
`
`Provide a way for the user to "record" a sequence
`of actions and easily "play them back" at any time.
`The playback should be as easy as giving a single
`command, pressing a single button, or dragging
`and dropping an object.
`
`The user should be able to give the macro a name
`of her choice. Let her review the action sequence
`somehow, so she can check her work or revisit a
`forgotten sequence to see what it did (as in the
`Command History pattern). Make it possible for
`one macro to refer to another so they can build on
`each other.
`
`Users will want to save macros from one day to the
`
`next, so make sure they’re persistent--save them
`
`to files or a database. Present them in a search-
`able, sortable, and even categorizable list, depend-
`
`ing on your users’ needs.
`
`DOING TItIIX G S
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`The macro itself could be played back literally, to
`keep things simple; or, if it acts upon an object that
`can change from one invocation to another, you
`could allow the sequence to be parameterized
`(e.g., use a placeholder or variable instead of a lit-
`eral object). Macros also should be able to act on
`many things at once.
`
`How the names of the macros (or the controls
`that launch them) are presented depends heavily
`
`upon the nature of the application, but consider
`
`displaying them with built-in actions rather than
`making them second-class citizens.
`
`The ability to record these sequences, plus the fa-
`cility for macros to build on one another, create
`the potential for a user to invent entirely new lin-
`guistic or visual 9rammar--a grammar that is finely
`
`tuned to their own environment and work habits.
`This is a very powerful capability. In reality, it’s pro-
`gramming; but if your users don’t think of them-
`selves as programmers, don’t call it that or you’ll
`scare them off. ("1 don’t know how to program
`anything; I must not be able to do this.")
`
`0.882
`0 867
`0.948
`
`0.522
`0345
`0.257
`
`0.736
`0.549
`0.485
`
`,~
`
`~hortcut key:
`Cel+~"
`E~c~ion:
`
`~or~ macro ~n:
`IThis Workbook
`
`3 i
`4
`5
`
`7
`
`10’
`11
`
`13
`14~
`15
`
`1’~ ~ i N ~Sheet 1 Z Sheet2 X’ Sheet3
`Ready
`
`Microsoft Excel allows macros to be
`recorded, named, stored along with the document, and
`
`even assigned to a keyboard shortcut. The user also can
`
`choose to run it from a button on the toolbar, or an
`
`ActiveX control in the document itself (which means you
`
`can use them as callbacks for buttons or text fields).
`
`I(~ ..... "
`
`"_J I’~
`
`"_i
`
`These Excel macros are written
`
`in Visual Basic, and the user can hand-edit them
`if desired. This is when it becomes programming.
`
`Because Visual Basic provides access to so much
`
`general-purpose functionality--most of it not
`
`directly related to spreadsheet operations--
`macros can be a serious security risk for Office
`
`applications. By sharply constraining the
`
`functionality available to macros and limiting the
`number of ways users can run macros (e.g.,
`
`clicking on toolbar buttons), you can trade power
`
`for safety.
`
`M A C R 0 S
`
`] 59
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`SHOWING COMPLEX DATA:
`
`’II{EI(,%, ’IABLI!S, AN[)
`
`INFORMA’FION GI~.APHICS
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`Information graphics including maps, tables, and graphs communi-
`cate knowledge visually rather than verbally. When clone well, they let
`people use their eyes and minds to draw their own conclusions; they
`show, rather than tell.
`These graphics are my favorite kinds of interfaces. However, poor tools or inadequate design can sharply
`limit what you can do with them, and many information-rich interfaces just don’t quite work as well as
`they could.
`
`The patterns in this chapter will help you make the best of the tools you have, and introduce you to some
`useful and interesting innovations in visualization design. The ideas described in this introduction can help
`you sort out which design aspects are most important to you in a given interface.
`
`"Information graphics" simply means data presented visually, with the goal of imparting knowledge to the
`user. I’m including tables and tree views in that description because they are inherently visual, even though
`they’re constructed primarily from text instead .of lines and polygons. Other familiar static information
`graphics includes maps, flowcharts, bar plots, and diagrams of real-world objects.
`
`But we’re dealing with computers, not paper. You can make almost any good static design better with
`interactivity. Interactive tools let the user hide and show information as she needs it, and they put the user
`in the "driver’s seat" as she chooses how to view and explore it.
`
`Even the mere act of manipulating and rearranging the data in an interactive graphic has value--the user
`becomes a participant in the discovery process, not just a passive observer. This can be invaluable. The
`user may not produce the world’s best-designed plot or table, but the process of manipulating that plot
`or table puts her face-to-face with aspects of data that she may never have noticed on paper.
`
`Ultimately, the user’s goal in using information graphics is to learn something. But the designer needs to
`understand what the user needs to learn. The user might look for something very specific, like a particular
`street on a map, in which case she needs to be able to find it--say by searching directly, or by filtering out
`extraneous information. She needs to get a "big picture" only to the extent necessary to reach that spe-
`cific data point. The abilities to search, filter, and zero in on details are critical.
`
`On the other hand, she might try to learn something less concrete. She might look at a map to grasp the
`layout of a city, rather than to find a specific address. Or she may be a scientist visualizing a biochemical
`process, trying to understand how it works. Now overviews are important; she needs to see how the parts
`interconnect into the whole. She may want to zoom in, zoom back out again, look at the details occasion-
`ally, and compare one view of the data to another.
`
`TIlE BASICS OF INFORMATION GRAPIIlCS 161
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`Good interactive information graphics offer users answers to these questions:
`
`How is this data organized?
`
`What’s related to what?
`
`How can I explore this data?
`
`Can I rearrange this data to see it differently?
`
`Show me only what I need to know.
`
`What are the specific data values?
`
`In these sections, remember that the term "information graphics" is a very big umbrella. It
`covers plots, graphs, maps, tables, trees, timelines, and diagrams of all sorts. The data can
`be huge and multilayered, or small and focused. Many of these techniques apply surpris-
`ingly well to graphic types that you wouldn’t expect.
`
`Before describing the patterns themselves, let’s set the stage by discussing some of the
`questions posed above.
`
`The first thing a user sees in any information visualization is the shape you’ve chosen for the
`
`data. Ideally, the data itself has an inherent structure that suggests this shape to you. Which
`
`of these models fits your data best?
`
`Model
`
`Linear
`
`Tabular
`
`Diagram
`
`Common graphics
`
`0~0~0
`
`List or single-variable plot
`
`Spreadsheet, multi-column list, Sortable Table,
`Multi-Y Plot, or other multi-variable plots
`
`Hierarchical
`
`.<;<:
`
`Tree, Cascaded Lists, Tree Table, Treemap, or
`directed graph
`
`Network (or
`organic)
`
`Geographic (or
`spatial)
`
`Other
`
`Directed graph or flowchart
`
`Map or schematic
`
`Plots of various sorts, such as parallel coordinate
`plots, or Treemaps
`
`Try these models against the data you try to show. If two or more might fit, consider which
`ones play up which aspects of your data. If your data could be both geographic and tabular,
`for instance, showing it as only a table may obscure its geographic nature--a viewer may
`miss interesting features or relationships in the data if you do not show it as a map too.
`
`162 SIIOWING COMPLEX 1)ATA
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`The organizational model you choose tells the user a lot about the shape of the data. Part
`
`of this message operates at a subconscious level; people recognize trees, tables, and maps,
`
`and they immediately jump to conclusions about the underlying data before they even start
`
`to think consciously about it. But it’s not just the shape that does this. The look of the indi-
`
`vidual data elements also works at a subconscious level in the user’s mind: things that look
`
`alike must be associated with one another.
`
`If you’ve read Chapter 4, that should sound familiar--you already know about the Gestalt
`
`principles. (If you jumped ahead in the book, this might be a good time to go back and read
`
`the introduction to Chapter 4.) Most of those principles, especially similarity and continuity,
`
`will come into play here, too. I’ll tell you a little more about how they seem to work.
`
`Certain visual features operate "preattentively:" they convey information before the viewer
`
`pays conscious attention. Look at Figure 6-] and find the blue objects.
`
`00000
`
`,~:’: ::~.i / Find the blue objects
`
`I’m guessing that you can do that pretty quickly. Now look at Figure 6-2 and do the same.
`
`0000000000
`
`0000000000
`0000000
`
`0000000000
`
`You did that pretty quickly too, right? In fact, it doesn’t matter how many red objects there
`are; the amount of time it takes you to find the blue ones is constant! You might think it
`should be linear with the total number of objects--order-N time, in algorithmic terms--but
`it’s not. Color operates at a primitive cognitive level. Your visual system does the hard work
`for you, and it seems to work in a "massively parallel" fashion.
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`On the other hand, visually monotonous text forces you to read the values and think about
`them. Figure 6-:3 shows exactly the same problem with numbers instead of colors. How fast
`can you find the numbers that are greater than
`
`0.103
`
`0.176
`
`0.387
`
`0.300
`
`0.379
`
`0.276
`
`0.179
`
`0.321
`
`0.192
`
`0.250
`
`0.333
`
`0.384
`
`0.564
`
`0.587
`
`0.857
`
`1.064
`
`0,698
`
`0.621
`
`0,232
`
`0.316
`
`0.421
`
`0.309
`
`0.654
`
`0.729
`
`0.228
`
`0.529
`
`0.832
`
`0.935
`
`0.452
`
`0.426
`
`0.266
`
`0.750
`
`1.056
`
`0.936
`
`0.911
`
`0.820
`
`0.723
`
`1.201
`
`0.935
`
`0.819
`
`0.225
`
`0.326
`
`0.643
`
`0.337
`
`0.721
`
`0.837
`
`0.682
`
`0.987
`
`0.984
`
`0.849
`
`0.187
`
`0.586
`
`0.529
`
`0.340
`
`0.829
`
`0.835
`
`0.873
`
`0.945
`
`1.103
`
`0.710
`
`0.153
`
`0.485
`
`0.560
`
`0.428
`
`0.628
`
`0.335
`
`0.956
`
`0.879
`
`0.699
`
`0.424
`
`FtGURE ~3,3 , Find the values greater than one
`
`When dealing with text like this, your "search time" really is linear with the number of items.
`What if we still used text, but made the target numbers physically larger than the others, as
`in Figure 6-4?
`
`0.103
`
`0.176
`
`0.387
`
`0.300
`
`0.379
`
`0.276 0.179
`
`0.321
`
`0.192
`
`0.250
`
`0.333
`
`0.384
`
`0.564
`
`0.587
`
`0.857
`
`1.064 0.698
`
`0.621
`
`0.232
`
`0.316
`
`0,421
`
`0.309
`
`0.654
`
`0.729
`
`0.228
`
`0.529
`
`0.832
`
`0.935
`
`0.452
`
`0.426
`
`0.266
`
`0,750
`
`1.056 0.936
`
`0.911
`
`0.820
`
`0.723
`
`1,201
`
`0.938
`
`0.819
`
`0.225
`
`0.326
`
`0,643
`
`0.337
`
`0.721
`
`0.837
`
`0.682
`
`0.987
`
`0.984
`
`0.849
`
`0.187
`
`0.586
`
`0.529
`
`0.340
`
`0.829
`
`0.835
`
`0.873
`
`0.945
`
`1.103 0.710
`
`0.1 $3
`
`0.485
`
`0.560
`
`0.428
`
`0.628
`
`0.338
`
`0.956
`
`0.879
`
`0.699
`
`0.424
`
`FIGURE 6-4 ,’ Again
`
`Now we’re back to constant time again. Size is, in fact, another preattentive variable. The
`fact that the larger numbers protrude into their right margins also helps you find them--
`alignment is yet another preattentive variable.
`
`Figure 6-5 shows many known preattentive variables.
`
`J
`
`164
`
`SHOWING COMPI. EX DATA
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`Color hue
`
`Position and alignment
`
`Color brightness
`
`Orientation
`
`Color saturation
`
`Size
`
`abcdef
`abcdef
`abcdef
`
`abcdef abcdef abcdef
`abcdef abcdef abcdef
`abcdef ebcdef abcdef
`
`abcdef abcdef
`abcdef
`abcdef
`abcdef abcdef
`abcdef
`abcdef
`abcdef abcdef abcdef abcdef
`
`Texture
`
`Shape
`
`FIGURE 6-5 / Eight preattentive variables
`
`This concept has profound implications for text-based information graphics, like the table
`of numbers in Figure 6-:5. If you want some data points to stand out from the others, you
`have to make them look different by varying their color, size, or some other preattentive
`variable. More generally, you can use these variables to differentiate classes or dimensions
`of data on any kind of information graphic. This is sometimes called "encoding."
`
`When you have to plot a multidimensional data set, you can use several different visual
`variables to encode a!l those dimensions in a single static display. Consider the scatter plot
`shown in Figure 6-6. Position is used along the X and Y axes; color hue encodes a third vari-
`able. The shape of the scatter markers could encode yet a fourth variable, but in this case,
`
`THE BASICS OF INFORMATION G:RAPHICS
`
`165
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`shape is redundant with color hue. The redundant encoding helps a user visually separate
`the three data groups.
`
`[] []
`
`45
`
`40
`
`~5
`
`1508 2000 2500 3000 3600 4000 4500 5000
`Car Weight in Pounds
`
`Encoding three variables in a scatter plot
`
`Encoding via preattentive factors relates to a general graphic design concept called "layer-
`ing." When you look at well-designed graphics of any sort, you perceive different classes of
`information on the page. Preattentive factors like color cause some of them to "pop out" of
`the page, and similarity causes you to see them as connected to one another, as if each
`were on a transparent layer over the base graphic. It’s an extremely effective way of seg-
`menting data--each layer is simpler than the whole graphic, and the viewer can study each
`in turn, but relationships among the whole are preserved and emphasized.
`
`NAVIGATION ANI::) BROVVStNG: HOW CAN I EXPLORE ] HIS DATA?
`
`A user’s first investigation of an interactive data graphic may be browsing--just looking
`around to see what’s there. He also may navigate through it to find some specific thing he’s
`seeking. Filtering and searching can serve that purpose too, but navigation through the
`"virtual space" of a dataset often is better. Spatial Memory (Chapter 1) kicks in, and the user
`can see points of interest in context with the rest of the data.
`
`A famous mantra in the information visualization field is: "focus plus context." A good visu-
`alization should permit a user to focus on a point of interest, while simultaneously showing
`enough material around that point of interest to give the user a sense of where it is in the
`big picture.
`
`Here are some common techniques for navigation and browsing:
`
`If the data display won’t fit onscreen at once, you could put it in a scrolled window,
`giving the user easy and familiar access to the offscreen portions. Scrollbars are
`
`166 5HO\VlNG COMPI. EX DATA
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`familiar to almost everyone, and are easy to use. However, some displays are too
`big, or their size is indeterminate (thus making scrollbars inaccurate), or they have
`data beyond the visible window that you need to retrieve or recalculate (thus making
`scrollbars too slow to respond). Instead of using scrollbars in those cases, try setting
`up buttons that the user has to click to retrieve the next screenful of data; think about
`how MapQuest or MapBlast works. Other applications do panning instead, in which
`the cursor "grabs" the information graphic and drags it until the point of interest is
`found, as in Google Maps.
`
`These techniques are appropriate for different situations, but the basic idea is the
`same: to interactively move the visible part of the graphic. Sometimes Overview
`Plus Detail can help the user stay oriented. A small view of the whole graphic can be
`shown with an indicator rectangle displaying the visible "viewport;" the user might
`pan by dragging that rectangle, in addition to scrollbars or whatever else is used.
`
`Zooming changes the scale of the viewed section, whereas scrolling changes the
`location. When you present a data-dense map or graph, consider offering the user
`the ability to zoom in on points of interest. It means you don’t have to pack every
`single data detail into the full view--if you have lots of labels, or very tiny features
`(especially on maps), it may be impossible anyway. As the user zooms in, those fea-
`tures can emerge when they have enough space.
`
`Host zooms are implemented with a mouse click or button press, and the whole view-
`ing area changes scale at once. But that’s not the only way to zoom. Some applica-
`tions create nonlinear distortions of the information graphic as the user moves the
`mouse pointer over the graphic: whatever is under the pointer zooms, but the stuff
`far away from the pointer doesn’t change scale. See the Local Zooming pattern for
`more information.
`
`Tree views typically let users open and close parent items at will, so they can inspect
`the contents of those items. Some hierarchically structured diagrams and graphs
`also give users the chance to open and close parts of the diagram "in place," without
`having to open a new window or go to a new screen. With these devices, the user
`can explore containment or parent/child relationships easily, without leaving that
`window. The Cascading Lists pattern describes another effective way to explore a
`hierarchy; it works entirely through single-click opening and closing of items.
`
`Some information graphics just present a "top level" of information. A user might
`click or double-click on a map to see information about the city she just clicked on,
`or she might click on key points in a diagram to see subdiagrams. This "drilling down"
`might reuse the same window, use a separate panel on the same window, or bring
`up a new one (see Chapter 2 for a discussion of window mechanics). This technique
`resembles opening and closing points of interest, except that the viewing occurs
`separately from the graphic, and is not integrated into it.
`
`Till-: BASICS ()F INY(IIIM.,\TI()N (~I{.\PIIICS
`
`167
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`If you also provide a search facility for an interactive information graphic, consider linking
`the search results to whatever technique listed previously that you use. In other words, when
`a user searches for the city of Sydney on a map, show the map zooming and/or panning to
`that point. Then the search user gets some of the benefits of context and spatial memory.
`
`Sometimes just rearranging an information graphic can reveal unexpected relationships.
`Look at the following graphic, taken from the National Cancer Institute’s online mortality
`charts (Figure 6-7). It shows the number of
`deaths from lung cancer in the state of Texas.
`The major metropolitan regions in Texas are ar-
`ranged alphabetically--not an unreasonable de-
`fault order if you look up specific cities, but as
`presented, the data doesn’t prompt many inter-
`estin9 questions. It’s not clear why Abilene, Al-
`ice, Amarillo, and Austin all seem to have similar
`numbers, for instance; it may just be chance.
`
`¯
`
`From http.//cance~gov/atlasplus/,
`
`sorted alphabetically
`
`mllml
`
`The same chart, sorted numerically
`
`But this web application lets you reorder the data
`into numerically descending order, as in Figure
`6-8. Suddenly the graph becomes much more in-
`teresting. Galveston is ranked first--why is that,
`when its neighbor, Houston, is further down the
`scale? What’s special about Galveston? (Okay,
`you needed to know something about Texas ge-
`ography to ask these questions, but you get my
`point.) Likewise, why the difference between
`ne!ghbors Dallas and Fort Worth? And appar-
`ently, the Mexico-bordering southern cities of El
`Paso, Brownsville, and Laredo have less lung can-
`cer than the rest of Texas; why might that be?
`
`People who can interact with data graphics this
`way have more opportunities to learn from the
`graphic. Sorting and rearranging puts different
`data points next to each other, thus letting users
`make different kinds of comparisons--it’s far
`easier to compare neighbors than widely scat-
`tered points. And users tend to zero in on the
`extreme ends of scales, as I did in this example.
`
`How else can you apply this concept? The pat-
`tern Sortable Table talks about one obvious way:
`when you have a many-columned table, users
`might want to sort the rows according to their
`choice of column. This pattern is pretty common.
`
`168 SIIOWING COMPI, I!X DATA
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`(Many table implementations also permit rearrangement of the columns themselves, by
`dragging.) Trees can have their children reordered. Diagrams and connected graphs might
`allow spatial repositionin9 of their elements, while retaining their connectivity. Use your
`imagination!
`
`Consider these methods of sorting and rearranging:
`
`Alphabetically
`
`Numerically
`
`By date or time
`
`By physical location
`
`By category or tag
`
`Popularity--heavily versus lightly used
`
`User-designed arrangement
`
`Completely random (you never know what you might see)
`
`For a subtle example, look at Figure 6-9. Bar charts that show multiple data values on each
`bar ("stacked" bar charts) might also be amenable to rearranging--the bar segments near-
`est the baseline are the easiest to evaluate and compare, so you might want to let users
`determine which variable is next to the baseline.
`
`The light blue variable in this example might be the same height from bar to bar. Does it
`vary, and how? Which light blue bars are the tallest? You really can’t tell until you move that
`data series to the baseline--that transformation lines up the bases of all blue rectangles.
`Now a visual comparison is easy: light-blue bars 6 and 12 are the tallest, and the variation
`seems loosely correlated to the overall bar heights.
`
`lllnnull
`
`Rearrangement of a stacked bar chart
`
`Sometimes you don’t want to see an entire dataset at once. You might start with the whole
`thing, and then narrow it down to what you need--filtering. Or, you might build up a subset
`of the data via searching or querying. Most users won’t even distinguish between filtering
`and querying (though there’s a big difference from, say, a database’s point of view). What-
`ever term you use, the user’s intent is the same: to zero in on whatever part of the data is
`of interest, and get rid of the rest.
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`The simplest filtering and querying techniques offer users a choice of which aspects of the
`
`data to view. Checkboxes and other one-click controls turn parts of the interactive graphic
`
`on and off. A table might show some columns and not others, per the user’s choice; a map
`
`might show only the points of interest (e.g., restaurants) that the user selects. The Dynam-
`
`ic Ql~eries pattern, which can offer rich interaction, is a logical extension of simple filter
`
`controls like these.
`
`Sometimes simply highlighting a subset of the data, rather than hiding or removing the
`
`rest, is sufficient. That way a user can see that subset in context with the rest of the data.
`
`Interactively, you can highlight with simple controls, as described earlier. The Data Brushing
`
`pattern describes a variation of data highlighting; data brushing highlights the same data
`
`in several data graphics at once.
`
`Look at Figure 6-]0. This interactive ski-trail map can show four categories of trails, coded
`
`by symbol, plus other features like ski lifts and base lodges. When everything is "turned on"
`
`at once, it’s so crowded that it’s hard to read anything! But users can click on the trail sym-
`
`bols, as shown, to turn the data "layers" on and off. The first screenshot shows no high-
`
`lighted trails; the second switches on the trails rated black-diamond with a single click.
`
`1:~ From httta.~//ww~zsundayrive~com/trailmap.htm
`
`Searching mechanisms vary heavily from one type of graphic to another. A table or tree
`should permit textual searches, of course; a map should offer searches on addresses and
`other physical locations; numeric charts and plots might let users search for specific data
`values or ranges of values. What are your users interested in searching on?
`
`When the search is done, and results obtained, you might set up the interface to see the
`results in context, on the graphic--you could scroll the table or map so that the searched-
`for item is in the middle of the viewport, for instance. Seeing the results in context with the
`rest of the data helps the user understand the results better. The Jump to Item pattern is a
`
`common way to search and scroll in one step.
`
`170
`
`SHOWING COMPI. EX [),t~’l’A
`
`MemoryWeb Ex. 2033
`Apple v. MemoryWeb – IPR2022-00031
`
`

`

`The best filtering and querying interfaces are:
`
`They respond as quickly as possible to the user’s searching and filtering. Implementing
`this admittedly isn’t easy for web applications and other interfaces that need to get
`data from across a network.
`
`They let a user refine the search, query, or filter until she gets the desired results.
`They also might combine these operations: a user might do a search, get a screenful
`of results, and then filter those results down to what she wants.
`
`They show results in context with surrounding data to make it easier for a user
`to understand where they are in a data space. This is also true for other kinds of
`searches; the best web search engines show a keyword embedded in a sentence, or
`an image embedded in its web page.
`
`Several common techniques help a viewer get specific values out of an information graph-
`ic. Know your audience--if they’re only interested in getting a qualitative sense of the data,
`then there’s no need for you to spend large amounts of time or pixels labeling every little
`thing, But some actual numbers or text usually are necessary.
`
`Since these techniques all involve text, don’t forget the graphic design principles that make
`text look good: readable fonts, appropriate font size (not too big, not too small), proper
`visual separation between unrelated text items, alignment of related items, no heavy-bor-
`dered boxes, and no unnecessary obscuring of data.
`
`Many information graphics put labels directly on the graphic, such as town names
`on a map. Labels also can identify the values of symbols on a scatter p