_.
`
`~j~ _n ~ ~_..~.t --
`NeHt
`PrelJlous
`Initial
`L6St
`Help
`
`-...--.- --- --
`
`- -
`
`_ _ _ _
`
`_ __ _ _ _
`
`--
`
`_
`
`_
`
`Window
`North
`North-East
`East
`south-East
`South
`South-West
`West
`North-West
`Suspend
`C
`NC
`N
`___~~~/--------- -- -- ---'wr=-r=---- - -
`IHeip
`t7~/
`
`-- -.- ---------- --~
`
`189
`
`The neighborhood of a document is not determined unless the user judges it to be
`
`relevant or selects the Expand option from the content menu associated with the document
`
`text window.
`
`If either of these choices is made, the node is redrawn farther away to
`
`provide space for its neighborhood. The user selects the Expand option and the map is
`
`redrawn showing the neighborhood of the selected document (figure 6.30).
`
`N~ighborhood Map
`
`'i
`
`_
`
`_
`
`_
`
`_
`
`8~ ~ /8
`0- c ~ d.3. 1;.3- C -C~
`V- ,~'P ~~
`/
`cEJ
`
`-
`
`Figure 6.30: Neighborhood Map with expanded document neigh(cid:173)
`borhood.
`
`The user is not restricted to selecting documents that appear on the neighborhood
`
`map. He may go back to the current document of interest and select any list of documents
`
`associated with it. These are the reference list, the citation list, nearest neighbor list, the
`
`author list, or. the journal issue list. Figure 6.31 shows the neighborhood map after the
`
`user has examined the recommended node and has decided to examine the other node eon-
`
`neeted by the nearest neighbor link, marked "N"
`
`001
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`190
`
`The user is not restricted to just moving from document to document, but may also
`
`examine concepts.
`
`In figure 6.32, the user has decided that the term "multidimensional" is
`
`possibly a fruitful path to follow. He selects the 'I'e r.m option from the content menu uno
`
`then the term of interest. This tells the system that he is selecting this as a node to examine
`
`and not that it is an interesting term. The node icon with the term's number appears on the
`
`neighborhood map and is shaded to indicate that it has been visited. The link is marked
`
`with the number of occurrences of that term in the document. The display for the term
`
`would appear in its own window, figure 6.20. In this case, there are as of yet no connec(cid:173)
`
`tions to any other concepts. The user could add a synonymous phrase like "a-dimensional"
`
`at this point. The content menu shows the different possibilities for domain knowledge
`
`entry. The Select option allows the user to follow a concept in the same way that the
`
`T e rm option was used in a document display content menu. The Documents option tells
`
`the system that he would like to see the documents that are associated with this term, where
`
`the Expand option would signal to expand the neighborhood using concept links.
`
`002
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`191
`
`North
`North-East
`East
`South-East
`South
`South -West
`West
`North-West
`Suspend
`Help
`
`NeHt
`Pre nln us
`Initial
`Last
`IJlelp
`
`8/
`" /
`8C-d3163 C-8
`
`NC
`
`C
`
`-
`
`-
`
`- ---- ---
`
`- -- -
`
`- 1- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`Multidimensional Binary Search Trees Used for e at Entry
`Rssociatiue Searching
`Suspend
`Help
`
`Bently, J.l.
`
`2722
`This paper deuelops the multidimensional binary
`search tree (or Ie-d tree, where k is the
`dimensionality of the search space) as a data
`structure for storage of information to be
`retrleue d by assuclattue searches. The Ic-d tree
`
`erm
`[Hpand
`N. Neighbors
`References
`Citations
`Broader
`Narrower
`Reiloted
`SYlilonym
`Phrase
`Enltry Ole
`canc et
`Reneuant
`Done
`Help
`
`Figure 6.31: User views document 2722 (text is incomplete).
`
`003
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`192
`
`, ' .
`
`. .
`
`I
`
`.,
`
`'
`
`. . . . . ·Nei 9 h'b'orJH)(fdl.: .!I<·i 6i( . :. . .' ....-:- :··> ,.'.:::~t ur:~:· ·
`
`'~
`. . . • .
`
`Co nt en t
`NeHt
`Previo u s
`Initial
`Lost
`Help
`
`North
`North-East
`Eost
`South-East
`South
`South-West
`West
`North-West
`Suspend
`Help
`
`t23000
`
`~-,
`
`2"8- c
`
`[Hpand
`Oroodar
`Narrower
`Related
`Synonym
`Phrase
`Entry 0 Ie:
`Cancel
`RelalJant
`Done
`Help
`
`Multidimensional Binary Search Trees Used for
`assoctatlue Searching
`
`Bently, J.l.
`
`2722
`This paper deuelops the
`search tree (or Ie -d tree, where Ie is the
`dimensionality of the search space) as a data
`structure for storage of inform ation to be
`re trieueu by associative searches. The Ie-d tree
`
`binary
`
`Figure 6.32: User selects a term to examine from document 2722.
`
`004
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`:;: :' ~'''l": ~
`
`.
`
`: "~:
`
`.
`
`'
`
`:
`
`II
`
`193
`
`..
`
`Win dow .
`lreHt Entry
`Suspend
`Help
`
`;~ i-co-n t"e nt.. .'.
`Con c e p t
`.
`Name: multidimensional
`Select
`Stem: muttldlmen
`[Hpond
`Term# 30742
`Synonym
`Occurrences: 8
`Related
`*** Nearest Neighbors ***
`Brooder
`Narrower
`Component
`Port Of
`Phrase
`Entry Ole
`Cancel
`Documents
`Heleuont
`Done
`Help
`
`*** Synonym ***
`
`*** Related ***
`
`*** Broader ***
`
`Figure 6.33: Display for the concept mul t idimens ional.
`
`---
`
`-
`
`-- - --
`
`-
`
`-
`
`Upon selecting the Documents option (figure 6.34), the browse maps are both
`
`-
`-
`expanded and a document list similar to the search results list appears (figure 6.21) with the
`
`titles of the documents that have the term "multidimensional" in them.
`
`4. Operations on Generalized Arrays with t
`Compiler
`
`Show Rei
`
`5. An Efficient Procedure for the Generatio
`Closed Subsets
`
`Show Rei
`
`6. An Efficient Composite Formula for
`Multidimensional Quadrature
`
`Top
`Scroll-Up
`Scroll-Down
`Bottom
`Suspend
`Hel
`
`Show Rei
`
`7. Use of Tree Structures for Processing Files
`
`Figure 6.34: User selects Documents option.
`
`005
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`194
`
`The user selects the seventh document as an interesting one since it has the term "tree" in it.
`
`In this document, he sees the teffi1 "trie,' (which is a particular kind of tree data structure
`
`IHorowitz 84]) and decides to find out what documents are connected to it. In the collec(cid:173)
`
`tion, there are only three, but one of the other two is interesting.
`
`Figure 6.35 shows the context map that results from the session as it has developed
`
`so far. The context map is smaller scale version of the neighborhood map. The links are
`
`still marked as to type, and the nodes marked as to whether they have been recommended,
`
`visited, or judged relevant. No node identification information is given since that is
`
`available from the neighborhood map. Both maps remain in correspondence with each
`
`other, centered on the same node. When the user selects a node in the neighborhood map,
`
`the context map is updated at the same time, and vice versa. An example of a context map
`
`is figure 6.35. From this map, the user can go to any node by scrolling it into view and
`
`selecting it (placing the cursor on it and clicking with the left mouse button). This will
`
`cause the neighborhood map to be redrawn with the new node of interest in the center with
`
`its immediate neighborhood. For example, in this session the user might decide that he
`
`wants to continue investigating the area around the second document he determined to be
`
`relevant (marked with an "a").
`
`One problem that may arise in this session occurs if the user decides to expand in
`
`the direction of the node marked "b." The neighborhood of this node can be drawn as
`
`shown in figure 6.36. If the user would take the recommended node from the expansion of
`
`the marked node, it would overwrite the node representing the term "trie." The interface
`
`manager attempts to avoid this by expanding in a direction that appears to be clear, such as
`
`the location marked "c."
`
`006
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`I
`l. '"
`
`.
`
`: .
`
`'
`; .;
`
`.
`
`.
`
`.
`
`Con t e H t Map
`
`I
`I
`
`195
`
`•
`
`@~ 0 ~O
`0 1 "' I'
`C5
`
`3
`
`1
`
`1
`
`2
`
`1
`
`o~ @
`C NC
`"'
`
`. . .--
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-1
`
`~
`
`1
`
`1'o
`
`(c)
`
`Figure 6.35: Context Map after examining document #2846
`(menus not shown, but are the same as those with the neighborhood map).
`
`If this location was taken, as in figure 6.36, the system would draw the node in
`
`another region of the map and place a connector icon on a link leading to it (figure 6.37).
`
`Selecting the connector would put the map back in the region where the node originally was
`
`drawn.
`
`007
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`196
`
`o~ ~
`C NC
`"
`
`N NC C
`
`,0
`C" -o,
`[j
`
`C
`
`Figure 6.36: Context Map showing crowded region around node
`"A," and user desires to expand node "B."
`
`008
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`197
`
`-- --
`
`-- -
`
`- -
`
`- - - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- - -f-- -
`
`-
`
`-
`
`-
`
`.----~
`
`Figure 6.37: Use of connector to expand node "8." Connectors are
`placed on the links to facilitate the movement between nodes.
`
`The connectors are useful primarily on the neighborhood map, where the context is
`
`limited to the immediate neighborhood. When the user selects connector Cl (figure 6.37),
`
`the node marked "A" is placed in the center of the map. When connector C2 is selected the
`
`node marked "B" is centered. In this way the user can follow the path he has selected. The
`
`connectors can be useful in the context map too. If it is very dense in one region the new
`
`expanded node can be very far away from the node it was expanded from.
`
`6.4
`
`Possible Behavioral Changes
`
`There are other behaviors that could be observed by changing the control expert and
`
`by significantly enhancing the UMB. Currently, theUMB makes only a simple initial
`
`009
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`198
`
`assessment of the user, and this assessment remains fixed throughout the session. If the
`
`user model had confidence factors associated with it assessments, and if the UMB moni(cid:173)
`
`tored the activity of the user, it could continually adjust the model to reflect how the user's
`
`activity corresponded to the initial assessment. If this fell below a threshold the CE could
`
`recognize this fact and move the system back to a state where the UMB could again ques(cid:173)
`
`tion the user and change the model. This would simply require for the CE, the addition of
`
`one transition (rule) that recognizes the UMB's confidence in the model falling below the
`
`threshold and moves the system back to the $CU state and the addition of the UMB to the
`
`priority lists of the $GIN, $DNC, and $ER states.
`
`6.5 Summary
`
`In this chapter, scenarios demonstrating the operation of I3R have been presented.
`
`These scenarios show how I3R provides different facilities to different kinds of users.
`
`They also show how the user can get direct access to the information in the con(cid:173)
`
`cept/document knowledge base by browsing. Indications have been made as to how the
`
`behavior of the system could be altered by the addition of transitions to the control expert.
`
`010
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`CHAPTER 7
`
`CONCLUSION
`
`7.1
`
`Summary
`
`This thesis has presented an information retrieval system that embodies the concept
`
`of an intelligent interface. This intelligent interfac e is implemented by an architecture that
`
`has a number of innovative features that support facilities that help the end-user overcom e
`
`the difficulties of previous IR systems.
`
`The first major feature is the implementation of the major functions of the system as
`
`individual rule-based systems, called experts. These individual rule systems operate in(cid:173)
`
`depende-ntly, paS-ling--TIle results of theIr actIvIty on a short ferm memory. lhisprom6te s ·
`
`the clean separation of functions and allows new major functions to be added with minor
`
`impact on the other experts in the system. The implementation of the experts using rules
`
`allows the incremental development of each expert, so that minor changes can be made to
`
`the expert without grossly affecting its current operation. The idea of multiple independent
`
`cooperating experts communicating using a blackboard [Erman 80] as an architecture for
`
`}3R was developed from a system analysis point of view [Croft 85]. A similar architecture
`
`was independently specified by Belkin [1983, 1984]. A later system, CODER, has also
`
`adopted this basic architecture, but uses two blackboards with separate knowledge sources,
`
`one for documents analysis and the other for retrieval [France 861.
`
`The second innovative feature is the coordination of the operation of the experts by
`
`a flexible control structure based on an analysis [Brooks 83, 85, 86, Daniels 85, 86] of
`
`actual end-user/search intermediary interactions. This control structure, implemented as a
`
`transition network , moderates the activity of the experts to provide a logical dialogue with
`
`the user , so that he does not become confused by numerous independent demands for
`
`199
`
`011
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`200
`
`information from the experts. The course of the dialogue is determined by the progress that
`
`the user has made in expressing his information need and retrieving documents relevant to
`
`his information need. The measurement of the progress is determined by the use of '
`
`expectations of the number of relevant documents retrieved and the number of searches
`
`performed.
`
`The operation of the system and its adaptation to a particular user is controlled by
`
`stereotypes, which are judgements about the user's domain experience, IR system experi(cid:173)
`
`ence, and interest in either an exhaustive or selective search. These stereotypes determine
`
`the expectations used by the control expert to determine the progress of a search session.
`
`T hey also determine what facilities are available to the user. An expert user has more fa(cid:173)
`
`cilities available to him and can take more initiative to control the course of a session than a
`
`novice user can. This idea has been subsequently used in the IOTA system [Chiaramella
`
`87] as a means to adapt its natural language responses to different kinds of users.
`
`Another important innovation in I3R is the use of user supplied domain knowledge.
`
`This obviates the need for a significant investment of resources to derive a global thesaurus
`
`for system use.
`
`Instead, the effort is spread out, so that the users themselves supply the
`
`relevant domain knowledge for the particular information need.
`
`If global domain
`
`knowledge is available, it can be used. Since the user and global domain knowledge are
`
`represented in the same way, the knowledge supplied by the users can be migrated into the
`
`the globally available domain knowledge.
`
`In this way, as the system is used, its base of
`
`domain knowledge can increase and domain knowledge obtained from experts can be made
`
`available to all the users. An extension of this concept would be to provide facilities to
`
`allow users to explicitly share their domain knowledge models.
`
`I3 R was the first system to propose and implement the use of multiple search
`
`strategies in order to take advantage of the differences in performance of techniques based
`
`on a variety of retrieval models.
`
`In previous systems the concept of adaptable search
`
`012
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`201
`
`strategies was based on the experience of human search intermediaries in manipulating the
`
`form of Boolean queries to achieve the best possible results.
`
`Another major innovative feature of I3R is the incorporation of browsing as a
`
`method for both query formulation and search. In fact, browsing merges these two phases
`
`of information retrieval. In the process of query formulation, browsing can help the user to
`
`find concepts that more accurately represent his information need by providing a path from
`
`the concepts that he already knows to new ones. He can then check their relevance by
`
`examining their use in documents.
`
`In the process of search, the user can examine
`
`documents to determine their relevance, and from a relevant document can follow links to
`
`other documents that are related by citations or by similar content as determined by nearest
`
`neighbor links. The user is not restricted to moving from documents to documents but can
`
`....-
`
`- - -- -----.- m ove to~concepts-or-toJournal issues. As the user browses, and makes judgements a60m--------- --
`
`the relevance of concepts and documents, the system records this information and uses it to
`
`further enhance its model of the user's information need.
`
`The user is assisted during browsing by recommendations made by the Browsing
`
`Expert. These recommendations are based on the structure of the concept/document data
`
`and the information in the request model. This information provides evidence for the
`
`browsing expert so that it can determine what concept or document is most likely to be
`
`useful to the user at his current location in the concept/document information.
`
`Another interesting feature of 13R is the use of graphics to provide the user with vi(cid:173)
`
`sual context to aid him in the browsing process. The maps that the system constructs gives
`
`the user a context that helps him determine where he has been, so that he does not get lost
`
`while browsing in the highly interconnected network of documents and concepts.
`
`Finally, the underlying organization of concept/document knowledge in 13R allows
`
`it to support multiple search strategies, browsing, and the use of different domain knowl(cid:173)
`
`edge models. This organization is significantly more sophisticated than previous ones,
`
`013
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`since it fuses what are normally separate sources of knowledge into a single knowledge
`
`202
`
`base.
`
`7.2
`
`Future Directions
`
`7.2.1
`
`Evaluation of the system
`
`The primary work to be performed on }3R in the future is the evaluation of its
`
`performance. This encompasses not only its effectiveness in retrieving relevant
`
`documents, but also its usability with regard to the extra facilities. In other words, how
`
`well do the extra facilities help the user to get the results he desires. The difficulties of
`
`evaluating a highly interactive system have been discussed in sections 2.2.3 and 6.2.
`
`7.2.2
`
`Extension of the experts
`
`The prototype implementation used basic implementations of many of the experts to
`
`demonstrate the functionality of the overall system. All of the experts can be extended in a
`
`number of ways.
`
`The most important expert to develop further is the user model builder. By
`
`developing the user model to a greater extent, the basis is formed for developing more
`
`sophisticated patterns of interaction with the user. At present, the user model is constructed
`
`solely on the basis of the user's responses to questions posed by the UMB. A more
`
`sophisticated UMB would monitor the activity of the user, as well as asking him questions,
`
`to determine the stereotypes that apply. Modelling the user based on his activity requires
`
`that the VMB have models of different patterns of system usage. These models can be
`
`developed only by studying the patterns of usage by many different users over a long
`
`period. Initial work on the functions and structure of a sophisticated UMB has been the
`
`014
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`203
`
`subject of study by Dainiels [1985, 1986, 1987]. The stereotypes of the users could take
`
`on the structure of those proposed by Rich [1979].
`
`The request model builder can be extended by feeding back to it from the results of
`
`the search the terms that were used to make the decision about what documents to retrieve.
`
`This information can be shown to the user, so that he may understand what contributed to
`
`the selection of documents that were shown to him. Also, in relation to the request model,
`
`the user should be given a way to look at and alter the request directly in the form of a
`
`request model display. Whether this display is alterable and the kinds of information
`
`shown on the display would be controlled by the user stereotypes. For example, an expert
`
`user would definitely be allowed to alter the model and could be shown the probabilistic
`
`weights of the terms. A novice, perhaps, would only be shown the relative importance of
`_.._ -
`_.
`In addition to this the user should have
`
`the terms by their ordering based on their weight.
`
`- _ ._ -----
`
`the ability to invoke a search manually.
`
`It is also important to make the control expert more flexible in its operation. This is
`
`also based on the extension of the user model builder. With a better user model, the system
`
`can act more intelligently about the course of the retrieval session.
`
`7.2.3
`
`Addition of an Explainer
`
`One of the major experts defined in the requirements specified in chapter three was
`
`an explainer. The need for this expert as well as other considerations lead to the selection
`
`of rules as the basis for the implementation of the experts as rule-based systems, so that
`
`their reasoning for their actions could be made available to the user.. Furthermore, this
`
`expe11 is particularly important in assisting novice users of the system. Because of these
`
`considerations, an intelligent explainer is an important extension to the current capabilities
`of 13R.
`
`015
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`204
`
`To accomplish this, the previously mentioned extensions to the user model builder
`
`must be made. The explainer needs for its basis a good model of the user representing not
`
`only who the user is, but how he interacts with the system. For example, a user may
`
`usually express his information need as a full text statement.
`
`If he then chooses to use a
`
`complex Boolean formulation, the user model builder would notice this, making an
`
`assertion that the user is using an unfamiliar method to enter his query. In response to this
`
`assertion, the explainer might be more likely to offer assistance These requirements
`
`involve some significant research issues.
`
`An initial pass at implementing an explainer would be to provide to the user with
`
`help facility like that available on the VAX/VMS operating system and then record the
`
`kinds of help that the user requests. The explainer would be integrated into the system in
`
`much the way that the interface manager is. It would run as a separate expert, not under the
`
`coordination of the control expert
`
`7.2.4
`
`Natural
`
`language Techniques
`
`I3R represents a beginning in the integration of AI techniques into IR systems.
`
`There are a number of ways that 13R can be extended. The first way is the incorporation of
`
`natural language processing techniques in a number of different areas. Since, in the docu(cid:173)
`
`ment retrieval domain, the primary means of expressing knowledge is natural language,
`
`natural language processing techniques are prime candidates for inclusion into 13R. This
`
`has already begun in the context of improving the performance of the search techniques.
`
`This work is being pursued as an independent system called ADRENAL [Croft 87]. The
`
`focus of this work is to define and demonstrate the use of NL techniques that can be
`
`applied effectively in systems that contain large numbers of documents. The basic idea is
`
`to apply the techniques to compare the query to sets of documents that have been retrieved
`
`using traditional methods. This limits the comparison to those documents that have a like-
`
`016
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`205
`
`lihood of being relevant to the information need. These techniques could be added to 13R
`
`by extending the search controller or by adding a new expert.
`
`Another place where NL techniques could be added to I3R is in the analysis of the
`
`context of citations. While it has been shown that citations are a valuable source of infor-
`
`mation for finding relevant documents, more information could be determined about what a
`
`citation means from its context. For example, some citations point to articles that have a
`
`differing point of view than the author's, or they can point to work that contains similar
`
`ideas. This kind of information would be valuable for the operation of the browsing ex-
`
`pert, since it would allow the BE to make a more informed recommendation to the user.
`
`These kinds of citation link types have been partially defined by Trigg [1986J. This kind of
`
`- --
`
`information could be incorporated into I3R by adding link types to the citations links, ex-
`---- -
`-
`- -- - - - -
`-
`-
`-
`-
`-
`-
`---- -- -------- ------ -- ~---- ----
`-
`-
`-
`-
`tending the heuristics of the browsing expert, and adding an expert to do the analysis of the
`
`documents.
`
`7.2.5
`
`Representation
`
`Another area in which I3R can be extended is in the information kept about different
`
`kinds of journals and particular issues. For example, in the publications of the ACM,
`
`Computing Surveys is primarily directed at providing tutorial kinds of articles, whereas the
`
`Journal of the ACM is focussed at highly theoretical articles requiring a relatively high
`
`degree of mathematical sophistication. Having this kind of information as evidence would
`
`allow the system to provide documents that are better matched to the user's level of domain
`
`expertise.
`
`7.2.6
`
`Interface Considerations
`
`Finally, an area of major concern in the further development of I3R is the enhance-
`
`ment of the graphical presentation of information. One of the ways that this can be ac-
`
`017
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`206
`
`complished is the use of color to convey other kinds of information such as relevance. For
`
`example, the most relevant documents could be displayed in red and decreasing relevance
`
`shown by scaling the colors to blue to represent the least relevant. Another use of color is
`
`highlighting different groups of links.
`
`Another possibility is to use three dimensional representation to show more of the
`
`neighborhood of a node in the concept/document knowledge. Specifically, extending the
`
`current organization of the neighborhood and context maps to three dimensions would
`
`allow the map to show 18 more nodes for a total of 25. Use of three dimensions to
`
`represent the neighborhoods would be available to system experts, since very few people
`
`are trained to visualize in three dimensions.
`
`On a purely implementational note, the entire interface could be rewritten using the
`
`currently developing X-standard [Scheifler 86J. This would allow 13R to operate on a
`
`much wider variety of architectures.
`
`018
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`BIBLIOGRAPHY
`
`[Barbi 84]
`BARBI, E. ET. AL. "A conceptual approach to document retrievaL" Proc. ofthe Znd
`ACM-SIGOA Conf. on Office Info. Sys. 219-226, 1984.
`
`[Bates 79a]
`BATES, M.J. "Information search tactics." J. of the Amer. Soc. for Info. Sci, 30:
`206-214, 1979.
`
`[Bates 79b]
`BATES, M.J. "Idea tactics." J. of the Amer. Soc.for Info. Sci, 30: 280-289, 1979.
`
`[Bates 86]
`BATES, M.J. "Terminological assistance for the online subject searcher." Proc. of
`the 2nd Conf. on Computer Interfaces & Intermediaries for Info. Ret. Defense
`Technical Information Center, Alexandria VA, 1986.
`
`[Belkin 83]
`BELKIN, N.J., SEEGER, T., & WERSIG, G. "Distributed expert problem treatment
`as a model for information system analysis and design." J. of Info. Science.S:
`153-167, 1983
`-
`-
`-
`~~---------~ ~ -
`[Belkin 84]
`BELKIN, N.J., HENNINGS, R.D., SEEGER, T.,".Simulation of a distributed ex(cid:173)
`pert-based information provision mechanism.".Information Technology, 3:122(cid:173)
`141, 1984.
`
`[Belkin 86]
`BELKIN, N.J. & KWASNIK, B.H. "Using structural representations of anomalous
`states of knowledge for choosing document retrieval strategies." Proc. ofthe ACM
`SIGIR Int' l Conf. on Research and Development in Info. Retrieval. 11-22, Pisa,
`Italy 1986.
`
`[Belkin 87aj
`BELKIN. N.J. & CROfT, W.B. "Retrieval techniques." in Annual Review of Info.
`Science and Technology, American Society for Info. Science, 22: 109-145, 1987.
`
`[Belkin 87b]
`BELKIN, N.J. ET. AL. "Distributed expert-based information systems: an inter(cid:173)
`disciplinary approach." Information Processing and Management, 23: 395-409,
`1987.
`
`(Borgman 8TI
`BORGMAN, c.L. "Information system functionality: a user-driven perspective."
`Position paper presented at Workshop on Distributed Expert-Based Info. Systems,
`1987.
`
`(Brajnik 851
`BRAJNIK, G. GUIDA, G. & TASSO, C. "An expert interface for effective man-ma(cid:173)
`chine interaction." Cooperative Interactive Systems, L.Eak (Ed.). Springer-Verlag:
`New York, 1985.
`
`207
`
`019
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`fBrajnik &71
`BRAJNIK, G. GUIDA, G. & TASSO, C. "User modeling in intelligent information
`retrieval." Info. Proc. & Management, 23: 305-320, 1987.
`
`(Brooks 83J
`BROOKS, H.M. & BELKIN, N.J. "Using discourse analysis for the design of in(cid:173)
`formation retrieval interaction mechanisms." Proc. ofthe Sixth Annual International
`ACM SIGIR Conf.: Research and Development in Info. Retrieval, 31-47,1987.
`
`[Brooks 85]
`BROOKS, H.M., DANIELS, P.l., & BELKIN, N.l. "Problem descriptions and user
`models: Developing an intelligent interface for document retrieval systems." Proc.of
`Informatics 8, Oxford, April 1985, London Aslib, 1985.
`
`IBrooks 861
`BROOKS, H.M. "Developing and representing problem descriptions." IRIFS 6: In(cid:173)
`telligent Info . Systems for the Info. Society, Frascati, Italy 1985., Amstersdam,
`North Holland, 1986.
`
`[Brooks 87a]
`BROOKS, H.M. "The functions of an information system: the Monstrat model."
`Position paper presented at Workshop on Distributed Expert-Based Info. Systems,
`1987.
`
`[Brooks 87b]
`BROOKS, H.M. "Experts systems and intelligent information retrieval." Info. Proc .
`& Management, 23: 367-382, 1987.
`
`fBurket 79 j
`BURKET, T.G . EMRATH, P. & KUCK, 0.1. "The use of vocabulary files for on(cid:173)
`line information retrieval." Info. Proc. & Management, 15: 181-189, 1979.
`
`[Caplinger 861
`CAPLINGER, M. "Graphical Database Browsing." Proc . of Third ACM-SIGOIS
`Conference on Office Inf. Sys., October 1986, Providence Rhode Island, 113-121.
`
`[Carberry 87]
`CARBERRY, S. "Workshop report: First international workshop on user-model(cid:173)
`ing." The Al Magazine, Fall: 71- 74, 1987.
`
`[Charniak 85]
`CHARNIAK, E. & MCDERMOTT, D. Introduction to Artificial Intelligence, Read(cid:173)
`ing , Massachusetts, Addison-Wesley, 1985.
`
`[Chiaramella 87]
`CHIARAMELLA & Y. DEFUDE, B. " A prototype of an intelligent system for infor(cid:173)
`mation retrieval: IOTA." Info. Proc. & Management, 23: 285-304, 1987.
`
`[Cleverdon 66]
`CLEVERDON,C.W., MILLS.,J., KEEN, E.M. "Factors determining the performance
`of indexing systems.' Aslib Cranfield Project, Vol. I-Design, Cranfield, Eng(cid:173)
`land, 1966.
`
`020
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`[Cohen 87]
`COHEN, P. R. & KJELDSEN, R. " Information retrieval by constrained spreading
`activation in semantic networks." Info. Proc & Management 23: 255-268, 1987.
`
`209
`
`[Conklin 87]
`CONKLIN, J. "Hypertext: an introduction and survey." IEEE Computer 20 (Sept):
`17-41, 1987.
`
`[Cooper 1971]
`COOPER, W.S., "A definition of relevance for information retrieval." Information
`Storage and Retrieval, 7:19-37, 1971.
`
`[Croft 79]
`CROFT, W.B. & HARPER, D.l. "Using probabilistic models of document re(cid:173)
`trieval without relevance information." 1. ofDocumentation, 35 : 285-289, 1979.
`
`[Croft 83]
`CROFf, W.B., WOLF, R., THOMPSON, R.H. "A network organization used for
`document retrieval." Proc. of Sixth Annual lnt' I ACM SIGIR Can! on Research
`and Development in I.R., lune 6-8, 1983, Bethesda, Maryland, SIGIR Forum
`17:4, 178-188, 1983.
`
`[Croft 821r
`CROFT, W. B. & THOMPSON, R.H. "The use of adaptive mechanisms for selec(cid:173)
`tion of search strategies in document retrieval systems." Research and Development
`in Info. Retrieval: Proc of the 3rd joint BCS and ACM symposium, July 2-6,
`Cambridge, England, Cambridge University Press, 95-110,1984.
`
`[Croft 8Sa]
`CROFT, W.B. & PARENTY, T.J. "A comparison of a network structure and a
`database system used for document retrieval." Info. Sys. to: 377-390, 1985.
`
`[Croft 85b]
`CROFT, W.B.& THOMPSON, R.H., An expert assistant for document retrieval.
`Amherst, MA: University of Massachusets, Dept of Compo and Info. Science, Tech
`Report 85-05, 1985.
`
`[Croft 86a]
`CROFT, W.B. "Boolean queries and dependencies in probabilistic retrieval mod (cid:173)
`els." J. of the Amer. Soc. for Info. Sci. 37: 71-77, 1986.
`
`[Croft 86b]
`CROff, W.B. "User-specified domain knowledge for document retrieval." Proc. of
`the ACM SIGIR Int'I Can! on Research and Development in Info. Retrieval. 201(cid:173)
`206, Pisa, Italy 1986.
`
`[Croft 86c]
`CROFT, W.B. & THOMPSON, R.B . "I 3R: a new approach to the design of docu (cid:173)
`ment retrieval systems." J. of the Amer. Soc for Info. Sci. (to appear), 1986.
`
`021
`
`Facebook Inc. Ex. 1214 Part 5
`
`

`
`ICroft 87al
`CROFf, W.B. & LEWIS, D.D. "An Approach to natural language processing for
`document retrieval." Proc. ofthe Tenth Annual International ACM SIGIR Conf.:
`Research and Development in Info. Retrieval, 26-32, 1987.
`
`210
`
`[Croft 87b]
`CROFT, W.B. "Approaches to intelligent information retrieval." Info. Proc. &
`Management, 23: 249-254, 1987.
`
`[Daniels 85J
`DANIELS. PJ., BROOKS, H.M., & BELKIN, N.J. "Using problem structures for
`driving human-computer dialogues." in RIAO 85. Actes of the Conf.: Rechere
`d'Informations assistee par Ordinateur,203-221, Grenoble, France, I.M.A.G.,
`1985.
`
`[Daniels 861
`DANIELS, PJ. 'The user modelling function of an intelligent interface for document
`retrieval systems." IRIFS 6: Intelligent Info. Systems for the Info. Society,
`Frascati, Italy 1985., Amstersdam, North Holland, 1986.
`
`[Daniels 87]
`DANIELS, PJ.