US008220055B1
`
`(12) United States Patent
`Kennedy
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,220,055 B1
`Jul. 10, 2012
`
`(54) BEHAVIOR BLOCKING UTILIZING
`POSITIVE BEHAVIOR SYSTEM AND
`METHOD
`
`7/2002 Redlich et a1. .............. .. 713/201
`2002/0099959 A1 *
`2003/0061514 A1: 3/2003 Bardsley et a1.
`713/201
`2004/0083372 A1
`4/2004 WllllamSOn et a1.
`713/188
`2004/0117648 A1 *
`6/2004 Klssel ......................... .. 713/200
`
`(75) Inventor: Mark K. Kennedy, Redondo Beach, CA
`(Us)
`
`(73) Assignee: Symantec Corporation, Mountain View,
`CA (U S)
`
`FOREIGN PATENT DOCUMENTS
`1202228 Al * 5/2002
`EP
`2367714 A * 4/2002
`GB
`* Cited by examiner
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U'S'C' 1546)) by 2814 days‘
`(21) APPI' No‘: 10/774,177
`
`Primary Examiner i Kim vu
`Assistant Examineri Randal Moran
`(74) Attorney, Agent, orFirm * McKay and Hodgson, LLP;
`Serge J. Hodgson; Sean P. LeW1s
`
`(51) Int- Cl-
`(2006-01)
`H041‘ 29/06
`US. Cl- .......................................... ..
`(58) Field of Classi?cation Search ............. .. 726/22e25
`See application ?le for Complete Search history.
`
`(56)
`
`References Clted
`
`U.S. PATENT DOCUMENTS
`
`7,076,803 B2* 7/2006 Bruton et a1. ................. .. 726/23
`7,191,252 B2* 3/2007 Redlich et a1. .............. .. 709/246
`
`A method includes decreasing a suspicion of a negative action
`by an application if the application has previously performed
`a positive aetion_ The positive action is an action that is never
`or rarely taken by malicious code. In one example, the posi
`tive action is use of a user interface element by the application
`to have a user interaction With a user of the computer system.
`By taking into consideration the positive action by the appli
`cation, the occurrence of false positives is minimized.
`
`21 Claims, 2 Drawing Sheets
`
`Hacker
`Computer
`System 104
`
`Host Computer System 102
`
`CPU
`108 T
`
`Memory 114
`
`Behavior
`Blocking
`Application
`1 06
`
`U0 Interface 110
`
`Key
`Board
`116
`
`Printer
`120
`
`Mouse
`118
`
`\l
`
`1,9
`Device
`123
`
`_
`I
`Display Devlce
`122
`
`Display Device 132
`
`Processor
`1 34
`
`' “100
`
`\
`Network
`Interface 1 38
`
`Server System 130
`
`Memory 1 3 6
`
`000001
`
`Symantec 1009
`IPR of U.S. Pat. No. 8,141,154
`
`

`
`US. Patent
`
`Jul. 10, 2012
`
`Sheet 1 of2
`
`US 8,220,055 B1
`
`Host Computer System 102
`
`CPU
`108 ’
`
`M°m°ry “4
`
`Behavior
`Blocking
`Application
`1 06
`
`U0 Interface 110
`
`I; 221
`1 16
`
`Printer
`120
`
`Mouse
`1 l8
`
`_
`_
`Dlsplay Device
`122
`
`Display Device 132
`
`Processor
`134
`
`Server System 130
`
`FIG. 1
`
`Hacker
`Computer
`System 104
`
`Network 124
`
`‘\100
`
`\
`Network
`Interface 138
`
`Memory 136
`
`000002
`
`

`
`US. Patent
`
`Jul. 10, 2012
`
`Sheet 2 of2
`
`US 8,220,055 B1
`
`202
`
`FIG. 2
`
`Host Computer
`Process
`200
`\
`
`204
`/
`Hook :3‘Ct19n(s)by
`appl1cat1on(s)
`l
`Hooked action
`
`206
`/
`
`5' ___ "505x555?
`
`l _ _ _7 _ _ _ _ _ _ _ _ _ _ _ _ _ _ -l
`
`208
`
`210
`
`Action is
`posltive?
`
`Action is
`negative‘?
`
`212
`
`\
`
`_______________________ _
`Decrement suspicion
`level counter for
`
`222
`
`Set Increment
`value to low
`1
`
`226
`’ Increment suspicion level /
`counter for application
`
`Positive
`Action(s)?
`
`Set Increment
`value to high
`
`224
`
`230
`
`/
`
`Take Protective Action
`
`. ---- - -_ ---------------- — -.
`
`{
`: Notlfy Host Computer :
`5
`System
`i
`:
`User/Administrator
`
`228
`
`susplclon level
`counter exceeds
`threshold?
`
`232
`
`V
`
`-' """""""""" “ '
`
`E
`
`i
`' ' '7 """"""" ' '
`214
`
`000003
`
`

`
`US 8,220,055 B1
`
`1
`BEHAVIOR BLOCKING UTILIZING
`POSITIVE BEHAVIOR SYSTEM AND
`METHOD
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The present invention relates to the protection of computer
`systems. More particularly, the present invention relates to a
`behavior blocking system.
`2. Description of the Related Art
`In the computer security domain, there Were behavior
`blocking applications that Would block a suspicious action by
`an application on a computer system. HoWever, a large set of
`these blocked suspicious actions are not malicious, i.e., are
`false positives.
`Typically, the user of the computer system is noti?ed that
`the suspicious action has been blocked and the user is
`required to select hoW the blocked suspicious action should
`be handled, e.g., blocked, released, blocked in the future or
`released in the future. Thus, these false positives are intrusive
`and annoying to the user of the computer system at a mini
`mum and result in lost productivity due to the time spent by
`the user in responding to the false positives.
`
`SUMMARY OF THE INVENTION
`
`A method includes decreasing a suspicion of a negative
`action by an application if the application has previously
`performed a positive action. The positive action is an action
`that is never or rarely taken by malicious code. In one embodi
`ment, the positive action is use of a user interface element by
`the application to have a user interaction With a user of a
`computer system. By taking into consideration the positive
`action by the application, the occurrence of false positives is
`minimized.
`Embodiments in accordance With the present invention are
`best understood by reference to the folloWing detailed
`description When read in conjunction With the accompanying
`draWings.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`FIG. 1 is a diagram of a client-server system that includes
`a behavior blocking application executing on a host computer
`system in accordance With one embodiment of the present
`invention; and
`FIG. 2 is a How diagram of a host computer process in
`accordance With one embodiment of the present invention.
`Common reference numerals are used throughout the draW
`ings and detailed description to indicate like elements.
`
`DETAILED DESCRIPTION
`
`In accordance With one embodiment, referring to FIG. 2, a
`method includes determining that an action by an application
`is negative (check operation 218). Upon a determination that
`the action is negative, the method includes determining if the
`application has had at least one positive action prior to the
`negative action (check operation 220). A suspicion level
`counter for the application is incremented more (operations
`224, 226) or less (operations 222, 226) depending upon
`Whether the application has had at least one positive action
`prior to the negative action. By taking into consideration the
`positive action by the application, the occurrence of false
`positives is minimized.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`More particularly, FIG. 1 is a diagram of a client-server
`system 100 that includes a behavior blocking application 106
`executing on a host computer system 102, e.g., a ?rst com
`puter system, in accordance With one embodiment of the
`present invention.
`Host computer system 102, sometimes called a client or
`user device, typically includes a central processing unit
`(CPU) 108, hereinafter processor 108, an input output (I/O)
`interface 110, and a memory 114.
`Host computer system 102 may further include standard
`devices like a keyboard 116, a mouse 118, a printer 120, and
`a display device 122, as Well as, one or more standard input/
`output (I/O) devices 123, such as a compact disk (CD) or
`DVD drive, ?oppy disk drive, or other digital or Waveform
`port for inputting data to and outputting data from host com
`puter system 102. In one embodiment, behavior blocking
`application 106 is loaded into host computer system 102 via
`I/O device 123, such as from a CD, DVD or ?oppy disk
`containing behavior blocking application 106.
`Host computer system 102 is coupled to a server system
`130 of client-server system 100 by a netWork 124. Server
`system 130 typically includes a display device 132, a proces
`sor 134, a memory 136, and a netWork interface 138.
`Further, host computer system 102 is also coupled to a
`hacker computer system 104 of client-server system 100 by
`netWork 124. In one embodiment, hacker computer system
`104 is similar to host computer system 102, for example,
`includes a central processing unit, an input output (I/O) inter
`face, and a memory. Hacker computer system 104 may fur
`ther include standard devices like a keyboard, a mouse, a
`printer, a display device and an I/O device(s). The various
`hardware components of hacker computer system 104 are not
`illustrated to avoid detracting from the principles of the inven
`tion.
`NetWork 124 can be any netWork or netWork system that is
`of interest to a user. In various embodiments, netWork inter
`face 138 and I/O interface 110 include analog modems, digi
`tal modems, or a netWork interface card.
`Behavior blocking application 106 is stored in memory
`114 of host computer system 102 and executed on host com
`puter system 102. The particular type of and con?guration of
`host computer system 102, hacker computer system 104, and
`server system 130 are not essential to this embodiment of the
`present invention.
`FIG. 2 is a How diagram ofa host computer process 200 in
`accordance With one embodiment of the present invention.
`Referring noW to FIGS. 1 and 2 together, execution of behav
`ior blocking application 106 by processor 108 results in the
`operations of host computer process 200 as described beloW
`in one embodiment.
`From an enter operation 202, How moves to a hook action
`(s) by application(s) operation 204. In hook action(s) by
`application(s) operation 204, one or more applications
`executed on ho st computer system 102 are hooked. Generally,
`an application is hooked by hooking and intercepting speci?c
`action(s), sometimes called hooked action(s), of the applica
`tion.
`More particularly, in hook action(s) by application(s)
`operation 204, one or more actions of one or more applica
`tions are hooked. To illustrate, a ?le system ?lter driver in the
`WindoWs operating system hooks ?le events by installing a
`layer betWeen the user and ?le system for the ?le events and
`intercepts the ?le events betWeen the user and ?le system.
`In accordance With one embodiment, an application is
`hooked by installing one or more user mode hooks to inter
`cept actions by the application that are interactions With the
`user. These actions by the application that are interactions
`
`000004
`
`

`
`US 8,220,055 B1
`
`3
`With the user are thus hooked actions. Hooking of applica
`tions and actions is Well knoWn to those of skill in the art and
`typically depends upon the particular operating system of
`host computer system 102. The particular hooking technique
`used is not essential to the present invention.
`From hook action(s) by application(s) operation 204, How
`moves to a hooked action operation 206. In hooked action
`operation 206, a hooked action, i.e., an action hooked in hook
`action(s) by application(s) operation 204, is made by a
`hooked application. The hooked action is sometimes herein
`referred to as “the action” or “the action by the hooked appli
`cation” for simplicity of discussion.
`From hooked action operation 206, How moves, optionally,
`to a stall action operation 208 (or directly to an action is
`positive check operation 210 if stall action operation 208 is
`not performed).
`In stall action operation 208, the action by the hooked
`application is stalled, i.e., is prevented from being executed or
`otherWise implemented. From stall action operation 208, How
`moves to action is positive check operation 210.
`In action is positive check operation 210, a determination is
`made as to Whether the action by the hooked application is
`positive, i.e., is a positive action. Generally, a positive action
`is an action that is rarely or never performed by malicious
`code. In one embodiment, malicious code is de?ned as any
`computer program, module, set of modules, or code that
`enters a computer system Without an authorized user’ s knoWl
`edge and/ or Without an authorized user’s consent.
`For example, malicious code rarely if ever interacts With
`the user, e.g., a human, of host computer system 102. As an
`illustration, malicious code has no user interaction about 95%
`of the time and about 5% of the time uses a message box to
`have a very minimal user interaction. Accordingly, in one
`embodiment, a positive action by an application occurs When
`the application interacts With the user of host computer sys
`tem 102, i.e., has a user interaction. Because use of a message
`box is a very minimal user interaction, in one embodiment,
`use of a message box is not de?ned as a positive action
`although use of a message box can be a positive action if
`desired to be de?ned as such.
`For example, a positive action by an application occurs
`When the application uses a user interface element to have a
`user interaction With the user.
`Examples of user interactions include interactions With the
`user in setting up the application or using the application. For
`example, a user interaction occurs When the application is
`con?gured by the user. As another example, a user interaction
`occurs When the user selects the recipient(s) of an e-mail
`message or the information, e.g., attachments, to be sent With
`an e-mail message. Although speci?c examples of user inter
`actions are provided, in light of this disclosure, it is under
`stood that other user interactions With an application can
`occur, and the particular user interactions depend, for
`example, on the particular application.
`Generally, a user interface element is an element used by a
`user in providing input or otherWise interacting With an appli
`cation. Examples of user interface elements include: (1)
`check boxes; (2) radio boxes; (3) list boxes; (4) combo boxes;
`(5) text boxes; (6) common dialog boxes; and (7) message
`boxes. Although speci?c examples of user interface elements
`are provided, in light of this disclosure, it is understood that
`other user interface elements can be used by the user, and the
`particular user interface element depends, for example, on the
`particular application.
`If a determination is made that the action by the hooked
`application is a positive action in action is positive check
`operation 210, How moves, optionally, to a decrement suspi
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`cion level counter for application operation 212 (or directly to
`an optional release action operation 214 if operation 212 is
`not performed or directly to an exit operation 216 if opera
`tions 212 and 214 are not performed).
`In one embodiment, each application has an associated
`suspicion level counter, Which is a measure of the suspicion
`associated With the application. This suspicion level counter
`is decremented in decrement suspicion level counter for
`application operation 212 thus reducing the suspicion asso
`ciate With the application.
`Decrement suspicion level counter for application opera
`tion 212 is optional and in one embodiment is not performed.
`In accordance With this embodiment, the suspicion level
`counter associate With the application is not decremented and
`the suspicion associated With the application remains
`unchanged.
`From decrement suspicion level counter for application
`operation 212, How moves to, optionally, release action
`operation 214. As discussed above, stall action operation 208
`is optional. Accordingly, if stall action operation 208 is per
`formed and the action Was stalled, release action operation
`214 is performed to release the action.
`Conversely, if stall action operation 208 Was not performed,
`release action operation 214 is unnecessary and thus not
`performed.
`From release action operation 214 (or directly from decre
`ment suspicion level counter for application operation 212 if
`operation 214 is not performed), ?oW moves to and exits at
`exit operation 216 or returns to hooked action operation 206.
`Returning again to action is positive check operation 210,
`if a determination is made that the action is not a positive
`action, ?ow moves to an action is negative check operation
`218.
`In action is negative check operation 218, a determination
`is made as to Whether the action by the hooked application is
`negative, i.e., is a negative action. Generally, a negative action
`is an action that is highly suspicious or suggestive of mali
`cious code.
`Examples of negative actions include: (1) attacking secu
`rity softWare; (2) sending of executable attachments; (3)
`copying of an application across a netWork; and (4) sending
`executable instant messengering attachments. Although spe
`ci?c examples of negative actions are provided, in light of this
`disclosure, it is understood that other negative actions can
`occur, and the particular negative actions depend, for
`example, on the particular application.
`If a determination is made that the action by the hooked
`application is not a negative action in action is negative check
`operation 218, How moves to optional release action opera
`tion 214 (or directly to exit operation 216 if operation 214 is
`not performed).
`Conversely, if a determination is made that the action by
`the hooked application is a negative action in action is nega
`tive check operation 218, How moves to a previous positive
`action(s) check operation 220. In previous positive action(s)
`check operation 220, a determination is made as to Whether
`the hooked application has performed any positive actions
`prior to the present negative action.
`If a determination is made that the hooked application has
`performed at least one positive action prior to the present
`negative action, How moves to a set increment value to loW
`operation 222.
`In set increment value to loW operation 222, the increment
`value for the suspicion level counter for the application is set
`to loW. Stated another Way, in set increment value to loW
`operation 222, the increment value for the suspicion level
`counter for the application is set to a ?rst increment value,
`
`000005
`
`

`
`US 8,220,055 B1
`
`5
`sometimes called a loW increment value. From set increment
`value to loW operation 222, How moves to an increment
`suspicion level counter for application operation 226.
`Conversely, if a determination is made that the hooked
`application has not performed at least one positive action
`prior to the present negative action, How moves to a set
`increment value to high operation 224.
`In set increment value to high operation 224, the increment
`value for the suspicion level counter for the application is set
`to high. Stated another Way, in set increment value to high
`operation 224, the increment value for the suspicion level
`counter for the application is set to a second increment value,
`sometimes called a high increment value. Generally, the sec
`ond or high increment value is greater than the ?rst or loW
`increment value. From set increment value to high operation
`224, How moves to increment suspicion level counter for
`application operation 226.
`In increment suspicion level counter for application opera
`tion 226, the suspicion level counter for the application is
`increment by the set increment value.
`More particularly, the suspicion level counter for the appli
`cation is increment by the loW increment value set in set
`increment value to loW operation 222 if the hooked applica
`tion has performed at least one positive action prior to the
`present negative action. Conversely, the suspicion level
`counter for the application is increment by the high increment
`value set in set increment value to high operation 224 if the
`hooked application has not performed at least one positive
`action prior to the present negative action.
`Thus, the suspicion level associate With the application,
`i.e., the suspicion level counter, is increment more or less by
`the negative action depending upon Whether the application
`has previous exhibited at least one positive action.
`From increment suspicion level counter for application
`operation 226, How moves to a suspicion level counter
`exceeds threshold check operation 228. In suspicion level
`counter exceeds threshold check operation 228, a determina
`tion is made as to Whether the suspicion level counter exceeds
`the suspicion level threshold for the application. In one
`embodiment, the suspicion level threshold is a value Which
`When exceeded by the suspicion level counter indicates With
`reliability that the application is malicious, i.e., contains mali
`cious code.
`If a determination is made that the suspicion level counter
`does not exceed the suspicion level threshold for the applica
`tion in suspicion level counter exceeds threshold check opera
`tion 228, How moves to release action operation 214 (or
`directly to exit operation 216 if operation 214 is not per
`formed). In this event, the negative action by the application,
`and, more generally, the suspicion associated With the appli
`cation, is not su?icient to conclude that the application is
`malicious.
`Conversely, if a determination is made that the suspicion
`level counter does exceed the suspicion level threshold for the
`application in suspicion level counter exceeds threshold
`check operation 228, How moves to a take protective action
`operation 230. In take protective action operation 230, pro
`tective action is taken to prevent the application from causing
`damage to or exploiting host computer system 102. For
`example, the action by the application is terminated. As dis
`cussed above, optionally, the action by the application Was
`stalled in stall action operation 208. As another example of
`protective action, the application and/or a malicious thread
`running Within the context of the application is terminated.
`Termination of applications is Well knoWn to those of skill in
`the art and so is not discussed further for clarity of discussion.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`More particularly, because the suspicion level counter has
`exceeded the suspicion level threshold for the application, the
`likelihoodthat the application is malicious code is signi?cant.
`HoWever, by taking protective action, the application is pre
`vented from exploiting and/ or damaging host computer sys
`tem 102.
`In one embodiment, the user of host computer system 102
`and/or administrator are given a set of choices on the protec
`tive action to be taken in take protective action operation 230
`(or Whether no protective action is to be taken).
`To illustrate, take the case When operation 212 is not per
`formed, the loW increment value is 0.4, the high increment
`value is 0.6, and the suspicion level threshold is 0.5.
`In accordance With this example, if the hooked application
`has not performed at least one positive action prior to the
`present negative action, the suspicion level counter Will be
`incremented by and equal the high increment value of 0.6.
`Accordingly, a determination Will made in check operation
`228 that the suspicion level counter is 0.6, Which is greater
`than the suspicion level threshold of 0.5, and protective action
`Will be taken in take protective action 230.
`HoWever, continuing With this example, if the hooked
`application has performed at least one positive action prior to
`the present negative action, the suspicion level counter Will be
`incremented by and equal the loW increment value of 0.4.
`Accordingly, a determination Will made in check operation
`228 that the suspicion level counter is 0.4, Which is less than
`the suspicion level threshold of 0.5, and the action Will be
`released in release action operation 214.
`As this example demonstrates, by taking into consideration
`Whether or not the hooked application has performed at least
`one positive action prior to the present negative action, pro
`tective action is taken or not taken, respectively, thus mini
`miZing the incidence of false positives.
`To illustrate another example, take the case When the dec
`rement value of operation 212 is 0.34, the loW increment
`value is 0.34, the high increment value is 0.6, and the suspi
`cion level threshold is 1.0.
`In accordance With this example, if the hooked application
`has not performed at least one positive action and has per
`formed tWo negative actions, the suspicion level counter Will
`be equal to the high increment value of 0.6 multiplied by tWo,
`i.e., 1.2. Accordingly, a determination Will made in check
`operation 228 that the suspicion level counter is 1.2, Which is
`greater than the suspicion level threshold of 1.0, and protec
`tive action Will be taken in take protective action 230.
`HoWever, continuing With this example, if the hooked
`application has performed at least one positive action, three
`subsequent negative actions are alloWed before protective
`action is taken in take protective action 230. More particu
`larly, the suspicion level counter Will be —0.34, 0.00, 0.34,
`0.68 after the positive action, ?rst negative action, second
`negative action, and third negative action, respectively.
`Accordingly, if the application has previous exhibited at least
`one positive action, the suspicion associated With subsequent
`negative actions by the application is reduced thus minimiZ
`ing the occurrence of false positives.
`FloW moves from take protective action operation 230,
`optionally, to a notify host computer system user/administra
`tor operation 232 (or directly to exit operation 216 if opera
`tion 232 is not performed). In notify host computer system
`user/administrator operation 232, the user of host computer
`system 102 and/or the administrator are noti?ed that protec
`tive action has been taken on host computer system 102, e. g.,
`that an action and/ or an application have been terminated. The
`user and/or administrator can be noti?ed using any one of a
`number of techniques, e.g., by using a pop up WindoW, by
`
`000006
`
`

`
`US 8,220,055 B1
`
`7
`Writing to a ?le and/or otherwise by logging the event. Fur
`ther, a noti?cation or a sample of the malicious code can be
`provided to a security center.
`From notify host computer system user/ administrator
`operation 232, How moves to and exits at exit operation 216 or
`returns to hooked action operation 206.
`Referring again to FIG. 1, behavior blocking application
`106 is in computer memory 114. As used herein, a computer
`memory refers to a volatile memory, a non-volatile memory,
`or a combination of the tWo.
`Although behavior blocking application 106 is referred to
`as an application, this is illustrative only. Behavior blocking
`application 106 should be capable of being called from an
`application or the operating system. In one embodiment, an
`application is generally de?ned to be any executable code.
`Moreover, those of skill in the art Will understand that When it
`is said that an application or an operation takes some action,
`the action is the result of executing one or more instructions
`by a processor.
`While embodiments in accordance With the present inven
`tion have been described for a client-server con?guration, an
`embodiment of the present invention may be carried out using
`any suitable hardWare con?guration involving a personal
`computer, a Workstation, a portable device, or a netWork of
`computer devices. Other netWork con?gurations other than
`client-server con?gurations, e. g., peer-to-peer, Web-based,
`intranet, internet netWork con?gurations, are used in other
`embodiments.
`Herein, a computer program product comprises a medium
`con?gured to store or transport computer readable code in
`accordance With an embodiment of the present invention.
`Some examples of computer program products are CD-ROM
`discs, DVDs, ROM cards, ?oppy discs, magnetic tapes, com
`puter hard drives, servers on a netWork and signals transmit
`ted over a netWork representing computer readable code.
`As illustrated in FIG. 1, this medium may belong to the
`computer system itself. HoWever, the medium also may be
`removed from the computer system. For example, behavior
`blocking application 106 may be stored in memory 136 that is
`physically located in a location different from processor 108.
`Processor 108 should be coupled to the memory 136. This
`could be accomplished in a client-server system, or alterna
`tively via a connection to another computer via modems and
`analog lines, or digital interfaces and a digital carrier line.
`More speci?cally, in one embodiment, host computer sys
`tem 102 and/or server system 130 is a portable computer, a
`Workstation, a tWo-Way pager, a cellular telephone, a digital
`Wireless telephone, a personal digital assistant, a server com
`puter, an lntemet appliance, or any other device that includes
`components that can execute the behavior blocking function
`ality in accordance With at least one of the embodiments as
`described herein. Similarly, in another embodiment, host
`computer system 102 and/or server system 130 is comprised
`of multiple different computers, Wireless devices, cellular
`telephones, digital telephones, tWo-Way pagers, or personal
`digital assistants, server computers, or any desired combina
`tion of these devices that are interconnected to perform, the
`methods as described herein.
`In vieW of this disclosure, the behavior blocking function
`ality in accordance With one embodiment of present invention
`can be implemented in a Wide variety of computer system
`con?gurations. In addition, the behavior blocking function
`ality could be stored as different modules in memories of
`different devices. For example, behaviorblocking application
`106 could initially be stored in server system 130, and then as
`necessary, a portion of behavior blocking application 106
`could be transferred to host computer system 102 and
`
`50
`
`55
`
`60
`
`65
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`8
`executed on host computer system 102. Consequently, part of
`the behavior blocking functionality Would be executed on
`processor 134 of server system 130, and another part Would
`be executed on processor 108 of host computer system 102. In
`vieW of this disclosure, those of skill in the art can implement
`various embodiments of the present invention in a Wide
`variety of physical hardWare con?gurations using an operat
`ing system and computer programming language of interest
`to the user.
`In yet another embodiment, behavior blocking application
`106 is stored in memory 136 of server system 130. Behavior
`blocking application 106 is transferred over netWork 124 to
`memory 114 in host computer system 102. In this embodi
`ment, netWork interface 138 and I/O interface 110 Would
`include analog modems, digital modems, or a netWork inter
`face card. If modems are used, netWork 124 includes a com
`munications netWork, and behavior blocking application 106
`is doWnloaded via the communications netWork.
`This disclosure provides exemplary embodiments of the
`present invention. The scope of the present invention is not
`limited by these exemplary embodiments. Numerous varia
`tions, Whether explicitly provided for by the speci?cation or
`implied by the speci?cation or not, may be implemented by
`one of skill in the art in vieW of this disclosure.
`
`What is claimed is:
`1. A method comprising:
`decreasing a suspicion of a negative action by an applica
`tion if said application has previously performed a posi
`tive action comprising setting an increment value of a
`suspicion level counter for said negative action to a ?rst
`value, Wherein said positive action is use of a user inter
`face element by said application to have a user interac
`tion With a user,
`Wherein if a determination is made that said application has
`not had said previous positive action prior to said nega
`tive action, said method further comprising setting said
`increment value of said suspicion level counter for said
`negative action to a second value greater than said ?rst
`value; and
`incrementing said suspicion level counter by said incre
`ment value.
`2. The method of claim 1 Wherein said positive action
`comprises interacting With said user of a computer system.
`3. The method of claim 1 Wherein said positive action is an
`action that is never performed by malicious code.
`4. The method of claim 1 Wherein said positive action is an
`action that is rarely performed by malicious code.
`5. The method of claim 1 Wherein said user interaction
`comprises an interaction With said user in setting up said
`application.
`6. The method of claim 1 Wherein said user interaction
`comprises an interaction With said user in using said applica
`tion.
`7. The method of claim 1 Wherein said user interaction
`comprises said user selecting recipient(s) of an e-mail mes
`sage.
`8. The method of claim 1 Wherein said user interaction
`comprises said user selecting information to be sent With an
`e-mail message.
`9. The method of claim 1 Wherein said user interface ele
`ment is an element used by said user in interacting With sai