==Phrack Inc.==

               Volume 0x0b, Issue 0x39, Phile #0x0b of 0x12

|=------------=[ HOLISTIC APPROACHES TO ATTACK DETECTION ]=--------------=|
|=-----------------------------------------------------------------------=|
|=-----------------------------=[ sasha ]=-------------------------------=|


  "The art of writing a beautiful fugue lies precisely in [the] ability to
   manufacture several different lines, each one of which gives the illusion of
   having been written for its own beauty, and yet which when taken together
   form a whole which does not seem forced in any way.  Now, this dichotomy
   between hearing a fugue as a whole, and hearing its component voices, is a
   particular example of a very general dichotomy, which applies to many kinds
   of structures built up from lower levels.

   A similar analysis could be made of dozens of Escher pictures, which rely
   heavily upon the recognition of certain basic forms, which are then put
   together in nonstandard ways;  and by the time the observer sees the
   paradox on a high level, it is too late - he can't go back and change his
   mind about how to interpret the lower-level objects."

     - Douglas R. Hofstadter [Hofstadter, 1979].

  "Oddly enough, one of the things that got me started was a joke, the title of
   a book by Douglas Adams - Dirk Gently's Holistic Detective Agency.  And I
   thought, that's an interesting phrase - what would it mean to solve a crime
   holistically?  It would mean that you'd have to 'solve' not just the crime,
   but the whole world in which the crime took place."

     - Alan Moore [Moore, 2000].


----|  1.  Introduction


This article concerns various approaches to the problem of detecting attacks.

Specifically, we are interested in enterprise environments in which weaknesses
in traditional security monitoring methods become apparent.

Holistic methods are proposed as a partial solution to some of the shortcomings
in traditional reductionist approaches.

Existing research literature will be reviewed, an example enterprise security
monitoring architecture that employs a holistic approach is described, and
some predictions regarding the future of security monitoring are made in the
concluding section.


----|  2.  Problem Space


Modern enterprise networks generate a vast amount of real-time environmental
data relating to security status, system status, network status, application
status, and so on.  Network management technologies and architectures have
evolved over time to solve the problems inherent in processing large amounts of
event data:  event correlation, event reduction, and root-cause analysis are
all employed.  Security monitoring technologies and architectures however, have
not yet matured to the same extent.  Most, if not all, security monitoring
technologies focus on reporting low-level events (such as observed attacks) in
as much detail as possible.  That approach is useful in a small environment but
fails in an enterprise environment for the following reasons:

*      The contextual information surrounding the detection of events might not
       be available due to the rate of change in the network and the possible
       geographic separation of event generators and management consoles.

*      The "signal-to-noise" ratio is much higher in an enterprise environment
       due to the large number of event generators.

*      The people performing monitoring may not have the privilege or mandate
       to connect to machines to investigate possible incidents, therefore they
       must rely purely on the event data available to them.

Current security monitoring technologies are difficult to scale for the above
reasons and are therefore difficult to deploy and use in an enterprise
environment.

Traditional approaches to attack detection focus exclusively on analysis based
on reductionism.  This article advocates a holistic approach that can work in
conjunction with traditional reductionist methods and add additional value.
These terms are now described below.


----|  3.  Reductionism and Holism


Traditional security monitoring technologies such as network and host based IDS
(Intrusion Detection Systems) and host based integrity checkers, operate on a
reductionist basis.  The reductionist approach is based on the belief that a
whole can be largely understood by examining its constituent parts;  i.e. it is
possible to infer the existence of an attack if a specific observation can be
made.  Such tools attempt to detect unauthorized change(s) or to match current
activity against known indicators of misuse.

Alongside the reductionist approach is the holistic approach.  Holism is based
on the belief that a whole is greater than the sum of its parts;  i.e. it is
possible to infer the existence of an attack if a set of observations (that
are perhaps superficially unrelated) can be approximately matched to a
structure that represents knowledge of the methods that attacks employ at a
high(er) level.

Another way to describe this distinction is as follows:  reductionist methods
reason by induction - they reason from particular observations to generate
supposed truths.  Holistic methods do the reverse - they start with general
knowledge and predict a specific set of observations.  In reality, the solution
of complex problems is best achieved by long strings of mixed inductive and
deductive inferences that weave back and forth between observations and
internal models.


----|  4.  Epiphenomena and the Connection Chain Problem


The following quote is from [Hofstadter, 1979] -

  "I would like to relate a story about a complex system.  I was talking one
   day with two systems programmers for the computer I was using.  They
   mentioned that the operating system seemed to be able to handle up to about
   thirty-five users with great comfort, but at about thirty five users or so,
   the response time all of a sudden shot up, getting so slow that you might as
   well log off and go home and wait until later.  Jokingly, I said, "Well,
   that's simple to fix - just find the place in the operating system where the
   number '35' is stored, and change it to '60'!".  Everyone laughed.  The
   point is, of course, that there is no such place.  Where, then, does the
   critical number - 35 users - come from?.  The answer is: it is a visible
   consequence of the overall system organization - an 'Epiphenomemon'.

   Similarly, you might ask about a sprinter, "Where is the '9.3' stored, that
   makes him be able to run 100 yards in 9.3 seconds?".  Obviously, it is not
   stored anywhere.  His time is a result of how he is built, what his
   reaction time is, a million factors all interacting when he runs.  The time
   is quite reproducible, but it is not stored in his body anywhere.  It is
   spread around among all of the cells of his body and only manifests itself
   in the act of the sprint itself."

The two examples above illustrate the sort of thinking that gives rise to
holistic solutions.  If we concede that an event that occurs in a security
monitoring architecture can often only acquire significance when viewed in the
context of other activity, then we can theorize that it is possible to detect
the presence of an attack by looking for epiphenomenon that occur as the
by-product of attacks.  This approach has been taken to the connection chain
problem.

To explain the connection chain problem it is necessary to first introduce
some terminology.  When an individual (or a program) connects to one computer,
and from there connects to another computer, and another, that is referred to
as a "connection chain".

The ability to detect a connection chain is advantageous - since it is the
traditional mechanism used by attackers to attempt to obfuscate their "real"
(i.e. initial) location.

In [Staniford-Chen, 1995] a system is described that can thumbprint a
connection chain by monitoring the content of connections.

This is achieved by forming a signature for the data in a network connection.
This signature is a small quantity which does not allow complete reconstruction
of the data, but does allow comparison with signatures of other connections to
determine with reasonable confidence whether the underlying connection is the
same or not.

The specific technology developed to perform this task is called local
thumbprinting.  This involves forming linear combinations of the frequencies
with which different characters occur in the network data sampled.  The optimal
linear combinations are chosen using a statistical methodology called principle
component analysis which is shown to work successfully when given at least a
minute and a half of a reasonably active network connection.

Thumbprinting relies on the fact that the content of an extended connection is
invariant at all points of the chain (once protocol details are abstracted
out).  Thus, if the system can compute thumbprints of the content of each
connection, these thumbprints can then be compared to establish whether two
connections have the same content.

A weakness in this method is that disguising the content of the extended
connection (such as encrypting it differently on each link of the chain) can
circumvent the technology.

In [Zhang et al., 2000] the connection chain problem is approached by employing
methods that do not rely on packet contents - by leveraging the distinct
properties of interactive network traffic (smaller packet sizes and longer idle
periods for interactive traffic than for machine generated traffic) to develop
an algorithm.

These examples shows that it is possible to detect attacks in a way that does
not rely on the detection of individual attack techniques.


----|  5.  Attack-Strategy Based Intrusion Detection


Another advantage to holistic methods that work on a "higher" layer of
inference than reductionist methods is in the area of attack strategy analysis.

In [Huang et al., 2000], an IDS framework is described that can perform
"intention analysis".  Intention analysis takes the form of "If A occurs, then
B occurs, we can predict that C will occur".

The suggested implementation mechanism in the paper is to employ a goal-tree
with the root node the ultimate goal of an attack.  Lower level nodes represent
alternatives or ordered sub-goals in achieving the upper node / goal.  Leaves
(end nodes) are sub-goals that can be substantiated using events that can be
identified in the environment using monitoring.

The addition of a temporal aspect to the model enables the model to "predict"
likely future steps in an attack as an attacker attempts to climb logically
higher in the goal-tree.

This example shows the significant extra value that can be provided by
"stepping back" and analyzing event data at a higher layer.  The reductionist
tendency is to step forwards and look into activity in detail;  the holistic
tendency is to step backwards and look at activity only in the context of other
activity.

Of course, a holistic model still relys on data gathered from the environment
using reductionist techniques, and this is discussed along with other issues
in the section below.


----|  6.  An Example Model for an Enterprise Security Monitoring System


Employing a holistic approach to attack detection is especially useful in
enterprise environments.  In such environments, the large number of event
generators can report such a large amount of data that the task of detecting
attacks within that dataset can only realistically be achieved
programmatically;  that is where holistic methods can add value.

The "event generators" mentioned above can be any component within the IT
infrastructure that generates information regarding the status of some aspect
of the infrastructure.  The form and function of event generators is
irrelevant to this discussion, although they would likely include host and
network based IDS, RMON probes, firewalls, routers, hosts, and so on.  Each
event generator will employ an event delivery mechanism such as SNMP, syslog,
ASCII log file, etc.  In this article we will abstract out the delivery
mechanism used to transport events prior to processing.

I propose the following model.

The data from event generators can be used to populate a knowledge structure
that isomorphically describes a number of common attack methodologies.  Think
about the ordered set of steps that are carried out when attacking a system;
this is a methodology.  There are a large number of ways in which each step
in an attack can be carried out, but the relationship between the steps
usually remains static in terms of the underlying methodology.

An isomorphism is an information preserving transformation.  It applies when
two structures can be mapped onto each other in such a way that for each part
of one structure there is a corresponding part in the other structure, where
"corresponding" means that the two parts play similar roles in their respective
structures.

A set of structures that map isomorphically to common attack methodologies can
therefore be constantly compared to a structure that is being constantly
populated by event data from the monitored environment.

The process used to determine when an attack is detected would use a
"soft-decision" approach.  A soft-decision process can report partial evidence
when a predetermined amount of a knowledge structure is populated.  A
soft-decision process can also output a level of confidence in the result at
any given time, i.e. it accumulates and integrates data (events) and reports
partial conclusions and the associated level of (un)certainty as new data
arrives.

The advantage in this approach is that an attacker can often hide or obfuscate
components of their attack by exploiting weaknesses in specific attack
detection technologies or by simply being stealthy (remember - we still rely
on reductionist event gathering technologies "underneath"). However, the weight
of data collected within the environment can be used to indicate the presence
of an attack on a higher, more abstract layer, in which seemingly unrelated
changes or events that occur within the environment can be shown to be related
by using codified knowledge of the sequence of events that comprise different
types of attacks (methodologies).

In addition, weaknesses in the ability of individual event detectors to make an
accurate decision about activity (see [Ptacek, 2000]) become less damaging.
Instead of relying on the absolute determination of the existence of an attack,
an event detector can contribute information about what it thinks it _might_
have seen, and leave attack determination to a higher layer.

The attack structure of attacks that employ automated agents as in
[Jitsu et al., 2000], or distributed agents as in [Stewart, 2000], will likely
be the most simplistic to codify as they employ techniques based on programmed
internal rules.


----|  7.  Concluding Remarks


The difficulties involved in performing security monitoring of enterprise
environments has driven the recent demand for outsourced managed security
monitoring services.  Companies such as Guardent (www.guardent.com),
Counterpane (www.counterpane), and Internet Security Systems (www.issx.com) all
offer managed security services.  These companies are employing technologies
which are based in part on a holistic approach, for example - those described in
[Counterpane, 2001].

The individual components of an attack, such that an individual event generator
might detect, are not "context free".  The reductionist idea that each
component within an attack contributes to the entirety of the attack in a
manner that is independent of the other components, must be rejected.  The
holistic concept is that an attack cannot be considered to be built up from the
context free functions of its components (a declarative approach);  rather, it
is considered how the components interact (a procedural approach).

From an attackers perspective, it will soon not be enough to obfuscate against
detection by specific technologies.  Attacks that attempt to shield themselves
against detection by specific approaches to intrusion detection (for example -
by modulating shellcode to escape detection by specific signatures), and/or
against detection by specific products, will become less effective.  The next
generation of security monitoring and intrusion detection technologies will
employ a strategy based on holistic methods in which the underlying form and
structure of attacks is codified and can subsequently be recognized.


----|  8.  References 


   [Counterpane, 2000]  Counterpane Internet Security, Socrates and Sentry.
                        http://www.counterpane.com/integrated.html

    [Hofstadter, 1979]  Douglas R. Hofstadter, "Godel, Escher, Bach: an Eternal
                        Golden Braid", 20th-Anniversary Edition, Penguin Books,
                        2000.

  [Huang et al., 1998]  Ming-Yuh Huang and Thomas M. Wicks, "A Large-scale
                        Distributed Intrusion Detection Framework Based on
                        Attack Strategy Analysis", Proc. 1st International
                        Workshop on the Recent Advances in Intrusion Detection,
                        Louvain-la-Neuve, Belgium, September 14-16, 1998.

  [Jitsu et al., 2000]  Jitsu-Disk, Simple Nomad, Irib, "Project Area52",
                        Phrack Magazine, Volume 10, Issue 56, File 6 of 16,
                        May 2000.

         [Moore, 2000]  http://independent-sun-01.whoc.theplanet.co.uk/enjoymen
                        t/Books/Interviews/2000-07/alanmoore210700.shtml

 [Ptacek et al., 2000]  Thomas H. Ptacek and Timothy N. Newsham, "Insertion,
                        Evasion, and Denial of Service: Eluding Network
                        Intrusion", January 1998.
                        http://www.securityfocus.com/data/library/ids.ps

[Staniford-Chen, 1995]  Stuart Staniford-Chen, "Distributed Tracing of
                        Intruders", Masters Thesis, University of California,
                        Davis, 1995.

       [Stewart, 2000]  Andrew J. Stewart, "Distributed Metastasis: A
                        Computer Network Penetration Methodology", September,
                        1999.  http://www.securityfocus.com/data/library/distri
                        buted_metastasis.pdf

  [Zhang et al., 2000]  Yin Zhang and Vern Paxson, "Detecting Stepping Stones",
                        Proc. 9th USENIX Security Symposium, Denver, Colorado,
                        August 2000.


|=[ EOF ]=---------------------------------------------------------------=|