In-Depth

Meet the SIMs

• By John K. Waters
• June 1, 2005

The SIM market is new enough that it’s not easy to make apples-to-apples comparisons of these products. Amrit Williams, a Gartner analyst, offers these criteria to help focus your evaluation.

Product Scope
What information does it collect and what does it report against? Does it capture data from the network devices, the operating system logs, the database management system logs, the firewalls, the intrusion detection system sensors—in other words, all of the security resources that I need to capture data from?

Also look at things that are important for your particular environment. “One organization may be concerned primarily with correlating firewall log data and IDS log data,” Williams says. “Another organization may focus on HIPAA requirements and care only that they are capturing log data from ID and access management applications.”

Data Collection
How is the data collected? Does the SIM solution require agents to be installed directly on the monitored system, or is there some type of aggregation point, such as a syslog server, from which the data is collected? Can you invoke data collection from a command-line interface with the tool itself when you need to? When is the data collected? What’s the period of time during which the data is pulled? Does the product integrate with or have an API that allows other systems to collect data from it?

“Keep in mind that we’re talking about potentially hundreds of thousands of events being generated every day and filtering that down to the one to 20 things that an organization really needs to take action on,” Williams says.

Correlation Capabilities
You’re going to want a SIM solution that operates against current event data and data that is more static. For example, firewall, IDS and host logs generate new data into the system frequently; that data needs to be correlated with things like vulnerability assessment data or asset classification data, which isn’t changing much in comparison.

Taxonomy
How well does the product map information from disparate security sources to a common classification? A quality taxonomy helps aid in pattern recognition and improves the scope and stability of the correlation rules.

Incident Management and Workflow
Security organizations don’t usually have the authority or the responsibility to do very much to the network or the desktops. They capture this data and they advise the operations team. This is especially common in large companies. So at some point, the security people are going to need to pass the data collected by the SIM to another organization in the enterprise.

“There should be some workflow embedded in the product to enable efficient incident response,” Williams says. “If I’m a security engineer and I see evidence that the firewall is under attack or I notice that our network is so out of compliance that a critical event might occur, I need to be able to be able to let [the operations people] know so that they can act.”

Scalability
Scalability is generally achieved through a hierarchy of SIM servers. Tiers of systems aggregate, correlate and store data, which can be reported against centrally. Then there are servers with specialized functions such as being able to act as a database for reporting and display or for correlating data.

Deployment Flexibility
Does the SIM require you to install agents on the monitored systems to collect data? The security organization may not be allowed to install agents everywhere.

Enterprise Administration
Does the SIM offer strong authentication and role-based access controls? Does it support integration with enterprise directories? “That’s been a pretty strong requirement for a lot of large organizations,” Williams says, “because they’re usually adopting these enterprise-style directories.” Does it perform auto discovery? Does it provide asset classification features to help you to prioritize threats?

“These things aren’t specific to SIMs,” Williams adds, “but should be considered in the evaluation of many security products.”

Embedded Security Knowledge
How much security know-how does the vendor bring to the table? How sharp is the support team? How savvy are the tools? These are fair questions to ask during any product evaluation, but considering the relative youth of this product category, they’re essential when assessing SIMs.

Look for things like a library of pre-defined correlation rules, some pre-defined corrective actions and pre-defined threat analysis; some regulator compliance reports, and threat and vulnerability information displayed in the context of an incident.

Back to feature: Traveling at a Zillion Events Per Second