Security: Incident response is everything

A competent CSIRT will mitigate the damage of a security breach

This year so far has had a number of information security incidents that have cost likely billions of dollars, and they've cost even more as damage to the reputations of some well-known businesses. US retailer Target, Home Depot, Apple, Sony, JP Morgan Chase, and the list goes on.

But for each attack that makes the mainstream news headlines, such as the Russian attack on JP Morgan, there are at least thousands or possibly millions of attacks that don't make the news. They usually affect smaller businesses and institutions, but there are many, many more of them.

According to the 2014 Symantec Internet Security Threat Report, in 2013:

• Website, web database, and point-of-sale system attacks caused the identities of 552 million users to be stolen, with an average of 2,181,891 identities per breach. 552 million online identities is nearly one in five of the roughly 2.8 billion people estimated to use the Internet that year.

• One in 196 emails was estimated to contain malware.

• One in 566 websites were estimated to contain malware.

• 351 web browser vulnerabilities were found in Microsoft Internet Explorer, Mozilla Firefox, Google Chrome, Apple Safari, and Opera. As most lesser known web browsers share a layout engine, such as WebKit, Gecko, or Trident with one of the top five web browsers, many vulnerabilities that affect a major web browser also affect many less common web browsers.

Something as simple as a single malware file, malicious email, or network vulnerability can cost a business a lot of money in stolen data, litigation, or even mere computer and network downtime. All businesses with computer networks are at risk of information security attacks.

Prevention is indeed the best medicine. Creating and enforcing a well-designed IT security policy makes your business less likely to be subject to attacks, and less often. But absolutely nothing can be made 100% secure.

So, the perfect complement to sound prevention with a good IT security policy is well prepared incident response. A well trained and vigilant Computer Security Incident Response Team is crucial.

There will be times when attacks will slip through your security measures, and when they do, appropriate incident response from a competent CSIRT will mitigate the damage and get your systems back to normal. Incident response is everything.

How to build a CSIRT

Choosing the right people for your CSIRT is essential. A CSIRT may consist of any or all of your employees and executives who have computer security experience and credentials.

They may include people with roles such as Network Administrators, System Administrators, IT Department Managers and Directors, and Chief Technology Officers who have certifications such as CompTIA Security+ or CISSP. Larger businesses and institutions may have CSIRTs containing roles such as Information Security Architects, Directors, and Auditors, Cryptographers, Cryptanalysts, Software Developers, and Disaster Recovery Specialists.

The largest businesses and institutions may have CSIRT members who are specifically employed for incident response. In a nutshell, a smaller business or institution typically has a CSIRT that consists of all employees and executives with a computer security background, whereas larger organizations typically have CSIRTs with a combination of dedicated members and people in different information technology roles.

Either way, for the sake of confidentiality and efficiency, I recommend that your CSIRT should consist entirely of people inside of your organization.

What a CSIRT Needs

Effective application, operating system, and network appliance (such as firewalls) logs must be kept. At least once every business day, make sure your network administrators check to assure that your software is logging properly! Log analysis programs must also be used unless you have a very small network.

When an incident strikes, whether or not it’s your logs that detect it, they must be reviewed in all occurrences. As much as possible, make sure your logs have lots of automated backups.

It’d be helpful to have clients and servers that record logs not only store to local disks, but to automatically copy files to external RAID configurations as well. Proper log recording and analysis can make or break the work your CSIRT must do.

As your CSIRT learns about a specific incident, all of their findings must be properly documented, in as much detail as possible. That’s vital not only for compliance and auditing purposes, but also perhaps for changing IT policy and procedures as threats and technology evolves over time.

Make sure that each CSIRT member who contributes to an incident report can be traced to their user account. CSIRT members may be in many of the same user groups, but they should not share any single user accounts. Each single CSIRT member must be accountable in a legally verifiable way.

No matter who your vendors are, patches to applications and operating systems may correct security vulnerabilities, introduce new vulnerabilities, or both.

If your network has a large number of clients and network appliances, you may need to operate applications from your main servers to centralize patch management throughout your network, rather than letting individual machines handle patches on their own. If a patch introduces a security vulnerability that enables an attack, you must be able to determine the specific patch for a useful bug report. Submit bug reports as frequently as possible.

Your network should be penetration tested by a third party at least once a year, when significant changes are made to your network design, or whichever scenario is more frequent. Whether the pen testing is black box, grey box, or white box, the pen tester’s reports must be shared with the entire CSIRT. It’s always best to err on the side of more documentation than less, especially when compliance is considered.

Incident documentation alerts, and vulnerability handling are reactive CSIRT services. Interfacing with penetration testers, intrusion detection system management, security audits, security tool development, and pre-incident log analysis are proactive services. An effective CSIRT will balance their efforts between reactive and proactive services (PDF).

All proactive services must have set routines and schedules, so that all necessary activities are never forgotten or neglected. Routine proactive procedures will reduce possible risks and losses when reactive work must be done after an incident occurs.


Even if your CSIRT consists of people with other roles in your organization, such as network administrators, it’s prudent for CSIRT members to have roles specific to their CSIRT.

The greatest responsibility and authority should be assigned to a CSIRT Team Leader. Their job is to coordinate the activities of all of the other CSIRT members, and to make final decisions when there’s little consensus.

The member who reports directly to the Team Leader should be a CSIRT Incident Lead. They will coordinate the response to each and every incident.

Most CSIRT members should be Associate Members. They all report to the Incident Lead. Associate Members will have various roles with specific scopes. Here are some Associate Member roles:

• Information security incidents sometimes become news or people may talk about them on the Internet. To mitigate possible reputation damage, one member should be a Public Relations Officer.

• Your team should have a Legal Representative who will be a lawyer with IT security policy knowledge and technical incident response experience. An effective Legal Representative in your team can mitigate litigation risk. When applicable, they’ll also cooperate with law enforcement.

• At least one member should be an IT Contact. They’re the link between the CSIRT and the IT department. All incidents affect the IT department, so communication and coordination is vital.

A CSIRT is a must in the 21st century. Proper preparedness and incident response can mitigate the worst. Never rest on your laurels, because absolutely nothing is completely secure. Your business can never be too vigilant in information security.

Kim Crawley is a security researcher for the InfoSec Institute, an IT security training company.

Join the Computerworld newsletter!

Error: Please check your email address.

Tags securitytraining

More about AppleCompTIAGoogleHome DepotindeedJP MorganLeaderMicrosoftMorganMozillaSonySymantecTechnologyTrident

Show Comments