Analysis of the Incident Handling Six-Step Process - GIAC Research in the Common Body of Knowledge

Analysis of the Incident Handling Six-Step Process: Category: Telecommunications and Network Security
Author: Jim Murray
Date Added: February 6th, 2007
Relevant Courses: SANS SEC 504 Hacker Tech

Introduction

Incident handling is sometimes portrayed as the glamour job of information security. What is more exciting then thwarting a Denial of Service (DoS) attack by a Russian syndicate or nabbing that hacker before he makes off with your company's credit card numbers? In cinema and television, the incident handler is usually portrayed as the lone gunman with MacGyveresque skills that can face and defeat anything that comes his way.

Although these cinematic images are certainly exciting, to properly handle an incident, you need more than a turkey baster and some bubble gum. SANS defines the incident-handling process as a six-step methodology, and this paper provides an analysis of that six-step process to help you understand what it takes to be successful when an incident arises.

The Six—Step Process

Step One-Preparation

The preparation phase is where you spend the bulk of your time as an incident handler. When discussing the skill sets required to be successful, Incident Handling has often been compared to emergency medicine. Both are usually performed in a highly stressful environment and require quick thinking and a calm presence. The amount of preparation you do before an event occurs is the most important factor in determining how successful you can handle an incident. Paramedics spend approximately 70 percent of their time preparing for emergency calls. The same should hold true for the incident handler. In the Preparation phase of incident handling, many things need to be accomplished. The following list contains items that the incident handler should have in place before an incident occurs:

Establish applicable policies—Policies are the rules by which an entity operates. Incident Response policies should be in place to identify who is responsible for responding to an incident. Policies should also clearly state what the company's stance is regarding monitoring and privacy. This is an important step because having the appropriate policies in place protects both the incident handler and the company.
Build relationships with key players—Although working alone can make for great movie fodder, in the real world of incident handling, it is a death sentence. If you talk to a successful incident handler, you find that they have a network of contacts that would make the people at LinkedIn proud. The following is a list of people that you should get to know well because, inevitably, there will come a time when you need their assistance.
- Law enforcement—Either local, state or federal, get to know your law enforcement contacts.
- Human resources—You HR group can help with those pesky policy issues and gray areas.
- System administrators—This is a group that you might need to rely on for their technical expertise and they can help gain access to systems that are involved in the incident.
- Legal counsel—Because you don't want to have to update your resume prematurely or have to trade smokes for favors (end up in jail), you should have legal counsel.
- Fellow incident handlers—If you have not seen a particular incident before, chances are that someone out there has. It's best to have access to many people with incident-handling expertise.
Build your response kit—This can be a duffle bag or a small carry-on suitcase. Regardless of what it is, this is what you have with you whenever you work an incident. You want to make sure that you spend enough time putting this together, so that you are ready at a moment's notice. You should never steal from your response kit. Sometimes we are testing something or working on an issue and we need a network cable or installation software and know it is there in our response kit. We tell ourselves that we are just going to borrow it and put it back as soon as we are done. Don't do it because you know it will never make it back there. Here is a list of things that you should consider having in your response kit:
- Network cables—Include various sizes, both crossover and straight-through
- A small hub or tap
- USB jump drive or external hard drive
- Response laptop-This laptop should have everything you need on it, for instance, checklists, forms, response software
- Various peripheral cables—USB, Firewire, parallel, serial, console, and so on
- Clean binaries and diagnostic software
- Call list
- Notebooks, pens, pencils, and small audio recorder
- Plastic/anti-static bags for evidence
- Forensic software and imaging media
- Blank CDs for burning software from the response laptop
Create incident checklists—Incident checklists are like memory joggers. They are there to help keep you on track. During an incident, things can get pretty hectic, and having checklists are a way to help you remember things that you might forget in the heat of the moment. One word of caution though: Don't use checklists as the Ten Commandments that you cannot stray from. When incident handlers do not stray from the checklist as needed, things can get difficult. Checklists are guidelines; if something comes up in the course of an incident that is not covered by the checklist, don't be afraid to work outside the box. Just make sure that you do it in a methodical and calm manner.
Establish communication plan—This task addresses how incident handlers communicate with each other during an incident and what identifies the escalation process. For communication within the team, you need to consider both in-band and out-of-band communication mechanisms. This protects you in the event that there is an availability problem with the in-band mechanism or if there is suspicion that the security of the in-band mechanism has been compromised. If possible, you should have a member of the team that is solely responsible for communication both within the team and with management. This provides a central point of contact and consistency of the messages that are communicated.
Perform threat modeling—This step requires that you think of the kinds of attacks that can occur and how you respond to them. You want to map out the types of incidents that you expect to encounter both technical and physical. This could include DoS attacks, policy violations, compromise of information, compromise of systems, tornados, hurricanes, and so on. After you have a list of the incidents that expect, you should have a plan on how you address these incidents. This plan should be outlined in the six-step process answering the following questions:
- How will we prepare for the incident?
- How will we identify the incident?
- How will we contain the incident?
- How will we eradicate the incident?
- How will we recover from the incident?
- How will we capture the lessons learned from the incident?
Build an incident response team—This is one area that varies from company to company, but at a minimum, you should have a team of individuals that are identified to respond when an incident occurs. In most cases, incident response is not their primary responsibility. When building a team, some of the qualities you want to look for include:
- Technical skills—You want to make sure that your team members are familiar with the technology.
- Organizational skills—Because of the hectic pace of incident handling, it is essential to have someone on the team that can keep things straight.
- Diplomatic skills—During an incident, there are times when compromises need to be made between the various parties involved. Having someone who is level headed and can negotiate between the various parties is an advantage.
Practice, practice, practice—To hone your skills as an incident handler, you need to practice working incidents. One way to do this is to take part in Capture the Flag events at security conferences. These events are sometimes hosted by local security organizations, such as the Information Systems Security Association (ISSA) and Infragard. You can also work with other incident handlers in your area to set up practice sessions.

Step Two—Identification

In this phase of the process, you determine whether or not an incident exists. To start off, let's define the difference between an event and an incident. An event is defined as any observable occurrence in a system and/or network. An incident is defined as an adverse event in a system and/or network or the threat of such an event. Essentially, an event is anything that happens in your environment and an incident occurs when that event threatens or actually does harm to your environment. In a lot of cases you are called to action because someone sees an event or events that he believes are suspicious. Your job is to evaluate those events objectively and determine if they truly define an incident.

In this phase, it is extremely important that you don't allow outside influences to cloud your judgment. There are several times that I have been called to the scene by an excited end user or system administrator that is certain that she has uncovered nefarious activity. These people supply you with all types of conjecture as to what they think happened, and sometimes it is easy to get sucked into their excitement. Don't do it. Gather all of the facts and make your judgment based on those facts.

Step Three—Containment

After you have identified that an incident has actually, the next step in the process is to contain that incident. During the containment phase, you want to ensure that the incident does not get any worse; if your company is planning on pursuing legal action, this is the phase in which you gather evidence. Containment is the step of the process that can at times get political, especially when dealing with production systems. To contain the incident, you might want to remove the system from the network; however, because it is a revenue generating system, management might not allow this to take place. This is where the negotiating begins and it is important that you or your team has discussed these scenarios in advance with management in the preparation phase, so that you are prepared for them when they arise. Some tasks that occur during the containment phase include:

Prevent further contamination of the system or network—This is a balancing act because you want to stop the bleeding, although you also want to make sure that you preserve the state of the system as much as possible in case of litigation. Some of the tasks that can accomplish this include:
- Remove the network cable or isolating the system on a separate VLAN
- Use a firewall or access lists to prevent into or out of the system
- Change DNS entries to direct traffic away from compromised system
Preserve Evidence—To do this, you might image the entire system or part of the system or capture volatile data, such as running process, ram, network connections, and so on. Whatever method you use for preserving evidence, make sure that you are using clean binaries and document everything that you do. In some cases, it is inevitable that a task you perform will change something on the system. Be prepared to explain what changed and why you performed that action.

Step Four—Eradication

Now that you have contained the incident, you need to begin the clean up process. It is during this phase that you analyze the information that you have gathered to determine how the attack took place. To prevent the incident from happening again, it is important for you to understand how it was carried out.

In some cases, you might be able to eradicate the attack without having to rebuild the system. In most cases, though, especially with malware or rootkit attacks, the only way to truly be assured that eradication is successful is to perform a complete rebuild of the system. If this is the case, make sure that the media you are rebuilding with or the backups you are rebuilding from are not compromised as well. When Code Red attacked in June of 2000, many companies attempted to recover their systems from backups only to find that the backups of the systems were also infected with Code Red.

After you have restored the system, make sure that you perform a vulnerability analysis on the system. This ensures that you have successfully eradicated the threat and have not introduced new vulnerabilities to the system.

Step Five—Recovery

The recovery phase of this methodology is where you place the system back into the production environment. In this phase, you work with your QA and business partners to validate that the system recovery is successful. This involves testing the system to make sure that all business processes and function are back to normal.

You also want to monitor the system or process to ensure that the system is not compromised again and to look for additional signs of attack.

Step Six—Lessons Learned

In this final step, you utilize what you learned during the handling of the incident to enhance and improve your incident-handling process. Unfortunately, this is the phase of the process that often gets overlooked or simply is not done. Many times, we are relieved to have systems back to normal or we get busy trying to catch up on things that were not done while we were working the incident that we don't make the time to document the incident or look for ways to improve the process.

In this phase, you should:

Complete the incident report and present findings to management.
Look for ways to improve the process both from a technical and administrative aspect.
Have a clearly defined plan for implementing these improvements.

Summary:

In the third episode of the Lord of the Rings trilogy, there is a scene just before the start of the epic battle where one of the characters says, "I don't want to be in a battle, but waiting on the edge of one I can't escape is even worse." There is no doubt that during your information security career, you will encounter an incident. By following this six-step methodology, you ensure that you are prepared when the battle takes place and the outcome is victory.

^ Top

Domain 2: Telecommunications and Network Security