June 1, 2004
This is the seventh in a series where I am trying to pin down what the true effect of a given certification is. If you hold a GIAC or other information security certification and are willing to be interviewed by email, send a note to email@example.com
Stephen Northcutt - The SANS Institute
Q. I was wondering how SANS decides which courses to offer at the respective conferences. Looking at the schedule of training events for the next several months, I was surprised to see few, if any, scheduled classes for GISF and GSLC. Since these are the Certs where I am a grader, I expect that the limited number of classes will mean that there will not be that many certification attempts in the coming months. (Other than the online students and local mentor program participants, of course.) I just wondered how SANS decides which courses to include in the various conferences that are scheduled.
A. This is one of those "best fit" algorithms. Keep in mind that each hotel is different, we have to balance the "large room" classes, such as SANS Security Essentials and the Hacker Track, with courses that fit in smaller rooms, such as Consultant, Introduction to Information Security and SANS Security Leadership. In addition, there is the instructor availability issue, we have many instructors that can teach Security Essentials, but only one that can teach Securing Windows, for example. I disagree that there are few scheduled classes for GSLC, we have already run 4 times this year and we will be running SANS Security Leadership at SANSFIRE in Monterey in July, Network Security 04 in Las Vegas in October, probably at CDI East in Washington DC in December and in Kona Hawaii in January 05.
Q. My question pertains to the direction of SANS' menu of certifications. We currently offer 13 different programs, and it sounds like there are a couple more on tap. My concern is that with the increased number, it may become confusing to the industry with respect to the distinct value offered by each Cert. How does SANS decide when a new cert is warranted and has anyone determined a range of maximum and minimum offerings that we would like to provide on an ongoing basis?
A. We are reaching completion of the courseware for Information Security, but have a long way to go in the other disciplines SANS intends to service: Audit, Legal, System Administration or Operations, Management and Networking. The goal is 30 courses, that are each of the caliber of our existing courses and we believe they will form the basis for a high quality masters degree in Information Science.
Q. What was the original objective of the practical assignment? If the exams are testing the course material, how does the practical differ? Should the practical test only the material presented in class or beyond that scope? If it should test beyond, how far and to what extent?
A. There are exactly two requirements for a practical assignment, it should test the subject matter knowledge of the student and the result should be a benefit to the Defensive Information Community. This is the heart of GIAC. It should not be limited only to the material presented in class, the practical assignment is a learning tool and on the main it works well. Many times, students have told be that while the courses was great and they learned a lot, they learned just as much completing the practical assignment.
Q. When grading papers, how much deviation from the assignment should be permitted?
A. That is a tough question and each track lead should have the authority to adjust this answer to meet the specifics of a specific certification. Any student should have the ability to request permission to substitute a unique practical assignment as long as it demonstrates knowledge of the subject matter and is of value to the community. For me, the second is the most important criteria. We had a saying in the early days that if Marty Roesch should decide to complete the GCIA and submitted Snort as his practical that would pass.
Q. What standard should we be holding the students to, and how far should the student have to stretch to achieve the certification?
A. I hope we can all agree that partially depends on the certification. I realize I caused a lot of pain and confusion with the changes in Track 9, but it stands to reason there has to be an Introduction to Information Security track and that practical should be something that someone just starting out should have an honest shot at passing. In the same vein, I expect the failure rate on the reverse engineering malware certificate will be fairly high and that is OK too, it is probably our most advanced course. We do not want the certifications to become nearly impossible to complete such as the situation that occurred in our Advanced Audit track and we also do not want to have their value diluted. The advisory boards have done an incredible job of keeping the balance and all of the defensive information community applauds your work. According to www.giac.org, since we first started in year 2000, we have certified 6,386 people across fourteen tracks. That is fantastic, it means that if you hold a GSNA, GCIA, or any other GIAC certification it really means something.
Q. In your opinion, since we only see about a half dozen honors practicals a year should the grader's and advisory boards be lowering the standard we are holding students to? To what level?
A. Well, we want honors to be something truly special, I think we can all agree on that. I tend to agree that if the approval rate is 6 per year that is probably a bit on the low side. Maybe we should take a look at our process. But this is one of those situations where we want to make small careful adjustments, not introduce wholesale changes. We do not want to dilute the value of an honors score!