Security Musings

Security Musings

Musings on Certifications

Collected musings on information security certifications.

Other Related Articles in Musings on Certifications

Response to SecurityFocus Op Ed, "It is time for a new certification"

By Stephen Northcutt
A number of people have written me for my response to the SecurityFocus Op Ed, It is time for a new certification. I have the following personal comments and opinions.

In terms of research into assessment and certification, we have learned a lot at GIAC through the years. And, the process of applying for ISO certification has been really educational.

I think we all know and agree there are limits to what we can do with multiple choice tests. However, those of you that have been with the GIAC program for a long time have probably noticed measurable improvements in test quality. There are a large number of psychometric scripts running behind the scenes evaluating quality in a number of ways. A year ago, I would have dismissed psychometrics as voodoo; now we are paying for training to get a number of our staff educated to work at the practitioner level.

In addition, the Director of GIAC, Jeff Frisk, has led the charge to bring in scenario questions. We are monitoring these carefully, but they do allow you to test more than regurgitation. They allow you to test knowledge.

I believe GIAC is doing better than any other family of security certifications in actually testing the job someone might do. This is becoming known as role based training and role based assessment or certification, and it is important.

What job does the CISSP test your qualification for, what job does CISSP training prepare you to do? Now, to be sure, it makes sense to have an exam for minimal competence, can you speak the language of security, do you understand the core concepts of security. The Common Body of Knowledge does a fine job of that, and when they developed it they were far ahead of their time; there were clearly men and women of vision associated with the ISC2 to lead that charge.

We feel the same way about the GIAC Security Essentials Certification; though it is not a job per se, we feel it establishes the minimum baseline for the knowledge, skills and abilities that a person with hands on responsibility for system should know from a security perspective.

However, the GSEC is only one of over 20 GIAC certifications and the overwhelming majority of GIAC certifications are based on a role, a set of tasks that one actually accomplishes in the workplace. For instance, when I was doing work for the Missile Defense Agency (BMDO) I was helping them with their perimeter and also with their intrusion detection. So, I had two primary roles, and there is a GIAC certification for both roles, Perimeter Protection In-Depth, the GCFW and Intrusion Detection In-Depth, the GCIA.

Further, there is a need for more senior people to demonstrate mastery and integration of multiple roles and that is the GSE, GSM, GSC and so forth. And, those certifications are not just multiple choice, they are hands-on and require discussion of theory as well.

But you always want to do more, to push the envelope; that is why I am really excited that at Network Security 2007, in conjunction with White Wolf Security, we will have our first Marathon Capstone. It is a cyber exercise, not a capture the flag event, but, rather, a multiple day hands-on event where participants will have one role and they will be evaluated on their performance in their role. A seasoned (all GIAC certified) set of penetration testers will provide the attacks against the various stations. We have already formed an advisory board to determine what "passing" is and they will start to work as soon as we finish with all the legal paperwork. Participants that pass will be given a certificate with a joint GIAC/White Wolf Security trustmark.