Security Laboratory

Security Laboratory


Interview with Stephen Northcutt


Dave Elfering
David Elfering is the Director of Network & Information Security Werner Enterprises, Stephen Northcutt is President of The SANS Technology Institute, a post graduate security college

Stephen, as a long time practitioner and instructor
[1], do you think security has changed from 1997 to 2007?
Well, Dave, in 1997 you had to write or compile on your SunOS 4.1.3 most tools, and you could name all of the tools.

Today, many organizations buy more commercial software and nobody can name all the tools or even all the vendors. The scale was also radically different: the cost of assurance as a portion of the IT budget was lower, although it was hard to manage and non-uniform.[2,3] And, the percent of compromised systems was also far lower, primarily because the worm was still being invented and to be compromised, in general, someone had to personally take the time to hack your system.

Thank you, Stephen, do you see changes in the skill sets of security people in the past 10 years?
When there were only a few security people, if you didn't have skills the only place you could hide was in US Government certification and accreditation. Today, the majority, repeat, the majority of the people in the so-called security field simply do not have any hard skills.[4] So, you ask, how do they get things done? Well, they send trouble tickets to operations. You would be amazed how many cases a ticket has to be passed over three times before the work actually gets done.

One other nice topic dear to my heart is using open source or semi-open source tools; what do you think of tools like Nessus, Splunk, Xenos, etc., instead of six figure commercial stuff? My new analyst has probably saved me $200,000 by doing this and is producing better overall results.
I have to agree. At SANS we use a lot of open source tools, it is important to keep in mind that requires a certain amount of sophistication from your sys admin staff, but you want your staff to have a certain amount of sophistication so that works out. And, of course, we want to add Jabber, Snort, and Nagios to the list.

Consider this quote from Epa.gov: "The Romans were aware that lead could cause serious health problems, even madness and death. However, they were so fond of its diverse uses that they minimized the hazards it posed." Does this sound any different than the way we are treating our information systems?
Nice, I just finished reading Rome INC[5] so this is timely. And I think you have it exactly right.

The second round of scathing testimony on the Hill this week said, "Foreign intelligence agencies must weep with joy when they contemplate U.S. government networks," said James Lewis, director of the technology and public policy program at the Center for Strategic and International Studies, who went on to describe "an unparalleled looting of U.S. government databases." And also: "We are a nation unprepared to properly defend ourselves and recover from a strategic cyberattack," said O. Sami Saydjari, president of Professionals for Cyber Defense.

So, our lovely shiny technology can certainly be used against us if we let it. Just like the lead pipes and lead goblets in Rome. A bigger problem for Rome than lead though, was greed and avarice, the inordinate love for riches.[6] When the younger generations in the hundred years of peace had not experienced the fight for liberty (or expansion) they became soft targets, this is what is happening to us in IT today.

Years ago we used to think that overall we were smarter than the hackers. Has that changed? Are we in for the fight of our lives?
Not me, I have never felt smarter than hackers. Ten years ago I was in a US DoD facility and was continually impressed with the novelty of attacks. Why the best and brightest go to the dark side used to confuse me, but not anymore. They do it for money - that is the primary impact of organized crime in the space. So yes, we are in the fight of our lives. Today, we pretty much have to accept some percent of loss of purity on systems and reload the OS from time to time.

TJ Maxx recently lost credit card information for over 40 million accounts. When will people start to push back against these issues?
People will not push back! Credit cards don't feel like money, and when the letter comes saying your card was compromised, maybe you had a bit of outrage, but by now this is your third letter. If 45.7 million credit cards[7] compromised will not spur outrage, nothing will. However, the problem is self leveling; either the liability insurance companies will force a change or the credit card industry will. According to attorney Ben Wright, "Retailers, financial institutions and other players in the credit card industry have a forum for working out their relationships."[8] By the way Dave, there is already a calculator[9] to figure out the expense of a data breach.

Do you think corporations are proving themselves incapable of self-governance with regard to information security?
Some like TJ Maxx, Neiman Marcus, and Astroglide, yes, and that is self evident from recent news stories. However, other companies are all over security; which ones will still be in business in three years or five years? And what takes these companies off the corporate landscape is a number of subtle things, "A study by the Ponemon Institute found that when a company announces a security breach, its stock price drops between 0.6% and 2.1%."[10] 2.1% is not that much, but factor in the distraction caused by the press,[11] and some staff turnover, and the loss of valuable intellectual property to competitors that didn't have to develop it themselves. You can probably never prove the reason for a company failing is poor security, but there are some hints, "Complacency, it seems, abounds. A large proportion of security execs admitted they're not in compliance with regulations that specifically dictate security measures their organization must undertake or risk stiff sanctions, up to and including prison time for executives."[12]

Do you think that the biggest issue with security is software engineering and quality or with how companies/people treat information?
Very perceptive question, Dave. Clearly there is a gap in software engineering where developers are not aware of what they can do in the software development lifecycle to engineer security into the product. "Governments, companies, and educational institutions are doomed to deal with endless streams of software vulnerabilities unless programmers learn to write much more secure code.

Several initiatives are underway to improve secure programming skills and knowledge. Symantec, Oracle, Microsoft, and a few other software companies are conducting short courses for their programmers; software firms like SPI Dynamics and Fortify Technology are working with universities to provide automated, real-time feedback to student programmers; and dozens of universities are creating elective courses on secure programming. Yet, even if all of those initiatives are successful, they are unlikely to affect even two percent of the existing 1.5 million programmers already in the work force or those who will be entering the work force over the next five years."[13] The new software exams by the Software Security Institute is a step towards building metrics that can one day help improve the situation.

Deming revolutionized the auto industry by teaching the Japanese about quality control. To me software seems a lot like cars in the 60's; great fun but lacking safety and overall quality (just buy a new one when the old one falls apart). Do you think someone will do for software what Deming did for automobiles and will it come from outside the U.S? (As it was for automobiles)?
Sure, and the 14 points will work just as well today as they did when Deming first put them to paper. There is a great summary of these here;[14] read through the list, think about the software security problem, and I think you can see the roadmap to improvement.

Are we security people guilty of promising things we can't deliver, just to be recognized and given a spot at the executive bar? Do you think we assess true business risk consistently?
The C*O inflation is clear evidence that we aren't getting the job done. This is actual text from an email I got. "As a CAE, I look up to the CEO, CFO, and COO, then sideways to the CPO, CRO, CMO, CIO, CSO, CAO, and down to the CJO (chief janitor)." This is a strong indicator we are not getting the job done and that organizations are thinking, security is too big a problem; we have to slide it up. However, it is not too big of a problem, we just are not doing what we need to do.

Worse, instead of latching onto the five architectures of Defense in Depth[13] or the 14 points of Deming and starting to fix the problems, people just keep making up new titles and acronyms like GRC, which right now stands for a great security web site, Gibson Research Corporation,[14] but may soon stand for Governance, Risk, and Compliance.

How can we deliver security without established methods to measure it? Without metrics are we just do-gooders who continually tell people that a 3 headed sea monster is going to eat them? (FUD)
Sure thing, I am excited about the new Security Metrics book by Andrew Jaquith and web site,[15] this should be good times. Metrics is as metrics does, the people that crave metrics can get them and use them to make decisions, but I tell you what Dave, it is interesting bringing up metrics when you are in a hiring interview. The overwhelming majority of people I interview cannot even begin to discuss where to get sources of information and that is scary.

There is a study from the IT Process Institute showing that there are 21 foundational controls responsible for the majority of effective control. Have you had a chance to study this? And if so, what are your thoughts about it?
Sure thing, CoBIT has 312 controls, and I love the way ITPI is pushing to get to the ones that matter. You can't fight the Pareto principle.[16]

Do you think companies are improving in their ability to quantify their risks and put appropriate (reasonable) controls in place?
This goes back to the dividing line some are choosing. I am sorry that I cannot name names, but those that take security seriously that I have worked with are adamant that we do not mention them. However, I can tell you they take no shortcuts in the protection of their intellectual property.[17]

The majority, however, are the TJ Maxx and Department of Homeland Security's of the world. It is easy to tell though which side of the line a company is on if you just interview the staff. If no one actually knows how to do security at a technical level, then that company is totally screwed. It reminds me of Star Control III where the "K'tang are portrayed as an intellectually inferior species using advanced technology they do not fully understand."[18]

I've read an article suggesting that anti-virus is no longer effective against the new malware. Is anti-virus an afterthought as it works today?
Well as Anton Chuvakin put it, "I just wanted to express my genuine shock about how poorly the tools, built for blasting away the threats of the 90s, fare against the threats of 00's."[19] There is an article by Drew Robb worth reading, "The situation is so bad that none of the AV/AS vendors can remove regenerating malware or detect rootkits."[20] The short answer is we are just going to have to rebuild operating systems from the ground up every six months or so, and more often if the user insists on surfing the web as administrator. Of course none of the computer vendors want to ship media anymore, so that is going to be a challenge.

How long until most of the world's computers are part of 2 or 3 botnets?
Kind of unlikely for the present, and I would not even want to try to guess what the five year mark looks like. Right now, there is too much competition, so if the number started to shrink, the competition would increase and sections of some bot herder's network would be ripped off. There is also the command and control problem; the excellent research by Kevin Bong and John Brozycki points that out. A sophisticated bot herder would rather have four or five parallel bot nets with command and control operating in four or five different countries than one monster net pointing right to him.[21]

Still, this is a fun question. We are headed for search engine convergence when all the world relies on Google, Wikipedia has surpassed Britannica, and there are two major ISPs at least in the USA, Verizon and ATT, the future could be very interesting: all that knowledge and information under the control of very few entities, I guess it would make sense the underworld might also converge. Maybe that Interface[22] scenario is not so far fetched.

Any cool projects coming up for SANS?
Sure, we are heads down on VoIP and Secure Web Applications. There is alway something going on! Thanks for the interview, seems quite odd to have the tables turned on me like that.


1. http://www.sans.org/training/instructors.php#Northcutt
2. http://infosecuritymag.techtarget.com/2003/mar/cisosurvey.shtml
3. http://www.pwc.com/Extweb/ncpressrelease.nsf/docid/021E13BF834E1FB680257206002B6AB6
4. http://blogs.ittoolbox.com/security/dmorrill/archives/top-10-information-security-skills-10317
5. http://www.amazon.com/Rome-Inc-Multinational-Corporation-Enterprise/dp/0393329453/
6. http://www.newadvent.org/cathen/02148b.htm
7. http://news.com.com/TJX+says+45.7+million+customer+records+were+compromised/2100-1029_3-6171671.html
8. http://www.sans.edu/resources/leadershiplab/tjx_security_comment.php
9. http://www.tech-404.com/calculator.html
10. http://www.pr-squared.com/2007/04/good_ol_fashioned_pr.html
11. http://www.networkcomputing.com/channels/security/showArticle.jhtml?articleID=199000300
12. http://www.cio.com/article/24979/The_Global_State_of_Information_Security_
13. http://www.sans-ssi.org/
14. http://www.mftrou.com/edwards-deming.html
15. http://www.securitymetrics.org/content/Wiki.jsp
16. http://en.wikipedia.org/wiki/Pareto_principle
17. http://www.sans.edu/resources/leadershiplab/what_is_ip.php
18. http://en.wikipedia.org/wiki/Universal_translator
19. http://chuvakin.blogspot.com/2007/04/more-on-anti-virus-and-anti-malware.html
20. http://www.enterprisenetworksandservers.com/monthly/art.php?2668
21. http://www.sans.edu/resources/student_projects/200704_001.doc
22. http://www.amazon.com/Interface-Neal-Stephenson/dp/0553383434/