I’m happy to announce the launch of my new podcast, the Reality Check Security Podcast with Gary McGraw:

The Reality Check Podcast with Gary McGraw focuses directly on software security practitioners and practical software security. Reality Check’s sister podcast, the Silver Bullet Security Podcast with Gary McGraw, follows a free form interview style tailored highlight the ideas and experience of security gurus. By contrast, Reality Check is concerned with practical questions centered on running large-scale software security initiatives in the real world.

Reality Check targets experienced leaders working to solve software security problems in large organizations every day. We use a standard script to guide each conversation with questions about history, methodology, best practice, and measurement. We plan to interview leaders of mature software security programs and leaders of programs just getting started.

Your feedback is absolutely welcome. Please subscribe to the series through or RSS feed or through iTunes.

In the last entry, I responded to dashboard initiatives and organizations’ attempt to measure. Examples focused on vulnerability-detection because a lot of software security expenditure has as well.

Some practitioners, as well as software security managers, have realized that a focusing only on uncovering the problem won’t get it fixed. They’ve begun to document secure sample code, construct secure-development frameworks, and even harden existing open-source frameworks in order to make secure application development easier. This trend cuts across those IT shops supporting non-software businesses (such as finance, insurance, gaming, and the like) as well as software vendors. Let’s look at the 2nd surprising finding from SAMM data gathering, as published in Gary’s InformIT article:

8. Secure-by-default frameworks can be very helpful, especially if they are presented as middleware classes (but watch out for an over focus on security “stuff”).

Secure-by-default frameworks for aiding in development are absolutely my favorite part of software security. Not only do they make sense as part of any program, they represent an essential corner-turn from a hopeless attempt to keep developers abreast of what they’re doing wrong to the downward push towards the finish line by helping developers release better software. In 2001, plainly sick of documenting and demonstrating the same vulnerabilities over-and-over again in meetings and reports, I begun to talk about, seek money for, and outline what such a framework might look like for use of Servlets in the Java EE platform. Frankly, sometimes it’s just plain easier to describe how to do something correctly than it is to conceive of pen-tests or static analysis rules that will find all instances of vulnerability, some omission, or an incorrect implementation.

But, I’m not the only one that’s been climbing this hill: A broad range of approaches to constructing such frameworks are visible within my own client base.

Some organizations have taken their cue from what the network and host security guys have done in their own spaces. These software security groups see software security as a mapping between vulnerability and the need for control. This drives two sets of behavior:

With regards to the software they build, these organizations tend to strive towards having some central engineering team (or a particular development group) provide a ’security control’ that they can plug in for each vulnerability. This has had two effects in my experience:

1. The ’security framework’ ends up as an amalgam of functions designed in very point-solution fashion. For example OWASP ESAPI’s addCSRFToken() method. (see code)
2. ‘design review’ becomes a checklist activity in which security folk (playing the role of auditor) walk through a list of features they expected to include, such as those API calls described in #1 or at a higher level the inclusion of a particular security package: IE: “You’ve used ClearTrust for authentication–right?”

With regards to the software, infrastructure, and COTS these organizations purchase, they tend to apply a modified checklist (such as #2) during acquisition, and then–in the case in which they’re acquiring a security component itself–add it to the checklist for future reviews.

There are benefits to this level of framework creation/use (namely, it raises awareness of the problem, standardizes solutions making compliance easier to gauge, and does in fact place some useful functionality in the hands of developers they might have otherwise omitted or bumbled themselves).

However, I think this approach misses the mark for several important reasons.

Others build their security frameworks from scratch. Convinced that their organization is different enough to warrant their own unique approach they set to construction. Some organizations cull framework material from ‘centers of excellence’ within their organization: development teams that have already done a great job at something like demanding authentication and invalidating sessions. Others architect the security framework centrally, construct it, and then deploy it mandating development teams integrate with it as they continue otherwise normal maintenance of their apps.

There’s intrinsic benefit and difficulty to a purely bottom-up or top-down approach. If an organization goes purely bottom-up, their difficulty lies in coordinating and architecturally cohering different teams’ widgets into a common framework. The strength they can rely on is buy-in: developers are hard-pressed to complain about what they and their peers have written. The top-down approach lacks buy-in and (at times) practicality, but often enjoys a coherent unified design.

Early in ‘07, I wrote on a model for mixing the two approaches successfully.

Whether executed top-down, bottom-up, or through some compromise, I like this approach better than the one I described above. My biggest issues with it are:

1. It can become a political monster, especially if top-down
2. Being overly unique (ala: “not-invented here” or “I’m a snowflake” pathologies) can be damning

The more top-down an approach is, the more likely developers are to balk at mandates for the framework’s use. “Pfft, but that won’t work the way we built our app.”, they’ll lament. If you hear that, prepare for a deluge of exception forms. ;-)

So, how are people overcoming limitations of an ineffective feature-centric audit-model and the pains suffered from inventing their own framework from scratch? By, “meeting the developer on ‘their side of the river’”, as it was once said to me.

When a developer sits down to write or maintain a web application, they’re doing so within a very rich platform already. The .NET platform has well-integrated set of tools from the webserver up to application-support functionality provided by a single vendor: Microsoft. The Java EE platform focuses on open standards, and as such has a bevy of open-source packages that support the lower-levels of the stack that Sun provides. Simply put: a developer writing a Java EE web app is already conforming to the Servlet framework, if not the Struts, JSF, or some other MVC package. They know the platform’s packages, they’re already using them, and there are tons of great books, tutorials, and examples out there.
This is their side of the river. Demanding they build on top of those packages, but integrate with another package–for the sake of security alone–is folly.

A better answer to creating one’s own security framework–and I’ve seen this in play in more than one organization–is to secure a framework the developers already know. This:

• Maintains current platform design idiom - If Spring is ‘the right design paradigm’–great, you’re using with without having to have invented it;
• Allows consideration of the framework’s threat model - It’s easier to secure a system you know the design/deployment model of, than it is to write ‘the perfect’ security feature in a vacuum;
• Maintains transparency, as best as possible - Those securing the framework can place implementation ‘under the hood’ allowing developers to call functions as normal;
• Moves security ‘lower in the stack’
• Reduces integration points for developers

To me, these advantages are overwhelming. Showing the benefits of faithfully maintaining the underlying framework’s design could be the subject of its own entry entirely and–I feel–places the nail in the coffin of those arguing that security can be represented simply as features. I’ll defer that for now however.

Fixing problems instead of identifying them? Security guys getting traction by helping developers write code and get it out the door faster? Meeting them within the realm of the frameworks they’re already using: hardening their configuration, implementation, and deployment? This works?

Not surprising ;-)

Like Ken and Brian I’m pleased that results from this study are getting out there. I recently participated in the OWASP EU Summit, where Pravir and I conducted a session on the maturity model. The session had valuable industry participation.

One of the things I’ve harped on for years now is that experts need to pull their heads up from whatever they’ve gotten so good at doing in the weeds and avail themselves of some of the lessons learned by organizations that have been working on software security themselves. In 2006, I presented on “building one’s own software security group” and one of my first five slides uncovered a dirty little secret–true now as it was then: Fortune 500 security groups often possess more man-power and budget than security security consultancies. I see this maturity model study and the response/involvement of the people I’ve mentioned here as a giant step in the public recognition of the value security groups within large commercial entities have been providing.

To follow up on Gary’s InformIT article, I’ll be writing a series of posts on why these “Top 10 surprises” don’t come as a shock to us at Cigital and what it might mean for organizations. I’ll go in reverse order:

9. Not only are there are no magic software security metrics, bad metrics actually hurt.

First, a clarification. Gary’s article indicates that:

but even the most advanced programs don’t use any sort of balanced scorecard approach.

I certainly talk to organizations that rely on a scorecard, especially at the executive level, to report on software security activities. What’s interesting to me is the horror stories organizations tell about chasing the wrong metric. First, they lament how tracking “the wrong thing” motivated unexpected (and undesirable) behavior changes in the staff they were trying to effect. I’ve personally seen things like developers manipulating their code to break static analysis tools, signing a training sign-in sheet then leaving for the day, and vulnerability fixes that cause terrible residual risk (But pass a regression test in the form of a penetration testing exercise). Developers ‘hacking’ the system of control? We shouldn’t be surprised.

In my opinion, organizations charged the scorecard hill too early. They were frustrated with how “Fear-driven” security was but simply didn’t know what underlying measures would raise their visibility into key areas. Just as bad, they didn’t know how to take measurements precisely and inexpensively. I want to explicitly except organizations like GE from this analysis, BTW, as their rich history of scorecard management deserves its own treatment. For information on this, see GE’s presentation at Metricon 2.0.

Though there is often some pretty easy low-hanging fruit, leading organizations have begun to realize that deploying sensors to answer questions about software security spending is trickier than expected. Success involves looking at things on different levels than their budgetary line-items have defined. For instance, people are paying for penetration testing and they may have allotted money for a static analysis tool or code review services (or both). However, to determine and increase the efficiency of their assessment practices, organizations are finding more success with normalizing vulnerability findings reports, tying them back to what detected them, and generating “cost per findings” from there, rather than micro-optimizing each assessment technique itself. In my experience, this causes organizations to shift focus between assessment activities in ways that surprise them, but provide a profoundly cheaper overall assessment cost. As a result, organizations are surprised that a slight resurgence in penetration testing allows them to uncover authentication problems in their web applications more cheaply than the recently more favored static analysis tools are allowing them to. Likewise, simple customization of static analysis tools has allowed them to delete whole swaths of test cases that distracted or bloated their penetration testing efforts. More on this phenomenon can be found in my recent article.

I believe a scorecard will emerge. I believe metrics like “cost per finding” will begin to make sense and provide valuable decision support when organizations further mature the consistent application of their software security practices and realign the things they’re measuring. But, right now, great software security managers are conducting more “under the hood” diagnostics than “roll-up” reporting to increase effectiveness of their groups, reduce their cost, and weather this economic storm. No surprise there ;-)

I’ve been an organizer of ACSAC in one capacity or another for close to 20 years now, and I’ve managed to attend most years. The conference always meets in early December in a southern US city (2008 in Anaheim, 2009 in Honolulu). This year’s keynote speakers were Sami Saydjari (formerly of NSA and DARPA, and now president of his own consulting company) and Whit Diffie (Distinguished Fellow at Sun, and famous inventor of public key cryptography).

I found Sami’s talk (”Structuring for Strategic Cyber Defense: A Cyber Manhattan Project Blueprint”) both energizing and frightening. I’ve known Sami for many years, and worked with him in several capacities, and have seen his concerns for the safety of the US develop over the years. Sami views cyberspace not just as an extension of “traditional” warfare (which he dubbed “kinetic” some years ago), but as a space in its own right, with its own assets - instead of land and people controlled (or damaged/destroyed/killed), it’s information that’s at risk. He pointed out that our enemies - whom he doesn’t hesitate to name, most notably the Chinese government - are investing in all-out attacks against the US, focusing on destroying our infrastructure including power and banking. (If you think the meltdown on Wall Street of the past few months has been bad, think of how much worse it could have been if coupled with a cyberattack to wipe out banking records!) While Sami’s focus is on the US, he points out that the threat is not only to the US, but also to any other technology-dependent country, which includes most of the developed world. He believes the threat is the same order as the nuclear threat during the cold war, which was a threat to national sovereignty.

Sami noted “It’s time to stop speculating about the threat, and time to do a risk assessment”, and then described the “Dark Angel” tabletop exercises that showed how an adversary with $500M (a tiny fraction of what we’re spending on bailing out the auto industry!) and three years could take out 70% of the power infrastructure of the US for an extended period of time - think about the lawlessness in the wake of Hurricane Katrina, and extend that across the entire country for a period of months or longer.

Sami noted that “the debate between talking heads has to stop” and called for an expert congressional commission similar to the Rogers commission that studied the Space Shuttle Challenger disaster, in which Richard Feynman played a key role.

On the positive side, the incoming Obama administration seems more aware of the issues of cybersecurity than any prior administration.

Sami is calling for a Cyber Manhattan Project with the vision (to overcome national strategic vulnerabilities ASAP) and urgency (because there’s plenty of means, motive, and opportunity for an attacker) to develop strategic capabilities, including continuity of critical information infrastructure operations (perhaps including “dark power” similar to “dark fiber” available in case of emergency), addressing the systems that are designed to “fail safe fail by having catastrophic failures”, figuring out how to “reboot the power grid” in case of a failure (the power grid requires power to get started), asymmetric threats (a $500M attack could cause $1T damage), situation awareness, and metrics-based quantifiable security.

Finally, he called for the Right organizing model (along the lines of DARPA for research and NASA for operations rather than a bureaucracy like DHS), at the Right Place (in the White House, to be above the fray), with the Right Clout (a presidential statement of support), and the Right Authority (bureaucracies cannot have power over the money).

Sami certainly isn’t the only one commenting on the urgent need to address cybersecurity threats. If you haven’t read the report of the “Commission on Cybersecurity for the 44th Presidency” it’s well worth it, although it’s not as demanding as I’d like to see. Also, see a recent Wall Street Journal article (titled “Internet Attacks Are a Real and Growing Problem: A new report says cyberwar isn’t science fiction”) .

After his talk, several of us asked Sami what we could do as individuals to further the Cyber Manhattan Project. Following is his answer, reproduced with his permission.

1. Educate yourself. Learn about the nature and gravity of the national strategic cyber risk and bold and effective moves that could mitigate those risks. Learn about the policy realm and what the issues are so that youcan speak intelligently about good and bad cyber defense policy. Take a policy expert out to lunch. Learn about related history such as the early formation of NASA and of the (U.S. and U.K) Air Force and how these big changes were possible against the barriers that the establishment always puts up to such big changes.
2. Educate your peers. Once you educate yourself, help educate your professional peers through one-on-one debates and discussions. Give professional talks to peer group at conferences and to graduate level students about to emerge into the professional world.
3. Educate the public. Securing the national cyber vulnerability will require significant investment of resources and significant educational, legal, and cultural attitude changes. This requires that the public be educated on this issue and that a national public discussion take place. Facilitate this discussion by educating the public through lectures and talking to the press in terms that they can understand. I find that analogies like the Manhattan project and the space race are helpful to frame the debate. Wrote Op Ed pieces.
4. Educate leadership. Advocate for sound public policy in strategic cyber defense. Talk to executive branch leadership at whatever level you have access to help them understand the types of actions our country needs to address these risks and encourage them to take an active role. Meet with members of the legislative branch and their staff and advocate for legal and policy changes and appropriate investments to mitigate national risk. Government has a responsibility to “provide for the common defense” as laid out in the constitution. Hold the government accountable to step up to that duty to the emerging cyber space territory in the information age.
5. Serve. This is a hard one. There are at least two important ways to serve, but both will involve your most precious asset: your time. Consider investing 10% of your career in public service. This could include joining non-profit advocacy groups that help move for change and improvement in some aspect of national cyber defense. The second major category is to serve inside the government to change things from within. This can be extremely hard because it means disrupting your normal career path and perhaps your family. It is also hard when the degree of change is great and the system resists that change with all its might. Yet, this can be one of the most effective ways to cause significant change. Try hard to answer the call, particularly in later parts of your career when you have the experience and network to be most effective.

Brian Chess, Sammy Migues and I have been building a maturity model for software security. We decided to base our model on data gathered by interviewing 9 top software security programs. We developed a framework to guide a series of interviews for data acquisition.

Though we have not completed the maturity model (analysis continues apace), a number of surprises bubbled up from the data soup. We wrote them up in an article that we thought you might find interesting.

Though our approach is only “science-y” and may well be closer to anthropology than computer science, we do think it’s important to focus on what works in the real world. If you have any questions or comments about our study, we welcome contact.

Two of Cigital’s thought leaders, Paco Hope and Ben Walther, just published a new book from O’Reilly called the Web Security Testing Cookbook. I wrote the foreword for the book which is reprinted below. More information about the book can also be found on Facebook.

Web applications suffer more than their share of security attacks. Here’s why. Websites and the applications that exist on them are in some sense the virtual front door of all corporations and organizations. Growth of the Web since 1993 has been astounding, outpacing even the adoption of the television and electricity in terms of speed of wide spread adoption.

Web applications are playing a growing and increasingly prominent role in software development. In fact, pundits currently have us entering the era of Web 3.0. The problem is that security has frankly not kept pace. At the moment we have enough problems securing Web 1.0 apps that we haven’t even started on Web 2.0, not to mention Web 3.0.

Before I go on, there’s something I need to get off my chest. Web applications are an important and growing kind of software, but they’re not the only kind of software! In fact, considering the number of legacy applications, embedded devices, and other code in the world, my bet is that web applications make up only a small percentage of all things software. So when all of the software security attention of the world is focused solely on web applications, I get worried. There are plenty of other kinds of critical applications out there that don’t live on the Web. That’s why I think of myself as a software security person and not a Web application security person.

In any case, Web application security and software security do share many common problems and pitfalls (not surprising since one is a subset of the other). One common problem is treating security as a feature, or as “stuff.” Security is not “stuff.” Security is a property of a system. That means that no amount of authentication technology, magic crypto fairy dust, or service-oriented architecture (SOA) ws-* security API will automagically solve the security problem. In fact, security has more to do with testing and assurance than anything else.

Enter this book. Boy, do we need a good measure of web application security testing! You see, many “tests” devised by security experts for web app testing are not carried out with any testing rigor. It turns out that testing is its own discipline, with an entire literature behind it. What Paco and Ben bring to the table is deep knowledge of testing clue. That’s a rare combination.

One critical factor about tests that all testers worth their salt understand is that results must be actionable. A bad test result reports something vague like “You have an XSS problem in the bigjavaglob.java file.” How is a developer supposed to fix that? What’s missing is a reasonable explanation of what XSS is (cross-site scripting, of course), where in the bazillion-line file the problem may occur, and what to do to fix it. This book has enough technical information in it for decent testers to report actionable results to actual living developers.

Hopefully the lessons in this book will be adopted not only by security types but also by testing people working on web applications. In fact, Quality Assurance (QA) people will enjoy the fact that this book is aimed squarely at testers, with the notions of regression testing, coverage, and unit testing built right in. In my experience, testing people are much better at testing than security people are. Used properly, this book can transform security people into better testers, and testers into better security people. Another critical feature of this book is its clear focus on tools and automation. Modern testers use tools, as do modern security people. This book is full of real examples based on real tools, many of which you can download for free on the Net. In fact, this book serves as a guide to proper tool use since many of the open source tools described don’t come with built-in tutorials or how-to guides. I am a fan of hands-on material, and this book is about as hands-on as you can get.

An overly optimistic approach to software development has certainly led to the creation of some mind-boggling stuff, but it has likewise allowed us to paint ourselves into the corner from a security perspective. Simply put, we neglected to think about what would happen to our software if it were intentionally and maliciously attacked. The attackers are at the gates, probing our web applications every day.

Software security is the practice of building software to be secure and function properly under malicious attack. This book is about one of software security’s most important practices—security testing.

—Gary McGraw, July 2008

This is a guest post from Meera Subbarao, a senior consultant at Cigital.

Continuous Integration commonly known as CI is a process that consists of continuously compiling, testing, inspecting, and deploying source code. In any typical CI environment, this means running a new build every time code changes within a version control repository. Martin Fowler describes CI as:

A software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build to detect integration errors as quickly as possible.

Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.

While CI is actually a process, the term Continuous Integration often is associated with three important tools in particular. As shown in the screen above the three pillars of CI are:

• A version control repository like Subversion, or CVS.
• A CI Server such as Hudson, or Cruise Control
• An automated build process like Ant or Nant

So, let’s look at each of these in detail:

Version Control Repository:

Version control repositories also known as SCM (source code management) play a crucial role in any software development environment. They also play a very important role for a successful CI process. The SCM is a central place for the team to store every needed artifact for the project. It is mandatory for the teams to put everything needed for a successful build into this repository. This includes the build scripts, property files, database scripts, all the libraries required to build the software and so on.

The CI Server:

For CI to function properly, we also need to have an automated process that monitors a version control repository and runs a build when any changes are detected. There are several CI servers available, both open source and commercial. Most of them are similar in their basic configuration and monitor a particular version control repository and run builds when any changes are detected.

Some of the most commonly used open source CI servers are; Cruise Control, Continuum, and Hudson. Hudson is particularly interesting because of its ease of configuration and compelling plug-ins, which makes integration with test and static analysis tools much easier.

Automated Build:

The process of CI is about building software often, which is accomplished through the use of a build. A sturdy build strategy is by far the most important aspect of a successful CI process. In the absence of a solid build that does more than compile your code, CI withers. With automated builds, teams can reliably perform (in an automated fashion) otherwise manual tasks like compilation, testing, and even more interesting things like software inspection and deployment.

Now that we have seen the important tools in our CI process, let’s see how a typical CI scenario looks like for a developer:

• CI server is configured to poll the version control repository continuously for changes.
• Developer commits code to the repository.
• CI server detects this change, and retrieves the latest code from the repository.
• This causes the CI server to invoke the build script with the given targets and options.
• If configured, CI Server will send out an e-mail to the specified recipients when a certain important event occurs.
• The CI server continues to poll for changes.

Why is CI Important?

This is one of the most frequently asked questions, and here are a few points to note about this powerful technique:

• Building software often greatly increases the likelihood that you will spot defects early, when they still are relatively manageable.
• Extends defect visibility.
• CI ensures that you have production ready software at every change.
• CI also ensures that you have reduced the risk of integration issues by building software at every change.
• CI server can also be configured to run continuous inspection which can assist the development team in finding potential bugs, bad programming practice, automatically check coding standards, and also provide valuable feedback on the quality of code being written

Now that we have a basic understanding of CI, why CI is important for us, and the tools required to run a successful CI process, in a series of upcoming articles we will see how to setup CI, how to maintain code quality, continuously run code inspections, generate documentations, provide feedbacks to the teams and so on. Stay tuned….

I have been known to take the Web application security community to task for a myopic focus on Web and Web only. Being constrained by HTTP does serve to make things pretty easy! Lately, I have adjusted my thinking.

Jeremiah Grossman and I cross paths out there on the evangelism circuit pretty often and have talked about Web app security versus software security many times. Jeremiah is a great guy, and always willing to listen and think carefully. It was only natural that he would end up as a Silver Bullet victim.

Episode 32 of the Silver Bullet Security Podcast features a chat with Web security guru Jeremiah. Among other things, we talk about the relationship between Web app security and software security.

Near the end of our conversation, we raised the idea of whether all Web security problems have analogs in the software security space and what that might mean. After thinking more about that issue, I made it the subject of this month’s informIT column.

In the end, Web application security is important, but we must be careful not to overemphasize the solutions that work only for Web apps and forget about the rest of software out there. In the meantime, we have plenty to learn from each subdomain.

Brian Chess and I just published an article on the Software Security Framework displayed below.

Our plan is to use this framework to build a maturity model for software security by interviewing executives running many of the top ten large-scale software security initiatives. Please check out the article, and stay tuned for more.

My last project for my former employer (Software AG) was a study of what software vendors do to achieve software assurance. The goal of the study was to see whether we (Software AG) were at, above, or below the norm, and to adjust investments in assurance accordingly. All but one of the vendors who participated are household names - these weren’t mom & pop shops, but major multi-national ISVs, most of them with sales of a billion dollars a year or more.

I presented a brief summary of the study results at the recent “Making the Business Case for Software Assurance” workshop hosted by Carnegie Mellon’s Software Engineering Institute, and sponsored by the US Department of Homeland Security. I’ll also be presenting an even briefer summary of the results at the 24th Annual Computer Security Applications Conference in December.

In my new role at Cigital, I’m hoping to be able to expand the survey beyond software vendors into e-commerce vendors, embedded software suppliers, financial institutions, etc., as well as to systematize the survey so it can be done by filling out a web form instead of as an interview. I welcome your suggestions as to how to make this project more relevant to vendors and software purchasers - and also welcome your participation in the survey, as well as suggestions on how to fund the ongoing work!

And finally, thanks to the (anonymous) vendors who participated in the first phase of the project. While I can’t thank them by name, I very much appreciate their input.