So it seems like almost everywhere you turn for advice about securing programs or resolving known security problems leads you to a ‘security guy’ telling you something along the lines of ‘well, you have to validate your inputs to prevent these kinds of issues’.

Perhaps I’ve heard it too many times or perhaps I’m just jaded, but I’m throwing the BS card. Of course, I’d never leave it at just that… I think I’ve got a pretty good case for why it’s BS.

Consider my favorite red-headed stepchild, cross-site scripting (XSS). The mechanics of this problem are simple: an application accepts some input data and then offers that data in the form of output back to a user without checking the content of the data along the way (this is the case for both reflected or stored XSS problems, fundamentally).

Now, consider a small flashback I’m about to have to a Computer Networking & Communications from my undergrad days. I know serial communications are sooo last week, but anyone remember putting together simple protocols to transfer data over a line? In a simple message-based protocol, you’d pick a few byte-values to represent a few control commands like ‘end of message’ or ‘close this channel down’. This seemed like a great plan until you tested it out and noticed that some messages were getting truncated in weird ways and occasionally the whole channel went down. If you didn’t just chalk it up to bit-gnomes and listened to the professor, what you learned was that since you intermixed the CONTROL channel with the DATA channel, your data was inadvertently being interpreted as control commands when the appropriate byte-values were present in the data being transferred. Hopefully, you then learned that to make the protocol reliable, you needed to have a mechanism to escape data that contained values that would be interpreted as control codes. How’d you implement the fix? Well, you certainly didn’t try to trace the origin and content of every byte that might enter a message. What you did was augment the send_message() function with logic to zip through the pending message and escape anything that was a control code and then you’d do the normal stuff of writing it to the wire.

Let’s pop back to reality and our XSS problem. The problem is that we’re mixing user DATA with our application’s CONTROL and allowing the user data to be interpreted as control commands (in this case, HTML and/or javascript elements that run in the victim’s browser). I can understand how input validation might help for common, pathological cases of XSS vulns, but in no way is it a complete or adequate solution because what’s really important (and the correct chokepoint to fix the issue at) is the point of output. Yeah, we need to ensure that every time we put content into the user-bound HTML stream we first encode it to be safe in that control channel/language.

This is not limited to XSS. SQL injection (or really any injection attack) is about taking input data and passing it with unchecked contents to a DB command (or any API with a control language of its own, e.g. LDAP queries). Again, input validation can help in simple cases, but you’ve gotta know a priori all the ways in which data might be used by an app and choose some kind of mutually safe set of characters to let through. Either that or encode the potentially unsafe characters so they don’t cause trouble somewhere down the line. Not very extensible, usable, or maintainable in many circumstances since it’s overly restricting and fragile to change.

Take for instance, a ‘Comments’ field in a web app. Many real-world applications really do need to allow users to use characters like ‘$’, ‘%’, ‘<’, or ‘>’ to represent things like money, percentages, and value comparisons, so I simply reject the idea of banning those characters because it’s unnecessary and indicative of misunderstanding of the real problem. Some folks (which shall remain nameless) have said, ‘well, if you need those characters, just HTML/URL encode them as part of the input validation process and you’re all set’. Well, now you’ve added another problem where you’ve gotta go decode and re-encode appropriate to any other output vectors. To supporters of this strategy I ask, how many loading dock foremen and warehouse employees do you know that would correctly interpret a printout containing ‘Boxes &amp; packing must be &lt;50lbs’? How about ‘Boxes%20%26%20packing%20must%20be%20%3C50lbs’?

Now, I’m not saying you should not do input validation. It adds great usability features and might limit impact of other programming mistakes. What I’m saying is that input validation alone isn’t enough. You’ve gotta have output encoding to truly solve it right. We need to ensure that architects & developers have a deeper understanding of what the problem really is in order for them to naturally build systems to these types of attacks.

Software security veterans have all certainly thought about the idea of ‘securing the SDLC’… I can tell because every consulting firm’s collateral that I’ve seen in the past year has a new bullet under their ‘services’ section referring to something like ‘Secure development process integration’ or ‘Secure SDLC services’. That being said, let’s talk about what this means for a second. Fundamentally, there are a few ‘different’ schools of thought out there (and as it’ll turn out, they’re not all that different at all).

I know of three popular ways of looking at the problem, 1) Microsoft SDL 3.0 (with a recent book by Howard and Lipner to codify the subject), 2) Software Security Touchpoints from Gary’s book Software Security, and 3) CLASP (originally developed by Secure Software, Inc, and now an open project through OWASP). BTW, if anyone knows of other publicly usable process methodologies, by all means email me since I’d love to read about them.

After spending a bit of time thinking through all these different ideas, a few interesting points emerge. First, there’s not much difference between SDL, the Touchpoints, and CLASP. There’s just about nothing I can see where these processes fundamentally disagree. The differences are really only in the timing and the extent of the prescribed activities (i.e. they each cover the bases of what you should be doing, some just give different orderings to the activities and talk about the sub-steps in different ways). My personal opinion is that SDL is particularly suited for companies like MS (large ISVs with large user populations) and process like the Touchpoints and CLASP are a bit more flexible and widely applicable.

So what’s the deal? Do we have the problem of dev process augmentation solved and put to bed? Heck no. Consider the following quote that popped up in a discussion my buddy Gunnar Peterson and I had at the recent OWASP conference in Milan: “Amateurs talk about tactics, debutants talk about strategy, but professionals talk about logistics.??? (this quote has many variations and is hard to find a definitive source, but it’s likely from a US military officer many years ago). As the software security space was emerging, you bet we had to crawl from the primordial ooze by figuring out some tactics to stop the bleeding. Logically following, lots of smart folks sat down and figured out the right way (via experimentation, mostly) to look at the problem from a high-level. Hence, strategy for software security was born. Now, the proverbial last mile is the logistics of how you get the job done within an organization that’s got 50,000 real-world constraints that complicate everything.

Regardless of your favorite security-enhanced SDLC method, you’ll notice that they really are, at their core, a collection of activities, procedures and artifacts (tactics). Don’t get me wrong, it’s great stuff in terms of what’s needed to do the job well and it’s generally assembled and presented in a full-blown, whole-hog, flying-car way (strategy). If you’re in the shoes of the person in charge of augmenting your company’s dev processes, you’re handed a large collection of great things to think about, but little that’s directly actionable in terms of answering ‘what do I do tomorrow?’ (logistics).

What I’m getting at is that I think we’ve gotten to the point where if you’re still debating tactics of what to do or the strategic vision of what needs done for process integration, you’re solving the wrong problem. It’s about rubber-to-the-road logistics. We need to build on the work that’s been done already and come up with plans that make it accessible and usable for an average human that hasn’t made a career on thinking about these things. That’s a serious challenge, but not an impossible one. At Cigital, that’s what our SDLC process gigs are all about (providing the company a detailed plan of how to get it done). What’s needed now is to get a more abstract way of looking at the various factors that contribute to logistical differences (e.g. type of business, market vertical, organizational hierarchy, regulatory constraints, etc.). I strongly believe that we can formalize these factors and I think that goes a long way to breaking the back of the problem. I fact, I’ve been working with folks in the OWASP community on this very problem (and would love to get anyone else with field experience involved). Much of that work will be released in a new version of CLASP in the next week or so, so stay tuned if you’re interested (I’ll post another entry here announcing it).

Technorati Tags: software security, sdlc

If I were to say penetration testing, what would you expect? After conversations with lots of colleagues and friends it’s become fairly clear to me that the term is massively overloaded. It actually got me thinking about the evolution of pen-testing over the last several years. And I’m not going to preach from the you-need-design-and-code-review-too pulpit. I’m just gonna talk about the innards of the pen-test itself (breaking a running system).

The old-school notion of a pen-test was one where you’d hire a bunch of l33+ h4×0r5 and turn them loose on a system to see what they are able to bust. The general idea here was to simulate a real-world attack scenario (black-box pen-test). Given adequate time, Cheetos^® and Red Bull^® you’ll probably be left with some findings that are interesting. But take a closer look at the breakdown of how those testers spent their time. Those guys likely spent a ton of their time trying to figure out how the app works, trolling for non-obvious interfaces and partially reverse-engineering the logic of the app. When you’re paying by the hour, this is bad news since they didn’t spend as much time as they could have in actually trying to break the app (they spent a bunch in just getting their arms around it.)

Evolving from these obvious limitations in the completely black-box approach the notion of the “pen-test with code” was born. This was a similar scene, but instead of poking at the running app as much, the testers would go and read code. While the testers are still at arm’s length from the dev team, it’s still better since access to the code gives a truer picture of what’s actually happening under the hood without time spent on reverse engineering. Sometimes, there would even be design docs available which really helped move things along. In the end, this is close to the modern-day version of what many expect when they buy a “pen-test.”

The major weakness of this style of pen-test is that there’s often no business context to ground the direction of the testing (on which aspects of the system you concentrate) or to baseline the value of the findings (to the company using the app, what do the discovered exploits really mean?) Some pen-testers are starting to bridge the technical-to-business gap, but they’re definitely in the minority. This step in the advancement of the pen-test is definitely a step in the right direction since it focuses the testers on generating and prioritizing findings that show demonstrable impact to your core business (as opposed to sheets of meaningless XSS vulns that are all prioritized “high” since XSS is bad.) The pen-tester gets to concentrate on breaking business logic in your application and scheming about combinations of technical vulns that lead to an interesting business problem. You’ll still get all the technical vulns, they’ll just be low priority if they can’t directly contribute to a bigger problem. Since this all plays into making risk management decisions, I call this style of white-box pen-testing the “risk-based pen-test.”

If you’re using pen-test today, you really want as close to a risk-based pen-test as you can get. Although I have heard a few reasons why you might not:

• “I don’t have access to code or design docs.” I hear this reason all the time as justification for why a black-box approach is used. In the end, even if you don’t have code, someone has gotta know something about how the application actually works. Even if it’s only from a user or sysadmin perspective, it goes miles to setting the testers off in the right direction. Find those people and link them up with the pen-testers! And business risk mapping is still possible even without code (but you’ll probably need at least ad hoc design info).
• “I use black-box pen-test nowadays as a way to simulate real-world attacks.” This notion is just off-base due, perhaps, to the lack of understanding about real-world attacks. Yes, the people “in the wild” that might attack you will have a similar skill-set to your pen-testers. The kicker is that the time available to an outside attacker is virtually infinite compared to that of a for-hire pen-tester. When it comes down to exploiting software often it’s just a matter of how much time you spend bashing your head against the problem. Outsiders can spend 2 hours a day for 6 months working on getting a single exploit working. Your pen-testers have 8 hours a day for a week to find lots of impressive results. Thus, cheat the problem by giving the pen-testers all the info you have.

In any case, let’s get back to talking about that notion of the risk-based pen-test. When I think about the requisite skills to complete that job I see many similarities to what QA folks do on a routine basis. Foremost, the QA testers will already know the application components and UIs. They should know the business value of the app and in many cases they’ll also know how the business logic of the app is supposed to work. And also, they are routinely asked to assess the specific business impact of problems in the application (they do it every time they open a bug and assign a priority.) That’s a huge advantage over someone from the “outside” coming into a pen-test engagement. Now what about differences? The biggest red-flag is that the QA people aren’t trained to attack applications to find security flaws. That’s a big disadvantage since you need those skills in order to be even remotely effective at a pen-test. But do keep in mind, QA folks are very much trained to break applications in general (it is, in fact, their job.)

So where are we going with this? Simple: let’s give the pen-testing responsibility to the QA team. They’ve already got a leg-up since they know the application. To counter the point about them not being trained in security attacks employ two simple techniques: tools and training. Some of the automated pen-testing tools out there are really great now. They’re basically extremely potent packages of electronic subject-matter expertise. Now, I would never advocate just buying a tool and hitting the “go” button and calling the job done (you don’t get very good results at all this way.) Enter the training. The automated pen-testing tools, in many ways, are very similar to other types of tools in a QA tester’s belt. They run a series of attack test-cases and report vulnerabilities when a test fails (this is a massive oversimplification of what they’re doing under the hood, but in terms of usability, the analogy holds.) So, train the QA testers on effectively using those tools. Teach them about how to feed application-specific details to the tools to make sure your coverage is high and ensure the results are more accurate.

Further, teach them about the classic notion of security testing (starting with requirements and deriving test-cases to ensure that functional requirements are implemented securely) and show them how to automate it with the tooling. In fact, that’s where this is all ending up: a risk-based security test with a wide blast-radius for getting the bulk of the benefit of a pen-test. It’s cheaper in the long-run (front-loaded cost for buying tools and building the internal skill vs. repeated, indefinite fixed-price cost.) In terms of effectiveness vs. an external pen-test, it’s definitely on the positive side of the 80%-20% rule (and all the things you might miss in that 20% will likely be caught by code review or architecture analysis.) It’s even “faster” in the sense that an organization could move through each assessment more quickly, thus enabling more assessments within a given timeframe.

In summary, I see the future of pen-testing as a push toward QA environments. Since the notion of a code review and architecture review are becoming more mainstream, this makes sense. Why continue to spend a lot on a service that ultimately should be used as a sanity-check of the running system?

Red Bull^® and Cheetos^® are trademarks of their respective companies.

Technorati Tags: penetration testing, software security