Application Security Over-Confidence: Facts & Myths Revealed
In this exclusive interview, Roger Thornton, founder and CTO of Fortify Software, discusses the survey results and his own market perspective, discussing:
TOM FIELD: Hi. This is Tom Field, Editorial Director with Information Security Media Group. The topic today is application security, and we're talking with Roger Thornton, Founder and Chief Technology Officer with Fortify Software. Roger, thanks so much for joining me today.
ROGER THORNTON: Thank you, Tom. It's a pleasure to be with you.
FIELD: To start our conversation, I wanted to ask you, how do these survey results and this confidence issue map against what you hear from your own customers at Fortify?
THORNTON: Sure, Tom. You know, there's I guess the first and most important thing to say is, you know, a lot of the respondents to this survey were in the banking sector and financial services community where they've, for the most part, been working on this problem four, five, in some cases, six years. And so a lot of these institutions have confidence in not only their ability to locate vulnerabilities in software, but also to produce software that is relatively free of vulnerabilities. But there are a lot of companies we talked to that are confident that their applications are safe without that five years of practice doing it, and so you always have to, when somebody tells you they're confident their applications are secure, you have to dig a little bit deeper and find out if they're confident because they have processes in place to ensure and measure and their confidence comes from seeing those metrics and those measurements get better over time, or if they're just generally confident because, you know, their developers are good developers and they would never make security mistakes.
You know, from this survey, given who was responding, I think the confidence on internal was probably merited because, again, a lot of the people doing the survey were from that industry. But their lack of confidence outside their own development shops is very warranted. At Fortify we've been working in this space since 2002 and, for the first several years of our work with the marketplace, overwhelmingly it was the, you know, top-tier global banks, government agencies that were real active in this. Now we are starting to see the independent software vendors that service the banking industry, as well as, manufacturing companies and retail companies starting to get on board but, naturally, many of these companies are literally years behind where the banks are.
FIELD: You know, Roger, I want to pick up on that confidence issue that you spoke to on both levels. Let's talk about it in terms of in-house applications. As you say, the survey respondents indicate they've got some confidence in what they develop in-house, and you say this jibes with what you see. Where does this confidence come from?
THORNTON: Well, as I was mentioning,- when I talk to somebody and ask them what their confidence on their internal is, it'll come from either a lack of awareness of the problem and that would be, let's say the naive confidence; and, you know, I'll hear people say things like, "Well, you know, our developers are really good and we only hire from the top colleges and they're all smart and, therefore, we can't have any security problems. Or for all we know, we haven't been breached." And one of the things is, if that's the basis of confidence, it's probably not well-placed and, when I'm talking to someone like that, one of the first things I'll tell them or ask them to think about is, clearly, companies like Microsoft have - and continue to have but, you know, in the past they had - some really bad security problems. They have since implemented processes to make those problems go away but, clearly, they have problems. And now, if you think about the average developer at Microsoft, these are very smart, very capable, hard-working people. Right? So security vulnerabilities in your code aren't -- they can be created by the best developers on Earth. What's important is that, when you look at your software engineering or your software procurement process, that you see the controls in place to assure and measure that your software is indeed secure. And I want to stress, Microsoft is probably one of the industry leaders in doing that around their operating systems. A lot of care is taken to the development of that software and with a lot of banks, so if the person you're talking to is feeling a confidence because he's able to pull up the security assessments of every major application within their portfolio, then that confidence is coming from knowledge or from activity. But they're both out there. And if someone's confident, you've always got to dig a little deeper and find out, "Are you confident because you've got the data from the effective measurements that are showing that you're making improvements around security, or are you just generally confident because you've got smart developers in here doing a good job?"
FIELD: Well, that's a good distinction. Roger, let's turn this around now and look at the third-party service providers. There's a lot of regulation in the banking industry stressing the necessity of vendor management and making sure that the services that your vendors provide for you are just as safe as yours are supposed to be, just as secure, and, yet, we see that there's not as much confidence from our respondents in those applications that are developed or managed by their vendors. Why is this? Why not the confidence? Are they not seeing the type of metrics that they need to?
THORNTON: Yeah. For the most part - and, again, you know, your top-tier global banks do a very good job of first risk management, identifying which third parties could put the bank at risk and, in a lot of cases, it's a third party that holds information that the bank has been entrusted with or third parties that are doing processes that the banks count on to be in business. And so the first thing they do well is the risk management activities to find, you know, the ones to go look at first. And then most of the top tier banks are really good at doing audits on their partners and going in and scratching the surface. And for years they've been doing audits on network security and infrastructure security and these types of things and, over the last couple of years, they've started to do audits on application security. And the data they're getting back is, I would stress, as expected, not where it needs to be. Right? So if you're a major bank and you've been working on application security for five years and now you're reaching out to your business partners, they're not going to be at the same stage that you're at day one. Right? So they're not expecting third parties to be there immediately, but they're expecting them to take this seriously and start to implement controls and auditing processes so that their code is secure.
And, you know, one other thing to stress, and I'll use the example, probably the best example of this. When we read a newspaper article about credit cards being compromised, we see the brands Visa, American Express, MasterCard over and over and over again. Right? Well, those companies have some of the best security in the world and normally, when we're reading about a credit card breach in the newspaper, it wasn't those companies; it was their business partners. It could be a retailer, it could be a university, it could be a, you know, the bank. And so the companies - the credit card companies - realized that the security practices of their business partners would affect the brand of their company. Right? And so it's incumbent upon them to bring their broad set of business partners up to a level of security that was commensurate with their own and then we all know, of course, that's PCI, which many, many, many companies today are held accountable to. And when I meet people who feel the wrath of, you know, the PCI regiment, you know they feel a little bit like, "Gosh, why is this? You know, why all of a sudden am I having to jump through these hoops?" And one of the things I stress to remind them is, "You're jumping through those hoops because if you have a problem, you're gonna harm your business partner. And your business partner doesn't want that, and he wants you to do good and actually in a way is sharing with you what state-of-the-art information security is." And so sometimes we can look at this as a larger company asking small companies to do things that's quite punitive when actually it's the reverse: the larger company sharing with all their business partners the best practices. sure that their business partners don't cost them more money than they planned. Right?
FIELD: No, you're right. Now, Roger, so the confidence issue aside here, one of the things that seemed to me to be a bit of a disconnect is that whatever the level of confidence, institutions don't seem to be putting a lot of effort into consistent processes for identifying and remediating application security vulnerabilities, and I want to ask you, what's behind this disconnect?
THORNTON: Well, you know, from my experience talking to a lot of companies and getting to know people during this process, I truly believe people go through a series of stages of awareness or maturity around the problem, and so the first - you know, there's a lot of companies out there that think, "Hey. You know what? I've got a firewall. I've got access control. My systems are safe." Right? And the reality is (and this is kind of backing up a little bit), there was once upon a time where our network and our infrastructure was wide open and people would come in and steal things and harm things and that was a big problem. And then came all of the core infrastructure security we have today to solve that problem. Well, what happened is so bad guys didn't go away; they went, "Then, okay, well your infrastructure's secure. Where am I gonna go next? Go up the stack and then look at your applications and see if I can use your software as a basis to get into your system, steal data, break things, what have you." And so there's a lot of companies that feel like, "Hey, I've got this infrastructure security. I must be secure." Or you fast-forward to, "Hey, you're not." And so there's this realization that the software matters as well as the network and, from that point of initial awareness of the problem, the first thing that generally makes sense to people is, "Okay, well, I've got a bunch of software running my enterprise. How many vulnerabilities do I have? What kind of vulnerabilities do I have? Where are the vulnerabilities?" And so they'll set up, either formally or informally, you know, internally using technology like Fortify or externally they might hire consultants to do it for them, but they'll go into this assessment phase and, in the assessment phase they find out, "Yep, sure enough, I've got vulnerabilities, some really bad ones over here and some medium ones over there." And that's very useful data, but, if you think about it, knowing you have a problem is only the first step. Right? You've got to go fix it. What's really key to stress is banking regulations mandate that all public-facing applications have an assessment once a year, and so there's a lot of applications that once a year they'll do that assessment and find out, "Yeah, an application's got problems. Okay, check the box we did the assessment. Move on." Right?
There's not long after doing that over and over, people figure out, "Well, wait a minute. I've got these problems. Let me go back to the IT Department and let me ask them to fix these issues and assess and remediate." And there are companies that we work with that have pretty big organizations that, you know, get the code as it shipped or legacy code or what have you. They do assessments. They come up with a work list, project plan, and send it back to IT to be reworked. And that's where a lot of companies are today. But there really is a better place to be, and that is pushing into the software development lifecycle while you're building - or maintaining, right; maybe it's the legacy stuff and you're in it for maintenance - while that work is going on, looking for the vulnerabilities so that when it's shipped and as it's tested, you've got the assurance that it is indeed free of vulnerabilities. And what that does is it saves you from building code, finding vulnerability and fixing it; it's building code and fixing it at the same time, and that's kind of a preventative model.
And so when you look at companies and you look at their processes, a lot of them, as I said, will have no processes whatsoever. Right? An occasional assessment. And that's clearly not the place to be. There's a growing number that have a process where they can assess code, they can fix those vulnerabilities and get that code patched up and repaired. And there's a couple of things to note. You know, if you're building your own systems in your IT Department, you definitely want to be in that third bucket, right, because you want to build things, you want to have those things secure. You don't want to build them and rework them. But when you buy something from a third party - whether a big third party like a Microsoft or an Oracle, or a specialized third party and, you know, maybe someone that does a custom software for your vertical industry - you still need to do assessment.
And you won't be doing remediation; you're going to be doing rejecting. Right? So whenever somebody gives you a piece of software, you need to have as part of your process either preferably that third party shows you their assurance that the software's secure - in other words, they're building it to be secure and they're actually showing you the artifacts that attest to that fact - or you can do a third-party assessment of their code and, if you find vulnerabilities, send it back to them. And it's kind of the same thing. Right? If someone's given you code and you're doing an assessment and you're finding vulnerabilities and you're giving it back to them, you definitely haven't hit the state of the art, right, because this is a very cumbersome and expensive process and then there's lag time to everything. But ideally a third party is handing you evidence --- they're able to produce all sorts of test results and an analysis of these kinds of things that show you that, "Yes, indeed, we do security as part of building the software for you, and here are the metrics from our internal security process."
Unfortunately, that level of sophistication hasn't quite hit the second tier of companies yet, and this is, though, what the banks are looking for in their third party partners. Right? They don't want you to just ship an application and fix it when they get it back to you; they want you to build it right in the first place.
FIELD: Well, you've got banking institutions captured well because, as you know, they respond to regulators. They do what they're told and do it when they're told to do it. And now we're at a time where the regulatory agencies are telling them about application security and are giving them direction on vendor management. So in this environment, you've got a lot of institutions now, you've got their attention. So what are some proactive ways that these banking institutions can start to tackle application security if they've not done it well?
THORNTON: Well, you would be - we'd be - talking the second tier banks because really just about every first tier major global bank is doing something. There might be a few exceptions, but, I mean, really all the major guys have pretty good processes in place. But we meet every day with regional banks and smaller banks, banks in other countries, and simply for them it's to follow what the leaders have done and the very first thing is to make sure that somebody in the organization owns application security, that there is a, you know, a person responsible for the process. Now, ultimately, your IT developers are going to own the building of software correctly, just as they own the building of software functionally, but you really need a single person to spearhead that. And one of the most common things that we'll see is the establishment of a Center of Excellence or a security team that can do audits on programs that can perform all the various, different types of hacking techniques to validate internally that there is or is not a problem. Now, for some companies that are smaller, you might work with a third party like Fortify or one of our consulting partners to play that role for you, but you ultimately want to have some group in-house that can be a gold standard.
And then the next thing is developer awareness, making sure that developers and program managers and all your various people inside the IT organization have both awareness and training on the importance of application security. And that, you know, some of our customers have fabulous documentaries that you come on board to work there as a developer, you have computer-based training classes that show videos on why this is so important, and then it doesn't have to be that sophisticated, but some level of training and awareness.
And then the next thing is looking at the software development and software procurement processes, and not - you don't have to revamp them, and you don't have to re-haul them, but you've just got to put a couple of extra processes in to make sure that security is being considered as things are built (which will be a little bit easier, because it's more in your control), and as things are purchased, which is a little bit more of a contractual exercise, making sure your contracts specify that any software that's given to you has assurance of security and vendor management and audit[ph]. "I'm sure that, you know, I'm doing a great job. You can trust me. Here's my code." You want to reserve the right to audit that fact.
And, again, large financial institutions are already doing that, and doing it well. You know, the early adopters who thought of different approaches and different ways to go; but, over the years, they've all kind of fallen into very similar patterns and it basically makes sense. Right? You know, I want to go from today, I've got all this software and I have no idea if it's secure or not and where am I putting my business and data at risk, to a future where every piece of code I have, I have a pretty good assurance and all the test cases and all the data to attest to it, that that software is going to perform well in a hostile environment. And that's obviously something that takes time to do, but it's never going to start until you start with the first application and the first development team working on the most important stuff and work your way down that way.
FIELD: Roger, one last question for you. As you know, the economic situation in the financial services industry just changed historically in the last week. In that context, what are some of the cost-effective strategies that institutions can employ now to get a better handle on their application security?
THORNTON: Great question and, I mean, there's no doubt and we see it in our customer base. So if I were to have told you two years ago that two of the customers in our financial services portfolio weren't going to be around today, you would never believe that in a million years, but we're seeing prime, pristine, major companies go under. You know, the fact is is that security, unfortunately, is not an activity that's going to reduce operational cost. Right? But security itself can be performed more effectively or less effectively. Right? And so when you go into bad economic times, I don't think there's any companies that can say, "These times are so bad, let's not defend our business." Right? But, instead, we can look at security and say, "Are we doing this the most cost-effective way possible?" because we want to take all of our dollars and focus them more on the core business. When you look at the "do nothing" strategy on application security, which basically would say, "I've got my network security, I've got my infrastructure security, but my apps, well, I'm gonna let those be exposed," that strategy costs more than you would think because what's going to happen is there will be some compelling event - could be a breach, could be a business partner who says, "Hey, you know what? You've gotta get these applications shored up"; it could be some banking regulatory body that comes and tells you you have to - and when you've got to go into your applications in an emergency, you know, go help fix everything and triage it, can cause massive disruptions, not just in your finding people to get that work done in an emergency mode, but can disrupt all the other IT projects you're working on. And so looking at application security as proactive starting today, having every development group build security into applications, is far less expensive.
And there's a corollary to this, you know, from the manufacturing world and also from the software world. If I build something and I build it wrong and then at some later point in time I've got to go back into that and correct it, I've got to rework it, the cost is going to be much higher than if I incrementally had built it right in the first place. We've run these models and done ROI's with customers and you can expect 10, 20 times more expensive per issue fix in a piece of code that has actually been built, QA'd, shipped and delivered. And in the era that we're in right now, it's more important than ever that you want to look at each of these functions and say, "Okay, what's the absolutely, absolutely most cost-effective way I can do this?" With application security, that is as far upstream in development as you can get.
FIELD: Roger, you make some great points. I appreciate your time and your insight today.
THORNTON: Great, Tom. It was a real pleasure chatting with you and, hopefully, those are some ideas that those are listening can take and put to work in their own organizations. And, of course, if anybody would like to learn more, feel free to reach out and contact us at Fortify. I'm always more than willing to chat with people about their application security initiatives.
FIELD: Very good. The topic has been Application Security. We've been talking with Roger Thornton, Founder and Chief Technology Officer at Fortify Software. For Information Security Media Group and for Fortify Software, I'm Tom Field. Thank you very much.