Industry Insights with Carl Scaffidi

Governance , Managed Security Service Provider (MSSP)

3 Reasons to Do a 'Proof of Concept' With MDR Providers

Tips for Acquiring the Right Managed Detection and Response System
3 Reasons to Do a 'Proof of Concept' With MDR Providers

When it comes to security, it's easy to spot a few themes among small and mid-sized companies.

Many have strong security programs despite having a relatively small staff, which means that everyone in the organization wears multiple hats. The "wears many hats" approach works well for a while - it did for us here at Baker Donelson - but at some point, every organization needs additional (and often outside) support as their security operations mature.

As the saying goes, "Hope is not a strategy." While a POC won't tell you every last detail about a vendor, it'll get you one step closer to finding a partner that can help your security operation grow and mature. 

Earlier this year, we took a look at our program. We'd recently added more tools, which meant there was more to monitor. We knew we needed more analysts to ensure we were staying on top of all of it so we'd be able to demonstrate to our clients that we're keeping their data secure.

We set out to find a managed detection and response provider. As we began our search, one thing was clear: We'd need to put these vendors to the test with a proof of concept.

Why Do a Proof of Concept?

Since taking on the role of CISO at Baker Donelson, I've made it a policy that we do a POC before we invest in any new technology.

Every security vendor promises the moon in their marketing materials. As a small organization, we need to be confident that any tool we buy will do what we need it to do in our environment. The only way to do that is to kick the tires.

PowerPoint presentations and demos are a helpful starting point. But just like buying a new car, you need to take technology for a test drive. Otherwise, you risk wasting time and money, not to mention your own reputation.

A POC won't answer every possible question about a piece of technology or a service provider. But a provider's willingness to engage with you - and the way they engage with you - will help you learn a great deal about what your relationship will be like once the ink on the contract is dry.

Three Factors to Evaluate

One of the most surprising things I found as we evaluated managed detection and response providers was how few of them would even agree to do a POC. When vendors refused, I could only conclude that our business wasn't a good fit for them or they couldn't back up their marketing claims. Ultimately, we selected Expel. Here are the three areas we focused on in our POC:

Will it integrate with the security tools we use?

We prioritized finding a provider that would work with the technology and controls we already had in place. For just about any organization, it's going to be important that your MDR or managed security service provider integrate with your SIEM. We needed an MDR that could quickly grasp what we care most about when we're watching our SIEM - so they can add their eyes to ours.

The same is true of endpoint detection and response technology. We needed a MDR that could quickly learn which EDR alerts require an investigation, which ones mean it's time to go threat hunting and which should trigger containment actions. Our EDR is process-oriented, so we wanted a provider who could work in partnership with us from the initial alert all the way through the incident response lifecycle.

Can the service adapt as we evolve?

Things change constantly - especially when it comes to cybersecurity. What works one day won't necessarily work the next. And if you operate as if it will, you'll fall behind. When you're choosing a MDR, look for one that's adaptable enough to keep up with you as you improve your processes and add new tools.

One way I like to test this concept of adaptability is to understand their past integrations and timelines as well their future roadmap. If one of your tools is on the roadmap and it aligns with the POC time, being an early adopter of that integration helps both you and the provider during the process.

Are they easy to work with?

Look for a vendor whose culture aligns with yours. Our operation is fairly lean, and we wanted a provider that could work as an extension of our team and integrate with our processes and workflows, rather than generate a torrent of alerts with general guidance to "check them out."

We don't have an internal security operations center, so it was critical to find a provider who understood our environment and could get even smarter over time. All alerts aren't equal. For example, decisions on when to escalate or investigate can be very different depending on time of day, users or the data being accessed.

Get a Second (Unbiased) Opinion

A POC is a great way to get a real-world understanding about whether a new technology or service provider will add value for your organization.

But don't forget about the power of your professional network. I've gotten great insights from my peers when I've attended local roundtable events with fellow CISOs. Whether it's a roundtable or a Slack channel or something else, find those security leaders you trust most and ask their opinion of the technology and the vendors you're evaluating. Getting a list of references from the providers is a good start, but it's essential to vet those reviews against feedback from a network you trust.

As the saying goes, "Hope is not a strategy." While a POC won't tell you every last detail about a vendor, it'll get you one step closer to finding a partner that can help your security operation grow and mature.



About the Author

Carl Scaffidi

Carl Scaffidi

CISO, Baker Donelson

Carl Scaffidi is CISO at Baker Donelson, a large U.S. based law firm, where he's responsible for developing and executing strategy, governance, risk and compliance to build and lead a comprehensive information security program. He has over 18 years of IT and security experience. Prior to this role, he was the global architecture lead for EY cyber defense and threat & vulnerability management and an information security consultant for PwC and Accenture.




Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing careersinfosecurity.com, you agree to our use of cookies.