No company or industry is safe from attack these days. Financial Services, Manufacturing and Healthcare were among the most impacted across the 3,205 compromises in 2023 — a 72% increase from 2021. Additionally, the average cost of a breach is $4.45 million. To make sure that they’re proactively keeping their organizations safe, security teams need the right tools and solutions to support them — especially when 59% say their teams are understaffed.
A major job function of a CISO is assessing security solutions for their organization — but what exactly should they be assessing? How will they know which tools will keep a major threat out, and which tools may fail at the time when they’re needed the most?
As the Chief Evangelist at Team Cymru, I’ve helped hundreds of CISOs evaluate new security tools for their organization that elevate their security teams above the competition to become a harder target to breach. According to security practitioners surveyed in our recent “Voice of a Threat Hunter 2024,” what makes a threat hunting program most effective is the tools. Here are nine factors to consider when evaluating a security solution to guarantee you’re making a wise investment.
1. Performance
Start with evaluating the performance of the solution, like whether it will be able to assess your data at a reasonable rate or handle your network capacities. Also look at what its performance will be in the real world, like whether it will perform without causing a flood of false positives — the key measurement here is accuracy of data and if it leads to successful outcomes when using it. You don’t want to purchase a tool only to discover after deployment that it doesn’t meet your performance standards.
2. Compliance and Standards Alignment
Also, consider how the solution enables you to better achieve compliance and standards alignment. If you happen to be in an industry that’s regulated, you really want to make sure that the tool is going to be able to meet your regulatory needs in a safe way and that the security tool on its own doesn’t violate any regulations or policies. That’s often overlooked, particularly around privacy. Arm your procurement teams with minimum acceptable criteria for security standards, ensuring that only those over the bar can join your roster of trusted suppliers.
3. Integration
No one gets a new green field where you’re building a single monolithic system. Regardless where you are or move to, you’ll need to address how a solution integrates into your current systems. Integrations are almost certain to happen with either previous solutions you’ve inherited as the new CISO or with the disparate pieces of your complete footprint. Confirm with the vendor that your new solution can integrate with existing tools and can connect to each other seamlessly. Ask your security team leaders to create a vendor integration matrix — what needs to be connected to what, and why. This will help you understand where there may be overlap, but also an opportunity to consolidate further to save budget.
4. Automation and Scalability
Another consideration is how the tool achieves scalability through automation. One of the things CISOs overlook is they often buy a security solution for the network and enterprise they have now, but they fail to look at purchasing for what their organization will look like three or five years into the future. During evaluation, your security teams should be specifically instructed to assess how the tool can be automated, and the measurable gains of doing so — regardless if they will leverage it from Day 1. You don’t want to quickly outgrow your investment within a year and have to then rebuild — start with scalability in mind. Understanding the business objectives helps feed into your overall strategy to scale. Most Boards of large enterprise organizations have a Cyber representative, responsible for communication between the most senior layers and management within the organization. Maximize this relationship by providing strategic insights whilst gaining the long term visibility needed for more effective planning.
5. Usability and Manageability
Additionally, using the tool is a very critical piece, but so is managing the tool. For example, if you have a large security team, all of those individuals will need accounts for the tool, if the suppliers license model isn’t favorable to large user counts, this will take budget and resources to happen. Use license scaling as part of your evaluation assessment to truly understand the long term investment impact. Make sure that the workflows built around these tools are something that your team can manage in a reasonable amount of time. Additionally, check to make sure that you’ll have sufficient support for your solution. Presumably, the vendor sells or offers support — check that their reputation is good by asking in forums or finding user reviews.
6. Cost Effectiveness
What often gets overlooked in the purchasing process is the cost of the solution relative to the risk that you’re trying to offset with it. For example, purchasing a solution that’s only used for a situation where the impact of the risk is actually far lower than the cost of the solution. Take into account not just cost but the lifecycle cost to begin with.
7. Innovation and Future Proofing
Similar to scalability, another consideration is innovation and future-proofing. You don’t want to find out in a year that the solution you bought is already antiquated because technology or society has changed so much that your tool doesn’t work anymore. The recommendation is to obtain the vendor roadmap under NDA before or during an evaluation, ask them pressing questions about the future product vision, and establish if that fits your strategy. When you’re in the planning stages on what you are going to use and how you’re going to use it, consider what technology might be changing or what new technology might come on the scene. Obviously, it’s not possible to plan for every possible future outcome, but keep your ears open for what might be coming in the future.
8. Reporting and Analytics
Next, consider reporting and analytics. Make sure that the tool is going to be able to tell you something meaningful. A tool that just produces graphs and charts may be useful to some of your team, but some of your team may need statistics. Some reporting that comes from it may need to go to the board. You want to be able to make sure that the tool can produce the type of data and results that you can communicate clearly and effectively to all the various stakeholders, whether they be the practitioners on the ground or whether they are in the boardroom.
9. User Reviews and Case Studies
Finally, ensure both you and your team take time to read user reviews and case studies to learn more about which tools might be best for you. As a C-level executive, I receive all kinds of advertisements about products — and many of those are actually worthwhile reading, like the case studies that they publish. These allow you to see how someone else actually used this tool for the very same thing that you’re looking to use it for, or of equal value, knowing what to avoid investing in.
Finding Better Security Solutions Today
Your cyber landscape is likely going to expand, becoming more challenging to manage and exposed to an increasing array of sophisticated adversaries — which is why security teams need the right tools and solutions to stave off those attacks. By evaluating a new solution’s performance, how it integrates, scalability, cost-effectiveness, and more, CISOs can be certain they’re not just investing in the right tool for today, but the right one for tomorrow as well.
