Threat Intelligence: Quality versus Quantitiy


I really enjoyed Charlotte Henry’s article, “Is threat intelligence being devalued by an information overload?” in the Computer Business Review (read article). The central point of the piece is that it is likely that companies feel safer by raising large number of alerts, many of which are false positives, because these same alerts surely must contain the majority of true positives. But this attitude assumes that the organization will still be able to respond to a real breach quickly. In short, the industry is smitten with quantity over quality.

The article also points out some key contributing factors to the state of affairs. First, there is the obvious skill shortage – which is not surprising: you cannot expect to have a senior threat analyst straight out of college any more than you can have a senior programmer. Organizations also fail to triage incoming information as its coming in so that the most relevant information is quickly analyzed while data that can be consumed as time permits can be put aside… that is, if there ever is time.

I think these points are spot on but I don’t think she goes far enough. She misses a couple of key factors that contribute to the information overload. Most organizations simply do not have the will, let alone the wherewithal to properly staff a threat analysis team. In order to achieve quality analysis, you must have a diverse team so that their analysis is not plagued by self-confirmation bias. Assembling a good team of analysts will not be cheap which puts this goal out of the reach of most organizations. Basically, nobody wants to have more overhead work heaped on to their business operations. If that means that they strive for quantity over quality because it meets the minimum standard of quality to avoid negligence, that is what they will do.

A famous memo once declared, Bin Laden determined to attack the United States, which is an interesting tidbit representing countless hours of valuable analysis. What is an executive supposed to do with this? President Bush, despite his faults, could not be expected to alter the day-to-day operation of the government without realistic recommendations of how to mitigate the threat. There simply weren’t any. We cannot expect more from business operations leadership. The threat intelligence they pay for needs to be scalable so that their teams are fed pre-curated data along with prescriptive recommendations that are relevant to their business lines and, to some degree, to their specific policies and technologies. Once the heavy-lifting is done, these organizations can do the analysis needed to tailor fit and cost-out the recommendations. They can balance these costs against the potential loss and decide what to do, if anything, to mitigate the risk. Threat intelligence fails to provide these recommendations and therefore organizations prefer to not let the great be the enemy of the good enough (to avoid a charge of negligence).

Another problem that organizations face when assembling threat intelligence teams is creating the authority needed for these teams to do their jobs. Few business operations are comfortable with providing their security teams carte-blanche over their revenue generating business-lines. No online-retailer is going to block IP addresses en-masse knowing that it will cut their revenues, for instance. Nor are they going to risk a “short-term” cessation of activity while the security teams harden their operations. However, top-talent security analysts, given the skills shortage, are only going to work where they are not frustrated by “bureaucratic nonsense”. They can get a job wherever they want – and they talk to one another so people know where they have control. Why stick around where they are not appreciated? Once again, organizations seeking to balance business and security will be forced to accept quantity over quality.

In order to solve this problem executive leadership needs proof that there is return-on-investment for inconveniencing, let alone interrupting, their core businesses. They need to understand the benefit of implementing better security controls and policies before they budge. Unfortunately, security analysts are not in the business of writing business cases and there is a dearth of tools to help them build them.

In conclusion, threat intelligence needs to move beyond blindly providing data and putting the onus on their customers. They need to do the heavy-lifting and provide actionable information along with the ability to drill down into the details. Until this happens, threat intelligence will be a nice-to-have. This analysis is quite possible and I would love to talk to anybody interested in doing it.

Comments are closed. Posted by: on

Tags: ,