Solution

• It is important for intelligence users to understand each intelligence system or tool capabilities and limitations; what they can and what they cannot do. This will help saving time and resources through all the security intelligence lifecycle through optimizing the usage of those resources and setting the right expectations.

Intelligence Capabilities

• Intelligence systems/tools can help in one or more of the following tasks.

• Support in the decision-making process (long and short term). This is a major goal in all information and intelligence collection activities. Current decisionmaking processes require support of information and intelligence at different level of details and sources. The quality and timeliness of the right information can be an invaluable asset to any decision support system.

• Long- and short-term alerts of potential threats. This is also another major goal for most of security intelligence collection and monitoring activities. The types and natures of alerts that intelligence systems can provide should be communicated properly between intelligence providers and consumers to make sure they are on the same levels of expectations.

• For example, an intelligence collection system can fall from the “can” to the “can’t” based on the threat alert level of details, accuracy, timeliness, etc.

• News alerts. Decision-makers may need to be summarized on certain types/categories of news that are happening in some countries, industries, user groups etc.

• Security and situational awareness. In addition to news’ alerts that are directed to a limited category of audience (e.g., decision-makers), security awareness programs target a larger category of audience. For example, recently, phishingattacks (through emails, SMS messages, etc.) showed significant increase in volumes. This triggers many US organizations to conduct security awareness programs to inform employees of how to best deal with such security threats.

• Reports on specific topics: For example, a crime or incident, local, national, or international, may trigger different types of security intelligence collection and analysis activities (See digital forensics, malware analysis, etc.).

• Persons of interest (PoI) intelligence collection. This can be strategic, long term, ongoing based on certain profiles, or can be targeted based on specific national or international incidents.

Intelligence Limitations

• Different types of limitations can be discussed about intelligence systems. One important aspect in this regard is to understand each system limitations and communicate such limitations properly with intelligence collection, analysis, and usage teams.

• For different reasons (e.g., job security, system acquisition problems), miscommunication between those different teams will eventually different types of problems.

Predictive vs Prescriptive Analytics

• Intelligence limitations are affected by intelligence collection and analysis challenges. One of the main intelligence challenges is that in many cases expectations are not clear. In other words, evaluating the accuracy and success measures on the intelligence collection and analysis is not trivial.

• We described, for example, one simple success factor, for most security intelligence systems; the ability to successfully detect/mitigate all security threats. Clearly this is simple to say and unrealistic to expect. The security intelligence process is a continuous and evolutionary process, we continuously learn from the past to improve future responses.

• The ability to alert and mitigate for future security threats can be seen as part of a larger scope (the ability to predict future events or activities). Some collection activities target predictive analytics, rather than intelligence.

• Intelligence in most of the tasks we described earlier is considered as “Prescriptive analytics”: What happened in the past and what is happening now. Predictive analytics (PA) focus on predicting what will happen in the future. Intelligence and PA complements each other; the best models of PA are those who are built on rich levels or prescriptive analytics.

• Predicting future events face limitations related to the accuracy of the prediction. We collect certain variables related to those events and in our prediction models, we assume, for practical reasons that only those variables will impact the occurrence of such events.

• A typical PA model will have a “target, class, label” variable and a number of “predictor” variables. We typically include predictor variables that we can collect. In reality, in most cases, we will have missing variables; variables that influence the occurrence of those events that we can’t collect data about, or we can with very low accuracy.