Solution

• Intelligence collection and analysis activities produce results tied with accuracy or confidence levels. Intelligence audience should not look at the intelligence information while ignore its accuracy. Rumors may convey important, serious, or valuable information, but with very low accuracy and hence significance. Intelligence can vary from “facts: with very high confidence” to “rumors, or assumptions: with very low confidence.”

• Data analysis algorithms (e.g., clustering, classification, prediction) produce different types of performance measures or metrics (e.g., confusion matrix, area under curve, AUC, F-measure, root mean square error, RMSE). For data analytics, it is very important to study all relevant performance metrics to an analytic activity, their implications, meanings, interpretations, etc.

• For intelligence users, trying to study and understand such metrics can be very time-consuming and confusing. Hence, it is the job of intelligence collection and analysis teams to create accuracy or confidence levels that are more readable or easier to interpret and understand by users who are typically with no technical background/skills. For example, accuracy of intelligence collection or analysis results can be summarized into three confidence levels:

High confidence

• Some aspects that support this selection:

  • Intelligence is correlated from more than one source

  • Intelligence is correlated from more than one system/tool

  • Source of information is trustworthy (e.g., based on previous intelligence)

  • Minimum assumptions and strong logical reasoning/inference

Medium confidence

• Some aspects that support this selection:

  • Intelligence is partially collaborated from more than one source.

  • Intelligence source or system is partially tested or has previous accepted levels of confidence.

  • Low contradictions, assumptions, etc.

Low confidence

• Some aspects that support this selection:

  • Intelligence system or source is new, unverified, etc.

  • Several assumptions and/or contradictions exist.

  • Intelligence comes from difference sources with conflicting information.

• Such three levels’ categorization of the accuracy makes the assessment simple for users or decision-makers while at the same time help them always correlatenintelligence with accuracy or confidence levels. As confidence levels indicate probabilities, a continuous percentage range (i.e., from 100 to 0%) can be used where 100% indicates top or absolute confidence and 0% indicates no confidence. Percentage confidence can also be converted to ranges (e.g., >90% highly likely,60–90% probable, and 40–60% possible).

• It is important as part of showing the right confidence level is to use the right terms when expressing intelligence information. Here is a list of possible terms to use in each level:

• High confidence: Certainly, most likely, etc.

• Medium confidence: Likely, probably, etc.

• Low confidence: Possibly, may or may not, etc.

• Different levels of confidence can also be associated to different components of the same intelligence case. Each statement or information can be given its own confidence level and then the overall intelligence case can be given one unified confidence level.