Intelligence Analysis

intelligence_analysis_notes.txt

notes on the field of intelligence analysis

summary: it appears there is a mature science being applied by police, military, and intelligence organizations
    and that these tactics, techniques, and procedures have a direct correlation to 
    the emerging modern $190B industry of information security / cyber security ...
    yet most cybersec/infosec professionals are aware of its existence, certainly not practicing it, much less sworn to it.
    this is of concern as many groups are learning costly/painful lessings reinventing-the-wheel with regard to ethics.
    but there is also a race to introduce AI/ML in this space, but so few commercial vendors are aware of the academic body of work
    on information/intelligence analysis that already exists from the pre-9/11 intelligence community.

it started with researching the origin of the
  "link chart" / [social] graph / link diagram / link analysis /investigative chart / link board / conspiracy wall / "connect the dots"
  Anacapa charts
  Law Enforcement and Intelligence Analysis
    Development of Systems and Procedures
      Crowd Management Research and Development
      Computer-aided intelligence analysis
        produces network diagrams and flow charts from fragmented information inputs, in response to specific analytical requests of users
        Criminal Network Analysis (CNA)
          a four-day training course (Advanced Intelligence Analysis) to train investigators and intelligence analysts
          Critical Thinking Training for Intelligence Analysis
            a web-based training program
            "While in the last decade or so a number of useful tools have been developed to aid data collection, evaluation, collation and integration, 
            analysis remains highly dependent on the cognitive capabilities, specifically the critical thinking skills, of the human analyst."
            understand the inherent capabilities and limitations of the analyst and,
            in particular, the cognitive challenges of intelligence analysis ...
            principal challenges are COMPLEXITY, BIAS, UNCERTAINTY and DOMAIN EXPERTISE.
            the ultimate objective of intelligence analysis:
              the development of inferences that can be acted on with confidence!
            provides a comprehensive view of critical thinking skills and maps them to intelligence functions and tasks
              (over 11 modules addressing specific crit think skill)

            Analysts and investigators from more than 1,500 law enforcement agencies at the federal, state, county, and municipal levels 
            have now attended these Anacapa training courses. (California Department of Justice and Anacapa Sciences, Inc.) 

      Analyzed the organizational and training requirements
        Provided a plan for incorporating an analytical investigation function within the office
        (a tech tree; a syllabus; a career ladder; a self-metric and organizational goal)
        system analysis effort designed to reduce fraud, abuse, and waste within the Department

      Digital Forensics Investigation Training

  Anacapa Sciences, Inc. is an employee-owned corporation formed in 1969 by Jim McGrath, Ted Parker and Doug Harris 
  to improve human performance in systems and organizations.

  Anacapa is the Chumash Indian word for an island off the coast of Santa Barbara, California; it means "visible through the fog."
  meaning ‘now you see it now you don’t’ -- usually shrouded in mist but occasionally is revealed by the sun in total clarity.

  Completed more than 1,000 separate projects for more than 250 government agencies, nonprofit and commercial orgs, incl.
    National Aeronautics and Space Administration
    Royal Aircraft Establishment of the United Kingdom
    Electric Power Research Institute
    Army Research Institute
    US Department of Transportation
    Singapore Police
    London Metropolitan Police (Scotland Yard)
    US Air Force
    US Army
    US Navy
    US Special Operations Command
    Hughes Training, Inc.
    State Farm Insurance Companies
    University of North Carolina

  - Research on human behavior and performance in systems and organizations.
  - Development of training in real and virtual environments.
  - Human-centered design and testing of systems, equipment, and procedures.
  - Analytical techniques for the investigation of complex criminal activities.
  - Software for assisted analysis and technical control and defense/protection technology.


  how do analysis classify crimes?
  what does a criminal intelligence analyst do?
  origins of network analysis
  Network Vulnerabilities and Strategic Intelligence in Law
  https://www.unodc.org/documents/organized-crime/Law-Enforcement/Criminal_Intelligence_for_Analysts.pdf

  Anacapa Sciences has long been recognised as the gold standard for analytical investigation. 
    This document is a one-upped modernized summary of their entire course material.
    The other half of which is based on the seminal CIA tome, "Psychology of Intelligence Analysis"
      tome: a book, especially a large, heavy, scholarly one.
      tomb: a large vault, typically an underground one, for burying the dead.
            a monument to the memory of a dead person, erected over their burial place.

  useful visualizations
    Time Event Chart
    Association / Activity Matrix
    Link Analysis Diagram
    http://crimeanalystsblog.blogspot.com/2016/12/introduction-to-link-analysis-part-3.html
    http://crimeanalystsblog.blogspot.com/2009/08/step-8-use-problem-analysis-triangle.html
    "Crazy Wall"
    https://www.esquire.com/uk/culture/film/news/a7703/detective-show-crazy-walls/

  Information
  - Knowledge in raw form

  Intelligence
  - Information that is capable of being understood
  - Information with added value
  - Information that has been evaluated in context to its source and reliability

  Analysis (of either information or intelligence)
  - The resolving or separating of a thing into its component parts
  - Ascertainment of those parts
  - The tracing of things to their source to discover the general principles behind them
  - A table or statement of the results of this process

  [INFORMATION + EVALUATION = INTELLIGENCE]
  INFORMATION + ADDED VALUE = INTELLIGENCE
  INTELLIGENCE + HUMAN REVIEW = ACTIONABLE

  ANALYSIS GOES BEYOND THE FACTS
    It can tell you how good (or poor) your information/intelligence is
    It can tell you things you didn’t know before
    It can tell you what you need to know to understand a situation
    It can tell you where to look further
    It can help you to communicate your understanding to others

   Intelligence: the process of interpreting information to infer and ascribe a meaning; processed information. 
   Can also describe a department of people [or machines] who perform the processing activity.
   
   Narrowed down to law enforcement use, “intelligence” could be described as information that is
   ACQUIRED, EXPLOITED and PROTECTED by the activities of law enforcement institutions 
   to decide upon and support criminal investigations.

   It's so crazy how applicable this is to our job, yet we have no formal training or awareness, much less sworn oath,
   as to the ETHICS of our jobs. We are essentially investigative detectives, billed as cops, but not trained like either.
   
  [INTELLIGENCE: KNOWLEDGE (PROCESSED INFORMATION) DESIGNED FOR ACTION]
  ACTIONABLE INTEL = RELEVANT INTELLIGENCE DESIGNED FOR ACTION

  "Intelligence always involves a degree of interpretation resulting in an inevitable degree of speculation and risk. 
  The amount of speculation and risk is dependent upon the quality and quantity of information. 
  Intelligence is usually divided in two main areas:"

  Operational intelligence: Typically provides an investigative team with hypotheses and inferences 
    concerning specific elements of illegal operations of any sort. 
    These will include hypoth eses and inferences about specific criminal networks,  
    individuals or groups involved in unlaw ful activities, discussing their methods, capabilities, vulnerabilities, 
    limitations and intentions that could be used for effective law enforcement action.

  Strategic intelligence: Focuses on the long-term aims of law enforcement agencies.
    It typically reviews current and emerging trends changes in the crime  environment, 
    threats to public safety and order, opportunities for controlling action and the development of 
    counter programmes and likely avenues for change to policies, programmes and legislation.

  Operational intel is a prereqresite (should be the first goal prior) to have good Strategic intel.

  EVIDENCE: Data from which to establish proof.

  The Intelligence Cycle
  tasking->collection->evaluation->collation->analysis->"inference development"->dissemination  

  COLLECTION: THE GATHERING OF DATA
  COLLECTION PLAN: A FORMALLY DEFINED APPROACH TO DESCRIBING THE INFORMATION, JUSTIFICATION NEEDED, AND MEANS OF ACQUIRING IT
  EVALUATION: AN ASSESSMENT OF THE RELIABILITY OF THE SOURCE AND THE QUALITY OF THE INFORMATION
    we should be (xkeyscore) scoring our information (ie. if from humans on slack, low score)
    https://en.wikipedia.org/wiki/XKeyscore
    https://fossbytes.com/xkeyscore-nsas-search-engine-to-hack-into-your-lives-is-as-simple-as-google/
  COLLATION: THE ORGANIZATION OF THE DATA COLLECTED INTO A FORMAT FROM WHICH IT CAN BE RETREIVED AND ANALYSED
  ANALYSIS: THE CAREFUL EXAMINATION OF INFORMATION TO DISCOVERS ITS MEANING AND ESSENTIAL FEATURES
  DISSEMINATION: THE RELEASE OF THE RESULTS OF ANALYSIS TO THE CLIENT
    reports, leaderboards, dashbaords

  DATA INTEGRATION: COMBINING DATA IN PREPARATION TO DRAWING INFERENCES
  DATA INTERPRETATION: GIVING THE DATA A MEANING; GOING BEYOND THE INFORMATION AVAILABLE

Common charting techniques:
  Link charting
    show relationships among entities featuring in the investigation
  Event charting
    show chronological relationships among entities or sequences of events
  Commodity flow charting
    explore the movement of money, narcotics, stolen goods or other commodities
  Activity charting
    identify activities involved in a criminal operation
  Financial profiling
    identify concealed income of individuals or business entities and to identify indicators of economic crime
      for us this could be people who make the most api calls or similar 'in-app purchase/economy' type metrics
  Frequency charting
    organize, summarize and interpret quantitative information
  Data correlation
    illustrate relationships between different variables

  for correlation i should be using
    http://intelmsl.com/insights/other/some-notes-on-the-origins-of-network-analysis-part-2-the-age-of-anacapa/


Four types of inferences:
  Hypothesis
    a tentative explanation, a theory that requires additional information for confirmation or rejection.
  Prediction
    an inference about something that will happen in the future.
  Estimation
    an inference made about the whole from a sample, typically quantitative in nature.
  Conclusion
    an explanation that is well supported.

Source evaluation (AI scoring strategy)
  A
    No doubt regarding authenticity, trustworthiness, integrity, competence, or History of complete reliability
  B
    Source from whom information received has in most instances proved to be reliable
  C
    Source from whom information received has in most instances proved to be unreliable
  X
    Reliability cannot be judged

Information evaluation (AI scoring strategy; degrees of separation)
  2
    Information known personally to the source but not known personally to the official who is passing it on.
    Agrees with other information on the subject (strongly weighted/connected)
  3
    Information not known personally to the source but corroborated by other information already recorded
  4
    Information which is not known personally to the source and can not be independently corroborated

Dissemination Handling Codes (similar to ACL but we should also make it explicitly printed as "do not circulate outside of X group" somewhere on each page)
  1
    Dissemination permitted within law enforcement agencies in the country of origin.
  2
    Dissemination permitted to other national agencies.
  3
    Dissemination permitted to international law enforcement agencies.
  4
    Dissemination within originating agency only.
  5
    Permits dissemination, but receiving agency to observe the conditions specified.

any inference should contain: (like our indicators, each event should be enhanced by machine or human until it reaches 100% completeness score)
  Key individual or individuals - WHO? 
  Criminal activities - WHAT? 
  Method of operation - HOW? 
  Geographical scope - WHERE?
  Motive - WHY? 
  Time-frame - WHEN?

AXIOMS FOR AN INTELLIGENCE ANALYST
  Believe in your own professional judgment
      You are the expert. Believe in your work and stand your ground if the intelligence supports your position
  Be a risk taker
      Intelligence is perishable; its usefulness often decreases exponentially over time.
      Knowing when to call an analysis "good enough" is essential.
      Don't spend too long refining it in order to be a hero or play it safe.
      Optimize for efficiency, and entrust accuracy as an exercise for the reader or upon follow-up request. (see also: the 80/20 Pareto principle)
      Be aware of the conditions under which additional information may not add to the accuracy of your inference. Stop when your sample size is statistically significant.
      Do not be afraid of being wrong when forecasting trends or events.
      Taking risks is part of your job description. Only by taking risks you can maximize your value to your agency.

    6.4.1 Consider value-cost-risk tradeoffs in seeking additional information  

    A dilemma faced by intelligence analysts is whether to stop and report an inference based on 
    available information, or to collect additional information. More information might produce an 
    inference  with  greater  usefulness  at  a  higher  level  of  confidence,  but  seeking  additional 
    information adds to intelligence costs and also risks a result that is not timely enough to be of 
    value. This dilemma might be encountered early in the intelligence process or, more critically, 
    ater  during  the  testing  of  hypotheses.  This  skill,  then,  is  the  ability  to  evaluate  the  need  for 
    new information by considering the value, cost and risk tradeoffs that are involved.  
      
    The analyst faces value-cost-risk tradeoffs principally during the stage of analysis in which
    hypotheses  are  being  tested;  this  is  a  critical  part  of  the  process  of  developing  a  useful  
    inference. Typically, one or more hypotheses would have been developed at this stage of the 
    analysis and additional information might be required to help confirm or refute them. With 
    limited time and resources available for collecting additional information, the analyst must 
    employ these resources in the manner that will produce the greatest value for the resources 
    expended.  The  analyst  must  also  be  sensitive  to  producing  an  inference  in  sufficient  time 
    and at a high enough level of confidence for it to be of use.

    pg. 227
    http://cdn.intechopen.com/pdfs/35820/InTech-Critical_thinking_skills_for_intelligence_analysis.pdf

  Intelligence is of no value if it is not disseminated
      Identifying correlation, finding probable causation, and proving causation are a series of exponentially more costly steps in the refinement of an analysis. 
      In a large operation, these are typically separate duties and departments.
      If you are stretched too thin, err on the side of dissemination of potentially actionable but under-analyzed information.
      Communicate the intelligence, conclusions and recommendations clearly and effectively and in a timely manner.
      What your client does not know has no value.
  Make your assumptions transparent
      Understand and respect the limitations of our (human) mental equipment
      Overcome the limitations of working memory in the face of complexity by externalizing, chunking and filtering
      Be prepared to show your linkages from data to premises to inference, and the basis for the assessment of your inference
      Be aware of and combat the negative influences of cognitive biases—the power of vivid, personal information, the problem presented by the lack of key information, and the negative impact of confirmation bias
      Always indicate the confidence one should place in an inference by providing a numerical probability assessment
  It is better to make a mistake than to do nothing at all
      If you are wrong, and the facts call for it, admit it. Only those who don’t do anything make no mistakes.
  Avoid mirror imaging at all costs
      Mirror imaging is projecting your thought process or value system onto someone else. Your targets are criminals. 
      Their mentality is completely different. You must learn to think like they do.
  When everyone agrees on an issue, something probably is wrong
      It is rare and not natural for a group of people in the intelligence community to fully agree on anything.
      If it does occur, it’s time to worry.
  Your client does not care how much you know, tell them just what they need to know 
      Excessive details merely obscure the important facts.
  Form is never more important than the substance
      A professional appearance and appropriately selected formats are important, but they do not outweigh substance. 
      Clients want to know what intelligence means, and they want it when they need it.
  Aggressively pursue collection of information that you need
      Never settle for less than all you need. If you fail to get access to the vital data source for any reason, you will be held responsible.
  Do not take the editing process personally
      If editorial changes do not alter the meaning of your message, accept them. 
      If they do, speak up. Even then, it might be that a brighter mind has seen what you have missed.
      Believe in your product, but be self-critical.
  Know your intelligence community counterparts and talk to them
      You are not competitors; you are of the same breed.
      Become part of the network. Do not pick up the phone only when you need something.
  Do not take your job, or yourself, too seriously
      Avoid burnout. Writing you off as an asset will be a net loss to your agency (although it may not immediately see it exactly like this). 
      The welfare of your family and your health is more important than nailing down a criminal, or scaling another rung on the career ladder. 
      Your role in the larger order of things is not self-important.
      Your commitment, perseverance and dedication to the job will bring results only over a long term.

TEN STANDARDS FOR ANALYSTS
  1. Analysed data (i.e., intelligence) should be used to direct law enforcement operations and investigations
  2. Analysis should be an integral part of every major investigation the agency pursues.
  3. Analytical products should contain, as a minimum, a written report. Visual products may also be presented, 
    but are only acceptable as an addition to, rather than in replacement of, a written report.
  4. Analytical products should contain conclusions and recommendations. These are presented to management for their consideration regarding decision-making.
  5. The development of an analytical product requires the application of thought to data. Data compilation that does not reflect comparison or other considerations is not analysis.
  6. Analytical products must be accurate. Consumers must be able to rely on the data provided to them by analysts.
  7. Analysis must be produced in a timely manner.
  8. Analytical products should reflect all relevant data available through whatever sources and means available to the analyst.
  9. Analyses should incorporate the best and most current computer programs, compilation, visualization, and analytical techniques available in the analyst’s environment.
  10. Analyses should both reflect, and be evaluated upon, their qualitative and quantitative contribution to the mission and priorities of 
      the agency or organization for which they are being produced.

Basis analysis techniques (link analysis)
  this whole chapter is amazingly good. from pg 35 on!
  this should all be automated into AI data structures and scores
  to automatically draw correlations

THE IMPORTANT FEATURES OF A NETWORK ARE CONTAINED IN FOUR CONCEPTS: (edge scores in a graph)
  ENTITY, RELATIONSHIP, DIRECTIONALITY, STRENGTH

Figure 7-3. Example of an event matrix chart
  this is exactly what BigPanda is doing with their event timeline for operations! 
  we must abstract it to be a reusable timeline for any of our event data.

Flow analysis
  directional graph showing influence and control. A acts/acted upon B.

Call frequency table (telephone/comms analysis)
  association matrix for FROM id : TO id
    could be a count of interactions between two network identities

Probability is derived as a ratio between “number of times the event will occur” and “number of opportunities for the event to occur”.

There are three sources of probability estimates:
  Relative frequency of past events
    Where over a given period the number of times an event has occurred in the past is used as a guide to the likelihood of future events occurring.
  Theoretical estimation
    Where some definite formula, however derived, is used as a basis for prediction
  Subjective estimation
    Where the prediction relies solely upon the personal opinion or judgement, usually as a privilege of experience, expertise or position

Types of probability values:
  Simple
    the probability of occurrence of a single event
  Joint
    the probability of two events occuring at the same time
  Conditional
    the probability of a second event, given that a first event has occurred


Data Collection Tradeoffs
  a common dilemma faced by intelligence analysts: whether to report an inference based on available information or to collect additional information.
  In short, the decision to seek additional information requires tradeoffs among value, cost and risk. We want to get the greatest value at the least cost and without risking the timeliness of the intelligence product. 

Critical Thinking Strategies for Intelligence Analysis
  Externalizing
    Get ideas and concepts out of the head early and 
    into a model that shows key elements and how they relate—
    much of data integration involves techniques for doing this, such as link, flow, event, and activity charting
  Chunking
    Group information into meaningful “chunks” that facilitate its management and understanding.
    The inductive reasoning process used in intelligence analysis is facilitated by chunking collected data into a relatively small number of premises—no fewer than 5 or more than 9.
  Filtering
    Filtering requires laying out specific analytical objectives and filtering out information not appropriate to meeting those objectives
    Examples of Analytical Objectives:
      defining the flow of money into an organization
      clarifying the span of control of a key individual
      using only information above a specified validity
      tracking events during a specified period of time
      examining financial transactions above a specified amount
  Probability
    

TRAINING MATERIALS

this should be mandatory for IR/SOC and NOC teams
  Course Objectives:
    - Understand the role of the analyst, the components of the analytical process, and the importance of critical thinking.
    - Identify types of attacks (ddos, credential stuffing, keylogging, ransomwhere, phishing, etc.), their primary sources and the primary methods of distribution.
    - Identify sources of information, including non-infosec sources, computer databases, and confidential sources.
    - Understand how to search and filter graphs, flowcharts, event timelines, etc. to find what you are looking for.
    - Learn the purposes of link analysis, link charts, and association charts, communication analysis, and other correlation strategies.
    - Accurately describe attacker tactics: spoofing, tampering, data exfiltration, denial of service, elevation of privilege, advanced persistent threat
    - Learn about the regulation we are subject to: GDPR, SOX, PCI, etc.

  For the IR team (cont'd, level 2 from analyst => investigator)
    IQ ACCREDITED LEVEL 3 CERTIFICATE IN ANACAPA SCIENCES CRIMINAL INTELLIGENCE ANALYSIS
    http://www.focustraining.co.uk/home/course/IQ-Accredited-Level-3-Certificate-in-Anacapa-Sciences-Criminal-Intelligence-Analysis

    - Describe and apply the internationally recognised intelligence cycle
    - Describe the techniques of data collection and evaluation
    - Describe the components of the analytical process
    - Develop association matrices
    - Identify and target intelligence gaps
    - Construct charts of their analysis on any identified subject
    - Distinguish between inductive and deductive logic
    - Apply inductive logic to analysis
    - Identify the components of critical thinking
    - Apply case charting techniques
    - Conduct Concealed IP Analysis
    - Apply comparative case analysis techniques
    - Develop inferences from analysis
    - Deliver clear and accurate briefings

  Books and articles in the literature of cognitive psychology, such as Cognitive Psychology 4th Edition, Medin, Ross and Markman, Wiley, 2004.
  Psychology of Intelligence Analysis, Richards J. Heuer, Jr. Central Intelligence Agency, 1999. The book is readable or downloadable on-line at:
  https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/index.html

  Threat Hunting training that Allan probably received from SANS
  https://www.sans.org/cyber-security-summit/archives/file/summit-archive-1536354143.pdf
  http://folk.ntnu.no/chrhall/HALLSTENSEN_COINS_SummerSchool_DefendingNetworks_pub.pdf

tertiary degree programs
Degrees and Programs in Applied Intelligence 
https://scs.georgetown.edu/programs/423/master-of-professional-studies-in-applied-intelligence/curriculum
https://start.amu.apus.edu/intelligence/overview?gclid=EAIaIQobChMI0tzJuovq4wIVyJ-zCh3tJwpkEAAYASAAEgK81PD_BwE
https://www.dia.mil/Training/National-Intelligence-University-NIU/
https://www.ciaagentedu.org/schools-by-state/

Trade school courses and certifications
http://gicis.co.uk/short-courses/policing/digital-forensics-investigation-training/
http://gicis.co.uk/short-courses/policing/investigation/

Free online course materials
https://sites.google.com/site/lawenforcementintelligence/training-on-line
https://web.archive.org/web/20120119100713/http://www.anacapatraining.com/webtraining/index.html
https://en.wikipedia.org/wiki/Intelligence_analysis
https://en.wikipedia.org/wiki/Intelligence_cycle