10 Principles for Better Transaction Monitoring

Building an effective and efficient transaction monitoring system (TMS) continues to be a challenge for most financial institutions (FIs). Meeting table-stake requirements, such as data quality and completeness, a sound tuning methodology, and robust model governance, may help keep the regulators at bay. However, building a cost-effective and efficient transaction monitoring system requires financial institutions to go much further.

In the Wolfsberg Group’s recent statement on effective monitoring of suspicious activity, two challenges are highlighted:

1. Demonstrating expanding red flag and typology coverage across the entire customer and product range, even when an FI’s data shows that these result in little to no escalations when performed via systemic monitoring.
2. Reporting potentially suspicious activity in all cases in which red flags and typologies potentially indicative of financial crime have set alarms and in which the legitimacy of the underlying transactions could not be fully verified.

Addressing these challenges demands a more principled scientific approach rooted in probabilistic reasoning and causal inference. Here, I propose 10 principles that are essential to building such a system.

1. Every institution’s transaction monitoring system should be underpinned by a provisional “Theory of Financial Crime” in that institution’s universe.

Every institution should have a “Theory of Financial Crime” that identifies:

1. major risk factors or red flags,
2. financial crime typologies of interest (e.g., human trafficking) and
3. the controls (e.g., models and scenarios) that are in place to mitigate those risks.

Explicitly laying out such a theory will make an institution’s assumptions transparent to regulators while making it easy for an institution to recognize gaps and inadequacies in their program.

The theory itself can be captured by a symbolic and potentially causal model such as a Bayesian network.

2. Use probabilities to quantify uncertainty.

Anti-money laundering (AML) is a highly uncertain discipline. It is impossible to say with certainty how money launderers operate, so every red flag, alert, case or label has a certain irreducible level of uncertainty. Probability is the science of uncertainty, and FIs need to include a probabilistic estimate with every metric.

The institution’s theory of financial crime and the model that captures it should be probabilistic. Every prediction made by a TMS, such as an alert or a case, should have a probability that estimates the likelihood of the case being truly suspicious. When a compliance office identifies a red flag of interest to an institution, this should be accompanied by a prior probability.

One way of quantifying the quality of a red flag is the Bayes factor or likelihood ratio. Only a probabilistic approach can help institutions take a risk-based approach to AML.

3. Financial institutions should employ the scientific method in a principled manner to continuously improve while ensuring defensibility.

The key tenet of the scientific method is the revision of theories based on collected evidence. Financial institutions often end up doing things that don’t work just to placate regulators, because they are not applying the scientific method in a principled way.

The TMS is a manifestation of the “Theory of Financial Crime”; therefore, every alert and case represents an experiment to evaluate the theory.

By ensuring investigators provide the right feedback, institutions can update their “Theory of Financial Crime,” and consequently their TMS, in a defensible way. If an investigator determines that a case is suspicious for a reason not accounted for in the institution’s “Theory of Financial Crime,” then it indicates the theory needs to be updated to account for new risk factors and red flags. If an investigator determines the case is suspicious for a reason accounted for in the institution’s theory, it will increase the belief in the institution’s theory.

This is how the first challenge identified by the Wolfsberg Group can be resolved.

4. Priors matter.

Understand and apply the Bayes rule to avoid base-rate neglect. For example, consider a medical test for a rare disease with a 1% prevalence rate. If the test has a 99% accuracy rate, a false positive result might still be more likely than a true positive because of the low base rate of the disease.

Similarly, in AML, the prior probability of a customer being a money-laundering risk significantly influences the posterior probability after an alert. The posterior probability of a customer being a money-laundering risk given that a set of scenarios trigger an alert can vary significantly depending on the prior likelihood of the customer being a money launderer.

Segmenting customers into precise, homogenous groups ensures accurate assessment of prior probabilities, enhancing performance of the TMS.

5. A case is suspicious or “SAR worthy” because the focal entity’s activity conforms with one or more risk factors or red flags as outlined in an institution’s “Theory of Financial Crime.”

The 5 W’s (who, where, what, when and why – and how) are critical to an AML investigation. The hardest to pin down among these is the “why?”

Answer the question “Why is the activity suspicious?” by looking at whether the activity of the customer is consistent with one or more risk factors or red flags of interest to an institution. The institution’s “Theory of Financial Crime” will allow us to infer the cause or “why?” behind any alert or case.

The answer to the “how?” is the evidence that explains how the observed activity is consistent with the red flag or risk factor.

6. AML investigations are a form of hypothesis testing.

Every AML investigation is a form of hypothesis testing in which the null hypothesis and alternative hypothesis can be framed as:

H0: The focal entity did not carry out activity consistent with the red flag.

HA: The focal entity did carry out activity consistent with the red flag.

The hypothesis to be tested is always an implication of the underlying theory. An institution’s “Theory of Financial Crime” or, more concretely, the causal model that captures it should be queried to identify and prioritize the hypotheses an AML investigator should test.

The investigation itself should use the evidence collected to accept or reject the hypotheses.

7. Red flags are falsifiable, and all red flags are not created equal.

When regulators publish a red flag they are fairly broad. Only an institution’s risk assessment can determine if they are relevant to an FI. Furthermore, some red flags may be of higher quality (i.e., more specific and easier to detect). A red flag identified as relevant may also turn out to be immaterial. An institution might also realize later that it needs to consider red flags that it didn’t originally consider. This suggests that institutions should have a mechanism to assign different weights to red flags based on their quality as well as a feedback mechanism to “upvote” or “downvote” a red flag.

Having an explicit theory of financial crime that is revised based on investigator feedback can do just this.

8. Activity consistent with a red flag is a necessary – not sufficient – cause for suspicion.

If an entity demonstrates activity that is consistent with a red flag or typology, it is not necessarily suspicious. This is particularly so in the case of low-quality red flags.

As the Wolfsberg Group’s statement highlights, in the absence of evidence to confirm the illegitimacy of this activity, reporting all such activity is unlikely to be helpful. In such a case, a risk-based decision must be made. Specifically, the posterior probability of the case being effective in the light of all available evidence should be considered, including the quality of the red flags in question.

If this posterior probability is greater than a threshold as determined by the risk tolerance of the institution, it may make sense to report such cases. Calculating this posterior probability will constitute the evidence that can explain a decision whether to file a suspicious activity report (SAR). Demonstrating this for each SAR that an institution chooses NOT to file, can address the second challenge highlighted by the Wolfsberg group.

9. Synthesizing signals from all available systems is essential to generate higher-quality predictions.

Core AML systems, such as transaction monitoring, Know Your Customer (KYC), customer due diligence (CDD) and sanctions, often operate in silos. To create richer, more meaningful cases, signals from all these systems as external data sources should be synthesized and consolidated.

A particular red flag that might be of low quality in normal circumstances could become highly relevant in the presence of negative news on the entity. A model that captures the causal logic of how all these signals interact can be used to estimate the posterior probability of a case being suspicious given all these signals.

10. Machine learning models are the future, but post hoc model explanations are insufficient.

Machine learning (ML) models represent the future of transaction monitoring. Even large, regulated institutions have replaced rule-based scenarios with ML models. Even when institutions have clean labels and high-performing models, investigating events generated by these models can be challenging.

If an institution develops an ML model based on all available data, it is likely to integrate signals across various risk factors and red flags, making investigations challenging. Therefore, it may be necessary for institutions to clearly state the red flags of interest for a given model up front and only consider those labels when developing a model.

Although post hoc explanation techniques such as LIME or SHAP are available, these only provide feature importances that are very hard for a nontechnical investigator to make sense of.

Further, ML models by design are inductive. They only care about association and not cause. Predictions of ML models or post hoc explanations will not have causal logic, which is essential to answer the “why?” of AML investigations.

This suggests that ML models also have to fit into an institution’s “Theory of Financial Crime.” The theory can then be used to derive the hypotheses an investigator can test when a case gets generated.

Conclusion

The scientific method has transformed humanity in the last 500 years. Applying the scientific method and ideas from probability is key to building an effective, efficient and compliant transaction monitoring system. It offers a systematic and principled approach to make risk- and evidence-based decisions while continually iterating toward a better solution.

I believe the principles laid out here are critical to addressing concerns of financial institutions and regulators.