Dick Kamp & Bjorn Blom: Data quality as a foundation of reputation
This column was originally written in Dutch. This is an English translation.
By Dick Kamp, Director of Pension Investment & Risk, and Bjorn Blom, Data Scientist, both at Milliman
With the introduction of the Future Pensions Act (Wtp), accurate data is more important than ever. Continuous monitoring and smart AI checks help pension funds maintain members’ trust.
With the transition to the new pension contract, pension funds are now operating exclusively under defined contribution schemes. In this environment, data quality is of crucial importance, partly due to the strict definition of the Maximum Permissible Deviation (MPD) in the data quality policy.
Pension funds have deliberately set the bar high for accepting deviations: after all, once contributions have been transferred into the scheme, errors are immediately visible to members via their periodic statements. This makes data quality not only an operational issue, but above all a reputational one for both the fund and the sector. In this respect, a comparison with the banking sector is apt: there too, data in daily statements, transaction overviews and banking apps must be error-free, up-to-date and consistent. The fact that this is largely the case in practice demonstrates that structurally high data quality is indeed achievable in an environment characterised by frequent changes and direct customer access.
Why data quality is more important than ever in the new pension contract
Under the Wtp, each member is periodically informed about their personal pension pot and expected benefits. This means that even minor data discrepancies or errors are immediately visible and can lead to questions, concern or even complaints. A single discrepancy can undermine a member’s trust in the fund, and if this occurs more frequently, it affects the reputation of the fund and the sector. Just as in the banking sector, members here also perceive digital information as a direct reflection of reality: what appears in the app, on the portal or on the receipt must therefore simply be correct.
The high standards for data quality are underlined by the MTA: the threshold above which discrepancies are not accepted. This forces funds to no longer view data quality as a KPI to be measured periodically, but as a process to be monitored continuously.
From periodic measurement to continuous quality monitoring
Traditionally, data quality was measured periodically, for example once a quarter or following major changes. Under the new pension scheme, this is no longer sufficient. For the sake of transparency and direct communication with members, continuous monitoring is essential. Continuous quality monitoring should therefore not be viewed as a standalone control tool, but as a standard component of the risk management policy and the fund’s regular monitoring. It is important to emphasise that prevention is better than cure: the sooner discrepancies are identified and rectified, the smaller the chance that they will affect communications with members, benefit payments, administrative reports or supervisory information. This calls for a fundamentally different approach:
- Continuous monitoring: Data quality must be monitored on an ongoing basis, using automated tools that immediately detect and report anomalies. This prevents errors from accumulating until the next audit and enables rapid correction.
- Event-driven checks: Additional checks during key events, such as large data imports, system migrations or bulk updates, remain essential but are supplementary to the continuous monitoring.
- Feedback loops: Every correction or update must be automatically followed by a data quality check to prevent new inconsistencies from arising.
Modular data quality control
The intended module (or modules) consists predominantly of programmed, deterministic checks. These are automatically triggered as soon as a transaction is received. By performing both input and processing checks immediately, data quality is monitored even before the information becomes visible to the participant.
In addition to these automated checks, an extra layer can be added in stages, comprising checks based on Machine Learning (ML) and Artificial Intelligence (AI). These ML/AI techniques recognise patterns in large volumes of data and use them to improve predictions or classifications without being explicitly programmed for every possible situation.
The ML/AI checks are trained using the organisation’s own participant population, pension scheme and historical transactions. In this way, these controls are trained to determine which values are plausible and which changes may indicate errors. ML/AI models are particularly effective in trend analyses and plausibility analyses over longer periods (such as in data profiling). They flag anomalous patterns that fall outside the MTA range, even before these lead to operational errors.
ML/AI techniques thus provide a powerful complement to existing controls. They support the data manager and process owner by highlighting patterns and anomalies that might otherwise go unnoticed with manual or deterministic controls alone. It is important to emphasise that these technologies do not replace human oversight, but rather reinforce it: the final assessment and decision-making remain the responsibility of humans.
The data manager plays a key role in this. This is the officer responsible for setting up, monitoring and further developing the data quality framework within the fund or the implementing organisation. The data manager is responsible for defining data quality rules, monitoring outcomes, analysing the structural causes of deviations and coordinating corrective actions with process owners, IT and risk functions. This officer also ensures that data quality is demonstrably embedded in processes, reports and governance. Responsibility for the substantive accuracy of data remains primarily with line management and the process owners. The data manager facilitates, monitors and sets the agenda.
IT integration
The deterministic checks and the ML and AI functionality are being added as separate modules to the pension platform’s existing data pipelines and workflows. This enables data quality checks to be carried out automatically for every transaction, import or calculation, without the need to directly modify the core systems. The new modules to be developed will enable the platform to automatically carry out corrections, such as cleaning up addresses, removing duplicate participants and standardising input fields.
Via central dashboards, which are fed by the module, data managers, process owners and first-line risk managers gain continuous insight into current data quality, including immediate alerts in the event of deviations exceeding the MTA. All checks and corrections carried out via the module are fully integrated with existing identity and access management, logging and auditing systems, ensuring that it is always possible to trace who has made which changes.
The proposed modular integration is scalable and future-proof, and thus aligns seamlessly with modern requirements for pension administration.
Process and governance framework
With this modular approach, roles and responsibilities relating to data quality are clearly defined, from the data manager right up to the board. The results of the deterministic checks and the ML and AI analyses are automatically incorporated into regular fund reports and decision-making.
Any deviation above the MTA is, in accordance with existing policy, immediately fed back to the process owner, who initiates corrective actions. Where necessary, the data manager and the first-line risk manager are also involved to assess whether the issue is an isolated error, a structural process problem or a control issue. Once a correction has been made, a new check is carried out automatically, so that the effect of the improvement can be measured immediately. Periodic reports on data quality, generated by the module, are automatically shared with the board and the regulator, thereby demonstrating compliance with the Wtp and other legislation. Staff are trained to interpret the module’s results and take proactive action in response to indications of reduced data quality.
Examples
Every month, checks are carried out to ensure that benefit amounts in the pension administration match those in the benefits administration. Furthermore, thousands of transactions are processed during the monthly allocation of investment returns. The allocation of investment returns to pension capital is compared with the returns from the investment administration. Deterministic checks monitor direct consistency; ML/AI analyses assess whether the flagged values fit within the long-term patterns of the population. In this way, every transaction is immediately checked for deviations exceeding the MTA. These are highlighted on the dashboard, after which the data manager or process owner can take targeted action. The full audit trail is automatically updated and included in the report to the board.
Reputation and sectoral importance
As each member receives a statement and has direct insight into their personal pension pot, the fund’s reputation depends on error-free data. Structural errors or recurring deviations can undermine trust, not only in the fund but also in the pension sector as a whole. Investing in continuous data quality monitoring is therefore not a luxury, but an absolute necessity.
Conclusion: data quality as the foundation for the new pension fund
Under the Wtp, data quality is no longer an indicator to be measured periodically, but a permanent process that requires constant attention. The strict MTA standard and the need for transparency towards participants make a proactive, real-time approach essential. Machine learning and AI offer powerful tools for this, but the foundation lies in a culture of data-driven working, ownership and continuous improvement.
Start, therefore, by systematically and continuously ensuring data quality – in other words, ‘stay clean’ – and thereby lay a solid foundation for trust, transparency and a sustainable reputation in the new world of pensions.
This is the fifty-second column in a series on risk management. The series aims to encourage readers to view risk management as an integral part of running a pension fund.