A Federated Architecture for a National Data Library

The UK National Data Library: Technical White Paper Challenge, initiated by Wellcome and the Economic and Social Research Council (ESRC), sought visionary approaches to enhancing access to UK public sector datasets. Evaluated by an expert panel, this paper was recognised as a distinguished and successful response to the call, offering innovative insights for creating a National Data Library that benefits both the research community and the public good.

The call for a National Data Library (NDL) is timely. Granting authorised researchers controlled access to sensitive public data at scale will be key to enabling more efficient services and research within the UK. In 2021, the DARE UK programme was set up by UKRI with a mission to put the UK at the forefront of sensitive data research and innovation by assembling the tools, technologies, and standards needed to streamline secure data linkage and use. Since then, the programme has conducted public and professional consultations, landscape reviews, and programmes of research, development, and proofs-of-concept to assemble most of the key elements needed for a federated approach to the NDL.

Our focus is access to sensitive data for approved research purposes, and this white paper describes a comprehensive and self-consistent way to achieve this. In defining the overall architecture, we have assimilated not only the pioneering results from the DARE UK programme to date but also current thinking and practice from large-scale scientific infrastructures, data spaces, and digital public infrastructures both domestically and internationally. The associated HDR UK white paper provides a health data perspective.