CQC Research and Evaluation team with Professor Carl Macrae.
Regulatory sandboxing is a new collaborative approach for CQC. It helps us to understand how to regulate innovative services.
CQC ran 3 regulatory sandboxes in 2019/20:
- Round 1: clinical services that use digital triage algorithms
- Round 2: clinical services that use artificial intelligence as part of screening or diagnostic services
- Round 3: personal assistants (homecare) managed by digital agencies - we put this round on hold during the coronavirus pandemic.
To deliver the sandboxes, we worked with:
- organisations that are delivering innovative services (the participants)
- other stakeholders, including regulators
- people with experience of care.
We evaluated the 3 rounds to help us decide whether we should run more sandboxes and to capture our learning. The evaluation focuses on the experiences of:
- the participants
- the CQC sandbox team
- other stakeholders in health and social care.
This paper brings together the evaluation findings from across all 3 rounds. It incorporates evidence from round 2 collected by Carl Macrae, Professor of Organisational Behaviour and Psychology at the University of Nottingham, as part of a project supported by Wellcome. The Wellcome project is exploring the governance and regulation of artificial intelligence in healthcare.
Findings from the evaluation
The sandbox was an effective way of addressing regulatory challenges.
The collaborative and fast-paced approach worked well. We were also able to understand the perspectives of a wide range of stakeholders.
Sandboxes need a clear scope.
We produced scoping documents for each sandbox. But it was not clear which types of services to include for round 3. This mean that asking ‘what does good look like?’ was the wrong starting point. To address this, more tailored questions should be included in future rounds.
Intensification of processes allowed regulatory challenges to be rapidly addressed.
The sandboxes allowed us to consider the regulatory challenges quickly and efficiently. The speed of the sandboxes created enthusiasm and momentum. But this also created challenges when the process needed to deviate from the timeline. This was particularly true in round 3. It took longer than expected to register the participants as services regulated by CQC.
Multidisciplinary teams proved valuable.
Having a multidisciplinary team working on the sandbox meant that people from a range of specialisms across CQC could contribute. The sandboxes also drew upon the contributions of people with lived experience of care. This helped keep a focus on enabling services to provide the right care for the people who need it.
Collaboration and co-production were key to the success of the sandboxes.
Most participants were engaged throughout the process. They were encouraged by the opportunity to play a central role in shaping the regulatory approach. The openness and explorative nature of the sandbox excited both CQC colleagues and sandbox participants. The workshops brought together the perspectives of all involved. They enabled a collaborative ethos which enthused participants and stakeholders. Participants also appreciated the opportunities to network and share challenges.
The sandbox approach helped to build better relationships with stakeholders.
We have built better relationships with stakeholders who were part of the sandbox. This has also helped to define regulatory boundaries with some other regulators.
Commercial sensitivities can create challenges for sandboxes.
In rounds 1 and 2, there were challenges around handling commercial sensitivities. This included concerns about intellectual property. This was managed through non-disclosure agreements and building trust between participants.
Sandboxes need to be clear about how findings and recommendations are acted upon.
The sandbox outputs have been well received because they draw together the perspectives of a wide range of stakeholders. Several recommendations from the sandboxes have been taken forward. But participants, CQC staff and stakeholders said that it could have been clearer how the recommendations would be implemented. A clear plan for implementing the recommendations should be started right at the beginning of the sandbox.
This work has improved many of the participants' and stakeholders' perceptions of CQC.
At times the sandbox trod a difficult line between supporting experimentation and CQC’s role as the regulator. Despite this, CQC was able to balance that risk. CQC showed an openness to innovation and a desire to work with providers and stakeholders in new ways.
Regulators' Pioneer Fund
This was made possible by a grant from the £10m Regulators’ Pioneer Fund launched by The Department for Business, Energy and Industrial Strategy (BEIS) and administered by Innovate UK.