Digital therapy for chronic tic disorders and Tourette syndrome: early value assessment

3.1 Evidence gaps and ongoing studies

Table 1 summarises the evidence gaps and ongoing studies that might address them. Information about evidence status is derived from the external assessment group's (EAG) report; evidence not meeting the scope and inclusion criteria is not included. The table shows the evidence available to the committee when the guidance was published. The EAG did not identify any ongoing studies that may address the evidence gaps.

3.2 Data sources

Data could be collected using a combination of primary data collection, suitable real-world data sources, and data collected through the technology itself (for example, engagement data).

NICE's real-world evidence framework provides detailed guidance on assessing the suitability of a real-world data source to answer a specific research question.

The NHS England Secure Data Environment (SDE) service could potentially support this research. This platform provides access to high-standard NHS health and social care data that can be used for research and analysis. SDEs are data storage and access platforms that bring together many sources of data, such as from primary and secondary care, to enable research and analysis. They could be used to collect data to address the evidence gaps. The sub-national SDEs are designed to be agile and can be modified to suit the needs of new projects. Within an SDE, the data may be linked to other useful data such as that from primary care and could provide information on important confounders (for example, comorbidities).

The NHS Talking Therapies, for anxiety and depression and Mental Health Services Data Set (MHSD) are real-world data sets that could also be used to collect information about the impact that disorders have on mental health.

The quality and coverage of real-world data collections are of key importance when used in generating evidence. Active monitoring and follow up through a central coordinating point is an effective and viable approach to ensuring good-quality data with broad coverage.

3.3 Evidence collection plan

It is important that evidence generation is aligned with how the technology could be used in the NHS in the future: under the supervision of a healthcare professional following a formal diagnosis.

The suggested approach to addressing the evidence gaps for Neupulse is a longitudinal, parallel cohort study. The study will follow an intervention arm and a control arm over 6 months (ideally longer) and compare their outcomes.

Statistically robust data about the effectiveness of the technology in different subgroups requires very large study populations that are not feasible in this context. Analysis of any data that is available at the end of the evidence generation period will nevertheless support future NICE decision making, so relevant characteristics should be recorded.

The studies should enrol a representative population, that is, people who would be offered standard care, including behavioural therapy, without digital technologies. This may include face-to-face appointments and monitoring. The studies should compare people with tic disorders or Tourette syndrome using digital technologies for self-management with a similar group having standard care. Eligibility for inclusion, and the point of starting follow up, should be clearly defined and consistent across comparison groups to avoid selection bias. The technology is indicated for use with or without a formal diagnosis. It is likely that people participating in future multi-centre trials will have a formal diagnosis. Further evidence on the clinical impact of Neupulse and potential adverse events in people without a diagnosis should also be collected if it is being used in this way.

Data should be collected in all groups from the point at which a person would become eligible for standard care. The data from both the intervention and comparison groups should be collected at appropriate time intervals and up to a minimum of 6 months. Data from people in different centres, with comparable standard care and patient population, but no access to digital technologies for self-management, should form the comparison group. Ideally, the studies should be run across multiple centres, aiming to recruit centres that represent the variety of care pathways in the NHS.

Despite consistent eligibility criteria, non-random assignment to interventions can lead to confounding bias, complicating interpretation of the treatment effect. So, approaches should be used that balance confounding factors across comparison groups, for example, using propensity score methods. To achieve this robustly, data collection will need to include prognostic factors related both to the intervention delivered and patient outcomes. These should be defined with input from clinical specialists. Also, analysis should be stratified according to the severity of the tic disorder. Incomplete records and demographically imbalanced groups can lead to bias if unaccounted for. Data collection should follow a predefined protocol and quality assurance processes should be put in place to ensure the integrity and consistency of data collection. See NICE's real-world evidence framework, which provides guidance on the planning, conduct, and reporting of real-world evidence studies. This document also provides best practice principles for robust design of real-world evidence when assessing comparative treatment effects using a prospective cohort study design.

3.4 Data to be collected

3.5 Evidence generation period

The evidence generation period should be 3 years. This will be enough time to implement the evidence generation study, collect the necessary information and analyse the collected data.