Idebenone for treating visual impairment in Leber's hereditary optic neuropathy in people 12 years and over

Leber's hereditary optic neuropathy (LHON) is caused by mutations in the genes encoding mitochondrial DNA (mtDNA). Mutations in mtDNA disrupt the synthesis of adenosine triphosphate (ATP) and produce free radicals. This damages retinal ganglion cells and destroys the optic nerve. LHON is normally inherited, if a mother carries the mutation, it may be transmitted to children. The 3 most common mutations are 11778G>A, 14484T>C and 3460G>A. These mutations are found in around 95% of the LHON population. LHON typically leads to progressive vision loss, particularly in young adults, and mainly affects boys and men. The patient experts explained that the experience of living with LHON varies from person to person. They explained that, for them, LHON:

• came on rapidly

• is painless

• is subacute

• has caused severe loss of visual acuity (VA) and colour vision, and loss of central but not peripheral vision.
  
  They explained that, even though peripheral vision is usually preserved initially, it may also deteriorate over time. This can lead to being registered as blind. Blurring and clouding of vision are usually the first symptoms of LHON and start in 1 eye, with the second eye following a similar progression within 4 to 6 months. The clinical experts explained that LHON is usually irreversible, but that spontaneous improvement may occur in a few people with certain LHON mutations such as 14484T>C and 3460G>A. The clinical experts noted the lack of understanding of the cause and natural history of LHON. Its course is split into subacute, dynamic and chronic phases. But this naming convention has limited significance because each phase may present differently for different people. There is what is described as a nadir, or lowest point, of VA, after which no further deterioration in central vision is expected. This may be different for each person with LHON. The committee noted the frequent rapidity of progression of LHON and the uncertainty around the mechanism of disease activity.

There are no licensed treatments for LHON available in the NHS in England. The clinical and patient experts explained that the rapid vision loss and deteriorating nature of the condition have a considerable effect on people's:

• independence

• education

• ability to work and occupational choices

• social life

• ability to carry out day-to-day tasks.
  
  The patient experts explained that the condition significantly affects the lives of people with LHON because it leads to a sudden and severe loss of central vision in 1 eye then, shortly afterwards, in the other eye. In most people, chronic visual impairment remains. The sudden change in sight can make daily activities such as reading, driving, travelling on public transport and recognising faces very difficult. The patient and clinical experts explained that many people must adapt to their reduced vision. This can include relying on assistive technologies, such as screen readers, magnifiers or speech to text software to access digital information. Also, people with LHON can feel emotional and psychological effects, including grief, frustration, anxiety and depression, and coping with vision loss also affects their families and carers. The effects can lead to an inability to work and socialise, a negative effect on education, missed career opportunities and difficulties in having relationships. The clinical experts explained that carers are often the family members of people with LHON, specifically mothers who may feel guilt for passing on the condition. The committee understood that there are no treatment options and people with LHON often have difficulty doing daily tasks. It also understood that the condition can have an impact on education, independence, travelling on public transport and career opportunities, cause financial burdens for people and their families, and result in difficulties in having relationships. The committee concluded that LHON is a rare, serious and debilitating condition that severely affects the lives of people with it, and their families and carers.

The patient and clinical experts explained that no treatment addresses the underlying cause of LHON. They explained that the current treatment option for people with LHON is limited to best supportive care (standard care from now). This includes neuro-ophthalmology outpatient appointments, referral to low-vision services, lifestyle advice and genetic counselling. They explained that genetic counselling can help people with LHON and their carers understand the condition, risk factors and its inheritance. Supportive therapies such as low-vision aids and infrared light therapy may be used to help people to adjust to changes in vision and maintain independence. Lifestyle modifications are often recommended for people with LHON. These can include avoiding certain things that could potentially worsen their condition, such as tobacco, alcohol and exposure to drugs and toxins with mitochondrial toxicity. Ubiquinone (coenzyme Q10) and other substances can be used to improve mitochondrial function, reduce oxidative stress and provide alternative ATP energy sources. The clinical experts clarified that ubiquinone may be effective in other mitochondrial conditions. But they noted that the evidence suggests it is not very effective in treating LHON because it does not pass the blood-brain barrier to get to the optic nerve. The committee noted that managing LHON is complex and individualised, and that there is no effective treatment for LHON. It concluded that standard care is the appropriate comparator and would be given in addition to idebenone.

For the clinical effectiveness and safety of idebenone in people with LHON, evidence from 5 studies was considered. These were:

• RHODOS: a double-blind randomised placebo-controlled trial assessing the efficacy and safety of idebenone in 85 people over 24 weeks of treatment. It included people aged 14 to 64 years with impaired VA in at least 1 eye because of LHON with an onset of visual loss of 5 years or less, and a confirmed diagnosis (m.11778G>A, m.14484T>C or m.3460G>A LHON mtDNA mutations identified).

• RHODOS observational follow up (OFU): a single-visit observational follow-up study of 58 people with LHON assessing the long-term efficacy of idebenone. It included people in the RHODOS trial in either the idebenone or placebo arms, but who were not expected to have idebenone after RHODOS finished. Median follow up was 30 months (range: 20.9 to 42.5 months).

• LEROS: an open-label intervention study assessing the efficacy and safety of long-term treatment with idebenone in 199 people 12 years and over with LHON. It lasted 24 months, with visits taking place at months 1, 3, 6, 9, 12, 18 and 24.

• Expanded access program (EAP): a real-world-evidence open-label multicentre retrospective analysis of long-term treatment with idebenone in 111 people with LHON with an onset less than 5 years from baseline. It included people 12 years and over with vision loss of less than 12 months before starting idebenone.

• PAROS: a phase 4 study, post-authorisation study with the primary objective to evaluate the long-term safety profile of idebenone in the treatment of people with LHON.
  
  The committee noted that the main evidence came from RHODOS. LEROS and EAP provided data on the long-term effectiveness of idebenone for LHON. The committee noted that RHODOS was a high-quality randomised controlled trial (RCT), but had a small sample size and provided limited evidence on the long-term effects of idebenone. RHODOS‑OFU provided data that was based on a single visit 30 months after RHODOS finished. The people included did not have idebenone between the end of RHODOS and their follow-up visit. The EAP provide long-term data on people with LHON who had idebenone on an individual basis for 36 months. The committee noted that, when the RHODOS study was started, there was a lack of detailed natural history studies on LHON. Also, the optimum length of time needed to detect the impact of treatment was not known. So, further non-randomised studies such as the EAP, LEROS and PAROS were included in the idebenone development program.

For the first committee meeting, the company used data from the case record survey (CaRS) natural history studies to inform the comparative effectiveness of the standard-care comparator. This was because of the lack of long-term RCT data in the EAP, RHODOS‑OFU and LEROS. The CaRS studies were retrospective non-interventional studies of existing medical records of people with a genetically confirmed diagnosis of LHON. The EAG explained that the CaRS studies had a large proportion of missing data, and a high degree of variability in the availability of data from people with LHON at different time points. The committee noted that the lack of long-term comparator data meant that the company used an indirect treatment comparison (ITC) to compare idebenone with standard care. The committee noted that the CaRS studies were international studies with CaRS‑1 reporting natural history data for 383 people with LHON and CaRS‑2 reporting natural history data for 219 people with LHON. The committee noted that the CaRS studies did not provide direct evidence on long-term treatment with idebenone compared with standard care. It also noted that there was a lot of missing data and a high degree of variability in the availability of data from different people at different time points. The committee concluded that using data from the CaRS studies was acceptable to inform comparative effectiveness in the context of this evaluation. But it thought that further characterisation of natural history using the data could be attempted, rather than only using a limited number of data points.

In response to consultation, the company did an integrated analysis to address the committee's concerns about the lack of comparative data between idebenone and standard care. The integrated data was done using, RHODOS‑OFU, EAP, CaRS‑1, CaRS‑2, LEROS and PAROS. It included data from 1,252 people, of whom 847 were included in the intention-to-treat (ITT) analyses. This included 409 people having idebenone and 438 people having standard care. The committee noted that the company did not include RHODOS 6_-month RCT data in the integrated analysis.

RHODOS measured outcomes such as VA, clinically relevant recovery, contrast sensitivity, visual field assessment and adverse effects. It provided analyses for many outcomes at the level of an individual eye (for example, change in the logarithm of the minimal angle of resolution [LogMAR] VA of individual eyes) and at the level of the patient (for example, change in the LogMAR VA of a patient's best eye). It based its economic model on VA based on the LogMAR VA transitions seen in the clinical evidence. The company noted the difficulty in collecting outcome data on VA and relating it to quality of life for LHON. This was because both eyes may be affected at different time points, so each person's VA at baseline was not always clear. The EAG thought that the change in a person's best eye would most closely be linked to quality of life. The patient experts explained that, during their visual field assessment, they noted that their peripheral vision improved over time to compensate for losses in central vision. They thought this might explain minor improvements in VA without idebenone. The committee was aware that common outcomes used to evaluate best VA and colour sensitivity were presented as secondary outcomes. The committee agreed that the best VA was broadly acceptable as an outcome to inform the assessment of efficacy of idebenone. But it noted potential limitations of the sensitivity of the outcome if describing smaller treatment effects.

In RHODOS, the primary outcome was best recovery of VA for people with improving VA in either eye or least worsening of VA for people whose VA was not improving in either eye, between baseline and week 24. This was identified using an Early Treatment Diabetic Retinopathy Study (ETDRS) chart and expressed using LogMAR values. The primary outcome improved both for people having idebenone and people having placebo. With idebenone, there was a mean LogMAR improvement of -0.135 (95% confidence intervals [CI] -0.216 to -0.054). This equated to an improvement of 6 letters on the ETDRS chart. With placebo, there was a LogMAR improvement of -0.071 (95% CI -0.176 to 0.034). This equated to an improvement of 3 letters on the ETDRS chart. The estimated mean difference between groups was not statistically significant (LogMAR -0.064, 95% CI -0.184 to 0.055; p=0.291). This equated to a 3‑letter change. In the RHODOS ITT population, for the change in best VA in the best eye at week 24 compared with the best VA in the best eye at baseline, the difference between idebenone and standard care did not reach statistical significance. In people having idebenone, the LogMAR slightly improved (change -0.035, 95% CI -0.126 to 0.055), which equated to an improvement of 1 letter on the ETDRS chart. For people having placebo there was a worsening of the LogMAR (change +0.085, 95% CI -0.032 to 0.203), which equated to a worsening of 4 letters on the ETDRS chart. The between-group difference was not statistically significant (LogMAR -0.120, 95% CI -0.255 to 0.014; p=0.078) and equated to a 6‑letter change. In the RHODOS trial, a higher proportion of people in the idebenone group (ITT 30.2%; n=16) than in the placebo group (ITT 10.3%, n=3) showed clinically relevant recovery (CRR) from baseline. The difference between the groups was not statistically significant (p=0.056).

The committee noted that the statistically significant changes in the best VA and VA of the best eye from baseline did not reach statistical significance. It acknowledged that even a small improvement in vision would be important, particularly for people with severe sight impairment. The company explained that RHODOS was short and was completed before a wide understanding of the natural history of LHON. So, it may not have shown the true benefit of idebenone. It presented further non-randomised evidence from longer-term trials from EAP and LEROS. In EAP, there was a slight improvement in best VA. The LogMAR decreased from 1.23 (95% CI -0.18 to 1.80) at baseline to 1.19 (95% CI -0.16 to 1.80) at last visit. In LEROS, there was also a slight improvement in best VA from baseline to 24 months (ITT population). There was a mean change in the LogMAR of -0.09 in people with LHON onset in the second eye of 1 year or less and a change in the LogMAR of -0.19 in people with LHON onset in the second eye of more than 1 year. The committee concluded that the evidence suggested that idebenone may have some benefit in terms of improving vision, and preventing vision deterioration and progression of LHON. But it thought that the results were uncertain because of the non-randomised nature of the available long-term evidence.

The committee noted small differences in baseline characteristics such as age and sex between individual studies, and between RHODOS (used in the updated company model from 0 to 6 months) and the integrated analysis (used in the company model from 6 months). The EAG's clinical experts thought that age at baseline and sex are not prognostic factors for LHON, so this was unlikely to have affected the results. The clinical experts explained that RHODOS was a small study that had a different profile of mutations compared to larger studies. They explained that some mutations were underrepresented and some were overrepresented in RHODOS. The committee was aware that the proportion of people with the m.11778G>A mutation (associated with a lower rate of spontaneous improvement) was higher in RHODOS compared with the integrated analysis. It particularly noted that m.14484T>C (associated with a higher rate of spontaneous improvement) was overrepresented in RHODOS compared with the integrated analysis. The clinical experts explained that overrepresentation of this mutation could have led to standard care performing better in RHODOS than would be expected in the NHS. The EAG explained that the results of a subgroup analysis of idebenone compared with standard care, which excluded m.14484T>C, did not reach statistical significance. The clinical experts explained that the proportion of the most common mutation (m.11778G>A) in clinical practice is more similar to that in the RHODOS than in the integrated analysis. The committee also noted that the proportion of people with a LogMAR above 1 was 85% in RHODOS compared with 55% in the integrated analysis. The clinical experts explained that, in their experience, over 90% of people present to their services with a LogMAR above 1 because early diagnosis is very uncommon. The committee noted considerable uncertainty with the generalisability of the evidence. It concluded that RHODOS was potentially more generalisable to NHS clinical practice. This was based on the proportion of people with the m.11778G>A mutation and the proportion of people with a LogMAR above 1 more closely resembling the proportions that would be expected in the NHS.

A direct comparison between idebenone and standard care was only available for 6 months. After this, no direct evidence was available comparing idebenone with standard care. At the first committee meeting the company derived relative treatment effects of idebenone compared with standard care from an ITC using 2 unmatched populations. These were the EAP population for idebenone and the CaRS natural history studies. The EAG noted that the imbalance in prognostic factors between the EAP and the CaRS studies, for example, the study differences in the prevalence of 3 mutations. The EAG explained that this could have biased the result of the ITC. So, it thought that matching the idebenone and standard-care cohorts would be less biased. At the clarification stage, the company provided a propensity score-matching analysis (PSM) of changes in best VA between LEROS and CaRS‑1 and CaRS‑2 at month 24. The EAG thought that, after matching, the baseline characteristics were reasonably balanced between LEROS and CaRS‑1 and CaRS‑2. But it noted that the age of first symptom onset was younger in the standard-care cohort than the idebenone cohort. Also, the prevalence of T14484C genotypes was higher in the idebenone cohort than in the standard-care cohort. The EAG commented that only a limited amount of the CaRS follow-up data was included in the PSM analyses. This was because the company chose to only analyse a single-visit pair (from baseline to 24 months), rather than all available data for standard care. The EAG commented that the median time between visits was 11.7 months in the CaRS studies. So, restricting the analysis to visit pairs 24 months apart likely did not use all the available data. It preferred matching people between LEROS and the CaRS studies at baseline and using all available follow-up data in the analysis. The committee noted that the results of the ITC suggested that idebenone appeared to improve change in best VA at 24 months by -0.02 on the LogMAR scale. This meant it was slightly more effective than standard care in improving VA. But this was not statistically significant, which meant that it was likely that there was no evidence of a difference in treatment effect between idebenone and standard care. The committee understood that PSM did not provide reliable evidence of the long-term treatment benefit of idebenone compared with standard care. But it thought that there were substantial uncertainties in the methods of the ITC because of the limitations of the evidence and limited time in which it had to be completed. The committee thought that there was insufficient sensitivity analysis and exploration of uncertainty of the population adjustment used in the ITC. There were also considerable limitations in the methodology of only using patient-level data for beyond 24 months in the CaRS dataset. So, it requested analyses based on a more comprehensive view of the entirety of the available CaRS evidence.

During consultation, the company did an integrated analysis (see section 3.6) to estimate treatment effect. This was used in the model after 6 months. As part of this, the company presented a propensity score weighted analysis (PSWA) to address the imbalance between idebenone and standard care requested by committee. The company's PSWA was a weighted stabilised inverse probability of treatment weights computed by a propensity score model. Sex, age at onset, time from onset at baseline, baseline best-corrected VA, unilateral or bilateral involvement at baseline and type of mutation (that is, G11778A, G3460A, T14484C) variables were included in the analysis. The committee noted that some potential prognostic factors, such as timing of nadir, were not included in the analysis. The EAG thought that the updated propensity score weighting was potentially less biased than original approach.

The EAG explained that the RHODOS was used to calculate transition probabilities for the first 6 months in the model (see section 3.11) but was not accounted for in the company's integrated analysis. The committee questioned the validity of excluding RHODOS from the integrated analysis (see section 3.6). The company clarified that it had excluded RHODOS because it was an RCT. Also, its integration with observational and retrospective studies would have introduced bias because of differences in study design. The EAG explained that the company's approach may not have been coherent. This was because it did not make the populations in the integrated analysis similar to those in RHODOS, despite using both data sources in the model. The committee noted that the company did not provide the results of 6 months of analysis including and excluding RHODOS, which was requested by the EAG at clarification. The committee concluded that it would have liked to have seen the impact of including the RHODOS data in the integrated analysis or use of PSWA to match the integrated analysis with the RHODOS data.

The company presented an economic model comparing idebenone with standard care. The model was based on a Markov state transition model that included 8 health states and an absorbing death state. Health states were based on VA expressed using the LogMAR (LogMAR less than 0.3, LogMAR 0.3 to 0.6, LogMAR 0.6 to 1.0, LogMAR 1.0 to 1.3, LogMAR 1.3 to 1.7.) counting figures, hand movement and light perception. The company explained that its model structure was in line with model structures used in NICE's highly specialised technologies guidance on voretigene neparvovec for treating inherited retinal dystrophies caused by RPE65 gene mutations (from now, HST11) and NICE's technology appraisals guidance on ranibizumab for treating diabetic macular oedema, ranibizumab for treating visual impairment caused by macular oedema secondary to retinal vein occlusion and ranibizumab for treating choroidal neovascularisation associated with pathological myopia. The patient and clinical experts agreed that the company's 8 health state model accurately captures the natural progression of LHON. At the second meeting, the committee noted that the EAG agreed with the 8 health-state model. The committee concluded that the 8 health-state model was appropriate for decision making.

In response to consultation, the company derived transition probabilities for idebenone and standard care using:

• RHODOS: baseline to month 6

• integrated analysis (month 6 to month 12): with propensity score weights based on stabilised inverse probability of treatment weights

• integrated analysis (month 12 to month 36): weighted and estimated using a logistic regression model with missing at random and missing not at random assumptions

• no change in LogMAR: month 36 onwards.
  
  The company explained that the baseline population distribution used in the model was based on the integrated analysis ITT baseline distribution. This included RHODOS and had a large sample size. The EAG preferred to use RHODOS to inform baseline characteristics and distribution in its model. This was because RHODOS was used to model transitions from baseline to month 6. The committee noted that the source of baseline distribution had a large impact on the cost-effectiveness results. This was mainly because more people started in better health states (lower LogMAR) when the integrated analysis distribution was used (see section 3.9). This led to greater quality-adjusted life year (QALY) gains once the transition probabilities were applied.
  
  The company thought that the change in LogMAR from baseline predicted by its model using the baseline distribution from the integrated analysis aligned well with:

• the results of the RHODOS trial at 6 months

• the outputs of the integrated analysis across all timepoints.
  
  The exact values for change in LogMAR from baseline estimated in the model is considered confidential by the company so cannot be reported here. The EAG thought that it was important to align the model outcomes with the RHODOS outcomes. It thought that the company's approach overestimated the deterioration in LogMAR for standard care compared with RHODOS at 6 months. It explained that using RHODOS baseline distribution led to outputs that more accurately captured the RHODOS trial effect outcomes. The clinical experts noted that the EAG's approach more accurately reflected the RHODOS standard-care outcomes at 6 months. But they thought that, in the longer term, the EAG's approach suggested an improvement in outcomes over time. They explained that this would not be expected in clinical practice. The committee acknowledged the clinical experts' concerns, but recalled that it had heard there can be some improvements in vision over time with standard care (see section 3.7). It also thought that the improvements from baseline were relatively small. The committee thought that the most important thing to consider was the difference in change in LogMAR from baseline between the treatment arms. The company's model predicted a much bigger improvement than the EAG's approach. The committee noted the small non-statistically significant difference in efficacy in the RHODOS trial (see section 3.8). So, it thought that the outputs of the EAG's model had more face validity. The committee also recalled that the RHODOS population may have been more reflective of the NHS than the integrated analysis population (see section 3.9). So, the committee concluded that it was more appropriate to use the baseline distribution from RHODOS. But the committee acknowledged that the EAG's approach was imperfect. This was mainly because there was no alternative to applying the integrated analysis at 6 months. The committee thought that the issues with the EAG's approach could have potentially been avoided if RHODOS had been included in the integrated rather applying treatment effect separately.

At the first meeting, the committee noted that both the company and the EAG assumed that people would stay on idebenone for up to 3 years. The company used pooled time on treatment data seen in RHODOS and the EAP to model time on treatment with idebenone. The committee noted that clinical opinion received by the EAG suggested that people may continue to have idebenone for more than 3 years if LHON responds to idebenone or has only recently stabilised. The committee noted that the company thought extrapolating beyond 3 years was highly uncertain and inappropriate. This was because of a lack of data because of the small number of people who had treatment for more than 3 years. The patient experts mentioned that they expect idebenone to be used until LHON stabilisation. The clinical experts explained that, in clinical practice, they would use idebenone for up to 2 years if LHON is responding or until LHON stabilisation. They explained that, based on the evidence from the natural history studies and RHODOS, cell death does not continue in LHON for the rest of a person's life. People with LHON do not have repeated episodes of cell death, either with or without treatment over time. The committee was aware that treatment duration ranged from 2.4 to 70.4 months in the EAP study. The committee noted that, in clinical practice, people may have idebenone for longer than 3 years and that this would likely be driven by LHON stabilisation. At the first meeting, the committee concluded that the time on treatment for idebenone was uncertain. It added that it would like to have seen further sensitivity analyses using alternative assumptions from expected use in clinical practice (for example, using assumptions about stability from the available clinical data).

In response to consultation, the company noted variability in treatment duration across the LHON studies. It modelled time on treatment for idebenone using data from the integrated analysis set. This was based on the time from the first dose to the 'indication' for when the treatment should be stopped based on clinical expert opinion in its base case. This was defined as:

• the first CRR is seen in 24 months, but no additional CRR is seen in the 6 months after the firstCRR

• the second CRR is seen in the 6 months after the first CRR, but no additional CRR seen in the 6 months after the second CRR.
  
  CRR was defined as an improvement of at least 2 lines in best-corrected VA or a change from off-chart to on-chart results by at least 5 letters. The company proposed that this could be translated into the following stopping rule in clinical practice:

• People will stay on treatment for a minimum of 24 months if there are no issues with tolerability.

• People who have not had a CRR within 24 months will stop treatment.

• People who have a CRR will stay on treatment until the improvement has plateaued for 2 successive periods (that is, there is no further improvement in VA at the following visit) up to a maximum treatment duration of 36 months.
  
  The EAG thought that it was more appropriate to model time on treatment using the actual treatment discontinuation data from the integrated analysis, which was longer than the company's time on treatment. Importantly, a significant proportion of people had treatment beyond 36 months. The exact figures are considered confidential by the company so cannot be reported here. The committee sought advice from the patient and clinical experts on the acceptability of the company's proposed stopping rule and appropriateness for use in clinical practice. The clinical and patient experts broadly agreed that the company's proposal of how idebenone would be stopped reflected clinical practice. The clinical experts clarified that, based on International Consensus Statement on the Clinical and Therapeutic Management of LHON 2017, there was a strong consensus that treatment with idebenone should continue for up to at least 24 months. They explained that, if the disease does not respond by 24 months, treatment should be stopped. But disease response can vary from person to person, and they would want to carry on treatment as long as there is a response. Generally, there is a plateau in improvement at 36 months. But there will be some people who may still see improvement after 36 months. For these people, the clinical experts would want the flexibility to carry on treatment. The patient experts explained that, although they experienced anxiety at the thought of stopping treatment, most benefit is derived from having idebenone at the early onset of the disease. They said that they thought it would be appropriate to stop treatment when disease response reaches the stabilisation phase. But they also expressed their concerns about feeling anxious if their disease worsens. The committee noted that both clinical and patient experts agreed that, in clinical practice, there would be some people who would want to have the treatment beyond 36 months, and for whom this would be appropriate. So, the committee was concerned that idebenone may not be used in line with the company's proposed stopping rule. It also noted that the efficacy in the model was based on treatment duration in the integrated analysis set. It added that the company had not proposed any adjustments to reflect the treatment duration associated with the stopping rule. So, the committee thought that the EAG's approach was more appropriate. The committee was aware that this had a large impact on the cost-effectiveness results. It concluded that it would like to see more evidence from the clinical and patient experts about the feasibility of the company's proposed stopping rule, and efficacy results that took account of it.

In RHODOS, health-related quality-of-life data was collected using the Visual Function Index (VF‑14), Clinicians Global Impression of Change and energy levels. The committee was aware that the NICE reference case recommends using EQ‑5D‑3L directly measured from people with a condition. When EQ‑5D‑3L is not available from clinical trial data, EQ‑5D data can be sourced from published literature or estimated by mapping from other measures of health-related quality of life collected in clinical trials, using published mapping algorithms. The company explained that no published mapping algorithm was available to map from VF‑14, collected in RHODOS, to the EQ‑5D. So, it used health-state utility values from Brown et al. (1999) derived from using time trade-off valuation from 325 people with vision loss caused by a range of vitreoretinal conditions. Most people had age-related macular degeneration (33%) or diabetic retinopathy (33%). This was in line with HST11 and NICE's technology appraisals guidance on ranibizumab for treating diabetic macular oedema, ranibizumab for treating visual impairment caused by macular oedema secondary to retinal vein occlusion and ranibizumab for treating choroidal neovascularisation associated with pathological myopia. The committee noted that the company also provided scenarios using alternative utility values identified by Lawrence et al. (2023), Czoski-Murray et al. (2009) and Rentz et al. (2014). The EAG explained that the utility values from Brown et al. were not based on EQ‑5D‑3L, and had a higher average age than people with LHON, US-based population. So, the EAG preferred to use utility values from Lawrence et al. in its base case. In this study, EQ‑5D data was collected from people in the UK with LHON with an average age of 46.5 years. The committee noted that the source of utility values had a minimal effect on the cost-effectiveness results in the EAG's base case. But it noted that this could have been, in part, because of the alternative model structure used by the EAG. At the first meeting, the committee concluded that Lawrence et al. was a more appropriate source to derive utility values from. It added that it would like to see further scenarios explored using varying utility values, in particular for reflecting a counting-fingers health state.

In response to consultation, the company updated its model using utility values from Lawrence et al. based on Health Utility Index‑3. The company thought that HUI‑3 utility values were more appropriate and in line with previous NICE technology appraisal guidance for measuring health-related quality as compared with EQ‑5D, which does not have a specific domain for visual impairment. It explained that the HUI‑3 used in the Lawerence et al. study measures quality of life specifically related to vision loss in LHON and collected from the UK and Republic of Ireland population with an average age of 46.5 years. The committee noted that the EAG used utility values from Lawrence et al. based on EQ‑5D in its base case. It was aware that it had done this because EQ‑5D is used as a standard for deriving utility for QALYs, ensuring consistency across evaluations. The committee noted that the company preferred to use HUI‑3 values instead of EQ‑5D. This was because HUI‑3 contains a vision component while EQ‑5D is known to have poor convergent validity in visual disorders. Although the EQ‑5D measurement method is preferred to measure health-related quality of life in adults. It recalled that, in some circumstances the EQ‑5D may not be the most appropriate measure. To make a case that the EQ‑5D is inappropriate, qualitative empirical evidence should be provided on the lack of content validity for the EQ‑5D, showing that key dimensions of health are missing as described in NICE's manual on health technology evaluation. No empirical evidence was presented by the company. But the committee recognised the difficulties in the ability to collect or generate clinical evidence in a rare condition and determined that it would consider HUI‑3 utility values in its decision making. The committee concluded that, in this case, it would consider utility values generated using the HUI‑3.

The committee noted that the company applied a utility decrement of 0.04 in line with HST11 for all people with a LogMAR of more than 1.0 to represent the disutility associated with LHON caregivers health-related quality of life. The clinical and patient experts explained that most people with LHON need constant support from family members and carers. A patient organisation submission highlighted that a child's diagnosis can have a significant effect on some parents' mental and physical wellbeing. The committee noted there may be effects on carers' quality of life, particularly for younger people with LHON. But it highlighted that the reference case refers to health-related quality of life because of LHON. So, it may be difficult to interpret this in the context of adults with LHON that live independently without fulltime carers. The committee noted that it had not been presented with evidence for carer quality of life associated with LHON. It also noted that the committee for HST11 thought that it was appropriate to apply carer disutilities for parents of children with a condition that causes blindness. But this was not applied to adults. So, it concluded that the EAG's approach of excluding disutility values for carers of adults in all health states could be appropriate, but it could consider scenarios including a carer disutility for adults with LHON if more quantitative evidence for carers of adults with LHON or other conditions that cause blindness was provided.

In response to consultation, the company maintained its position that although, in HST11, there was a carer disutility applied for parents of children with a condition that causes blindness but not carers of adults with LHON . It thought that it was appropriate to include caregiver disutility in the base case. This was to accurately reflect the impact that LHON has on caregiver quality for adults with the condition having a substantially challenging change in their daily living. The patient experts explained that LHON typically begins in early adulthood and affects people between their late teens and early adulthood. They explained that support for LHON-related vision loss is good in childhood, but tends to decline in adulthood. This can leave people to rely on their carers for many daily activities, including travelling to schools, universities and medical appointments, and using reading or technical equipment that is critical for people to be as independent as possible. The committee noted comments from stakeholders that the effect of LHON on the quality of life of families and carers should be taken into account. The committee acknowledged that adults with LHON may need some assistance in using public services for travelling, and that this could potentially have an effect on the quality of life of families and carers. It noted that it had not seen any evidence to support carer disutility for adults with LHON. It concluded that it would like to see the amount of assistance and impact on carer's quality of life quantified, including the number of carers per patient and the impact of treatment on carer quality of life.

The committee noted that the company included resource costs for each health state, assuming costs associated with blindness using Mead et al. (2003). It included costs of hospitalisations (assumed to be because of injurious falls), outpatient visits (obtaining low-vision aids and rehabilitation), blind registration, supportive living, residential care (aged 65 years and over) and depression. Blind registration and depression were assumed to be one-off costs applied in the first year, whereas all other costs are assumed to occur per cycle. The company explained that Meads et al. was not specifically based on people with LHON. It also had an older population who were classed as blind, so the reported resource use did not apply to the LHON population. So, it used estimates of each resource across the included model health states (on-chart), classified by the LogMAR value, from a survey of 3 international ophthalmologists. These estimates were validated by the 5 UK clinical experts. The EAG noted uncertainty in the estimates provided by the experts. For example, a wide range between the highest and lowest estimates was provided for many resource categories. The EAG noted that 1 expert said that they would not expect young people with vision equal to driving vision to fall regularly, as estimated by the company's resource use. The EAG explained that, in clinical practice, people who would incur health resource costs would be clinically visually impaired with a LogMAR of 1.0 to 1.3. So, the EAG only applied resource costs to people with a LogMAR of more than 1, except depression costs, which were assumed to apply to all health states. The clinical experts explained that they would expect costs for outpatient visits for low-vision aids and rehabilitation for people with a LogMAR of less than 1. The committee noted that although there were significant differences in the approaches used by the company and the EAG about the use of health-state resource utilisation, this had a minor impact on the cost-effectiveness results. It concluded that it was appropriate to apply the resource costs of outpatient visits (including obtaining low-vision aids and rehabilitation) for health stages with a LogMAR of less than 1.

In response to consultation, the company clarified it had removed the blind registration fee based on clinical opinion. It explained that it applied resource use as a mid-point proportion informed by clinical opinion and Mead et al., with outpatient care cost as a one-off cost and with supportive living applied across the life horizon. The committee noted that the EAG's applied resource use was based on Mead et al. It included one-off supportive living costs and standard care needing half the outpatient visits compared with idebenone. The committee concluded that using different approaches to calculate health-state resources used had a minor effect on cost-effectiveness results.

The committee noted that the company's updated deterministic base case gave an incremental cost-effectiveness ratio (ICER) below £30,000 per QALY gained for idebenone compared with standard care. The EAG made several changes to the company's base case. These changes increased the cost-effectiveness estimates to a level that was above what NICE normally considers an acceptable use of NHS resources. The committee noted that the EAG's deterministic base case showed that ICERs for idebenone compared with standard care were over £373,292 per QALY gained. The committee was also presented with a range of scenarios investigating the impact of different assumptions on the company's base case. The committee noted that there was still a high level of uncertainty concerning:

• exclusion of RHODOS from the company's PSWA (see section 3.9)

• using the baseline distribution from the integrated analysis instead of RHODOS (see section 3.11)

• when people would stop idebenone in clinical practice (see section 3.15)

• lack of quantifiable evidence on carer disutility (see section 3.15).

NICE's manual on health technology evaluation notes that above a most plausible ICER of £20,000 per QALY gained, judgements about the acceptability of technology as an effective use of NHS resources will take into account the degree of certainty around the ICER. The committee will be more cautious about recommending a technology if it is less certain about the ICERs presented. The committee was aware that it may accept a higher degree of uncertainty when evidence generation is particularly difficult because the condition is rare. It noted that several of the key uncertainties were affected by the rarity of LHON. These included limited evidence on the long-term effects of idebenone and the assumption used to model the standard-care treatment effect. So, the committee concluded that an acceptable ICER would be towards the upper end of the range NICE considers a cost-effective use of NHS resources (around £30,000 per QALY gained).

The committee agreed that its preferred assumptions to compare idebenone with standard care included:

• baseline characteristics and distribution of patients based on RHODOS and 8 ‑health-state model (see section 3.11)

• time to treatment discontinuation based on time on treatment from the integrated analysis (see section 3.15)

• HUI‑3 utility values from Lawrence et al. (2023; see section 3.16)

• no carer disutility for LHON adults (see section 3.17).
  
  When taking into account all the committee's preferred assumptions, the ICER for idebenone compared with standard care was 280,416 per QALY gained.

Because of the rarity of LHON, the committee recognised difficulties in the ability to collect or generate clinical evidence on idebenone's comparative effectiveness and the natural history of LHON. It agreed that this contributed to significant uncertainty in decision making. The committee also noted that there may be other factors not included in the analysis. These included the potential of idebenone to reduce anxiety and depression, and the effect of LHON on education, travelling and career opportunities from vision loss. The committee thought that, because of these factors, it would apply greater flexibility in accepting a higher degree of uncertainty, as described in section 6.2.34 of NICE's manual on health technology evaluations. The committee also accepted utility values based on HUI‑3 from Lawrence et al. (2023).

The committee was aware that the population for which idebenone is indicated included young people and adults. The committee noted that LHON is a genetic condition. It was aware that LHON is a devastating condition that can begin at a very young age and that people with the condition, and their families and carers, are affected in all aspects of life (see section 3.2). The committee agreed that, if idebenone were recommended, the recommendation would not restrict access for some people over others. No other equality or social value judgement issues were identified.

The clinical experts that that idebenone is innovative for treating LHON in people with a very high unmet need. They also thought that idebenone is a step change for LHON treatment because it has the potential to make a difference in health-related quality of life. They explained that, with idebenone, about 50% of people will have the opportunity of a better visual outcome. The committee acknowledged the benefits offered by idebenone and heard from the clinical and patient experts that idebenone could offer wide-ranging effects including:

• reduced anxiety and depression

• independence with daily activities

• increased socialising

• improved mental health

• rebuilding confidence

• better education and career opportunities.
  
  The committee thought that the uncertainties in the evidence meant that it was unclear whether these had been fully captured in the model. It concluded that it had not been presented with evidence of any additional benefits specific to idebenone that had not been captured in the QALY measurement.

The committee took into account its preferred assumptions, key uncertainties in the model and the other factors in its decision making. Taking these into account, the ICERs based on assumptions were higher than what NICE normally considers a cost-effective use of NHS resources It concluded that the most plausible ICER for idebenone compared with standard care was considerably above its preferred ICER threshold. So, idebenone is not recommended.