Health Economics, Evaluation and Equality Publications

Research papers published by our team

2021

Authors - Sebastian Hinde, Laura Bojke, Peter Coventry

https://www.mdpi.com/1660-4601/18/21/11599

Abstract

Internationally, shifts to more urbanised populations, and resultant reductions in engagements with nature, have been a contributing factor to the mental health crisis facing many developed and developing countries. While the COVID-19 pandemic reinforced recent trends in many countries to give access to green spaces more weight in political decision making, nature-based activities as a form of intervention for those with mental health problems constitute a very small part of patient pathways of care. Nature-based interventions, such as ecotherapy, are increasingly used as therapeutic solutions for people with common mental health problems. However, there is little data about the potential costs and benefits of ecotherapy, making it difficult to offer robust assessments of its cost-effectiveness. This paper explores the capacity for ecotherapy to be cost-effective as a healthcare intervention. Using a pragmatic scoping review of the literature to understand where the potential costs and health benefit lie, we applied value of information methodology to identify what research is needed to inform future cost-effectiveness assessments. We show that there is the potential for ecotherapy for people with mild to moderate common mental health problems to be cost-effective but significant further research is required. Furthermore, nature-based interventions such as ecotherapy also confer potential social and wider returns on investment, strengthening the case for further research to better inform robust commissioning.

Authors - Joseph Kwon, Yujin Lee, Tracey Young, Hazel Squires, Janet Harris

https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-021-07056-1

Background

High prevalence of falls among older persons makes falls prevention a public health priority. Yet community-based falls prevention face complexity in implementation and any commissioning strategy should be subject to economic evaluation to ensure cost-effective use of healthcare resources. The study aims to capture the views of older people on implementing the National Institute for Health and Care Excellence (NICE) guideline on community-based falls prevention and explore how the qualitative data can be used to inform commissioning strategies and conceptual modelling of falls prevention economic evaluation in the local area of Sheffield.

Methods

Focus group and interview participants (n = 27) were recruited from Sheffield, England, and comprised falls prevention service users and eligible non-users of varying falls risks. Topics concerned key components of the NICE-recommended falls prevention pathway, including falls risk screening, multifactorial risk assessment and treatment uptake and adherence. Views on other topics concerning falls prevention were also invited. Framework analysis was applied for data analysis, involving data familiarisation, identifying themes, indexing, charting and mapping and interpretation. The qualitative data were mapped to three frameworks: (1) facilitators and barriers to implementing the NICE-recommended pathway and contextual factors; (2) intervention-related causal mechanisms for formulating commissioning strategies spanning context, priority setting, need, supply and demand; and (3) methodological and evaluative challenges for public health economic modelling.

Results

Two cross-component factors were identified: health motives of older persons; and professional competence. Participants highlighted the need for intersectoral approaches and prioritising the vulnerable groups. The local commissioning strategy should consider the socioeconomic, linguistic, geographical, legal and cultural contexts, priority setting challenges, supply-side mechanisms spanning provider, organisation, funding and policy (including intersectoral) and health and non-health demand motives. Methodological and evaluative challenges identified included: incorporating non-health outcomes and societal intervention costs; considering dynamic complexity; considering social determinants of health; and conducting equity analyses.

Conclusions

Holistic qualitative research can inform how commissioned falls prevention pathways can be feasible and effective. Qualitative data can inform commissioning strategies and conceptual modelling for economic evaluations of falls prevention and other geriatric interventions. This would improve the structural validity of quantitative models used to inform geriatric public health policies.

Authors - Matthew Franklin, Tracey Young

https://ebmh.bmj.com/content/24/4/e5

Furukawa et al 1 posed the question: how can we estimate quality-adjusted life years (QALYs) based on Patient Health Questionnaire-9 (PHQ-9) scores? They recommend equipercentile linking analysis between the depression severity PHQ-9 and preference-based EQ-5D three-level version (EQ-5D-3L; UK value set), the latter used to estimate utility data for QALYs.

Furukawa et al 1 refer to the process of ‘cross-walking’, whereby the practice of fitting a statistical model to health utility data has been referred to as ‘mapping’ and 'cross-walking’.2 Furukawa et al 1 reference two mapping-related papers (their references 7 and 9); however, their analysis seems to have missed rigorous mapping methodology and previous studies which have used these mapping processes, alongside other conceptual considerations when wanting to ‘cross-walk’/‘map’ from a non-preference-based (often condition-specific) measure such as the PHQ-9 to the preference-based EQ-5D-3L.

Authors - Richard Pilbery, Tracey Young, Andrew Hodge

https://www.medrxiv.org/content/10.1101/2020.08.06.20169334v3

Abstract

Introduction NHS ambulance service conveyance rates in the UK are almost 70%, despite an increase in non-emergency cases. This is increasing the demands on crowded emergency departments (ED) and contributes to increased ambulance turnaround times. Yorkshire Ambulance Service introduced a specialist paramedic (SP) role to try and address this, but non-conveyance rates in this group have not been as high as expected.

Methods We conducted a controlled interrupted time series analysis using data from incidents between June 2017 and December 2019, to study appropriate non-conveyance rates before and after a GP placement. A costing analysis examined the average cost per appropriate non-conveyance achieved for patients receiving care from intervention group SPs pre- and post-placement was also conducted.

Results 7349 incidents attended by intervention group SPs were eligible for inclusion. Following removal of cases with missing data, 5537/7349 (75.3%) cases remained. Post-placement, the intervention group demonstrated an increase in appropriate non-conveyance rate of 35.0% (95%CI 23.8–46.2%, p<0.001), and a reduction in the trend of appropriate non-conveyance of −1.2% (95%CI −2.8–0.5%, p=0.156), relative to the control group.

Post-placement, the cost per appropriate non-conveyance for intervention group SPs was a mean of £509.38 (95% bootstrapped CI £455.32–£564.59) versus £1258.04 (95% bootstrapped CI £1232.64–£1284.04) for the same group in the pre-placement phase. This represents a mean saving of £748.66 per appropriate non-conveyance (95% bootstrapped CI £719.45–£777.32) and a cost-effectiveness ratio of £2141.15 per percentage increase in appropriate non-conveyance (95% bootstrapped CI £2057.62–£2223.12).

Conclusion In this single UK NHS ambulance service study, we found a clinically important and statistically significant increase in appropriate non-conveyance rates by specialist paramedics who had completed a 10-week GP placement. This improvement persisted for the 12-month period following the placement and demonstrated cost savings compared to usual care.

Authors - Sebastian Hinde, Laura Bojke, Gerry Richardson, Yvonne Birks, William Whittaker, Mark Wilberforce, Andrew Clegg

https://academic.oup.com/ageing/article-abstract/50/4/1073/6150370?redirectedFrom=fulltext

Abstract

Delayed transfers of care (DTOC), often unhelpfully referred to as ‘bed blocking’, has become a byword for waste and inefficiency in healthcare systems throughout the world. An estimated 2.7 million bed days are occupied each year in England by older people no longer in need of acute treatment, estimated to cost £820 million (2014/15) in inpatient care. Policy and media attention have often been drawn to this narrative of financial waste, resulting in policy setting that directly targets the level of DTOC, but has done little to put patient health first.


These figures and policies portray a misleading image of the delays as primarily of concern in terms of their financial burden on acute hospital care, with little consideration given to the quantification on patient health or wider societal impacts. In spite of the multi-factorial decision-making process that occurs for each patient discharge, current evaluation frameworks and national policy setting fail to reflect the complexity of the process.


In this commentary, we interrogate the current approach to the quantification of the DTOC impact and explore how policies and evaluation methods can do more to reflect the true impact of the delays.

Authors - Tessa Peasgood, Jen-Yu Chang, Robina Mir, Clara Mukuria & Philip A. Powell

https://link.springer.com/article/10.1007/s11136-021-02801-9

Purpose

Uncertainties exist in how respondents interpret response options in patient-reported outcome measures (PROMs), particularly across different domains and for different scale labels. The current study assessed how respondents quantitatively interpret common response options.

Methods

Members of the general public were recruited to this study via an online panel, stratified by age, gender, and having English as a first language. Participants completed background questions and were randomised to answer questions on one of three domains (i.e. loneliness (negatively phrased), happiness or activities (positively phrased)). Participants were asked to provide quantitative interpretations of response options (e.g. how many times per week is equal to “often”) and to order several common response options (e.g. occasionally, sometimes) on a 0–100 slider scale. Chi-squared tests and regression analyses were used to assess whether response options were interpreted consistently across domains and respondent characteristics.

Results

Data from 1377 participants were analysed. There was general consistency in quantifying the number of times over the last 7 days to which each response option referred. Response options were consistently assigned a lower value in the loneliness than happiness and activities domains. Individual differences, such as age and English as a second language, explained some significant variation in responses, but less than domain.

Conclusion

Members of the public quantify common response options in a similar way, but their quantification is not equivalent across domains or every type of respondent. Recommendations for the use of certain scale labels over others in PROM development are provided.

Authors - Paolo Candio, Andrew J. Hill, Stavros Poupakis, Anni-Maria Pulkki-Brännström, Chris Bojke & Manuel Gomes

https://link.springer.com/article/10.1007%2Fs40258-020-00629-x

Abstract

Sample selectivity is a recurrent problem in public health programmes and poses serious challenges to their evaluation. Traditional approaches to handle sample selection tend to rely on restrictive assumptions. The aim of this paper is to illustrate a copula-based selection model to handle sample selection in the evaluation of public health programmes. Motivated by a public health programme to promote physical activity in Leeds (England), we describe the assumptions underlying the copula selection, and its relative advantages compared with commonly used approaches to handle sample selection, such as inverse probability weighting and Heckman’s selection model. We illustrate the methods in the Leeds Let’s Get Active programme and show the implications of method choice for estimating the effect on individual’s physical activity. The programme was associated with increased physical activity overall, but the magnitude of its effect differed according to adjustment method. The copula selection model led to a similar effect to the Heckman’s approach but with relatively narrower 95% confidence intervals. These results remained relatively similar when different model specifications and alternative distributional assumptions were considered. The copula selection model can address important limitations of traditional approaches to address sample selection, such as the Heckman model, and should be considered in the evaluation of public health programmes, where sample selection is likely to be present.

2020

Authors - Simon Moore, Tracey Young, Andy Irving, Steve Goodacre, Alan Brennan, Yvette Amos

https://emj.bmj.com/content/38/7/504

Background

Alcohol intoxication management services (AIMS) provide an alternative care pathway for alcohol-intoxicated adults otherwise requiring emergency department (ED) services and at times of high incidence. We estimate the effectiveness and cost-effectiveness of AIMS on ED attendance rates with ED and ambulance service performance indicators as secondary outcomes.

Methods

A controlled longitudinal retrospective observational study in English and Welsh towns, six with AIMS and six without. Control and intervention cities were matched by sociodemographic characteristics. The primary outcome was ED attendance rate per night, secondary analyses explored hospital admission rates and ambulance response times. Interrupted time series analyses compared control and matched intervention sites pre-AIMS and post-AIMS. Cost-effectiveness analyses compared the component costs of AIMS to usual care before with results presented from the National Health Service and social care prospective. The number of diversions away from ED required for a service to be cost neutral was determined.

Results

Analyses found considerable variation across sites, only one service was associated with a significant reduction in ED attendances (−4.89, p<0.01). The services offered by AIMS varied. On average AIMS had 7.57 (mean minimum=1.33, SD=1.37 to mean maximum=24.66, SD=12.58) in attendance per session, below the 11.02 diversions away from ED at which services would be expected to be cost neutral.

Conclusions

AIMSs have variable effects on the emergency care system, reflecting variable structures and processes, but may be associated with modest reductions in the burden on ED and ambulance services. The more expensive model, supported by the ED, was the only configuration likely to divert patients away from ED. AIMS should be regarded as fledgling services that require further work to realise benefit.

Authors - Alexis Foster, Alicia O'Cathain, Janet Harris

https://bmjopen.bmj.com/content/10/10/e039116

Objectives

To identify the facilitators and barriers to implementing patient-reported outcome measures (PROMs) in third sector organisations (TSOs) delivering health and well-being services.

Design

A qualitative interview study. Participants were recruited using purposive, opportunistic and snowballing methods. Framework analysis was used.

Setting

TSOs including charities, community groups and not-for-profit organisations in England, UK.

Participants

Thirty interviewees including service users, TSO front-line workers and managers, commissioners of TSOs and other stakeholders such as academic researchers.

Results

TSOs primarily used PROMs because of pressures arising from the external funding context. However, organisations often struggled to implement PROMs, rarely getting the process right first time. Facilitators for implementation included having an implementation lead committed to making it work, investing resources in data management systems and support staff and taking a collaborative approach to designing the PROMs process. The latter helped to ensure an appropriate PROMs process for the specific TSO including choosing a suitable measure and planning how data would be collected, processed and used. There was a dilemma about whether TSOs should use standardised well-being measures (eg, the Warwick-Edinburgh Mental Well-being Scale) or design their own PROM. Not all TSOs sustained the collection and reporting of PROMs over time because this required a change in organisational culture to view PROMs as beneficial for the TSO and PROMs becoming part of front-line workers’ job specifications.

Conclusions

TSOs are trying to use PROMs because they feel they have no choice but often struggle with implementation. Having an implementation lead, designing an appropriate process, investing resources, training staff and taking mitigating action to address potential barriers can facilitate implementation. Some of the findings are consistent with the experiences of more clinical services so appear relevant to the implementation of PROMs irrespective of the specific context.

Authors - Paolo Candioa, David Meads, Andrew J. Hill, Laura Bojke

https://www.sciencedirect.com/science/article/abs/pii/S0168851020301974?via%3Dihub

Background

While several reviews have assessed economic evaluations of physical activity in public health and, in most cases, found the interventions to be cost-effective, the validity of the conclusions reached depends on the appropriateness of the modelling methods used in the individual studies.

Objective

To provide an overview and critique of modelling approaches and key structural assumptions used in applied studies to estimate the impact of physical activity on population health.

Methods

Electronic databases were systematically searched for relevant model-based economic evaluations. A thematic approach was used to assess the modelling studies. The critique determined the appropriateness of the modelling frameworks and plausibility of key structural assumptions.

Results

Twenty-five models were identified. Cohort models were most frequently used. High variability in the modelling of downstream diseases was found across studies analysing similar populations. Structural assumptions regarding the dynamics of change of physical activity were unrealistic in most cases. Heterogeneity was addressed in only a few studies, while health equity concerns were, at best, acknowledged by authors.

Conclusions

This literature is predominantly characterised by modelling approaches that may not adequately address the complexities associated with representing the physical activity behaviour- population health process. A consensus on how to model the impact of physical activity on public health and development of a reference model could help reduce these sources of uncertainty.

Authors - Paolo Candioa, David Meads, Andrew J. Hill, Laura Bojke

https://academic.oup.com/jpubhealth/article-abstract/43/4/876/5880643?redirectedFrom=fulltext

Background

The purpose of this paper is to assess the cost-effectiveness of a proportionate universal programme to reduce physical inactivity (Leeds Let us Get Active (LLGA)) in adults.


Methods

A continuous-time Markov chain model was developed to assess the cost implications and QALY gains associated with increases in physical activity levels across the adult population. A parametric survival analysis approach was applied to estimate the decay of intervention effect over time. Baseline model data were obtained from previous economic models, population-based surveys and other published literature. A cost-utility analysis was conducted from a health care sector perspective over the programme duration (39 months). Scenario and probabilistic sensitivity analyses were performed to test the robustness of cost-effectiveness results.


Results

In total, 51 874 adult residents registered to the programme and provided baseline data,19.5% of which were living in deprived areas. Under base case assumptions, LLGA was found to be likely to be cost-effective. However, variations in key structural assumptions showed sensitivity of the results.


Conclusions

Results from this study suggest a non-negligible level of uncertainty regarding the effectiveness, and therefore, cost-effectiveness of a universal offer of free leisure centre-based exercise that targets hard to reach groups. Further data collection and a shift towards prospective evaluations are needed.

Authors - Elnaz Aliakbari, Kara A. Gray-Burrows, Karen A. Vinall-Collier, Sakina Edwebi, Zoe Marshman, Rosemary R. C. McEachan, Peter F. Day

https://onlinelibrary.wiley.com/doi/10.1111/ipd.12658

Background

Dental caries is the most prevalent preventable condition in children. A key preventive home-based oral health behaviour is the adoption and maintenance of parental supervised toothbrushing until 8 years of age.

Aim

To examine interventions promoting parental supervised toothbrushing practices to reduce dental caries in young children (<8 years old).

Design

Interventions promoting parental involvement in home-based toothbrushing in children under 8 years old and their impact on caries were subjected to review. Electronic databases (MEDLINE, EMBASE, PubMed, Web of Science, PsycINFO, Scopus, and the Cochrane Library), references, and unpublished literature databases were searched for relevant literature.

Results

Of the 10 176 articles retrieved, forty-two articles were included. The Theoretical Domains Framework was used to code intervention content, with the main domains addressed being knowledge (41/42), skills (35/42), and environmental context and resources (22/42). Sufficient descriptions of the intervention development, delivery, and evaluation were lacking, with only 18 studies being underpinned by theory. Twenty-nine studies explored the impact on caries yielding mixed results.

Conclusions

There are few interventions targeting home-based oral health behaviours underpinned by theory and methodological rigour in their development and evaluation. This demonstrates a clear need for future interventions to be guided by complex intervention methodology.

Authors - A.Driscoll, S.Hinde, A.Harrison, L.Bojke, P.Doherty

https://www.sciencedirect.com/science/article/abs/pii/S0167527320302047?via%3Dihub

Background

Cardiac rehabilitation (CR) programs are effective in reducing cardiovascular mortality and readmissions. However, most patients are denied the benefits of CR due to low referral rates. Of those patients referred, commencement rates vary from 28.4% to 60%. This paper quantifies the scale of health loss in Australia due to poor engagement with the program, and estimates how much public funding can be justifiably reallocated to address the problem.

Methods

Economic decision modelling was undertaken to estimate the expected lifetime health loss and costs to Medicare. Key parameters were derived from Australian databases, CR registries and meta-analyses. Population health gains associated with uptake rates of 60%, and 85% were calculated.

Results

CR was associated with a 99.9% probability of being cost-effective, even at a cost-effectiveness threshold lower than conventionally applied. Importantly, an average of 0.52 years of life expectancy are lost due to national uptake being below 60% achieved in some best performing programs in Australia, equivalent to 0.28 quality adjusted life years.

The analysis indicates that $12.9 million/year could be justifiably reallocated from public funds to achieve a national uptake rate of 60%, while maintaining cost-effectiveness of CR due to the large health gains that would be expected.

Conclusion

CR is a cost-effective service for patients with coronary heart disease. In Australia, less than a third of patients commence CR, potentially resulting in avoidable patient harm. Additional investment in CR is vital and should be a national priority as the health gains for patients far outweigh the costs.

Authors - Elizabeth Taylor Buck, Christine M. Smith, Amanda Lane, Anju Devianee Keetharuth, Tracey Young and Jo Cooke

https://link.springer.com/article/10.1186/s41687-020-00202-z?wt_mc=Internal.Event.1.SEM.ArticleAuthorIncrementalIssue

Background

Patient reported outcome measures (PROMs) are a means of assessing the quality and effectiveness of care from the patient’s perspective. However, the routine use of PROMs in clinical practice can be difficult to implement. New challenges arise at different stages of the implementation process and organisations need to invest time and financial resources into designing an appropriate strategy, information systems, providing technical support and preparing staff.

Recovering Quality of Life (ReQoL) is a PROM that was specifically designed to measure mental health service users’ perspectives of recovery and quality of life. It is a co-produced, service user-centred outcome measure tested by over 6000 mental health service users. It is able to detect change across a broad spectrum of mild to severe mental health conditions.

In May 2016 a licence to use the ReQoL measures became freely available to the NHS and publicly funded research. At the time of publication, 149 licences had been issued worldwide and eleven official translations had been made available. The team that developed ReQoL has continued to collaborate with mental health trusts across the country to support the implementation of ReQoL. The National Institute for Health Research and Applied Research Collaboration (NIHR ARC YHFootnote1), and the preceding NIHR CLAHRC-YH, along with the ReQoL development team have helped to guide and fund this work. The NIHR CLAHRC-YH supported two national events, the first of which was the launch of ReQoL at the Houses of Parliament in October 2016. More recently in November 2018, over 70 people, from 23 organisations, attended a second event focused on the development of a ReQoL Community of Practice.

Community of Practices (CoPs) have existed in sectors such as education and business for over 30 years [9], operating as networks and support groups for people who share a common set of problems or interests. Members of CoPs maintain and build links with each other for the purposes of social interaction, knowledge sharing, knowledge creation and identity building.

Franklin, M et al (2020) Conducting Value for Money Analyses for Non-randomised Interventional Studies Including Service Evaluations: An Educational Review with Recommendations

Authors - Matthew Franklin, James Lomas, Gerry Richardson

https://link.springer.com/article/10.1007%2Fs40273-020-00907-5

Abstract

This article provides an educational review covering the consideration of conducting ‘value for money’ analyses as part of non-randomised study designs including service evaluations. These evaluations represent a vehicle for producing evidence such as value for money of a care intervention or service delivery model. Decision makers including charities and local and national governing bodies often rely on evidence from non-randomised data and service evaluations to inform their resource allocation decision-making. However, as randomised data obtained from randomised controlled trials are considered the ‘gold standard’ for assessing causation, the use of this alternative vehicle for producing an evidence base requires careful consideration. We refer to value for money analyses, but reflect on methods associated with economic evaluations as a form of analysis used to inform resource allocation decision-making alongside a finite budget. Not all forms of value for money analysis are considered a full economic evaluation with implications for the information provided to decision makers. The type of value for money analysis to be conducted requires considerations such as the outcome(s) of interest, study design, statistical methods to control for confounding and bias, and how to quantify and describe uncertainty and opportunity costs to decision makers in any resulting value for money estimates. Service evaluations as vehicles for producing evidence present different challenges to analysts than what is commonly associated with research, randomised controlled trials and health technology appraisals, requiring specific study design and analytic considerations. This educational review describes and discusses these considerations, as overlooking them could affect the information provided to decision makers who may make an ‘ill-informed’ decision based on ‘poor’ or ‘inaccurate’ information with long-term implications. We make direct comparisons between randomised controlled trials relative to non-randomised data as vehicles for assessing causation; given ‘gold standard’ randomised controlled trials have limitations. Although we use UK-based decision makers as examples, we reflect on the needs of decision makers internationally for evidence-based decision-making specific to resource allocation. We make recommendations based on the experiences of the authors in the UK, reflecting on the wide variety of methods available, used as documented in the empirical literature. These methods may not have been fully considered relevant to non-randomised study designs and/or service evaluations, but could improve and aid the analysis conducted to inform the relevant value for money decision problem.

Authors -Sebastian Hinde, Alexander Harrison, Laura Bojke, Patrick Doherty

https://journals.sagepub.com/doi/abs/10.1177/2047487320912625

Background

Despite its role as an effective intervention to improve the long-term health of patients with cardiovascular disease and existence of national guidelines on timeliness, many health services still fail to offer cardiac rehabilitation in a timely manner after referral. The impact of this failure on patient health and the additional burden on healthcare providers in an English setting is quantified in this article.

Methods

Two logistic regressions are conducted, using the British Heart Foundation National Audit of Cardiac Rehabilitation dataset, to estimate the impact of delayed cardiac rehabilitation initiation on the level of uptake and completion. The results of these regressions are applied to a decision model to estimate the long-term implications of these factors on patient health and National Health Service expenditure.

Results

We demonstrate that the failure of 43.6% of patients in England to start cardiac rehabilitation within the recommended timeframe results in a 15.3% reduction in uptake, and 7.4% in completion. These combine to cause an average lifetime loss of 0.08 years of life expectancy per person. Scaled up to an annual cohort this implies 10,753 patients not taking up cardiac rehabilitation due to the delay, equating to a loss of 3936 years of life expectancy. We estimate that an additional £12.3 million of National Health Service funding could be invested to alleviate the current delay.

Conclusions

The current delay in many patients starting cardiac rehabilitation is causing quantifiable and avoidable harm to their long-term health; policy and research must now look at both supply and demand solutions in tackling this issue.

Authors -Sebastian Hinde, Sally Fowler Davis, Steven Ariss

https://bmjopen.bmj.com/content/10/3/e029174

Abstract

NHS vanguards, under-pressure to perform, required better contracting and data management arrangements with evaluation teams, to ensure that integrated service outcomes could be reported effectively. This communication reflects the experience of evaluating an NHS vanguard and suggests how academic teams can improve capacity for complex programme evaluation of rapid improvements in integrated services. This should be based on a shared commitment to data collection and management. Also, robust knowledge exchange processes can enable systems change and sustainability. The identifying features of the particular site have been withheld.

2019

Authors - Sebastian Hinde, Louise Horsfield, Laura Bojke, Gerry Richardson

https://doi.org/10.1007/s40258-019-00538-8

Abstract

Since 2013, obesity services in the UK National Health Service (NHS) have focused on a tiered structure, with tiers 3 (specialist weight management services) and 4 (primarily bariatric surgery) commissioned by Clinical Commissioning Groups (CCGs) and widely reported as cost effective and recommended by national guidelines. However, CCGs have been reluctant to fully conform to the guidance. We explore how the different evaluative perspective of those generating evidence from local decision makers has contributed to this failure of the CCGs to provide services considered cost effective. We explore four elements where the conventional economic evaluation framework, as applied by the National Institute for Health and Care Excellence (NICE), differ from the reality faced by local decision makers: the cost-effectiveness threshold, the implications of decision uncertainty and budgetary excess, the valuation of future costs and outcomes, and the scope of included costs. We argue that the failure of the conventional framework to reflect the reality faced by local decision makers is rendering much of the existing literature and guidance inappropriate to the key commissioners. Our analysis demonstrates that it is not reasonable to assume that the framework of economic evaluation used to inform national guidance applies to local decision makers, such as in the commissioning of weight loss services. This failure is likely to apply to the majority of cases where evidence is generated to inform national decision makers but commissioning is at a local level.

Authors - Sebastian Hinde, Laura Bojke, Gerry Richardson

https://bmjopen.bmj.com/content/9/11/e029830

Objectives

In the context of tightening fiscal budgets and increased commissioning responsibility, local decision-makers across the UK healthcare sector have found themselves in charge of the implementation and evaluation of a greater range of healthcare interventions and services. However, there is often little experience, guidance or funding available at a local level to ensure robust evaluations are conducted. In this paper, we evaluate the possible scenarios that could occur when seeking to conduct a quantitative evaluation of a new intervention, specifically with regards to the availability of evidence.

Design

We outline the full set of possible data scenarios that could occur if the decision-maker seeks to explore the impact of the launch of a new intervention on some relevant quantifiable outcomes. In each case we consider the implicit assumptions associated with conducting an evaluation, exploring possible situations where such scenarios may occur. We go on to apply the scenarios to a simulated dataset to explore how each scenario can result in different conclusions as to the effectiveness of the new intervention.

Results

We demonstrate that, across the full set of scenarios, differences in the scale of the estimated effectiveness of a new intervention and even the direction of effect are possible given different data availability and analytical approaches.

Conclusions

When conducting quantitative evaluations of new interventions, the availability of data on the outcome of interest and the analytical approach can have profound effects on the conclusions of the evaluation. Although it will not always be possible to obtain a complete set of data and conduct extensive analysis, it is vital to understand the implications of the data used and consider the implicit assumptions made through its use.