AMWG Interim Report to Government - Attachment B

Page last updated: 18 November 2008

PDF printable version of Managing Uncertainty in the Assesment of Medicines for Listing on the Pharmaceutical Pharmaceutical Benefints Scheme - (PDF 70 KB)

Managing Uncertainty in the Assesment of Medicines for Listing on the Pharmaceutical Pharmaceutical Benefints Scheme

A paper prepared for the Access to Medicines Working Group

Executive Summary

The Access to Medicines Working Group (AMWG), under its terms of reference, is charged with considering issues relating to timely and appropriate access to effective new medicines on the Pharmaceutical Benefits Scheme (PBS), including exploring:

“the practical limitations to the evidence available to the Pharmaceutical Benefits Advisory Committee (PBAC) to facilitate decision making around access to new medicines and the development of options to manage uncertainty in such situations.”

This paper is the result of work conducted by the AMWG under this term of reference.

Uncertainty in the context of Pharmaceutical Benefits Advisory Committee (PBAC) submissions and evaluations refers to areas where the evidence, results or conclusions provided are unclear, ambiguous or open to various or different interpretations. Uncertainty can impact on four important aspects influencing decision making – clinical, economic, utilisation and financial. As a general rule, the more complex the submission, the higher the likelihood that there will be some areas where there are shortcomings in the available evidence that will need to be considered by PBAC when deciding whether to recommend that a medicines be listed on the PBS.

The result of uncertainty is an increase in the rate of rejections. This in turn can lead to an increase in the rate of resubmissions, which places a further demand on the resources and analysis required to evaluate the medicine, which further increases resource demand on Industry and the PBAC and can delay the listing of new medicines.

This paper discusses what gaps in the evidence arise in relation to the listing of medicines on the PBS; examines what the PBAC already does to address these gaps and identifies areas where further work may be required. It is accepted that while every effort can be made to minimise uncertainty it cannot be eliminated and therefore the remaining uncertainty needs to be managed.

Background

The Pharmaceutical Benefits Scheme (PBS), along with Medicare, is a key component of Australia's health system.

The PBS aims to provide timely, reliable and affordable access to effective and necessary medicines and vaccines for all Australians. Under the PBS the Australian Government subsidises medicine costs to help people pay for prescription medicines for most medical conditions.

The Government subsidises medicines and vaccines that meet its criteria. In the context of the ongoing development and release of new medicines which are often relatively expensive, it can be difficult to meet the community’s expectations regarding subsidised access to all available medicines. Both the effectiveness and cost effectiveness of the treatments need to be considered in making decisions about subsidisation.

A number of strategies are used to ensure that medicines listed on the PBS and National Immunisation Program (NIP) provide the best ‘value for money’. Broadly the Government only subsidises medicines that cost-effectively maintain the health of the community. This is achieved by carefully assessing the therapeutic benefits and costs of medicines, including comparisons with other treatments, where appropriate. If a medicine is found to be cost-effective, then the Government negotiates the price for it with its sponsor.

A medicine proposed for listing is considered acceptably cost-effective by the Pharmaceutical Benefits Advisory Committee (PBAC) if, for significant medical conditions, the improvement in health outcomes justify the additional costs (and any additional harms) compared with its main alternate therapy.

An acceptably cost-effective new medicine can be recommended for listing if:

It treats or prevents significant medical conditions that are not covered, or only partially covered, by currently listed medicines;
It is more effective and/or less harmful than a currently listed medicine; or
It is as effective and safe as an existing listed medication.

In relation to the third option, such a medicine would be recommended for listing on a cost-minimisation basis, which is a sub-set of a cost-effectiveness analysis. This approach applies where there are insufficient gains in health outcomes to justify a higher price for the proposed drug over currently listed alternatives.

Overview of the Listing Process

The PBS listing process, which is managed by Department of Health and Ageing (DoHA) determines which, and under what circumstances, medicines are eligible for subsidy by the Government. Medicines, prices and other terms and conditions are set out in the Schedule of Pharmaceutical Benefits. The schedule is updated monthly, being primarily used by prescribers and dispensers, but also available to the general public.

A submission for PBS listing can be made for any medicine for any use for which it is registered (or in the process of being registered) by the Therapeutic Goods Administration (TGA). A sponsor is required to agree to list a medicine.

A medicine can only be listed by the Government on the PBS following a positive recommendation to do so by PBAC.

Pharmaceutical Benefits Advisory Committee

PBAC is an independent statutory body which makes recommendations and provides advice to the Minister for Health and Ageing about which drugs and medicinal preparations should be made available as pharmaceutical benefits. It assesses the clinical benefit and cost effectiveness compared with other treatments or products for the same medical condition or use.

PBAC has two sub-committees that provide expert advice and comments on the medicines: the Economics Sub Committee (ESC); and the Drug Utilisation Sub Committee (DUSC). PBAC may refer submissions to one or both of these committees. PBAC is also assisted by the PBAC secretariat and expert medicine evaluators.

If PBAC decides to recommend that a medicines be listed, it usually also makes recommendations on any conditions or restrictions in the listing arrangements.

PBAC also provides advice to the Pharmaceutical Benefits Pricing Authority (PBPA) about comparators, the therapeutic relativity and cost effectiveness of recommended or listed medicines and related pricing matters from a clinical perspective.

Pharmaceutical Benefits Pricing Authority

When a medicine is recommended for listing by PBAC, the sponsor makes a pricing application to the PBPA.

The PBPA then negotiates the initial price with the sponsor of the medicine, taking into account the PBAC’s recommendations about the cost effectiveness of the medicine. The recommendations of PBAC and PBPA are then considered by Government. If the increased net cost to the Government is projected to be greater than $10 million per annum in any of the first four years of listing, the proposed medicine is considered by Cabinet before the Minister may declare a listing on the PBS.

The advice of PBAC, particularly in relation to clinical and cost effectiveness of proposed items, is a significant factor in determining initial prices. However other considerations include the sponsor’s proposed price, prices in comparable countries, the price of alternatives listed on the PBS, manufacturing and supply costs, prescription volumes, cost of the goods and margins and price calculation.

Types of Uncertainty in the PBS listing process

There are several stages and processes involved in listing a new medicine on the PBS. As a general rule, the more complex the submission (and evidence) provided to PBAC the higher the likelihood that there will be some areas where there are shortcomings in the available evidence that will need to be dealt with by PBAC before it makes its recommendation. This need for decision making in the absence of complete information is a characteristic of the environment, and is called uncertainty.

Uncertainty may impact several aspects of the evaluation and decision making process:

Clinical uncertainty – there may be uncertainty about whether a new medicine proposed for listing will effectively meet the clinical need it is proposed to treat. There may be uncertainty about the claims related to the long-term extent and nature of the comparative effectiveness of the medicine, its side effects, interaction with other medicines, which therapies it might replace, or its general place in clinical practice.
Economic uncertainty – while there might be certainty around a medicine’s clinical impact, there may still be uncertainties around its value. For example, the utility or value that patients place on the extra health outcomes to be gained may be uncertain, there may be uncertainty around the costs of administration, the wider impacts of the medicine on the use of other health care resources, or perhaps the assumptions and techniques used to model the economic evaluation of the medicine may be under question. Economic uncertainty inevitably increases with increasing clinical uncertainty.
Utilisation uncertainty – the extent to which the medicine will be used in practice in the Australian community may be unclear. Even in examples where there is a clinical and an economic benefit to eligible patients and the community to subsidise a new medicine, there may be uncertainty around exactly how many eligible patients there are in the community, the extent of substitution for other medicines, the rate of uptake and the doses used in practice.
Financial uncertainty – in some ways linked to utilisation uncertainty, uncertainty may exist around the overall cost of the medicine to the PBS and perhaps even greater uncertainty about the overall impact of the medicine on other areas of the Government’s health budget.

Uncertainty has been demonstrated to have an impact on PBAC’s view of the acceptability of the incremental cost effectiveness ratio (ICER, see Appendix A) associated with specific products. In their presentation Appraising the quality and other dimensions of clinical evidence in health technology assessments, Harris et al have argued that the greater the degree of certainty, the greater the chance of listing (Harris et al. 2006). These authors argued that where certainty was more evident, the probability of receiving a positive recommendation was approximately 60% at a threshold of $55,000/ extra Quality Adjusted Life Year (QALY, see Appendix A) gained. However if the data were uncertain, the probability of receiving a positive recommendation at a threshold of $55,000/ extra QALY gained was only 15%. Similarly, the National Health and Medical Research Council (NHMRC) published guidelines in its handbook series on preparing clinical practice guidelines (How to compare the costs and benefits: evaluation of the economic evidence, July 2001) which recommended that where there was uncertainty, the ICER should only be $30,000/ extra QALY gained. Where there was certainty, the ICER should be elevated to $70,000/ extra QALY gained.

Experiences of PBAC with uncertainty

PBAC is required to determine whether or not a medicine is cost effective, and as part of that process is required to consider the clinical effectiveness of a medicine, taking into account the comparator (the therapy that prescribers would most replace with the medicine in practice if the PBS subsidises it as requested) and the risks of harm associated with the new medicine.

From a PBAC perspective, clinical uncertainties stem from deficiencies in the primary clinical data and the need to translate from such data to address the question of whether the new medicine is cost-effective in the PBS listing as requested. A particular issue is the design of the trial. Ideally PBAC would like to see, wherever possible, direct “head-to-head” randomised trials against the prevailing medicine or treatment. It is acknowledged however that this can be difficult at times because the comparator in Australia may not be the same as it is in other countries, because of the changing nature of listings on the PBS, because of trial design issues or because of ethical considerations. It can be that the comparator changes in the years between the commencement of a trial and the eventual consideration of a submission by PBAC, which also makes direct comparisons more difficult.

In the absence of direct randomised trials, indirect comparisons are often provided as the primary source of clinical evidence. This matter was substantially addressed in the 2006 revision of the PBAC Guidelines, with a separate Section B(i) on presenting such evidence. The Economics Sub-Committee of PBAC has also established an Indirect Comparisons Working Group involving independent experts and representatives of both the committees and the industry, to look at this matter, which first met on 11 April 2008.

Another major source of clinical uncertainty is the reliance on translating surrogate measures reported in trials to final outcomes presented in modelled economic evaluations, and it needs to be noted that this issue has been identified both nationally (including at the 2006 Joint Policy Conference) and internationally. This issue was partially addressed in the 2006 revision of the PBAC Guidelines. The Economics Sub-Committee of PBAC has also similarly established a Surrogate to Final Outcomes Working Group to look at this matter, which also met on 11 April 2008.

PBAC experience indicates that companies could do more to tailor their clinical trials to meet the needs of the health technology assessment step which is necessary for listing a medicine for public subsidy. PBAC encourages companies to take advantage of the pre-submission meetings with the PBAC secretariat to identify issues of predictable uncertainty earlier in the process. Open and early dialogue can provide the company with feedback to take into account when finalising its trial design. Increasing the number of pre-submission meetings may have resource implications for DoHA.

How Government entities deal with uncertainty

PBAC and DoHA therefore and on balance prefer to deal with uncertainty by adopting a flexible approach to each situation within the constraints of the legislative framework within which they operate. Each submission is different and where uncertainties exist, the PBAC response is tailored to address each specific issue. A rigid ‘tick box’ set of criteria that passes or fails an application is not the best way to deal with uncertainty that arises in assessing a complex pharmaceutical listing submission. In fact while a “tick box” approach could enable some companies on some occasions to tailor their submission to meet the criteria, the degree of variation is such that DoHA and PBAC believe that such an approach will likely limit PBAC in finding pathways to list an expensive new medicine associated with elements of uncertainty.

Assessing the clinical benefit and cost effectiveness of a medicine on evidence supplied by an applicant is often not straight forward. The data presented may differ in quality between applications, there may be queries on the accuracy or relevance of data presented or on restrictions requested by the applicant, or further information may be required to support the request.

The existing flexible arrangements to deal with uncertainty permit PBAC to:

recommend (or not) PBS listings on the information provided;
defer a decision subject to further information from the applicant company;
based on input from the Restrictions Working Group, recommend variations to restrictions sought by the applicant company; or
provide additional advice alongside a recommendation to list, for example on pricing.

Examples of ways to address uncertainty, by using the existing flexibility in assessment include:

Restrictions - such as restricting to closely defined populations within the TGA approved indications or, less frequently, continuation rules where there is uncertainty over who will benefit sufficiently from the medicine to justify continuing subsidy on a cost effectiveness basis. DoHA has evolved a restriction taxonomy which allows quite fine adjustments to the level of restriction assurance required. This ranges from unrestricted benefits through to complex requirements requiring prescribers to supply written requests to Medicare Australia for authorisation to prescribe. These restrictions are primarily designed to ensure that only the right patient is treated under the right circumstances, but they can also have the effect of ensuring that very expensive medicines are used wisely and carefully and are only substituted for less expensive treatments where there is a clear clinical need to do so.
Deeds of Agreement – such as recommending a weighted average price across subgroups of a PBS eligible population that account for differing prices ‘justified’ by differing levels of clinical benefit and hence differing cost effectiveness. Deeds are also used to give Government greater certainty around projected expenditure for some listings by establishing usage thresholds over which payment of a rebate may be required. This type of risk-sharing arrangement is aimed at encouraging each sponsor to ensure its medicine is being prescribed appropriately. For example, rebate payments have been required for ‘leakage’ of the medicine beyond the PBS restriction into uses which have not been accepted as cost effective at the listed price, including uses for indications which have not been approved by the TGA. They have also been used to address financial uncertainties about the extent of use consistent with the restriction, for example where the extent of prescribing exceeds predicted behaviour.
Relevant randomised trials which measure more patient-relevant health outcomes – such as on-going trials conducted elsewhere which are reported to PBAC when updated and/or completed. For example, ongoing follow-up from Phase III trials designed for regulatory purposes or new trials, including Phase IV trials from the drug company, designed to address questions relevant to decisions for listing a medicine for public subsidy.
The inclusion of Quality Use of Medicines (QUM) programs and cooperation with other QUM partners with the view that, for some medicines, such programs may reduce uncertainty related to utilisation and clinical outcomes.
Consumer Impact Statements – PBAC is currently trialling Consumer Impact Statements, which allow patients to present in their own words, details about how a condition affects their daily life and the impact on carers. Initial feedback from the PBAC about the usefulness of the information provided has been positive.

It is important when dealing with uncertainty to acknowledge that uncertainty is not simply resolved by agreeing to list an item. Populations, treatment algorithms and the emergence of new medicines and technologies can cause changes over time and create new uncertainties around the clinical effectiveness of a medicine, or exacerbate existing uncertainties. PBAC has dealt with this in part through “predicted versus actual” utilisation reviews conducted by its Drug Utilisation Sub-Committee.

Another post-listing action has been to meet with treating clinicians and affected companies to consider how the treatment algorithm is changing. For example at the present time, PBAC is concerned about a rare condition for which there were no effective medicines 2 or 3 years ago, but for which there are now 5 or 6. PBAC has therefore initiated a meeting with clinicians to look at how each of the medicines is being used in practice, the evidence for that use, and the extent to which current restriction arrangements help or hinder best quality cost effective medication use.

A summary of international examples of dealing with uncertainty is contained at Appendix B.

Principles around managing uncertainty in the PBS listing process

It is acknowledged that uncertainty in the PBS listing process cannot be eliminated, but it can be minimised and/or managed. Guiding principles in managing uncertainty include:

Uncertainty should be accepted and managed, and where possible minimised;
Stakeholders should recognise that a flexible approach acknowledges that there are many different types of uncertainty and its management requires a suite of options to draw from, tailored to the specific issue;
Stakeholders in the Australian system can learn from and contribute to global developments in dealing with uncertainty;
Patient access, benefit and safety and the viability of the PBS should be foremost in efforts to manage uncertainty.

Possible future options for dealing with uncertainty

As noted above, PBAC adopts a flexible approach when dealing with uncertainty to enable it to respond to individual situations in the most constructive manner within the legislative framework governing its operation. The nature of particular uncertainties will vary from submission to submission and will require ongoing flexibility and responsiveness from PBAC.

However, the following activities not already explicitly undertaken by PBAC, might be worthy of further examination and discussion about any possible practical application in the PBS listing process. This list is not arranged in priority order.

The encouragement of more detailed and extensive pre-submission meetings between the sponsor and DoHA. In these meetings key areas of uncertainty would be more clearly identified and discussions held relating to how these may be addressed either prior to or during the assessment period. (NB this project would have significant overlap with elements of the AMWG’s work on streamlining the registration and reimbursement processes).
The examination of a case-manager role who would liaise between the sponsor and the PBAC (its sub-committees) and Departmental officers. The case-manager would assist in finding solutions to problems as they emerge during the assessment process, with a particular focus on addressing areas of uncertainty. It is not expected that every submission would need a case-manager, but only those that have been identified at the pre-submission meeting as being sufficiently complex that it would benefit from such an intervention.
Further exploration of the use of Coverage with Evidence Development (CED, see Appendix B) as a tool for managing uncertainty under pre-defined circumstances. Significant theoretical work and debate has already been undertaken in this area, and it is proposed that the AMWG work towards examining its possible application to the PBS.

Conclusion

This paper has identified three options for managing uncertainty in the PBAC decision making process that merit further consideration. The AMWG will develop a work plan of projects to explore the management of uncertainty in greater detail and will keep the Minister informed in a future report.

Appendix A – Cost effectiveness and the incremental cost effectiveness ratio (ICER)

Cost effectiveness assessment compares the price advantage sought by a sponsor to the net additional health gains conferred by the new drug over its main comparator, together with any cost implications arising from changes in the provision of health care resources. The main comparator is the current therapy that prescribers would most replace with the new drug in practice. The main comparator can be another PBS-listed drug or, where there is no existing effective drug therapy, placebo for standard medical management of the condition.

The preferred metric for describing the overall changes in health outcomes is the number of Quality Adjusted Life Years (QALYs) gained. This summarises the effects of a health intervention on both survival and quality of life.

In calculating the costs associated with listing a drug on a cost effectiveness basis, a wide range of costs (and cost offsets) can be taken into account beyond drug costs. These can include diagnostic, medical, hospital, residential age care and allied health services costs. For the sake of simplicity, the discussion and examples below focus on the drug costs of the new drug and its main comparator only and not any other sources of costs and cost offsets.

The ratio of additional costs to additional health gains is known as the Incremental Cost Effectiveness Ratio (ICER), which is simplified below as:

ICER =

The ICER presents the extra cost for each additional QALY gained offered by the new drug therapy. The larger the ICER, the more “expensive” is the additional health gain and the less favourable the economic credentials are for a PBAC recommendation to list. Increasing the price advantage requested over the comparator increases the value of the ICER. Increasing the additional QALY gained decreases the value of the ICER.

While there is no single maximum ICER value set by PBAC – no “cost effectiveness threshold” – there is evidence that medicines are more likely to be recommended for listing with an ICER around $30,000 than with an ICER above $70,000.

Table 1 provides an example of a new drug with an ICER of $30,000. A $1,500 price advantage is requested for the new drug, and there is an additional health gain of 0.05 QALYs. In this simple example there are no non-drug costs or cost offsets to take into account.

Table 1
Comparator cost/year	$500
New drug cost/year	$2,000
Price advantage requested	$1,500
Additional QALY gain/year	0.05
ICER	$30,000

Appendix B Some International examples for handling uncertainty

An analysis conducted of published international initiatives aimed at handling uncertainty identified a number of solutions which to varying degrees have been applied in Australia. These initiatives from the EU, USA and Canada are categorised below:

Risk sharing linked to clinical outcomes (UK, Canada, EU)

- Reimbursement is agreed on the basis that additional data is forthcoming from studies that will further clarify or strengthen the claim in regard to clinical outcomes. Studies must be of appropriate quality and size to address the area/s of uncertainty.

Coverage with Evidence Development (Mostly USA)

- As for risk sharing linked to clinical outcomes. Coverage with Evidence Development (CED) is a term used mainly in the United States, and implies Medicare coverage (reimbursement) with the condition that additional data (evidence) is being developed. This evidence development may be via various methods, again with consideration of size and quality of the research. The reimbursement during this period of further development of evidence may occur via limited and formal trial participation, or on a wider basis with separate evidence development on an ongoing basis. The former may also be termed Coverage with Study Participation.

Price Volume/dosage Arrangements (USA Managed Care)

- Where the uncertainty is related to the daily cost of the medication and where that may vary significantly, reimbursement decisions may be made on the assumption of a certain daily cost of medication. This may be based on the average daily dose (and cost) and the risk sharing arrangement may require some change to the price or coverage decision if actual daily doses vary from that average.