Evidence guidelines

Session One

Diane Wilkinson

Hello and welcome to ‘Evidence Guidelines for Listed Medicines Information Session One: Overview’. I'd like to start by acknowledging the traditional owners and custodians of the lands on which we meet today and pay my respects to elders, past, present, and emerging. I would also like to extend that acknowledgment and my respect to any Aboriginal and Torres Strait Islander peoples here today.

The purpose of this session is to help sponsors understand how to use the Listed Medicines Evidence Guidelines so that they can compile a robust evidence package that supports the efficacy of their listed medicine. Having a robust evidence package will help achieve a favourable outcome for a listed medicine if it is selected for a TGA efficacy compliance review.

I'd like to point out that although the Evidence Guidelines are not mandatory requirements, they do show sponsors the things that the TGA delegate considers in conducting a compliance review.

Sponsors may choose to generate their own scientific evidence for their medicine, that is, conduct a clinical trial on their unique formulation. Or they may choose to rely on existing published literature. Most commonly, evidence packages for listed medicines are based on published literature, therefore, the Evidence Guidelines are primarily focused on evidence packages that are based on published literature.

The Evidence Guidelines are structured to provide a step-by-step process to assist sponsors to: find evidence; critically assess the evidence; select indications based on evidence; and present that evidence. This structure is based on the scenario where a sponsor first selects a research question, then researches the evidence landscape and then, selects therapeutic indications based on the evidence they find. However, it is possible to follow the guidelines in a different order. For example, a sponsor may choose to first select the indications for their medicine and then survey the evidence landscape.

I’ll now go through each of the sections of the Guidelines in more detail.

Section one of the Guidelines is a general introduction. It outlines the purpose of the Guidelines and provides the legal responsibilities of a sponsor in relation to holding evidence that demonstrates the efficacy of their listed medicine. It also explains when and how the TGA reviews the efficacy of listed medicines.

Under the regulatory framework listed medicines are not pre-market evaluated by the TGA for quality, safety, and efficacy. Instead, they are automatically included in the Australian Register of Therapeutic Goods based on a certification by the sponsor, its product owner, that the medicine meets all the applicable legislative requirements in relation to safety, quality, and efficacy. Regulatory requirements for listing include that the sponsor holds appropriate evidence to substantiate the indications and can demonstrate that the efficacy of the medicine is acceptable.

The overall purpose of an evidence package is for the sponsor to demonstrate to the TGA that their medicine will do what the medicine’s indications say it will do, that is, that the medicine is efficacious. The Evidence Guidelines set out the factors that the TGA considers when assessing the efficacy of a listed medicine is acceptable. They provide a roadmap to help sponsors put together a robust evidence package.

Under the Therapeutic Goods Act, sponsors make a number of certification when they list their medicine. This includes that:

• they hold evidence to support all indications and comply with all requirements for those indications
• that they hold evidence to support any claims made for the medicine; and
• that they must, at all times while the medicine remains listed, hold information or evidence that supports indications.

The TGA can cancel a medicine from the ARTG if the sponsor certifications are found to be incorrect or if the efficacy of the medicine appears to be unacceptable. If a medicine is cancelled from the ARTG, in general, it can no longer be imported, manufactured, or exported from Australia by the sponsor.

For the purpose of the Evidence Guidelines, we have defined efficacy as: ‘the capacity of a medicine to produce a therapeutic effect’. That is, that the medicine will do what its indications say it will do.

We expect medicines to work as described by its indications, whether medicine relies on scientific evidence sources or evidence of traditional use, is important in this context. The emphasis is, that, the medicine will do what the indications say it will do, whether the evidence base is traditional or scientific. So, a medicine using a permitted traditional indication must have evidence of traditional use in the specified paradigm. For scientific indications, efficacy is supported by scientific evidence, such as clinical studies or systematic reviews.

The TGA can review a medicine at any time it is listed in the ARTG. We conduct random and targeted post-market compliance reviews of listed medicines on an ongoing basis. During an efficacy review the key questions we consider include, but are not limited to:

• What therapeutic effect is described by the indication?
• On what basis can it be concluded that the medicine will result in a therapeutic effect?
• What is the reason for this conclusion?
• And, is the above conclusion based on data or information that we have confidence in to be true and accurate?

Since these are important questions being considered by the TGA when reviewing efficacy, it's a good idea of sponsors to consider these questions when critically analysing your evidence.

Section two of the Evidence guidelines provides guidance on how to conduct and document a literature search. It outlines the different types of evidence, such as evidence of traditional use and scientific evidence.

Evidence of traditional use is based on an extensive history of use in a specific paradigm. This history underpins the safe use of these medicines in a traditional setting. The use of many traditional medicines is well documented in pharmacopeia, monographs, Materia Medica etc.

Scientific evidence refers to quantifiable data and, usually, includes reports of clinical trials in humans, animal studies or pharmacological studies.

The Guidance also provides links to NHMRC guidelines that can be an additional information source.

Section two provides guidance on how to conduct a literature search using search methodology and when to conduct a non-systematic literature search and what to document. This is to ensure that a balanced view of the evidence landscape is obtained and considered by sponsors.

It's important to note that the TGA does not require sponsors to undertake a particular search strategy for listed medicines - it is up to the sponsor to consider what type of literature search is most appropriate in their circumstance.

Evidence Guideline Section three is on how to assess evidence. It provides guidance on how to determine if the evidence you have found is relevant to your medicine and, of good quality. Once you weigh up the relevance and quality of an evidence source, you can consider it in the context of the body of evidence holistically.

While this section is more focused on assessing individual evidence sources, it's also important to consider how each evidence source contributes to your evidence package as a whole. There are a range of factors that influence a medicine's efficacy, such as ingredient dose and method of preparation. It is important that the evidence source should be specific to each of these factors.

The Evidence Guidelines includes a decision tree to help determine the relevance of evidence sourced to your medicine.  Where you have an evidence source that does not match all of these factors, additional justification should be included in your evidence package. A justification can be the provision of an explanation to show how the evidence can be reasonably extrapolated to your medicine. It could also provide other evidence sources that address any data gaps.

Scientific evidence sources can be categorised into a hierarchy based on how much the source is impacted by bias. Each source of evidence should be assessed to determine if it is of sufficient quality. In this context, quality means how confident you can be that the therapeutic effect described in an evidence source is correct.

(While scientific evidence sources can be categorised into hierarchy based on how much the source is impacted by bias, in contrast, traditional evidence sources do not have categories based on bias).

You can see that we've given the hierarchy sources of Category A, Category B and Category C. Certain sources of scientific evidence provide a lower risk of bias than others, due to the design methodology or level of review. When designed and implemented appropriately, double blinded, randomised controlled trials and systematic reviews of multiple randomised controlled trials are the most likely to achieve low bias and high precision when studying treatment effects. Conversely, if a randomised, controlled trial is not appropriately designed and implemented, the results generated may not be robust and reliable.

It's important to note that the hierarchy should not be used in place of conducting your own assessment of the quality and relevance of your evidence sources.

The Guidelines also include a number of links to other resources and tools that can give more detailed guidance on assessing quality.

Evidence guidelines Section four shows how to use evidence. This describes the different types of claims and indications and what kinds of evidence are required to support them. ‘Indication’ is defined in the Act as a specific therapeutic use of the medicine.

Conversely, a claim is a statement that does not describe a therapeutic use. Claims still need to be supported by evidence, but are not required to be included in the ARTG entry. However, it is important to note the claims that imply therapeutic uses are assessed as indications and a similar indication should be included in the ARTG entry for that implied therapeutic use.

Listed medicines may only use low level indications that are included in the Permissible Indications Determination. The types of indications they can make are:

• general health maintenance
• health enhancement
• prevention of dietary deficiencies
• benefit for a non-serious form of a disease or a condition. It's important to note that low level indications may not refer or imply the prevention, cure, alleviation of any disease, defect, or injury

Because of the different types based of evidence that is used to demonstrate efficacy, scientific indications are based on scientific evidence, and traditional indications are based on evidence of traditional use.

Scientific indications are further categorised into ‘specific’ or ‘nonspecific’ indications. This classification system is a means to determine when it is acceptable for sponsors to hold a lower level of evidence. While we recognise all listed medicine indications are low level, there are some that go beyond general health maintenance, and we would expect a slightly higher level of evidence to support them.

‘Nonspecific’ indications for listed medicines refer to general health and wellbeing. The types of nonspecific indications include:

• health maintenance
• relief of general symptoms
• general vitamin, mineral, nutritional supplementation that imply a general health benefit

(The evidence requirements for medicines with traditional indications is the same whether the indication is ‘specific’ or ‘nonspecific’. As such, this classification system is not used for traditional indications).

Specific indications are indications that go beyond maintaining general health and wellbeing. These indications include:

• health enhancement
• reduction of the occurrence or frequency of condition or symptoms, discrete event or named condition
• relief of symptoms linked to a known condition, disease, or disorder
• nutritional supplementation that restore correct or modify a physiological or mental process function or state

The Evidence Guidelines include a tool to assist with the process of classifying indications as specific or nonspecific. We've included several explained examples in the Guidelines to assist sponsors in using the tool. The tool is intended to increase the consistency of indication classification. However, there may be some indications that are difficult to categorise for individual medicines. The decision tool and examples will be discussed in more detail in Session two. For different types of indications, we have different expectations for the minimum level and type of evidence that is required to support efficacy.

This table summarises the evidence requirements for traditional indications. This indicates that a minimum of two from these sources is required. But keep in mind, you should assess all of the evidence sources for quality and relevance. Depending on the quality and relevance assessment, additional sources may be needed, or a justification provided, to cover any information gaps.

The next table summarises the minimum evidence requirements for scientific indications. There are different requirements for nonspecific and specific indications. These are general minimum evidence requirements which may not necessarily be sufficient in all cases. Again, you need to assess quality and relevance for each source and consider whether you need to include a justification or additional evidence sources.

The Guidelines provide some more detail about vitamin, mineral and nutrient supplementation. Supplementation claims convey that, by consuming the medicine, a consumer will ingest additional vitamin, mineral or nutrient than otherwise ingested from dietary sources alone. They do not describe a therapeutic use or effect and therefore, are not included in the ARTG. The minimum evidence requirement for a supplementation claim is that the medicine provides at least 25% RDI (which is recommended daily intake for the nutrient) and, it is in a form able to be absorbed by the body.

Additionally, the medicine must also have at least one permissible indication on the ARTG entry and label. This is because for a medicine to be included in the ARTG, there needs to be at least one permissible indication.

All scientific indications that refer to the effect of the state of vitamin and mineral on region are categorised as specific or nonspecific, using the same principles as for any other scientific indication. We have provided two exceptions to these, and these have special evidence requirements. These are listed underneath:

• “Maintain/ support (state vitamin, mineral, nutrient) levels in the body’
• ‘Maintain support (the mineral) within normal range’

For these, the medicine must provide at least 25% RDI of the nutrient in a form absorbed by the body. In addition, when uses on a medicine label, there should be a label statement stating that they ‘can only be of assistance if dietary intake is inadequate’.

The rationale for these special evidence requirements is, that, a consumer is likely to have a similar expectation for these indications as for a similar supplementation claim. Therefore, these indications have similar evidence requirements as supplementation claims.

We also have biomarker indications in the Permitted Indications Determinations and listed medicines,  can only use low level biomarker indications relating to general health. These are:

• ‘Help maintain/ support healthy blood sugar/ glucose’
• ‘Helps maintain/support healthy cholesterol’

Indications for listed medicines, generally, should only target healthy individuals with biomarker levels that lie within the normal healthy range. Because of the way the permissible biomarker indications are worded, these indications are classified as ‘nonspecific’.

Section five shows how to present evidence. It provides guidance on how to document and present a critical appraisal of the evidence, including providing justifications where appropriate.

The purpose of a critical appraisal is to demonstrate that the sponsor's conclusion of the medicine’s efficacy is logical, reasonable, and based on data we have confidence in to be true and accurate. Having a strong and clear narrative or critical appraisal that draws the body of evidence together is very important in the evidence package to ensure that you can demonstrate to the TGA that the efficacy of your medicine is acceptable. This is especially important where there are gaps and or discrepancies between the evidence sources you include in your evidence package, compared to the design formulation of your medicine.

A critical appraisal should be focused on evidence sources that are most relevant to the research question and are of high quality. Avoid relying on irrelevant or weak evidence sources. Distinguish evidence from opinion, assumptions, misreporting and assess the validity of the results and conclusions in the relevant literature. Assess the impact of competing datasets and assess the impact of any potential for bias.

If you find that you don't have a convincing argument, you should reconsider the design of your medicine, such as the indication, dose, target, or population.

Justifications may be needed when you rely on evidence sources that may be missing critical pieces of information, may not perfectly match your medicine, may be subject to significant bias or compounding or offer an evidence landscape mixed with positive and negative findings. Holding several evidence sources that meet the relevance and quality criteria when combined, may help strengthen arguments to support your conclusion.

You may not always need to provide a detailed, critical appraisal. It can be quite simple, depending on your circumstances. The Guidelines refer to sponsors needing to consider the balance of evidence and provide justifications for any data discrepancies. There will be individual circumstances for each listed medicine and the critical appraisal is your opportunity to provide a clear explanation for why your medicine works as described by its indications.

To assist understanding with using the guidance, we have provided a number of hypothetical case studies in Appendix two. There is also a detailed example of a literature search showing both systematic and non-systematic search strategies and the best practice way to document the searches - that's included in Appendix three.

And that's the conclusion of section one. Thank you very much for your attention.

Session Two

Ji-Won Jung

And so for the second part of this evidence guidelines information session, I'll be going over some details into how to use the decision tool. So as Diane's mentioned, this decision tool is designed so that you can specify exactly how to categorise, each of these permissible indications into specific or non-specific.

Before I go into more details, I’d like to point out that the indications in this has to be taken at face value. So in other words, the presentation or the context of the indications in which they're presented are not considered when using this tool as it's not possible to capture all those nuances. So if you have a look at the tool, you would realise that there are four main questions outlined in these four green boxes. And each of these questions, you have a yes and no answer to them.

Outside these boxes, you have these permissible indications to show how each of these indications would answer to each of these questions. And if you look closer, you'll see that some of these indications are bolded and that means that these bold indications have been explained further in the evidence guidelines in pages 46 to 48. So the tool can be broken down into two major parts.

So the first part is the first question, and it can be considered to be the hardest part of the tool. And the thought processes in how to answer this question will be outlined in the next slides. And the second and third and fourth question – overall, in general, what this is trying to achieve is that it tries to filter out the indications that refer to a single symptom which is not connected to a condition.

And as condition, we define it by a set of signs and symptoms. So these indications which we have filtered out, there would generally be indications which refer to the relief of general symptoms only. And that would change into non-specific indication. So the first question here is, is the indication represented to restore, correct or modify, (which means to increase or decrease) the physiological or mental process, function or state.

So if you break it down, it is essentially asking you, is it restoring, correcting or changing something. And, if it's helpful for you, you can actually simplify it down further by asking yourself, is it actually changing something in the body? And I'll go through some examples here to help you understand this a bit better. So for the first example, ‘maintains support intestinal health’.

If you think about the question, is it actually changing something in the body? So when you think through it, it is not changing anything specific in the body. It's actually just maintaining the intestinal health that person already has. So when you follow the arrow through on the right and start to answer this question ‘No’, you would end up this indication being non-specific. For the next example, ‘analgesic or relief pain’.

So when you think about this, it is decreasing something which is pain. And how this is done is it can go through the physiological pathway in which it can decrease or inhibit proteins or enzymes involved in this pain processes in the body. So essentially when you are decreasing something and you go back to what the original question is asking, it is essentially modifying or decreasing physiological process.

And this case, you would answer ‘Yes’ to this question. So you would go to the next question. So the second question is, does the indication explicitly refer to ‘symptoms of’, or ‘reduced occurrence of’? And this is pretty much, a much simpler question. So the example we have here is ‘decrease (which is relieve) symptoms of dehydration’. So does this indication have the words in there?

And the answer is ‘Yes’, it has the words ‘symptoms of’. Therefore, this will be classified to be specific indication, because when you go out to the right hand side of this box, this goes straight into the specific box. So for the second example ‘helps reduce occurrence of symptoms of mild allergies’. And again, this indication has both of these words. So it would also be categorised as specific indication.

So what about ‘analgesic or relieve pain’? This indication does not refer to any of these words specifically. So you will have to go to the next question. So for your first question, does the indications suggest action of relief? So essentially it’s asking you, is it relieving something? So when you go to the first example, ‘analgesic or relief pain’?

The answer is ‘Yes’, because it refers to the relief and that is the relief of pain. For the second example ‘helps enhance or promotes body energy reserves’ – if you look at the indication itself, it has nothing to do with relieving. So in this case you would go towards answering ‘No’ to this question. Therefore, turns out to be a specific indication.

So for the next example ‘decrease, (which is hair loss) or thinning. This one is a little bit more tricky. So you have to think more carefully about if you replace the words ‘decrease’ or ‘reduce’, and replace it with the word ‘relief’ in there, do you think the indication still make sense? So essentially, it’ll be ‘relieving hair loss or thinning’? And when you think about it carefully, it does not really make sense.

So the answer to this one would be ‘No’, it would not be an action of relief. And it turns out to be a specific indication. For our last example there ‘decrease, reduce irritation or inflammation’. You can replace those words again ‘decrease’ or ‘reduce’ with ‘relief’ to see if that makes sense. And in this case, you say ‘relieving ear irritation or inflammation’ – it does make sense.

Therefore, ‘Yes’. Go to the next question for this case. So as to the final question, ‘does the indication refer only to a single symptom or sign?’ In other words, can you determine what condition it is referring to solely by the symptom or sign? And if you’re not very sure of whether it is only talking about a single symptom, you can break it down to see whether you can break this single symptom into more or multiple symptoms or signs. Because if you can, that would not be considered to be a single symptom as the word condition is defined as a characteristic set of signs or symptoms.

So when you go to the example ‘decrease, reduce, relieve diarrhea’ – diarrhea is a bit more difficult in this case. You might think that it is a single symptom, but when you think more carefully about what it constitutes, it can be separated out into a set of symptoms, and those separate symptoms would be ‘loose, or watery stools’, ‘frequent stools’, which also includes ‘abdominal cramps and pain’.

Therefore, this indication would be considered to be specific because you answered ‘No’ to this question. So what about ‘analgesic or relieve pain’? Pain is a single symptom and a condition cannot be deducted from it. So in this case, this will be answered ‘Yes’ to this question. And as this is the final question, this would be categorised into non-specific indication.

So let's go through one final example, going through each of these questions altogether. So what about ‘relieve itchy eyes’. Is it essentially restoring, correcting, or changing something? And yes, the answer to this question would be ‘Yes’, it is decreasing the itchiness of the eye. Therefore, you would go down to the next question. Does the indication explicitly refer to ‘symptoms of’ or ‘reduce occurrence of’?

And this one’s simple – it is not talking about any of those words. So you answer ‘No’ to this question and go to the next question. Is it relieving something? And in this case, yes, it is relieving itchy eyes. So you go to the next question. Is it referring only to a single symptom or sign, or can you tell what condition it is simply from this single symptom or sign?

Answer is ‘Yes’ because it is referring only to a single symptom or sign. So, as you cannot also determine what condition it is from this single symptom, this will be considered to be a non-specific indication. I’d like to point out here that you must be extra careful in how these indications are presented, especially when you are combining multiple non-specific indications such as these four listed here.

So individually these may be considered to be non-specific indications which only refer to a single symptom or sign. However, when you are combining these, it may be presented in a way that’s implying a condition. And the condition here it’s implying may be considered to be symptoms of hay fever. So make sure that the overall presentation, when you are doing that, you keep in mind that a condition is not being implied.

So in the case where the overall presentation, you are implying a condition – this is when that is expected that you include a parent indication that links these individual symptoms to a condition in the ARTG entry. And further on from that – the parents indications. They are generally considered to be specific indications as they are referring to a named disease or condition. So therefore, you need to have specific level of evidence for the parent indication which supports each of these linked single symptoms.

If your advertised indication implies a condition by combining those symptom-based indications, you may be changing the meaning or intent of the indication. And by advertising the indication that has changed in meaning or intent, you may be considered to be advertising an indication not in the ARTG entry, which is a legislated breach. This is why how these individual non-specific indications are used, and how they're presented is particularly important. And it’s something that this tool cannot take into consideration given the complexity of the matter.

Please be aware that combining indications isn't the only way the meaning and intent of the indication may be changed. And this can also be done through the way an indication is presented. Hence, this is the reason why indications are taken at face value when using this tool, as it cannot capture all those other factors.

But those other factors are also taken into account and consideration in a compliance review. So this finalises this portion of the tutorial. I hope this has helped you a little bit on how to use the tool. And thank you for listening.