Back HIV Prevention Pre-exposure (PrEP) New HIV PrEP Studies Will Be a Challenge, Statisticians Warn

New HIV PrEP Studies Will Be a Challenge, Statisticians Warn


Two statisticians involved in the PROUD and iPrEx trials of HIV pre-exposure prophylaxis (PrEP) warn that future trials to test new PrEP drugs and formulations may be extremely difficult to design. David Dunn of the UK Medical Research Council and David Glidden of the University of California at San Francisco say that statisticians will need to choose and analyze their trial populations very carefully if they wish to be able to demonstrate meaningful results.

[Produced in collaboration with]


While pre-exposure prophylaxis has been remarkably effective in preventing HIV, both in scientific trials such as PROUD and Ipergay, and in public availability programs in the U.S., there is a general consensus that alternatives to tenofovir/emtricitabine (Truvada) need to be found. This is the only drug combination licensed for PrEP in the U.S. and some other countries, and under consideration for licensing in Europe.

Truvada can cause short-term initial side-effects such as nausea that may put off people wanting to take it, and while long-term serious side effects are uncommon, studies do document slight, reversible declines in kidney function and also in bone mineral density among people taking PrEP. There is a new formulation of tenofovir, tenofovir alafenamide or TAF, that produces lower rates of these side effects and is already licensed as part of an HIV treatment combination pill (Genvoya). Its manufacturer, Gilead Sciences, is known to be keen to test TAF plus emtricitabine as PrEP.

In addition, HIV resistant to both drugs in Truvada does occur, though rarely, and can arise when people start Truvada soon after becoming unwittingly infected with HIV, though this appears to fade over time.

However, the most urgent need for formulations of PrEP other than Truvada is that while knowledge of the effectiveness of PrEP has spread, certain groups --young women in some African trials, and young gay men and black men in the U.S., for instance -- still struggle to adhere to Truvada PrEP, taken either daily or intermittently. For these groups and others in situations that may make PrEP adherence difficult (homelessness, poor mental health, incarceration) other drugs and, especially, long-acting drugs including injectable formulations already studied for treatment may be needed to offer protection from HIV.

We also have no direct data for the efficacy of Truvada as PrEP for people who inject drugs, only for solo tenofovir, and that comes from a trial with hard-to-interpret results.

It may not even be a new drug or formulation we need to test. We also need to test Truvada against itself in the form of different regimens -- such as intermittent versus daily dosing. So far, we only have the result from the placebo-controlled Ipergay trial, which found that intermittent PrEP was effective in a high-incidence population, but did not compare daily with intermittent PrEP. We await final effectiveness figures from the 3 branches of the ADAPT study, but it will be difficult to get a final answer from a study with only 180 participants per site.

Devising New PrEP Trials

The problem with devising trials of new PrEP options is the very effectiveness of Truvada. First, this now makes it not only unethical, but also scientifically meaningless, to do a placebo-controlled trial in which half of the group receive dummy pills instead of the new drug -- meaningless because the new PrEP drug should be tested against the best alternative, which is now Truvada rather than other forms of prevention.

However, directly comparing a new pill against Truvada might require an unfeasibly large trial. Dunn and Glidden cite a design pitting Truvada against a drug which in reality has exactly the same effectiveness. This, of course, would not be known beforehand -- that’s the point of doing the trial. The trial design used would be a "non-inferiority" one, in which the object is to find out whether the drugs are equivalent to each other in terms of effectiveness, or whether one is clearly better. The researchers define that the new product would be considered "superior" or "inferior" to Truvada if it was over 25% more or less effective.

If this trial was conducted in a population with a fairly typical annual HIV incidence (infection rate) of 2.25% -- which is roughly equivalent to the annual incidence seen in U.K. gay men attending STI clinics who have had an HIV test within the last year -- then it would require a 2-year trial with 19,000 participants to be able to determine for sure that the new product was 25% more or less effective than Truvada -- clearly impractical. To rule out that it was 10% more or less effective, we would need an even larger trial.

A smaller trial would be required if there were advance indications that the product was very much superior to Truvada but, given the already high biological efficacy of Truvada, this is quite unlikely and would depend on the product being much easier to adhere to for the trial population.

There are ways of getting round this. The first is to select a trial population from groups that have clearly found it difficult to adhere to Truvada. This is the hope of researchers testing injectable formulations, where the adherence is to quarterly clinic appointments rather than daily pill-taking.

Statistical Reinterpretations

However there are statistical ways around it too. Non-inferiority trials report something called a hazard ratio, which is the proportional or percentage difference between the effectiveness of one drug and another.

For a public health prevention tool like PrEP, an absolute rather than proportional measure called the rate difference is more important. This is simply the absolute difference between the number of HIV infections among people in different arms of the trial, rather than the proportion. Because even in a high-incidence group the majority of the trial population would not have caught HIV anyway, the rate difference is often a numerically larger and more relevant figure, especially as it translates directly into the most important figure of all -- the NNT or number needed to treat, which says how many people would need to be given PrEP in order to prevent 1 HIV infection. This is the figure that tells how cost-effective PrEP might be.

However, Dunn and Glidden add, in order to put this rate difference into its true perspective, one needs a third piece of information -- the "background incidence," or what the HIV infection rate in the trial population would have been if they had not been offered PrEP of any kind. In the placebo-controlled trials one could assume that the background incidence was that seen in the placebo group, though in practice that was not always the case.

In a trial without a placebo arm, however, establishing background incidence is going to be crucial but difficult. We may need several years’ worth of data on the population in which we intend to do the trial, and even then we cannot be certain that the people who eventually consent to take part in the trial reflect the full population from whom they are selected.

For instance, look at a situation in which a new experimental drug or regimen in fact turns out to be less effective than Truvada -- which could easily be the case. If in a 2-year trial with 5000 participants, 15 HIV infections were seen among participants on the new regimen and 8 among those on Truvada, then it looks like Truvada is nearly twice as effective.

Compare it, however, with 2 different background incidences. In one, the background rate is 0.4% a year, meaning that 20 infections would be expected to occur in a year among people not taking any PrEP at all. In this case, the 15 infections on the new drug and the 8 on Truvada look like a huge difference -- 25% versus 60% effectiveness -- for a hazard ratio of 2.4.

In the other study, the background incidence rate is 4% a year, meaning that among 5000 people one would expect 200 new HIV cases without PrEP. That then means that the new drug is 93% effective and Truvada is 98% effective -- a hazard ratio of 1.03, which is statistically indistinguishable from 1.0, i.e., no difference.

But the rate difference in both trials between the experimental drug or regimen and Truvada remains the same in both trials: 0.14%, or 1.4 cases of HIV per 1000 patients per year. That is the number of extra infections per year we would see using the new drug or regimen or formulation instead of Truvada. If the new formulation had other advantages such as cost, availability, or a better side effect profile, then we might consider it worth using.

Establishing Background Incidence

As we said, it is going to be difficult to establish a reliable figure for background HIV incidence. One could use a historical control, such as incidence in the placebo arm of a previous study, as in the Partners PrEP demonstration study, but incidence changes over time and a suitable historical control group may not be available.

In some situations one could simply use the proportion of patients among people applying for the trial who are diagnosed upon screening, but this would not work in situations where people test frequently.

The only other alternative is to try and work out the background incidence from HIV diagnosis figures in the population one hopes will be interested in PrEP -- see this report for a quite detailed analysis of this sort conducted in Barcelona.

However, as Dunn and Glidden show, background incidence in the population targeted for a trial has been notoriously bad at predicting actual incidence in the trial. In almost all the early placebo-controlled PrEP trials, incidence in the trial was 2 to 4 times lower than that predicted by pre-trial surveys. But in PROUD and Ipergay, the opposite was the case, and incidence in the control arms was 3 to 4 times higher than predicted.

In the former case, as well as being cautious because they knew they might be on a placebo, participants may have joined at a period of particularly high risk and then "reverted to the mean": their risk declined to what it had been more typically. In the latter case, standard HIV tests may fail to detect recent infections and thus underestimate recent incidence.

This introduces another idea: in the PrEP trials so far, the effectiveness figure given has excluded cases where people caught HIV immediately before starting PrEP, during their "window period" when no test can detect HIV. This is because there is no way PrEP could have stopped these people from getting HIV. There were 3 cases like this in PROUD.

But there is no perfect way to eliminate every person who may have just caught HIV when they come forward for PrEP, and testing requirements in rollout programs are unlikely to be as elaborate as they are in trials. For this reason, Dunn and Glidden say, cases like this should be henceforward included in realistic estimates of PrEP’s effectiveness in widespread provision. In the PROUD study, for instance, this would have changed the effectiveness from 86% to 78%.  

"Future trials of PrEP are highly challenging to design," concluded Dunn and Glidden. "New statistical paradigms for non-inferiority trials are required, with statisticians and expert clinicians working closely to develop these."



DT Dunn and DV Glidden DV. Statistical issues in trials of preexposure prophylaxis. Current Opinion in HIV and AIDS 11(1):116-121. January 2016.