> One of the virtues of meta-analysis is that it allows conclusions to be adjusted with changing evidence. One Randomized Controlled Trial included in the BIRD Group's meta-analysis of Ivermectin has been challenged in The Guardian. It is unclear if the study in question is fatally flawed. In this clip, Tess Lawrie shows us what happens to the meta-analysis when the suspect study is eliminated. Answer: the estimate of the medication's effectiveness change somewhat (prevention of death drop's from 62% to 49%. Prophylactic effectiveness goes up slightly from 86% to 87%. Confidence intervals become somewhat wider). The conclusion of the analysis remains the same: Ivermectin reduces deaths substantially and is a very effective preventive measure against the contraction of COVID-19.
Tess Lawrie is an MD and PHD, external analyst for the World Health Organization, and an expert in analysis of medical evidence.
It's interesting to me how different people come to different conclusions even on technical issues like "is this meta analysis sound after removing that study". Here's someone saying the exact opposite:
In that analysis, he appears to remove multiple papers (2?) not just the single paper.
The problem with all research at the moment is it's moving so fast and it's not peer reviewed. Luckily, Ivermectin carries very little risks, is well studied and has minimal drug interactions (and is cheap @ ~$1 per pill).
For anyone interested, there's a database of papers and results: https://c19early.com/
I recommend actually diving into the papers, many are not very good, but that's the nature of the research all around (drugs, vaccine, covid19, etc) at the moment.
There's another analysis (and a better explanation) on this blog - including a section which looks solely at what happens "If you remove the Elgazzar paper from their model..."
"There’s no benefit seen whatsoever for people who have severe COVID-19, and the confidence intervals for people with mild and moderate disease become extremely wide."
Later there's another section in the blog post - but it's not that he removes a second paper. Since it's supposed to be a meta-analysis of all the available studies, "If you include another study that was published after the Bryant meta-analysis came out, which found no benefit for ivermectin on death, the benefits seen in the model entirely disappear."
The Los Angeles Times had a Pulitzer-prize winning writer explore one of the recent meta-analyses - who found this key detail:
All the members of the research team are associated with a British pro-invermectin organization and the money for the research was raised by a Gofundme campaign headlined "Help us get a life-saving drug approved for COVID-19."
The Times also points out that Elgazzar's dubious data "was not peer-reviewed before publication, so it probably should never have been incorporated in the meta-analysis at all."
Peer reviews do not involve reviews of actual collected data, just a review of the paper that draws conclusions from it. I don't know if there's such a thing as peer reviewed data in the medical world. If there is, it can't be a consistent standard, as a lot of studies are published despite appearing to have made up data:
Tess Lawrie co-wrote a meta-analysis that was crowd-funded, on the premises that ivermectin is working against covid.. That doesn't bode well for its neutrality, does it ?
Tess Lawrie ? The same Tess Lawrie who wrote a fraudulent pharmacovigilance analysis claiming that covid vaccines are unsafe, based on the arbitrary claim that anything bad happening to vaccinated people can only be caused by the vaccine. An old anti-vaccine trick.
Recommend watching the full video, basically meta-analysis are only as good as the underlying data. Because these studies test various protocols, it's likely you're going to underestimate the drugs effectiveness. (Later studies are more likely to have better protocols).
In this case, the confidence interval drops, but the reduction in deaths and hospitalizations is still a very strong signal. However, the confidence interval has widened (note, the lower bound is still showing positive results).
What I'm saying is when the largest trial and the one with probably the most leverage in the meta-analysis included turns out to be completely fraudulent, it doesn't bode well for that not being the case for other included trials. This one was 'preregistered' after data was collected among other major red flags and shouldn't have passed QC. That is the type of bias/error that can't be accounted for in the analysis itself.
Sure, but it strikes me that people latched on to this drug as a miracle cure since this study stated that chance of dying with ivermectin was 90% less than compared with a placebo. It just strikes me that any subsequent studies should be viewed with the highest amount of skepticism.
I think this epidemiologist provides a better argument than I will ever do on an online forum: https://gidmk.medium.com/is-ivermectin-for-covid-19-based-on... (TL;DR: efficacy goes to 0 when you add in a new study and remove an additional study that shouldn't have been in the meta-analysis)
Also, let's not forget recent history: the EUA for Hydroxychloroquine as a treatment for covid got pulled by the FDA even though it was celebrated by many as the miracle cure for covid last year both on this site and in the media. https://www.fda.gov/media/138945/download
Yeah, I don't think it's good to grab onto anything in particular.
That being said, the risks with certain drugs are very low. Hydroxychloroquine has some serious side effects and appear to marginally improve results, hence they revoked the EUA.
> Hydroxychloroquine has some serious side effects and appear to marginally improve results, hence they revoked the EUA.
I agree. HCQ was neither safe, nor truly effective (though the NIH study showed no effectiveness even though the FDA left that question open when they revoked the EUA). If ivermectin were truly 90% effective as the original paper claimed, then it should be very easy to replicate it, at any sample size. (e.g. The effect of alcohol on humans can be seen on a sample size of one. HIV retroviral drug studies had 90% effectiveness, and could be quickly measured by the T-cell count in any infected person)
Researchers are humans. I believe there's some kind of "heroic/hope bias" with covid studies that warrants that extra skeptical look. If ivermectin had marginally successful data, the authors (or one of them) might have "juiced" it up to make it more efficacious than it was. It doesn't make them evil, but it does make them untrustworthy. On the other hand, I also wasn't a doctor in an emergency room last year.
All strong antivirals are likely to have some side effects. Almost every medical intervention has effects, the most common of which is a lighter wallet.
I support Remdesivir/Veklury which is a potent IV antiviral. Likewise, it has the following side effects
Hypersensitivity Including Infusion-Related and Anaphylactic Reactions
Hypersensitivity reactions, including infusion-related and anaphylactic reactions,
have been observed during and following administration of VEKLURY. Signs and
symptoms may include hypotension, hypertension, tachycardia, bradycardia,
hypoxia, fever, dyspnea, wheezing, angioedema, rash, nausea, diaphoresis, and
shivering. Slower infusion rates, with a maximum infusion time of up to 120
minutes, can be considered to potentially prevent these signs and symptoms.
Monitor patients under close medical supervision for hypersensitivity reactions
during and following administration of VEKLURY. If signs and symptoms of a
clinically significant hypersensitivity reaction occur, immediately discontinue
administration of VEKLURY and initiate appropriate treatment. The use of
VEKLURY is contraindicated in patients with known hypersensitivity to VEKLURY
or any components of the product
Increased Risk of Transaminase Elevations
Transaminase elevations have been observed in healthy volunteers who
received 200 mg of VEKLURY followed by 100 mg doses up to 10 days; the
transaminase elevations were mild (Grade 1) to moderate (Grade 2) in severity
and resolved upon discontinuation of VEKLURY. Transaminase elevations have
also been reported in patients with COVID-19 who received VEKLURY. Because
transaminase elevations have been reported as a clinical feature of COVID-19,
and the incidence was similar in patients receiving placebo versus VEKLURY in
clinical trials of VEKLURY, discerning the contribution of VEKLURY to
transaminase elevations in patients with COVID-19 can be challenging
> "Phase 3 double-blind randomized clinical trials are needed to provide definitive data."
Let me LOL at this statement. If you think Phase 3 studies can't be hacked in one way or another, well please meet the Pharma industry with its hordes of 1) cherry picking patients for phase 3 recruitment, 2) cherry picking patients who should be removed from the analysis as "dropouts", 3) p-hacking and multiple other practices well known under the sun.
Believing that randomized Phase 3s remove all bias is incredibly naive.
Based on my experience (I designed pharmaceutical clinical trials in an earlier life) this is much harder todo than it sounds. You have to start early (Phase 1) and what you end up with goes on your label (basically the package insert). You don’t always end up where you want to.
The real shenanigans appear once your drug is approved.
Cherry picking patients is not hard, since this is entirely up to the trial's recruitment criteria. This is also what makes comparing drugs virtually impossible since no 2 drugs in the same market apply the same recruitment criteria. This is how manufacturers invite the discussion about "superiority" while competitive drugs are almost never compared directly in the same trials.
I wonder if there are cryptography based secure protocols that can ensure that the studies have been carried out the proper way. Given your experience would love to hear your thoughts.
My second question to you is to understand what apart from deep pockets prevent a trial to be repeated until an outcome favorable to the drug company is obtained. I understand pharma companies are encouraged to make all their trials public, but to me that seems like a generic guidance with no teeth.
> I wonder if there are cryptography based secure protocols that can ensure that the studies have been carried out the proper way. Given your experience would love to hear your thoughts.
Cryptographic methods would be overkill and yet couldn't be applied to the places where such fraud might occur. When you submit trial results the documentation is immense -- you submit pretty much everything. There is a long, highly documented chain of custody and a lot of people (and independent entities) involved so it would be very difficult to pull off a conspiracy. That's not where the problems lie.
For example if the agency thinks something is irregular they can go back and look at the underling data and the instructions on how those data were collected.
The amount of data supplied is so large that before the days of electronic filing every drug company had a group whose job was simply to manage the logistics of organizing the trucks needed to transport a single submission! That sounds therefore like there would be a needle-in-a-haystack problem, and of course there always is, but you'd be surprised by the tiny details you get asked about as someone sees something they don't understand or are suspicious about and just follow the trail backwards.
> My second question to you is to understand what apart from deep pockets prevent a trial to be repeated until an outcome favorable to the drug company is obtained. I understand pharma companies are encouraged to make all their trials public, but to me that seems like a generic guidance with no teeth.
So this is the place to look: at a high level. However what you ask is hard. You can't legally give humans unapproved drugs (the agency doesn't give you permission to do so; they say "you proposed to use X on people who meet criteria C, following a specify procedure P, in order to answer questions Q, R and S. As long as you do precisely as you asked, only on the population C and on no more people than discussed, we will not prosecute you for using an unlicensed drug". And you submit an enormous amount of data you have already collected that explains why it's worth giving it a try and why it isn't a risk to human life.
So you do specific pre-clinical (animal) work just to get it into humans at all (phase 1), then use the safety data you collected to justify some kind of limited study of efficacy (Phase 2) and based on that you do a large, representative (of the people in the US who might need this drug) population of subjects.
If you tried to repeat a trial the agency would say "but you already did this and didn't like the results, so no". And in fast why would you? You'd be desperately trying to figure out why it failed and try to do something different.
Regarding cryptographic methods, I was indeed clutching at straws. It seems to me that current methods can convey that there are 'no obvious indicators of fraud', it would be nice if pharma had recourse to methods by which they could 'prove' that fraud couldn't have happened. I suppose it would be quite impossible to design such a protocol, 'best effort' is probably all one can do. I think mandatory pre-registration itself fixes many of the potential problems.
On a different note: compilers and medical trials … I don't see those going together a lot.
> On a different note: compilers and medical trials … I don't see those going together a lot.
Or power generation or... I have enjoyed working in different domains requiring completely different technical skills. I have learned a lot.
It's one thing to learn chemistry and lab techniques and protocol. It's another to present before the FDA who doesn't have any interest in what my previous career might have been.
Some folks, perhaps many folks like 'more wood behind fewer arrows' culture. I lean the other way, so I take interest in stories where people have moved from things to other things. For understandable reasons CS seems to have a lot of them.
The Google reference is a little ironic, because at one point in time, say around 2004, it was a place where anything that's intellectually challenging and gratifying was fair game.
I would assume Bell Labs would have been something like that.
Perhaps phase 3s aren't perfectly free of bias, but they are essentially impossible to effectively manipulate by the drugmaker.
Even if we assume all the physicians involved in the trial are willing to be bribed, the pharma company doesn't know which ones are handing out placebos.
> Perhaps phase 3s aren't perfectly free of bias, but they are essentially impossible to effectively manipulate by the drugmaker.
Not completely accurate.
Studies with negative outcomes are not required to be published by the drug maker. You can virtually keep trying the same drug and "get lucky" with a single trial, and just register that one with the authorities since the standards for replication are still very low.
Preregistering does not mean anything about sharing actual results. the large majority of negative outcomes are NEVER published. Please don't spread misinformation.
Pretty sure FDAAA has mandated reporting results for a while now (although some sponsors choose to break the law).
Do you have an example of your scenario of an approved drug with many phase 3 trials pre-registered but only one with reported results?
> pharma company doesn't know which ones are handing out placebos..
Not sure I get you. If all the physicians are bribed, then why would the company need to know anything? Everyone can cook up some data and go along with the result that the company needs, right?
Also, about this
>If all the physicians are bribed
I think you should assume that any physician that ends up in a trial is there because they are known to be complaisant. Same as politicians and authority figures are often planted by powerful people with vested interests.
> By phase 3 they should have to preregister the study and identify the outcomes they will measure. There isn't much room for p-hacking at this point.
Not all Phase 3 with negative outcomes are published. It's actually the opposite. Most of them are not. So preregistration achieves nothing in terms of transparency.
And p-hacking still happens far more often than you think - I have personally seen trials where they identified "sub-groups" of patients in order to prove that there were patients for which said drugs worked. That's a thing, no matter what you decided to set in the first place as objective.
And then there's stuff like Biogen's alzheimer drug for which no efficacy was proven but still received FDA commercialization approval. Everything is working just peachy, right?
Necessary but not sufficient. Those are issues that can plague all studies, but at least some sources of bias and error can be mitigated against with randomisation and blinding.
> It appeared that the authors had run entire paragraphs from press releases and websites about ivermectin and Covid-19 through a thesaurus to change key words. “Humorously, this led to them changing ‘severe acute respiratory syndrome’ to ‘extreme intense respiratory syndrome’ on one occasion,”
The thing I found most interesting was the ungodly mess they found in the spreadsheet of the raw data. I almost never see papers backed by a link to the spreadsheet or other software used to calculate the results, but I feel like that ought to be expected for publication nowadays. I wonder how many other papers would implode if their raw data were easily available for download.
An interesting comment I read somewhere else was that the realpolitik case for discrediting treatments is that deploying the fast tracked vaccines is predicated on a legal emergency measure, for which the key requirement is there are no other viable treatments.
I don't think there is serious controversy that the vaccine rollout is being used as leverage for imposing digital identity schemes and for establishing a variety of other ostensibly temporary powers that will not be rolled back, and so it is difficult to "trust the science" on this as the question has become 90%+ political. The consequences of a non-vaccine treatment being available would obviate the need for all the supporting systems for it, and derail a large co-ordinated effort that has been some years in the making. If you know anything about digital identity, you know what's been going on in that space right now and who/what is driving it. Conspiracy theories about mere pharma profits are distracting and don't think nearly big enough.
If we really want to get everyone vaccinated for c19, we need to commit to no domestic vaccine passports. If we require the passports, the rollout is going to stall, you're going to mass-disenfranchise 20%+ of the middle class, and when we have to deal with a threat with a more serious fataility rate, there will be no institutional credibility left to respond to it.
I'm not sure we should be haggling the details of chemistry and biology we aren't equipped to reason about when the real policy issue of passports is something we all have a stake in.
You seem to be implying that the reason so many people refuse to get vaccinated is because of some passport mechanism?
That does not match my experience whatsoever; the people refusing a vaccine are all worried about side effects because they think it’s still “experimental”, and they don’t believe C19 poses a serious threat to them. I’ve never heard anyone mention anything about “digital identity schemes”.
> legal emergency measure, for which the key requirement is there are no other viable treatments.
Dexamethasone is an approved, cheap drug that is now also recommended by the FDA to treat covid-19. The theory there’s some conspiracy by government authorities to deny any approved drug is working on covid is clearly bogus.
The FDA will be granting full, not emergency, approval to the Pfizer covid-19 vaccine in the coming months, and then this argument falls apart completely.[1]
> I don't think there is serious controversy that the vaccine rollout is being used as leverage for imposing digital identity schemes and for establishing a variety of other ostensibly temporary powers that will not be rolled back
Seems like nonsense, only two states have rolled out a vaccine “passport” (nothing like the actual passport schemes created in China and some foreign countries to reopen economies faster - the US version is just a digital document similar to how you can buy a railroad fare digitally) and the latest reporting is that Biden’s covid team is blocking incoming travel from Europe because they are opposed to vaccine passports, fearing that conspiracy theorists wouldn’t get a vaccine if they allow vaccinated travelers from Europe to prove their status and visit.[2]
Agreed that treatments and FDA approval will change the dynamics and medical need for c19 vaccination - when they happen. Dexamethasone is a corticocosteroid used as an immunosuppressor to clear airways, and not an alleged prophalactic treatment like Ivermectin, or an antiviral like HCQ, so that's a bit of a misdirection in comparing it to a treatment. It's like saying a defibrillator is an alternative heart disease treatment.
And if there are treatments, passports are unnecessary. If you have followed the hesitancy reasoning and indeed, now movement judging by european protests, there is a consistent logic in their objections. They aren't standard hippie anti-vaxers. Whether the facts their logic uses as premises are true or falsifiable should be the focus of debate.
Maybe there's an example of what level of c19 vaccination is strictly necessary to reopen economies that bears some scrutiny. If we keep enabling this constant apopleptic personal response to people with legitimate questions, we're going to lose a good 30%+ of people who don't respond to that kind of thing.
> I don't think there is serious controversy that the vaccine rollout is being used as leverage for imposing digital identity schemes and for establishing a variety of other ostensibly temporary powers that will not be rolled back
Well you're flat wrong about the controversy bit, not even reaching the substance.
It's not seriously controversial. If you work in identity or health, you know what technologies are being used, tech standards are being pushed out via public health networks, (SMARThealth, WHO's covax, etc.) and the massive changes in privacy regulations and information sharing in support of it are unprecedented. There are probably details that are still confidential in government, but that parties are leveraging the pandemic crisis to drive prior agendas is no secret. To deny that is unserious to the point of the very gaslighting that is creating the mistrust that is impacting the rollout.
Data on ths subject is so politicized it's not even science anymore, it's scientistic policy activism. Arguing in the ivermectin/HCQ debate isn't going to impact policy or rollouts, but a tangible commitment to not exploiting the crisis to impose centralized identity governance would.
I'm curious about what you talk about here, but I haven't heard anything about it. Any reading or further search terms beyond "SMARThealth" and "WHO covax"?
Why do the researchers do this? It doesn’t sound like they are peddling a (cheap, generic, unbranded) drug. Is it literally just to get a name in a journal?
If this is the objective, I’m actually surprised at the amateurishness of the fraud. I’d imagine you could cook the data much better.
Ivermectin has been know about since April 2020 when the Caly paper showed in-vitro that at high doses, ivermectin had anti-viral properties (sars-cov-2 sends proteins into the cell nucleus to turn off your immune response (interferon)). However, western scientists thought this anti-viral effect was implausible (https://www.microbe.tv/twiv/twiv-599/) - they were mostly correct as the effect is most likely immunomodulatory (Melo et al). Thus, NO clinicial trials of Ivermectin have been completed in the West (apart from a tiny 20 person trial in Barcelona).
In contrast, the developing world has completed 30 randomized controlled trials (RCTs) and 60 trials in total - https://ivmmeta.com/. Ivermectin is safe (a WHO approved essential medicine), so we only care about efficacy. It is safer than paracetamol and has been taken billions of times.
No trial completed to date has shown that it does not have a positive effect. Most trials have shown statistically significance on the most important clinical outcomes - hospitalization and death.
Ivermectin works best when given early. It is transforming treatment and having incredible success in places like Uttar Pradesh (200+ million inhabitants, with only 38 new cases and 1 death yesterday - https://www.covid19india.org/).
So, Ivermectin is an enormous threat to the Pharma industry. Merck (who discovered Ivermectin and help cure river blindness by donating it) are down-talking it, so they can bring Molpuvanir to market (they got a couple of billion from the US govt for this recently). Vaccine manufacturers run negative ads on google against Ivermectin (https://www.biznews.com/thought-leaders/2021/05/12/mailbox-i...). AND the person who discovered the fake data in the trial linked was supposedly a masters student, but he runs an investigative service and the level of sophistication in finding the data showed higher involvement - https://trialsitenews.com/ivermectin-study-appears-fraudulen....
This is like the tobacco industry funding scientific trials. We already had a discredited Jama paper (https://jamaletter.com/) on Ivermectin/Covid. So they can do it, but I really hope the truth on Ivermectin - that it is an effective treatment for Covid - comes out. Otherwise, I will despair for society and advocate revolutionary change. However, the 2 big western trials being funded are being funded by the vaccine manufacturers (indirectly, of course), via Macmaster Univ (Gates-funded) and Oxford (Gates funded). RCTs can be designed to not succeed (it can't fail with Ivermectin, only not shown huge effect), and these are designed so - Bill
I don't know whether ivermectin is an effective, let alone transformative, treatment for covid. I certainly hope it is! We could all do with some good news.
I don't have time or expertise to evaluate, let alone conduct, clinical trial methodologies or meta-analyses, so I have to rely on secondary sources. The institutional consensus (from WHO, etc) is that the treatment is unproven, and further research is required. https://www.who.int/news-room/feature-stories/detail/who-adv...
It is apparent that there is a community (ivmmeta, BIRD, FLCCC) who strongly disagree and believe that ivermectin efficacy is already proven.
So the question becomes: who to believe?
We definitely have historical examples of astroturfing, regulatory capture, etc occurring - such as oil companies going to enormous lengths to muddy the science around climate change. And we also have recent examples (HCQ is the obvious one) where a large number of people became very excited about a treatment which has, I think, been fairly conclusively shown to be ineffective.
I have no magic method to determine which of these is the case this time, but my money is on IVM being a washout after further research.
I am skeptical of the BIRD folks, because they are acting and talking like a pressure group, not an open-minded body. I frankly doubt their impartiality when evaluating the evidence - I'm not alleging a nefarious purpose, just the omnipresent danger of becoming too attached to your pet hypothesis. To paraphrase your statement, I hope the truth on ivermectin comes out. And I hope it is an effective treatment. But these are two separate hopes.
BTW: You're providing an example here of how, once you start looking at a particular issue with a conspiracy mindset, you can easily construct an impervious epistemic bubble. The large trials cannot be trusted because they have been designed to fail; shadowy forces lie behind the doubts cast on the Elgazzar trial; and so on. This is not to say that conspiracies never exist. But it is worth asking yourself: what could I learn at this point which would change my mind?
You're implying I have selective bias in the evidence I put forward. However, you have only put forward a hunch - "my money is on".
Your method of critique is post-modern - you have your evidence, they have their evidence. But that is not true. There are no studies (apart from one) that show Ivermectin is not effective - it's just the effect level that differs between them. There are now 30 RCTs! When you know that Ivermectin works best when given early and at higher (but safe!) doses, you can design studies that show a lower effect level. Just give it late at low doses on a healthy cohort. Then you conclude "Ivermectin has no effect". But meta analyses account for variances between RCTs - and they show it works.
If you look at the MacMaster and Oxford trials - it allows participants who have had first sympthoms of Covid up to 14 days ago, from age 30. And they only dose for the first 2/3 days at 200/300 mcg/kg, respectively. We know from the meta analyses there is a dose response, and ivermectin has safety data with over 1200 mcg/kg. So, yes, you can design a RCT to give the answer you want - if you are incentivized to do so. Show me the incentive and i will show you the outcome.
https://odysee.com/@DarkHorsePodcastClips:b/ivermectin-meta-...
> One of the virtues of meta-analysis is that it allows conclusions to be adjusted with changing evidence. One Randomized Controlled Trial included in the BIRD Group's meta-analysis of Ivermectin has been challenged in The Guardian. It is unclear if the study in question is fatally flawed. In this clip, Tess Lawrie shows us what happens to the meta-analysis when the suspect study is eliminated. Answer: the estimate of the medication's effectiveness change somewhat (prevention of death drop's from 62% to 49%. Prophylactic effectiveness goes up slightly from 86% to 87%. Confidence intervals become somewhat wider). The conclusion of the analysis remains the same: Ivermectin reduces deaths substantially and is a very effective preventive measure against the contraction of COVID-19.
Tess Lawrie is an MD and PHD, external analyst for the World Health Organization, and an expert in analysis of medical evidence.