Presenting the true positive rate without mentioning false positives seems problematic. I couldn’t find any information about that in the article, and they don’t link the paper / publication.
This is a key point. If the false positive rate is 18% (as noted in other comments), and the rate of Alzheimer's is 36/100k[1], then for every patient the test correctly identifies, there are 500 patients that tested positive but did not have Alzheimers. There needs to be significant improvement before such a test becomes useful.
[1] that's mortality rate which I am using as a proxy for incidence rate, though the actual incidence rate is presumably higher (mortality rate from the CDC)
I'm not good at this (yet), but my understanding is that your prior for just this test would be the likelihood of having Alzheimer's based on the statistical occurrence within the general population.
After this test you now have updated priors which may not be good enough to make a certain diagnosis, but wouldn't it still be useful combined with another independent test that has similar rates, since you'd be using these updated priors?
The paper itself [0] claims 82% specificity and 100% sensitivity.
Meaning, 100% of positive cases in the test set were correctly identified, and 82% of negative cases in the test set were correctly identified. That corresponds to a 0% false negative rate and an 18% false positive rate. The test set is 40 imaging studies from 40 patients.
Wouldn’t 82% of negative cases being correctly identified mean that 18% of negative cases were incorrectly identified as positive(false positive), or in other words it will mistake 18% of healthy people as having Alzheimer’s, but will always identify an Alzheimer’s patient?
82% true negative rate == 18% false positive rate. Your description is correct. It's likely that those statistics will be somewhat worse when the algorithm is tested on a broader and more diverse test set, which the authors claim to plan to do in the future.
Thanks for digging up the paper, false positives are probably more acceptable than false negatives in this scenario (wrong positive diagnoses can be rectified with additional tests but a negative diagnosis will often not be confirmed by additional testing).
Both things matter, and you need to consider what the follow-up test might be and what it would cost not just financially but also in terms of pain, lost time, risk of harm and so on.
There are medical tests which have such a high rate of false positives that in practice in the absence of other indications you would ignore a positive result, because the only possible follow-up test would be a non-risk-free biopsy, and therefore, for your peace of mind, it's probably better not to do the first test at all.
A false positive rate (currently) that high may preclude this being used as a widely available diagnostic test, but if used within a patient base already showing other signs of problems it might be more acceptable. After all, if you're going to see a memory specialist you're already thinking there's a problem.
Firstly, I agree. You can't hide from that, and research should be upfront with the limitations of the work.
Other commenters have noted that the sensitivity is 100% with specificity 82% (false positive rate being 18%). I wanted to provide my own take on what that means for me as a future provider (I'm considering becoming a neurologist after medical school). I'm assuming to the data science crowd here, it's almost second nature to know what these numbers really mean in practice, but maybe I can provide some outside perspective.
When considering screening tests in medicine, we have to always balance the benefits and risks of conducting these tests on patients. We have to figure out how reliable a test is, and how patients benefit from earlier detection.
In this case, it's an imaging test, so noninvasive. However, it's a test that requires radiation (positron emission tomography or PET). So we already have to determine if that extra radiation dose is worth the increased risk for something like cancer. In an elderly population, it's almost certainly worth the scan given the radiation would not significantly impact their decades of accrued radiation damage.
Now, what happens if a patient has a positive test, 6 years earlier than they would normally? Let's assume for a moment that it was a true positive test. They could receive treatment earlier, which could improve their quality of life. They could adjust their risk factors to slow down the progression of disease. The treatment could have side effects, such as increased blood pressure. If we apply that to a large population scale, would we increase mortality associated with heart disease by treating patients with suspected Alzheimer's earlier?
Lastly, I wanted to touch on the effects of a screening test with a large amount of false positives. I would argue that a screening test should be allowed to have lots of false positives, simply because we can always do other tests (neuropsychiatric evaluation like MOCA score) to improve our accuracy of detection. However, if we do these other tests, we have to again balance the harms with the risks. Mammograms, for instance, could give false positives that could warrant a breast biopsy. In the case of Alzheimer's, a false positive could lead to stress and depression in an elderly population already at higher risk for stress related disease.
In short, screening tests are great if there's a way to do something about the disease. As far as I know, early prophylactic treatment (with current medications) of Alzheimer's does improve quality of life, slowing progression of the disease manifestations, with relatively few side effects.
You missed some downsides: anguish, more tests for something that isn't a problem, costs, possibly suicides. You may want to read up more on the downsides to breast cancer screenings for reference.
I didn't delve into all the downsides, but I did mention the stress and related depression that could be associated with a false positive. I also noted how there are downsides to mammograms, such as a breast biopsy. We could go a step further and note that some women opt for mastectomies as well, especially for genetic screening (BRCA).
There's plenty to delve into regarding the risks and benefits of screening tests: Here's a good framework for those interested.
Woah is the radiation in an imaging test really so dangerous that you carefully consider whether its worth doing even once? Because uhh i get those pretty often
No, not especially. It's mostly dependent on how early in your life you receive doses of radiation. It's why we tend to stay away from CT scans in pregnant women, or other high dose radiation in kids. However, the obstetrician-gynecologists I worked with drilled into me that if you need a CT scan, do it anyways because technically one CT scan does not exceed the dosage that will cause damage in babies. XKCD made a nice little graphic comparing the dosage of different radiations:
Yes, but whatever the false positive rate is, my guess is it will decrease significantly in coming years. For instance, if a patient gets a positive rating, there might be another type of test that would eliminate most of the false positives. It seems to me quite unlikely researchers would not try to make improvements.
It's not problematic. It's on purpose and it's criminally misleading. If "AI companies" had to show their false positive rates, 99% of them would go bankrupt because then nobody would want to buy their bullshit.
Typically Alzheimer’s diagnoses are given with extreme reluctance, it changes how every person that interacts with that patient will view their condition, prognosis and treatment options.
I think that has more to do with the extended time frame than the actual ability of family and medical providers to see changes in a patient cognition and abilities.
Well, as inhumane as it seems I can promise you no one wishes Alzheimer's on anyone and when the person gets it they are no longer the person you knew. In a sense, the disease has killed off their mind and the person you knew. Spiritual death. But the body lives on. The other thing with this disease is the rate at which it can come on. Early on set Alzheimer's can come on so quickly that everyone is caught off guard. If you were in your 40's married to someone in their 50's who suddenly needed full time care, as harsh as it sounds I would not judge that person for leaving the marriage. In my personal opinion the person with advanced Alzheimer's would have little insight to what exactly is going on around them, like their spouse is no longer with them, and divorcing them would not even register. I more picture these divorces happening after the person is well on their way and really lack the mental capacity to understand or worry about it. It would be a little harsher if the machine could tell you 6 years before you got the disease and your partner left you but again I would not judge.
Well maybe.. forcing someone to stay married may also be considered inhumane. It also probably means the spouse with Alzheimer legally cannot consent to a divorce as well.
I feel the opposite. If I was unhappy in my marriage and found out my spouse now also has Alzheimer's, granting a quick exit from a situation that can only get worse seems humane to me. Why force somebody who is suffering to suffer even more?
Maybe because you took a vow to care for each other in sickness and in health....
Marriage is a social contract. Some of the benefits (lower income taxes, preferential inheritance, etc) are in recognition that you are taking on some of the responsibilities that would otherwise fall to society.
That's a very culturally specific view. In the culture where I was married (Japan) one person (either one) is removed from their existing family registry and inserted in another one. You do it at the city hall and it is entirely composed of paperwork. They even give you a free pen :-) You don't vow anything.
What you are responsible for and what benefits you enjoy from being married is very much a product of that culture. In my case, if my wife is ill, I do have to provide for her (Fun fact: in Japan you require your spouse's written permission to get a divorce except in certain circumstances). However, it would not surprise me that some other cultures (perhaps Norway) have different responsibilities. The world is large and "marriage" means different things to different people.
Making it easier to divorce your spouse if they get Alzheimer's is more humane for spouses of Alzheimer's victims (who get to escape that unpleasant situation more easily) and more inhumane for Alzheimer's victims themselves (who are more likely to find themselves abandoned at a time when they are increasingly unable to cope on their own).
> who are more likely to find themselves abandoned at a time when they are increasingly unable to cope on their own
That often happens rapidly, and in many cases putting the Alzheimer's patient in full-time care is the only option -- even with a caring, involved spouse.
I watched my grandfather go from a slightly forgetful goof to confused, violent, and impossible to manage in <3 years. My grandmother, not doing so well herself, struggled to keep up but after a couple years putting grandpa in a facility was the only real tenable option.
Not necessarily. They might die of other causes first.
There are several things in this discussion that make not much sense to me (a non-Norwegian):
* How would a divorce court refer to a (confidential) medical diagnosis? In some countries, at least, even a criminal court cannot easily access such things.
* Why would the law refer to a diagnosis of a particular disease (when the patient might still be healthy) rather than refer to actual cognitive impairment?
* What's the hurry? Can't you get a divorce fairly quickly just by moving out and filling in some forms?
* One of the major legal implications of divorce is that you don't automatically inherit or get insurance pay-outs. Not the most obvious thing to want when your spouse has a terminal illness, though in some cases you know there's no money involved. Even if you're named in the will there may be bad tax implications if you're not married to the person you're inheriting from.
* Someone mentioned children, but divorce doesn't have to relate directly to children: courts have to deal with the children of unmarried couples, and they have to deal with the children of couples that are separated but still married, so I'd expect a court to worry about the relationships and the welfare of the children and not to care very much whether the parents/guardians are officially married or not.
I would guess that these are all areas of law in which there are a lot of differences between jurisdictions.
> more inhumane for Alzheimer's victims themselves
I think you are making the assumption that the unhappy and trapped spouse is going to provide loving care. The Alzheimer patient may be better off with other family or in state care than being dependent on somebody who may be hostile to them.
It also forms a legal separation for new debts: both debt associated with treatment (in the US, treatment is extremely expensive) and debt acquired due to them being exploited by less ethical forces in our society.
Since a marriage is a legal binding there are procedures for a divorce.
You need to live separated for one year to have it approved for one thing.
And if you have children under guardianship you are required to attend family counseling or such.
I assume that what is refereed to in the upper comment is the waiting period and other requirements.
Personally I did not know this was the case with altzheimer even though I am Norwegian.
Unless you want to involve a judge and a legal process I imagine you usually need two signatures, at the very least. In Norway there are other requirements as well.
Seems like a great incentive to pay off a doctor to diagnose someone so you can lose less money, get the kids, have another bargaining chip in a messy divorce, etc, etc.
Edit: Pointing out flaws/possible exploits in a system is not the same thing as endorsing their use.
In that case the same could be done with sexual assault or abuse allegations. There's no end to possible duplicity, this doesn't really extend it, especially as Alzheimer's is going to be a super specific diagnosis. What is the Venn diagram of people who are fighting over kids and also suffering from Alzheimer's?
That seems like a problem subject to heavy confirmation bias - e.g. people decide their relative is cognitively impaired and see any transient shortfall as being more evidence, rather than any number of other possible causes.
(Not that you can't necessarily see cognitive decline externally earlier than a diagnosis, just that it seems like it'd be hard to make a clinical argument about persistent decline versus several poor nights of sleep, for example.)
i think these early diagnostic techniques are a necessary brick on the road to curing AD. by the time people are showing clinical symptoms the disease has already progressed and has been doing so for decades. treatments that slow progression will be more effective if administered earlier on in the progression.
No questions here, but you guys might be interested to see the implementation I took with this, where I also used machine learning for Alzheimer's / dementia diagnosis.
I was using OASIS's public dataset so I only had ~150 images to work with, instead of ~2000. I used transfer learning from ImageNet's dataset to try and get usable results. I also had super limited testing (15-20 patients), but got ~60% accuracy with ~13% false positive rate.
It could be useful to apply those same transfer learning techniques in your team's model.
I find it really interesting that you used machine learning to do the diagnosis given the data you had available - scans seem to be naturally similar to images that we use neural networks on today so t he fit seems good. I was wondering though, did you have any ability to dissect the algorithm after it was trained and see what exact characteristics it was looking for?
The reason I ask is because there's some description of the processes we know indicate Alzheimers, but there could be new signs this algorithm has identified that could be applied more directly?
We are, sort of. We've added lithium to drinking water which has been linked to decrease in dementia.
People that suffer from bipolar disorder and are being treated with lithium seems rarely if ever to develop dementia, but the dosages are much much (1000 times, or so) higher than what we add to drinking water.
According to my doctor the primary issue with adding higher concentration of lithium (still, way below therapeutic levels for mental disorders) to drinking water is quacks that make a lot of noise when the topic comes up. They see it as adding mind altering chemicals to water, which couldn't be further from truth.
> We've added lithium to drinking water which has been linked to decrease in dementia.
Who's the "we" here? The facility I worked at adds only the following chemicals in order: sulfuric acid (pH control, since the algal sex orgies in summertime causes pH to shoot way up), poly-aluminum chloride and a long-chained coagulant polymer (coagulants), ozone and sodium hypochlorite (disinfection), sodium hydroxide (more pH control), fluoride (the obvious), ammonium (for keeping the chloride around longer), and zinc orthophosphate (pipe corrosion control). There's also unused tanks for potassium permanganate (raw water control), alum (unused alternative for PACL as a coagulant) and calcium thiosulfate (ozone quench).
There's no lithium in the system at all--I've walked past every tank and pump.
"Lithium can cause nausea, diarrhea, dizziness, muscle weakness, fatigue, and a dazed feeling. These unwanted side effects often improve with continued use. Fine tremor, frequent urination, and thirst can occur and may persist with continued use. Weight gain and swelling from excess fluid can also occur. Lithium can also cause or make skin disorders such as acne, psoriasis, and rashes worse. The amount of lithium in the body must be carefully controlled and is checked by blood tests."
"Lithium can poison a developing baby (fetus) and can increase the risk of birth defects, including heart problems.
Lithium treatment is UNSAFE in women who are breast-feeding. Lithium can enter breast milk and cause unwanted side effects in a nursing infant.
Heart disease: Lithium may cause irregular heart rhythms. This may be a problem, especially for people who have heart disease.
Kidney disease: Lithium is removed from the body by the kidneys. In people with kidney disease, the amount of lithium that is given might need to be reduced.
Surgery: Lithium might change levels of serotonin, a chemical that affects the central nervous system. There is some concern that lithium might interfere with surgical procedures that often involve anesthesia and other drugs that affect the central nervous system. Lithium use should be stopped, with the approval of a healthcare provider, at least two weeks before a scheduled surgery.
Thyroid disease: Lithium might make thyroid problems worse."
Clearly Lithium isn't side-effect free, so it would be pretty reckless to add something like that to drinking water.
Everything you're quoting is at theraputic levels for bipolar disorder, way above those added to water. It occurs naturally in some waters at low levels.
Apparently "low-dose" lithium is not without side-effects either [1]:
"In general, the only significant problems with low-dose lithium are tolerability and thyroid issues. About 1 person in 10 to 15 gets dull, flat, and “blah” (the “lithium made me a zombie” effect, overrepresented in online testimonials). I explain to my patients in advance that if this happens, we’ll give up on it. This adverse effect does not diminish with time and generally persists even if the dose is reduced. Then there’s weight gain: is it dose-related? To my knowledge, this has not been established. I nurture some hope this is so.
That leaves the thyroid issue. Thyroid-stimulating hormone (TSH) levels must be monitored even with low-dose lithium. In women, induction of hypothyroidism is extremely common—and almost predictable in women with a family history of thyroid problems. The latter may be an uncovering of an autoimmune disorder. If your patient has a high-normal TSH value before lithium (eg, 2.5 mIU/L or above, and certainly above 3 mIU/L), she is at even higher risk for lithium-induced hypothyroidism.1
So monitor closely, and even more closely in those at greater risk: for example, every 6 weeks until a trend (up, or flat) is established. Once you have established that the TSH level is not rising, the probability of later hypothyroidism due to lithium is much diminished and you can back off to getting a TSH level with your 6- to 12-month check of creatinine.2"
A therapeutic low dosage would still be considerably higher compared to the levels that has been suggested to add in drinking water.
Normal therapeutic dosage is around 300mg and 500mg lithium per day, and blood levels should be around 0,5-1,2 mEq/L. A concentration at 0,7 mEq/L as quoted in the article is within normal values and dosage, and not really considered low.
The real dangers is NSAIDS or ACE inhibitors which can push your concentration up quite rapidly, but that's easy to avoid, and still not very relevant until you're at around 1,0 mEq/L, give or take. Concentration above 1,6 mEq/L is considered an overdosage and comes with a lot of risks, thyroid being one of them.
Anyone put on a therapeutic dosages of lithium will have their blood checked frequently, often starting with twice a week for an extended period.
But again - in dosages that are 100-1000x times higher than that they want to add in the drinking water.
I hike to a desert oasis in the Mohave Desert of California where a crack in a cliff wall has scalding hot water seeping down the cliff wall, which happens to have a high lithium content. There is a creek the hot water flows into, and people have sand bagged various hot pools to soak. The placed is called "Deep Creek Hot Springs", and it internationally famous. About a decade ago I looked up the lithium content, it is published, and everyone there talks about how calming the hot springs are, lasting for days.
Well, if they’re reducing the incidence of dementia, and you would’ve gotten dementia otherwise... they -did- alter your mind. And that’s a good thing.
People and their fear of chemicals though. Reminds me of http://www.dhmo.org
Lithium is linked to all sorts of behavioral changes though. Who knows what microdosing the entire population would do? I really don't think it's fair to describe opposition to this sort of project as arising from "quacks".
I used to see people rant about 'chemicals in the water changing our behavor' and think insanity. Now I am somewhat relieved I have a private well on a secluded aquifier. Just a byproduct of rural life, overblown I am sure, but it puts the mind at ease when I read this kind of thing. PSA: suppliment floride if on a well, your otc toothpaste is not enough.
Dementia is definitely known to be a brain disease. Perhaps what you're saying is that lithium does not alter the mind? Here is a study that suggests that it reduces aggressive behavior:
>The authors suggest that lithium can have a clinically useful effect upon impulsive aggressive behavior when this behavior is not associated with psychosis.
That's a quote from the abstract. Lithium is a behavior-altering chemical.
Here is another study where lithium is found to alter behavior (although, admittedly, in the brains of children which were identified as misbehaving):
The claim that there could be a non-dietary chemical element that altered the behavior of miscreants without any impact at all on the behavior of regular people would take extraordinary evidence before I'd believe it. Every other psychoactive drug impacts healthy brains if for some reason it is administered.
We don't know how it interacts with the body, but we're fairly certain that it doesn't alter your mind. You can stop taking it and all the symptoms will return.
>You can stop taking it and all the symptoms will return.
It sounds like you're agreeing with me that it alters your mind while you are taking it. I'm confused by the English language use going on here. The plan is to micro-dose entire cities with lithium, which means we are interested in what lithium does when you are on it, because entire cities will be on it for decades of elapsed time. The evidence clearly indicates that the people drinking this water will have their behaviors altered, potentially for their entire lives if they just so happen to stay in cities that medicate their populations in this way.
In fact, the entire point is that people's behaviors will be altered. If lithium was mentally inert, then it would not be useful as a mental health drug, and nobody would be suggesting adding it to the water supply to improve public mental health.
The only real argument against it, and I'll grant you that, that we have is that we don't know how it interacts. It's a good assumption that is has something to do with brain chemistry, but we don't know, or if it's a direct relation or indirect interaction. Or if there even is one.
It is known that lithium alters human behavior. Given that, the "assumption" of it altering brain chemistry is a red herring: because "drug that alters human behavior" and "mind altering drug" are the same thing, with or without theories about brain chemistry. Even in an alternate universe where the mind resided in the spleen, if someone suggested that a drug should be added to the water to improve mental health, it would be tautologically identical to say that they wanted to add a mind-altering drug to the water supply.
Now, to make the argument more longwinded but also more correct, if lithium treated stomach ulcers and the children were misbehaving because of discomfort, it would be impacting human behavior without being a "mental drug." However the evidence does not indicate that the counterfactual situation I proposed is true - in fact, just about every study agrees that lithium is a mental health drug prescribed to treat mental health problems.
Can you define "mind"? It is well known that lithium can be used to treat some mental disorders. It must be doing something to the brain in order to have that effect. In my view, that literally means lithium is mind-altering.
My grandmother died from this, and it is a horrible death. I really fear becoming a victim to this disease. Anything that can help, even a little, is so encouraging.
Anecdotally, and slightly off-topic, I was able to smell when my wife was pregnant before we were able to find out with a test or late period. Not sure if that's relatively normal or not though.
There was a specific case of a woman who realized, after her husband acquired a peculiar smell and was 6 years later diagnosed with Parkinson's, then everyone at a Parkinson's support group had that smell, that she could smell Parkinson's in people well before any clinical tests.
She went to researchers who tested her with sweat from 6 diagnosed patients and 6 controls - she identified the 6 diagnosed patients correctly, and one of the controls who was later diagnosed with it but had not been yet.
(Also, as a fascinating aside which probably isn't what the GP comment meant, _loss_ of sense of smell is one of the earliest symptoms of Alzheimer's or Parkinson's. Why? Good question.)
There's all sorts of anecdotes of pets reacting to people being ill well before the people put 2 and 2 together.
The primary problem, I think, is that you'd have difficulty convincing someone to fund this research without a priori knowledge - e.g. if you don't already know that there are (relatively) readily externally visible biochemistry changes from a degenerative brain problem, why would you see if you can train an animal to smell it?
It's similar to the question people posed after the PS3 signing key leak came out - while it is the case that Sony was signing all PS3 (and PSP, if memory serves) binaries with the same (all-zeroes?) random input, so they leaked information sufficient to eventually retrieve the private key from enough samples, why would you think to check if they did that without already knowing?
(Or, more generally, the large domain of problems that is relatively-trivial to verify a correct solution but infeasible to test all possible solutions in order to find one.)
I don't have references, but my understanding is that the human nose is in general just as sensitive has a dog's. However humans tend to ignore what their nose tells them and so they don't know how to use it as well as the dog's. When a human is trained their become just as good as a dog.
That would point towards some molecule being eliminated from the body after some kind of silent poisoning. That could be a reasonable explanation for parkinson and other diseases, if proven true.
Trying to match the smell with some common products could be very interesting.
Good point. It would be relatively cheap to do a larger study on this and perhaps we can prove some easy facts about Parkinson's, namely that it is correlated with a smell. It would be a cheap easy win towards answering the hard question, "What causes Parkinson's disease?".
I was recently reading "Surely You're Joking, Mr. Feynman", the last chapter contains a story about a researcher who was studying rats in mazes. The researcher wanted to put the rats into a long corridor of identical doors, and teach them to go exactly 3 doors down from where they were put in and get some food. The rats always went to the exact door where they last got food from though, no matter where they where put in. The researcher didn't know how they could tell the doors apart; was it smell, sight, sound, etc? He was determined to eliminate all these possibilities so that the rats would only have their entry point to go off of, and then they would learn to go 3 doors and get food. The final change he made before succeeding was to put the rat maze on sand, and then the rats finally stopped going to the door which last had food. Apparently the rats were able to use the subtle sounds the floor made to determine their exact location in the long corridor.
Then Richard Feynman mentioned that this researcher was largely ignored and unrecognized. He hadn't discovered anything about rats, so much as he had discovered something about experiments on rats. Other researchers continued to do rat maze experiments without putting their mazes on sand, and would publish papers without considering the possibility that the rats were navigating by the sound of the floor.
Coming back to my original point, I wonder if it was proven that Parkinson's is correlated with a smell, would such an odd experiment even be considered by anyone in the research community?
*Computational Statistics (a better name for both, given current methods, since it avoids anthropomorphism and sci-fi implications in uninformed readership).
How do they compare the AI performance to that of radiologists? The abstract says that radiologists aren't as good at spotting subtle, diffuse changes but what metric did they use to compare?
> The data show high doses of radiation may cause cancers. But there are no data to establish a firm link between cancer and doses below about 10,000 mrem (100 mSv – 100 times the NRC limit).
