>Chang’s team asked Bravo-1 to imagine saying one of 50 common words nearly 10,000 times
I dropped out of a PhD back in 2018 all about the application of Machine Learning to Neurosignal Decoding, and that quote just perfectly sums up everything wrong with the state-of-the-art approach.
A bizarre prompt, an infeasibly boring task, and they somehow magically expect a machine learning to smooth everything over despite the fact that the subject isn't even generating a consistent signal to start with.
I would imagine that after about the 300th word, they'd start to lose all meaning and their associated signals would collapse much closer together. It definitely does feel like an extremely optimistic experiment. Do you think there's a good way to capture enough useful data in a lab environment, short neural implants + microphones with 24/7 recording?
>Do you think there's a good way to capture enough useful data in a lab environment, short neural implants + microphones with 24/7 recording?
It's a hard problem, even neutral implants and mics isn't enough. I had two months to come up with some ideas about how we could get a human to produce reliable signals, and failed completely. That was when I walked away and got a job.
I know nothing about neuroscience and sounded like a recipe for failure, you obviously have to use true communications, like asking the test subject between food choices and actually bringing the food requested, so the next time they are asked about food choices they know is a real interaction and have to make a real decision, the same for everything else that is tested, meaning make it as real as physically possible.
You are missing the point, they are not decoding semantics. The signals being decoded correspond to imagined motor activations needed to produce the word. It's similar to asking you to mentally rehearse throwing a ball.
I don't think it is anything like simulating a mechanical action like throwing a ball, just saying a word alters brains in completely different ways depending on the person, for example saying "spider" reminds me of spiderman and there is little I can to stop such through from happening, to someone else it may remind them of something completely different, even my own thoughts about that word may be different any other day.
To actually capture words you have much better chances reading the brain while writing the word with a pen because you are actually sending a signal from the brain to your hand, which is what they did here (even if he doesn't actually have a hand to move, the brain still can emit the very same commands): https://www.cnet.com/google-amp/news/brain-implants-let-para...
>I don't think it is anything like simulating a mechanical action like throwing a ball, just saying a word alters brains in completely different ways depending on the person
It's exactly like imagining throwing a ball. A disproportionately large amount of the motor cortex is used for facial muscle and tongue control. Look up the "cortical homunculus".
>for example saying "spider" reminds me of spiderman and there is little I can to stop such through from happening
These BCIs can't tell whether or not you're thinking about Spider-Man, otherwise they'd be used on terrorists to get information. Instead, they're mainly focused on broad fitting synchronisation of M1 neurons (indicating rest)
>To actually capture words you have much better chances reading the brain while writing the word with a pen because you are actually sending a signal from the brain to your hand
Real movement does produce much more consistent results than imagined movements, but it doesn't translate well to the target market for BCIs (people with severe motor disabilities)
>even if he doesn't actually have a hand to move, the brain still can emit the very same commands
It doesn't work like that in real life. With no feedback, we're back to square one with the "imagined movement".
Sure, a word can evoke all sorts of meaning, other brain process, etc. The decoding principle leveraged here is exactly as I indicated. It's decoding motor activation associated with imagining speaking a word, from the speech-motor cortex.
Given the diversity of pronunciation even within one local group, and that everyone learns to make the sounds their own mechanical way, how does this generalize between individuals?
Maybe just like everything in the history of mankind it is a material reality and we just dont understand it yet, just like we didn't understood how to fly for millions of years, maybe it will take a few more to read a brain.
Just like people used to believe the sun was a god or something magical for centuries but now we know it's a mass of gas under nuclear fusion. In general believing that something we don't understand happens due magic (or however you wanna call non-material reality) has a bad track record.
Use the phrase 'god' or 'something magical' as a synonym for 'something beyond our understanding and control' and you'll realize we're a lot closer to that level of understanding than we are to having figured everything out.
Abstraction makes it much easier to reason about complex systems, such as the world :)
Don't you have knowledge of immaterial realities, like the lines, points, and angles of geometry? How is that possible if all you've got to think with is matter?
The machinations of the brain are as immaterial as the bytes of a hard drive; which technically speaking are both material given that the information there is an electrical property of matter.
On the contrary - the more we focus on technology and treat everything as if it's a machine, the less we understand the true nature of the human mind. That's why the general trend in literature, art and architecture is down instead of up.
Misleading title, it’s more of a delay according to Facebook’s quote in the article:
“While we still believe in the long-term potential of head-mounted optical [brain-computer interface] technologies, we’ve decided to focus our immediate efforts on a different neural interface approach that has a nearer-term path to market,” the company said.
I don’t think it’s misleading, they have removed funding and suspended work.
Could they start again in the future? Sure, but in practical terms removing funding and stopping people from working on a project indefinitely is the same as cancelling. This is just corporate-speak, which requires translation - if removing funding, staff, research space and equipment and stopping all work indefinitely isn’t cancelling something, what extra steps are required for cancelling something?
The different neural approach is reading signals from muscles in your arms, so it’s definitely not the brain-speech interface they were talking about, but the fact they say it’s closer to market will placate investors.
I was on the list for a SDK at CTRL-Labs before FB swooped in and bought them. I'm really bummed I couldn't get ahold of one first.
In having on-again, off-again RSI issues over the years from coding, I think this tech would be hugely beneficial and is more realistic to ship in the near future.
I also have RSI and it stops me from being a productive programmer, playing any sort of video game, and completely ruined my life for a couple years. If I get really good sleep I can use the computer for a couple hours without any pain at all but normally I'm relegated to only using my hands for a little bit each day otherwise I'll injure them. I tried using voice to text but then I injured my vocal chords using it. Physical therapy and Rehabilitation is slow and I doubt I'll ever be able to have the full use of my voice and hands again. Having a BCI would improve my quality of life immensely.
Have you tried strength training? Do you use a tenting split keyboard and vertical mouse? Do you have a good chair, decent posture, and a desk at the proper height? Those things solved my RSI.
i would guess that anything that works for coding/computing use cases is a long ways off. like speech recognition, these things rely heavily on language models...
i suppose we have language models for code now, but i suspect it would still be frustrating- at least in the short term.
Wrt to the wristband, the concept and hardware is simple enough that were it not for Facebook nightmare lawyers and patents it could be a cheap maker community staple. Under Facebook, it will fail - nobody wants Facebook tracking literally everything they type.
Huh? This technology (using electromyography to detect hand/arm movements) has been around for at least 6 years (https://www.pcmag.com/reviews/myo-gesture-control-armband) but has not become a "community staple". The company producing the armband actually shelved the product because it was so bad and focused their efforts on AR glasses instead.
This is a really hard problem. NIRS is already difficult; very weak signal, limited to <2 cm of cortical surface, and haemodynamic response is 5-7 seconds.
I think it's definitely a useful imaging technique for certain (highly specific) tasks. But to detect language? They would need many, many more source/detector pairs, considering the vast swathes of the brain partially responsible. Language is already very fuzzy within fMRI.
I guess there is also the possibility that they just changed their mind and didn't want it to be usable consumer technology at this time? Maybe i am being overly suspicious, but i guess it seems to be at least a little strange that DARPA would put someone on the task who says "it's closer than you realize" and then do a complete 180.
Also, i don't really follow the tech, but i thought i remembered seeing (on HN maybe?) positive reports of actually useful BCI prototypes. Maybe those were overblown or limited, though.
I studied a PhD in this field (dropped out after one year).
There are useful prototypes out there ("useful" as in someone with motor neurone disease moving a computer cursor on a text to speech machine), but all of them use invasive, penetrative electrodes that fail after a matter of months as the body rejects them.
The less invasive prototypes have seen no real breakthroughs in the last decade and a half. Only a small percentage of people can use them with any real accuracy, nobody knows why and frankly nobody's doing the research on it.
All the grant money goes to people throwing random ML models at standard datasets and patting themselves on the back when it gets a 2% more accurate fit on one (and only one) set.
Couple this with the fact that technology relying on other biosignals is yielding amazing progress (mainly the translation of voluntary muscle/tendon movement into control signals for a computer, like what Stephen Hawking had but more recently prosthetic hands with dinner and finger control), I suspect that most of the research into BCIs will be put on ice until we have a more complete understanding of how the brain works or can no longer make progress with other control technologies.
My take is that Facebook is being a lot more realistic about this than Neurallink is.
Nothing wrong with being lofty, but with the money and manpower that FB has put behind BCI, versus the talent flight from Neurallink – FB will likely be the winner.
A shame, because I don't think anyone else can even come close to really stacking up.
It depends what you mean. They chose very different approaches. Neuralink is much more technically grounded than attempting high bandwidth optical non-invasive decode, and obviously requires implantation.
Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
METHODS
We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke.
Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words.
We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them.
RESULTS
We decoded sentences from the participant’s cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%.
In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period.
CONCLUSIONS
In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model.
(Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149. opens in new tab.)
“ The company says it now plans to open-source the software it developed for brain decoding and also provide access to prototype devices, so other researchers can benefit from its work”
Anyone found this repo ? I would love to see some of their pipeline’s parameters. They ‘should’ have a fine-tuned processing pipeline that’s interesting for different kind of bio-signal sensors.
There is something similar to this: `Spotify granted patent to use mic to infer emotional state, age, gender, and accent`[0]. Not surprising, given how data hungry most tech companies are now. Without data, they are dead. But how much is really needed to serve ADs? Do we really need to be doing creepy things like 'sentiment analysis' or peering inside the brain? How much access is too much access? I think we are slowly learning the threshold piece by piece.
We arnt slowly learning the threshold. It's known, and we passed it a long time ago. People turn off tracking when given the choice and told what is being tracked already. Much less some future peering inside the brain.
Interestingly, I interviewed at one of the many startups who wants to implant thousands of sensors directly into the brain. The only issue is they had the same problem that MIT Lincoln Labs has: they're snobs about hiring only PhD EEs, creating an ideological monoculture.
Maybe when putting things into peoples brains, competence is a highly necessary requirement. It's not enough to have "liked coding since high school" and "read a bunch of blogs" and "taught myself javascript" which may suffice for the average web shop, but when opening skulls, a broad and deep education definitely helps. Those crying about gatekeeping tend to be firmly on one side of the gate without putting in the necessary sweat.
The comment was not about heterogeneous or not. It was about Ph.D. or not. Everything else equal, I'd prefer a heterogeneous bunch of Ph.D.s to have designed and built the thing that's put inside my brain over a similarly heterogeneous bunch of "I've taught this myself in 1 year using youtube". Snobs or not.
Ph.D.s are trained in two things. "Reasoning about an unknown" is one of them. But the particular niche knowledge they acquired while training this is the other. That niche can save my life if the niche matches the thing to be implanted into my brain. No matter how many youtube-educated wannabe-experts take issue with formal education.
(I have many issues with the academic system, but this kind of critisism is ridiculous.)
Would you rather have a team of 10 PhDs and 3 "self taught hackers" or 13 PhDs?
It might just be me, but I actually feel more comfortable with the former.
If you ONLY hire people from specific backgrounds, you get blind spots. That's dangerous. Excluding anyone without a PhD should not be expected to improve safety.
That makes it sound like Ph.D. graduates are a homogenuous bunch. That's nonsense. Somebody with a Ph.D. degree is just that -- a person with a certain degree certifying a certain competence. Apart from that, it does not say anything. Not what background the person has, how they think, what they like. It's part of a Ph.D. degree to aquire knowledge and skills alone. "Self-taught" if you will.
The distinction you make does not exist in that sense. The only actual difference is that the "extra" 3 Ph.D. graduates have had a certification for a somewhat structured education in some research field, while the youtoube fans don't. So, everything else equal, that group has an advantage.
Interesting - out of curiosity a few years ago, I glanced at neuralink's hiring site and was surprised to see the same. I'm sure these folks are very qualified at what they do, but if we're hoping to put these devices in humans, at some point all of the models and "hard" data will have to be matched up to something messier - disease characteristics, quality of life, etc. How many PhD EE's does it take to come up with an operational definition of things like "Depression" or "Multiple Sclerosis symptoms" so you can just "turn this activity down"? Good luck...
I dropped out of a PhD back in 2018 all about the application of Machine Learning to Neurosignal Decoding, and that quote just perfectly sums up everything wrong with the state-of-the-art approach.
A bizarre prompt, an infeasibly boring task, and they somehow magically expect a machine learning to smooth everything over despite the fact that the subject isn't even generating a consistent signal to start with.