The headline is wildly inaccurate, and the article papers over huge problems. They did not build a translation device.
For one thing, ASL has certain morphemes that are expressed through use of external objects, facial expressions, etc. Things other than hands.
Another, bigger problem is that ASL is not English. You can't just take a word, translate it into a different language and assume you have a valid sentence in the target language.
There is a very detailed explanation written in this letter [1], by the linguistics department of UW. That is in contrast to the press office of UW. Among other things: [...] Deaf signers who have viewed the demonstration video of SignAloud do not recognize what is being demonstrated as ASL: [...]
Unfortunately (IMHO) this letter[1] got mixed with a lot of politics in it, but the technical aspect is solid.
"Unfortunately (IMHO) this letter[1] got mixed with a lot of politics in it, but the technical aspect is solid."
Totally agree, question is why didn't the Linguistics Department finish the letter after that section, i.e. why the convoluted discussions about audism, appropriation and such? Not having a deaf person on your team or checking the results with the Deaf Community is certainly to be criticized with respect to the naivete of developing an accurate translation system but the letter treats this as a colonial assault.
If I want to develop my own French to English translation system (I don't know anything about French) should this be interpreted as my (perhaps) subconscious feelings of superiority? Evidently so.
If you built a system that translated word-by-word from a pidgin French into English, and then tried to sell it as a tool that would unlock "the fundamental right of communication" for those poor French people, then yeah, I think you should expect some push-back.
Sure, but if the linguistics department wrote a response which spent half a page explaining why the app didn't actually interpret French very well and might have benefited from a native speaker on the team and devoted considerably more space to attacking the creators' motivation with specious drivel about how it was "cultural appropriation" to take their language "without permission", I'd think they were the ones displaying the arrogant colonialist mentality.
Well, I don't agree but that's probably because I don't think "cultural appropriation" is an inherently specious concept, as you seem to.
Whatever else you might think about that letter, I thought the authors made it very explicit that they were not attacking the SignAloud creators' motivation.
I realise the letter starts with a disclaimer about not attacking the creators, who seem to have been browbeaten into actually endorsing it, but it also spends two pages bashing them with phrases like "a feather in the cap—for members of the non-marginalized culture—hearing people", "reaping the rewards...of cultural appropriation" and "this is exploitation" which clearly apply to them more than to the PR office; internal consistency is not one of its strongpoints.
FWIW, I don't think the concept of cultural appropriation is inherently specious, but I do think many if not most accusations of cultural appropriation are, and suggesting that the idea to demo a tech product as a potential translation gadget is "exploitation" is stupid and counterproductive beyond belief. I mean, if a student at this university in future has a choice between pursuing a neat idea for an app they think might help deaf people or a different project, which way is this letter going to nudge their decision? They could have made the same point about the disadvantages of not engaging the deaf community with the project at its inception and the resulting practical shortcomings by structuring the letter as an offer of help...
The key phrase in that letter is "where is the deaf person on the team?"
It's arrogant to take someone else's culture and redesign/"enhance" it, particularly if members of that culture are impaired or disadvantaged. Privilege creates blind spots, and the most invidious kind of privilege is the kind that you don't realize you have.
The typographer Adrian Frutiger notoriously designed a "universalized" version of Devanagari script in the 60s or 70s. Not having the necessary insight into the history of the letterforms, he made an embarrassing mess of it. I don't think anyone really feels that this made him a bad person, but it's indicative of the tendency to march in and appropriate cultural artifacts in an insensitive way.
French is a particularly poor example to choose. ASL has been continuously delegitimized as a language. Up until the 60s no one thought ASL was a "real" language, or a language as rich as English, because no one was studying the grammar of ASL. And because ASL was seen as broken English, pantomime, or otherwise deficient, ASL was actively out of the reach of deaf people. Throughout history, deaf kids were punished for using sign language, and their hands were cuffed so they could not sign.
French, on the other hand, has experience elevated status (perhaps not so much now that English is most often the lingua Franca, but even the term...).
If someone comes out and suggests that French is simple, or basically a re-encoding of English, that will sound absurd but more importantly it will not be associated with a history of language oppression behind it.
This project has been bothering me for (what feels like) months and I am glad to see there was an official response by the linguistics department of UW. How'd you find it?
By politics, do you mean that the letter mentions the social issues surrounding this project/its advertisement? Why did you think it's unfortunate?
One of my friends helped me see that one of the great practical applications of linguistics is to provide justice and value to minority languages and dialects.
My girlfriend works there, we discussed it at length.
>Why did you think it's unfortunate?
Primarily because they botched it. I wish it was well-enough written so that I could use to convince the people who actually need serious convincing (unlike the PR office or the two kids). Alas it's far too inflammatory, and on so many levels. It's also mushed together in a way that thinking back I can't separate one argument from another. If they separated politics from the technical aspects at least I could use the latter part, but as it is, it's pretty much hopeless.
So now if I am to convince anyone either in the need for scientific rigor or in the need to pay attention to the plights of the Deaf I am entirely on my own, using my own clumsy word. This is not how enlightenment is supposed to work. The most educated and knowledgeable, those with the most authority, those who can write well, or should be able to, they should be providing ideological ammunition to those who are willing to spread the message. Instead they ended up publicly flogging the PR office, which might have been misguided but is fundamentally on "our" side, in a fit of anger. A squandered opportunity.
Another challenge is that there is very little corpus data available for ASL (for one thing, there's no natural orthography so notation systems can vary wildly between studies and have only started coalescing recently). So you really can't build a statistical MT system the likes of Google Translate because there simply isn't the raw data to train it on.
That letter spends a large portion on detailing how the gloves are not capable of two way communication but it seems like the path from speaker -> deaf could be tackled using supplementary technology. One such method (ignoring problems with real-time language processing) would be a speech to text displayed on a HUD (think google glass).
Perhaps their criticism against the marketing of the project still stands as it is definitely not a panacea for the problem of communication between ASL and English.
They also bring up the problem of hand motions not being enough to fully express the ASL statement in English, however one could imagine a dialect of ASL that could utilize alternate syntax that would allow for better compatibility with English. As with the other criticism, it seems the gloves are merely the first piece of the puzzle but certainly not the full solution.
So basically they didn't understand what was being signalled? If they did and it would actually take a new "bridge sign landuage" that would be easy to translate and the device would be easily accessible (cheap and open source), then I guess it would be worth it.
Fun fact: In Poland, there are two sign languages used: Polish Sign Language ("Polski Język Migowy", PJM) [1] and Signed Polish (język migany) or System Językowo-Migowy (SJM) [2].
This letter seems absurdly critical of a promising, if incomplete, technology. Really, it's like criticizing babelfish back in the day because it can't convey the tone of voice used. The gloves are really quite an achievement.
As a Sign student I've commented on this project a few times as I wanted to give some relevant examples.
As others pointed out ASL is not simply English replaced with symbols but is its own nuanced language. Many Sign speakers learn to read and write English as well and so can use a bridge called PSE (pidgin sign english) which is commonly used in the workplace, but it is certainly not ASL.
In ASL facial expression not only accounts for ones grammar but can signifigantly change the meaning of the sentence, for example:
-I pay him on a regular basis
-I always pay him diligently
-I always pay with ease
-I always manage to pay
These four sentences have very different meaning but can all be made with the exact same ASL signs and only a different facial expression.
Beyond that the grammar, syntax and even word choice of ASL is not the same as English. For an example, just the other day my ASL teacher who is deaf came away with the sentence:
"I was swimming there (pointed at building down the street), then I drove to the food court at the mall and ate the best burger."
There are a lot of things we don't use in English such as directional pointers, or the fact that he finger-spelled CM as the initials of our local mall to clarify what he meant. Then there are different meanings such as using "finish" to list of a series of events, or "champ" to represent something is the best he's had.
If you take a pair of gloves and stick them on an ASL speaker and simply spit out English with no understanding of the language and no note of facial expression - you're going to have a bad time. And you are likely unable to easily build a system to translate ASL syntax to English as there isn't an easy to use large corpus of samples.
Lastly, in the article the founders are saying how communication is a fundamental human right and this is supposed to empower the ASL community. The ASL community is a vibrant community, who can communicate (even surprisingly well with non-ASL speakers). You'll certainly face a lot of backlash if you suggest they are disadvantaged or lack a voice. Many deaf individuals will be sure to tell you that they are not disadvantaged they just communicate differently.
All in all, it's an interesting technology but I think if you move forward with a technology like this for people to use it needs to be with the involvement of that people and an understanding of their needs, culture and language. For example, using this as input device for a computer may be a more natural use case for a lot of ASL users.
For one thing, ASL has certain morphemes that are expressed through use of external objects, facial expressions, etc. Things other than hands.
Another, bigger problem is that ASL is not English. You can't just take a word, translate it into a different language and assume you have a valid sentence in the target language.
There is a very detailed explanation written in this letter [1], by the linguistics department of UW. That is in contrast to the press office of UW. Among other things: [...] Deaf signers who have viewed the demonstration video of SignAloud do not recognize what is being demonstrated as ASL: [...]
Unfortunately (IMHO) this letter[1] got mixed with a lot of politics in it, but the technical aspect is solid.
[1] https://catalyst.uw.edu/workspace/file/download/5c706e255e89...