SignAloud: Gloves That Translate Sign Language into Text and Speech
lemelson.mit.edu
lemelson.mit.edu
And then there's the fact that ASL isn't serialized English (it's actually unrelated to English at all, and has its roots in the French Sign Language). There's no one-to-one mapping between ASL signs and English words, just as there's no one-to-one mapping between Chinese and French, or any two languages. And ASL has grammar features not seen in any spoken language, to complicate things further.
These sensors could perhaps be used to translate other sign languages, but I'm really skeptical.
I have often wondered if it would be possible to create a notation using the simplest and most abstract rendition of an actual SL speaker's signing. Lines and curves that hold the semantic essence of the movement of face, hand and posture. It would take a native speaker to be able to abstract what is truly essential to meaning, and a community to agree on its elegance and clarity.
Off topic, but connected: I discovered from experience that small children can form the SL alphabet with their hands before they acquire the coordination to draw written letters. I found it very useful in teaching my children to read.
We have all sorts of representations for spoken speech---a waveform, spectrogram, vocal tract models, IPA--and these all get used in a speech recognition (or synthesis) system. For ASL we have movies, still-image cartoons that depict motion, categories for some parameters like handshape, some linguistic insights, and I'm not sure what else.
1)ASL isn't just about hand movements, there is a ton of grammar that comes from facial expression. It's not just like English tone, but can actually change a word. Like can become "don't like" from facial expression.
2) Context is very key, there are a lot of words with multiple meanings that matter based on the sentence. It could be figured out from a really smart system but wouldn't be easy.
3) Lastly, ASL isn't English. This is the biggest mistake people make. You aren't simply putting words into symbols. ASL has it's own grammar, syntax and even slang. I remember spending 15 minutes trying to figure out what my ASL teacher was saying when he said he "ate champ burger" and it turns out he meant it was the best burger he ever had. To 'translate' ASL you need to move from symbols to words and then add in the context.
Ultimately there is a reason that professionals in this industry are called "sign language interpreters" and not "sign language translators" ASL is really hard to put into spoken or written language.
It's also worth noting at really high level events an ASL interpreter will translate spoken word into sign and then a "Certified Deaf Interpreter" will re-sign it because the ASL of someone who is has communicated in sign their entire life has far more facial expression and meaning than even professional interpreters who are hearing.
I think it's great to see advancements in technology trying to assist the deaf community, but I think it's important to get early involvement rather than assume we know what people need.
I hope they keep up the great work - it's on the right path.
1) This is an informal shorthand agreed between people who already know each other in the particular context of work, so there is no surprise or ambiguity. Outside this context, I would use, and expect to see the dictionary sign.
2) It functions as an 'in-group' slang, boosting cohesion and good will.
3) There is a certain humorousness in bending a symbol into an informal use. This is more pronounced in cases where the alternate meaning is irreverent or vulgar. I have no intention of expanding on the second point.
4) One does not have to be facing the SL speaker. The 't' symbol can be recognized from beside the person or even behind. Point 5 gives the rationale for this.
5) It can be communicated without changing focus or posture, allowing concentration on the task at hand without breaking the workflow.
6) Though 'thanks' is a brief gesture, 't' can be signed and understood even more quickly. Just a flash of meaning.
7) The sign for 'good', while it is also used to communicate thanks and has advantages 4,5,6 listed above, can be ambiguous, violating the principle of least surprise (see point 1 above). It is more generic, while 't' is specific.
However, I think it's fair to say that a rudimentary system like this one is scripted, and is not tuned to the informal and complicated contexts you describe. You can argue that it's fair game to pre-program the device to emit "Lemelson" for #L, just as you might prepare an interpreter with proper nouns before a gig, but I think it goes against the spirit of the claims made in the video.
Nonetheless, great work by these students. I think their work is likely to have more applications in other areas where mimicking the movement of hands and fingers is an essential component.
I've never heard of a case where a casual user gets RSI.
http://www.youtube.com/watch?v=0QNiZfSsPc0
There are others: https://github.com/melling/ErgonomicNotes#gesture-computing
In the case of the former, you're using the language differently than a typical user. Most people don't produce language in the same amounts or in the same ways that an interpreter will (that is; speaking/signing isn't most people's job). There are steps you can take to reduce RSI as an interpreter, including proper rest while working and supportive braces at night.
For the latter, it's possible to over-produce signs - basically "signing too hard." That can be painful, but it's correctable.
It isn't really because of how much they sign (deaf people don't typically get RSI from sign language), but rather because of how they sign it. The signing is more precise and formal, the hands/arms are elevated more, and generally less "colloquial" than you would see in day-to-day sign language.
But, you're right - the manner of signing has something to do with it as well, and I've amended my comment to reflect that. :)
http://katies.online/katiesblog/index.php/2016/04/27/yes-ive...
These are students, young/new engineers. This is impressive and should be lauded as should all attempts to push what an individual person is capable of. It may not be new or novel to everyone, but it is to them...basically, cut them a bit of slack.
This was 2-3 years ago, so machine learning wasn't advanced (and even now, getting a functional NN running and trained in a weekend isn't feasible) so they only had it recognize a small set of commands, but it could do that.
http://www.washington.edu/news/2016/04/12/uw-undergraduate-t...
SL is an entirely distinct language. People who are fluent in multiple languages switch from linguistic context to another when they speak different languages. They don't translate from their native language. So it makes sense that you don't do this either.