subtitles.png

Not everyone loves subtitles, but it can make TV really accessible for deaf people. A new research is looking into a way to give people with hearing difficulties the option to subtitle their everyday lives using crowdsourced transcribers. This is interesting!

Researchers from the University of Rochester have developed an app which lets deaf users read subtitles that correspond to their happenings around them. The app is called Scribe, and it beams an audio track from the user's phone to a central server.

The system then recruits workers from Amazon's Mechanical Turk crowdsourcing service. Workers will listen to the audio stream from the user's phone and is asked to transcribe what they hear. New Scientists:
All workers hear the full audio stream but the volume of different sections is raised and lowered, encouraging each person to focus on transcribing a particular part. Scribe then combines the partial transcriptions with software normally used to align evolutionarily related sequences of DNA. Bigham modified the software to account for common typos based on the layout of a keyboard - for example, if someone types "fqll", it is more likely they mean "fall" than "fill", because "a" is nearer to "q" than "i" is. The software then chooses the words that a majority of the workers have typed.
Scribe is 74 percent accurate compared to 88.5 percent from a trained stenographer—but at a fraction of the cost. The app is still in development, but the team behind it hope to have it in beta version ready for release.

[University of Rochester via New Scientist]