It is essentially on-the-fly converting of your words to a different language, and with your own voice to boot. It is not perfect yet, and gets one in eight words wrong on average, but just think about never having to learn a foreign language, and still practice your best sarcastic remarks on the locals when they serve you food that still looks like it's moving.
The software has to be provided first with hours of recordings, so as it can analyze the subtleties of your vocal cords, and then learn to speak Cantonese with them, just like the first voice recognition efforts required you to talk to the computer for quite a while until it gets your brand of crazy accent. Rick Rashid, the head of Microsoft Research who demonstrated the technology, clarified how this became possible:
Just over two years ago, researchers at Microsoft Research and the University of Toronto made another breakthrough. By using a technique called Deep Neural Networks, which is patterned after human brain behavior, researchers were able to train more discriminative and better speech recognizers than previous methods.
[...] We have been able to reduce the word error rate for speech by over 30% compared to previous methods. This means that rather than having one word in 4 or 5 incorrect, now the error rate is one word in 7 or 8.
[...] We have been able to reduce the word error rate for speech by over 30% compared to previous methods. This means that rather than having one word in 4 or 5 incorrect, now the error rate is one word in 7 or 8.
If anyone is still wondering why do we need desktop-class processors in our phones and tablets, it's features like these that will evidently require all the horsepower your Windows Phone can muster, and we hope this direct translation tech will arrive soon to our handsets. The fun starts about 5:30 into the video below, but it is worth watching in its entirety.
via TNW
