From Reddit:
By real-time I mean "seconds away".
I'm looking for a technology that would allow to watch any youtube video from common languages (eg: english, french, german, spanish) in other common languages. Possibly with seconds delay but that would not be a concern since we can delay the video rendering to sync audio/video.
I'm a nerd and deep into IT and AI but didn't took time to look into that particular field deeply.
Pixel phones already have "interpreter mode" - they will translate audio being spoken between two languages. They do this in real time like an interpreter would.
I imagine the additional challenge for what you're asking for is just the audio mixing. The only added step is that you would need to take the "background noise" and mix the new spoken audio back in.
I went on a trip to Norway this June, 10000km all the way to Nordkapp. I expected at least something to go wrong, and one of my rear brake calipers got stuck and I lost braking, Immediately stopped and somehow stopped right in front of a service.
With the help of my Pixel 7's interpreter mode I managed to talk to the mechanic just closing the shop and he looked at the smoking brake, told me to wait a few minutes for it to cool down and see if it works again, and it did.
At that moment I realized how much more of a hassle that would have been if not for my P7, truly we're at a level where we can travel anywhere and do anything without any help.... As long as there's internet connection lol
Noise cancelling headphones connected to your phone while your phone listens to the video from your pc. It's not pretty but it works
I work on dubbed translation software. Right now, our translation process is theoretically realtime (without lip syncing), taking about a minute per minute of audio. It just has some added setup time per each request that's mostly a scaling issue, and the website's UI is not actually set up to do realtime translation yet (instead it just takes in a video, translates the whole thing, and spits it out).
But realtime dubbed translation is absolutely possible with current tech, there's just some mostly non-ML-related barriers. u/perrochon is right about the legal issues. Realistically, what you'll more likely see soon is YouTubers adding multilingual audio tracks to their videos themselves as YouTube begins to roll out that feature more widely. If you know any YouTubers that might be interested, send them our contact :)
Some languages' grammar is backwards compared to English. So for them instantaneous translation would not be possible.
now. Ai is replacing translators AND interpreters in every legal and administrative space already. Humans hopefully will have literature as a last bastion for a few years still. Source: I’m a translator. The industry is in shambles. The worst aspect of it is not losing you job, but losing respect for your work, clients saying shit like: but the AI said something different when I translated it, and having to explain idioms to an idiot.
YouTube has had machine generation of subtitles and then machine translation of those subtitles for a decade. All it needs after that is text-to-speech which is not hard. Translation is much quicker than the necessary transcoding of the video stream.
The reasons it's not there on YouTube are manifold, and include product (many people hate dubbing and want subtitles) and legal (you are making another audio track with the copyrighted sounds, you are wholesale using someone else's video to make a new video, the owner of the original content may not like the auto generated new sound track, etc.)
I have this today on my Google pixel. Love it. I live half time US and other half Thailand
I don't think you need AI for this. Firefox is already beta testing auto website translation and you've been able to copy and paste text into google translate for years. If you're into to this and want to create it you just need a three step program.
Audio transcription
Text translation
Text to speech
That's it. If you're looking to incorporate AI then maybe something that samples the speakers voice on the fly and reuses it in step 3 would provide for a babblefish like experience rather than an automatic voiceover.
While you can do this without AI, AI can potentially offer some benefit. Primarily, it can learn to understand the context of what's being discussed and translate meaning more so than just words.
Translating words for words is one thing, but when a certain phrase has an unconventional meaning in one language when used in a specific context, and you translate the words to a different language, that meaning can be lost. Being able to use AI to understand those nuances, and hold context on an ongoing conversation, can potentially make the real-time translation much more meaningful (by translating the underlying message, not just the words).
You should be able to set that up with OpenAI's "realtime" version of the 4o model.
It costs like 20 cents a minute though...
Google translate does it on my pixel phone.
I was at the planetarium on Cozumel, and watched their video presentation with live subtitles courtesy of my phone. It wasn't perfect but good enough to follow what was going on.
I'm a professional interpreter, you won't be getting a service from AI like you can from me any time soon. Interpreting is human activity for humans and AI often confuses natural glitches in conversations as ends of sentences to name just one of its fundamental shortcomings.
I really dislike the current adoption of AI. Technically it works. But the quality is barely understandable most of the time.
But it seems this is good enough for a lot of people. Which is scary when people in charge see it as a worthwhile costcutting option.
The AI of tomorrow will be vastly different from the AI of today
Will you take my call whenever I feel like I need a translation for $20 dollars a month?
Do you know English, Italian, Korean, Japanese, Chinese, Portuguese, Spanish, German, Russian and Arabic?
There are actually some apps that do a pretty damn okay job at that already...They're getting better very rapidly...
It honestly wouldn't surprise me if you could get almost real time translation through an ear bud from your phone or other device within the next decade at the rate we're moving...
Assuming society doesn't implode before then...lol
Now it's just expensive and requires Internet connection
There are fundamental linguistic challenges to actual real time challenges. Take this German sentence from an article in Der Spiegel:
In den aktuellen Gesprächen werde es darum gehen müssen, die Notwendigkeit eines von der Regierung in Gänze getragenen wirtschaftspolitischen Konzeptes zu betonen.
google translate renders it as
The current talks will have to focus on emphasizing the need for an economic policy concept that is fully supported by the government.
In the English, “emphazing” is the 9th word. In the German, it comes from “zu betonen“, right at the end. You could not construct the full English translation until you had the full German sentence, right to the end. Fundamentally the structure of the languages are different in this way. This means the best you can hope for with on the fly translation is for a sentence to be translated after it is fully complete.
This exists!! I learned about this app a few months ago from someone on Reddit and tried it during a conference call with people who were speaking a different language and it worked (mostly) flawlessly. I fully understood the conversation in real time.
The app is called 3P0 and from what I understand it's from an independent dev who frequents Reddit. Pretty cool tech, honestly.
Never, because you don’t fully know what the person actually said until you hear the full sentence.
I've noticed there's increased interactions in certain apps between users of different languages thanks to quick translate buttons.
I think it's a positive thing that I have faith it.
Samsung phones do it with the Samsung earbuds. In real time.
Even if the computer was hyper advanced, and the algorithms have been perfectly trained.
It would never be possible because different languages have different sentence structures. Japanese sentence structure is nearly reverse of English, for instance.
So even a perfect translator would have to wait for them to finish their sentence before it could spit it out for you.
Hearing aids that can translate foreign languages - Really!
Originally introduced to the world in 2018, Livio AI hearing aids significantly altered how many individuals use and think of hearing aids. And why is that? Because Livio AI was the first multi-purpose hearing device that not only sounds better than other hearing aids out there, it also lets you track your brain and body health, stream music, phone calls and more from your smartphone, and translate languages as you hear them.
#translation #language #hearingaids #technology
How does the translation tool work?
Select the language you speak and then choose the language of the person you plan to engage in a conversation with.*
When you speak into your iPhone®, the Thrive Hearing app translates your speech and displays it on the screen in the other person’s language. Simply then show them the screen so they can see what you’re saying.
Alternatively, when the other person speaks into your phone, the app will translate their speech, display it in your language on the phone, and also stream the translated text to your hearing aids in your language.
Hi, @taskmaster4450le,
This post has been voted on by @darkcloaks because you are an active member of the Darkcloaks gaming community.
Get started with Darkcloaks today, and follow us on Inleo for the latest updates.
Now, this same great technology has been updated to create an even better sounding hearing aid - Evolv AI. These outstanding hearing aids have the ability to translate 27 languages as you hear them! It’s easier to communicate with individuals who speak a different language - simply by using the translation tool in the Thrive Hearing app.
How Will Real-Time Translation Shape the Future of Language Learning?
With fast advancements in real-time translation and AI tools like video dubbing, I wonder if language learning will become less popular in the future.
Learning a new language takes a lot of time and effort. In 10-20 years, as technology reduces language barriers, will fewer people find it worth the investment? Could fluency in foreign languages become rarer as a result?
People who learn a language because they genuinely want to will likely continue to do so, but people who learn a language because they have to probably won't do it.
I'm curious to hear your thoughts. This is just a thought experiment—I believe language learning has many unique benefits that technology can't replace!
Old school analog face to face communication will always be welcome over the people who rely on the machines.
It'll be like Star Trek where the machine isn't even visible during a face to face conversation. People will just have their headphones popped in listening to the live translation. People already walk around with headphones in all the time
I can't imagine "real time" translation will truly happen, there will always be a delay of some sort.
(To clarify: when I think "real time translation", I'm thinking, like, an earpiece you wear that translates what somebody is saying as they're saying it, giving the impression that they are speaking your language)
Sentence structure varies so much between languages that this type of translation simply isn't feasible. For example: type a moderately complex sentance into DeepL (example) and watch how the translated sentence transforms wildly with nearly every new word.
For instance, the sentance "he has been taking care of his grandmother for twenty dollars an hour".
"he has" -> "tiene", as in has "he possesses"
"he has been" -> "ha sido". as in "he has been [a doctor]"
"he has been taking" -> "ha estado tomando" as in "he has been taking [medicine]"
etc etc.
Yeah, a real time would be hard with changes in syntax, not only things like noun/verb order would demand a delay, but bigger issues of syntax like SVO, SOV etc. It could get close to instantaneous, but never quite the speed of someone who just speaks the language.
Meaning gets lost in translation, anyone who speaks more than one language knows this. French poems translated to English don’t feel the same, English literature translated to French doesn’t feel the same.
People learn languages to feel other cultures, not simply understand them.
I don't think it will change much of anything. Unless we can forgo language entirely and just skip straight to transmitting direct meaning. THAT will be revolutionary in many many many ways. For good AND for bad....
It is just my opinion, but I don't think computer programs will ever be as good as humans. I spent an entire career as a software programs, so I know how programs work.
Programs are created by humans. Human language experts work out the grammar of a language, and they work with programmers who turn all the rules into numbers. No matter how smart the humans are, they are creating a set of rules in advance. The computer has zero intelligence. It can only run the rules (after they are turned into numbers) that the humans create. So nobody intelligent sees the actual sentence you are translating. A computer program is just following a large set of rules created by humans in the past.
For translation, you have the grammars of different languages. Only a human who is fluent in both languages can create the set of rules. And how many rules are needed? The good thing is that many different human experts can all contribute to the same set of rules. Once the rules are written down, they won't be forgotten. A good translation program is probably many man-years of work.
The real issue is this: can a set of rules translate every sentence correctly? Or are languages too complicated?
Don't worry. Won't be advanced enough in at least a couple decades. No need to stop studying.
And even then, literal translations will lose much of the nuance and cultural differences.
If I know how to swim, I don’t need a swimming ring. If I love swimming, it doesn’t matter whether there are swimming rings.
Good "AI" translation is straight-up impossible because 1:1 translation is impossible, things can be conveyed in one language that there's no easy way to express in another and while there's ways for creative translators to still convey those ideas, it's something that "AI" simply can't do. "AI" translations are going to create more language learners if they catch on at all, because "AI" translations are so shallow and low quality (if not outright incomprehensible) that they force you to learn the language if you want to get the original point at all.
are people forgetting that actual humans, communities, and societies speak the languages that you learn ????? why learn a language for translation purposes and ONLY use it for translation purposes??? whatever happened to actually TALKING TO PEOPLE ....... what the hell
The thing that OP is referring to would be instant translation that DOES allow you to talk to people seemlessly and as easily as if you were talking to them directly.
I love learning languages and really hope AI doesn’t replace human translation, but I’m sure it will.
All these folks mentioning the nuances of languages and the impossibility of machines grasping the intricacies of human communication don’t seem recognise the sheer power and the barely imaginable future abilities of computers, I believe.
Hope I’m dead long before it happens, but I reckon AI’s coming for all you translators’ jobs!!
Already because of how prevalent English has become, and because of online translation tools, there is less of a need for translators.
The universities in my country have abandoned some of the smaller language choices and reduced the number of students they accept.
I do find it worrying. Although you might be able to communicate with all these people, then especially politically, it is important to be able to understand what is going on in countries in their own language. If you get it all in a filtered way, you are going to lose out on details and attitudes that might be important.
I think real-time translation will always be a bit laborious and won't really replace learning language learning
I think that needing to be comfortable in the written version of the language is already much less important. Translation and generative AI can already do the writing for you, good enough for 90% of situations.
But, I do think speaking is something that will still be an important skill to learn.
Imagine you're at a business networking event and the guy next to you has learnt English and can speak it himself, while you're trying to rely on some real-time translator tech. The other guy will have the edge.
Imagine going on a date or having a romantic relationship with someone where you only communicate by translation tech. It just doesn't seem the same.
I don’t think it’s gonna replace language learning soon but god I wish it’d happen soon to replace my language learning. Absolutely hate it, but have to. Real time face to face translation would also be great in eliminating code switching so I don’t have to bother to speak my native language ever again, that’d be a massive improvement in quality of life as well.
But yeah, I agree that language learning purely as a tool might get rarer, just like people don’t ride horses around as a means of transportation that commonly anymore; but riding a horse for fun and even for transportation too didn’t get replaced by the invention and proliferation of cars.