Field Report · October 29, 2025
Please Stop Using Grok
Putting Grok's video translations to the test
I built @TranslateMom in May 2023, before X rolled out Grok - it's a simple bot that translates videos on the platform to a user's desired language by being tagged (e.g., "@TranslateMom English"). It started as a simple Whisper + Google Translate loop and has grown into a much more elaborate system and my own $2k+ MRR side project with over 100k active users on X.
Grok was rolled onto X in March 2025, and has had quite the bumpy road, including spewing out racist comments and calling itself mecha hitler. Despite that, the bot has now become a staple on the platform, with over 61.5 million posts (mostly replies) at the time of this writing
As Grok got smarter, TranslateMom mentions dropped slightly, sometimes replaced with a desperate "@grok is this true?"
So I ran a little experiment: every time someone tagged Grok to translate a video, TranslateMom quietly replied with its own answer too. The goal was to make an objective comparison.
Methodology
Experiment parameters
- Grok Attack. Check for @grok mentions and anything that includes 'translate' or 'english' or looks vaguely like a translation request
- Observation window. Sample clips were collected between October 27 and October 29, 2025. This article might be outdated as @grok and @TranslateMom both may have improved since the time writing this.
- Source Transcript By default, TranslateMom will also provide the source transcript which was used for the translation, but Grok does not do so unless you ask explicitly. Most users don't, and it was reflected in this experiment too.
Results
French Example
Kris Letang’s farewell message in french
Weird behaviour where Grok refuses the request outright, which can be considered a positive since it does not hallucinate an invalid response. Not very helpful for the user, however!
TranslateMom
TranslateMom
View transcript
I hope we’ll have fun one last time, it’s the way everyone wants to see you retire, so for me it will be a special moment. You’ve been someone who helped me enormously in my career, so I wish you a lovely evening and we hope we have a great time together one last time. Thank you.
Grok
View response
Sorry, the video doesn't have built-in subtitles, and I couldn't extract the exact French audio transcript from the clip. If you can provide the spoken French text, I'll translate it to English for you!
So why is Grok bad at translating videos?
- Actual transcript availability seems to be hit or miss. Sometimes Grok has access auto-generated captions and sometimes it doesn't and it's not clear why.
- Hallucination loves high-context prompts. Grok frequently borrowed claims from quote-tweets instead of the audio. If X exposes suggested context, the model absorbs it wholesale.
- Sampling video frames might help with translations. While translatemom doesn't currently do this, based on this experiment it might be something we add for the future.
- UX matters. Many times, Grok would misinterpret the user's command to translate the video and translate ONLY the post's text instead. In a traditional UI or with a specialized bot like @TranslateMom this won't happen
Mistranslation as Misinformation
Having run TranslateMom for nearly two years, I've noticed a troubling political trend: people intentionally misquoting those they politically disagree with by uploading videos alongside misleading translations, like quoting someone as saying "We must eat the poor," when they're actually sharing a soup recipe in Arabic.
Fighting misinformation online has never been more critical, and unfortunately, tools like Grok are easily manipulated for this purpose.
Until then, please stop only using "@grok translate this". @TranslateMom is not perfect either, but it will get you further to the truth, and using multiple sources of translations will get you a clearer picture. Better yet, ask a real human who knows the language!