Field Report · October 29, 2025
Please Stop Using Grok
Putting Grok's video translations to the test
Every time someone tags Grok to translate a video, TranslateMom replies with its own answer. I wanted to see who wins.
I built @TranslateMom in May 2023, before X rolled out Grok. What started as a simple Whisper + Google Translate loop has grown into a $2k+ MRR side project with over 100k active users on X.
Grok was rolled onto X in March 2025, and has had quite the bumpy road, including spewing out racist comments and calling itself mecha hitler. Despite the rough start, Grok's now a staple: 61.5M posts and counting.
As Grok got smarter, TranslateMom mentions dropped slightly, sometimes replaced with a desperate "@grok is this true?"
Methodology
Experiment parameters
- Grok Attack. I flagged any reply where Grok mentions "translate" or "english". Basically, any time it looked like a translation attempt. TranslateMom then replied with its own answer.
- Observation window. Sample clips were collected between October 27 and October 29, 2025. This article might be outdated as @grok and @TranslateMom both may have improved since the time writing this.
- Source Transcript By default, TranslateMom will also provide the source transcript which was used for the translation, but Grok does not do so unless you ask explicitly. Most users don't, and it was reflected in this experiment too.
Results
French Example
Kris Letang’s farewell message in french
The worst behavior is when Grok refuses outright. This can be a positive (no hallucination), but not helpful for the user.
TranslateMom
TranslateMom
View transcript
I hope we’ll have fun one last time, it’s the way everyone wants to see you retire, so for me it will be a special moment. You’ve been someone who helped me enormously in my career, so I wish you a lovely evening and we hope we have a great time together one last time. Thank you.
Grok
View response
Sorry, the video doesn't have built-in subtitles, and I couldn't extract the exact French audio transcript from the clip. If you can provide the spoken French text, I'll translate it to English for you!
So why is Grok bad at translating videos?
After reviewing dozens of these, a few patterns emerged:
- Grok's transcript access is inconsistent. Sometimes it has auto-captions, sometimes it doesn't, and there's no clear pattern why.
- Context poisoning is rampant. Grok frequently borrows claims from quote-tweets instead of the actual audio. If X exposes surrounding context, the model absorbs it wholesale, often at the expense of accuracy.
- Visual context could help. Sampling video frames might improve translations. TranslateMom doesn't currently do this, but this experiment suggests it's worth exploring.
- Grok often misreads the request entirely, translating the tweet text instead of the video. A specialized UI or a purpose-built bot like @TranslateMom prevents this.
The Scoreboard
Out of the 5 clips reviewed in this experiment, Grok failed meaningfully on 4. TranslateMom handled all 5 with usable translations (one was a tie due to poor audio quality).
Mistranslation as Misinformation
Having run TranslateMom for nearly two years, I've noticed a troubling political trend: people intentionally misquoting those they politically disagree with by uploading videos alongside misleading translations, like quoting someone as saying "We must eat the poor," when they're actually sharing a soup recipe in Arabic.
Fighting misinformation online has never been more critical. Tools like Grok are easily manipulated for this purpose.
Please stop only using "@grok translate this". @TranslateMom is not perfect either, but it will get you further to the truth. Using multiple sources of translations will get you a clearer picture. Or better yet, use a tool built specifically for this.