All the solutions I’m seeing are some third party service where I would have to upload my videos to them to get them transcribed.
Might be janky, but if you really wanted this for free you could get a speech to text program like futo, play the video and have it transcribe it and save it to a text file, then copy and paste in the subtitles
If you need it for media that has subtitles available somewhere, then there are plugins for that (ex. Jellyfin/Plex/Kodi)
If you’re looking for something to automatically transcribe audio locally, I’m not as sure but others already suggested some
Which are you looking for?
My friend makes youtube videos and wants an easy way to subtitle them, as YouTube auto caption is crap.
What does he edit the video with? My video editing software does speech to text subtitles. Which is nice because I can edit where and how the subs are displayed.
Maybe you don’t even need that, at least for accessibility.
Windows for example now has exactly this feature, which is a speech-to-text-transformer powered by some “AI”.
But, in contrast to the Bing chat, this works (afaik) offline by some FOSS-backend, which I don’t know the name of anymore (maybe someone else will?) You could use that tool for live transcription. That is supposed to work extremely well!
Please correct my if I’m wrong, I don’t use Windows anymore personally, and at work we have a business edition that doesn’t ship this brand new feature yet.
(Side note: as strongly as I hate Windows, this feature is absolutely godsend for hearing-impaired people and should be adopted by every other OS!)
If you want to transcript movies and thereof in bulk by uploading them, I can’t give you any information, sorry.
But I believe there are some sites that give you the “subtitle file” for download freely, which you can add manually for each movie in Plex/ Jellyfin.
Android does on-device transcription of any Audio source as well in recent versions!
The issue with providing this with open-source software is that it tends to require deep integration into the OS, which needs pretty much the same kinds of APIs that spyware also needs, so they get locked down a lot …
For example on Android I’m pretty sure that a 3rd party play store app could not provide the same feature without requiring the user to click through some unavoidable, scary sounding warnings from the OS (if at all).
AFAIK iOS and macOS have had this for some time now. I think since a couple years back it’s all local-only on newer devices as well.
Maybe Whishper would be suitable?
Oh that looks really cool, thank you for the link.
That looks perfect! Thank you!
Okay yeah, I spun up a docker instance and this is cool as fuck. It seems to be exactly what OP is looking for. This is cool enough to be a post on its own tbh. It would be perfect in a ytdl workflow, as you can do the transcription by linking a video. I’ve been holding off on adding youtube to my Jellyfin setup for just this sort of tool. I hope the add the GPU accelerated faster-whisper models soon.
Luckily I still had the project in my history! Glad it was useful.
Here’s one I’ve been playing with https://github.com/jhj0517/Whisper-WebUI
The small model of fast Whisper has been amazing for the 3 options it gives (files, YT, or recording), tho I have in mind the limitations and I’ve only used it with somewhat clear audio.I used this CLI tool for it: https://github.com/m1guelpf/auto-subtitle
Whisper-based, worked great for subtitling S2 of ReGenesis as for some reason the DVDs lacked subtitles. I used the largest model, I think. It even managed to correctly identify which character was crying, which I thought was impressive.
That looks pretty much what I’m looking for, thank you! I see instructions to run in Windows, but does it also work on Macs?
I second this. Also, I have been experiencing with automating it, using https://hub.docker.com/r/onerahmet/openai-whisper-asr-webservice as a provider for Bazarr
