It’s possible to combine Whisper with another tool, like PyAnnote to do this though, which I’d like to add as a feature in the future. Right now, you would have to manually clean up and assign segments to a speaker. One big feature I personally wish Whisper had was speaker diarization, where the model identifies different speakers throughout the recording and determines when that person spoke. Even using the medium model, minus some specific technology names, it transcribed videos without a single mistake. Overall, I was thoroughly impressed with the transcription quality. Since it is natively exposed in Python, it was the perfect candidate to team up with Algolia’s Python API Client. It can recognize many different languages and even has the ability to translate between them. You can check out the results on our demo site here! Machine Learning – WhisperĪs mentioned above, OpenAI released Whisper recently which is a general-purpose speech recognition model.

In the rest of this post, I will describe the toolchain I used to build the A/VSearch CLI, an integrated command line for generating and indexing transcripts from a YouTube channel or playlist. Fortunately for us, at almost the exact same time, the team at OpenAI released a new neural network called Whisper for automatic speech recognition. We’ve tried to do this in the past using YouTube’s captioning capabilities with mixed results. This meant indexing the transcripts of the videos. I wanted to be able to help developers find the exact spot in the video that matched their search. I knew I wanted the videos to be searchable by title and description and discoverable by category, but I wanted to do more. The theme of our 2022 Algolia Developer Conference was “Index the world and put your data in motion” so naturally, as soon as the last video was uploaded to YouTube, talk turned to how we could put all of this great new content in motion for our customers.

Download transcript of youtube video