OpenAI Whisper and it’s amazing power to do fine-tuning demonstrated on my mother-tongue
Kurian Benoy (~kurianbenoy) |
Description:
The core of the talk covers how we can fine tune a whisper model. I will specifically talk about fine tuning whisper to achieve state of the art results in a low resource language like Malayalam. We have got great results on fine-tuning on Malayalam whisper model. The original model weights of Whisper was reported with a WER of 108% and we were able to fine tune to reach a WER of close to 10%(approx 90 percentage accuracy) in CommonVoice 11 Malayalam subset and even 1% WER in MSC dataset.
Talk Outline:
- What is OpenAI Whisper?
- Features of OpenAI Whisper
- What is Fine-tuning and how to fine-tune Whisper?
- About my mother tongue
- Methodology of benchmarking whisper models
- Results on benchmarking Whisper model
- Future Ideas & Conclusion
Prerequisites:
- Any developer with 1 years experience, they needn't have any prior ML experience. I will try to demystify any jargons during my talk that's why I am saying no experience is needed in ML.
- Lot of interest in your own mother-tongue is always appreciated.
Content URLs:
Github project:
- https://github.com/kurianbenoy/malayalam_asr_benchmarking
Benchmarking results:
- https://huggingface.co/datasets/kurianbenoy/malayalam_msc_benchmarking/tree/main
- https://huggingface.co/datasets/kurianbenoy/malayalam_common_voice_benchmarking
Speaker Info:
Kurian is a Team Lead and AI Engineer working in sentient.io, a fast-paced startup based in Singapore. I have multiple years of experience in Python and Machine learning experience, where I am now looking more into the MLOPs side. I have contributed to various open source organizations like Keras, DVC, HuggingFace, fast.ai, Swathanthra Malayalam Computing, CloudCV etc.
More details about my previous talks can be found in below link.