Add support for lite-whisper #1886
Conversation
|
Hey @Eyoel-gebre, thank you for your work. I've successfully converted a model to ct2 format using your code: https://huggingface.co/bzikst/lite-whisper-large-v3-acc-ct2, but when I try to load it it throws |
There's a small miss-match between what the converted model produces and what the C++ model execution code expects due to fused layer logic: #1887 I think I have a work around and hope to get things wrapped up soon though. |
| """ | ||
| self._model_name_or_path = model_name_or_path | ||
| self._model_processor_name = model_name_or_path | ||
| if model_name_or_path.startswith('efficient-speech/lite-whisper'): |
There was a problem hiding this comment.
I don't think this is the best approach, since a different org might upload their own model and then this condition will evaluate to false
| if hasattr(transformers, loader.architecture_name): | ||
| model_class = getattr(transformers, loader.architecture_name) | ||
| model = self.load_model(model_class, self._model_name_or_path, **kwargs) | ||
| elif self._model_name_or_path.startswith('efficient-speech/lite-whisper'): |
|
Thanks for your effort! . I've successfully converted a model to ct2 format using your code, https://github.com/Eyoel-gebre/CTranslate2.git. But, when I used the converted model to infer, the results were not normal. Any hints? Or could you please provide the example to convert and infer? |
4 participants
Adding low rank linear layer logic to support the new lite-whisper model:
https://huggingface.co/efficient-speech/lite-whisper-large-v3-turbo
Adds low rank linear layer logic.