* Fix: Prevent <|nocaptions|> tokens in BatchedInferencePipeline
- Add nocaptions component tokens [1771, 496, 9799] to suppress_tokens list
- Add segment filtering to remove any remaining <|nocaptions|> segments
- Resolves issue where BatchedInferencePipeline would generate malformed
special tokens during periods of silence or low-confidence transcription
- Includes comprehensive tests to verify the fix
The issue occurred because while bracket tokens ('<', '|', '>') were
already suppressed, the content tokens ('no', 'ca', 'ptions') were not,
leading to partial token generation that formed complete <|nocaptions|>
tags in the output.
Files changed:
- faster_whisper/transcribe.py: Core fix implementation
- test_nocaptions_comprehensive.py: Comprehensive test suite
- tests/test_nocaptions_fix.py: Unit tests
* removed
* Fix: Prevent <|nocaptions|> tokens in BatchedInferencePipeline
* Fix: Implement proper <|nocaptions|> token suppression using single token approach
* ci: trigger tests
* fix: remove trailing whitespace from blank lines
* Update faster_whisper/transcribe.py
Co-authored-by: Mahmoud Ashraf <hassouna97.ma@gmail.com>
* Update faster_whisper/tokenizer.py
Co-authored-by: Mahmoud Ashraf <hassouna97.ma@gmail.com>
* Update faster_whisper/tokenizer.py
Co-authored-by: Mahmoud Ashraf <hassouna97.ma@gmail.com>
* Rename no_speech to no_captions in tokenizer
* nocaptions has been renamed to nospeech
* break line
* line break
* Refactor no_speech method for improved readability by adjusting line breaks
---------
Co-authored-by: Mahmoud Ashraf <hassouna97.ma@gmail.com>
* Added test for `multilingual` option with english-german audio
* removed `output_language` argument as it is redundant, you can get the same functionality with `task="translate"`
* use the correct `encoder_output` for language detection in sequential transcription
* enabled `multilingual` functionality for batched inference
* Supported new options for batched transcriptions:
* `language_detection_threshold`
* `language_detection_segments`
* Updated `WhisperModel.detect_language` function to include the improved language detection from #732 and added docstrings, it's now used inside `transcribe` function.
* Removed the following functions as they are no longer needed:
* `WhisperModel.detect_language_multi_segment` and its test
* `BatchedInferencePipeline.get_language_and_tokenizer`
* Added tests for empty audios
* transcribe: return all language probabilities if requested
If return_all_language_probs is True, TranscriptionInfo structure
will have a list of tuples reflecting all language probabilities
as returned by the model.
* transcribe: fix docstring
* transcribe: remove return_all_lang_probs parameter
* Implement VadOptions
* Fix line too long
./faster_whisper/transcribe.py:226:101: E501 line too long (111 > 100 characters)
* Reformatted files with black
* black .\faster_whisper\vad.py
* black .\faster_whisper\transcribe.py
* Fix import order with isort
* isort .\faster_whisper\vad.py
* isort .\faster_whisper\transcribe.py
* Made recommended changes
Recommended in https://github.com/guillaumekln/faster-whisper/pull/198
* Fix typing of vad_options argument
---------
Co-authored-by: Guillaume Klein <guillaumekln@users.noreply.github.com>
* Use cache_dir instead of local_dir
* Fix unit test
* Use cache_dir and preserve local_dir parameter
* Remove blank line at the end
* Disable ut
* Implement download_root suggestion
* Use cache_dir=download_root
* Support VAD filter
* Generalize function collect_samples
* Define AudioSegment class
* Only pass prompt and prefix to the first chunk
* Add dict argument vad_parameters
* Fix isort format
* Rename method
* Update README
* Add shortcut when the chunk offset is 0
* Reword readme
* Fix end property
* Concatenate the speech chunks
* Cleanup diff
* Increase default speech pad
* Update README
* Increase default speech pad
* Automatically download converted models from the Hugging Face Hub
* Remove unused import
* Remove non needed requirements in dev mode
* Remove extra index URL when pip install in CI
* Allow downloading to a specific directory
* Update docstring
* Add argument to disable the progess bars
* Fix typo in docstring
