If ICU is already available on the system there's no point downloading and
building it again. To save time and space, add an option to find and use the
system version of ICU
Stop using deprecated API functions that were removed in OBS v32 (See: https://github.com/obsproject/obs-studio/pull/12488)
* Use the return value from `obs_scene_add` instead of looking it up after creating it using the deprecated `obs_scene_sceneitem_from_source` function
* Use v2 sceneitem transform API instead of deprecated v1 API
* Change all uses of `circlebuf` to `deque` as it's been renamed in OBS
Additional bug fixes:
* Wait until scenes are loaded to create text source if it's missing. Previously the creation of the text source could fail due to there not being any scenes yet to create it in
* Fix build on MacOS and some Linux distributions
* Catch exceptions thrown by ONNX runtime and log an error instead of crashing
Other changes:
* Update OBS source dependencies
* OBS: 30.1.2 -> 31.1.1
* OBS deps: 2024-03-19 -> 2025-07-11
* Qt6 deps: 2024-03-19 -> 2025-02-04
* Update build files to more closely match the current plugin template repository
Fixes: #230Fixes: #227
* Fix `whisper_buffer` and `resampled_buffer` data race
`media_unpause` was causing `wisper_buffer` to be freed while
`vad_based_segmentation`/`hybrid_vad_segmentation` need that buffer
to not be modified for the duration of those calls
* Slightly improve handling for weird subtitle output filenames
* Squashed 'deps/c-webvtt-in-video-stream/' content from commit 5579ca6
git-subtree-dir: deps/c-webvtt-in-video-stream
git-subtree-split: 5579ca6dc9dcf94e3c14631c6c01b2ee4dfcf005
* Add WIP webvtt sei functionality
* Add webvtt recording/streaming settings
* Make latency_to_video_in_msecs and send_frequency_hz configurable
* Make webvtt languages configurable
* Add translation and main language separately
* Add rust CI integration
* Update version to 0.3.9 in README.md and buildspec.json; add available versions section for clarity
* Add support for custom API translation provider and update configuration options
* Enable ccache support in CMake build scripts for improved build performance
* Add status message for detected ccache in CMake build scripts
* Add support for disabled VAD mode and enhance CMake configuration
* Enhance VAD processing and transcription filter data structure with additional comments and logic adjustments
* Enhance VAD processing with improved logging, adjust single segment default, and update inference handling
* Refactor whisper parameter handling and enhance utility functions for better clarity and maintainability
* Add whisper parameters group properties and clean up related code
* Add whisper parameters handling and update related files for improved functionality
* Refactor whisper parameter type casting for improved clarity and consistency
* trigger build
* Fix logging message to use the correct variable for saved sentence
* Add cloud translation support with multiple providers and configuration options
* Refactor CMakeLists.txt for cloud translation sources formatting
* Add support for translating only full sentences in cloud translation
* Update ICU build configuration and fix header include case sensitivity
* Fix CURL helper function signatures and improve URL encoding
* Fix character type casting in DeepLTranslator for language conversion
* Refactor file saving logic in transcription filter to streamline sentence handling and add support for saving translated sentences
* Add support for Deepl Free API endpoint and enhance cloud translation configuration
* Add ccache detection to ICU build configuration for improved compilation speed
* Enhance ICU build configuration to use ccache as a compiler wrapper for improved performance
This commit modifies the `load_models_info` function in `model-infos.cpp` to enable downloading the models directory from GitHub. Previously, the download functionality was commented out, but now it is uncommented to allow for successful downloading of the directory. This change improves the functionality of the code.
Ref: #<issue_number>
Simplify the translation options in the transcription-filter-properties.cpp file by adding a new option "translate_only_full_sentences". This option will be visible only when the "translate_enabled" flag is true and the "is_advanced" flag is set.
Remove unnecessary code in model-infos.cpp
Remove the code that logs a warning message when the "sha256" field is missing or invalid in the model JSON file. This code is no longer needed as it does not affect the functionality of the program.
Comment out download_json_from_github in model-infos.cpp
Comment out the call to the "download_json_from_github" function in the load_models_info() function in model-infos.cpp. This function is currently not working as intended and needs further investigation.
* refactor: Handle file exceptions when writing raw sentence and translations
This commit modifies the code in transcription-filter-callbacks.cpp to handle file exceptions when writing raw sentence and translations to files. It adds exception handling using try-catch blocks to ensure that file operations are properly handled. This change improves the robustness of the code and prevents crashes or unexpected behavior when file operations fail.
* refactor: Update models_info function to use cached models information
The models_info function in model-downloader.cpp has been updated to use a cached version of the models information. This improves performance by avoiding unnecessary file reads and JSON parsing. The function now returns a const reference to the cached models_info map. This change ensures that the models_info function is more efficient and reduces the overhead of loading the models information.
Refactor the code in model-downloader.cpp to use the updated models_info function and remove the unnecessary file read and JSON parsing code.
Closes#123
* refactor: Simplify file handling in transcription-filter-callbacks.cpp
* refactor: Add script to query Hugging Face models and update models_directory.json
This commit adds two new scripts, hugging_face_model_query.py and hugging_face_model_query_all.py, to query Hugging Face models and update the models_directory.json file. The hugging_face_model_query.py script fetches model information from the Hugging Face API and adds new models to the models_directory.json file. The hugging_face_model_query_all.py script fetches a list of models matching a specific search criteria and adds the matching models to the models_directory.json file. These scripts will help keep the models_directory.json file up to date with the latest models available on Hugging Face.
Refactor the file handling in transcription-filter-callbacks.cpp
This commit simplifies the file handling in the transcription-filter-callbacks.cpp file. The changes aim to improve the readability and maintainability of the code by reducing complexity and removing unnecessary code.
Update the models_info function to use cached models information
This commit updates the models_info function to use cached models information instead of fetching it every time the function is called. This change improves the performance of the function by reducing the number of API calls and improves the overall efficiency of the code.
Handle file exceptions when writing raw sentence and translations
This commit adds exception handling code to handle file exceptions when writing raw sentence and translations. The changes ensure that any file-related exceptions are caught and properly handled, preventing the program from crashing or producing incorrect results.
Simplify the Onnxruntime installation in FetchOnnxruntime.cmake
This commit simplifies the Onnxruntime installation process in the FetchOnnxruntime.cmake file. The changes aim to make the installation steps more concise and easier to understand, improving the overall maintainability of the code.
Update the version to 0.3.6 and adjust the website URL
This commit updates the version of the software to 0.3.6 and adjusts the website URL accordingly. The changes ensure that the software is properly versioned and the website URL is up to date.
* refactor: Add ExtraInfo struct to ModelInfo and update models_info function
* refactor: Update model names in models_directory.json and fix URL in transcription-filter.h
- Add filter-replace-utils for serializing and deserializing in the src/ui directory.
- Update CMakeLists.txt to include the new files in the target_sources.
- Update FindLibAvObs.cmake to read buildspec.json from the CMAKE_SOURCE_DIR.
- Update model-infos.cpp to include two new Whisper models.
- Add a new CMakeLists.txt file in the src/tests directory.
- Add localvocal-offline-test.cpp in the src/tests directory.
- Add clear_current_caption function in localvocal-offline-test.cpp.
* refactor: Update version to 0.3.5 and clear current caption in transcription filter callbacks
* feat: Refactor whisper-processing.cpp for improved VAD segmentation and token buffer thread
* feat: Update prebuilt Whispercpp version to 0.0.6
* refactor: Remove trailing whitespace in translation-language-utils.h
* refactor: Add case-insensitive flag to regex in set_text_callback
The code change adds the `std::regex_constants::icase` flag to the regex used in the `set_text_callback` function in `transcription-filter-callbacks.cpp`. This allows for case-insensitive matching when replacing filter words in the `str_copy` string.
Refactor the code to improve VAD segmentation and token buffer thread in whisper-processing.cpp
The code change refactors the `whisper-processing.cpp` file to improve the VAD (Voice Activity Detection) segmentation and token buffer thread. This aims to enhance the performance and accuracy of the transcription filtering process.
refactor: Add prepopulated filter options and corresponding map entries in FilterReplaceDialog
The code change adds prepopulated filter options, such as "English Swear Words," "English Hallucinations," and "Korean Hallucinations," to the `FilterReplaceDialog` UI. It also adds the corresponding map entries to the `filter_words_replace` map, allowing users to easily add predefined filter patterns and replacement values.
refactor: Update version to 0.3.5 and clear current caption in transcription filter callbacks
The code change updates the version to 0.3.5 and clears the current caption in the transcription filter callbacks. This ensures that the correct version is displayed and any previous captions are removed.
refactor: Remove trailing whitespace in translation-language-utils.h
The code change removes trailing whitespace in the `translation-language-utils.h` file, improving code readability and consistency.
* refactor: Add filter-replace-utils for serializing and deserializing filter words replacements
* refactor: Add filter-replace-utils for serializing and deserializing filter words replacements
* refactor: Add filter-replace-utils for serializing and deserializing filter words replacements
* Fix hangups and VAD segmentation
* feat: Add max_sub_duration field to transcription filter data
* chore: Update VAD parameters for better segmentation accuracy
* feat: Add segment_duration field to transcription filter data
* feat: Optimize VAD processing for better performance
* feat: Refactor token buffer thread and whisper processing
The code changes involve refactoring the token buffer thread and whisper processing. The token buffer thread now uses the variable name `word_token` instead of `word` for better clarity. In the whisper processing, the log message format has been updated to include the segment number and token number. These changes aim to improve the performance and accuracy of VAD processing, as well as add new fields to the transcription filter data.
* Refactor token buffer thread and whisper processing
* refactor: Update translation context in transcription filter
The code changes in this commit update the translation context in the transcription filter. The `translate_add_context` property has been changed from a boolean to an integer slider, allowing the user to specify the number of context lines to add to the translation. This change aims to provide more flexibility in controlling the context for translation and improve the accuracy of the translation output.
* refactor: Update last_text variable name in transcription filter callbacks
* feat: Add translation language utilities
This commit adds a new file, `translation-language-utils.h`, which contains utility functions for handling translation languages. The `remove_start_punctuation` function removes any leading punctuation from a given string. This utility will be used in the translation process to improve the quality of the translated output.
* feat: Update ICU library configuration and dependencies
This commit updates the configuration and dependencies of the ICU library. The `BuildICU.cmake` file has been modified to use the `INSTALL_DIR` variable instead of the `ICU_INSTALL_DIR` variable for setting the ICU library paths. Additionally, the `ICU_IN_LIBRARY` variable has been renamed to `ICU_IN_LIBRARY` for better clarity. These changes aim to improve the build process and ensure proper linking of the ICU library.
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
* refactor: Update ICU library configuration and dependencies
This commit updates the `BuildICU.cmake` file to set the `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` environment variables to `-fPIC` for Linux platforms. This change aims to ensure that the ICU library is built with position-independent code, improving compatibility and security. Additionally, the `icuin` library has been renamed to `icui18n` to align with the naming convention. These updates enhance the build process and maintain consistency in the ICU library configuration.
* Fix `ModelDownloader` not being aware of child object deletions
* Delete `ModelDownloader` after it's done processing
Otherwise this is only deleted when OBS exits, effectively leaking
memory
* Protect logging with a mutex
Main thread and worker thread output could get interleaved weirdly
without this
* Move segments.json saving to different thread
This was taking a considerable amount of time, especially for longer
input files, reducing overall utilization
* Check whether offline test can push more data before waiting
* Fix offline test with large files
In
```
circlebuf_push_back(
&gf->input_buffers[c],
audio[c].data() +
frames_count * frame_size_bytes,
frames_size_bytes);
```
`frames_count * frame_size_bytes` would overflow with `int` on
a 4 hour file; using `size_t` (on a 64 bit platform) fixes that
* look at the front of the whisper buffer instead of the back
this should mostly not make a difference, but feels semantically
more correct
* Initialize `resampled_buffer` for offline tests
* Read relevant audio bytes
There are two issues here:
1. `line_size` may contain padding (didn't happen in my tests)
2. from: 2b5f000d3f:/libavutil/frame.h#l405
> For audio, only linesize[0] may be set. For planar audio, each
> channel plane must be the same size.
* log running time in addition to local time
* Run whisper test "as fast as possible"
This kind of behaves like libobs, where each chunk of audio is
inspected individually by VAD/whisper, until processing of either
takes longer than the window length, in which case audio continues
to stream in
* Only ever send a single chunk of audio
* Add additional files to tests copy command
* Use condition variable to signal input thread if available
* Only wait in whisper thread if input buffers are empty
* Add accessor for VAD window size in samples
* Feed buffered audio data to VAD in proper window sizes
* Wake whisper thread whenever audio is received
* Update silero VAD to v5
* Only reset VAD state between chunks of activity
* refactor: Update whisper model path and enable hipBLAS acceleration
* refactor: Update whisper model path and enable hipBLAS acceleration
* refactor: Update whisper model path and enable hipBLAS acceleration
* refactor: Update whisper model path and enable hipBLAS acceleration
* refactor: Update whisper model path and enable hipBLAS acceleration
* refactor: Update whisper model path and enable CoreML acceleration
* refactor: Enable partial transcription with a latency of 1000ms
* refactor: Update CMakePresets.json and buildspec.json
- Remove the "QT_VERSION" variable from CMakePresets.json for all platforms
- Update the "version" of "obs-studio" and "prebuilt" dependencies in buildspec.json
- Update the "version" of "qt6" dependency in buildspec.json
- Update the "version" of the project to "0.3.3" in buildspec.json
- Update the "version" of the project to "0.3.3" in CMakePresets.json
- Remove unused code in whisper-processing.cpp
* refactor: Add -Wno-error=deprecated-declarations option to compilerconfig.cmake
* refactor: Update language codes in translation module
* refactor: Add boolean flag for whisper model loaded status
* refactor: Improve handling of whisper model paths in transcription filter
* refactor: Update whisper model path and add flag for model loaded status
* refactor: Add transcription-filter-properties.cpp for managing filter properties
* refactor: Add translation_monitor to transcription filter
- Add translation_monitor to the transcription filter data structure
- Initialize and stop the translation_monitor in the transcription_filter_update function
- Update the send_caption_to_source function to use the translation_monitor for sending translated captions
- Clear the translation_monitor when disabling buffered output in the transcription_filter_update function
* refactor: Simplify UI and improve error handling in transcription filter
