Compare commits

...

28 Commits

Author SHA1 Message Date
github-actions[bot]
f33d27f836 chore(release): Update version to v1.4.280 2025-08-10 12:34:52 +00:00
Kayvan Sylvan
1694324261 Merge pull request #1686 from ksylvan/0810-fix-openai-streaming-bug
Prevent duplicate text output in OpenAI streaming responses
2025-08-10 05:32:17 -07:00
Changelog Bot
3a3f5c50a8 chore: incoming 1686 changelog entry 2025-08-10 05:27:29 -07:00
Kayvan Sylvan
b1abfd71c2 fix: prevent duplicate text output in OpenAI streaming responses
## CHANGES

- Skip processing of ResponseOutputTextDone events
- Prevent doubled text in stream output
- Add clarifying comment about API behavior
- Maintain delta chunk streaming functionality
- Fix duplicate content issue in responses
2025-08-10 05:24:30 -07:00
github-actions[bot]
f5b7279225 chore(release): Update version to v1.4.279 2025-08-10 12:13:45 +00:00
Kayvan Sylvan
b974e1bfd5 Merge pull request #1685 from ksylvan/0810-fix-gemini-roles-in-sessions
Fix Gemini Role Mapping for API Compatibility
2025-08-10 05:11:16 -07:00
Changelog Bot
8dda68b3b9 chore: incoming 1685 changelog entry 2025-08-10 05:07:06 -07:00
Kayvan Sylvan
33c24e0cb2 fix(gemini): map chat roles to Gemini user/model in convertMessages
CHANGES
- Map assistant role to model per Gemini constraints
- Map system, developer, function, tool roles to user
- Default unrecognized roles to user to preserve instruction context
- Add unit test validating convertMessages role mapping logic
- Import chat package in tests for role constants
2025-08-10 04:55:33 -07:00
github-actions[bot]
8fb0c5b8a8 chore(release): Update version to v1.4.278 2025-08-09 17:26:37 +00:00
Kayvan Sylvan
d82122b624 Merge pull request #1681 from ksylvan/0803-youtube-transcript-lang-fix
Enhance YouTube Support with Custom yt-dlp Arguments
2025-08-09 10:24:09 -07:00
Kayvan Sylvan
f5966af95a docs: update release notes 2025-08-09 10:09:32 -07:00
Changelog Bot
9470ee1655 chore: incoming 1681 changelog entry 2025-08-09 10:06:50 -07:00
Kayvan Sylvan
9a118cf637 refactor: replace custom arg parser with shellquote; precompile regexes
CHANGES
- Precompile regexes for video, playlist, VTT tags, durations.
- Parse yt-dlp additional arguments using shellquote.Split for safety.
- Validate user-provided yt-dlp args and surface quoting errors.
- Reuse compiled regex in GetVideoOrPlaylistId extractions for stability.
- Simplify removeVTTTags by leveraging precompiled VTT tag matcher.
- Parse ISO-8601 durations with precompiled pattern for efficiency.
- Replace inline VTT language regex with cached compiled matcher.
- Remove unused findVTTFiles helper and redundant language checks.
- Add go-shellquote dependency in go.mod and go.sum.
- Reduce allocations by eliminating per-call regexp.MustCompile invocations.
2025-08-09 10:01:24 -07:00
Kayvan Sylvan
d69757908f docs: update release notes 2025-08-08 23:49:55 -07:00
Changelog Bot
30525ef1c0 chore: incoming 1681 changelog entry 2025-08-08 23:44:42 -07:00
Kayvan Sylvan
8414e72545 feat: add smart subtitle language fallback when requested locale unavailable
CHANGES
- Introduce findVTTFilesWithFallback to handle subtitle language absence
- Prefer requested language VTT, gracefully fallback to available alternatives
- Auto-detect downloaded subtitle language and proceed without interruption
- Update yt-dlp processing to use fallback-aware VTT discovery
- Document language fallback behavior and provide usage example
- Return first available VTT when no specific language requested
- Detect language-coded filenames using regex for robust matching
2025-08-08 23:39:12 -07:00
Kayvan Sylvan
caca366511 docs: update YouTube processing documentation for yt-dlp argument precedence control
## CHANGES

- Add user argument precedence over built-in flags
- Document argument order and override behavior
- Include new precedence section with detailed explanation
- Add override examples for language and format
- Update tips section with precedence guidance
- Modify Go code to append user args last
- Add testing tip for subtitle language discovery
- Include practical override use case examples
2025-08-08 23:14:28 -07:00
Kayvan Sylvan
261eb30951 feat: add --yt-dlp-args flag for custom YouTube downloader options
### CHANGES

- Introduce `--yt-dlp-args` flag for advanced control
- Allow passing browser cookies for authentication
- Improve error handling for YouTube rate limits
- Add comprehensive documentation for YouTube processing
- Refactor YouTube methods to accept additional arguments
- Update shell completions to include new flag
2025-08-08 23:03:02 -07:00
Kayvan Sylvan
bdb36ee296 Merge branch 'main' into 0803-youtube-transcript-lang-fix 2025-08-08 16:42:37 -07:00
github-actions[bot]
1351f138fb chore(release): Update version to v1.4.277 2025-08-08 07:40:13 +00:00
Kayvan Sylvan
8da51968dc Merge pull request #1679 from ksylvan/0807-desktop-notification
Add cross-platform desktop notifications to Fabric CLI
2025-08-08 00:37:40 -07:00
Kayvan Sylvan
30d23f15be chore: format fix 2025-08-08 00:35:04 -07:00
Changelog Bot
0a718be622 chore: incoming 1679 changelog entry 2025-08-08 00:30:20 -07:00
github-actions[bot]
21f258caa4 feat(cli): add cross-platform desktop notifications with secure custom commands
CHANGES
- Integrate notification sending into chat processing workflow
- Add --notification and --notification-command CLI flags and help
- Provide cross-platform providers: macOS, Linux, Windows with fallbacks
- Escape shell metacharacters to prevent injection vulnerabilities
- Truncate Unicode output safely for notification message previews
- Update bash, zsh, fish completions with new notification options
- Add docs and YAML examples for configuration and customization
- Add unit tests for providers and notification integration paths
2025-08-08 00:20:51 -07:00
github-actions[bot]
3584f83b30 chore(release): Update version to v1.4.276 2025-08-08 02:24:57 +00:00
Kayvan Sylvan
056791233a Merge pull request #1677 from ksylvan/0807-fix-release-notes-ci-cd-permission
Grant GITHUB_TOKEN write permissions for release notes job
2025-08-07 19:22:22 -07:00
Kayvan Sylvan
dc435dcc6e ci: add write permissions to update_release_notes job
## CHANGES

- Add contents write permission to release notes job
- Enable GitHub Actions to modify repository contents
- Fix potential permission issues during release process
2025-08-07 19:17:19 -07:00
Kayvan Sylvan
2b195f204d ci: separate release notes generation into dedicated job
## CHANGES

- Move changelog generation to separate workflow job
- Add fallback logic for YouTube subtitle language detection
- Remove changelog commands from main release job
- Create dedicated update_release_notes job with Go setup
- Implement retry mechanism without language specification
- Improve yt-dlp command argument construction flexibility
- Add proper checkout and Go configuration steps
2025-08-03 21:46:24 -07:00
27 changed files with 1324 additions and 83 deletions

View File

@@ -142,6 +142,8 @@ jobs:
update_release_notes:
needs: [build, get_version]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v4

View File

@@ -80,6 +80,7 @@
"Laomedeia",
"ldflags",
"libexec",
"libnotify",
"listcontexts",
"listextensions",
"listmodels",
@@ -93,6 +94,7 @@
"matplotlib",
"mattn",
"mbed",
"metacharacters",
"Miessler",
"nometa",
"numpy",
@@ -102,6 +104,7 @@
"opencode",
"openrouter",
"Orus",
"osascript",
"otiai",
"pdflatex",
"pipx",

View File

@@ -1,5 +1,55 @@
# Changelog
## v1.4.280 (2025-08-10)
### PR [#1686](https://github.com/danielmiessler/Fabric/pull/1686) by [ksylvan](https://github.com/ksylvan): Prevent duplicate text output in OpenAI streaming responses
- Fix: prevent duplicate text output in OpenAI streaming responses
- Skip processing of ResponseOutputTextDone events
- Prevent doubled text in stream output
- Add clarifying comment about API behavior
- Maintain delta chunk streaming functionality
## v1.4.279 (2025-08-10)
### PR [#1685](https://github.com/danielmiessler/Fabric/pull/1685) by [ksylvan](https://github.com/ksylvan): Fix Gemini Role Mapping for API Compatibility
- Fix Gemini role mapping to ensure proper API compatibility by converting chat roles to Gemini's user/model format
- Map assistant role to model role per Gemini API constraints
- Map system, developer, function, and tool roles to user role for proper handling
- Default unrecognized roles to user role to preserve instruction context
- Add comprehensive unit tests to validate convertMessages role mapping logic
## v1.4.278 (2025-08-09)
### PR [#1681](https://github.com/danielmiessler/Fabric/pull/1681) by [ksylvan](https://github.com/ksylvan): Enhance YouTube Support with Custom yt-dlp Arguments
- Add `--yt-dlp-args` flag for custom YouTube downloader options with advanced control capabilities
- Implement smart subtitle language fallback system when requested locale is unavailable
- Add fallback logic for YouTube subtitle language detection with auto-detection of downloaded languages
- Replace custom argument parser with shellquote and precompile regexes for improved performance and safety
## v1.4.277 (2025-08-08)
### PR [#1679](https://github.com/danielmiessler/Fabric/pull/1679) by [ksylvan](https://github.com/ksylvan): Add cross-platform desktop notifications to Fabric CLI
- Add cross-platform desktop notifications with secure custom commands
- Integrate notification sending into chat processing workflow
- Add --notification and --notification-command CLI flags and help
- Provide cross-platform providers: macOS, Linux, Windows with fallbacks
- Escape shell metacharacters to prevent injection vulnerabilities
## v1.4.276 (2025-08-08)
### Direct commits
- Ci: add write permissions to update_release_notes job
- Add contents write permission to release notes job
- Enable GitHub Actions to modify repository contents
- Fix potential permission issues during release process
## v1.4.275 (2025-08-07)
### PR [#1676](https://github.com/danielmiessler/Fabric/pull/1676) by [ksylvan](https://github.com/ksylvan): Refactor authentication to support GITHUB_TOKEN and GH_TOKEN
@@ -24,8 +74,7 @@
### Direct commits
- Chore: remove redundant words
Signed-off-by: queryfast <queryfast@outlook.com>
- Remove redundant words from codebase
- Fix typos in t_ patterns
## v1.4.272 (2025-07-28)

View File

@@ -551,6 +551,10 @@ Application Options:
--voice= TTS voice name for supported models (e.g., Kore, Charon, Puck)
(default: Kore)
--list-gemini-voices List all available Gemini TTS voices
--notification Send desktop notification when command completes
--notification-command= Custom command to run for notifications (overrides built-in
notifications)
--yt-dlp-args= Additional arguments to pass to yt-dlp (e.g. '--cookies-from-browser brave')
Help Options:
-h, --help Show this help message

View File

@@ -1,3 +1,3 @@
package main
var version = "v1.4.275"
var version = "v1.4.280"

Binary file not shown.

View File

@@ -80,6 +80,7 @@ _fabric() {
'(--transcript-with-timestamps)--transcript-with-timestamps[Grab transcript from YouTube video with timestamps]' \
'(--comments)--comments[Grab comments from YouTube video and send to chat]' \
'(--metadata)--metadata[Output video metadata]' \
'(--yt-dlp-args)--yt-dlp-args[Additional arguments to pass to yt-dlp]:yt-dlp args:' \
'(-g --language)'{-g,--language}'[Specify the Language Code for the chat, e.g. -g=en -g=zh]:language:' \
'(-u --scrape_url)'{-u,--scrape_url}'[Scrape website URL to markdown using Jina AI]:url:' \
'(-q --scrape_question)'{-q,--scrape_question}'[Search question using Jina AI]:question:' \
@@ -117,6 +118,8 @@ _fabric() {
'(--think-start-tag)--think-start-tag[Start tag for thinking sections (default: <think>)]:start tag:' \
'(--think-end-tag)--think-end-tag[End tag for thinking sections (default: </think>)]:end tag:' \
'(--disable-responses-api)--disable-responses-api[Disable OpenAI Responses API (default: false)]' \
'(--notification)--notification[Send desktop notification when command completes]' \
'(--notification-command)--notification-command[Custom command to run for notifications]:notification command:' \
'(-h --help)'{-h,--help}'[Show this help message]' \
'*:arguments:'
}

View File

@@ -13,7 +13,7 @@ _fabric() {
_get_comp_words_by_ref -n : cur prev words cword
# Define all possible options/flags
local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --language -g --scrape_url -u --scrape_question -q --seed -e --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --dry-run --serve --serveOllama --address --api-key --config --search --search-location --image-file --image-size --image-quality --image-compression --image-background --suppress-think --think-start-tag --think-end-tag --disable-responses-api --voice --list-gemini-voices --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --yt-dlp-args --language -g --scrape_url -u --scrape_question -q --seed -e --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --dry-run --serve --serveOllama --address --api-key --config --search --search-location --image-file --image-size --image-quality --image-compression --image-background --suppress-think --think-start-tag --think-end-tag --disable-responses-api --voice --list-gemini-voices --notification --notification-command --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
# Helper function for dynamic completions
_fabric_get_list() {
@@ -85,7 +85,7 @@ _fabric() {
return 0
;;
# Options requiring simple arguments (no specific completion logic here)
-v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key | --search-location | --image-compression | --think-start-tag | --think-end-tag)
-v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | --yt-dlp-args | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key | --search-location | --image-compression | --think-start-tag | --think-end-tag | --notification-command)
# No specific completion suggestions, user types the value
return 0
;;

View File

@@ -76,6 +76,7 @@ complete -c fabric -l strategy -d "Choose a strategy from the available strategi
complete -c fabric -l think-start-tag -d "Start tag for thinking sections (default: <think>)"
complete -c fabric -l think-end-tag -d "End tag for thinking sections (default: </think>)"
complete -c fabric -l voice -d "TTS voice name for supported models (e.g., Kore, Charon, Puck)" -a "(__fabric_get_gemini_voices)"
complete -c fabric -l notification-command -d "Custom command to run for notifications (overrides built-in notifications)"
# Boolean flags (no arguments)
complete -c fabric -s S -l setup -d "Run setup for all reconfigurable parts of fabric"
@@ -94,6 +95,7 @@ complete -c fabric -l transcript -d "Grab transcript from YouTube video and send
complete -c fabric -l transcript-with-timestamps -d "Grab transcript from YouTube video with timestamps"
complete -c fabric -l comments -d "Grab comments from YouTube video and send to chat"
complete -c fabric -l metadata -d "Output video metadata"
complete -c fabric -l yt-dlp-args -d "Additional arguments to pass to yt-dlp (e.g. '--cookies-from-browser brave')"
complete -c fabric -l readability -d "Convert HTML input into a clean, readable view"
complete -c fabric -l input-has-vars -d "Apply variables to user input"
complete -c fabric -l dry-run -d "Show what would be sent to the model without actually sending it"
@@ -108,4 +110,5 @@ complete -c fabric -l list-gemini-voices -d "List all available Gemini TTS voice
complete -c fabric -l shell-complete-list -d "Output raw list without headers/formatting (for shell completion)"
complete -c fabric -l suppress-think -d "Suppress text enclosed in thinking tags"
complete -c fabric -l disable-responses-api -d "Disable OpenAI Responses API (default: false)"
complete -c fabric -l notification -d "Send desktop notification when command completes"
complete -c fabric -s h -l help -d "Show this help message"

View File

@@ -0,0 +1,183 @@
# Desktop Notifications
Fabric supports desktop notifications to alert you when commands complete, which is especially useful for long-running tasks or when you're multitasking.
## Quick Start
Enable notifications with the `--notification` flag:
```bash
fabric --pattern summarize --notification < article.txt
```
## Configuration
### Command Line Options
- `--notification`: Enable desktop notifications when command completes
- `--notification-command`: Use a custom notification command instead of built-in notifications
### YAML Configuration
Add notification settings to your `~/.config/fabric/config.yaml`:
```yaml
# Enable notifications by default
notification: true
# Optional: Custom notification command
notificationCommand: 'notify-send --urgency=normal "$1" "$2"'
```
## Platform Support
### macOS
- **Default**: Uses `osascript` (built into macOS)
- **Enhanced**: Install `terminal-notifier` for better notifications:
```bash
brew install terminal-notifier
```
### Linux
- **Requirement**: Install `notify-send`:
```bash
# Ubuntu/Debian
sudo apt install libnotify-bin
# Fedora
sudo dnf install libnotify
```
### Windows
- **Default**: Uses PowerShell message boxes (built-in)
## Custom Notification Commands
The `--notification-command` flag allows you to use custom notification scripts or commands. The command receives the title as `$1` and message as `$2` as shell positional arguments.
**Security Note**: The title and message content are properly escaped to prevent command injection attacks from AI-generated output containing shell metacharacters.
### Examples
**macOS with custom sound:**
```bash
fabric --pattern analyze_claims --notification-command 'osascript -e "display notification \"$2\" with title \"$1\" sound name \"Ping\""' < document.txt
```
**Linux with urgency levels:**
```bash
fabric --pattern extract_wisdom --notification-command 'notify-send --urgency=critical "$1" "$2"' < video-transcript.txt
```
**Custom script:**
```bash
fabric --pattern summarize --notification-command '/path/to/my-notification-script.sh "$1" "$2"' < report.pdf
```
**Testing your custom command:**
```bash
# Test that $1 and $2 are passed correctly
fabric --pattern raw_query --notification-command 'echo "Title: $1, Message: $2"' "test input"
```
## Notification Content
Notifications include:
- **Title**: "Fabric Command Complete" or "Fabric: [pattern] Complete"
- **Message**: Brief summary of the output (first 100 characters)
For long outputs, the message is truncated with "..." to fit notification display limits.
## Use Cases
### Long-Running Tasks
```bash
# Process large document with notifications
fabric --pattern analyze_paper --notification < research-paper.pdf
# Extract wisdom from long video with alerts
fabric -y "https://youtube.com/watch?v=..." --pattern extract_wisdom --notification
```
### Background Processing
```bash
# Process multiple files and get notified when each completes
for file in *.txt; do
fabric --pattern summarize --notification < "$file" &
done
```
### Integration with Other Tools
```bash
# Combine with other commands
curl -s "https://api.example.com/data" | \
fabric --pattern analyze_data --notification --output results.md
```
## Troubleshooting
### No Notifications Appearing
1. **Check system notifications are enabled** for Terminal/your shell
2. **Verify notification tools are installed**:
- macOS: `which osascript` (should exist)
- Linux: `which notify-send`
- Windows: `where.exe powershell`
3. **Test with simple command**:
```bash
echo "test" | fabric --pattern raw_query --notification --dry-run
```
### Notification Permission Issues
On some systems, you may need to grant notification permissions to your terminal application:
- **macOS**: System Preferences → Security & Privacy → Privacy → Notifications → Enable for Terminal
- **Linux**: Depends on desktop environment; usually automatic
- **Windows**: Usually works by default
### Custom Commands Not Working
- Ensure your custom notification command is executable
- Test the command manually with sample arguments
- Check that all required dependencies are installed
## Advanced Configuration
### Environment-Specific Settings
Create different configuration files for different environments:
```bash
# Work computer (quieter notifications)
fabric --config ~/.config/fabric/work-config.yaml --notification
# Personal computer (with sound)
fabric --config ~/.config/fabric/personal-config.yaml --notification
```
### Integration with Task Management
```bash
# Custom script that also logs to task management system
notificationCommand: '/usr/local/bin/fabric-notify-and-log.sh "$1" "$2"'
```
## Examples
See `docs/notification-config.yaml` for a complete configuration example with various notification command options.

298
docs/YouTube-Processing.md Normal file
View File

@@ -0,0 +1,298 @@
# YouTube Processing with Fabric
Fabric provides powerful YouTube video processing capabilities that allow you to extract transcripts, comments, and metadata from YouTube videos and playlists. This guide covers all the available options and common use cases.
## Prerequisites
- **yt-dlp**: Required for transcript extraction. Install on MacOS with:
```bash
brew install yt-dlp
```
Or use the package manager of your choice for your operating system.
See the [yt-dlp wiki page](https://github.com/yt-dlp/yt-dlp/wiki/Installation) for your specific installation instructions.
- **YouTube API Key** (optional): Only needed for comments and metadata extraction. Configure with:
```bash
fabric --setup
```
## Basic Usage
### Extract Transcript
Extract a video transcript and process it with a pattern:
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern summarize
```
### Extract Transcript with Timestamps
Get transcript with timestamps preserved:
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --transcript-with-timestamps --pattern extract_wisdom
```
### Extract Comments
Get video comments (requires YouTube API key):
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --comments --pattern analyze_claims
```
### Extract Metadata
Get video metadata as JSON:
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --metadata
```
## Advanced Options
### Custom yt-dlp Arguments
Pass additional arguments to yt-dlp for advanced functionality. **User-provided arguments take precedence** over built-in fabric arguments, giving you full control:
```bash
# Use browser cookies for age-restricted or private videos
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--cookies-from-browser brave"
# Override language selection (takes precedence over -g flag)
fabric -g en -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--sub-langs es,fr"
# Use specific format
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--format best"
# Handle rate limiting (slow down requests)
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--sleep-requests 1"
# Multiple arguments (use quotes)
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--cookies-from-browser firefox --write-info-json"
# Combine rate limiting with authentication
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--cookies-from-browser brave --sleep-requests 1"
# Override subtitle format (takes precedence over built-in --sub-format vtt)
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --yt-dlp-args "--sub-format srt"
```
#### Argument Precedence
Fabric constructs the yt-dlp command in this order:
1. **Built-in base arguments** (`--write-auto-subs`, `--skip-download`, etc.)
2. **Language selection** (from `-g` flag): `--sub-langs LANGUAGE`
3. **User arguments** (from `--yt-dlp-args`): **These override any conflicting built-in arguments**
4. **Video URL**
This means you can override any built-in behavior by specifying it in `--yt-dlp-args`.
### Playlist Processing
Process entire playlists:
```bash
# Process all videos in a playlist
fabric -y "https://www.youtube.com/playlist?list=PLAYLIST_ID" --playlist --pattern summarize
# Save playlist videos to CSV
fabric -y "https://www.youtube.com/playlist?list=PLAYLIST_ID" --playlist -o playlist.csv
```
### Language Support
Specify transcript language:
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" -g es --pattern translate
```
## Combining Options
You can combine multiple YouTube processing options:
```bash
# Get transcript, comments, and metadata
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" \
--transcript \
--comments \
--metadata \
--pattern comprehensive_analysis
```
## Output Options
### Save to File
```bash
# Save output to file
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern summarize -o summary.md
# Save entire session including input
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern summarize --output-session -o full_session.md
```
### Stream Output
Get real-time streaming output:
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern summarize --stream
```
## Common Use Cases
### Content Analysis
```bash
# Analyze video content for key insights
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern extract_wisdom
# Check claims made in the video
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern analyze_claims
```
### Educational Content
```bash
# Create study notes from educational videos
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern create_study_notes
# Extract key concepts and definitions
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern extract_concepts
```
### Meeting/Conference Processing
```bash
# Summarize conference talks with timestamps
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" \
--transcript-with-timestamps \
--pattern meeting_summary
# Extract action items from recorded meetings
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern extract_action_items
```
### Content Creation
```bash
# Create social media posts from video content
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern create_social_posts
# Generate blog post from video transcript
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" --pattern write_blog_post
```
## Troubleshooting
### Common Issues
1. **"yt-dlp not found"**: Install yt-dlp using pip or your package manager
2. **Age-restricted videos**: Use `--yt-dlp-args "--cookies-from-browser BROWSER"`
3. **No subtitles available**: Some videos don't have auto-generated subtitles
4. **API rate limits**: YouTube API has daily quotas for comments/metadata
5. **HTTP 429 errors**: YouTube is rate limiting subtitle requests
### Error Messages
- **"YouTube is not configured"**: Run `fabric --setup` to configure YouTube API
- **"yt-dlp failed"**: Check video URL and try with `--yt-dlp-args` for authentication
- **"No transcript content found"**: Video may not have subtitles available
- **"HTTP Error 429: Too Many Requests"**: YouTube rate limit exceeded. This is increasingly common. Solutions:
- **Wait 10-30 minutes and try again** (most effective)
- Use longer sleep: `--yt-dlp-args "--sleep-requests 5"`
- Try with browser cookies: `--yt-dlp-args "--cookies-from-browser brave --sleep-requests 5"`
- **Try a different video** - some videos are less restricted
- **Use a VPN** - different IP address may help
- **Try without language specification** - let yt-dlp choose any available language
- **Try English instead** - `fabric -g en` (English subtitles may be less rate-limited)
### Language Fallback Behavior
When you specify a language (e.g., `-g es` for Spanish) but that language isn't available or fails to download:
1. **Automatic fallback**: Fabric automatically retries without language specification
2. **Smart file detection**: If the fallback downloads a different language (e.g., English), Fabric will automatically detect and use it
3. **No manual intervention needed**: The process is transparent to the user
```bash
# Even if Spanish isn't available, this will work with whatever language yt-dlp finds
fabric -g es -y "https://youtube.com/watch?v=VIDEO_ID" --pattern summarize
```
## Configuration
### YAML Configuration
You can set default yt-dlp arguments in your config file (`~/.config/fabric/config.yaml`):
```yaml
ytDlpArgs: "--cookies-from-browser brave --write-info-json"
```
### Environment Variables
Set up your YouTube API key:
```bash
export FABRIC_YOUTUBE_API_KEY="your_api_key_here"
```
## Tips and Best Practices
1. **Use specific patterns**: Choose patterns that match your use case for better results
2. **Combine with other tools**: Pipe output to other commands or save to files for further processing
3. **Batch processing**: Use playlists to process multiple videos efficiently
4. **Authentication**: Use browser cookies for accessing private or age-restricted content
5. **Language support**: Specify language codes for better transcript accuracy
6. **Rate limiting**: If you encounter 429 errors, use `--sleep-requests 1` to slow down requests
7. **Persistent settings**: Set common yt-dlp args in your config file to avoid repeating them
8. **Argument precedence**: Use `--yt-dlp-args` to override any built-in behavior when needed
9. **Testing**: Use `yt-dlp --list-subs URL` to see available subtitle languages before processing
## Examples
### Quick Video Summary
```bash
fabric -y "https://www.youtube.com/watch?v=dQw4w9WgXcQ" --pattern summarize --stream
```
### Detailed Analysis with Authentication
```bash
fabric -y "https://www.youtube.com/watch?v=VIDEO_ID" \
--yt-dlp-args "--cookies-from-browser chrome" \
--transcript-with-timestamps \
--comments \
--pattern comprehensive_analysis \
-o analysis.md
```
### Playlist Processing
```bash
fabric -y "https://www.youtube.com/playlist?list=PLrAXtmRdnEQy6nuLvVUxpDnx4C0823vBN" \
--playlist \
--pattern extract_wisdom \
-o playlist_wisdom.md
```
### Override Built-in Language Selection
```bash
# Built-in language selection (-g es) is overridden by user args
fabric -g es -y "https://www.youtube.com/watch?v=VIDEO_ID" \
--yt-dlp-args "--sub-langs fr,de,en" \
--pattern translate
```
For more patterns and advanced usage, see the main [Fabric documentation](../README.md).

View File

@@ -0,0 +1,21 @@
# Example Fabric configuration with notification support
# Save this to ~/.config/fabric/config.yaml to use as defaults
# Enable notifications by default for all commands
notification: true
# Optional: Use a custom notification command
# Examples:
# macOS with custom sound:
# notificationCommand: 'osascript -e "display notification \"$2\" with title \"$1\" sound name \"Ping\""'
#
# Linux with custom urgency:
# notificationCommand: 'notify-send --urgency=normal "$1" "$2"'
#
# Custom script:
# notificationCommand: '/path/to/custom-notification-script.sh "$1" "$2"'
# Other common settings
model: "gpt-4o"
temperature: 0.7
stream: true

1
go.mod
View File

@@ -37,6 +37,7 @@ require (
require (
github.com/google/go-cmp v0.7.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
)
require (

2
go.sum
View File

@@ -155,6 +155,8 @@ github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=

View File

@@ -3,12 +3,14 @@ package cli
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/danielmiessler/fabric/internal/core"
"github.com/danielmiessler/fabric/internal/domain"
"github.com/danielmiessler/fabric/internal/plugins/db/fsdb"
"github.com/danielmiessler/fabric/internal/tools/notifications"
)
// handleChatProcessing handles the main chat processing logic
@@ -115,9 +117,65 @@ func handleChatProcessing(currentFlags *Flags, registry *core.PluginRegistry, me
}
}
}
// Send notification if requested
if chatOptions.Notification {
if err = sendNotification(chatOptions, chatReq.PatternName, result); err != nil {
// Log notification error but don't fail the main command
fmt.Fprintf(os.Stderr, "Failed to send notification: %v\n", err)
}
}
return
}
// sendNotification sends a desktop notification about command completion.
//
// When truncating the result for notification display, this function counts Unicode code points,
// not grapheme clusters. As a result, complex emoji or accented characters with multiple combining
// characters may be truncated improperly. This is a limitation of the current implementation.
func sendNotification(options *domain.ChatOptions, patternName, result string) error {
title := "Fabric Command Complete"
if patternName != "" {
title = fmt.Sprintf("Fabric: %s Complete", patternName)
}
// Limit message length for notification display (counts Unicode code points)
message := "Command completed successfully"
if result != "" {
maxLength := 100
runes := []rune(result)
if len(runes) > maxLength {
message = fmt.Sprintf("Output: %s...", string(runes[:maxLength]))
} else {
message = fmt.Sprintf("Output: %s", result)
}
// Clean up newlines for notification display
message = strings.ReplaceAll(message, "\n", " ")
}
// Use custom notification command if provided
if options.NotificationCommand != "" {
// SECURITY: Pass title and message as proper shell positional arguments $1 and $2
// This matches the documented interface where custom commands receive title and message as shell variables
cmd := exec.Command("sh", "-c", options.NotificationCommand+" \"$1\" \"$2\"", "--", title, message)
// For debugging: capture and display output from custom commands
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
// Use built-in notification system
notificationManager := notifications.NewNotificationManager()
if !notificationManager.IsAvailable() {
return fmt.Errorf("no notification system available")
}
return notificationManager.Send(title, message)
}
// isTTSModel checks if the model is a text-to-speech model
func isTTSModel(modelName string) bool {
lowerModel := strings.ToLower(modelName)

166
internal/cli/chat_test.go Normal file
View File

@@ -0,0 +1,166 @@
package cli
import (
"strings"
"testing"
"github.com/danielmiessler/fabric/internal/domain"
)
func TestSendNotification_SecurityEscaping(t *testing.T) {
tests := []struct {
name string
title string
message string
command string
expectError bool
description string
}{
{
name: "Normal content",
title: "Test Title",
message: "Test message content",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Normal content should work fine",
},
{
name: "Content with backticks",
title: "Test Title",
message: "Test `whoami` injection",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Backticks should be escaped and not executed",
},
{
name: "Content with semicolon injection",
title: "Test Title",
message: "Test; echo INJECTED; echo end",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Semicolon injection should be prevented",
},
{
name: "Content with command substitution",
title: "Test Title",
message: "Test $(whoami) injection",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Command substitution should be escaped",
},
{
name: "Content with quote injection",
title: "Test Title",
message: "Test ' || echo INJECTED || echo ' end",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Quote injection should be prevented",
},
{
name: "Content with newlines",
title: "Test Title",
message: "Line 1\nLine 2\nLine 3",
command: `echo "Title: $1, Message: $2"`,
expectError: false,
description: "Newlines should be handled safely",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
options := &domain.ChatOptions{
NotificationCommand: tt.command,
Notification: true,
}
// This test mainly verifies that the function doesn't panic
// and properly escapes dangerous content. The actual command
// execution is tested separately in integration tests.
err := sendNotification(options, "test_pattern", tt.message)
if tt.expectError && err == nil {
t.Errorf("Expected error for %s, but got none", tt.description)
}
if !tt.expectError && err != nil {
t.Errorf("Unexpected error for %s: %v", tt.description, err)
}
})
}
}
func TestSendNotification_TitleGeneration(t *testing.T) {
tests := []struct {
name string
patternName string
expected string
}{
{
name: "No pattern name",
patternName: "",
expected: "Fabric Command Complete",
},
{
name: "With pattern name",
patternName: "summarize",
expected: "Fabric: summarize Complete",
},
{
name: "Pattern with special characters",
patternName: "test_pattern-v2",
expected: "Fabric: test_pattern-v2 Complete",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
options := &domain.ChatOptions{
NotificationCommand: `echo "Title: $1"`,
Notification: true,
}
// We're testing the title generation logic
// The actual notification command would echo the title
err := sendNotification(options, tt.patternName, "test message")
// The function should not error for valid inputs
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
})
}
}
func TestSendNotification_MessageTruncation(t *testing.T) {
longMessage := strings.Repeat("A", 150) // 150 characters
shortMessage := "Short message"
tests := []struct {
name string
message string
expected string
}{
{
name: "Short message",
message: shortMessage,
},
{
name: "Long message truncation",
message: longMessage,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
options := &domain.ChatOptions{
NotificationCommand: `echo "Message: $2"`,
Notification: true,
}
err := sendNotification(options, "test", tt.message)
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
})
}
}

View File

@@ -113,11 +113,11 @@ func processYoutubeVideo(
}
}
if flags.YouTubeTranscriptWithTimestamps {
if transcript, err = registry.YouTube.GrabTranscriptWithTimestamps(videoId, language); err != nil {
if transcript, err = registry.YouTube.GrabTranscriptWithTimestampsWithArgs(videoId, language, flags.YtDlpArgs); err != nil {
return
}
} else {
if transcript, err = registry.YouTube.GrabTranscript(videoId, language); err != nil {
if transcript, err = registry.YouTube.GrabTranscriptWithArgs(videoId, language, flags.YtDlpArgs); err != nil {
return
}
}

View File

@@ -54,6 +54,7 @@ type Flags struct {
YouTubeTranscriptWithTimestamps bool `long:"transcript-with-timestamps" description:"Grab transcript from YouTube video with timestamps and send to chat"`
YouTubeComments bool `long:"comments" description:"Grab comments from YouTube video and send to chat"`
YouTubeMetadata bool `long:"metadata" description:"Output video metadata"`
YtDlpArgs string `long:"yt-dlp-args" yaml:"ytDlpArgs" description:"Additional arguments to pass to yt-dlp (e.g. '--cookies-from-browser brave')"`
Language string `short:"g" long:"language" description:"Specify the Language Code for the chat, e.g. -g=en -g=zh" default:""`
ScrapeURL string `short:"u" long:"scrape_url" description:"Scrape website URL to markdown using Jina AI"`
ScrapeQuestion string `short:"q" long:"scrape_question" description:"Search question using Jina AI"`
@@ -91,6 +92,8 @@ type Flags struct {
DisableResponsesAPI bool `long:"disable-responses-api" yaml:"disableResponsesAPI" description:"Disable OpenAI Responses API (default: false)"`
Voice string `long:"voice" yaml:"voice" description:"TTS voice name for supported models (e.g., Kore, Charon, Puck)" default:"Kore"`
ListGeminiVoices bool `long:"list-gemini-voices" description:"List all available Gemini TTS voices"`
Notification bool `long:"notification" yaml:"notification" description:"Send desktop notification when command completes"`
NotificationCommand string `long:"notification-command" yaml:"notificationCommand" description:"Custom command to run for notifications (overrides built-in notifications)"`
}
var debug = false
@@ -427,25 +430,27 @@ func (o *Flags) BuildChatOptions() (ret *domain.ChatOptions, err error) {
}
ret = &domain.ChatOptions{
Model: o.Model,
Temperature: o.Temperature,
TopP: o.TopP,
PresencePenalty: o.PresencePenalty,
FrequencyPenalty: o.FrequencyPenalty,
Raw: o.Raw,
Seed: o.Seed,
ModelContextLength: o.ModelContextLength,
Search: o.Search,
SearchLocation: o.SearchLocation,
ImageFile: o.ImageFile,
ImageSize: o.ImageSize,
ImageQuality: o.ImageQuality,
ImageCompression: o.ImageCompression,
ImageBackground: o.ImageBackground,
SuppressThink: o.SuppressThink,
ThinkStartTag: startTag,
ThinkEndTag: endTag,
Voice: o.Voice,
Model: o.Model,
Temperature: o.Temperature,
TopP: o.TopP,
PresencePenalty: o.PresencePenalty,
FrequencyPenalty: o.FrequencyPenalty,
Raw: o.Raw,
Seed: o.Seed,
ModelContextLength: o.ModelContextLength,
Search: o.Search,
SearchLocation: o.SearchLocation,
ImageFile: o.ImageFile,
ImageSize: o.ImageSize,
ImageQuality: o.ImageQuality,
ImageCompression: o.ImageCompression,
ImageBackground: o.ImageBackground,
SuppressThink: o.SuppressThink,
ThinkStartTag: startTag,
ThinkEndTag: endTag,
Voice: o.Voice,
Notification: o.Notification || o.NotificationCommand != "",
NotificationCommand: o.NotificationCommand,
}
return
}

View File

@@ -25,28 +25,30 @@ type ChatRequest struct {
}
type ChatOptions struct {
Model string
Temperature float64
TopP float64
PresencePenalty float64
FrequencyPenalty float64
Raw bool
Seed int
ModelContextLength int
MaxTokens int
Search bool
SearchLocation string
ImageFile string
ImageSize string
ImageQuality string
ImageCompression int
ImageBackground string
SuppressThink bool
ThinkStartTag string
ThinkEndTag string
AudioOutput bool
AudioFormat string
Voice string
Model string
Temperature float64
TopP float64
PresencePenalty float64
FrequencyPenalty float64
Raw bool
Seed int
ModelContextLength int
MaxTokens int
Search bool
SearchLocation string
ImageFile string
ImageSize string
ImageQuality string
ImageCompression int
ImageBackground string
SuppressThink bool
ThinkStartTag string
ThinkEndTag string
AudioOutput bool
AudioFormat string
Voice string
Notification bool
NotificationCommand string
}
// NormalizeMessages remove empty messages and ensure messages order user-assist-user

View File

@@ -336,6 +336,19 @@ func (o *Client) convertMessages(msgs []*chat.ChatCompletionMessage) []*genai.Co
for _, msg := range msgs {
content := &genai.Content{Parts: []*genai.Part{}}
switch msg.Role {
case chat.ChatMessageRoleAssistant:
content.Role = "model"
case chat.ChatMessageRoleUser:
content.Role = "user"
case chat.ChatMessageRoleSystem, chat.ChatMessageRoleDeveloper, chat.ChatMessageRoleFunction, chat.ChatMessageRoleTool:
// Gemini's API only accepts "user" and "model" roles.
// Map all other roles to "user" to preserve instruction context.
content.Role = "user"
default:
content.Role = "user"
}
if msg.Content != "" {
content.Parts = append(content.Parts, &genai.Part{Text: msg.Content})
}

View File

@@ -4,6 +4,8 @@ import (
"testing"
"google.golang.org/genai"
"github.com/danielmiessler/fabric/internal/chat"
)
// Test buildModelNameFull method
@@ -51,6 +53,30 @@ func TestExtractTextFromResponse(t *testing.T) {
}
}
// Test convertMessages handles role mapping correctly
func TestConvertMessagesRoles(t *testing.T) {
client := &Client{}
msgs := []*chat.ChatCompletionMessage{
{Role: chat.ChatMessageRoleUser, Content: "user"},
{Role: chat.ChatMessageRoleAssistant, Content: "assistant"},
{Role: chat.ChatMessageRoleSystem, Content: "system"},
}
contents := client.convertMessages(msgs)
expected := []string{"user", "model", "user"}
if len(contents) != len(expected) {
t.Fatalf("expected %d contents, got %d", len(expected), len(contents))
}
for i, c := range contents {
if c.Role != expected[i] {
t.Errorf("content %d expected role %s, got %s", i, expected[i], c.Role)
}
}
}
// Test isTTSModel method
func TestIsTTSModel(t *testing.T) {
client := &Client{}

View File

@@ -115,7 +115,11 @@ func (o *Client) sendStreamResponses(
case string(constant.ResponseOutputTextDelta("").Default()):
channel <- event.AsResponseOutputTextDelta().Delta
case string(constant.ResponseOutputTextDone("").Default()):
channel <- event.AsResponseOutputTextDone().Text
// The Responses API sends the full text again in the
// final "done" event. Since we've already streamed all
// delta chunks above, sending it would duplicate the
// output. Ignore it here to prevent doubled results.
continue
}
}
if stream.Err() == nil {

View File

@@ -0,0 +1,128 @@
package notifications
import (
"fmt"
"os"
"os/exec"
"runtime"
)
// NotificationProvider interface for different notification backends
type NotificationProvider interface {
Send(title, message string) error
IsAvailable() bool
}
// NotificationManager handles cross-platform notifications
type NotificationManager struct {
provider NotificationProvider
}
// NewNotificationManager creates a new notification manager with the best available provider
func NewNotificationManager() *NotificationManager {
var provider NotificationProvider
switch runtime.GOOS {
case "darwin":
// Try terminal-notifier first, then fall back to osascript
provider = &TerminalNotifierProvider{}
if !provider.IsAvailable() {
provider = &OSAScriptProvider{}
}
case "linux":
provider = &NotifySendProvider{}
case "windows":
provider = &PowerShellProvider{}
default:
provider = &NoopProvider{}
}
return &NotificationManager{provider: provider}
}
// Send sends a notification using the configured provider
func (nm *NotificationManager) Send(title, message string) error {
if nm.provider == nil {
return fmt.Errorf("no notification provider available")
}
return nm.provider.Send(title, message)
}
// IsAvailable checks if notifications are available
func (nm *NotificationManager) IsAvailable() bool {
return nm.provider != nil && nm.provider.IsAvailable()
}
// macOS terminal-notifier implementation
type TerminalNotifierProvider struct{}
func (t *TerminalNotifierProvider) Send(title, message string) error {
cmd := exec.Command("terminal-notifier", "-title", title, "-message", message, "-sound", "Glass")
return cmd.Run()
}
func (t *TerminalNotifierProvider) IsAvailable() bool {
_, err := exec.LookPath("terminal-notifier")
return err == nil
}
// macOS osascript implementation
type OSAScriptProvider struct{}
func (o *OSAScriptProvider) Send(title, message string) error {
// SECURITY: Use separate arguments instead of string interpolation to prevent AppleScript injection
script := `display notification (system attribute "FABRIC_MESSAGE") with title (system attribute "FABRIC_TITLE") sound name "Glass"`
cmd := exec.Command("osascript", "-e", script)
// Set environment variables for the AppleScript to read safely
cmd.Env = append(os.Environ(), "FABRIC_TITLE="+title, "FABRIC_MESSAGE="+message)
return cmd.Run()
}
func (o *OSAScriptProvider) IsAvailable() bool {
_, err := exec.LookPath("osascript")
return err == nil
}
// Linux notify-send implementation
type NotifySendProvider struct{}
func (n *NotifySendProvider) Send(title, message string) error {
cmd := exec.Command("notify-send", title, message)
return cmd.Run()
}
func (n *NotifySendProvider) IsAvailable() bool {
_, err := exec.LookPath("notify-send")
return err == nil
}
// Windows PowerShell implementation
type PowerShellProvider struct{}
func (p *PowerShellProvider) Send(title, message string) error {
// SECURITY: Use environment variables to avoid PowerShell injection attacks
script := `Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.MessageBox]::Show($env:FABRIC_MESSAGE, $env:FABRIC_TITLE)`
cmd := exec.Command("powershell", "-Command", script)
// Set environment variables for PowerShell to read safely
cmd.Env = append(os.Environ(), "FABRIC_TITLE="+title, "FABRIC_MESSAGE="+message)
return cmd.Run()
}
func (p *PowerShellProvider) IsAvailable() bool {
_, err := exec.LookPath("powershell")
return err == nil
}
// NoopProvider for unsupported platforms
type NoopProvider struct{}
func (n *NoopProvider) Send(title, message string) error {
// Silent no-op for unsupported platforms
return nil
}
func (n *NoopProvider) IsAvailable() bool {
return false
}

View File

@@ -0,0 +1,168 @@
package notifications
import (
"os/exec"
"runtime"
"testing"
)
func TestNewNotificationManager(t *testing.T) {
manager := NewNotificationManager()
if manager == nil {
t.Fatal("NewNotificationManager() returned nil")
}
if manager.provider == nil {
t.Fatal("NotificationManager provider is nil")
}
}
func TestNotificationManagerIsAvailable(t *testing.T) {
manager := NewNotificationManager()
// Should not panic
_ = manager.IsAvailable()
}
func TestNotificationManagerSend(t *testing.T) {
manager := NewNotificationManager()
// Test sending notification - this may fail on systems without notification tools
// but should not panic
err := manager.Send("Test Title", "Test Message")
if err != nil {
t.Logf("Notification send failed (expected on systems without notification tools): %v", err)
}
}
func TestTerminalNotifierProvider(t *testing.T) {
if runtime.GOOS != "darwin" {
t.Skip("Skipping macOS terminal-notifier test on non-macOS platform")
}
provider := &TerminalNotifierProvider{}
// Test availability - depends on whether terminal-notifier is installed
available := provider.IsAvailable()
t.Logf("terminal-notifier available: %v", available)
if available {
err := provider.Send("Test", "Test message")
if err != nil {
t.Logf("terminal-notifier send failed: %v", err)
}
}
}
func TestOSAScriptProvider(t *testing.T) {
if runtime.GOOS != "darwin" {
t.Skip("Skipping macOS osascript test on non-macOS platform")
}
provider := &OSAScriptProvider{}
// osascript should always be available on macOS
if !provider.IsAvailable() {
t.Error("osascript should be available on macOS")
}
// Test sending (may show actual notification)
err := provider.Send("Test", "Test message")
if err != nil {
t.Errorf("osascript send failed: %v", err)
}
}
func TestNotifySendProvider(t *testing.T) {
if runtime.GOOS != "linux" {
t.Skip("Skipping Linux notify-send test on non-Linux platform")
}
provider := &NotifySendProvider{}
// Test availability - depends on whether notify-send is installed
available := provider.IsAvailable()
t.Logf("notify-send available: %v", available)
if available {
err := provider.Send("Test", "Test message")
if err != nil {
t.Logf("notify-send send failed: %v", err)
}
}
}
func TestPowerShellProvider(t *testing.T) {
if runtime.GOOS != "windows" {
t.Skip("Skipping Windows PowerShell test on non-Windows platform")
}
provider := &PowerShellProvider{}
// PowerShell should be available on Windows
if !provider.IsAvailable() {
t.Error("PowerShell should be available on Windows")
}
// Note: This will show a message box if run
// In CI/CD, this might not work properly
err := provider.Send("Test", "Test message")
if err != nil {
t.Logf("PowerShell send failed (expected in headless environments): %v", err)
}
}
func TestNoopProvider(t *testing.T) {
provider := &NoopProvider{}
// Should always report as not available
if provider.IsAvailable() {
t.Error("NoopProvider should report as not available")
}
// Should never error
err := provider.Send("Test", "Test message")
if err != nil {
t.Errorf("NoopProvider send should never error, got: %v", err)
}
}
func TestProviderIsAvailable(t *testing.T) {
tests := []struct {
name string
provider NotificationProvider
command string
}{
{"TerminalNotifier", &TerminalNotifierProvider{}, "terminal-notifier"},
{"OSAScript", &OSAScriptProvider{}, "osascript"},
{"NotifySend", &NotifySendProvider{}, "notify-send"},
{"PowerShell", &PowerShellProvider{}, "powershell"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
available := tt.provider.IsAvailable()
// Cross-check with actual command availability
_, err := exec.LookPath(tt.command)
expectedAvailable := err == nil
if available != expectedAvailable {
t.Logf("Provider %s availability mismatch: provider=%v, command=%v",
tt.name, available, expectedAvailable)
// This is informational, not a failure, since system setup varies
}
})
}
}
func TestSendWithSpecialCharacters(t *testing.T) {
manager := NewNotificationManager()
// Test with special characters that might break shell commands
specialTitle := `Title with "quotes" and 'apostrophes'`
specialMessage := `Message with \backslashes and $variables and "quotes"`
err := manager.Send(specialTitle, specialMessage)
if err != nil {
t.Logf("Send with special characters failed (may be expected): %v", err)
}
}

View File

@@ -25,17 +25,33 @@ import (
"time"
"github.com/danielmiessler/fabric/internal/plugins"
"github.com/kballard/go-shellquote"
"google.golang.org/api/option"
"google.golang.org/api/youtube/v3"
)
var timestampRegex *regexp.Regexp
var languageFileRegex *regexp.Regexp
var videoPatternRegex *regexp.Regexp
var playlistPatternRegex *regexp.Regexp
var vttTagRegex *regexp.Regexp
var durationRegex *regexp.Regexp
const TimeGapForRepeats = 10 // seconds
func init() {
// Match timestamps like "00:00:01.234" or just numbers or sequence numbers
timestampRegex = regexp.MustCompile(`^\d+$|^\d{1,2}:\d{2}(:\d{2})?(\.\d{3})?$`)
// Match language-specific VTT files like .en.vtt, .es.vtt, .en-US.vtt, .pt-BR.vtt
languageFileRegex = regexp.MustCompile(`\.[a-z]{2}(-[A-Z]{2})?\.vtt$`)
// YouTube video ID pattern
videoPatternRegex = regexp.MustCompile(`(?:https?:\/\/)?(?:www\.)?(?:youtube\.com\/(?:live\/|[^\/\n\s]+\/\S+\/|(?:v|e(?:mbed)?)\/|(?:s(?:horts)\/)|\S*?[?&]v=)|youtu\.be\/)([a-zA-Z0-9_-]*)`)
// YouTube playlist ID pattern
playlistPatternRegex = regexp.MustCompile(`[?&]list=([a-zA-Z0-9_-]+)`)
// VTT formatting tags like <c.colorE5E5E5>, </c>, etc.
vttTagRegex = regexp.MustCompile(`<[^>]*>`)
// YouTube duration format PT1H2M3S
durationRegex = regexp.MustCompile(`(?i)PT(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?`)
}
func NewYouTube() (ret *YouTube) {
@@ -76,18 +92,14 @@ func (o *YouTube) initService() (err error) {
}
func (o *YouTube) GetVideoOrPlaylistId(url string) (videoId string, playlistId string, err error) {
// Video ID pattern
videoPattern := `(?:https?:\/\/)?(?:www\.)?(?:youtube\.com\/(?:live\/|[^\/\n\s]+\/\S+\/|(?:v|e(?:mbed)?)\/|(?:s(?:horts)\/)|\S*?[?&]v=)|youtu\.be\/)([a-zA-Z0-9_-]*)`
videoRe := regexp.MustCompile(videoPattern)
videoMatch := videoRe.FindStringSubmatch(url)
// Extract video ID using pre-compiled regex
videoMatch := videoPatternRegex.FindStringSubmatch(url)
if len(videoMatch) > 1 {
videoId = videoMatch[1]
}
// Playlist ID pattern
playlistPattern := `[?&]list=([a-zA-Z0-9_-]+)`
playlistRe := regexp.MustCompile(playlistPattern)
playlistMatch := playlistRe.FindStringSubmatch(url)
// Extract playlist ID using pre-compiled regex
playlistMatch := playlistPatternRegex.FindStringSubmatch(url)
if len(playlistMatch) > 1 {
playlistId = playlistMatch[1]
}
@@ -113,17 +125,27 @@ func (o *YouTube) GrabTranscriptForUrl(url string, language string) (ret string,
func (o *YouTube) GrabTranscript(videoId string, language string) (ret string, err error) {
// Use yt-dlp for reliable transcript extraction
return o.tryMethodYtDlp(videoId, language)
return o.GrabTranscriptWithArgs(videoId, language, "")
}
func (o *YouTube) GrabTranscriptWithArgs(videoId string, language string, additionalArgs string) (ret string, err error) {
// Use yt-dlp for reliable transcript extraction
return o.tryMethodYtDlp(videoId, language, additionalArgs)
}
func (o *YouTube) GrabTranscriptWithTimestamps(videoId string, language string) (ret string, err error) {
// Use yt-dlp for reliable transcript extraction with timestamps
return o.tryMethodYtDlpWithTimestamps(videoId, language)
return o.GrabTranscriptWithTimestampsWithArgs(videoId, language, "")
}
func (o *YouTube) GrabTranscriptWithTimestampsWithArgs(videoId string, language string, additionalArgs string) (ret string, err error) {
// Use yt-dlp for reliable transcript extraction with timestamps
return o.tryMethodYtDlpWithTimestamps(videoId, language, additionalArgs)
}
// tryMethodYtDlpInternal is a helper function to reduce duplication between
// tryMethodYtDlp and tryMethodYtDlpWithTimestamps.
func (o *YouTube) tryMethodYtDlpInternal(videoId string, language string, processVTTFileFunc func(filename string) (string, error)) (ret string, err error) {
func (o *YouTube) tryMethodYtDlpInternal(videoId string, language string, additionalArgs string, processVTTFileFunc func(filename string) (string, error)) (ret string, err error) {
// Check if yt-dlp is available
if _, err = exec.LookPath("yt-dlp"); err != nil {
err = fmt.Errorf("yt-dlp not found in PATH. Please install yt-dlp to use YouTube transcript functionality")
@@ -141,30 +163,93 @@ func (o *YouTube) tryMethodYtDlpInternal(videoId string, language string, proces
// Use yt-dlp to get transcript
videoURL := "https://www.youtube.com/watch?v=" + videoId
outputPath := filepath.Join(tempDir, "%(title)s.%(ext)s")
lang_match := language
if len(language) > 2 {
lang_match = language[:2]
}
cmd := exec.Command("yt-dlp",
baseArgs := []string{
"--write-auto-subs",
"--sub-lang", lang_match,
"--skip-download",
"--sub-format", "vtt",
"--quiet",
"--no-warnings",
"-o", outputPath,
videoURL)
}
args := append([]string{}, baseArgs...)
// Add built-in language selection first
if language != "" {
langMatch := language
if len(langMatch) > 2 {
langMatch = langMatch[:2]
}
args = append(args, "--sub-langs", langMatch)
}
// Add user-provided arguments last so they take precedence
if additionalArgs != "" {
additionalArgsList, err := shellquote.Split(additionalArgs)
if err != nil {
return "", fmt.Errorf("invalid yt-dlp arguments: %v", err)
}
args = append(args, additionalArgsList...)
}
args = append(args, videoURL)
cmd := exec.Command("yt-dlp", args...)
var stderr bytes.Buffer
cmd.Stderr = &stderr
if err = cmd.Run(); err != nil {
err = fmt.Errorf("yt-dlp failed: %v, stderr: %s", err, stderr.String())
return
stderrStr := stderr.String()
// Check for specific YouTube errors
if strings.Contains(stderrStr, "429") || strings.Contains(stderrStr, "Too Many Requests") {
err = fmt.Errorf("YouTube rate limit exceeded. Try again later or use different yt-dlp arguments like '--sleep-requests 1' to slow down requests. Error: %v", err)
return
}
if strings.Contains(stderrStr, "Sign in to confirm you're not a bot") || strings.Contains(stderrStr, "Use --cookies-from-browser") {
err = fmt.Errorf("YouTube requires authentication (bot detection). Use --yt-dlp-args '--cookies-from-browser BROWSER' where BROWSER is chrome, firefox, brave, etc. Error: %v", err)
return
}
if language != "" {
// Fallback: try without specifying language (let yt-dlp choose best available)
stderr.Reset()
fallbackArgs := append([]string{}, baseArgs...)
// Add additional arguments if provided
if additionalArgs != "" {
additionalArgsList, parseErr := shellquote.Split(additionalArgs)
if parseErr != nil {
return "", fmt.Errorf("invalid yt-dlp arguments: %v", parseErr)
}
fallbackArgs = append(fallbackArgs, additionalArgsList...)
}
// Don't specify language, let yt-dlp choose
fallbackArgs = append(fallbackArgs, videoURL)
cmd = exec.Command("yt-dlp", fallbackArgs...)
cmd.Stderr = &stderr
if err = cmd.Run(); err != nil {
stderrStr2 := stderr.String()
if strings.Contains(stderrStr2, "429") || strings.Contains(stderrStr2, "Too Many Requests") {
err = fmt.Errorf("YouTube rate limit exceeded. Try again later or use different yt-dlp arguments like '--sleep-requests 1'. Error: %v", err)
} else {
err = fmt.Errorf("yt-dlp failed with language '%s' and fallback. Original error: %s. Fallback error: %s", language, stderrStr, stderrStr2)
}
return
}
} else {
err = fmt.Errorf("yt-dlp failed: %v, stderr: %s", err, stderrStr)
return
}
}
// Find VTT files using cross-platform approach
vttFiles, err := o.findVTTFiles(tempDir, language)
// Try to find files with the requested language first, but fall back to any VTT file
vttFiles, err := o.findVTTFilesWithFallback(tempDir, language)
if err != nil {
return "", err
}
@@ -172,12 +257,12 @@ func (o *YouTube) tryMethodYtDlpInternal(videoId string, language string, proces
return processVTTFileFunc(vttFiles[0])
}
func (o *YouTube) tryMethodYtDlp(videoId string, language string) (ret string, err error) {
return o.tryMethodYtDlpInternal(videoId, language, o.readAndCleanVTTFile)
func (o *YouTube) tryMethodYtDlp(videoId string, language string, additionalArgs string) (ret string, err error) {
return o.tryMethodYtDlpInternal(videoId, language, additionalArgs, o.readAndCleanVTTFile)
}
func (o *YouTube) tryMethodYtDlpWithTimestamps(videoId string, language string) (ret string, err error) {
return o.tryMethodYtDlpInternal(videoId, language, o.readAndFormatVTTWithTimestamps)
func (o *YouTube) tryMethodYtDlpWithTimestamps(videoId string, language string, additionalArgs string) (ret string, err error) {
return o.tryMethodYtDlpInternal(videoId, language, additionalArgs, o.readAndFormatVTTWithTimestamps)
}
func (o *YouTube) readAndCleanVTTFile(filename string) (ret string, err error) {
@@ -303,8 +388,7 @@ func isTimeStamp(s string) bool {
func removeVTTTags(s string) string {
// Remove VTT tags like <c.colorE5E5E5>, </c>, etc.
tagRegex := regexp.MustCompile(`<[^>]*>`)
return tagRegex.ReplaceAllString(s, "")
return vttTagRegex.ReplaceAllString(s, "")
}
// shouldIncludeRepeat determines if repeated content should be included based on time gap
@@ -428,7 +512,7 @@ func (o *YouTube) GrabDuration(videoId string) (ret int, err error) {
durationStr := videoResponse.Items[0].ContentDetails.Duration
matches := regexp.MustCompile(`(?i)PT(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?`).FindStringSubmatch(durationStr)
matches := durationRegex.FindStringSubmatch(durationStr)
if len(matches) == 0 {
return 0, fmt.Errorf("invalid duration string: %s", durationStr)
}
@@ -588,8 +672,9 @@ func (o *YouTube) normalizeFileName(name string) string {
}
// findVTTFiles searches for VTT files in a directory using cross-platform approach
func (o *YouTube) findVTTFiles(dir, language string) ([]string, error) {
// findVTTFilesWithFallback searches for VTT files, handling fallback scenarios
// where the requested language might not be available
func (o *YouTube) findVTTFilesWithFallback(dir, requestedLanguage string) ([]string, error) {
var vttFiles []string
// Walk through the directory to find VTT files
@@ -612,14 +697,28 @@ func (o *YouTube) findVTTFiles(dir, language string) ([]string, error) {
return nil, fmt.Errorf("no VTT files found in directory")
}
// Prefer files with the specified language
// If no specific language requested, return the first file
if requestedLanguage == "" {
return []string{vttFiles[0]}, nil
}
// First, try to find files with the requested language
for _, file := range vttFiles {
if strings.Contains(file, "."+language+".vtt") {
if strings.Contains(file, "."+requestedLanguage+".vtt") {
return []string{file}, nil
}
}
// Return the first VTT file found if no language-specific file exists
// If requested language not found, check if we have any language-specific files
// This handles the fallback case where yt-dlp downloaded a different language
for _, file := range vttFiles {
// Look for any language pattern (e.g., .en.vtt, .es.vtt, etc.)
if languageFileRegex.MatchString(file) {
return []string{file}, nil
}
}
// If no language-specific files found, return the first VTT file
return []string{vttFiles[0]}, nil
}

View File

@@ -199,6 +199,9 @@ schema = 3
[mod."github.com/json-iterator/go"]
version = "v1.1.12"
hash = "sha256-To8A0h+lbfZ/6zM+2PpRpY3+L6725OPC66lffq6fUoM="
[mod."github.com/kballard/go-shellquote"]
version = "v0.0.0-20180428030007-95032a82bc51"
hash = "sha256-AOEdKETBMUC39ln6jBJ9NYdJWp++jV5lSbjNqG3dV+c="
[mod."github.com/kevinburke/ssh_config"]
version = "v1.2.0"
hash = "sha256-Ta7ZOmyX8gG5tzWbY2oES70EJPfI90U7CIJS9EAce0s="

View File

@@ -1 +1 @@
"1.4.275"
"1.4.280"