Compare commits

...

18 Commits

Author SHA1 Message Date
github-actions[bot]
bd39d98ffc Update version to v1.4.150 and commit 2025-03-07 07:44:52 +00:00
Eugen Eisler
36b0afa692 Merge pull request #1343 from jmd1010/input-component-update
Rename input.svelte to Input.svelte for proper component naming convention
2025-03-07 08:43:32 +01:00
jmd1010
53d09d8a5a Rename input.svelte to Input.svelte for proper component naming convention 2025-03-06 21:08:30 -05:00
github-actions[bot]
703144edad Update version to v1.4.149 and commit 2025-03-05 12:50:38 +00:00
Eugen Eisler
717e50e570 Merge pull request #1340 from ksylvan/03-04-youtube-live-fix
Fix for youtube live links plus new youtube_summary pattern
2025-03-05 13:49:22 +01:00
Eugen Eisler
4af6d5eeed Merge pull request #1338 from jmd1010/directory-structure-update
Update Web V2 Install Guide layout
2025-03-05 10:09:23 +01:00
Kayvan Sylvan
de356ddeb5 chore: update version 2025-03-04 22:00:26 -08:00
Kayvan Sylvan
59c50def2a feat: update yt commands and docs to support timestamped transcripts
CHANGES
- Add argument validation to yt for usage errors
- Enable -t flag for transcript with timestamps
- Refactor PowerShell yt function with parameter switch
- Update README to dynamically select transcript option
- Document youtube_summary feature in pattern explanations
- Introduce youtube_summary pattern.
2025-03-04 21:46:50 -08:00
Kayvan Sylvan
1dafb09e07 remove spurious newline 2025-03-04 21:20:24 -08:00
Kayvan Sylvan
e8caf9fc10 feat: update YouTube regex to support live URLs 2025-03-04 21:18:51 -08:00
jmd1010
59537c4bf5 Update Web V2 Install Guide layout 2 2025-03-04 12:29:31 -05:00
jmd1010
2f4569177d Update Web V2 Install Guide layout 2025-03-04 10:58:48 -05:00
Eugen Eisler
f2b85af0ea Merge pull request #1330 from jmd1010/directory-structure-update
Fixed ALL CAP DIR as requested and processed minor updates to documentation
2025-03-04 12:17:03 +01:00
Eugen Eisler
36be4c747c Merge pull request #1333 from asasidh/main
Update QUOTES section to include speaker names for clarity
2025-03-04 09:35:53 +01:00
asasidh
be6049e577 Update QUOTES section to include speaker names for clarity 2025-03-02 17:42:52 -06:00
jmd1010
a8ae09d4d6 Update Web V2 Install Guide with improved instructions V2 2025-02-28 19:57:13 -05:00
jmd1010
30059c46a0 Update Web V2 Install Guide with improved instructions 2025-02-28 13:23:43 -05:00
jmd1010
2d10c71e39 Reorganize documentation with consistent directory naming and updated guides 2025-02-28 11:34:17 -05:00
12 changed files with 445 additions and 34 deletions

View File

@@ -204,8 +204,19 @@ for pattern_file in $HOME/.config/fabric/patterns/*; do
done
yt() {
if [ "$#" -eq 0 ] || [ "$#" -gt 2 ]; then
echo "Usage: yt [-t | --timestamps] youtube-link"
echo "Use the '-t' flag to get the transcript with timestamps."
return 1
fi
transcript_flag="--transcript"
if [ "$1" = "-t" ] || [ "$1" = "--timestamps" ]; then
transcript_flag="--transcript-with-timestamps"
shift
fi
local video_link="$1"
fabric -y "$video_link" --transcript
fabric -y "$video_link" $transcript_flag
}
```
@@ -263,10 +274,34 @@ function $patternName {
function yt {
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[Parameter()]
[Alias("timestamps")]
[switch]$t,
[Parameter(Position = 0, ValueFromPipeline = $true)]
[string]$videoLink
)
fabric -y $videoLink --transcript
begin {
$transcriptFlag = "--transcript"
if ($t) {
$transcriptFlag = "--transcript-with-timestamps"
}
}
process {
if (-not $videoLink) {
Write-Error "Usage: yt [-t | --timestamps] youtube-link"
return
}
}
end {
if ($videoLink) {
# Execute and allow output to flow through the pipeline
fabric -y $videoLink $transcriptFlag
}
}
}
```

View File

@@ -1,41 +1,43 @@
# How to Install Fabric, Web Interface and PDF-to-Markdown
# How to Install the Web Interface and PDF-to-Markdown
Until it is fully integrated in GO, here's a safe way to test Web V2.
If Fabric is already installed and you see fabric/web, go to step 3
## Step 1: Clone the Repository
If fabric is not installed, ensure Go is installed https://go.dev/doc/install and node / npm for web https://nodejs.org/en/download.
Clone the fabric repository to your desired location:
There are many ways to install fabric. Here's one approach that usually works well:
```bash
# For Mac (specific location)
git clone https://github.com/danielmiessler/fabric.git ~/Documents/Github/fabric
# Or from any parent directory
## Step 1: clone the repo
In terminal, from the parent directory where you want to install fabric:
git clone https://github.com/danielmiessler/fabric.git
## Step 2 : Install Fabric
cd fabric
go install github.com/danielmiessler/fabric@latest
Step 2: Install GUI
## Step 3: Install GUI
Navigate to the web directory and install dependencies:
cd web
npm install
npx svelte-kit sync
Step 3: Install PDF-to-Markdown
## Step 4: Install PDF-to-Markdown
Install the PDF conversion components in the correct order:
# Make sure you're in the fabric/web directory
cd fabric/web
cd web
# Install dependencies in this specific order
npm install -D patch-package
npm install -D pdfjs-dist@2.5.207
npm install -D github:jzillmann/pdf-to-markdown#modularize
No build step is required after installation.
Step 4: Update Shell Configuration
## Step 5: Update Shell Configuration if not already done from your fabric installation
For Mac/Linux users:
Add environment variables to your ~/.bashrc (Linux) or ~/.zshrc (Mac) file:
@@ -55,6 +57,7 @@ REFER TO OFFICIAL FABRIC README.MD FILE FOR OTHER OPERATING SYSTEMS
Step 5: Create Aliases for Patterns
Add the following to your .zshrc or .bashrc file to create shorter commands:
```bash
# Define the base directory for Obsidian notes,
obsidian_base="/path/to/obsidian"
@@ -111,6 +114,10 @@ fabric --serve
Terminal 2: Start the Development Server
npm run dev
If you get an ** ERROR **.
It would be much appreciated that you copy /paste your error in your favorite LLM before opening a ticket, 90% of the time your llm will point you to the solution.
_____ ______ ______
OPTIONAL: Create Start/Stop Scripts
@@ -188,4 +195,9 @@ chmod +x ~/Desktop/stop-fabric.command
You can customize with icons by finding suitable .icns files, right-clicking each .command file, selecting "Get Info", and dragging your icon file onto the small icon in the top-left corner.
Note: You might need to allow the scripts to execute in your security settings by going to System Preferences → Security & Privacy after trying to run them the first time.
Note: You might need to allow the scripts to execute in your security settings by going to System Preferences → Security & Privacy after trying to run them the first time.
## 🎥 Demo Video
https://youtu.be/XMzjgqvdltM

View File

@@ -1,21 +1,20 @@
https://youtu.be/bhwtWXoMASA# Enhanced Pattern Selection,
Pattern Descriptions, New
Pattern TAG System, Language Support and other WEB UI Improvements V3
This Cummulative PR adds several Web UI and functionality improvements to make pattern selection more intuitive (pattern descriptions), ability to save favorite patterns, powerful multilingual capabilities, a Pattern TAG system, a help reference section, more robust Youtube processing and a variety of ui improvements.
This Cummulative PR adds several Web UI and functionality improvements to make pattern selection more intuitive with the addition of pattern descriptions, ability to save favorite patterns, a Pattern TAG system, powerful multilingual capabilities, PDF-to-markdown functionnalities, a help reference section, more robust Youtube processing and a variety of other ui improvements.
## 🎥 Demo Video
https://youtu.be/IhE8Iey8hSU
https://youtu.be/XMzjgqvdltM
## 🌟 Key Features
### 1. Web UI and Pattern Selection Improvements
- Enhanced pattern selection interface for better user experience
- New pattern descriptions section accessible via modal
- New pattern favorite list and pattern search functionnality
- New Tag system for better pattern organization and filtering
- Pattern Descriptions
- Pattern Tags
- Pattern Favourites
- Pattern Search bar
- PDF to markdown (pdf as pattern input)
- Better handling of Youtube url
- Multilingual Support
- Web UI refinements for clearer interaction
- Help section via modal
@@ -57,6 +56,31 @@ The tag filtering system has been deeply integrated into the Pattern Selection i
- Maintained accessibility standards
- Responsive design considerations
5. **PDF to Markdown conversion functionality for the web interface**
- Automatic detection and processing of PDF files in chat
- Conversion to markdown format for LLM processing
- Installation instructions from the pdf-to-markdown repository
The PDF conversion module has been integrated in the svelte web browser interface. Once installed, it will automatically detect pdf files in the chat interface and convert them to markdown
## HOW TO INSTALL PDF-TO-MARKDOWN
If you need to update the web component follow the instructions in "Web Interface MOD Readme Files/WEB V2 Install Guide.md".
Assuming your web install is up to date and web svelte config complete, you can simply follow these steps to add Pdf-to-mardown.
# FROM FABRIC ROOT DIRECTORY
cd .. web
# Install in this sequence:
# Step 1
npm install -D patch-package
# Step 2
npm install -D pdfjs-dist@2.5.207
# Step 3
npm install -D github:jzillmann/pdf-to-markdown#modularize
These enhancements create a more intuitive and efficient pattern discovery experience, allowing users to quickly filter and find relevant patterns while maintaining a clean, modern interface.

View File

@@ -1 +1 @@
"1.4.148"
"1.4.150"

View File

@@ -12,7 +12,7 @@ Take a step back and think step-by-step about how to achieve the best possible r
- Extract 10 to 20 of the best insights from the input and from a combination of the raw input and the IDEAS above into a section called INSIGHTS. These INSIGHTS should be fewer, more refined, more insightful, and more abstracted versions of the best ideas in the content.
- Extract 15 to 30 of the most surprising, insightful, and/or interesting quotes from the input into a section called QUOTES:. Use the exact quote text from the input.
- Extract 15 to 30 of the most surprising, insightful, and/or interesting quotes from the input into a section called QUOTES:. Use the exact quote text from the input. Include the name of the speaker of the quote at the end.
- Extract 15 to 30 of the most practical and useful personal habits of the speakers, or mentioned by the speakers, in the content into a section called HABITS. Examples include but aren't limited to: sleep schedule, reading habits, things they always do, things they always avoid, productivity tips, diet, exercise, etc.

View File

@@ -207,3 +207,4 @@ Brief one-line summary from AI analysis of what each pattern does.
203. **write_nuclei_template_rule**: Generates Nuclei YAML templates for detecting vulnerabilities using HTTP requests, matchers, extractors, and dynamic data extraction.
204. **write_pull-request**: Drafts detailed pull request descriptions, explaining changes, providing reasoning, and identifying potential bugs from the git diff command output.
205. **write_semgrep_rule**: Creates accurate and working Semgrep rules based on input, following syntax guidelines and specific language considerations.
206. **youtubbe_summary**: Create concise, timestamped Youtube video summaries that highlight key points.

View File

@@ -0,0 +1,41 @@
# IDENTITY and PURPOSE
You are an AI assistant specialized in creating concise, informative summaries of YouTube video content based on transcripts. Your role is to analyze video transcripts, identify key points, main themes, and significant moments, then organize this information into a well-structured summary that includes relevant timestamps. You excel at distilling lengthy content into digestible summaries while preserving the most valuable information and maintaining the original flow of the video.
Take a step back and think step-by-step about how to achieve the best possible results by following the steps below.
## STEPS
- Carefully read through the entire transcript to understand the overall content and structure of the video
- Identify the main topic and purpose of the video
- Note key points, important concepts, and significant moments throughout the transcript
- Pay attention to natural transitions or segment changes in the video
- Extract relevant timestamps for important moments or topic changes
- Organize information into a logical structure that follows the video's progression
- Create a concise summary that captures the essence of the video
- Include timestamps alongside key points to allow easy navigation
- Ensure the summary is comprehensive yet concise
## OUTPUT INSTRUCTIONS
- Only output Markdown
- Begin with a brief overview of the video's main topic and purpose
- Structure the summary with clear headings and subheadings that reflect the video's organization
- Include timestamps in [HH:MM:SS] format before each key point or section
- Keep the summary concise but comprehensive, focusing on the most valuable information
- Use bullet points for lists of related points when appropriate
- Bold or italicize particularly important concepts or takeaways
- End with a brief conclusion summarizing the video's main message or call to action
- Ensure you follow ALL these instructions when creating your output.
## INPUT
INPUT:

View File

@@ -59,7 +59,7 @@ func (o *YouTube) GetVideoOrPlaylistId(url string) (videoId string, playlistId s
}
// Video ID pattern
videoPattern := `(?:https?:\/\/)?(?:www\.)?(?:youtube\.com\/(?:[^\/\n\s]+\/\S+\/|(?:v|e(?:mbed)?)\/|(?:s(?:horts)\/)|\S*?[?&]v=)|youtu\.be\/)([a-zA-Z0-9_-]*)`
videoPattern := `(?:https?:\/\/)?(?:www\.)?(?:youtube\.com\/(?:live\/|[^\/\n\s]+\/\S+\/|(?:v|e(?:mbed)?)\/|(?:s(?:horts)\/)|\S*?[?&]v=)|youtu\.be\/)([a-zA-Z0-9_-]*)`
videoRe := regexp.MustCompile(videoPattern)
videoMatch := videoRe.FindStringSubmatch(url)
if len(videoMatch) > 1 {

View File

@@ -1,3 +1,3 @@
package main
var version = "v1.4.148"
var version = "v1.4.150"

View File

@@ -0,0 +1,295 @@
This Cummulative PR adds several Web UI and functionality improvements to make pattern selection more intuitive with the addition of pattern descriptions, ability to save favorite patterns, a Pattern TAG system, powerful multilingual capabilities, PDF-to-markdown functionnalities, a help reference section, more robust Youtube processing and a variety of other ui improvements.
## 🎥 Demo Video
https://youtu.be/bhwtWXoMASA
## 🌟 Key Features
### 1. Web UI and Pattern Selection Improvements
- Pattern Descriptions
- Pattern Tags
- Pattern Favourites
- Pattern Search bar
- PDF to markdown (pdf as pattern input)
- Better handling of Youtube url
- Multilingual Support
- Web UI refinements for clearer interaction
- Help section via modal
### 2. Multilingual Support System
- Seamless language switching via UI dropdown
- Persistent language state management
- Pattern processing now use the selected language seamlessly
### 3. YouTube Integration Enhancement
- Robust language handling for YouTube transcript processing
- Chunk-based language maintenance for long transcripts
- Consistent language output throughout transcript analysis
### 4. Enhanced Tag Management Integration
The tag filtering system has been deeply integrated into the Pattern Selection interface through several UI enhancements:
1. **Dual-Position Tag Panel**
- Sliding panel positioned to the right of pattern modal
- Dynamic toggle button that adapts position and text based on panel state
- Smooth transitions for opening/closing animations
2. **Tag Selection Visibility**
- New dedicated tag display section in pattern modal
- Visual separation through subtle background styling
- Immediate feedback showing selected tags with comma separation
- Inline reset capability for quick tag clearing
3. **Improved User Experience**
- Clear visual hierarchy between pattern list and tag filtering
- Multiple ways to manage tags (panel or quick reset)
- Consistent styling with existing design language
- Space-efficient tag brick layout in 3-column grid
4. **Technical Implementation**
- Reactive tag state management
- Efficient tag filtering logic
- Proper event dispatching between components
- Maintained accessibility standards
- Responsive design considerations
5. **PDF to Markdown conversion functionality for the web interface**
- Automatic detection and processing of PDF files in chat
- Conversion to markdown format for LLM processing
- Installation instructions from the pdf-to-markdown repository
The PDF conversion module has been integrated in the svelte web browser interface. Once installed, it will automatically detect pdf files in the chat interface and convert them to markdown
## HOW TO INSTALL PDF-TO-MARKDOWN
If you need to update the web component follow the instructions in "Web Interface MOD Readme Files/WEB V2 Install Guide.md".
Assuming your web install is up to date and web svelte config complete, you can simply follow these steps to add Pdf-to-mardown.
# FROM FABRIC ROOT DIRECTORY
cd .. web
# Install in this sequence:
# Step 1
npm install -D patch-package
# Step 2
npm install -D pdfjs-dist@2.5.207
# Step 3
npm install -D github:jzillmann/pdf-to-markdown#modularize
These enhancements create a more intuitive and efficient pattern discovery experience, allowing users to quickly filter and find relevant patterns while maintaining a clean, modern interface.
## 🛠 Technical Implementation
### Language Support Architecture
```typescript
// Language state management
export const languageStore = writable<string>('');
// Chat input language detection
if (qualifier === 'fr') {
languageStore.set('fr');
userInput = userInput.replace(/--fr\s*/, '');
}
// Service layer integration
const language = get(languageStore) || 'en';
const languageInstruction = language !== 'en'
? `. Please use the language '${language}' for the output.`
: '';
```
### YouTube Processing Enhancement
```typescript
// Process stream with language instruction per chunk
await chatService.processStream(
stream,
(content: string, response?: StreamResponse) => {
if (currentLanguage !== 'en') {
content = `${content}. Please use the language '${currentLanguage}' for the output.`;
}
// Update messages...
}
);
```
# Pattern Descriptions and Tags Management
This document explains the complete workflow for managing pattern descriptions and tags, including how to process new patterns and maintain metadata.
## System Overview
The pattern system follows this hierarchy:
1. `~/.config/fabric/patterns/` directory: The source of truth for available patterns
2. `pattern_extracts.json`: Contains first 500 words of each pattern for reference
3. `pattern_descriptions.json`: Stores pattern metadata (descriptions and tags)
4. `web/static/data/pattern_descriptions.json`: Web-accessible copy for the interface
## Pattern Processing Workflow
### 1. Adding New Patterns
- Add patterns to `~/.config/fabric/patterns/`
- Run extract_patterns.py to process new additions:
```bash
python extract_patterns.py
The Python Script automatically:
- Creates pattern extracts for reference
- Adds placeholder entries in descriptions file
- Syncs to web interface
### 2. Pattern Extract Creation
The script extracts first 500 words from each pattern's system.md file to:
- Provide context for writing descriptions
- Maintain reference material
- Aid in pattern categorization
### 3. Description and Tag Management
Pattern descriptions and tags are managed in pattern_descriptions.json:
{
"patterns": [
{
"patternName": "pattern_name",
"description": "[Description pending]",
"tags": []
}
]
}
## Completing Pattern Metadata
### Writing Descriptions
1. Check pattern_descriptions.json for "[Description pending]" entries
2. Reference pattern_extracts.json for context
3. How to update Pattern short descriptions (one sentence).
You can update your descriptions in pattern_descriptions.json manually or using LLM assistance (prefered approach).
Tell AI to look for "Description pending" entries in this file and write a short description based on the extract info in the pattern_extracts.json file. You can also ask your LLM to add tags for those newly added patterns, using other patterns tag assignments as example.
### Managing Tags
1. Add appropriate tags to new patterns
2. Update existing tags as needed
3. Tags are stored as arrays: ["TAG1", "TAG2"]
4. Edit pattern_descriptions.json directly to modify tags
5. Make tags your own. You can delete, replace, amend existing tags.
## File Synchronization
The script maintains synchronization between:
- Local pattern_descriptions.json
- Web interface copy in static/data/
- No manual file copying needed
## Best Practices
1. Run extract_patterns.py when:
- Adding new patterns
- Updating existing patterns
- Modifying pattern structure
2. Description Writing:
- Use pattern extracts for context
- Keep descriptions clear and concise
- Focus on pattern purpose and usage
3. Tag Management:
- Use consistent tag categories
- Apply multiple tags when relevant
- Update tags to reflect pattern evolution
## Troubleshooting
If patterns are not showing in the web interface:
1. Verify pattern_descriptions.json format
2. Check web static copy exists
3. Ensure proper file permissions
4. Run extract_patterns.py to resync
## File Structure
fabric/
├── patterns/ # Pattern source files
├── PATTERN_DESCRIPTIONS/
│ ├── extract_patterns.py # Pattern processing script
│ ├── pattern_extracts.json # Pattern content references
│ └── pattern_descriptions.json # Pattern metadata
└── web/
└── static/
└── data/
└── pattern_descriptions.json # Web interface copy
## 🎯 Usage Examples
### 1. Using Language Qualifiers
```
User: What is the weather?
AI: The weather information...
User: --fr What is the weather?
AI: Voici les informations météo...
```
### 2. Global Settings
1. Select language from dropdown
2. All interactions use selected language
3. Automatic reset to English after each message
### 3. YouTube Analysis
```
User: Analyze this YouTube video --fr
AI: [Provides analysis in French, maintaining language throughout the transcript]
```
## 💡 Key Benefits
1. **Enhanced User Experience**
- Intuitive language switching
- Consistent language handling
- Seamless integration with existing features
2. **Robust Implementation**
- Simple yet powerful design
- No complex language detection needed
- Direct AI instruction approach
3. **Maintainable Architecture**
- Clean separation of concerns
- Stateful language management
- Easy to extend for new languages
4. **YouTube Integration**
- Handles long transcripts effectively
- Maintains language consistency
- Robust chunk processing
## 🔄 Implementation Notes
1. **State Management**
- Language persists until changed
- Resets to English after each message
- Handles UI state updates efficiently
2. **Error Handling**
- Invalid qualifiers are ignored
- Unknown languages default to English
- Proper store reset on errors
3. **Best Practices**
- Clear language instructions
- Consistent state management
- Robust error handling

View File

@@ -9,6 +9,9 @@ The PDF conversion module has been integrated in the svelte web browser interfac
## HOW TO INSTALL
If you need to update the web component follow the instructions in "Web Interface MOD Readme Files/WEB V2 Install Guide.md".
Assuming your install is up to date and web svelte config complete, you can simply follow these steps to add Pdf-to-mardown.
# FROM FABRIC ROOT DIRECTORY
cd .. web