Update version to v1.4.73 and commit

Create a basic translator pattern, edit file to add desired language

.github/workflows/release.yml

@@ -94,11 +94,19 @@ jobs:
           path: fabric-windows-${{ matrix.arch }}.exe
 
       - name: Get latest tag
+        if: matrix.os != 'windows-latest'
         id: get_latest_tag
         run: |
           latest_tag=$(git tag --sort=-creatordate | head -n 1)
           echo "latest_tag=$latest_tag" >> $GITHUB_ENV
 
+      - name: Get latest tag
+        if: matrix.os == 'windows-latest'
+        id: get_latest_tag_windows
+        run: |
+          $latest_tag = git tag --sort=-creatordate | Select-Object -First 1
+          Add-Content -Path $env:GITHUB_ENV -Value "latest_tag=$latest_tag"
+
       - name: Create release if it doesn't exist
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Alma.md

@@ -172,6 +172,118 @@ S3 Bucket Security | Review and secure all S3 buckets to prevent data breaches |
 
 SQL Injection Mitigation | Implement measures to eliminate SQL injection vulnerabilities | High | Tigan Wang | 2024-01-20 | 2024-05-20 | Not Started | $60K one-time 
 
+## SECURITY POSTURE (To be referenced for compliance questions and security questionnaires)
+
+July 2019
+Admin accounts still not required to use 2FA.
+Company laptops distributed to employees, no MDM yet for device management.
+AWS IAM roles created for engineers, but root access still frequently used.
+Started basic vulnerability scanning using open-source tools.
+December 2019
+
+MFA enforced for all Google Workspace accounts after a phishing attempt.
+Introduced ClamAV for basic endpoint protection on corporate laptops.
+AWS GuardDuty enabled for threat detection, but no formal incident response team.
+First incident response plan table-top exercise conducted, but findings not fully documented.
+April 2020
+
+Migrated from Google Workspace to Office 365, with MFA enabled for all users.
+Rolled out SentinelOne for endpoint protection on 50% of company laptops.
+Implemented least-privilege access control for AWS IAM roles.
+First formal vendor risk management review completed for major SaaS providers.
+August 2020
+
+Completed full deployment of SentinelOne across all endpoints.
+Implemented AWS CloudWatch for real-time alerts; however, logs still not monitored 24/7.
+Began encrypting all AWS S3 buckets at rest using server-side encryption.
+First internal review of data retention policies, started drafting data disposal policy.
+January 2021
+
+Rolled out Jamf MDM for centralized management of macOS devices, enforcing encryption (FileVault) on all laptops.
+Strengthened Office 365 security by implementing phishing-resistant MFA using authenticator apps.
+AWS KMS introduced for managing encryption keys; manual key rotation policy documented.
+Introduced formal onboarding and offboarding processes for employee account management.
+July 2021
+
+Conditional access policies introduced for Office 365, restricting access based on geography (US-only).
+Conducted company-wide security awareness training for the first time, focusing on phishing threats.
+Completed first backup and disaster recovery (DR) drill with AWS, documenting recovery times.
+AWS Config deployed to monitor and enforce encryption and access control policies across accounts.
+December 2021
+
+Full migration to AWS for all production systems completed.
+Incident response playbook finalized and shared with the security team; still no 24/7 monitoring.
+Documented data classification policies for handling sensitive customer data in preparation for SOC 2 audit.
+First third-party penetration test conducted, critical vulnerabilities identified and remediated within 30 days.
+March 2022
+
+Rolled out company-wide 2FA for all critical systems, including Office 365, AWS, GitHub, and Slack.
+Introduced AWS Secrets Manager for managing sensitive credentials, eliminating hardcoded API keys.
+Updated all documentation for identity and access management in preparation for SOC 2 Type 1 audit.
+First external vulnerability scan completed using Qualys, with remediation SLAs established.
+April 2022
+
+Updated and consolidated all security policies (incident response, access control, data retention) in preparation for SOC 2 audit.
+Conducted tabletop exercise for ransomware response, documenting gaps in the incident response process.
+Implemented Just-In-Time (JIT) access for administrative privileges in AWS, reducing unnecessary persistent access.
+October 2022
+
+Passed SOC 2 Type 1 audit, with recommendations to improve monitoring and asset management.
+Launched quarterly phishing simulations to raise employee awareness and track training effectiveness.
+Fully enforced encryption for all customer data in transit and at rest using AWS KMS.
+Extended GuardDuty to cover all AWS regions; started monitoring alerts daily.
+January 2023
+
+Hired a dedicated CISO and expanded security team by 30%.
+Integrated continuous vulnerability scanning across all externally facing assets using Qualys.
+Conducted first third-party vendor risk assessment to ensure alignment with SOC 2 and internal security standards.
+Implemented automated patch management for all AWS EC2 instances, reducing time to deploy critical patches.
+July 2023
+
+Rolled out continuous attack surface monitoring (ASM) to identify and remediate external vulnerabilities.
+Performed annual data retention review, ensuring compliance with SOC 2 and GDPR requirements.
+Conducted a disaster recovery drill for AWS workloads, achieving a recovery time objective (RTO) of under 4 hours.
+Completed SOC 2 Type 2 readiness assessment, with focus on improving incident response times.
+November 2023
+
+Updated incident response documentation and assigned 24/7 monitoring to a third-party SOC provider.
+Rolled out zero-trust network architecture across the organization, removing reliance on VPN for remote access.
+Passed SOC 2 Type 2 audit with no major findings; recommendations included improved asset inventory tracking.
+Conducted full audit of access control policies and JIT access implementation in preparation for ISO 27001 certification.
+April 2024
+
+Implemented AI-driven threat detection to reduce time to detect security incidents from 10 hours to under 2 hours.
+Completed full encryption audit across all databases, ensuring compliance with GDPR, HIPAA, and other privacy regulations.
+Updated employee training programs to include privacy regulations (GDPR, CCPA) and data handling best practices.
+Completed internal review and audit of vendor access to critical systems as part of SOC 2 compliance effort.
+Completed move of all AWS services to us-west-2  and us-east-1 regions for 100% us-based cloud services.
+October 2024
+
+Conducted organization-wide review of data retention and disposal policies, implementing automated data deletion for expired data.
+Implemented continuous compliance monitoring for SOC 2, with automated alerts for deviations in access controls and encryption settings.
+Finalized implementation of AI-based monitoring and response systems, significantly reducing time to remediate critical vulnerabilities.
+Passed SOC 2 Type 2 and ISO 27001 audits with zero non-conformities, achieving full compliance across all control areas.March 2018
+
+Personal Gmail accounts used for internal and external communication.
+No 2FA enabled on any accounts.
+AWS accounts shared with engineers, no IAM roles or formal access control policies.
+No centralized endpoint protection; employees use personal laptops with no security controls.
+No documented security policies or incident response plan.
+September 2018
+
+Initiated migration from personal Gmail to Google Workspace (G Suite) for business email.
+Password complexity requirements introduced (minimum 8 characters).
+AWS root credentials still shared among team members, no MFA enabled.
+No formal logging or monitoring in place for AWS activity.
+February 2019
+
+Completed migration to Google Workspace; no email encryption yet.
+Introduced a basic password manager (LastPass) but no enforcement policy.
+AWS CloudTrail enabled for logging, but no one is reviewing logs.
+First draft of the incident response plan created, but not tested.
+June 2019
+
+Enforced MFA for Google Workspace admin accounts; standard user
 ## CURRENT STATE (KPIs, Metrics, Project Activity Updates, etc.)
 - October 2022: Current time to detect malicious behavior is 81 hours
 - October 2022: Current time to start investigating malicious behavior is 82 hours

README.md

@@ -43,6 +43,7 @@
   - [Just use the Patterns](#just-use-the-patterns)
 - [Custom Patterns](#custom-patterns)
 - [Helper Apps](#helper-apps)
+- [pbpaste](#pbpaste)
 - [Meta](#meta)
   - [Primary contributors](#primary-contributors)
 
@@ -50,7 +51,7 @@
 
 ## Updates
 
-> [!NOTE] 
+> [!NOTE]
 September 15, 2024 — Lots of new stuff!
 > * Fabric now supports calling the new `o1-preview` model using the `-r` switch (which stands for raw. Normal queries won't work with `o1-preview` because they disabled System access and don't allow us to set `Temperature`.
 > * We have early support for Raycast! Under the `/patterns` directory there's a `raycast` directory with scripts that can be called from Raycast. If you add a scripts directory within Raycast and point it to your `~/.config/fabric/patterns/raycast` directory, you'll then be able to 1) invoke Raycast, type the name of the script, and then 2) paste in the content to be passed, and the results will return in Raycast. There's currently only one script in there but I am (Daniel) adding more.
@@ -124,10 +125,10 @@ curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric
 # MacOS (arm64):
 curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-darwin-arm64 > fabric && chmod +x fabric && ./fabric --version
 
-# MacOS (amd64): 
+# MacOS (amd64):
 curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-darwin-amd64 > fabric && chmod +x fabric && ./fabric --version
 
-# Linux (amd64): 
+# Linux (amd64):
 curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-linux-amd64 > fabric && chmod +x fabric && ./fabric --version
 
 # Linux (arm64):
@@ -208,18 +209,15 @@ fabric -h
 
 ```bash
 
-usage: fabric -h
 Usage:
   fabric [OPTIONS]
 
 Application Options:
-  -p, --pattern=             Choose a pattern
-  -v, --variable=            Values for pattern variables, e.g. -v=$name:John -v=$age:30
-  -C, --context=             Choose a context
-      --session=             Choose a session
+  -p, --pattern=             Choose a pattern from the available patterns
+  -v, --variable=            Values for pattern variables, e.g. -v=#role:expert -v=#points:30"
+  -C, --context=             Choose a context from the available contexts
+      --session=             Choose a session from the available sessions
   -S, --setup                Run setup for all reconfigurable parts of fabric
-      --setup-skip-patterns  Run Setup for specific vendor, one of Ollama, OpenAI, Anthropic, Azure, Gemini, Groq, Mistral, OpenRouter, SiliconCloud. E.g. fabric --setup-vendor=OpenAI
-      --setup-vendor=        Run Setup for specific vendor. E.g. --setup-vendor=OpenAI
   -t, --temperature=         Set temperature (default: 0.7)
   -T, --topp=                Set top P (default: 0.9)
   -s, --stream               Stream
@@ -234,7 +232,7 @@ Application Options:
   -c, --copy                 Copy to clipboard
   -m, --model=               Choose model
   -o, --output=              Output to file
-      --output-prompt        Output the used prompt (or entire session) before the result
+      --output-session       Output the entire session (also a temporary one) to the output file
   -n, --latest=              Number of latest patterns to list (default: 0)
   -d, --changeDefaultModel   Change default model
   -y, --youtube=             YouTube video "URL" to grab transcript, comments from it and send to chat
@@ -253,7 +251,7 @@ Application Options:
       --version              Print current version
 
 Help Options:
-  -h, --help                        Show this help message
+  -h, --help                 Show this help message
 
 ```
 
@@ -277,6 +275,8 @@ https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/syste
 
 ## Examples
 
+> The following examples use the macOS `pbpaste` to paste from the clipboard. See the [pbpaste](#pbpaste) section below for Windows and Linux alternatives.
+
 Now let's look at some things you can do with Fabric.
 
 1. Run the `summarize` Pattern based on input from `stdin`. In this case, the body of an article.
@@ -294,7 +294,7 @@ pbpaste | fabric --stream --pattern analyze_claims
 3. Run the `extract_wisdom` Pattern with the `--stream` option to get immediate and streaming results from any Youtube video (much like in the original introduction video).
 
 ```bash
-fabric -y "https://youtube.com/watch?v=uXs-zPc63kM" | --stream --pattern extract_wisdom
+fabric -y "https://youtube.com/watch?v=uXs-zPc63kM" --stream --pattern extract_wisdom
 ```
 
 4. Create patterns- you must create a .md file with the pattern and save it to ~/.config/fabric/patterns/[yourpatternname].
@@ -318,7 +318,7 @@ The wisdom of crowds for the win.
 
 You may want to use Fabric to create your own custom Patterns—but not share them with others. No problem!
 
-Just make a directory in `~/.config/custompatterns/` (or wherever) and put your `.md` files in there. 
+Just make a directory in `~/.config/custompatterns/` (or wherever) and put your `.md` files in there.
 
 When you're ready to use them, copy them into:
 
@@ -363,6 +363,29 @@ go install github.com/danielmiessler/fabric/to_pdf@latest
 
 Make sure you have a LaTeX distribution (like TeX Live or MiKTeX) installed on your system, as `to_pdf` requires `pdflatex` to be available in your system's PATH.
 
+## pbpaste
+
+The [examples](#examples) use the macOS program `pbpaste` to paste content from the clipboard to pipe into `fabric` as the input. `pbpaste` is not available on Windows or Linux, but there are alternatives.
+
+On Windows, you can use the PowerShell command `Get-Clipboard` from a PowerShell command prompt. If you like, you can also alias it to `pbpaste`. If you are using classic PowerShell, edit the file `~\Documents\WindowsPowerShell\.profile.ps1`, or if you are using PowerShell Core, edit `~\Documents\PowerShell\.profile.ps1` and add the alias,
+
+```powershell
+Set-Alias pbpaste Get-Clipboard
+```
+
+On Linux, you can use `xclip -selection clipboard -o` to paste from the clipboard. You will likely need to install `xclip` with your package manager. For Debian based systems including Ubuntu,
+
+```sh
+sudo apt update
+sudo apt install xclip -y
+```
+
+You can also create an alias by editing `~/.bashrc` or `~/.zshrc` and adding the alias,
+
+```sh
+alias pbpaste='xclip -selection clipboard -o'
+```
+
 ## Meta
 
 > [!NOTE]

cli/cli.go

@@ -2,14 +2,15 @@ package cli
 
 import (
 	"fmt"
-	"github.com/danielmiessler/fabric/converter"
+	"github.com/danielmiessler/fabric/core"
+	"github.com/danielmiessler/fabric/plugins/ai"
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"github.com/danielmiessler/fabric/plugins/tools/converter"
+	"github.com/danielmiessler/fabric/restapi"
 	"os"
 	"path/filepath"
 	"strconv"
 	"strings"
-
-	"github.com/danielmiessler/fabric/core"
-	"github.com/danielmiessler/fabric/db"
 )
 
 // Cli Controls the cli. It takes in the flags and runs the appropriate functions
@@ -29,39 +30,35 @@ func Cli(version string) (err error) {
 		return
 	}
 
-	fabricDb := db.NewDb(filepath.Join(homedir, ".config/fabric"))
+	fabricDb := fsdb.NewDb(filepath.Join(homedir, ".config/fabric"))
+
+	if err = fabricDb.Configure(); err != nil {
+		if !currentFlags.Setup {
+			println(err.Error())
+			currentFlags.Setup = true
+		}
+	}
+
+	registry := core.NewPluginRegistry(fabricDb)
 
 	// if the setup flag is set, run the setup function
-	if currentFlags.Setup || currentFlags.SetupSkipPatterns || currentFlags.SetupVendor != "" {
-		_ = fabricDb.Configure()
-		if currentFlags.SetupVendor != "" {
-			_, err = SetupVendor(fabricDb, currentFlags.SetupVendor)
-		} else {
-			_, err = Setup(fabricDb, currentFlags.SetupSkipPatterns)
-		}
+	if currentFlags.Setup {
+		err = registry.Setup()
 		return
 	}
 
-	var fabric *core.Fabric
-	if err = fabricDb.Configure(); err != nil {
-		fmt.Println("init is failed, run start the setup procedure", err)
-		if fabric, err = Setup(fabricDb, currentFlags.SetupSkipPatterns); err != nil {
-			return
-		}
-	} else {
-		if fabric, err = core.NewFabric(fabricDb); err != nil {
-			fmt.Println("fabric can't initialize, please run the --setup procedure", err)
-			return
-		}
+	if currentFlags.Serve {
+		err = restapi.Serve(registry, currentFlags.ServeAddress)
+		return
 	}
 
 	if currentFlags.UpdatePatterns {
-		err = fabric.PopulateDB()
+		err = registry.PatternsLoader.PopulateDB()
 		return
 	}
 
 	if currentFlags.ChangeDefaultModel {
-		err = fabric.SetupDefaultModel()
+		err = registry.Defaults.Setup()
 		return
 	}
 
@@ -83,7 +80,11 @@ func Cli(version string) (err error) {
 	}
 
 	if currentFlags.ListAllModels {
-		fabric.GetModels().Print()
+		var models *ai.VendorsModels
+		if models, err = registry.VendorManager.GetModels(); err != nil {
+			return
+		}
+		models.Print()
 		return
 	}
 
@@ -133,23 +134,27 @@ func Cli(version string) (err error) {
 	// if none of the above currentFlags are set, run the initiate chat function
 
 	if currentFlags.YouTube != "" {
-		if fabric.YouTube.IsConfigured() == false {
+		if registry.YouTube.IsConfigured() == false {
 			err = fmt.Errorf("YouTube is not configured, please run the setup procedure")
 			return
 		}
 
 		var videoId string
-		if videoId, err = fabric.YouTube.GetVideoId(currentFlags.YouTube); err != nil {
+		if videoId, err = registry.YouTube.GetVideoId(currentFlags.YouTube); err != nil {
 			return
 		}
 
 		if !currentFlags.YouTubeComments || currentFlags.YouTubeTranscript {
 			var transcript string
 			var language = "en"
-			if currentFlags.Language != "" {
-				language = currentFlags.Language
+			if currentFlags.Language != "" || registry.Language.DefaultLanguage.Value != "" {
+				if currentFlags.Language != "" {
+					language = currentFlags.Language
+				} else {
+					language = registry.Language.DefaultLanguage.Value
+				}
 			}
-			if transcript, err = fabric.YouTube.GrabTranscript(videoId, language); err != nil {
+			if transcript, err = registry.YouTube.GrabTranscript(videoId, language); err != nil {
 				return
 			}
 
@@ -158,7 +163,7 @@ func Cli(version string) (err error) {
 
 		if currentFlags.YouTubeComments {
 			var comments []string
-			if comments, err = fabric.YouTube.GrabComments(videoId); err != nil {
+			if comments, err = registry.YouTube.GrabComments(videoId); err != nil {
 				return
 			}
 
@@ -174,11 +179,11 @@ func Cli(version string) (err error) {
 		}
 	}
 
-	if (currentFlags.ScrapeURL != "" || currentFlags.ScrapeQuestion != "") && fabric.Jina.IsConfigured() {
+	if (currentFlags.ScrapeURL != "" || currentFlags.ScrapeQuestion != "") && registry.Jina.IsConfigured() {
 		// Check if the scrape_url flag is set and call ScrapeURL
 		if currentFlags.ScrapeURL != "" {
 			var website string
-			if website, err = fabric.Jina.ScrapeURL(currentFlags.ScrapeURL); err != nil {
+			if website, err = registry.Jina.ScrapeURL(currentFlags.ScrapeURL); err != nil {
 				return
 			}
 
@@ -188,7 +193,7 @@ func Cli(version string) (err error) {
 		// Check if the scrape_question flag is set and call ScrapeQuestion
 		if currentFlags.ScrapeQuestion != "" {
 			var website string
-			if website, err = fabric.Jina.ScrapeQuestion(currentFlags.ScrapeQuestion); err != nil {
+			if website, err = registry.Jina.ScrapeQuestion(currentFlags.ScrapeQuestion); err != nil {
 				return
 			}
 
@@ -203,12 +208,16 @@ func Cli(version string) (err error) {
 	}
 
 	var chatter *core.Chatter
-	if chatter, err = fabric.GetChatter(currentFlags.Model, currentFlags.Stream, currentFlags.DryRun); err != nil {
+	if chatter, err = registry.GetChatter(currentFlags.Model, currentFlags.Stream, currentFlags.DryRun); err != nil {
 		return
 	}
 
-	var session *db.Session
-	if session, err = chatter.Send(currentFlags.BuildChatRequest(), currentFlags.BuildChatOptions()); err != nil {
+	var session *fsdb.Session
+	chatReq := currentFlags.BuildChatRequest(strings.Join(os.Args[1:], " "))
+	if chatReq.Language == "" {
+		chatReq.Language = registry.Language.DefaultLanguage.Value
+	}
+	if session, err = chatter.Send(chatReq, currentFlags.BuildChatOptions()); err != nil {
 		return
 	}
 
@@ -221,43 +230,19 @@ func Cli(version string) (err error) {
 
 	// if the copy flag is set, copy the message to the clipboard
 	if currentFlags.Copy {
-		if err = fabric.CopyToClipboard(result); err != nil {
+		if err = CopyToClipboard(result); err != nil {
 			return
 		}
 	}
 
 	// if the output flag is set, create an output file
 	if currentFlags.Output != "" {
-		if currentFlags.OutputPrompt {
+		if currentFlags.OutputSession {
 			sessionAsString := session.String()
-			err = fabric.CreateOutputFile(sessionAsString, currentFlags.Output)
+			err = CreateOutputFile(sessionAsString, currentFlags.Output)
 		} else {
-			err = fabric.CreateOutputFile(result, currentFlags.Output)
+			err = CreateOutputFile(result, currentFlags.Output)
 		}
 	}
 	return
 }
-
-func Setup(db *db.Db, skipUpdatePatterns bool) (ret *core.Fabric, err error) {
-	instance := core.NewFabricForSetup(db)
-
-	if err = instance.Setup(); err != nil {
-		return
-	}
-
-	if !skipUpdatePatterns {
-		if err = instance.PopulateDB(); err != nil {
-			return
-		}
-	}
-	ret = instance
-	return
-}
-
-func SetupVendor(db *db.Db, vendorName string) (ret *core.Fabric, err error) {
-	ret = core.NewFabricForSetup(db)
-	if err = ret.SetupVendor(vendorName); err != nil {
-		return
-	}
-	return
-}

cli/cli_test.go

@@ -5,7 +5,6 @@ import (
 	"os"
 	"testing"
 
-	"github.com/danielmiessler/fabric/db"
 	"github.com/stretchr/testify/assert"
 )
 
@@ -19,11 +18,3 @@ func TestCli(t *testing.T) {
 	assert.Error(t, err)
 	assert.Equal(t, core.NoSessionPatternUserMessages, err.Error())
 }
-
-func TestSetup(t *testing.T) {
-	mockDB := db.NewDb(os.TempDir())
-
-	fabric, err := Setup(mockDB, false)
-	assert.Error(t, err)
-	assert.Nil(t, fabric)
-}

cli/flags.go

@@ -14,13 +14,11 @@ import (
 
 // Flags create flags struct. the users flags go into this, this will be passed to the chat struct in cli
 type Flags struct {
-	Pattern            string            `short:"p" long:"pattern" description:"Choose a pattern" default:""`
-	PatternVariables   map[string]string `short:"v" long:"variable" description:"Values for pattern variables, e.g. -v=$name:John -v=$age:30"`
-	Context            string            `short:"C" long:"context" description:"Choose a context" default:""`
-	Session            string            `long:"session" description:"Choose a session"`
+	Pattern            string            `short:"p" long:"pattern" description:"Choose a pattern from the available patterns" default:""`
+	PatternVariables   map[string]string `short:"v" long:"variable" description:"Values for pattern variables, e.g. -v=#role:expert -v=#points:30"`
+	Context            string            `short:"C" long:"context" description:"Choose a context from the available contexts" default:""`
+	Session            string            `long:"session" description:"Choose a session from the available sessions"`
 	Setup              bool              `short:"S" long:"setup" description:"Run setup for all reconfigurable parts of fabric"`
-	SetupSkipPatterns  bool              `long:"setup-skip-patterns" description:"Run Setup for specific vendor, one of Ollama, OpenAI, Anthropic, Azure, Gemini, Groq, Mistral, OpenRouter, SiliconCloud. E.g. fabric --setup-vendor=OpenAI"`
-	SetupVendor        string            `long:"setup-vendor" description:"Run Setup for specific vendor. E.g. --setup-vendor=OpenAI"`
 	Temperature        float64           `short:"t" long:"temperature" description:"Set temperature" default:"0.7"`
 	TopP               float64           `short:"T" long:"topp" description:"Set top P" default:"0.9"`
 	Stream             bool              `short:"s" long:"stream" description:"Stream"`
@@ -36,7 +34,7 @@ type Flags struct {
 	Copy               bool              `short:"c" long:"copy" description:"Copy to clipboard"`
 	Model              string            `short:"m" long:"model" description:"Choose model"`
 	Output             string            `short:"o" long:"output" description:"Output to file" default:""`
-	OutputPrompt       bool              `long:"output-prompt" description:"Output the used prompt (or entire session) before the result"`
+	OutputSession      bool              `long:"output-session" description:"Output the entire session (also a temporary one) to the output file"`
 	LatestPatterns     string            `short:"n" long:"latest" description:"Number of latest patterns to list" default:"0"`
 	ChangeDefaultModel bool              `short:"d" long:"changeDefaultModel" description:"Change default model"`
 	YouTube            string            `short:"y" long:"youtube" description:"YouTube video \"URL\" to grab transcript, comments from it and send to chat"`
@@ -52,6 +50,8 @@ type Flags struct {
 	PrintSession       string            `long:"printsession" description:"Print session"`
 	HtmlReadability    bool              `long:"readability" description:"Convert HTML input into a clean, readable view"`
 	DryRun             bool              `long:"dry-run" description:"Show what would be sent to the model without actually sending it"`
+	Serve              bool              `long:"serve" description:"Serve the Fabric Rest API"`
+	ServeAddress       string            `long:"address" description:"The address to bind the REST API" default:":8080"`
 	Version            bool              `long:"version" description:"Print current version"`
 }
 
@@ -113,13 +113,14 @@ func (o *Flags) BuildChatOptions() (ret *common.ChatOptions) {
 	return
 }
 
-func (o *Flags) BuildChatRequest() (ret *common.ChatRequest) {
+func (o *Flags) BuildChatRequest(Meta string) (ret *common.ChatRequest) {
 	ret = &common.ChatRequest{
 		ContextName:      o.Context,
 		SessionName:      o.Session,
 		PatternName:      o.Pattern,
 		PatternVariables: o.PatternVariables,
 		Message:          o.Message,
+		Meta:             Meta,
 	}
 	if o.Language != "" {
 		langTag, err := language.Parse(o.Language)
@@ -140,6 +141,6 @@ func (o *Flags) AppendMessage(message string) {
 }
 
 func (o *Flags) IsChatRequest() (ret bool) {
-	ret = o.Message != "" || o.Session != ""
+	ret = (o.Message != "" || o.Context != "") && (o.Session != "" || o.Pattern != "")
 	return
 }

cli/flags_test.go

@@ -101,7 +101,8 @@ func TestBuildChatRequest(t *testing.T) {
 		SessionName: "test-session",
 		PatternName: "test-pattern",
 		Message:     "test-message",
+		Meta:        "test",
 	}
-	request := flags.BuildChatRequest()
+	request := flags.BuildChatRequest("test")
 	assert.Equal(t, expectedRequest, request)
 }

cli/output.go

@@ -0,0 +1,27 @@
+package cli
+
+import (
+	"fmt"
+	"github.com/atotto/clipboard"
+	"os"
+)
+
+func CopyToClipboard(message string) (err error) {
+	if err = clipboard.WriteAll(message); err != nil {
+		err = fmt.Errorf("could not copy to clipboard: %v", err)
+	}
+	return
+}
+
+func CreateOutputFile(message string, fileName string) (err error) {
+	var file *os.File
+	if file, err = os.Create(fileName); err != nil {
+		err = fmt.Errorf("error creating file: %v", err)
+		return
+	}
+	defer file.Close()
+	if _, err = file.WriteString(message); err != nil {
+		err = fmt.Errorf("error writing to file: %v", err)
+	}
+	return
+}

cli/output_test.go

@@ -0,0 +1,28 @@
+package cli
+
+import (
+	"os"
+	"testing"
+)
+
+func TestCopyToClipboard(t *testing.T) {
+	t.Skip("skipping test, because of docker env. in ci.")
+
+	message := "test message"
+	err := CopyToClipboard(message)
+	if err != nil {
+		t.Fatalf("CopyToClipboard() error = %v", err)
+	}
+}
+
+func TestCreateOutputFile(t *testing.T) {
+
+	fileName := "test_output.txt"
+	message := "test message"
+	err := CreateOutputFile(message, fileName)
+	if err != nil {
+		t.Fatalf("CreateOutputFile() error = %v", err)
+	}
+
+	defer os.Remove(fileName)
+}

common/domain.go

@@ -2,6 +2,8 @@ package common
 
 import goopenai "github.com/sashabaranov/go-openai"
 
+const ChatMessageRoleMeta = "meta"
+
 type Message struct {
 	Role    string `json:"role"`
 	Content string `json:"content"`
@@ -14,6 +16,7 @@ type ChatRequest struct {
 	PatternVariables map[string]string
 	Message          string
 	Language         string
+	Meta             string
 }
 
 type ChatOptions struct {

common/groups_items.go

@@ -0,0 +1,134 @@
+package common
+
+import (
+	"fmt"
+	"github.com/samber/lo"
+)
+
+func NewGroupsItemsSelector[I any](selectionLabel string,
+	getItemLabel func(I) string) *GroupsItemsSelector[I] {
+
+	return &GroupsItemsSelector[I]{SelectionLabel: selectionLabel,
+		GetItemKey:  getItemLabel,
+		GroupsItems: make([]*GroupItems[I], 0),
+	}
+}
+
+type GroupItems[I any] struct {
+	Group string
+	Items []I
+}
+
+func (o *GroupItems[I]) Count() int {
+	return len(o.Items)
+}
+
+func (o *GroupItems[I]) ContainsItemBy(predicate func(item I) bool) (ret bool) {
+	ret = lo.ContainsBy(o.Items, predicate)
+	return
+}
+
+type GroupsItemsSelector[I any] struct {
+	SelectionLabel string
+	GetItemKey     func(I) string
+
+	GroupsItems []*GroupItems[I]
+}
+
+func (o *GroupsItemsSelector[I]) AddGroupItems(group string, items ...I) {
+	o.GroupsItems = append(o.GroupsItems, &GroupItems[I]{group, items})
+}
+
+func (o *GroupsItemsSelector[I]) GetGroupAndItemByItemNumber(number int) (group string, item I, err error) {
+	var currentItemNumber int
+	found := false
+
+	for _, groupItems := range o.GroupsItems {
+		if currentItemNumber+groupItems.Count() < number {
+			currentItemNumber += groupItems.Count()
+			continue
+		}
+
+		for _, groupItem := range groupItems.Items {
+			currentItemNumber++
+			if currentItemNumber == number {
+				group = groupItems.Group
+				item = groupItem
+				found = true
+				break
+			}
+		}
+	}
+
+	if !found {
+		err = fmt.Errorf("number %d is out of range", number)
+	}
+	return
+}
+
+func (o *GroupsItemsSelector[I]) Print() {
+	fmt.Printf("\n%v:\n", o.SelectionLabel)
+
+	var currentItemIndex int
+	for _, groupItems := range o.GroupsItems {
+		fmt.Println()
+		fmt.Printf("%s\n", groupItems.Group)
+		fmt.Println()
+
+		for _, item := range groupItems.Items {
+			currentItemIndex++
+			fmt.Printf("\t[%d]\t%s\n", currentItemIndex, o.GetItemKey(item))
+
+		}
+	}
+}
+
+func (o *GroupsItemsSelector[I]) HasGroup(group string) (ret bool) {
+	for _, groupItems := range o.GroupsItems {
+		if ret = groupItems.Group == group; ret {
+			break
+		}
+	}
+	return
+}
+
+func (o *GroupsItemsSelector[I]) FindGroupsByItemFirst(item I) (ret string) {
+	itemKey := o.GetItemKey(item)
+
+	for _, groupItems := range o.GroupsItems {
+		if groupItems.ContainsItemBy(func(groupItem I) bool {
+			groupItemKey := o.GetItemKey(groupItem)
+			return groupItemKey == itemKey
+		}) {
+			ret = groupItems.Group
+			break
+		}
+	}
+	return
+}
+
+func (o *GroupsItemsSelector[I]) FindGroupsByItem(item I) (groups []string) {
+	itemKey := o.GetItemKey(item)
+
+	for _, groupItems := range o.GroupsItems {
+		if groupItems.ContainsItemBy(func(groupItem I) bool {
+			groupItemKey := o.GetItemKey(groupItem)
+			return groupItemKey == itemKey
+		}) {
+			groups = append(groups, groupItems.Group)
+		}
+	}
+	return
+}
+
+func ReturnItem(item string) string {
+	return item
+}
+
+func NewGroupsItemsSelectorString(selectionLabel string) *GroupsItemsSelectorString {
+	return &GroupsItemsSelectorString{GroupsItemsSelector: NewGroupsItemsSelector(selectionLabel, ReturnItem)}
+}
+
+type GroupsItemsSelectorString struct {
+	*GroupsItemsSelector[string]
+}

core/chatter.go

@@ -4,28 +4,26 @@ import (
 	"context"
 	"fmt"
 	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/db"
-	"github.com/danielmiessler/fabric/vendors"
+	"github.com/danielmiessler/fabric/plugins/ai"
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
 	goopenai "github.com/sashabaranov/go-openai"
+	"strings"
 )
 
+const NoSessionPatternUserMessages = "no session, pattern or user messages provided"
+
 type Chatter struct {
-	db *db.Db
+	db *fsdb.Db
 
 	Stream bool
 	DryRun bool
 
 	model  string
-	vendor vendors.Vendor
+	vendor ai.Vendor
 }
 
-func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (session *db.Session, err error) {
-	var chatRequest *Chat
-	if chatRequest, err = o.NewChat(request); err != nil {
-		return
-	}
-
-	if session, err = chatRequest.BuildChatSession(opts.Raw); err != nil {
+func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (session *fsdb.Session, err error) {
+	if session, err = o.BuildSession(request, opts.Raw); err != nil {
 		return
 	}
 
@@ -38,7 +36,7 @@ func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (s
 	if o.Stream {
 		channel := make(chan string)
 		go func() {
-			if streamErr := o.vendor.SendStream(session.Messages, opts, channel); streamErr != nil {
+			if streamErr := o.vendor.SendStream(session.GetVendorMessages(), opts, channel); streamErr != nil {
 				channel <- streamErr.Error()
 			}
 		}()
@@ -48,7 +46,7 @@ func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (s
 			fmt.Print(response)
 		}
 	} else {
-		if message, err = o.vendor.Send(context.Background(), session.Messages, opts); err != nil {
+		if message, err = o.vendor.Send(context.Background(), session.GetVendorMessages(), opts); err != nil {
 			return
 		}
 	}
@@ -56,59 +54,80 @@ func (o *Chatter) Send(request *common.ChatRequest, opts *common.ChatOptions) (s
 	if message == "" {
 		session = nil
 		err = fmt.Errorf("empty response")
+		return
 	}
 
 	session.Append(&common.Message{Role: goopenai.ChatMessageRoleAssistant, Content: message})
 
-	if chatRequest.Session != nil {
+	if session.Name != "" {
 		err = o.db.Sessions.SaveSession(session)
 	}
 	return
 }
 
-func (o *Chatter) NewChat(request *common.ChatRequest) (ret *Chat, err error) {
-	ret = &Chat{
-		Language: request.Language,
-	}
-
-	if request.ContextName != "" {
-		var ctx *db.Context
-		if ctx, err = o.db.Contexts.GetContext(request.ContextName); err != nil {
-			err = fmt.Errorf("could not find context %s: %v", request.ContextName, err)
-			return
-		}
-		ret.Context = ctx.Content
-	}
-
+func (o *Chatter) BuildSession(request *common.ChatRequest, raw bool) (session *fsdb.Session, err error) {
 	if request.SessionName != "" {
-		var sess *db.Session
-		if sess, err = o.db.Sessions.GetOrCreateSession(request.SessionName); err != nil {
+		var sess *fsdb.Session
+		if sess, err = o.db.Sessions.Get(request.SessionName); err != nil {
 			err = fmt.Errorf("could not find session %s: %v", request.SessionName, err)
 			return
 		}
-		ret.Session = sess
+		session = sess
+	} else {
+		session = &fsdb.Session{}
 	}
 
+	if request.Meta != "" {
+		session.Append(&common.Message{Role: common.ChatMessageRoleMeta, Content: request.Meta})
+	}
+
+	var contextContent string
+	if request.ContextName != "" {
+		var ctx *fsdb.Context
+		if ctx, err = o.db.Contexts.Get(request.ContextName); err != nil {
+			err = fmt.Errorf("could not find context %s: %v", request.ContextName, err)
+			return
+		}
+		contextContent = ctx.Content
+	}
+
+	var patternContent string
 	if request.PatternName != "" {
-		var pattern *db.Pattern
-		if pattern, err = o.db.Patterns.GetPattern(request.PatternName, request.PatternVariables); err != nil {
+		var pattern *fsdb.Pattern
+		if pattern, err = o.db.Patterns.GetApplyVariables(request.PatternName, request.PatternVariables); err != nil {
 			err = fmt.Errorf("could not find pattern %s: %v", request.PatternName, err)
 			return
 		}
 
 		if pattern.Pattern != "" {
-			ret.Pattern = pattern.Pattern
+			patternContent = pattern.Pattern
 		}
 	}
 
-	ret.Message = request.Message
+	systemMessage := strings.TrimSpace(contextContent) + strings.TrimSpace(patternContent)
+	if request.Language != "" {
+		systemMessage = fmt.Sprintf("%s. Please use the language '%s' for the output.", systemMessage, request.Language)
+	}
+	userMessage := strings.TrimSpace(request.Message)
+
+	if raw {
+		// use the user role instead of the system role in raw mode
+		message := systemMessage + userMessage
+		if message != "" {
+			session.Append(&common.Message{Role: goopenai.ChatMessageRoleUser, Content: message})
+		}
+	} else {
+		if systemMessage != "" {
+			session.Append(&common.Message{Role: goopenai.ChatMessageRoleSystem, Content: systemMessage})
+		}
+		if userMessage != "" {
+			session.Append(&common.Message{Role: goopenai.ChatMessageRoleUser, Content: userMessage})
+		}
+	}
+
+	if session.IsEmpty() {
+		session = nil
+		err = fmt.Errorf(NoSessionPatternUserMessages)
+	}
 	return
 }
-
-type Chat struct {
-	Context  string
-	Pattern  string
-	Message  string
-	Session  *db.Session
-	Language string
-}

core/chatter_test.go

@@ -1,21 +0,0 @@
-package core
-
-import (
-	"testing"
-)
-
-func TestBuildChatSession(t *testing.T) {
-	chat := &Chat{
-		Context: "test context",
-		Pattern: "test pattern",
-		Message: "test message",
-	}
-	session, err := chat.BuildChatSession(false)
-	if err != nil {
-		t.Fatalf("BuildChatSession() error = %v", err)
-	}
-
-	if session == nil {
-		t.Fatalf("BuildChatSession() returned nil session")
-	}
-}

core/fabric.go

@@ -1,295 +0,0 @@
-package core
-
-import (
-	"bytes"
-	"fmt"
-	"os"
-	"strconv"
-	"strings"
-
-	"github.com/danielmiessler/fabric/vendors/groq"
-	"github.com/danielmiessler/fabric/vendors/mistral"
-	goopenai "github.com/sashabaranov/go-openai"
-
-	"github.com/atotto/clipboard"
-	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/db"
-	"github.com/danielmiessler/fabric/jina"
-	"github.com/danielmiessler/fabric/vendors/anthropic"
-	"github.com/danielmiessler/fabric/vendors/azure"
-	"github.com/danielmiessler/fabric/vendors/dryrun"
-	"github.com/danielmiessler/fabric/vendors/gemini"
-	"github.com/danielmiessler/fabric/vendors/ollama"
-	"github.com/danielmiessler/fabric/vendors/openai"
-	"github.com/danielmiessler/fabric/vendors/openrouter"
-	"github.com/danielmiessler/fabric/vendors/siliconcloud"
-	"github.com/danielmiessler/fabric/youtube"
-	"github.com/pkg/errors"
-)
-
-const DefaultPatternsGitRepoUrl = "https://github.com/danielmiessler/fabric.git"
-const DefaultPatternsGitRepoFolder = "patterns"
-
-const NoSessionPatternUserMessages = "no session, pattern or user messages provided"
-
-func NewFabric(db *db.Db) (ret *Fabric, err error) {
-	ret = NewFabricBase(db)
-	err = ret.Configure()
-	return
-}
-
-func NewFabricForSetup(db *db.Db) (ret *Fabric) {
-	ret = NewFabricBase(db)
-	_ = ret.Configure()
-	return
-}
-
-// NewFabricBase Create a new Fabric from a list of already configured VendorsController
-func NewFabricBase(db *db.Db) (ret *Fabric) {
-
-	ret = &Fabric{
-		VendorsManager: NewVendorsManager(),
-		Db:             db,
-		VendorsAll:     NewVendorsManager(),
-		PatternsLoader: NewPatternsLoader(db.Patterns),
-		YouTube:        youtube.NewYouTube(),
-		Jina:           jina.NewClient(),
-	}
-
-	label := "Default"
-	ret.Configurable = &common.Configurable{
-		Label:           label,
-		EnvNamePrefix:   common.BuildEnvVariablePrefix(label),
-		ConfigureCustom: ret.configure,
-	}
-
-	ret.DefaultVendor = ret.AddSetting("Vendor", true)
-	ret.DefaultModel = ret.AddSetupQuestionCustom("Model", true,
-		"Enter the index the name of your default model")
-
-	ret.VendorsAll.AddVendors(openai.NewClient(), azure.NewClient(), ollama.NewClient(), groq.NewClient(),
-		gemini.NewClient(), anthropic.NewClient(), siliconcloud.NewClient(), openrouter.NewClient(), mistral.NewClient(), dryrun.NewClient())
-
-	return
-}
-
-type Fabric struct {
-	*common.Configurable
-	*VendorsManager
-	VendorsAll *VendorsManager
-	*PatternsLoader
-	*youtube.YouTube
-	Jina *jina.Client
-
-	Db *db.Db
-
-	DefaultVendor *common.Setting
-	DefaultModel  *common.SetupQuestion
-}
-
-type ChannelName struct {
-	channel chan []string
-	name    string
-}
-
-func (o *Fabric) SaveEnvFile() (err error) {
-	// Now create the .env with all configured VendorsController info
-	var envFileContent bytes.Buffer
-
-	o.Settings.FillEnvFileContent(&envFileContent)
-	o.PatternsLoader.SetupFillEnvFileContent(&envFileContent)
-
-	for _, vendor := range o.Vendors {
-		vendor.SetupFillEnvFileContent(&envFileContent)
-	}
-
-	o.YouTube.SetupFillEnvFileContent(&envFileContent)
-	o.Jina.SetupFillEnvFileContent(&envFileContent)
-
-	err = o.Db.SaveEnv(envFileContent.String())
-	return
-}
-
-func (o *Fabric) Setup() (err error) {
-	if err = o.SetupVendors(); err != nil {
-		return
-	}
-
-	if err = o.SetupDefaultModel(); err != nil {
-		return
-	}
-
-	_ = o.YouTube.SetupOrSkip()
-
-	if err = o.Jina.SetupOrSkip(); err != nil {
-		return
-	}
-
-	if err = o.PatternsLoader.Setup(); err != nil {
-		return
-	}
-
-	err = o.SaveEnvFile()
-
-	return
-}
-
-func (o *Fabric) SetupDefaultModel() (err error) {
-	vendorsModels := o.GetModels()
-
-	vendorsModels.Print()
-
-	if err = o.Ask(o.Label); err != nil {
-		return
-	}
-
-	index, parseErr := strconv.Atoi(o.DefaultModel.Value)
-	if parseErr == nil {
-		o.DefaultVendor.Value, o.DefaultModel.Value = vendorsModels.GetVendorAndModelByModelIndex(index)
-	} else {
-		o.DefaultVendor.Value = vendorsModels.FindVendorsByModelFirst(o.DefaultModel.Value)
-	}
-
-	//verify
-	vendorNames := vendorsModels.FindVendorsByModel(o.DefaultModel.Value)
-	if len(vendorNames) == 0 {
-		err = errors.Errorf("You need to chose an available default model.")
-		return
-	}
-
-	fmt.Println()
-	o.DefaultVendor.Print()
-	o.DefaultModel.Print()
-
-	err = o.SaveEnvFile()
-
-	return
-}
-
-func (o *Fabric) SetupVendors() (err error) {
-	o.Models = nil
-	if o.Vendors, err = o.VendorsAll.Setup(); err != nil {
-		return
-	}
-
-	if !o.HasVendors() {
-		err = errors.New("No vendors configured")
-		return
-	}
-
-	err = o.SaveEnvFile()
-
-	return
-}
-
-func (o *Fabric) SetupVendor(vendorName string) (err error) {
-	if err = o.VendorsAll.SetupVendor(vendorName); err != nil {
-		return
-	}
-	err = o.SaveEnvFile()
-	return
-}
-
-// Configure buildClient VendorsController based on the environment variables
-func (o *Fabric) configure() (err error) {
-	for _, vendor := range o.VendorsAll.Vendors {
-		if vendorErr := vendor.Configure(); vendorErr == nil {
-			o.AddVendors(vendor)
-		}
-	}
-	if err = o.PatternsLoader.Configure(); err != nil {
-		return
-	}
-
-	//YouTube and Jina are not mandatory, so ignore not configured error
-	_ = o.YouTube.Configure()
-	_ = o.Jina.Configure()
-
-	return
-}
-
-func (o *Fabric) GetChatter(model string, stream bool, dryRun bool) (ret *Chatter, err error) {
-	ret = &Chatter{
-		db:     o.Db,
-		Stream: stream,
-		DryRun: dryRun,
-	}
-
-	if dryRun {
-		ret.vendor = dryrun.NewClient()
-		ret.model = model
-		if ret.model == "" {
-			ret.model = o.DefaultModel.Value
-		}
-	} else if model == "" {
-		ret.vendor = o.FindByName(o.DefaultVendor.Value)
-		ret.model = o.DefaultModel.Value
-	} else {
-		ret.vendor = o.FindByName(o.GetModels().FindVendorsByModelFirst(model))
-		ret.model = model
-	}
-
-	if ret.vendor == nil {
-		err = fmt.Errorf(
-			"could not find vendor.\n Model = %s\n DefaultModel = %s\n DefaultVendor = %s",
-			model, o.DefaultModel.Value, o.DefaultVendor.Value)
-		return
-	}
-	return
-}
-
-func (o *Fabric) CopyToClipboard(message string) (err error) {
-	if err = clipboard.WriteAll(message); err != nil {
-		err = fmt.Errorf("could not copy to clipboard: %v", err)
-	}
-	return
-}
-
-func (o *Fabric) CreateOutputFile(message string, fileName string) (err error) {
-	var file *os.File
-	if file, err = os.Create(fileName); err != nil {
-		err = fmt.Errorf("error creating file: %v", err)
-		return
-	}
-	defer file.Close()
-	if _, err = file.WriteString(message); err != nil {
-		err = fmt.Errorf("error writing to file: %v", err)
-	}
-	return
-}
-
-func (o *Chat) BuildChatSession(raw bool) (ret *db.Session, err error) {
-	// new messages will be appended to the session and used to send the message
-	if o.Session != nil {
-		ret = o.Session
-	} else {
-		ret = &db.Session{}
-	}
-
-	systemMessage := strings.TrimSpace(o.Context) + strings.TrimSpace(o.Pattern)
-	if o.Language != "" {
-		systemMessage = fmt.Sprintf("%s. Please use the language '%s' for the output.", systemMessage, o.Language)
-	}
-	userMessage := strings.TrimSpace(o.Message)
-
-	if raw {
-		// use the user role instead of the system role in raw mode
-		message := systemMessage + userMessage
-		if message != "" {
-			ret.Append(&common.Message{Role: goopenai.ChatMessageRoleUser, Content: message})
-		}
-	} else {
-		if systemMessage != "" {
-			ret.Append(&common.Message{Role: goopenai.ChatMessageRoleSystem, Content: systemMessage})
-		}
-		if userMessage != "" {
-			ret.Append(&common.Message{Role: goopenai.ChatMessageRoleUser, Content: userMessage})
-		}
-	}
-
-	if ret.IsEmpty() {
-		ret = nil
-		err = fmt.Errorf(NoSessionPatternUserMessages)
-	}
-	return
-}

core/fabric_test.go

@@ -1,49 +0,0 @@
-package core
-
-import (
-	"os"
-	"testing"
-
-	"github.com/danielmiessler/fabric/db"
-)
-
-func TestNewFabric(t *testing.T) {
-	_, err := NewFabric(db.NewDb(os.TempDir()))
-	if err == nil {
-		t.Fatal("without setup error expected")
-	}
-}
-
-func TestSaveEnvFile(t *testing.T) {
-	fabric := NewFabricBase(db.NewDb(os.TempDir()))
-
-	err := fabric.SaveEnvFile()
-	if err != nil {
-		t.Fatalf("SaveEnvFile() error = %v", err)
-	}
-}
-
-func TestCopyToClipboard(t *testing.T) {
-	t.Skip("skipping test, because of docker env. in ci.")
-	fabric := NewFabricBase(db.NewDb(os.TempDir()))
-
-	message := "test message"
-	err := fabric.CopyToClipboard(message)
-	if err != nil {
-		t.Fatalf("CopyToClipboard() error = %v", err)
-	}
-}
-
-func TestCreateOutputFile(t *testing.T) {
-	mockDb := &db.Db{}
-	fabric := NewFabricBase(mockDb)
-
-	fileName := "test_output.txt"
-	message := "test message"
-	err := fabric.CreateOutputFile(message, fileName)
-	if err != nil {
-		t.Fatalf("CreateOutputFile() error = %v", err)
-	}
-
-	defer os.Remove(fileName)
-}

core/models.go

@@ -1,97 +0,0 @@
-package core
-
-import (
-	"fmt"
-	"sort"
-)
-
-func NewVendorsModels() *VendorsModels {
-	return &VendorsModels{VendorsModels: make(map[string][]string)}
-}
-
-type VendorsModels struct {
-	Vendors       []string
-	VendorsModels map[string][]string
-	Errs          []error
-}
-
-func (o *VendorsModels) AddVendorModels(vendor string, models []string) {
-	o.Vendors = append(o.Vendors, vendor)
-	o.VendorsModels[vendor] = models
-}
-
-func (o *VendorsModels) GetVendorAndModelByModelIndex(modelIndex int) (vendor string, model string) {
-	vendorModelIndexFrom := 0
-	vendorModelIndexTo := 0
-	for _, currenVendor := range o.Vendors {
-		vendorModelIndexFrom = vendorModelIndexTo + 1
-		vendorModelIndexTo = vendorModelIndexFrom + len(o.VendorsModels[currenVendor]) - 1
-
-		if modelIndex >= vendorModelIndexFrom && modelIndex <= vendorModelIndexTo {
-			vendor = currenVendor
-			model = o.VendorsModels[currenVendor][modelIndex-vendorModelIndexFrom]
-			break
-		}
-	}
-	return
-}
-
-func (o *VendorsModels) AddError(err error) {
-	o.Errs = append(o.Errs, err)
-}
-
-func (o *VendorsModels) Print() {
-	fmt.Printf("\nAvailable vendor models:\n")
-
-	sort.Strings(o.Vendors)
-
-	var currentModelIndex int
-	for _, vendor := range o.Vendors {
-		fmt.Println()
-		fmt.Printf("%s\n", vendor)
-		fmt.Println()
-		currentModelIndex = o.PrintVendor(vendor, currentModelIndex)
-	}
-	return
-}
-
-func (o *VendorsModels) PrintVendor(vendor string, modelIndex int) (currentModelIndex int) {
-	currentModelIndex = modelIndex
-	models := o.VendorsModels[vendor]
-	for _, model := range models {
-		currentModelIndex++
-		fmt.Printf("\t[%d]\t%s\n", currentModelIndex, model)
-	}
-	fmt.Println()
-	return
-}
-
-func (o *VendorsModels) GetVendorModels(vendor string) (models []string) {
-	models = o.VendorsModels[vendor]
-	return
-}
-
-func (o *VendorsModels) HasVendor(vendor string) (ret bool) {
-	ret = o.VendorsModels[vendor] != nil
-	return
-}
-
-func (o *VendorsModels) FindVendorsByModelFirst(model string) (ret string) {
-	vendors := o.FindVendorsByModel(model)
-	if len(vendors) > 0 {
-		ret = vendors[0]
-	}
-	return
-}
-
-func (o *VendorsModels) FindVendorsByModel(model string) (vendors []string) {
-	for vendor, models := range o.VendorsModels {
-		for _, m := range models {
-			if m == model {
-				vendors = append(vendors, vendor)
-				continue
-			}
-		}
-	}
-	return
-}

core/models_test.go

@@ -1,52 +0,0 @@
-package core
-
-import (
-	"errors"
-	"testing"
-)
-
-func TestNewVendorsModels(t *testing.T) {
-	vendors := NewVendorsModels()
-	if vendors == nil {
-		t.Fatalf("NewVendorsModels() returned nil")
-	}
-	if len(vendors.VendorsModels) != 0 {
-		t.Fatalf("NewVendorsModels() returned non-empty VendorsModels map")
-	}
-}
-
-func TestFindVendorsByModelFirst(t *testing.T) {
-	vendors := NewVendorsModels()
-	vendors.AddVendorModels("vendor1", []string{"model1", "model2"})
-	vendor := vendors.FindVendorsByModelFirst("model1")
-	if vendor != "vendor1" {
-		t.Fatalf("FindVendorsByModelFirst() = %v, want %v", vendor, "vendor1")
-	}
-}
-
-func TestFindVendorsByModel(t *testing.T) {
-	vendors := NewVendorsModels()
-	vendors.AddVendorModels("vendor1", []string{"model1", "model2"})
-	foundVendors := vendors.FindVendorsByModel("model1")
-	if len(foundVendors) != 1 || foundVendors[0] != "vendor1" {
-		t.Fatalf("FindVendorsByModel() = %v, want %v", foundVendors, []string{"vendor1"})
-	}
-}
-
-func TestAddVendorModels(t *testing.T) {
-	vendors := NewVendorsModels()
-	vendors.AddVendorModels("vendor1", []string{"model1", "model2"})
-	models := vendors.GetVendorModels("vendor1")
-	if len(models) != 2 {
-		t.Fatalf("AddVendorModels() failed to add models")
-	}
-}
-
-func TestAddError(t *testing.T) {
-	vendors := NewVendorsModels()
-	err := errors.New("sample error")
-	vendors.AddError(err)
-	if len(vendors.Errs) != 1 {
-		t.Fatalf("AddError() failed to add error")
-	}
-}

core/plugin_registry.go

@@ -0,0 +1,203 @@
+package core
+
+import (
+	"bytes"
+	"fmt"
+	"github.com/danielmiessler/fabric/common"
+	"github.com/danielmiessler/fabric/plugins/ai/azure"
+	"github.com/danielmiessler/fabric/plugins/tools"
+	"github.com/samber/lo"
+	"strconv"
+
+	"github.com/danielmiessler/fabric/plugins"
+	"github.com/danielmiessler/fabric/plugins/ai"
+	"github.com/danielmiessler/fabric/plugins/ai/anthropic"
+	"github.com/danielmiessler/fabric/plugins/ai/dryrun"
+	"github.com/danielmiessler/fabric/plugins/ai/gemini"
+	"github.com/danielmiessler/fabric/plugins/ai/groq"
+	"github.com/danielmiessler/fabric/plugins/ai/mistral"
+	"github.com/danielmiessler/fabric/plugins/ai/ollama"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
+	"github.com/danielmiessler/fabric/plugins/ai/openrouter"
+	"github.com/danielmiessler/fabric/plugins/ai/siliconcloud"
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"github.com/danielmiessler/fabric/plugins/tools/jina"
+	"github.com/danielmiessler/fabric/plugins/tools/lang"
+	"github.com/danielmiessler/fabric/plugins/tools/youtube"
+)
+
+func NewPluginRegistry(db *fsdb.Db) (ret *PluginRegistry) {
+	ret = &PluginRegistry{
+		Db:             db,
+		VendorManager:  ai.NewVendorsManager(),
+		VendorsAll:     ai.NewVendorsManager(),
+		PatternsLoader: tools.NewPatternsLoader(db.Patterns),
+		YouTube:        youtube.NewYouTube(),
+		Language:       lang.NewLanguage(),
+		Jina:           jina.NewClient(),
+	}
+
+	ret.Defaults = tools.NeeDefaults(ret.VendorManager.GetModels)
+
+	ret.VendorsAll.AddVendors(openai.NewClient(), ollama.NewClient(), azure.NewClient(), groq.NewClient(),
+		gemini.NewClient(), anthropic.NewClient(), siliconcloud.NewClient(), openrouter.NewClient(), mistral.NewClient())
+	_ = ret.Configure()
+
+	return
+}
+
+type PluginRegistry struct {
+	Db *fsdb.Db
+
+	VendorManager  *ai.VendorsManager
+	VendorsAll     *ai.VendorsManager
+	Defaults       *tools.Defaults
+	PatternsLoader *tools.PatternsLoader
+	YouTube        *youtube.YouTube
+	Language       *lang.Language
+	Jina           *jina.Client
+}
+
+func (o *PluginRegistry) SaveEnvFile() (err error) {
+	// Now create the .env with all configured VendorsController info
+	var envFileContent bytes.Buffer
+
+	o.Defaults.Settings.FillEnvFileContent(&envFileContent)
+	o.PatternsLoader.SetupFillEnvFileContent(&envFileContent)
+
+	for _, vendor := range o.VendorManager.Vendors {
+		vendor.SetupFillEnvFileContent(&envFileContent)
+	}
+
+	o.YouTube.SetupFillEnvFileContent(&envFileContent)
+	o.Jina.SetupFillEnvFileContent(&envFileContent)
+	o.Language.SetupFillEnvFileContent(&envFileContent)
+
+	err = o.Db.SaveEnv(envFileContent.String())
+	return
+}
+
+func (o *PluginRegistry) Setup() (err error) {
+	setupQuestion := plugins.NewSetupQuestion("Enter the number of the plugin to setup")
+	groupsPlugins := common.NewGroupsItemsSelector[plugins.Plugin]("Available plugins",
+		func(plugin plugins.Plugin) string {
+			var configuredLabel string
+			if plugin.IsConfigured() {
+				configuredLabel = " (configured)"
+			} else {
+				configuredLabel = ""
+			}
+			return fmt.Sprintf("%v%v", plugin.GetSetupDescription(), configuredLabel)
+		})
+
+	groupsPlugins.AddGroupItems("AI Vendors [at least one, required]", lo.Map(o.VendorsAll.Vendors,
+		func(vendor ai.Vendor, _ int) plugins.Plugin {
+			return vendor
+		})...)
+
+	groupsPlugins.AddGroupItems("Tools", o.Defaults, o.PatternsLoader, o.YouTube, o.Language, o.Jina)
+
+	for {
+		groupsPlugins.Print()
+
+		if answerErr := setupQuestion.Ask("Plugin Number"); answerErr != nil {
+			break
+		}
+
+		if setupQuestion.Value == "" {
+			break
+		}
+		number, parseErr := strconv.Atoi(setupQuestion.Value)
+		setupQuestion.Value = ""
+
+		if parseErr == nil {
+			var plugin plugins.Plugin
+			if _, plugin, err = groupsPlugins.GetGroupAndItemByItemNumber(number); err != nil {
+				return
+			}
+
+			if pluginSetupErr := plugin.Setup(); pluginSetupErr != nil {
+				println(pluginSetupErr.Error())
+			} else {
+				if err = o.SaveEnvFile(); err != nil {
+					break
+				}
+			}
+
+			if _, ok := o.VendorManager.VendorsByName[plugin.GetName()]; !ok {
+				if vendor, ok := plugin.(ai.Vendor); ok {
+					o.VendorManager.AddVendors(vendor)
+				}
+			}
+		} else {
+			break
+		}
+	}
+
+	err = o.SaveEnvFile()
+
+	return
+}
+
+func (o *PluginRegistry) SetupVendor(vendorName string) (err error) {
+	if err = o.VendorsAll.SetupVendor(vendorName, o.VendorManager.VendorsByName); err != nil {
+		return
+	}
+	err = o.SaveEnvFile()
+	return
+}
+
+// Configure buildClient VendorsController based on the environment variables
+func (o *PluginRegistry) Configure() (err error) {
+	for _, vendor := range o.VendorsAll.Vendors {
+		if vendorErr := vendor.Configure(); vendorErr == nil {
+			o.VendorManager.AddVendors(vendor)
+		}
+	}
+	_ = o.Defaults.Configure()
+	_ = o.PatternsLoader.Configure()
+
+	//YouTube and Jina are not mandatory, so ignore not configured error
+	_ = o.YouTube.Configure()
+	_ = o.Jina.Configure()
+	_ = o.Language.Configure()
+	return
+}
+
+func (o *PluginRegistry) GetChatter(model string, stream bool, dryRun bool) (ret *Chatter, err error) {
+	ret = &Chatter{
+		db:     o.Db,
+		Stream: stream,
+		DryRun: dryRun,
+	}
+
+	defaultModel := o.Defaults.Model.Value
+	defaultVendor := o.Defaults.Vendor.Value
+	vendorManager := o.VendorManager
+
+	if dryRun {
+		ret.vendor = dryrun.NewClient()
+		ret.model = model
+		if ret.model == "" {
+			ret.model = defaultModel
+		}
+	} else if model == "" {
+		ret.vendor = vendorManager.FindByName(defaultVendor)
+		ret.model = defaultModel
+	} else {
+		var models *ai.VendorsModels
+		if models, err = vendorManager.GetModels(); err != nil {
+			return
+		}
+		ret.vendor = vendorManager.FindByName(models.FindGroupsByItemFirst(model))
+		ret.model = model
+	}
+
+	if ret.vendor == nil {
+		err = fmt.Errorf(
+			"could not find vendor.\n Model = %s\n Model = %s\n Vendor = %s",
+			model, defaultModel, defaultVendor)
+		return
+	}
+	return
+}

core/plugin_registry_test.go

@@ -0,0 +1,16 @@
+package core
+
+import (
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"os"
+	"testing"
+)
+
+func TestSaveEnvFile(t *testing.T) {
+	registry := NewPluginRegistry(fsdb.NewDb(os.TempDir()))
+
+	err := registry.SaveEnvFile()
+	if err != nil {
+		t.Fatalf("SaveEnvFile() error = %v", err)
+	}
+}

core/vendors.go

@@ -1,120 +0,0 @@
-package core
-
-import (
-	"context"
-	"fmt"
-	"sync"
-
-	"github.com/danielmiessler/fabric/vendors"
-)
-
-func NewVendorsManager() *VendorsManager {
-	return &VendorsManager{
-		Vendors: map[string]vendors.Vendor{},
-	}
-}
-
-type VendorsManager struct {
-	Vendors map[string]vendors.Vendor
-	Models  *VendorsModels
-}
-
-func (o *VendorsManager) AddVendors(vendors ...vendors.Vendor) {
-	for _, vendor := range vendors {
-		o.Vendors[vendor.GetName()] = vendor
-	}
-}
-
-func (o *VendorsManager) GetModels() *VendorsModels {
-	if o.Models == nil {
-		o.readModels()
-	}
-	return o.Models
-}
-
-func (o *VendorsManager) HasVendors() bool {
-	return len(o.Vendors) > 0
-}
-
-func (o *VendorsManager) FindByName(name string) vendors.Vendor {
-	return o.Vendors[name]
-}
-
-func (o *VendorsManager) readModels() {
-	o.Models = NewVendorsModels()
-
-	var wg sync.WaitGroup
-	resultsChan := make(chan modelResult, len(o.Vendors))
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
-
-	for _, vendor := range o.Vendors {
-		wg.Add(1)
-		go o.fetchVendorModels(ctx, &wg, vendor, resultsChan)
-	}
-
-	// Wait for all goroutines to finish
-	go func() {
-		wg.Wait()
-		close(resultsChan)
-	}()
-
-	// Collect results
-	for result := range resultsChan {
-		if result.err != nil {
-			fmt.Println(result.vendorName, result.err)
-			o.Models.AddError(result.err)
-			cancel() // Cancel remaining goroutines if needed
-		} else {
-			o.Models.AddVendorModels(result.vendorName, result.models)
-		}
-	}
-}
-
-func (o *VendorsManager) fetchVendorModels(
-	ctx context.Context, wg *sync.WaitGroup, vendor vendors.Vendor, resultsChan chan<- modelResult) {
-
-	defer wg.Done()
-
-	models, err := vendor.ListModels()
-	select {
-	case <-ctx.Done():
-		// Context canceled, don't send the result
-		return
-	case resultsChan <- modelResult{vendorName: vendor.GetName(), models: models, err: err}:
-		// Result sent
-	}
-}
-
-func (o *VendorsManager) Setup() (ret map[string]vendors.Vendor, err error) {
-	ret = map[string]vendors.Vendor{}
-	for _, vendor := range o.Vendors {
-		fmt.Println()
-		if vendorErr := vendor.Setup(); vendorErr == nil {
-			fmt.Printf("[%v] configured\n", vendor.GetName())
-			ret[vendor.GetName()] = vendor
-		} else {
-			fmt.Printf("[%v] skipped\n", vendor.GetName())
-		}
-	}
-	return
-}
-
-func (o *VendorsManager) SetupVendor(vendorName string) (err error) {
-	vendor := o.FindByName(vendorName)
-	if vendor == nil {
-		err = fmt.Errorf("vendor %s not found", vendorName)
-		return
-	}
-	err = vendor.Setup()
-	if err != nil {
-		return
-	}
-	return
-}
-
-type modelResult struct {
-	vendorName string
-	models     []string
-	err        error
-}

core/vendors_test.go

@@ -1,131 +0,0 @@
-package core
-
-import (
-	"bytes"
-	"context"
-	"testing"
-
-	"github.com/danielmiessler/fabric/common"
-)
-
-func TestNewVendorsManager(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	if vendorsManager == nil {
-		t.Fatalf("NewVendorsManager() returned nil")
-	}
-}
-
-func TestAddVendors(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-
-	if _, exists := vendorsManager.Vendors[mockVendor.GetName()]; !exists {
-		t.Fatalf("AddVendors() did not add vendor")
-	}
-}
-
-func TestGetModels(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-
-	models := vendorsManager.GetModels()
-	if models == nil {
-		t.Fatalf("GetModels() returned nil")
-	}
-}
-
-func TestHasVendors(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	if vendorsManager.HasVendors() {
-		t.Fatalf("HasVendors() should return false for an empty manager")
-	}
-
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-	if !vendorsManager.HasVendors() {
-		t.Fatalf("HasVendors() should return true after adding a vendor")
-	}
-}
-
-func TestFindByName(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-
-	foundVendor := vendorsManager.FindByName("testVendor")
-	if foundVendor == nil {
-		t.Fatalf("FindByName() did not find added vendor")
-	}
-}
-
-func TestReadModels(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-
-	vendorsManager.readModels()
-	if vendorsManager.Models == nil || len(vendorsManager.Models.Vendors) == 0 {
-		t.Fatalf("readModels() did not read models correctly")
-	}
-}
-
-func TestSetup(t *testing.T) {
-	vendorsManager := NewVendorsManager()
-	mockVendor := &MockVendor{name: "testVendor"}
-	vendorsManager.AddVendors(mockVendor)
-
-	vendors, err := vendorsManager.Setup()
-	if err != nil {
-		t.Fatalf("Setup() error = %v", err)
-	}
-	if len(vendors) == 0 {
-		t.Fatalf("Setup() did not setup any vendors")
-	}
-}
-
-// MockVendor is a mock implementation of the Vendor interface for testing purposes.
-type MockVendor struct {
-	*common.Settings
-	name string
-}
-
-func (o *MockVendor) SendStream(messages []*common.Message, options *common.ChatOptions, strings chan string) error {
-	// TODO implement me
-	panic("implement me")
-}
-
-func (o *MockVendor) Send(ctx context.Context, messages []*common.Message, options *common.ChatOptions) (string, error) {
-	// TODO implement me
-	panic("implement me")
-}
-
-func (o *MockVendor) SetupFillEnvFileContent(buffer *bytes.Buffer) {
-	// TODO implement me
-	panic("implement me")
-}
-
-func (o *MockVendor) IsConfigured() bool {
-	return false
-}
-
-func (o *MockVendor) GetSettings() *common.Settings {
-	return o.Settings
-}
-
-func (o *MockVendor) GetName() string {
-	return o.name
-}
-
-func (o *MockVendor) Configure() error {
-	return nil
-}
-
-func (o *MockVendor) Setup() error {
-	return nil
-}
-
-func (o *MockVendor) ListModels() ([]string, error) {
-	return []string{"model1", "model2"}, nil
-}

db/patterns_test.go

@@ -1 +0,0 @@
-package db

db/sessions.go

@@ -1,62 +0,0 @@
-package db
-
-import (
-	"fmt"
-	"github.com/danielmiessler/fabric/common"
-)
-
-type Sessions struct {
-	*Storage
-}
-
-func (o *Sessions) GetOrCreateSession(name string) (session *Session, err error) {
-	session = &Session{Name: name}
-
-	if o.Exists(name) {
-		err = o.LoadAsJson(name, &session.Messages)
-	} else {
-		fmt.Printf("Creating new session: %s\n", name)
-	}
-	return
-}
-
-func (o *Sessions) PrintSession(name string) (err error) {
-	if o.Exists(name) {
-		var session Session
-		if err = o.LoadAsJson(name, &session.Messages); err == nil {
-			fmt.Println(session.String())
-		}
-	}
-	return
-}
-
-func (o *Sessions) SaveSession(session *Session) (err error) {
-	return o.SaveAsJson(session.Name, session.Messages)
-}
-
-type Session struct {
-	Name     string
-	Messages []*common.Message
-}
-
-func (o *Session) IsEmpty() bool {
-	return len(o.Messages) == 0
-}
-
-func (o *Session) Append(messages ...*common.Message) {
-	o.Messages = append(o.Messages, messages...)
-}
-
-func (o *Session) GetLastMessage() (ret *common.Message) {
-	if len(o.Messages) > 0 {
-		ret = o.Messages[len(o.Messages)-1]
-	}
-	return
-}
-
-func (o *Session) String() (ret string) {
-	for _, message := range o.Messages {
-		ret += fmt.Sprintf("\n\n\n[%v]>> \n\n\n%v", message.Role, message.Content)
-	}
-	return
-}

go.mod

@@ -5,6 +5,7 @@ go 1.22.5
 require (
 	github.com/anaskhan96/soup v1.2.5
 	github.com/atotto/clipboard v0.1.4
+	github.com/gin-gonic/gin v1.10.0
 	github.com/go-git/go-git/v5 v5.12.0
 	github.com/go-shiori/go-readability v0.0.0-20240923125239-59a7bd165825
 	github.com/google/generative-ai-go v0.18.0
@@ -17,7 +18,7 @@ require (
 	github.com/samber/lo v1.47.0
 	github.com/sashabaranov/go-openai v1.30.0
 	github.com/stretchr/testify v1.9.0
-	golang.org/x/text v0.18.0
+	golang.org/x/text v0.19.0
 	google.golang.org/api v0.197.0
 )
 
@@ -33,16 +34,26 @@ require (
 	github.com/ProtonMail/go-crypto v1.0.0 // indirect
 	github.com/andybalholm/cascadia v1.3.2 // indirect
 	github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de // indirect
+	github.com/bytedance/sonic v1.11.6 // indirect
+	github.com/bytedance/sonic/loader v0.1.1 // indirect
 	github.com/cloudflare/circl v1.4.0 // indirect
+	github.com/cloudwego/base64x v0.1.4 // indirect
+	github.com/cloudwego/iasm v0.2.0 // indirect
 	github.com/cyphar/filepath-securejoin v0.3.2 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/emirpasic/gods v1.18.1 // indirect
 	github.com/felixge/httpsnoop v1.0.4 // indirect
+	github.com/gabriel-vasile/mimetype v1.4.3 // indirect
+	github.com/gin-contrib/sse v0.1.0 // indirect
 	github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
 	github.com/go-git/go-billy/v5 v5.5.0 // indirect
 	github.com/go-logr/logr v1.4.2 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/go-playground/locales v0.14.1 // indirect
+	github.com/go-playground/universal-translator v0.18.1 // indirect
+	github.com/go-playground/validator/v10 v10.20.0 // indirect
 	github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c // indirect
+	github.com/goccy/go-json v0.10.2 // indirect
 	github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/google/s2a-go v0.1.8 // indirect
@@ -50,11 +61,20 @@ require (
 	github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
 	github.com/googleapis/gax-go/v2 v2.13.0 // indirect
 	github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/kevinburke/ssh_config v1.2.0 // indirect
+	github.com/klauspost/cpuid/v2 v2.2.7 // indirect
+	github.com/leodido/go-urn v1.4.0 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pelletier/go-toml/v2 v2.2.2 // indirect
 	github.com/pjbgf/sha1cd v0.3.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
 	github.com/skeema/knownhosts v1.3.0 // indirect
+	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
+	github.com/ugorji/go/codec v1.2.12 // indirect
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
 	go.opencensus.io v0.24.0 // indirect
 	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.55.0 // indirect
@@ -62,11 +82,12 @@ require (
 	go.opentelemetry.io/otel v1.30.0 // indirect
 	go.opentelemetry.io/otel/metric v1.30.0 // indirect
 	go.opentelemetry.io/otel/trace v1.30.0 // indirect
-	golang.org/x/crypto v0.27.0 // indirect
-	golang.org/x/net v0.29.0 // indirect
+	golang.org/x/arch v0.8.0 // indirect
+	golang.org/x/crypto v0.28.0 // indirect
+	golang.org/x/net v0.30.0 // indirect
 	golang.org/x/oauth2 v0.23.0 // indirect
 	golang.org/x/sync v0.8.0 // indirect
-	golang.org/x/sys v0.25.0 // indirect
+	golang.org/x/sys v0.26.0 // indirect
 	golang.org/x/time v0.6.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect

go.sum

@@ -32,11 +32,19 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkY
 github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
 github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
 github.com/bwesterb/go-ristretto v1.2.3/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
+github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc0=
+github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4=
+github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM=
+github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
 github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
 github.com/cloudflare/circl v1.3.3/go.mod h1:5XYMA4rFBvNIrhs50XuiBJ15vF2pZn4nnUKZrLbUZFA=
 github.com/cloudflare/circl v1.4.0 h1:BV7h5MgrktNzytKmWjpOtdYrf0lkkbF8YMlBGPhJQrY=
 github.com/cloudflare/circl v1.4.0/go.mod h1:PDRU+oXvdD7KCtgKxW95M5Z8BpSCJXQORiZFnBQS5QU=
+github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
+github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
+github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
+github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
 github.com/cyphar/filepath-securejoin v0.3.2 h1:QhZu5AxQ+o1XZH0Ye05YzvJ0kAdK6VQc0z9NNMek7gc=
 github.com/cyphar/filepath-securejoin v0.3.2/go.mod h1:F7i41x/9cBF7lzCrVsYs9fuzwRZm4NQsGTBdpp6mETc=
@@ -53,6 +61,12 @@ github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1m
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
 github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
+github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
+github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
+github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
+github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
+github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
 github.com/gliderlabs/ssh v0.3.7 h1:iV3Bqi942d9huXnzEF2Mt+CY9gLu8DNM4Obd+8bODRE=
 github.com/gliderlabs/ssh v0.3.7/go.mod h1:zpHEXBstFnQYtGnB8k8kQLol82umzn/2/snG7alWVD8=
 github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
@@ -68,10 +82,20 @@ github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
 github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
+github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
+github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
+github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
+github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
+github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
+github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
+github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
+github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
 github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c h1:wpkoddUomPfHiOziHZixGO5ZBS73cKqVzZipfrLmO1w=
 github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c/go.mod h1:oVDCh3qjJMLVUSILBRwrm+Bc6RNXGZYtoh9xdvf1ffM=
 github.com/go-shiori/go-readability v0.0.0-20240923125239-59a7bd165825 h1:CpSi7xiWqGaAqVn/2MsbRoDmPwXMvvQUu3hLjX1QrOM=
 github.com/go-shiori/go-readability v0.0.0-20240923125239-59a7bd165825/go.mod h1:YWa00ashoPZMAOElrSn4E1cJErhDVU6PWAll4Hxzn+w=
+github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
+github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
 github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f h1:3BSP1Tbs2djlpprl7wCLuiqMaUh5SJkkzI2gDs+FgLs=
 github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f/go.mod h1:Pcatq5tYkCW2Q6yrR2VRHlbHpZ/R4/7qyL1TCF7vl14=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -100,6 +124,7 @@ github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
 github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
 github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
 github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -115,8 +140,14 @@ github.com/jessevdk/go-flags v1.6.1 h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bB
 github.com/jessevdk/go-flags v1.6.1/go.mod h1:Mk8T1hIAWpOiJiHa9rJASDK2UGWji0EuPGBnNLMooyc=
 github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
 github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
+github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
+github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
 github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
 github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
+github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
+github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
+github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
+github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
@@ -124,9 +155,18 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
+github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
 github.com/liushuangls/go-anthropic/v2 v2.8.0 h1:0zH2jDNycbrlszxnLrG+Gx8vVT0yJAPWU4s3ZTkWzgI=
 github.com/liushuangls/go-anthropic/v2 v2.8.0/go.mod h1:8BKv/fkeTaL5R9R9bGkaknYBueyw2WxY20o7bImbOek=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
+github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
+github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
 github.com/ollama/ollama v0.3.11 h1:Fs1B5WjXYUvr5bkMZZpUJfiqIAxrymujRidFABwMeV8=
 github.com/ollama/ollama v0.3.11/go.mod h1:YrWoNkFnPOYsnDvsf/Ztb1wxU9/IXrNsQHqcxbY2r94=
 github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
@@ -135,6 +175,8 @@ github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
 github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
 github.com/otiai10/mint v1.5.1 h1:XaPLeE+9vGbuyEHem1JNk3bYc7KKqyI/na0/mLd/Kks=
 github.com/otiai10/mint v1.5.1/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM=
+github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
+github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
 github.com/pjbgf/sha1cd v0.3.0 h1:4D5XXmUUBUl/xQ6IjCkEAbqXskkq/4O7LmGn0AqMDs4=
 github.com/pjbgf/sha1cd v0.3.0/go.mod h1:nZ1rrWOcGJ5uZgEEVL1VUM9iRQiZvWdbZjkKyFzPPsI=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
@@ -158,15 +200,22 @@ github.com/skeema/knownhosts v1.3.0/go.mod h1:sPINvnADmT/qYH1kfv+ePMmOBTH6Tbl7b5
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
 github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
 github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
+github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
 github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
+github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
+github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
+github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
 github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
 github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
 github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
@@ -182,14 +231,17 @@ go.opentelemetry.io/otel/metric v1.30.0 h1:4xNulvn9gjzo4hjg+wzIKG7iNFEaBMX00Qd4Q
 go.opentelemetry.io/otel/metric v1.30.0/go.mod h1:aXTfST94tswhWEb+5QjlSqG+cZlmyXy/u8jFpor3WqQ=
 go.opentelemetry.io/otel/trace v1.30.0 h1:7UBkkYzeg3C7kQX8VAidWh2biiQbtAKjyIML8dQ9wmc=
 go.opentelemetry.io/otel/trace v1.30.0/go.mod h1:5EyKqTzzmyqB9bwtCCq6pDLktPK6fmGf/Dph+8VI02o=
+golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
+golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
+golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
 golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
 golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
-golang.org/x/crypto v0.27.0 h1:GXm2NjJrPaiv/h1tb2UH8QfgC/hOf/+z0p6PT8o1w7A=
-golang.org/x/crypto v0.27.0/go.mod h1:1Xngt8kV6Dvbssa53Ziq6Eqn0HqbZi5Z6R0ZpwQzt70=
+golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw=
+golang.org/x/crypto v0.28.0/go.mod h1:rmgy+3RHxRZMyY0jjAJShp2zgEdOqj2AO7U0pYmeQ7U=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
 golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
@@ -211,8 +263,8 @@ golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
 golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
-golang.org/x/net v0.29.0 h1:5ORfpBpCs4HzDYoodCDBbwHzdR5UrLBZ3sOnUJmFoHo=
-golang.org/x/net v0.29.0/go.mod h1:gLkgy8jTGERgjzMic6DS9+SP0ajcu6Xu3Orq/SpETg0=
+golang.org/x/net v0.30.0 h1:AcW1SDZMkb8IpzCdQUaIq2sP4sZ4zw+55h6ynffypl4=
+golang.org/x/net v0.30.0/go.mod h1:2wGyMJ5iFasEhkwi13ChkO/t1ECNC4X4eBKkVFyYFlU=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.23.0 h1:PbgcYx2W7i4LvjJWEbf0ngHV6qJYr86PkAV3bXdLEbs=
 golang.org/x/oauth2 v0.23.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
@@ -240,16 +292,16 @@ golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34=
-golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo=
+golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
 golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
 golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
 golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY=
-golang.org/x/term v0.24.0 h1:Mh5cbb+Zk2hqqXNO7S1iTjEphVL+jb8ZWaqh/g+JWkM=
-golang.org/x/term v0.24.0/go.mod h1:lOBK/LVxemqiMij05LGJ0tzNr8xlmwBRJ81PX6wVLH8=
+golang.org/x/term v0.25.0 h1:WtHI/ltw4NvSUig5KARz9h521QvRC8RmF/cuYqifU24=
+golang.org/x/term v0.25.0/go.mod h1:RPyXicDX+6vLxogjjRxjgD2TKtmAO6NZBsBRfrOLu7M=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
@@ -258,8 +310,8 @@ golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
 golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
-golang.org/x/text v0.18.0 h1:XvMDiNzPAl0jr17s6W9lcaIhGUfUORdGCNsuLmPG224=
-golang.org/x/text v0.18.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
+golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM=
+golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
 golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U=
 golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -314,3 +366,5 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
 honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50=
+rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=

patterns/create_design_document/system.md

@@ -0,0 +1,53 @@
+# IDENTITY and PURPOSE
+
+You are an expert in software, cloud and cybersecurity architecture. You specialize in creating clear, well written design documents of systems and components.
+
+# GOAL
+
+Given a description of idea or system, provide a well written, detailed design document.
+
+# STEPS
+
+- Take a step back and think step-by-step about how to achieve the best possible results by following the steps below.
+
+- Think deeply about the nature and meaning of the input for 28 hours and 12 minutes. 
+
+- Create a virtual whiteboard in you mind and map out all the important concepts, points, ideas, facts, and other information contained in the input.
+
+- Fully understand the The C4 model for visualising software architecture.
+
+- Appreciate the fact that each company is different. Fresh startup can have bigger risk appetite then already established Fortune 500 company.
+
+- Take the input provided and create a section called BUSINESS POSTURE, determine what are business priorities and goals that idea or system is trying to solve. Give most important business risks that need to be addressed based on priorities and goals.
+
+- Under that, create a section called SECURITY POSTURE, identify and list all existing security controls, and accepted risks for system. Focus on secure software development lifecycle and deployment model. Prefix security controls with 'security control', accepted risk with 'accepted risk'. Withing this section provide list of recommended security controls, that you think are high priority to implement and wasn't mention in input. Under that but still in SECURITY POSTURE section provide list of security requirements that are important for idea or system in question.
+
+- Under that, create a section called DESIGN. Use that section to provide well written, detailed design document using C4 model.
+
+- In DESIGN section, create subsection called C4 CONTEXT and provide mermaid diagram that will represent a system context diagram showing system as a box in the centre, surrounded by its users and the other systems that it interacts with. 
+
+- Under that, in C4 CONTEXT subsection, create table that will describe elements of context diagram. Include columns: 1. Name - name of element; 2. Type - type of element; 3. Description - description of element; 4. Responsibilities - responsibilities of element; 5. Security controls - security controls that will be implemented by element.
+
+- Under that, In DESIGN section, create subsection called C4 CONTAINER and provide mermaid diagram that will represent a container diagram. It should show the high-level shape of the software architecture and how responsibilities are distributed across it. It also shows the major technology choices and how the containers communicate with one another.
+
+- Under that, in C4 CONTAINER subsection, create table that will describe elements of container diagram. Include columns: 1. Name - name of element; 2. Type - type of element; 3. Description - description of element; 4. Responsibilities - responsibilities of element; 5. Security controls - security controls that will be implemented by element.
+
+- Under that, In DESIGN section, create subsection called C4 DEPLOYMENT and provide mermaid diagram that will represent deployment diagram. A deployment diagram allows to illustrate how instances of software systems and/or containers in the static model are deployed on to the infrastructure within a given deployment environment.
+
+- Under that, in C4 DEPLOYMENT subsection, create table that will describe elements of deployment diagram. Include columns: 1. Name - name of element; 2. Type - type of element; 3. Description - description of element; 4. Responsibilities - responsibilities of element; 5. Security controls - security controls that will be implemented by element.
+
+- Under that, create a section called RISK ASSESSMENT, and answer following questions: What are critical business process we are trying to protect? What data we are trying to protect and what is their sensitivity? 
+
+- Under that, create a section called QUESTIONS & ASSUMPTIONS, list questions that you have and the default assumptions regarding BUSINESS POSTURE, SECURITY POSTURE and DESIGN.
+
+# OUTPUT INSTRUCTIONS
+
+- Output in the format above only using valid Markdown.
+
+- Do not use bold or italic formatting in the Markdown (no asterisks).
+
+- Do not complain about anything, just do what you're told.
+
+# INPUT:
+
+INPUT:

patterns/extract_core_message/system.md

@@ -0,0 +1,39 @@
+# IDENTITY
+
+You are an expert at looking at a presentation, an essay, or a full body of lifetime work, and clearly and accurately articulating what the core message is.
+
+# GOAL
+
+- Produce a clear sentence that perfectly articulates the core message as presented in a given text or body of work.
+
+# EXAMPLE
+
+If the input is all of Victor Frankl's work, then the core message would be:
+
+Finding meaning in suffering is key to human resilience, purpose, and enduring life’s challenges.
+
+END EXAMPLE
+
+# STEPS
+
+- Fully digest the input. 
+
+- Determine if the input is a single text or a body of work.
+
+- Based on which it is, parse the thing that's supposed to be parsed.
+
+- Extract the core message from the parsed text into a single sentence.
+
+# OUTPUT
+
+- Output a single, 15-word sentence that perfectly articulates the core message as presented in the input.
+
+# OUTPUT INSTRUCTIONS
+
+- The sentence should be a single sentence that is 15 words or fewer, with no special formatting or anything else.
+
+- Do not include any setup to the sentence, e.g., "The core message is to…", etc. Just list the core message and nothing else.
+
+- ONLY OUTPUT THE CORE MESSAGE, not a setup to it, commentary on it, or anything else.
+
+- Do not ask questions or complain in any way about the task.

patterns/extract_latest_video/system.md

@@ -0,0 +1,23 @@
+# IDENTITY and PURPOSE
+
+You are an expert at extracting the latest video URL from a YouTube RSS feed.
+
+# Steps
+
+- Read the full RSS feed.
+
+- Find the latest posted video URL.
+
+- Output the full video URL and nothing else.
+
+# EXAMPLE OUTPUT
+
+https://www.youtube.com/watch?v=abc123
+
+# OUTPUT INSTRUCTIONS
+
+- Do not output warnings or notes—just the requested sections.
+
+# INPUT:
+
+INPUT:

patterns/extract_sponsors/system.md

@@ -5,7 +5,9 @@ You are an expert at extracting the sponsors and potential sponsors from a given
 # Steps
 
 - Consume the whole transcript so you understand what is content, what is meta information, etc.
+
 - Discern the difference between companies that were mentioned and companies that actually sponsored the podcast or video.
+
 - Output the following:
 
 ## OFFICIAL SPONSORS
@@ -15,36 +17,20 @@ You are an expert at extracting the sponsors and potential sponsors from a given
 - $SOURCE_CHANNEL$ | $SPONSOR3$ | $SPONSOR3_DESCRIPTION$ | $SPONSOR3_LINK$
 - And so on…
 
-## POTENTIAL SPONSORS
-
-- $SOURCE_CHANNEL$ | $SPONSOR1$ | $SPONSOR1_DESCRIPTION$ | $SPONSOR1_LINK$
-- $SOURCE_CHANNEL$ | $SPONSOR2$ | $SPONSOR2_DESCRIPTION$ | $SPONSOR2_LINK$
-- $SOURCE_CHANNEL$ | $SPONSOR3$ | $SPONSOR3_DESCRIPTION$ | $SPONSOR3_LINK$
-- And so on…
-
 # EXAMPLE OUTPUT
 
 ## OFFICIAL SPONSORS
 
-- AI Jason's YouTube Channel | Flair | Flair is a threat intel platform powered by AI. | https://flair.ai
-- Matthew Berman's YouTube Channel | Weaviate | Weviate is an open-source knowledge graph powered by ML. | https://weaviate.com
-- Unsupervised Learning Website | JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
-- The AI Junkie Podcast | JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
-
-## POTENTIAL SPONSORS
-
-- AI Jason's YouTube Channel | Flair | Flair is a threat intel platform powered by AI. | https://flair.ai
-- Matthew Berman's YouTube Channel | Weaviate | Weviate is an open-source knowledge graph powered by ML. | https://weaviate.com
-- Unsupervised Learning Website | JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
-- The AI Junkie Podcast | JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
+- Flair | Flair is a threat intel platform powered by AI. | https://flair.ai
+- Weaviate | Weviate is an open-source knowledge graph powered by ML. | https://weaviate.com
+- JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
+- JunaAI | JunaAI is a platform for AI-powered content creation. | https://junaai.com
 
 ## END EXAMPLE OUTPUT
 
 # OUTPUT INSTRUCTIONS
 
 - The official sponsor list should only include companies that officially sponsored the content in question.
-- The potential sponsor list should include companies that were mentioned during the content but that didn't officially sponsor.
-- Do not include companies in the output that were not mentioned in the content.
 - Do not output warnings or notes—just the requested sections.
 
 # INPUT:

patterns/raycast/capture_thinkers_work

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+# Required parameters:
+# @raycast.schemaVersion 1
+# @raycast.title Capture Thinkers Work
+# @raycast.mode fullOutput
+
+# Optional parameters:
+# @raycast.icon 🧠
+# @raycast.argument1 { "type": "text", "placeholder": "Input text", "optional": false, "percentEncoded": true}
+
+# Documentation:
+# @raycast.description Run fabric capture_thinkers_work on the input text
+# @raycast.author Daniel Miessler
+# @raycast.authorURL https://github.com/danielmiessler
+
+# Set PATH to include common locations and $HOME/go/bin
+PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$HOME/go/bin:$PATH"
+
+# Use the PATH to find and execute fabric
+if command -v fabric >/dev/null 2>&1; then
+    fabric -sp capture_thinkers_work "${1}"
+else
+    echo "Error: fabric command not found in PATH"
+    echo "Current PATH: $PATH"
+    exit 1
+fi

patterns/review_design/system.md

@@ -0,0 +1,61 @@
+# IDENTITY and PURPOSE
+
+You are an expert solution architect. 
+
+You fully digest input and review design.
+
+Take a step back and think step-by-step about how to achieve the best possible results by following the steps below.
+
+# STEPS
+
+Conduct a detailed review of the architecture design. Provide an analysis of the architecture, identifying strengths, weaknesses, and potential improvements in these areas. Specifically, evaluate the following:
+
+1. **Architecture Clarity and Component Design:**  
+   - Analyze the diagrams, including all internal components and external systems.
+   - Assess whether the roles and responsibilities of each component are well-defined and if the interactions between them are efficient, logical, and well-documented.
+   - Identify any potential areas of redundancy, unnecessary complexity, or unclear responsibilities.
+
+2. **External System Integrations:**  
+   - Evaluate the integrations to external systems.
+   - Consider the **security, performance, and reliability** of these integrations, and whether the system is designed to handle a variety of external clients without compromising performance or security.
+
+3. **Security Architecture:**  
+   - Assess the security mechanisms in place.
+   - Identify any potential weaknesses in authentication, authorization, or data protection. Consider whether the design follows best practices.
+   - Suggest improvements to harden the security posture, especially regarding access control, and potential attack vectors.
+
+4. **Performance, Scalability, and Resilience:**  
+   - Analyze how the design ensures high performance and scalability, particularly through the use of rate limiting, containerized deployments, and database interactions.
+   - Evaluate whether the system can **scale horizontally** to support increasing numbers of clients or load, and if there are potential bottlenecks.
+   - Assess fault tolerance and resilience. Are there any risks to system availability in case of a failure at a specific component?
+
+5. **Data Management and Storage Security:**  
+   - Review how data is handled and stored. Are these data stores designed to securely manage information?
+   - Assess if the **data flow** between components is optimized and secure. Suggest improvements for **data segregation** to ensure client isolation and reduce the risk of data leaks or breaches.
+
+6. **Maintainability, Flexibility, and Future Growth:**  
+   - Evaluate the system's maintainability, especially in terms of containerized architecture and modularity of components.
+   - Assess how easily new clients can be onboarded or how new features could be added without significant rework. Is the design flexible enough to adapt to evolving business needs?
+   - Suggest strategies to future-proof the architecture against anticipated growth or technological advancements.
+
+7. **Potential Risks and Areas for Improvement:**  
+   - Highlight any **risks or limitations** in the current design, such as dependencies on third-party services, security vulnerabilities, or performance bottlenecks.
+   - Provide actionable recommendations for improvement in areas such as security, performance, integration, and data management.
+
+8. **Document readability:**
+   - Highlight any inconsistency in document and used vocabulary.
+   - Suggest parts that need rewrite.
+
+Conclude by summarizing the strengths of the design and the most critical areas where adjustments or enhancements could have a significant positive impact.
+
+# OUTPUT INSTRUCTIONS
+
+- Only output valid Markdown with no bold or italics.
+
+- Do not give warnings or notes; only output the requested sections.
+
+- Ensure you follow ALL these instructions when creating your output.
+
+# INPUT
+
+INPUT:

patterns/transcribe_minutes/system.md

@@ -8,15 +8,15 @@ Take a step back and think step-by-step about how to achieve the best possible r
 
 - Fully digest the content provided.
 
-- Extract all actionables agreed within the meeting.
+- Extract all actionables agreed upon within the meeting.
 
 - Extract any interesting ideas brought up in the meeting. 
 
-- In a section called TITLE, write a 1 to 5 word title for the meeting
+- In a section called TITLE, write a 1 to 5 word title for the meeting.
 
 - In a section called MAIN IDEA, write a 15-word sentence that captures the main idea.
 
-- In a section called MINUTES, write 20 to 50 bullet points, highliting  of the most surprising, insightful, and/or interesting ideas that come up in the conversation. If there are less than 50 then collect all of them. Make sure you extract at least 20.
+- In a section called MINUTES, write 20 to 50 bullet points, highlighting of the most surprising, insightful, and/or interesting ideas that come up in the conversation. If there are less than 50 then collect all of them. Make sure you extract at least 20.
 
 - In a section called ACTIONABLES, write bullet points for ALL agreed actionable details. This includes cases where a speaker agrees to do or look into something. If there is a deadline mentioned, include it here.
 
@@ -24,16 +24,16 @@ Take a step back and think step-by-step about how to achieve the best possible r
 
 - In a section called CHALLENGES, identify and document any challenges or issues discussed during the meeting. Note any potential solutions or strategies proposed to address these challenges.
 
-- In a section caled NEXT STEPS, outline the next steps and actions to be taken after the meeting
+- In a section called NEXT STEPS, outline the next steps and actions to be taken after the meeting.
 
 # OUTPUT INSTRUCTIONS
 
 - Only output Markdown.
-- Write MINUTES as exxactly 15 words.
+- Write MINUTES as exactly 15 words.
 - Write ACTIONABLES as exactly 15 words.
 - Write DECISIONS as exactly 15 words.
 - Write CHALLENGES as 2-3 sentences.
-- Write NEXT STEPS a 2-3 sentences.
+- Write NEXT STEPS as 2-3 sentences.
 - Do not give warnings or notes; only output the requested sections.
 - Do not repeat ideas, quotes, facts, or resources.
 - You use bulleted lists for output, not numbered lists.

patterns/translate/system.md

@@ -0,0 +1,26 @@
+# IDENTITY and PURPOSE
+
+You are a an expert translator that takes sentence or documentation as input and do your best to translate it as accurately and perfectly in <Language> as possible.
+
+Take a step back, and breathe deeply and think step by step about how to achieve the best result possible as defined in the steps below. You have a lot of freedom to make this work well. You are the best translator that ever walked this earth.
+
+## OUTPUT SECTIONS
+
+- The original format of the input must remain intact.
+
+- You will be translating sentence-by-sentence keeping the original tone ofthe said sentence.
+
+- You will not be manipulate the wording to change the meaning.
+
+
+## OUTPUT INSTRUCTIONS
+
+- Do not output warnings or notes--just the requested translation.
+
+- Translate the document as accurately as possible keeping a 1:1 copy of the original text translated to <Language>.
+
+- Do not change the formatting, it must remain as-is.
+
+## INPUT
+
+INPUT:

plugins/ai/anthropic/anthropic.go

@@ -4,6 +4,8 @@ import (
 	"context"
 	"errors"
 	"fmt"
+
+	"github.com/danielmiessler/fabric/plugins"
 	goopenai "github.com/sashabaranov/go-openai"
 
 	"github.com/danielmiessler/fabric/common"
@@ -16,15 +18,15 @@ func NewClient() (ret *Client) {
 	vendorName := "Anthropic"
 	ret = &Client{}
 
-	ret.Configurable = &common.Configurable{
-		Label:           vendorName,
-		EnvNamePrefix:   common.BuildEnvVariablePrefix(vendorName),
+	ret.PluginBase = &plugins.PluginBase{
+		Name:            vendorName,
+		EnvNamePrefix:   plugins.BuildEnvVariablePrefix(vendorName),
 		ConfigureCustom: ret.configure,
 	}
 
 	ret.ApiBaseURL = ret.AddSetupQuestion("API Base URL", false)
 	ret.ApiBaseURL.Value = baseUrl
-	ret.ApiKey = ret.Configurable.AddSetupQuestion("API key", true)
+	ret.ApiKey = ret.PluginBase.AddSetupQuestion("API key", true)
 
 	// we could provide a setup question for the following settings
 	ret.maxTokens = 4096
@@ -39,9 +41,9 @@ func NewClient() (ret *Client) {
 }
 
 type Client struct {
-	*common.Configurable
-	ApiBaseURL *common.SetupQuestion
-	ApiKey     *common.SetupQuestion
+	*plugins.PluginBase
+	ApiBaseURL *plugins.SetupQuestion
+	ApiKey     *plugins.SetupQuestion
 
 	maxTokens                  int
 	defaultRequiredUserMessage string

plugins/ai/azure/azure.go

@@ -1,10 +1,10 @@
 package azure
 
 import (
+	"github.com/danielmiessler/fabric/plugins"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
 	"strings"
 
-	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/vendors/openai"
 	goopenai "github.com/sashabaranov/go-openai"
 )
 
@@ -19,7 +19,7 @@ func NewClient() (ret *Client) {
 
 type Client struct {
 	*openai.Client
-	ApiDeployments *common.SetupQuestion
+	ApiDeployments *plugins.SetupQuestion
 
 	apiDeployments []string
 }

plugins/ai/dryrun/dryrun.go

@@ -4,27 +4,18 @@ import (
 	"bytes"
 	"context"
 	"fmt"
+	"github.com/danielmiessler/fabric/plugins"
 	goopenai "github.com/sashabaranov/go-openai"
 
 	"github.com/danielmiessler/fabric/common"
 )
 
-type Client struct{}
+type Client struct {
+	*plugins.PluginBase
+}
 
 func NewClient() *Client {
-	return &Client{}
-}
-
-func (c *Client) GetName() string {
-	return "DryRun"
-}
-
-func (c *Client) IsConfigured() bool {
-	return true
-}
-
-func (c *Client) Configure() error {
-	return nil
+	return &Client{PluginBase: &plugins.PluginBase{Name: "DryRun"}}
 }
 
 func (c *Client) ListModels() ([]string, error) {

plugins/ai/gemini/gemini.go

@@ -4,6 +4,7 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"github.com/danielmiessler/fabric/plugins"
 	"strings"
 
 	"github.com/danielmiessler/fabric/common"
@@ -18,19 +19,19 @@ func NewClient() (ret *Client) {
 	vendorName := "Gemini"
 	ret = &Client{}
 
-	ret.Configurable = &common.Configurable{
-		Label:         vendorName,
-		EnvNamePrefix: common.BuildEnvVariablePrefix(vendorName),
+	ret.PluginBase = &plugins.PluginBase{
+		Name:          vendorName,
+		EnvNamePrefix: plugins.BuildEnvVariablePrefix(vendorName),
 	}
 
-	ret.ApiKey = ret.Configurable.AddSetupQuestion("API key", true)
+	ret.ApiKey = ret.PluginBase.AddSetupQuestion("API key", true)
 
 	return
 }
 
 type Client struct {
-	*common.Configurable
-	ApiKey *common.SetupQuestion
+	*plugins.PluginBase
+	ApiKey *plugins.SetupQuestion
 }
 
 func (o *Client) ListModels() (ret []string, err error) {

plugins/ai/groq/groq.go

@@ -1,7 +1,7 @@
 package groq
 
 import (
-	"github.com/danielmiessler/fabric/vendors/openai"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
 )
 
 func NewClient() (ret *Client) {

plugins/ai/mistral/mistral.go

@@ -1,7 +1,7 @@
 package mistral
 
 import (
-	"github.com/danielmiessler/fabric/vendors/openai"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
 )
 
 func NewClient() (ret *Client) {

plugins/ai/models.go

@@ -0,0 +1,13 @@
+package ai
+
+import (
+	"github.com/danielmiessler/fabric/common"
+)
+
+func NewVendorsModels() *VendorsModels {
+	return &VendorsModels{GroupsItemsSelectorString: common.NewGroupsItemsSelectorString("Available models")}
+}
+
+type VendorsModels struct {
+	*common.GroupsItemsSelectorString
+}

plugins/ai/models_test.go

@@ -0,0 +1,33 @@
+package ai
+
+import (
+	"testing"
+)
+
+func TestNewVendorsModels(t *testing.T) {
+	vendors := NewVendorsModels()
+	if vendors == nil {
+		t.Fatalf("NewVendorsModels() returned nil")
+	}
+	if len(vendors.GroupsItems) != 0 {
+		t.Fatalf("NewVendorsModels() returned non-empty VendorsModels map")
+	}
+}
+
+func TestFindVendorsByModelFirst(t *testing.T) {
+	vendors := NewVendorsModels()
+	vendors.AddGroupItems("vendor1", []string{"model1", "model2"}...)
+	vendor := vendors.FindGroupsByItemFirst("model1")
+	if vendor != "vendor1" {
+		t.Fatalf("FindVendorsByModelFirst() = %v, want %v", vendor, "vendor1")
+	}
+}
+
+func TestFindVendorsByModel(t *testing.T) {
+	vendors := NewVendorsModels()
+	vendors.AddGroupItems("vendor1", []string{"model1", "model2"}...)
+	foundVendors := vendors.FindGroupsByItem("model1")
+	if len(foundVendors) != 1 || foundVendors[0] != "vendor1" {
+		t.Fatalf("FindVendorsByModel() = %v, want %v", foundVendors, []string{"vendor1"})
+	}
+}

plugins/ai/ollama/ollama.go

@@ -3,6 +3,7 @@ package ollama
 import (
 	"context"
 	"fmt"
+	"github.com/danielmiessler/fabric/plugins"
 	"net/http"
 	"net/url"
 	"time"
@@ -17,21 +18,21 @@ func NewClient() (ret *Client) {
 	vendorName := "Ollama"
 	ret = &Client{}
 
-	ret.Configurable = &common.Configurable{
-		Label:           vendorName,
-		EnvNamePrefix:   common.BuildEnvVariablePrefix(vendorName),
+	ret.PluginBase = &plugins.PluginBase{
+		Name:            vendorName,
+		EnvNamePrefix:   plugins.BuildEnvVariablePrefix(vendorName),
 		ConfigureCustom: ret.configure,
 	}
 
-	ret.ApiUrl = ret.Configurable.AddSetupQuestionCustom("API URL", true,
+	ret.ApiUrl = ret.AddSetupQuestionCustom("API URL", true,
 		"Enter your Ollama URL (as a reminder, it is usually http://localhost:11434)")
 
 	return
 }
 
 type Client struct {
-	*common.Configurable
-	ApiUrl *common.SetupQuestion
+	*plugins.PluginBase
+	ApiUrl *plugins.SetupQuestion
 
 	apiUrl *url.URL
 	client *ollamaapi.Client

plugins/ai/openai/openai.go

@@ -4,6 +4,7 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"github.com/danielmiessler/fabric/plugins"
 	"io"
 	"log/slog"
 
@@ -24,9 +25,9 @@ func NewClientCompatible(vendorName string, defaultBaseUrl string, configureCust
 		configureCustom = ret.configure
 	}
 
-	ret.Configurable = &common.Configurable{
-		Label:           vendorName,
-		EnvNamePrefix:   common.BuildEnvVariablePrefix(vendorName),
+	ret.PluginBase = &plugins.PluginBase{
+		Name:            vendorName,
+		EnvNamePrefix:   plugins.BuildEnvVariablePrefix(vendorName),
 		ConfigureCustom: configureCustom,
 	}
 
@@ -38,9 +39,9 @@ func NewClientCompatible(vendorName string, defaultBaseUrl string, configureCust
 }
 
 type Client struct {
-	*common.Configurable
-	ApiKey     *common.SetupQuestion
-	ApiBaseURL *common.SetupQuestion
+	*plugins.PluginBase
+	ApiKey     *plugins.SetupQuestion
+	ApiBaseURL *plugins.SetupQuestion
 	ApiClient  *openai.Client
 }
 

plugins/ai/openrouter/openrouter.go

@@ -1,7 +1,7 @@
 package openrouter
 
 import (
-	"github.com/danielmiessler/fabric/vendors/openai"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
 )
 
 func NewClient() (ret *Client) {

plugins/ai/siliconcloud/siliconcloud.go

@@ -1,7 +1,7 @@
 package siliconcloud
 
 import (
-	"github.com/danielmiessler/fabric/vendors/openai"
+	"github.com/danielmiessler/fabric/plugins/ai/openai"
 )
 
 func NewClient() (ret *Client) {

plugins/ai/vendor.go

@@ -1,19 +1,15 @@
-package vendors
+package ai
 
 import (
-	"bytes"
 	"context"
+	"github.com/danielmiessler/fabric/plugins"
 
 	"github.com/danielmiessler/fabric/common"
 )
 
 type Vendor interface {
-	GetName() string
-	IsConfigured() bool
-	Configure() error
+	plugins.Plugin
 	ListModels() ([]string, error)
 	SendStream([]*common.Message, *common.ChatOptions, chan string) error
 	Send(context.Context, []*common.Message, *common.ChatOptions) (string, error)
-	Setup() error
-	SetupFillEnvFileContent(*bytes.Buffer)
 }

plugins/ai/vendors.go

@@ -0,0 +1,147 @@
+package ai
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"github.com/danielmiessler/fabric/plugins"
+	"sync"
+)
+
+func NewVendorsManager() *VendorsManager {
+	return &VendorsManager{
+		Vendors:       []Vendor{},
+		VendorsByName: map[string]Vendor{},
+	}
+}
+
+type VendorsManager struct {
+	*plugins.PluginBase
+	Vendors       []Vendor
+	VendorsByName map[string]Vendor
+	Models        *VendorsModels
+}
+
+func (o *VendorsManager) AddVendors(vendors ...Vendor) {
+	for _, vendor := range vendors {
+		o.VendorsByName[vendor.GetName()] = vendor
+		o.Vendors = append(o.Vendors, vendor)
+	}
+}
+
+func (o *VendorsManager) SetupFillEnvFileContent(envFileContent *bytes.Buffer) {
+	for _, vendor := range o.Vendors {
+		vendor.SetupFillEnvFileContent(envFileContent)
+	}
+}
+
+func (o *VendorsManager) GetModels() (ret *VendorsModels, err error) {
+	if o.Models == nil {
+		err = o.readModels()
+	}
+	ret = o.Models
+	return
+}
+
+func (o *VendorsManager) Configure() (err error) {
+	for _, vendor := range o.Vendors {
+		_ = vendor.Configure()
+	}
+	return
+}
+
+func (o *VendorsManager) HasVendors() bool {
+	return len(o.Vendors) > 0
+}
+
+func (o *VendorsManager) FindByName(name string) Vendor {
+	return o.VendorsByName[name]
+}
+
+func (o *VendorsManager) readModels() (err error) {
+	if len(o.Vendors) == 0 {
+
+		err = fmt.Errorf("no AI vendors configured to read models from. Please configure at least one AI vendor")
+		return
+	}
+
+	o.Models = NewVendorsModels()
+
+	var wg sync.WaitGroup
+	resultsChan := make(chan modelResult, len(o.Vendors))
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	for _, vendor := range o.Vendors {
+		wg.Add(1)
+		go o.fetchVendorModels(ctx, &wg, vendor, resultsChan)
+	}
+
+	// Wait for all goroutines to finish
+	go func() {
+		wg.Wait()
+		close(resultsChan)
+	}()
+
+	// Collect results
+	for result := range resultsChan {
+		if result.err != nil {
+			fmt.Println(result.vendorName, result.err)
+			cancel() // Cancel remaining goroutines if needed
+		} else {
+			o.Models.AddGroupItems(result.vendorName, result.models...)
+		}
+	}
+	return
+}
+
+func (o *VendorsManager) fetchVendorModels(
+	ctx context.Context, wg *sync.WaitGroup, vendor Vendor, resultsChan chan<- modelResult) {
+
+	defer wg.Done()
+
+	models, err := vendor.ListModels()
+	select {
+	case <-ctx.Done():
+		// Context canceled, don't send the result
+		return
+	case resultsChan <- modelResult{vendorName: vendor.GetName(), models: models, err: err}:
+		// Result sent
+	}
+}
+
+func (o *VendorsManager) Setup() (ret map[string]Vendor, err error) {
+	ret = map[string]Vendor{}
+	for _, vendor := range o.Vendors {
+		fmt.Println()
+		o.setupVendorTo(vendor, ret)
+	}
+	return
+}
+
+func (o *VendorsManager) SetupVendor(vendorName string, configuredVendors map[string]Vendor) (err error) {
+	vendor := o.FindByName(vendorName)
+	if vendor == nil {
+		err = fmt.Errorf("vendor %s not found", vendorName)
+		return
+	}
+	o.setupVendorTo(vendor, configuredVendors)
+	return
+}
+
+func (o *VendorsManager) setupVendorTo(vendor Vendor, configuredVendors map[string]Vendor) {
+	if vendorErr := vendor.Setup(); vendorErr == nil {
+		fmt.Printf("[%v] configured\n", vendor.GetName())
+		configuredVendors[vendor.GetName()] = vendor
+	} else {
+		delete(configuredVendors, vendor.GetName())
+		fmt.Printf("[%v] skipped\n", vendor.GetName())
+	}
+	return
+}
+
+type modelResult struct {
+	vendorName string
+	models     []string
+	err        error
+}

plugins/db/api.go

@@ -0,0 +1,13 @@
+package db
+
+type Storage[T any] interface {
+	Configure() (err error)
+	Get(name string) (ret *T, err error)
+	GetNames() (ret []string, err error)
+	Delete(name string) (err error)
+	Exists(name string) (ret bool)
+	Rename(oldName, newName string) (err error)
+	Save(name string, content []byte) (err error)
+	Load(name string) (ret []byte, err error)
+	ListNames() (err error)
+}

plugins/db/fsdb/contexts.go

@@ -1,13 +1,13 @@
-package db
+package fsdb
 
 import "fmt"
 
-type Contexts struct {
-	*Storage
+type ContextsEntity struct {
+	*StorageEntity
 }
 
-// GetContext Load a context from file
-func (o *Contexts) GetContext(name string) (ret *Context, err error) {
+// Get Load a context from file
+func (o *ContextsEntity) Get(name string) (ret *Context, err error) {
 	var content []byte
 	if content, err = o.Load(name); err != nil {
 		return
@@ -17,9 +17,9 @@ func (o *Contexts) GetContext(name string) (ret *Context, err error) {
 	return
 }
 
-func (o *Contexts) PrintContext(name string) (err error) {
+func (o *ContextsEntity) PrintContext(name string) (err error) {
 	var context *Context
-	if context, err = o.GetContext(name); err != nil {
+	if context, err = o.Get(name); err != nil {
 		return
 	}
 	fmt.Println(context.Content)

plugins/db/fsdb/contexts_test.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"os"
@@ -8,8 +8,8 @@ import (
 
 func TestContexts_GetContext(t *testing.T) {
 	dir := t.TempDir()
-	contexts := &Contexts{
-		Storage: &Storage{Dir: dir},
+	contexts := &ContextsEntity{
+		StorageEntity: &StorageEntity{Dir: dir},
 	}
 	contextName := "testContext"
 	contextPath := filepath.Join(dir, contextName)
@@ -18,7 +18,7 @@ func TestContexts_GetContext(t *testing.T) {
 	if err != nil {
 		t.Fatalf("failed to write context file: %v", err)
 	}
-	context, err := contexts.GetContext(contextName)
+	context, err := contexts.Get(contextName)
 	if err != nil {
 		t.Fatalf("failed to get context: %v", err)
 	}

plugins/db/fsdb/db.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"fmt"
@@ -14,17 +14,17 @@ func NewDb(dir string) (db *Db) {
 
 	db.EnvFilePath = db.FilePath(".env")
 
-	db.Patterns = &Patterns{
-		Storage:                &Storage{Label: "Patterns", Dir: db.FilePath("patterns"), ItemIsDir: true},
+	db.Patterns = &PatternsEntity{
+		StorageEntity:          &StorageEntity{Label: "Patterns", Dir: db.FilePath("patterns"), ItemIsDir: true},
 		SystemPatternFile:      "system.md",
 		UniquePatternsFilePath: db.FilePath("unique_patterns.txt"),
 	}
 
-	db.Sessions = &Sessions{
-		&Storage{Label: "Sessions", Dir: db.FilePath("sessions"), FileExtension: ".json"}}
+	db.Sessions = &SessionsEntity{
+		&StorageEntity{Label: "Sessions", Dir: db.FilePath("sessions"), FileExtension: ".json"}}
 
-	db.Contexts = &Contexts{
-		&Storage{Label: "Contexts", Dir: db.FilePath("contexts")}}
+	db.Contexts = &ContextsEntity{
+		&StorageEntity{Label: "Contexts", Dir: db.FilePath("contexts")}}
 
 	return
 }
@@ -32,9 +32,9 @@ func NewDb(dir string) (db *Db) {
 type Db struct {
 	Dir string
 
-	Patterns *Patterns
-	Sessions *Sessions
-	Contexts *Contexts
+	Patterns *PatternsEntity
+	Sessions *SessionsEntity
+	Contexts *ContextsEntity
 
 	EnvFilePath string
 }

plugins/db/fsdb/db_test.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"os"

plugins/db/fsdb/patterns.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"fmt"
@@ -7,14 +7,13 @@ import (
 	"strings"
 )
 
-type Patterns struct {
-	*Storage
+type PatternsEntity struct {
+	*StorageEntity
 	SystemPatternFile      string
 	UniquePatternsFilePath string
 }
 
-// GetPattern finds a pattern by name and returns the pattern as an entry or an error
-func (o *Patterns) GetPattern(name string, variables map[string]string) (ret *Pattern, err error) {
+func (o *PatternsEntity) Get(name string) (ret *Pattern, err error) {
 	patternPath := filepath.Join(o.Dir, name, o.SystemPatternFile)
 
 	var pattern []byte
@@ -23,13 +22,6 @@ func (o *Patterns) GetPattern(name string, variables map[string]string) (ret *Pa
 	}
 
 	patternStr := string(pattern)
-
-	if variables != nil && len(variables) > 0 {
-		for variableName, value := range variables {
-			patternStr = strings.ReplaceAll(patternStr, variableName, value)
-		}
-	}
-
 	ret = &Pattern{
 		Name:    name,
 		Pattern: patternStr,
@@ -37,7 +29,22 @@ func (o *Patterns) GetPattern(name string, variables map[string]string) (ret *Pa
 	return
 }
 
-func (o *Patterns) PrintLatestPatterns(latestNumber int) (err error) {
+// GetApplyVariables finds a pattern by name and returns the pattern as an entry or an error
+func (o *PatternsEntity) GetApplyVariables(name string, variables map[string]string) (ret *Pattern, err error) {
+
+	if ret, err = o.Get(name); err != nil {
+		return
+	}
+
+	if variables != nil && len(variables) > 0 {
+		for variableName, value := range variables {
+			ret.Pattern = strings.ReplaceAll(ret.Pattern, variableName, value)
+		}
+	}
+	return
+}
+
+func (o *PatternsEntity) PrintLatestPatterns(latestNumber int) (err error) {
 	var contents []byte
 	if contents, err = os.ReadFile(o.UniquePatternsFilePath); err != nil {
 		err = fmt.Errorf("could not read unique patterns file. Pleas run --updatepatterns (%s)", err)

plugins/db/fsdb/patterns_test.go

@@ -0,0 +1 @@
+package fsdb

plugins/db/fsdb/sessions.go

@@ -0,0 +1,88 @@
+package fsdb
+
+import (
+	"fmt"
+	"github.com/danielmiessler/fabric/common"
+)
+
+type SessionsEntity struct {
+	*StorageEntity
+}
+
+func (o *SessionsEntity) Get(name string) (session *Session, err error) {
+	session = &Session{Name: name}
+
+	if o.Exists(name) {
+		err = o.LoadAsJson(name, &session.Messages)
+	} else {
+		fmt.Printf("Creating new session: %s\n", name)
+	}
+	return
+}
+
+func (o *SessionsEntity) PrintSession(name string) (err error) {
+	if o.Exists(name) {
+		var session Session
+		if err = o.LoadAsJson(name, &session.Messages); err == nil {
+			fmt.Println(session.String())
+		}
+	}
+	return
+}
+
+func (o *SessionsEntity) SaveSession(session *Session) (err error) {
+	return o.SaveAsJson(session.Name, session.Messages)
+}
+
+type Session struct {
+	Name     string
+	Messages []*common.Message
+
+	vendorMessages []*common.Message
+}
+
+func (o *Session) IsEmpty() bool {
+	return len(o.Messages) == 0
+}
+
+func (o *Session) Append(messages ...*common.Message) {
+	if o.vendorMessages != nil {
+		for _, message := range messages {
+			o.Messages = append(o.Messages, message)
+			o.appendVendorMessage(message)
+		}
+	} else {
+		o.Messages = append(o.Messages, messages...)
+	}
+}
+
+func (o *Session) GetVendorMessages() (ret []*common.Message) {
+	if o.vendorMessages == nil {
+		o.vendorMessages = []*common.Message{}
+		for _, message := range o.Messages {
+			o.appendVendorMessage(message)
+		}
+	}
+	ret = o.vendorMessages
+	return
+}
+
+func (o *Session) appendVendorMessage(message *common.Message) {
+	if message.Role != common.ChatMessageRoleMeta {
+		o.vendorMessages = append(o.vendorMessages, message)
+	}
+}
+
+func (o *Session) GetLastMessage() (ret *common.Message) {
+	if len(o.Messages) > 0 {
+		ret = o.Messages[len(o.Messages)-1]
+	}
+	return
+}
+
+func (o *Session) String() (ret string) {
+	for _, message := range o.Messages {
+		ret += fmt.Sprintf("\n--- \n[%v]\n\n%v", message.Role, message.Content)
+	}
+	return
+}

plugins/db/fsdb/sessions_test.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"testing"
@@ -8,11 +8,11 @@ import (
 
 func TestSessions_GetOrCreateSession(t *testing.T) {
 	dir := t.TempDir()
-	sessions := &Sessions{
-		Storage: &Storage{Dir: dir, FileExtension: ".json"},
+	sessions := &SessionsEntity{
+		StorageEntity: &StorageEntity{Dir: dir, FileExtension: ".json"},
 	}
 	sessionName := "testSession"
-	session, err := sessions.GetOrCreateSession(sessionName)
+	session, err := sessions.Get(sessionName)
 	if err != nil {
 		t.Fatalf("failed to get or create session: %v", err)
 	}
@@ -23,8 +23,8 @@ func TestSessions_GetOrCreateSession(t *testing.T) {
 
 func TestSessions_SaveSession(t *testing.T) {
 	dir := t.TempDir()
-	sessions := &Sessions{
-		Storage: &Storage{Dir: dir, FileExtension: ".json"},
+	sessions := &SessionsEntity{
+		StorageEntity: &StorageEntity{Dir: dir, FileExtension: ".json"},
 	}
 	sessionName := "testSession"
 	session := &Session{Name: sessionName, Messages: []*common.Message{{Content: "message1"}}}

plugins/db/fsdb/storage.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"encoding/json"
@@ -10,14 +10,14 @@ import (
 	"github.com/samber/lo"
 )
 
-type Storage struct {
+type StorageEntity struct {
 	Label         string
 	Dir           string
 	ItemIsDir     bool
 	FileExtension string
 }
 
-func (o *Storage) Configure() (err error) {
+func (o *StorageEntity) Configure() (err error) {
 	if err = os.MkdirAll(o.Dir, os.ModePerm); err != nil {
 		return
 	}
@@ -25,7 +25,7 @@ func (o *Storage) Configure() (err error) {
 }
 
 // GetNames finds all patterns in the patterns directory and enters the id, name, and pattern into a slice of Entry structs. it returns these entries or an error
-func (o *Storage) GetNames() (ret []string, err error) {
+func (o *StorageEntity) GetNames() (ret []string, err error) {
 	var entries []os.DirEntry
 	if entries, err = os.ReadDir(o.Dir); err != nil {
 		err = fmt.Errorf("could not read items from directory: %v", err)
@@ -59,7 +59,41 @@ func (o *Storage) GetNames() (ret []string, err error) {
 	return
 }
 
-func (o *Storage) ListNames() (err error) {
+func (o *StorageEntity) Delete(name string) (err error) {
+	if err = os.Remove(o.BuildFilePathByName(name)); err != nil {
+		err = fmt.Errorf("could not delete %s: %v", name, err)
+	}
+	return
+}
+
+func (o *StorageEntity) Exists(name string) (ret bool) {
+	_, err := os.Stat(o.BuildFilePathByName(name))
+	ret = !os.IsNotExist(err)
+	return
+}
+
+func (o *StorageEntity) Rename(oldName, newName string) (err error) {
+	if err = os.Rename(o.BuildFilePathByName(oldName), o.BuildFilePathByName(newName)); err != nil {
+		err = fmt.Errorf("could not rename %s to %s: %v", oldName, newName, err)
+	}
+	return
+}
+
+func (o *StorageEntity) Save(name string, content []byte) (err error) {
+	if err = os.WriteFile(o.BuildFilePathByName(name), content, 0644); err != nil {
+		err = fmt.Errorf("could not save %s: %v", name, err)
+	}
+	return
+}
+
+func (o *StorageEntity) Load(name string) (ret []byte, err error) {
+	if ret, err = os.ReadFile(o.BuildFilePathByName(name)); err != nil {
+		err = fmt.Errorf("could not load %s: %v", name, err)
+	}
+	return
+}
+
+func (o *StorageEntity) ListNames() (err error) {
 	var names []string
 	if names, err = o.GetNames(); err != nil {
 		return
@@ -76,55 +110,21 @@ func (o *Storage) ListNames() (err error) {
 	return
 }
 
-func (o *Storage) BuildFilePathByName(name string) (ret string) {
+func (o *StorageEntity) BuildFilePathByName(name string) (ret string) {
 	ret = o.BuildFilePath(o.buildFileName(name))
 	return
 }
 
-func (o *Storage) BuildFilePath(fileName string) (ret string) {
+func (o *StorageEntity) BuildFilePath(fileName string) (ret string) {
 	ret = filepath.Join(o.Dir, fileName)
 	return
 }
 
-func (o *Storage) buildFileName(name string) string {
+func (o *StorageEntity) buildFileName(name string) string {
 	return fmt.Sprintf("%s%v", name, o.FileExtension)
 }
 
-func (o *Storage) Delete(name string) (err error) {
-	if err = os.Remove(o.BuildFilePathByName(name)); err != nil {
-		err = fmt.Errorf("could not delete %s: %v", name, err)
-	}
-	return
-}
-
-func (o *Storage) Exists(name string) (ret bool) {
-	_, err := os.Stat(o.BuildFilePathByName(name))
-	ret = !os.IsNotExist(err)
-	return
-}
-
-func (o *Storage) Rename(oldName, newName string) (err error) {
-	if err = os.Rename(o.BuildFilePathByName(oldName), o.BuildFilePathByName(newName)); err != nil {
-		err = fmt.Errorf("could not rename %s to %s: %v", oldName, newName, err)
-	}
-	return
-}
-
-func (o *Storage) Save(name string, content []byte) (err error) {
-	if err = os.WriteFile(o.BuildFilePathByName(name), content, 0644); err != nil {
-		err = fmt.Errorf("could not save %s: %v", name, err)
-	}
-	return
-}
-
-func (o *Storage) Load(name string) (ret []byte, err error) {
-	if ret, err = os.ReadFile(o.BuildFilePathByName(name)); err != nil {
-		err = fmt.Errorf("could not load %s: %v", name, err)
-	}
-	return
-}
-
-func (o *Storage) SaveAsJson(name string, item interface{}) (err error) {
+func (o *StorageEntity) SaveAsJson(name string, item interface{}) (err error) {
 	var jsonString []byte
 	if jsonString, err = json.Marshal(item); err == nil {
 		err = o.Save(name, jsonString)
@@ -135,7 +135,7 @@ func (o *Storage) SaveAsJson(name string, item interface{}) (err error) {
 	return err
 }
 
-func (o *Storage) LoadAsJson(name string, item interface{}) (err error) {
+func (o *StorageEntity) LoadAsJson(name string, item interface{}) (err error) {
 	var content []byte
 	if content, err = o.Load(name); err != nil {
 		return

plugins/db/fsdb/storage_test.go

@@ -1,4 +1,4 @@
-package db
+package fsdb
 
 import (
 	"testing"
@@ -6,7 +6,7 @@ import (
 
 func TestStorage_SaveAndLoad(t *testing.T) {
 	dir := t.TempDir()
-	storage := &Storage{Dir: dir}
+	storage := &StorageEntity{Dir: dir}
 	name := "test"
 	content := []byte("test content")
 	if err := storage.Save(name, content); err != nil {
@@ -23,7 +23,7 @@ func TestStorage_SaveAndLoad(t *testing.T) {
 
 func TestStorage_Exists(t *testing.T) {
 	dir := t.TempDir()
-	storage := &Storage{Dir: dir}
+	storage := &StorageEntity{Dir: dir}
 	name := "test"
 	if storage.Exists(name) {
 		t.Errorf("expected file to not exist")
@@ -38,7 +38,7 @@ func TestStorage_Exists(t *testing.T) {
 
 func TestStorage_Delete(t *testing.T) {
 	dir := t.TempDir()
-	storage := &Storage{Dir: dir}
+	storage := &StorageEntity{Dir: dir}
 	name := "test"
 	if err := storage.Save(name, []byte("test content")); err != nil {
 		t.Fatalf("failed to save content: %v", err)

plugins/plugin.go

@@ -1,4 +1,4 @@
-package common
+package plugins
 
 import (
 	"bytes"
@@ -9,41 +9,58 @@ import (
 
 const AnswerReset = "reset"
 
-type Configurable struct {
+type Plugin interface {
+	GetName() string
+	GetSetupDescription() string
+	IsConfigured() bool
+	Configure() error
+	Setup() error
+	SetupFillEnvFileContent(*bytes.Buffer)
+}
+
+type PluginBase struct {
 	Settings
 	SetupQuestions
 
-	Label         string
-	EnvNamePrefix string
+	Name             string
+	SetupDescription string
+	EnvNamePrefix    string
 
 	ConfigureCustom func() error
 }
 
-func (o *Configurable) GetName() string {
-	return o.Label
+func (o *PluginBase) GetName() string {
+	return o.Name
 }
 
-func (o *Configurable) AddSetting(name string, required bool) (ret *Setting) {
+func (o *PluginBase) GetSetupDescription() (ret string) {
+	if ret = o.SetupDescription; ret == "" {
+		ret = o.GetName()
+	}
+	return
+}
+
+func (o *PluginBase) AddSetting(name string, required bool) (ret *Setting) {
 	ret = NewSetting(fmt.Sprintf("%v%v", o.EnvNamePrefix, BuildEnvVariable(name)), required)
 	o.Settings = append(o.Settings, ret)
 	return
 }
 
-func (o *Configurable) AddSetupQuestion(name string, required bool) (ret *SetupQuestion) {
+func (o *PluginBase) AddSetupQuestion(name string, required bool) (ret *SetupQuestion) {
 	return o.AddSetupQuestionCustom(name, required, "")
 }
 
-func (o *Configurable) AddSetupQuestionCustom(name string, required bool, question string) (ret *SetupQuestion) {
+func (o *PluginBase) AddSetupQuestionCustom(name string, required bool, question string) (ret *SetupQuestion) {
 	setting := o.AddSetting(name, required)
 	ret = &SetupQuestion{Setting: setting, Question: question}
 	if ret.Question == "" {
-		ret.Question = fmt.Sprintf("Enter your %v %v", o.Label, strings.ToUpper(name))
+		ret.Question = fmt.Sprintf("Enter your %v %v", o.Name, strings.ToUpper(name))
 	}
 	o.SetupQuestions = append(o.SetupQuestions, ret)
 	return
 }
 
-func (o *Configurable) Configure() (err error) {
+func (o *PluginBase) Configure() (err error) {
 	if err = o.Settings.Configure(); err != nil {
 		return
 	}
@@ -54,8 +71,8 @@ func (o *Configurable) Configure() (err error) {
 	return
 }
 
-func (o *Configurable) Setup() (err error) {
-	if err = o.Ask(o.Label); err != nil {
+func (o *PluginBase) Setup() (err error) {
+	if err = o.Ask(o.Name); err != nil {
 		return
 	}
 
@@ -63,14 +80,14 @@ func (o *Configurable) Setup() (err error) {
 	return
 }
 
-func (o *Configurable) SetupOrSkip() (err error) {
+func (o *PluginBase) SetupOrSkip() (err error) {
 	if err = o.Setup(); err != nil {
 		fmt.Printf("[%v] skipped\n", o.GetName())
 	}
 	return
 }
 
-func (o *Configurable) SetupFillEnvFileContent(fileEnvFileContent *bytes.Buffer) {
+func (o *PluginBase) SetupFillEnvFileContent(fileEnvFileContent *bytes.Buffer) {
 	o.Settings.FillEnvFileContent(fileEnvFileContent)
 }
 
@@ -126,6 +143,10 @@ func (o *Setting) Print() {
 	fmt.Printf("%v: %v\n", o.EnvVariable, o.Value)
 }
 
+func NewSetupQuestion(question string) *SetupQuestion {
+	return &SetupQuestion{Setting: &Setting{}, Question: question}
+}
+
 type SetupQuestion struct {
 	*Setting
 	Question string
@@ -162,6 +183,11 @@ func (o *SetupQuestion) Ask(label string) (err error) {
 
 func (o *SetupQuestion) OnAnswer(answer string) (err error) {
 	o.Value = answer
+	if o.EnvVariable != "" {
+		if err = os.Setenv(o.EnvVariable, answer); err != nil {
+			return
+		}
+	}
 	err = o.IsValidErr()
 	return
 }

plugins/plugin_test.go

@@ -1,4 +1,4 @@
-package common
+package plugins
 
 import (
 	"bytes"
@@ -9,9 +9,9 @@ import (
 )
 
 func TestConfigurable_AddSetting(t *testing.T) {
-	conf := &Configurable{
+	conf := &PluginBase{
 		Settings:      Settings{},
-		Label:         "TestConfigurable",
+		Name:          "TestConfigurable",
 		EnvNamePrefix: "TEST_",
 	}
 
@@ -26,9 +26,9 @@ func TestConfigurable_Configure(t *testing.T) {
 		EnvVariable: "TEST_SETTING",
 		Required:    true,
 	}
-	conf := &Configurable{
+	conf := &PluginBase{
 		Settings: Settings{setting},
-		Label:    "TestConfigurable",
+		Name:     "TestConfigurable",
 	}
 
 	_ = os.Setenv("TEST_SETTING", "test_value")
@@ -42,9 +42,9 @@ func TestConfigurable_Setup(t *testing.T) {
 		EnvVariable: "TEST_SETTING",
 		Required:    false,
 	}
-	conf := &Configurable{
+	conf := &PluginBase{
 		Settings: Settings{setting},
-		Label:    "TestConfigurable",
+		Name:     "TestConfigurable",
 	}
 
 	err := conf.Setup()

plugins/tools/defaults.go

@@ -0,0 +1,71 @@
+package tools
+
+import (
+	"fmt"
+	"strconv"
+
+	"github.com/danielmiessler/fabric/plugins"
+	"github.com/danielmiessler/fabric/plugins/ai"
+	"github.com/pkg/errors"
+)
+
+func NeeDefaults(getVendorsModels func() (*ai.VendorsModels, error)) (ret *Defaults) {
+	vendorName := "Default"
+	ret = &Defaults{
+		PluginBase: &plugins.PluginBase{
+			Name:             vendorName,
+			SetupDescription: "Default AI Vendor and Model [required]",
+			EnvNamePrefix:    plugins.BuildEnvVariablePrefix(vendorName),
+		},
+		GetVendorsModels: getVendorsModels,
+	}
+
+	ret.Vendor = ret.AddSetting("Vendor", true)
+	ret.Model = ret.AddSetupQuestionCustom("Model", true,
+		"Enter the index the name of your default model")
+
+	return
+}
+
+type Defaults struct {
+	*plugins.PluginBase
+
+	Vendor           *plugins.Setting
+	Model            *plugins.SetupQuestion
+	GetVendorsModels func() (*ai.VendorsModels, error)
+}
+
+func (o *Defaults) Setup() (err error) {
+	var vendorsModels *ai.VendorsModels
+	if vendorsModels, err = o.GetVendorsModels(); err != nil {
+		return
+	}
+
+	vendorsModels.Print()
+
+	if err = o.Ask(o.Name); err != nil {
+		return
+	}
+
+	index, parseErr := strconv.Atoi(o.Model.Value)
+	if parseErr == nil {
+		if o.Vendor.Value, o.Model.Value, err = vendorsModels.GetGroupAndItemByItemNumber(index); err != nil {
+			return
+		}
+	} else {
+		o.Vendor.Value = vendorsModels.FindGroupsByItemFirst(o.Model.Value)
+	}
+
+	//verify
+	vendorNames := vendorsModels.FindGroupsByItem(o.Model.Value)
+	if len(vendorNames) == 0 {
+		err = errors.Errorf("You need to chose an available default model.")
+		return
+	}
+
+	fmt.Println()
+	o.Vendor.Print()
+	o.Model.Print()
+
+	return
+}

plugins/tools/jina/jina.go

@@ -7,12 +7,12 @@ import (
 	"io"
 	"net/http"
 
-	"github.com/danielmiessler/fabric/common"
+	"github.com/danielmiessler/fabric/plugins"
 )
 
 type Client struct {
-	*common.Configurable
-	ApiKey *common.SetupQuestion
+	*plugins.PluginBase
+	ApiKey *plugins.SetupQuestion
 }
 
 func NewClient() (ret *Client) {
@@ -20,9 +20,10 @@ func NewClient() (ret *Client) {
 	label := "Jina AI"
 
 	ret = &Client{
-		Configurable: &common.Configurable{
-			Label:         label,
-			EnvNamePrefix: common.BuildEnvVariablePrefix(label),
+		PluginBase: &plugins.PluginBase{
+			Name:             label,
+			SetupDescription: "Jina AI Service - to grab a webpage as clean, LLM-friendly text",
+			EnvNamePrefix:    plugins.BuildEnvVariablePrefix(label),
 		},
 	}
 

plugins/tools/lang/language.go

@@ -0,0 +1,42 @@
+package lang
+
+import (
+	"github.com/danielmiessler/fabric/plugins"
+	"golang.org/x/text/language"
+)
+
+func NewLanguage() (ret *Language) {
+
+	label := "Language"
+	ret = &Language{}
+
+	ret.PluginBase = &plugins.PluginBase{
+		Name:             label,
+		SetupDescription: "Language - Default AI Vendor Output Language",
+		EnvNamePrefix:    plugins.BuildEnvVariablePrefix(label),
+		ConfigureCustom:  ret.configure,
+	}
+
+	ret.DefaultLanguage = ret.AddSetupQuestionCustom("Output", false,
+		"Enter your default output language (for example: zh_CN)")
+
+	return
+}
+
+type Language struct {
+	*plugins.PluginBase
+	DefaultLanguage *plugins.SetupQuestion
+}
+
+func (o *Language) configure() error {
+	if o.DefaultLanguage.Value != "" {
+		langTag, err := language.Parse(o.DefaultLanguage.Value)
+		if err == nil {
+			o.DefaultLanguage.Value = langTag.String()
+		} else {
+			o.DefaultLanguage.Value = ""
+		}
+	}
+
+	return nil
+}

plugins/tools/patterns_loader.go

@@ -1,4 +1,4 @@
-package core
+package tools
 
 import (
 	"fmt"
@@ -8,8 +8,9 @@ import (
 	"sort"
 	"strings"
 
-	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/db"
+	"github.com/danielmiessler/fabric/plugins"
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+
 	"github.com/go-git/go-git/v5"
 	"github.com/go-git/go-git/v5/plumbing"
 	"github.com/go-git/go-git/v5/plumbing/object"
@@ -17,16 +18,21 @@ import (
 	"github.com/otiai10/copy"
 )
 
-func NewPatternsLoader(patterns *db.Patterns) (ret *PatternsLoader) {
+const DefaultPatternsGitRepoUrl = "https://github.com/danielmiessler/fabric.git"
+const DefaultPatternsGitRepoFolder = "patterns"
+
+func NewPatternsLoader(patterns *fsdb.PatternsEntity) (ret *PatternsLoader) {
 	label := "Patterns Loader"
 	ret = &PatternsLoader{
-		Patterns: patterns,
+		Patterns:       patterns,
+		loadedFilePath: patterns.BuildFilePath("loaded"),
 	}
 
-	ret.Configurable = &common.Configurable{
-		Label:           label,
-		EnvNamePrefix:   common.BuildEnvVariablePrefix(label),
-		ConfigureCustom: ret.configure,
+	ret.PluginBase = &plugins.PluginBase{
+		Name:             label,
+		SetupDescription: "Patterns - Downloads patterns [required]",
+		EnvNamePrefix:    plugins.BuildEnvVariablePrefix(label),
+		ConfigureCustom:  ret.configure,
 	}
 
 	ret.DefaultGitRepoUrl = ret.AddSetupQuestionCustom("Git Repo Url", true,
@@ -41,11 +47,13 @@ func NewPatternsLoader(patterns *db.Patterns) (ret *PatternsLoader) {
 }
 
 type PatternsLoader struct {
-	*common.Configurable
-	Patterns *db.Patterns
+	*plugins.PluginBase
+	Patterns *fsdb.PatternsEntity
 
-	DefaultGitRepoUrl *common.SetupQuestion
-	DefaultFolder     *common.SetupQuestion
+	DefaultGitRepoUrl *plugins.SetupQuestion
+	DefaultFolder     *plugins.SetupQuestion
+
+	loadedFilePath string
 
 	pathPatternsPrefix string
 	tempPatternsFolder string
@@ -58,6 +66,27 @@ func (o *PatternsLoader) configure() (err error) {
 	return
 }
 
+func (o *PatternsLoader) IsConfigured() (ret bool) {
+	ret = o.PluginBase.IsConfigured()
+	if ret {
+		if _, err := os.Stat(o.loadedFilePath); os.IsNotExist(err) {
+			ret = false
+		}
+	}
+	return
+}
+
+func (o *PatternsLoader) Setup() (err error) {
+	if err = o.PluginBase.Setup(); err != nil {
+		return
+	}
+
+	if err = o.PopulateDB(); err != nil {
+		return
+	}
+	return
+}
+
 // PopulateDB downloads patterns from the internet and populates the patterns folder
 func (o *PatternsLoader) PopulateDB() (err error) {
 	fmt.Printf("Downloading patterns and Populating %s..\n", o.Patterns.Dir)
@@ -90,7 +119,7 @@ func (o *PatternsLoader) PersistPatterns() (err error) {
 			if currentPattern.Name() == newPattern.Name() {
 				break
 			}
-			copy.Copy(filepath.Join(o.Patterns.Dir, newPattern.Name()), filepath.Join(newPatternsFolder, newPattern.Name()))
+			err = copy.Copy(filepath.Join(o.Patterns.Dir, newPattern.Name()), filepath.Join(newPatternsFolder, newPattern.Name()))
 		}
 	}
 	return
@@ -98,30 +127,26 @@ func (o *PatternsLoader) PersistPatterns() (err error) {
 
 // movePatterns copies the new patterns into the config directory
 func (o *PatternsLoader) movePatterns() (err error) {
-	os.MkdirAll(o.Patterns.Dir, os.ModePerm)
+	if err = os.MkdirAll(o.Patterns.Dir, os.ModePerm); err != nil {
+		return
+	}
 
 	patternsDir := o.tempPatternsFolder
 	if err = o.PersistPatterns(); err != nil {
 		return
 	}
 
-	copy.Copy(patternsDir, o.Patterns.Dir) // copies the patterns to the config directory
+	if err = copy.Copy(patternsDir, o.Patterns.Dir); err != nil { // copies the patterns to the config directory
+		return
+	}
+
+	//create an empty file to indicate that the patterns have been updated if not exists
+	_, _ = os.Create(o.loadedFilePath)
+
 	err = os.RemoveAll(patternsDir)
 	return
 }
 
-// checks if a pattern already exists in the directory
-// func DoesPatternExistAlready(name string) (bool, error) {
-// 	entry := db.Entry{
-// 		Label: name,
-// 	}
-// 	_, err := entry.GetByName()
-// 	if err != nil {
-// 		return false, err
-// 	}
-// 	return true, nil
-// }
-
 func (o *PatternsLoader) gitCloneAndCopy() (err error) {
 	// Clones the given repository, creating the remote, the local branches
 	// and fetching the objects, everything in memory:
@@ -152,10 +177,10 @@ func (o *PatternsLoader) gitCloneAndCopy() (err error) {
 		return err
 	}
 
-	var changes []db.DirectoryChange
+	var changes []fsdb.DirectoryChange
 	// ... iterates over the commits
 	if err = cIter.ForEach(func(c *object.Commit) (err error) {
-		// Get the files changed in this commit by comparing with its parents
+		// GetApplyVariables the files changed in this commit by comparing with its parents
 		parentIter := c.Parents()
 		if err = parentIter.ForEach(func(parent *object.Commit) (err error) {
 			var patch *object.Patch
@@ -167,7 +192,7 @@ func (o *PatternsLoader) gitCloneAndCopy() (err error) {
 			for _, fileStat := range patch.Stats() {
 				if strings.HasPrefix(fileStat.Name, o.pathPatternsPrefix) {
 					dir := filepath.Dir(fileStat.Name)
-					changes = append(changes, db.DirectoryChange{Dir: dir, Timestamp: c.Committer.When})
+					changes = append(changes, fsdb.DirectoryChange{Dir: dir, Timestamp: c.Committer.When})
 				}
 			}
 			return
@@ -186,7 +211,9 @@ func (o *PatternsLoader) gitCloneAndCopy() (err error) {
 		return changes[i].Timestamp.Before(changes[j].Timestamp)
 	})
 
-	o.makeUniqueList(changes)
+	if err = o.makeUniqueList(changes); err != nil {
+		return
+	}
 
 	var commit *object.Commit
 	if commit, err = r.CommitObject(ref.Hash()); err != nil {
@@ -250,7 +277,7 @@ func (o *PatternsLoader) writeBlobToFile(blob *object.Blob, path string) (err er
 	return
 }
 
-func (o *PatternsLoader) makeUniqueList(changes []db.DirectoryChange) {
+func (o *PatternsLoader) makeUniqueList(changes []fsdb.DirectoryChange) (err error) {
 	uniqueItems := make(map[string]bool)
 	for _, change := range changes {
 		if strings.TrimSpace(change.Dir) != "" && !strings.Contains(change.Dir, "=>") {
@@ -271,5 +298,6 @@ func (o *PatternsLoader) makeUniqueList(changes []db.DirectoryChange) {
 	}
 
 	joined := strings.Join(finalList, "\n")
-	os.WriteFile(o.Patterns.UniquePatternsFilePath, []byte(joined), 0o644)
+	err = os.WriteFile(o.Patterns.UniquePatternsFilePath, []byte(joined), 0o644)
+	return
 }

plugins/tools/youtube/youtube.go

@@ -5,15 +5,16 @@ import (
 	"encoding/json"
 	"flag"
 	"fmt"
-	"github.com/anaskhan96/soup"
-	"github.com/danielmiessler/fabric/common"
-	"google.golang.org/api/option"
-	"google.golang.org/api/youtube/v3"
 	"log"
 	"net/url"
 	"regexp"
 	"strconv"
 	"strings"
+
+	"github.com/anaskhan96/soup"
+	"github.com/danielmiessler/fabric/plugins"
+	"google.golang.org/api/option"
+	"google.golang.org/api/youtube/v3"
 )
 
 func NewYouTube() (ret *YouTube) {
@@ -21,9 +22,10 @@ func NewYouTube() (ret *YouTube) {
 	label := "YouTube"
 	ret = &YouTube{}
 
-	ret.Configurable = &common.Configurable{
-		Label:         label,
-		EnvNamePrefix: common.BuildEnvVariablePrefix(label),
+	ret.PluginBase = &plugins.PluginBase{
+		Name:             label,
+		SetupDescription: label + " - to grab video transcripts and comments",
+		EnvNamePrefix:    plugins.BuildEnvVariablePrefix(label),
 	}
 
 	ret.ApiKey = ret.AddSetupQuestion("API key", true)
@@ -32,8 +34,8 @@ func NewYouTube() (ret *YouTube) {
 }
 
 type YouTube struct {
-	*common.Configurable
-	ApiKey *common.SetupQuestion
+	*plugins.PluginBase
+	ApiKey *plugins.SetupQuestion
 
 	service *youtube.Service
 }

restapi/contexts.go

@@ -0,0 +1,19 @@
+package restapi
+
+import (
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"github.com/gin-gonic/gin"
+)
+
+// ContextsHandler defines the handler for contexts-related operations
+type ContextsHandler struct {
+	*StorageHandler[fsdb.Context]
+	contexts *fsdb.ContextsEntity
+}
+
+// NewContextsHandler creates a new ContextsHandler
+func NewContextsHandler(r *gin.Engine, contexts *fsdb.ContextsEntity) (ret *ContextsHandler) {
+	ret = &ContextsHandler{
+		StorageHandler: NewStorageHandler[fsdb.Context](r, "contexts", contexts), contexts: contexts}
+	return
+}

restapi/patterns.go

@@ -0,0 +1,35 @@
+package restapi
+
+import (
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"github.com/gin-gonic/gin"
+	"net/http"
+)
+
+// PatternsHandler defines the handler for patterns-related operations
+type PatternsHandler struct {
+	*StorageHandler[fsdb.Pattern]
+	patterns *fsdb.PatternsEntity
+}
+
+// NewPatternsHandler creates a new PatternsHandler
+func NewPatternsHandler(r *gin.Engine, patterns *fsdb.PatternsEntity) (ret *PatternsHandler) {
+	ret = &PatternsHandler{
+		StorageHandler: NewStorageHandler[fsdb.Pattern](r, "patterns", patterns), patterns: patterns}
+
+	// TODO: Add custom, replacement routes here
+	//r.GET("/patterns/:name", ret.Get)
+	return
+}
+
+// Get handles the GET /patterns/:name route
+func (h *PatternsHandler) Get(c *gin.Context) {
+	name := c.Param("name")
+	variables := make(map[string]string) // Assuming variables are passed somehow
+	pattern, err := h.patterns.GetApplyVariables(name, variables)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.JSON(http.StatusOK, pattern)
+}

restapi/serve.go

@@ -0,0 +1,28 @@
+package restapi
+
+import (
+	"github.com/danielmiessler/fabric/core"
+	"github.com/gin-gonic/gin"
+)
+
+func Serve(registry *core.PluginRegistry, address string) (err error) {
+	r := gin.Default()
+
+	// Middleware
+	r.Use(gin.Logger())
+	r.Use(gin.Recovery())
+
+	// Register routes
+	fabricDb := registry.Db
+	NewPatternsHandler(r, fabricDb.Patterns)
+	NewContextsHandler(r, fabricDb.Contexts)
+	NewSessionsHandler(r, fabricDb.Sessions)
+
+	// Start server
+	err = r.Run(address)
+	if err != nil {
+		return err
+	}
+
+	return
+}

restapi/sessions.go

@@ -0,0 +1,19 @@
+package restapi
+
+import (
+	"github.com/danielmiessler/fabric/plugins/db/fsdb"
+	"github.com/gin-gonic/gin"
+)
+
+// SessionsHandler defines the handler for sessions-related operations
+type SessionsHandler struct {
+	*StorageHandler[fsdb.Session]
+	sessions *fsdb.SessionsEntity
+}
+
+// NewSessionsHandler creates a new SessionsHandler
+func NewSessionsHandler(r *gin.Engine, sessions *fsdb.SessionsEntity) (ret *SessionsHandler) {
+	ret = &SessionsHandler{
+		StorageHandler: NewStorageHandler[fsdb.Session](r, "sessions", sessions), sessions: sessions}
+	return ret
+}

restapi/storage.go

@@ -0,0 +1,100 @@
+package restapi
+
+import (
+	"fmt"
+	"github.com/danielmiessler/fabric/plugins/db"
+	"github.com/gin-gonic/gin"
+	"io"
+	"net/http"
+)
+
+// StorageHandler defines the handler for storage-related operations
+type StorageHandler[T any] struct {
+	storage db.Storage[T]
+}
+
+// NewStorageHandler creates a new StorageHandler
+func NewStorageHandler[T any](r *gin.Engine, entityType string, storage db.Storage[T]) (ret *StorageHandler[T]) {
+	ret = &StorageHandler[T]{storage: storage}
+	r.GET(fmt.Sprintf("/%s/:name", entityType), ret.Get)
+	r.GET(fmt.Sprintf("/%s/names", entityType), ret.GetNames)
+	r.DELETE(fmt.Sprintf("/%s/:name", entityType), ret.Delete)
+	r.GET(fmt.Sprintf("/%s/exists/:name", entityType), ret.Exists)
+	r.PUT(fmt.Sprintf("/%s/rename/:oldName/:newName", entityType), ret.Rename)
+	r.POST(fmt.Sprintf("/%s/:name", entityType), ret.Save)
+	return
+}
+
+// Get handles the GET /storage/:name route
+func (h *StorageHandler[T]) Get(c *gin.Context) {
+	name := c.Param("name")
+	item, err := h.storage.Get(name)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.JSON(http.StatusOK, item)
+}
+
+// GetNames handles the GET /storage/names route
+func (h *StorageHandler[T]) GetNames(c *gin.Context) {
+	names, err := h.storage.GetNames()
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.JSON(http.StatusOK, names)
+}
+
+// Delete handles the DELETE /storage/:name route
+func (h *StorageHandler[T]) Delete(c *gin.Context) {
+	name := c.Param("name")
+	err := h.storage.Delete(name)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.Status(http.StatusOK)
+}
+
+// Exists handles the GET /storage/exists/:name route
+func (h *StorageHandler[T]) Exists(c *gin.Context) {
+	name := c.Param("name")
+	exists := h.storage.Exists(name)
+	c.JSON(http.StatusOK, exists)
+}
+
+// Rename handles the PUT /storage/rename/:oldName/:newName route
+func (h *StorageHandler[T]) Rename(c *gin.Context) {
+	oldName := c.Param("oldName")
+	newName := c.Param("newName")
+	err := h.storage.Rename(oldName, newName)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.Status(http.StatusOK)
+}
+
+// Save handles the POST /storage/save/:name route
+func (h *StorageHandler[T]) Save(c *gin.Context) {
+	name := c.Param("name")
+
+	// Read the request body
+	body := c.Request.Body
+	defer body.Close()
+
+	content, err := io.ReadAll(body)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+
+	// Save the content to storage
+	err = h.storage.Save(name, content)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, err.Error())
+		return
+	}
+	c.Status(http.StatusOK)
+}

setup_fabric.bat

@@ -0,0 +1,112 @@
+@echo off
+setlocal enabledelayedexpansion
+
+:: Check if running with administrator privileges
+net session >nul 2>&1
+if %errorlevel% neq 0 (
+    echo Please run this script as an administrator.
+    pause
+    exit /b 1
+)
+
+:: Install Chocolatey (package manager for Windows)
+if not exist "%ProgramData%\chocolatey\bin\choco.exe" (
+    echo Installing Chocolatey...
+    @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "[System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
+)
+
+:: Install Go
+where go >nul 2>&1
+if %errorlevel% neq 0 (
+    echo Installing Go...
+    choco install golang -y
+    set "PATH=%PATH%;C:\Program Files\Go\bin"
+)
+
+:: Install Git
+where git >nul 2>&1
+if %errorlevel% neq 0 (
+    echo Installing Git...
+    choco install git -y
+)
+
+:: Refresh environment variables
+call refreshenv
+
+:: Install Fabric
+echo Installing Fabric...
+go install github.com/danielmiessler/fabric@latest
+
+:: Run Fabric setup
+echo Running Fabric setup...
+fabric --setup
+
+:: Install yt helper
+echo Installing yt helper...
+go install github.com/danielmiessler/yt@latest
+
+:: Prompt user for YouTube API Key
+set /p YOUTUBE_API_KEY=Enter your YouTube API Key (press Enter to skip): 
+if not "!YOUTUBE_API_KEY!"=="" (
+    echo YOUTUBE_API_KEY=!YOUTUBE_API_KEY!>> %USERPROFILE%\.config\fabric\.env
+)
+
+:: Prompt user for OpenAI API Key
+set /p OPENAI_API_KEY=Enter your OpenAI API Key (press Enter to skip): 
+if not "!OPENAI_API_KEY!"=="" (
+    echo OPENAI_API_KEY=!OPENAI_API_KEY!>> %USERPROFILE%\.config\fabric\.env
+)
+
+:: Run Fabric
+:run_fabric
+cls
+echo Fabric is now installed and ready to use.
+echo.
+echo Available options:
+echo 1. Run Fabric with custom options
+echo 2. List patterns
+echo 3. List models
+echo 4. Update patterns
+echo 5. Exit
+echo.
+set /p CHOICE=Enter your choice (1-5): 
+
+if "%CHOICE%"=="1" (
+    set /p PATTERN=Enter pattern (or press Enter to skip): 
+    set /p CONTEXT=Enter context (or press Enter to skip): 
+    set /p SESSION=Enter session (or press Enter to skip): 
+    set /p MODEL=Enter model (or press Enter to skip): 
+    set /p TEMPERATURE=Enter temperature (or press Enter for default): 
+    set /p STREAM=Do you want to stream output? (Y/N): 
+
+    set "FABRIC_CMD=fabric"
+    if not "!PATTERN!"=="" set "FABRIC_CMD=!FABRIC_CMD! --pattern !PATTERN!"
+    if not "!CONTEXT!"=="" set "FABRIC_CMD=!FABRIC_CMD! --context !CONTEXT!"
+    if not "!SESSION!"=="" set "FABRIC_CMD=!FABRIC_CMD! --session !SESSION!"
+    if not "!MODEL!"=="" set "FABRIC_CMD=!FABRIC_CMD! --model !MODEL!"
+    if not "!TEMPERATURE!"=="" set "FABRIC_CMD=!FABRIC_CMD! --temperature !TEMPERATURE!"
+    if /i "!STREAM!"=="Y" set "FABRIC_CMD=!FABRIC_CMD! --stream"
+
+    echo Running Fabric with command: !FABRIC_CMD!
+    !FABRIC_CMD!
+    pause
+    goto run_fabric
+) else if "%CHOICE%"=="2" (
+    fabric --listpatterns
+    pause
+    goto run_fabric
+) else if "%CHOICE%"=="3" (
+    fabric --listmodels
+    pause
+    goto run_fabric
+) else if "%CHOICE%"=="4" (
+    fabric --updatepatterns
+    pause
+    goto run_fabric
+) else if "%CHOICE%"=="5" (
+    exit /b 0
+) else (
+    echo Invalid choice. Please try again.
+    pause
+    goto run_fabric
+)

version.go

@@ -1,3 +1,3 @@
 package main
 
-var version = "v1.4.46"
+var version = "v1.4.73"