chore(release): Update version to v1.4.375

feat(vertexai): add dynamic model listing and multi-model support

.github/FUNDING.yml

@@ -0,0 +1,2 @@
+github: [danielmiessler, ksylvan]
+buy_me_a_coffee: kayvansylvan

.github/ISSUE_TEMPLATE/bug.yml

@@ -7,29 +7,74 @@ body:
     attributes:
       value: |
         Thanks for taking the time to fill out this bug report!
+        Please provide as much detail as possible to help us understand and reproduce the issue.
+
   - type: textarea
     id: what-happened
     attributes:
       label: What happened?
       description: Also tell us, what did you expect to happen?
       placeholder: Tell us what you see!
-      value: "I was doing THIS, when THAT happened. I was expecting THAT_OTHER_THING to happen instead."
+      value: "Please provide all the steps to reproduce the bug. I was doing THIS, when THAT happened. I was expecting THAT_OTHER_THING to happen instead."
     validations:
       required: true
-  - type: checkboxes
+
+  - type: dropdown
+    id: os
+    attributes:
+      label: Operating System
+      options:
+        - macOS - Silicon (arm64)
+        - macOS - Intel (amd64)
+        - Linux - amd64
+        - Linux - arm64
+        - Windows
+    validations:
+      required: true
+
+  - type: textarea
+    id: os-version
+    attributes:
+      label: OS Version
+      description: Please provide details about your OS version by running one of the following commands.
+      placeholder: |
+        macOS: `sw_vers`
+        Linux: `uname -a` or `cat /etc/os-release`
+        Windows: `ver`
+      render: shell
+
+  - type: dropdown
+    id: installation
+    attributes:
+      label: How did you install Fabric?
+      description: "Please select the method you used to install Fabric. You can find this information in the [Installation section of the README](https://github.com/ksylvan/fabric/blob/main/README.md#installation)."
+      options:
+        - Release Binary - Windows
+        - Release Binary - macOS (arm64)
+        - Release Binary - macOS (amd64)
+        - Release Binary - Linux (amd64)
+        - Release Binary - Linux (arm64)
+        - Package Manager - Homebrew (macOS)
+        - Package Manager - AUR (Arch Linux)
+        - From Source
+        - Other
+    validations:
+      required: true
+
+  - type: textarea
     id: version
     attributes:
-      label: Version check
-      description: Please make sure you were using the latest version of this project available in the `main` branch.
-      options:
-        - label: Yes I was.
-          required: true
+      label: Version
+      description: Please copy and paste the output of `fabric --version` (or `fabric-ai --version` if you installed it via brew) here.
+      render: text
+
   - type: textarea
     id: logs
     attributes:
       label: Relevant log output
       description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
       render: shell
+
   - type: textarea
     id: screens
     attributes:

.github/pull_request_template.md

@@ -1,9 +1,12 @@
 ## What this Pull Request (PR) does
+
 Please briefly describe what this PR does.
 
 ## Related issues
+
 Please reference any open issues this PR relates to in here.
 If it closes an issue, type `closes #[ISSUE_NUMBER]`.
 
 ## Screenshots
+
 Provide any screenshots you may find relevant to facilitate us understanding your PR.

.github/workflows/ci.yml

@@ -4,13 +4,13 @@ on:
   push:
     branches: ["main"]
     paths-ignore:
-      - 'patterns/**'
-      - '**/*.md'
+      - "data/patterns/**"
+      - "**/*.md"
   pull_request:
     branches: ["main"]
     paths-ignore:
-      - 'patterns/**'
-      - '**/*.md'
+      - "data/patterns/**"
+      - "**/*.md"
 
 jobs:
   test:
@@ -20,18 +20,22 @@ jobs:
       contents: read
     steps:
       - name: Checkout code
-        uses: actions/checkout@v4
-
-      - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@main
+        uses: actions/checkout@v6
 
       - name: Set up Go
-        uses: actions/setup-go@v4
+        uses: actions/setup-go@v6
         with:
           go-version-file: ./go.mod
 
       - name: Run tests
         run: go test -v ./...
 
+      - name: Check for modernization opportunities
+        run: |
+          go run golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest ./...
+
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@v21
+
       - name: Check Formatting
         run: nix flake check

.github/workflows/patterns.yaml

@@ -3,7 +3,7 @@ name: Patterns Artifact
 on:
   push:
     paths:
-      - "patterns/**" # Trigger only on changes to files in the patterns folder
+      - "data/patterns/**" # Trigger only on changes to files in the patterns folder
 
 jobs:
   zip-and-upload:
@@ -11,23 +11,28 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
 
       - name: Verify Changes in Patterns Folder
+        id: check-changes
         run: |
           git fetch origin
-          if git diff --quiet HEAD~1 -- patterns; then
+          if git diff --quiet HEAD~1 -- data/patterns; then
             echo "No changes detected in patterns folder."
-            exit 1
+            echo "changes=false" >> $GITHUB_OUTPUT
+          else
+            echo "Changes detected in patterns folder."
+            echo "changes=true" >> $GITHUB_OUTPUT
           fi
-
       - name: Zip the Patterns Folder
-        run: zip -r patterns.zip patterns/
+        if: steps.check-changes.outputs.changes == 'true'
+        run: zip -r patterns.zip data/patterns/
 
       - name: Upload Patterns Artifact
-        uses: actions/upload-artifact@v4
+        if: steps.check-changes.outputs.changes == 'true'
+        uses: actions/upload-artifact@v6
         with:
           name: patterns
           path: patterns.zip

.github/workflows/release.yml

@@ -2,7 +2,7 @@ name: Go Release
 
 on:
   repository_dispatch:
-    types: [ tag_created ]
+    types: [tag_created]
   push:
     tags:
       - "v*"
@@ -15,12 +15,12 @@ jobs:
       contents: read
     steps:
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
 
       - name: Set up Go
-        uses: actions/setup-go@v4
+        uses: actions/setup-go@v6
         with:
           go-version-file: ./go.mod
 
@@ -28,101 +28,30 @@ jobs:
         run: go test -v ./...
 
   build:
-    name: Build binaries for Windows, macOS, and Linux
-    runs-on: ${{ matrix.os }}
+    # only run in main upstream repo
+    if: ${{ github.repository_owner == 'danielmiessler' }}
+    name: Build & Release with Goreleaser
+    needs: [test]
+    runs-on: ubuntu-latest
     permissions:
       contents: write
-    strategy:
-      matrix:
-        os: [ubuntu-latest, macos-latest, windows-latest]
-        arch: [amd64, arm64]
-        exclude:
-          - os: windows-latest
-            arch: arm64
-
     steps:
       - name: Checkout code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
-
       - name: Set up Go
-        uses: actions/setup-go@v4
+        uses: actions/setup-go@v6
         with:
           go-version-file: ./go.mod
-
-      - name: Determine OS Name
-        id: os-name
-        run: |
-          if [ "${{ matrix.os }}" == "ubuntu-latest" ]; then
-            echo "OS=linux" >> $GITHUB_ENV
-          elif [ "${{ matrix.os }}" == "macos-latest" ]; then
-            echo "OS=darwin" >> $GITHUB_ENV
-          else
-            echo "OS=windows" >> $GITHUB_ENV
-          fi
-        shell: bash
-
-      - name: Build binary on Linux and macOS
-        if: matrix.os != 'windows-latest'
-        env:
-          GOOS: ${{ env.OS }}
-          GOARCH: ${{ matrix.arch }}
-        run: |
-          go build -o fabric-${OS}-${{ matrix.arch }} .
-
-      - name: Build binary on Windows
-        if: matrix.os == 'windows-latest'
-        env:
-          GOOS: windows
-          GOARCH: ${{ matrix.arch }}
-        run: |
-          go build -o fabric-windows-${{ matrix.arch }}.exe .
-
-      - name: Upload build artifact
-        if: matrix.os != 'windows-latest'
-        uses: actions/upload-artifact@v4
+      - name: Run GoReleaser
+        uses: goreleaser/goreleaser-action@v6
         with:
-          name: fabric-${OS}-${{ matrix.arch }}
-          path: fabric-${OS}-${{ matrix.arch }}
-
-      - name: Upload build artifact
-        if: matrix.os == 'windows-latest'
-        uses: actions/upload-artifact@v4
-        with:
-          name: fabric-windows-${{ matrix.arch }}.exe
-          path: fabric-windows-${{ matrix.arch }}.exe
-
-      - name: Get latest tag
-        if: matrix.os != 'windows-latest'
-        id: get_latest_tag
-        run: |
-          latest_tag=$(git tag --sort=-creatordate | head -n 1)
-          echo "latest_tag=$latest_tag" >> $GITHUB_ENV
-
-      - name: Get latest tag
-        if: matrix.os == 'windows-latest'
-        id: get_latest_tag_windows
-        run: |
-          $latest_tag = git tag --sort=-creatordate | Select-Object -First 1
-          Add-Content -Path $env:GITHUB_ENV -Value "latest_tag=$latest_tag"
-
-      - name: Create release if it doesn't exist
+          distribution: goreleaser
+          args: release --clean
         env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: |
-          gh release view ${{ env.latest_tag }} || gh release create ${{ env.latest_tag }} --title "Release ${{ env.latest_tag }}" --notes "Automated release for ${{ env.latest_tag }}"
-
-      - name: Upload release artifact
-        if: matrix.os == 'windows-latest'
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Update Release Description
+        run: go run ./cmd/generate_changelog --release ${{ github.event.client_payload.tag || github.ref_name }}
         env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: |
-          gh release upload ${{ env.latest_tag }} fabric-windows-${{ matrix.arch }}.exe
-
-      - name: Upload release artifact
-        if: matrix.os != 'windows-latest'
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: |
-          gh release upload ${{ env.latest_tag }} fabric-${OS}-${{ matrix.arch }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/update-version-and-create-tag.yml

@@ -3,13 +3,19 @@ name: Update Version File and Create Tag
 on:
   push:
     branches:
-      - main  # Monitor the main branch
+      - main # Monitor the main branch
     paths-ignore:
-      - 'patterns/**'
-      - '**/*.md'
+      - "data/patterns/**"
+      - "data/strategies/**"
+      - "cmd/generate_changelog/*.db"
+      - "cmd/generate_changelog/incoming/*.txt"
+      - "scripts/pattern_descriptions/*.json"
+      - "web/static/data/pattern_descriptions.json"
+      - "**/*.md"
+      - .vscode/**
 
 permissions:
-  contents: write  # Ensure the workflow has write permissions
+  contents: write # Ensure the workflow has write permissions
 
 concurrency:
   group: version-update
@@ -17,17 +23,18 @@ concurrency:
 
 jobs:
   update-version:
-    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+    if: >
+      github.repository_owner == 'danielmiessler' &&
+      github.event_name == 'push' && github.ref == 'refs/heads/main'
     runs-on: ubuntu-latest
-
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
 
       - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@main
+        uses: DeterminateSystems/nix-installer-action@v21
 
       - name: Set up Git
         run: |
@@ -44,12 +51,13 @@ jobs:
         run: |
           latest_tag=$(git tag --sort=-creatordate | head -n 1)
           echo "Latest tag is: $latest_tag"
+          echo "tag=$latest_tag" >> $GITHUB_OUTPUT
           echo "tag=$latest_tag" >> $GITHUB_ENV  # Save the latest tag to environment file
 
       - name: Increment patch version
         id: increment_version
         run: |
-          latest_tag=${{ env.tag }}
+          latest_tag=${{ steps.get_latest_tag.outputs.tag }}
           major=$(echo "$latest_tag" | cut -d. -f1 | sed 's/v//')
           minor=$(echo "$latest_tag" | cut -d. -f2)
           patch=$(echo "$latest_tag" | cut -d. -f3)
@@ -57,20 +65,22 @@ jobs:
           new_version="${major}.${minor}.${new_patch}"
           new_tag="v${new_version}"
           echo "New version is: $new_version"
+          echo "new_version=$new_version" >> $GITHUB_OUTPUT
           echo "new_version=$new_version" >> $GITHUB_ENV  # Save the new version to environment file
           echo "New tag is: $new_tag"
+          echo "new_tag=$new_tag" >> $GITHUB_OUTPUT
           echo "new_tag=$new_tag" >> $GITHUB_ENV  # Save the new tag to environment file
 
       - name: Update version.go file
         run: |
-          echo "package main" > version.go
-          echo "" >> version.go
-          echo "var version = \"${{ env.new_tag }}\"" >> version.go
+          echo "package main" > cmd/fabric/version.go
+          echo "" >> cmd/fabric/version.go
+          echo "var version = \"${{ steps.increment_version.outputs.new_tag }}\"" >> cmd/fabric/version.go
 
       - name: Update version.nix file
         run: |
-          echo "\"${{ env.new_version }}\"" > nix/pkgs/fabric/version.nix
-      
+          echo "\"${{ steps.increment_version.outputs.new_version }}\"" > nix/pkgs/fabric/version.nix
+
       - name: Format source code
         run: |
           nix fmt
@@ -79,37 +89,47 @@ jobs:
         run: |
           nix run .#gomod2nix -- --outdir nix/pkgs/fabric
 
+      - name: Generate Changelog Entry
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          go run ./cmd/generate_changelog --process-prs ${{ steps.increment_version.outputs.new_tag }}
+          git add ./cmd/generate_changelog/changelog.db
       - name: Commit changes
         run: |
-          git add version.go
+          # These files are modified by the version bump process
+          git add cmd/fabric/version.go
           git add nix/pkgs/fabric/version.nix
           git add nix/pkgs/fabric/gomod2nix.toml
-          git add .
+
+          # The changelog tool is responsible for staging CHANGELOG.md, changelog.db,
+          # and removing the incoming/ directory.
+
           if ! git diff --staged --quiet; then
-            git commit -m "Update version to ${{ env.new_tag }} and commit $commit_hash"
+            git commit -m "chore(release): Update version to ${{ steps.increment_version.outputs.new_tag }}"
           else
             echo "No changes to commit."
           fi
 
       - name: Push changes
         env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}  # Use GITHUB_TOKEN to authenticate the push
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Use GITHUB_TOKEN to authenticate the push
         run: |
           git push origin main  # Push changes to the main branch
 
       - name: Create a new tag
         env:
-          GITHUB_TOKEN: ${{ secrets.TAG_PAT }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
-          git tag ${{ env.new_tag }}
-          git push origin ${{ env.new_tag }}  # Push the new tag
+          git tag ${{ steps.increment_version.outputs.new_tag }}
+          git push origin ${{ steps.increment_version.outputs.new_tag }}  # Push the new tag
 
       - name: Dispatch event to trigger release workflow
         env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}  # Use GITHUB_TOKEN to authenticate the dispatch
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Use GITHUB_TOKEN to authenticate the dispatch
         run: |
           curl -X POST \
             -H "Authorization: token $GITHUB_TOKEN" \
             -H "Accept: application/vnd.github.v3+json" \
             https://api.github.com/repos/${{ github.repository }}/dispatches \
-            -d '{"event_type": "tag_created", "client_payload": {"tag": "${{ env.new_tag }}"}}'
+            -d '{"event_type": "tag_created", "client_payload": {"tag": "${{ steps.increment_version.outputs.new_tag }}"}}'

.gitignore

@@ -131,9 +131,7 @@ celerybeat.pid
 # Environments
 .env
 .venv
-env/
 venv/
-ENV/
 env.bak/
 venv.bak/
 
@@ -349,5 +347,9 @@ web/package-lock.json
 .gitignore_backup
 web/static/*.png
 
-# Local VSCode project settings
-.vscode/
+# Local tmp directory
+.tmp/
+tmp/
+
+# Ignore .claude/
+.claude/

.goreleaser.yaml

@@ -0,0 +1,56 @@
+# Read the documentation at https://goreleaser.com
+# For a full reference of the configuration file.
+
+version: 2
+
+project_name: fabric
+before:
+  hooks:
+    - go mod tidy
+
+builds:
+  - id: default
+    env:
+      - CGO_ENABLED=0
+    goos:
+      - darwin
+      - linux
+    main: ./cmd/fabric
+    binary: fabric
+    ldflags:
+      - -s -w
+      - -X main.version={{.Version}}
+      - -X main.commit={{.ShortCommit}}
+      - -X main.date={{.Date}}
+      - -X main.builtBy=goreleaser
+      - -X main.tag={{.Tag}}
+
+  - id: windows-build
+    env:
+      - CGO_ENABLED=0
+    goos:
+      - windows
+    main: ./cmd/fabric
+    binary: fabric
+    ldflags:
+      - -s -w
+      - -X main.version={{.Version}}
+      - -X main.commit={{.ShortCommit}}
+      - -X main.date={{.Date}}
+      - -X main.builtBy=goreleaser
+      - -X main.tag={{.Tag}}
+
+archives:
+  - formats: [tar.gz]
+    # this name template makes the OS and Arch compatible with the results of `uname`.
+    name_template: >-
+      {{ .ProjectName }}_
+      {{- title .Os }}_
+      {{- if eq .Arch "amd64" }}x86_64
+      {{- else if eq .Arch "386" }}i386
+      {{- else }}{{ .Arch }}{{ end }}
+      {{- if .Arm }}v{{ .Arm }}{{ end }}
+    # use zip for windows archives
+    format_overrides:
+      - goos: windows
+        formats: [zip]

.vscode/extensions.json

@@ -0,0 +1,3 @@
+{
+	"recommendations": ["davidanson.vscode-markdownlint"]
+}

.vscode/settings.json

@@ -0,0 +1,251 @@
+{
+	"cSpell.words": [
+		"Achird",
+		"addextension",
+		"adduser",
+		"AIML",
+		"Anki",
+		"anthropics",
+		"Aoede",
+		"apikey",
+		"aplicar",
+		"Astley",
+		"atotto",
+		"Autonoe",
+		"azureml",
+		"badfile",
+		"Behrens",
+		"blindspots",
+		"Bombal",
+		"Buildx",
+		"byid",
+		"Callirhoe",
+		"Callirrhoe",
+		"Cerebras",
+		"colour",
+		"compadd",
+		"compdef",
+		"compinit",
+		"conceptmap",
+		"creatordate",
+		"curcontext",
+		"custompatterns",
+		"danielmiessler",
+		"davidanson",
+		"Debugf",
+		"debuglog",
+		"dedup",
+		"deepseek",
+		"Despina",
+		"direnv",
+		"DMARC",
+		"DOCKERHUB",
+		"dryrun",
+		"dsrp",
+		"editability",
+		"Eisler",
+		"elif",
+		"Elister",
+		"entrada",
+		"envrc",
+		"Erinome",
+		"Errorf",
+		"eugeis",
+		"Eugen",
+		"excalidraw",
+		"exolab",
+		"fabriclogo",
+		"flac",
+		"fpath",
+		"frequencypenalty",
+		"fsdb",
+		"gantt",
+		"genai",
+		"githelper",
+		"gjson",
+		"GOARCH",
+		"GODEBUG",
+		"godoc",
+		"godotenv",
+		"GOEXPERIMENT",
+		"gofmt",
+		"goimports",
+		"golint",
+		"GOMAXPROCS",
+		"gomod",
+		"gonic",
+		"goopenai",
+		"GOPATH",
+		"gopkg",
+		"Goreleaser",
+		"GOROOT",
+		"goroutines",
+		"Graphviz",
+		"greenteagc",
+		"grokai",
+		"Groq",
+		"hackerone",
+		"Haddix",
+		"hasura",
+		"hormozi",
+		"Hormozi's",
+		"horts",
+		"HTMLURL",
+		"imagetools",
+		"Jamba",
+		"jaredmontoya",
+		"jessevdk",
+		"Jina",
+		"joho",
+		"kballard",
+		"Keploy",
+		"kimi",
+		"Kore",
+		"ksylvan",
+		"Langdock",
+		"Laomedeia",
+		"ldflags",
+		"legibilidad",
+		"libexec",
+		"libnotify",
+		"listcontexts",
+		"listextensions",
+		"listmodels",
+		"listpatterns",
+		"listsessions",
+		"liststrategies",
+		"listvendors",
+		"lmstudio",
+		"Makefiles",
+		"markmap",
+		"matplotlib",
+		"mattn",
+		"mbed",
+		"Mdsvex",
+		"metacharacters",
+		"Miessler",
+		"modeline",
+		"modelines",
+		"mpga",
+		"mvdan",
+		"nicksnyder",
+		"nixpkgs",
+		"nometa",
+		"numpy",
+		"ollama",
+		"ollamaapi",
+		"Omri",
+		"openaiapi",
+		"opencode",
+		"opencontainers",
+		"openrouter",
+		"organise",
+		"Orus",
+		"osascript",
+		"otiai",
+		"pdflatex",
+		"pipx",
+		"PKCE",
+		"pkgs",
+		"porque",
+		"presencepenalty",
+		"printcontext",
+		"printsession",
+		"puede",
+		"Pulcherrima",
+		"pycache",
+		"pyperclip",
+		"qwen",
+		"readystream",
+		"restapi",
+		"rmextension",
+		"Sadachbia",
+		"Sadaltager",
+		"samber",
+		"sashabaranov",
+		"sdist",
+		"seaborn",
+		"semgrep",
+		"sess",
+		"sgaunet",
+		"shellquote",
+		"skeletonlabs",
+		"SSEHTTP",
+		"storer",
+		"Streamlit",
+		"stretchr",
+		"subchunk",
+		"Sulafat",
+		"swaggo",
+		"synctest",
+		"talkpanel",
+		"Telos",
+		"testpattern",
+		"testuser",
+		"Thacker",
+		"tidwall",
+		"topp",
+		"ttrc",
+		"unalias",
+		"unconfigured",
+		"unmarshalling",
+		"updatepatterns",
+		"useb",
+		"USERPROFILE",
+		"varnames",
+		"videoid",
+		"webp",
+		"WEBVTT",
+		"winget",
+		"wipecontext",
+		"wipesession",
+		"wireframes",
+		"Worktree",
+		"writeups",
+		"xclip",
+		"yourpatternname",
+		"youtu",
+		"YTDLP"
+	],
+	"cSpell.ignorePaths": [
+		"go.mod",
+		".gitignore",
+		"CHANGELOG.md",
+		"scripts/installer/install.*",
+		"web/static/data/pattern_descriptions.json",
+		"scripts/pattern_descriptions/*.json",
+		"data/patterns/pattern_explanations.md",
+		"internal/i18n/locales/es.json",
+		"internal/i18n/locales/fr.json",
+		"internal/i18n/locales/de.json",
+		"internal/i18n/locales/it.json",
+		"internal/i18n/locales/pt.json",
+		"internal/i18n/locales/zh.json",
+		"internal/i18n/i18n_test.go"
+	],
+	"markdownlint.config": {
+		"MD004": false,
+		"MD011": false,
+		"MD024": false,
+		"MD025": false,
+		"M032": false,
+		"MD033": {
+			"allowed_elements": [
+				"a",
+				"br",
+				"code",
+				"details",
+				"div",
+				"em",
+				"h",
+				"h4",
+				"img",
+				"module",
+				"p",
+				"summary",
+				"sup"
+			]
+		},
+		"MD041": false
+	}
+}

Dockerfile

@@ -1,41 +0,0 @@
-# Use official golang image as builder
-FROM golang:1.24.2-alpine AS builder
-
-# Set working directory
-WORKDIR /app
-
-# Copy go mod and sum files
-COPY go.mod go.sum ./
-
-# Download dependencies
-RUN go mod download
-
-# Copy source code
-COPY . .
-
-# Build the application
-RUN CGO_ENABLED=0 GOOS=linux go build -o fabric
-
-# Use scratch as final base image
-FROM alpine:latest
-
-# Copy the binary from builder
-COPY --from=builder /app/fabric /fabric
-
-# Copy patterns directory
-COPY patterns /patterns
-
-# Ensure clean config directory and copy ENV file
-RUN rm -rf /root/.config/fabric && \
-    mkdir -p /root/.config/fabric
-COPY ENV /root/.config/fabric/.env
-
-# Add debug commands
-RUN ls -la /root/.config/fabric/
-
-# Expose port 8080
-EXPOSE 8080
-
-# Run the binary with debug output
-ENTRYPOINT ["/fabric"]
-CMD ["--serve"] 

README.md

@@ -1,9 +1,20 @@
 <div align="center">
-Fabric is graciously supported by…
+    <a href="https://go.warp.dev/fabric" target="_blank">
+        <sup>Special thanks to:</sup>
+        <br>
+        <img alt="Warp sponsorship" width="400" src="https://raw.githubusercontent.com/warpdotdev/brand-assets/refs/heads/main/Github/Sponsor/Warp-Github-LG-02.png">
+        <br>
+        <h>Warp, built for coding with multiple AI agents</b>
+        <br>
+        <sup>Available for macOS, Linux and Windows</sup>
+    </a>
+</div>
 
-[![Github Repo Tagline](https://github.com/user-attachments/assets/96ab3d81-9b13-4df4-ba09-75dee7a5c3d2)](https://warp.dev/fabric)
+<br>
 
-<img src="./images/fabric-logo-gif.gif" alt="fabriclogo" width="400" height="400"/>
+<div align="center">
+
+<img src="./docs/images/fabric-logo-gif.gif" alt="fabriclogo" width="400" height="400"/>
 
 # `fabric`
 
@@ -12,11 +23,14 @@ Fabric is graciously supported by…
 ![GitHub top language](https://img.shields.io/github/languages/top/danielmiessler/fabric)
 ![GitHub last commit](https://img.shields.io/github/last-commit/danielmiessler/fabric)
 [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/danielmiessler/fabric)
 
 <div align="center">
-<p class="align center">
 <h4><code>fabric</code> is an open-source framework for augmenting humans using AI.</h4>
-</p>
+</div>
+
+![Screenshot of fabric](./docs/images/fabric-summarize.png)
+
 </div>
 
 [Updates](#updates) •
@@ -24,30 +38,96 @@ Fabric is graciously supported by…
 [Philosophy](#philosophy) •
 [Installation](#installation) •
 [Usage](#usage) •
+[REST API](#rest-api-server) •
 [Examples](#examples) •
 [Just Use the Patterns](#just-use-the-patterns) •
 [Custom Patterns](#custom-patterns) •
 [Helper Apps](#helper-apps) •
 [Meta](#meta)
 
-![Screenshot of fabric](images/fabric-summarize.png)
-
 </div>
 
 ## What and why
 
-Since the start of modern AI in late 2022 we've seen an **_extraordinary_** number of AI applications for accomplishing tasks. There are thousands of websites, chatbots, mobile apps, and other interfaces for using all the differnet AI out there.
+Since the start of modern AI in late 2022 we've seen an **_extraordinary_** number of AI applications for accomplishing tasks. There are thousands of websites, chat-bots, mobile apps, and other interfaces for using all the different AI out there.
 
 It's all really exciting and powerful, but _it's not easy to integrate this functionality into our lives._
 
-<p class="align center">
+<div class="align center">
 <h4>In other words, AI doesn't have a capabilities problem—it has an <em>integration</em> problem.</h4>
-</p>
+</div>
 
-**Fabric was created to address this by creating and organizating the fundamental units of AI—the prompts themselves!**
+**Fabric was created to address this by creating and organizing the fundamental units of AI—the prompts themselves!**
 
 Fabric organizes prompts by real-world task, allowing people to create, collect, and organize their most important AI solutions in a single place for use in their favorite tools. And if you're command-line focused, you can use Fabric itself as the interface!
 
+## Updates
+
+<details>
+<summary>Click to view recent updates</summary>
+
+Dear Users,
+
+We've been doing so many exciting things here at Fabric, I wanted to give a quick summary here to give you a sense of our development velocity!
+
+Below are the **new features and capabilities** we've added (newest first):
+
+### Recent Major Features
+
+- [v1.4.356](https://github.com/danielmiessler/fabric/releases/tag/v1.4.356) (Dec 22, 2025) — **Complete Internationalization**: Full i18n support for setup prompts across all 10 languages with intelligent environment variable handling—making Fabric truly accessible worldwide while maintaining configuration consistency.
+- [v1.4.350](https://github.com/danielmiessler/fabric/releases/tag/v1.4.350) (Dec 18, 2025) — **Interactive API Documentation**: Adds Swagger/OpenAPI UI at `/swagger/index.html` with comprehensive REST API documentation, enhanced developer guides, and improved endpoint discoverability for easier integration.
+- [v1.4.338](https://github.com/danielmiessler/fabric/releases/tag/v1.4.338) (Dec 4, 2025) — Add Abacus vendor support for Chat-LLM
+  models (see [RouteLLM APIs](https://abacus.ai/app/route-llm-apis)).
+- [v1.4.337](https://github.com/danielmiessler/fabric/releases/tag/v1.4.337) (Dec 4, 2025) — Add "Z AI" vendor support. See the [Z AI overview](https://docs.z.ai/guides/overview/overview) page for more details.
+- [v1.4.334](https://github.com/danielmiessler/fabric/releases/tag/v1.4.334) (Nov 26, 2025) — **Claude Opus 4.5**: Updates the Anthropic SDK to the latest and adds the new [Claude Opus 4.5](https://www.anthropic.com/news/claude-opus-4-5) to the available models.
+- [v1.4.331](https://github.com/danielmiessler/fabric/releases/tag/v1.4.331) (Nov 23, 2025) — **Support for GitHub Models**: Adds support for using GitHub Models.
+- [v1.4.322](https://github.com/danielmiessler/fabric/releases/tag/v1.4.322) (Nov 5, 2025) — **Interactive HTML Concept Maps and Claude Sonnet 4.5**: Adds `create_conceptmap` pattern for visual knowledge representation using Vis.js, introduces WELLNESS category with psychological analysis patterns, and upgrades to Claude Sonnet 4.5
+- [v1.4.317](https://github.com/danielmiessler/fabric/releases/tag/v1.4.317) (Sep 21, 2025) — **Portuguese Language Variants**: Adds BCP 47 locale normalization with support for Brazilian Portuguese (pt-BR) and European Portuguese (pt-PT) with intelligent fallback chains
+- [v1.4.314](https://github.com/danielmiessler/fabric/releases/tag/v1.4.314) (Sep 17, 2025) — **Azure OpenAI Migration**: Migrates to official `openai-go/azure` SDK with improved authentication and default API version support
+- [v1.4.311](https://github.com/danielmiessler/fabric/releases/tag/v1.4.311) (Sep 13, 2025) — **More internationalization support**: Adds de (German), fa (Persian / Farsi), fr (French), it (Italian),
+  ja (Japanese), pt (Portuguese), zh (Chinese)
+- [v1.4.309](https://github.com/danielmiessler/fabric/releases/tag/v1.4.309) (Sep 9, 2025) — **Comprehensive internationalization support**: Includes English and Spanish locale files.
+- [v1.4.303](https://github.com/danielmiessler/fabric/releases/tag/v1.4.303) (Aug 29, 2025) — **New Binary Releases**: Linux ARM and Windows ARM targets. You can run Fabric on the Raspberry PI and on your Windows Surface!
+- [v1.4.294](https://github.com/danielmiessler/fabric/releases/tag/v1.4.294) (Aug 20, 2025) — **Venice AI Support**: Added the Venice AI provider. Venice is a Privacy-First, Open-Source AI provider. See their ["About Venice"](https://docs.venice.ai/overview/about-venice) page for details.
+- [v1.4.291](https://github.com/danielmiessler/fabric/releases/tag/v1.4.291) (Aug 18, 2025) — **Speech To Text**: Add OpenAI speech-to-text support with `--transcribe-file`, `--transcribe-model`, and `--split-media-file` flags.
+- [v1.4.287](https://github.com/danielmiessler/fabric/releases/tag/v1.4.287) (Aug 16, 2025) — **AI Reasoning**: Add Thinking to Gemini models and introduce `readme_updates` python script
+- [v1.4.286](https://github.com/danielmiessler/fabric/releases/tag/v1.4.286) (Aug 14, 2025) — **AI Reasoning**: Introduce Thinking Config Across Anthropic and OpenAI Providers
+- [v1.4.285](https://github.com/danielmiessler/fabric/releases/tag/v1.4.285) (Aug 13, 2025) — **Extended Context**: Enable One Million Token Context Beta Feature for Sonnet-4
+- [v1.4.284](https://github.com/danielmiessler/fabric/releases/tag/v1.4.284) (Aug 12, 2025) — **Easy Shell Completions Setup**: Introduce One-Liner Curl Install for Completions
+- [v1.4.283](https://github.com/danielmiessler/fabric/releases/tag/v1.4.283) (Aug 12, 2025) — **Model Management**: Add Vendor Selection Support for Models
+- [v1.4.282](https://github.com/danielmiessler/fabric/releases/tag/v1.4.282) (Aug 11, 2025) — **Enhanced Shell Completions**: Enhanced Shell Completions for Fabric CLI Binaries
+- [v1.4.281](https://github.com/danielmiessler/fabric/releases/tag/v1.4.281) (Aug 11, 2025) — **Gemini Search Tool**: Add Web Search Tool Support for Gemini Models
+- [v1.4.278](https://github.com/danielmiessler/fabric/releases/tag/v1.4.278) (Aug 9, 2025) — **Enhance YouTube Transcripts**: Enhance YouTube Support with Custom yt-dlp Arguments
+- [v1.4.277](https://github.com/danielmiessler/fabric/releases/tag/v1.4.277) (Aug 8, 2025) — **Desktop Notifications**: Add cross-platform desktop notifications to Fabric CLI
+- [v1.4.274](https://github.com/danielmiessler/fabric/releases/tag/v1.4.274) (Aug 7, 2025) — **Claude 4.1 Added**: Add Support for Claude Opus 4.1 Model
+- [v1.4.271](https://github.com/danielmiessler/fabric/releases/tag/v1.4.271) (Jul 28, 2025) — **AI Summarized Release Notes**: Enable AI summary updates for GitHub releases
+- [v1.4.268](https://github.com/danielmiessler/fabric/releases/tag/v1.4.268) (Jul 26, 2025) — **Gemini TTS Voice Selection**: add Gemini TTS voice selection and listing functionality
+- [v1.4.267](https://github.com/danielmiessler/fabric/releases/tag/v1.4.267) (Jul 26, 2025) — **Text-to-Speech**: Update Gemini Plugin to New SDK with TTS Support
+- [v1.4.258](https://github.com/danielmiessler/fabric/releases/tag/v1.4.258) (Jul 17, 2025) — **Onboarding Improved**: Add startup check to initialize config and .env file automatically
+- [v1.4.257](https://github.com/danielmiessler/fabric/releases/tag/v1.4.257) (Jul 17, 2025) — **OpenAI Routing Control**: Introduce CLI Flag to Disable OpenAI Responses API
+- [v1.4.252](https://github.com/danielmiessler/fabric/releases/tag/v1.4.252) (Jul 16, 2025) — **Hide Thinking Block**: Optional Hiding of Model Thinking Process with Configurable Tags
+- [v1.4.246](https://github.com/danielmiessler/fabric/releases/tag/v1.4.246) (Jul 14, 2025) — **Automatic ChangeLog Updates**: Add AI-powered changelog generation with high-performance Go tool and comprehensive caching
+- [v1.4.245](https://github.com/danielmiessler/fabric/releases/tag/v1.4.245) (Jul 11, 2025) — **Together AI**: Together AI Support with OpenAI Fallback Mechanism Added
+- [v1.4.232](https://github.com/danielmiessler/fabric/releases/tag/v1.4.232) (Jul 6, 2025) — **Add Custom**: Add Custom Patterns Directory Support
+- [v1.4.231](https://github.com/danielmiessler/fabric/releases/tag/v1.4.231) (Jul 5, 2025) — **OAuth Auto-Auth**: OAuth Authentication Support for Anthropic (Use your Max Subscription)
+- [v1.4.230](https://github.com/danielmiessler/fabric/releases/tag/v1.4.230) (Jul 5, 2025) — **Model Management**: Add advanced image generation parameters for OpenAI models with four new CLI flags
+- [v1.4.227](https://github.com/danielmiessler/fabric/releases/tag/v1.4.227) (Jul 4, 2025) — **Add Image**: Add Image Generation Support to Fabric
+- [v1.4.226](https://github.com/danielmiessler/fabric/releases/tag/v1.4.226) (Jul 4, 2025) — **Web Search**: OpenAI Plugin Now Supports Web Search Functionality
+- [v1.4.225](https://github.com/danielmiessler/fabric/releases/tag/v1.4.225) (Jul 4, 2025) — **Web Search**: Runtime Web Search Control via Command-Line `--search` Flag
+- [v1.4.224](https://github.com/danielmiessler/fabric/releases/tag/v1.4.224) (Jul 1, 2025) — **Add code_review**: Add code_review pattern and updates in Pattern_Descriptions
+- [v1.4.222](https://github.com/danielmiessler/fabric/releases/tag/v1.4.222) (Jul 1, 2025) — **OpenAI Plugin**: OpenAI Plugin Migrates to New Responses API
+- [v1.4.218](https://github.com/danielmiessler/fabric/releases/tag/v1.4.218) (Jun 27, 2025) — **Model Management**: Add Support for OpenAI Search and Research Model Variants
+- [v1.4.217](https://github.com/danielmiessler/fabric/releases/tag/v1.4.217) (Jun 26, 2025) — **New YouTube**: New YouTube Transcript Endpoint Added to REST API
+- [v1.4.212](https://github.com/danielmiessler/fabric/releases/tag/v1.4.212) (Jun 23, 2025) — **Add Langdock**: Add Langdock AI and enhance generic OpenAI compatible support
+- [v1.4.211](https://github.com/danielmiessler/fabric/releases/tag/v1.4.211) (Jun 19, 2025) — **REST API**: REST API and Web UI Now Support Dynamic Pattern Variables
+- [v1.4.210](https://github.com/danielmiessler/fabric/releases/tag/v1.4.210) (Jun 18, 2025) — **Add Citations**: Add Citation Support to Perplexity Response
+- [v1.4.208](https://github.com/danielmiessler/fabric/releases/tag/v1.4.208) (Jun 17, 2025) — **Add Perplexity**: Add Perplexity AI Provider with Token Limits Support
+- [v1.4.203](https://github.com/danielmiessler/fabric/releases/tag/v1.4.203) (Jun 14, 2025) — **Add Amazon Bedrock**: Add support for Amazon Bedrock
+
+These features represent our commitment to making Fabric the most powerful and flexible AI augmentation framework available!
+
+</details>
+
 ## Intro videos
 
 Keep in mind that many of these were recorded when Fabric was Python-based, so remember to use the current [install instructions](#installation) below.
@@ -57,77 +137,73 @@ Keep in mind that many of these were recorded when Fabric was Python-based, so r
 - [My Own Intro to the Tool](https://www.youtube.com/watch?v=wPEyyigh10g)
 - [More Fabric YouTube Videos](https://www.youtube.com/results?search_query=fabric+ai)
 
-
 ## Navigation
 
 - [`fabric`](#fabric)
-  - [Navigation](#navigation)
-  - [Updates](#updates)
   - [What and why](#what-and-why)
+  - [Updates](#updates)
+    - [Recent Major Features](#recent-major-features)
   - [Intro videos](#intro-videos)
+  - [Navigation](#navigation)
+  - [Changelog](#changelog)
   - [Philosophy](#philosophy)
     - [Breaking problems into components](#breaking-problems-into-components)
     - [Too many prompts](#too-many-prompts)
   - [Installation](#installation)
-    - [Get Latest Release Binaries](#get-latest-release-binaries)
-      - [Windows](#windows)
-      - [macOS (arm64)](#macos-arm64)
-      - [macOS (amd64)](#macos-amd64)
-      - [Linux (amd64)](#linux-amd64)
-      - [Linux (arm64)](#linux-arm64)
+    - [One-Line Install (Recommended)](#one-line-install-recommended)
+    - [Manual Binary Downloads](#manual-binary-downloads)
     - [Using package managers](#using-package-managers)
       - [macOS (Homebrew)](#macos-homebrew)
       - [Arch Linux (AUR)](#arch-linux-aur)
+      - [Windows](#windows)
     - [From Source](#from-source)
+    - [Docker](#docker)
     - [Environment Variables](#environment-variables)
     - [Setup](#setup)
+    - [Supported AI Providers](#supported-ai-providers)
+    - [Per-Pattern Model Mapping](#per-pattern-model-mapping)
     - [Add aliases for all patterns](#add-aliases-for-all-patterns)
       - [Save your files in markdown using aliases](#save-your-files-in-markdown-using-aliases)
     - [Migration](#migration)
     - [Upgrading](#upgrading)
     - [Shell Completions](#shell-completions)
+      - [Quick install (no clone required)](#quick-install-no-clone-required)
       - [Zsh Completion](#zsh-completion)
       - [Bash Completion](#bash-completion)
       - [Fish Completion](#fish-completion)
   - [Usage](#usage)
+    - [Debug Levels](#debug-levels)
+    - [Dry Run Mode](#dry-run-mode)
+    - [Extensions](#extensions)
+  - [REST API Server](#rest-api-server)
+    - [Ollama Compatibility Mode](#ollama-compatibility-mode)
   - [Our approach to prompting](#our-approach-to-prompting)
   - [Examples](#examples)
   - [Just use the Patterns](#just-use-the-patterns)
     - [Prompt Strategies](#prompt-strategies)
+      - [Available Strategies](#available-strategies)
   - [Custom Patterns](#custom-patterns)
+    - [Setting Up Custom Patterns](#setting-up-custom-patterns)
+    - [Using Custom Patterns](#using-custom-patterns)
+    - [How It Works](#how-it-works)
   - [Helper Apps](#helper-apps)
     - [`to_pdf`](#to_pdf)
     - [`to_pdf` Installation](#to_pdf-installation)
-    - [`code_helper`](#code_helper)
+    - [`code2context`](#code2context)
+    - [`generate_changelog`](#generate_changelog)
   - [pbpaste](#pbpaste)
-  - [Web Interface](#web-interface)
-    - [Installing](#installing)
-    - [Streamlit UI](#streamlit-ui)
-      - [Clipboard Support](#clipboard-support)
+  - [Web Interface (Fabric Web App)](#web-interface-fabric-web-app)
   - [Meta](#meta)
     - [Primary contributors](#primary-contributors)
     - [Contributors](#contributors)
 
 <br />
 
-## Updates
+## Changelog
 
-> [!NOTE]
->
->June 17, 2025
->
->- Fabric now supports Perplexity AI. Configure it by using `fabric -S` to add your Perlexity AI API Key,
->   and then try:
->
->   ```bash
->   fabric -m sonar-pro "What is the latest world news?"
->   ```
->
->June 11, 2025
->
->- Fabric's YouTube transcription now needs `yt-dlp` to be installed. Make sure to install the latest
-> version (2025.06.09 as of this note). The YouTube API key is only needed for comments (the `--comments` flag)
-> and metadata extraction (the `--metadata` flag).
+Fabric is evolving rapidly.
+
+Stay current with the latest features by reviewing the [CHANGELOG](./CHANGELOG.md) for all recent changes.
 
 ## Philosophy
 
@@ -162,29 +238,25 @@ Fabric has Patterns for all sorts of life and work activities, including:
 
 ## Installation
 
-To install Fabric, you can use the latest release binaries or install it from the source.
+### One-Line Install (Recommended)
 
-### Get Latest Release Binaries
+**Unix/Linux/macOS:**
 
-#### Windows
+```bash
+curl -fsSL https://raw.githubusercontent.com/danielmiessler/fabric/main/scripts/installer/install.sh | bash
+```
 
-`https://github.com/danielmiessler/fabric/releases/latest/download/fabric-windows-amd64.exe`
+**Windows PowerShell:**
 
-#### macOS (arm64)
+```powershell
+iwr -useb https://raw.githubusercontent.com/danielmiessler/fabric/main/scripts/installer/install.ps1 | iex
+```
 
-`curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-darwin-arm64 > fabric && chmod +x fabric && ./fabric --version`
+> See [scripts/installer/README.md](./scripts/installer/README.md) for custom installation options and troubleshooting.
 
-#### macOS (amd64)
+### Manual Binary Downloads
 
-`curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-darwin-amd64 > fabric && chmod +x fabric && ./fabric --version`
-
-#### Linux (amd64)
-
-`curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-linux-amd64 > fabric && chmod +x fabric && ./fabric --version`
-
-#### Linux (arm64)
-
-`curl -L https://github.com/danielmiessler/fabric/releases/latest/download/fabric-linux-arm64 > fabric && chmod +x fabric && ./fabric --version`
+The latest release binary archives and their expected SHA256 hashes can be found at <https://github.com/danielmiessler/fabric/releases/latest>
 
 ### Using package managers
 
@@ -203,15 +275,50 @@ alias fabric='fabric-ai'
 
 `yay -S fabric-ai`
 
+#### Windows
+
+Use the official Microsoft supported `Winget` tool:
+
+`winget install danielmiessler.Fabric`
+
 ### From Source
 
 To install Fabric, [make sure Go is installed](https://go.dev/doc/install), and then run the following command.
 
 ```bash
 # Install Fabric directly from the repo
-go install github.com/danielmiessler/fabric@latest
+go install github.com/danielmiessler/fabric/cmd/fabric@latest
 ```
 
+### Docker
+
+Run Fabric using pre-built Docker images:
+
+```bash
+# Use latest image from Docker Hub
+docker run --rm -it kayvan/fabric:latest --version
+
+# Use specific version from GHCR
+docker run --rm -it ghcr.io/ksylvan/fabric:v1.4.305 --version
+
+# Run setup (first time)
+mkdir -p $HOME/.fabric-config
+docker run --rm -it -v $HOME/.fabric-config:/root/.config/fabric kayvan/fabric:latest --setup
+
+# Use Fabric with your patterns
+docker run --rm -it -v $HOME/.fabric-config:/root/.config/fabric kayvan/fabric:latest -p summarize
+
+# Run the REST API server (see REST API Server section)
+docker run --rm -it -p 8080:8080 -v $HOME/.fabric-config:/root/.config/fabric kayvan/fabric:latest --serve
+```
+
+**Images available at:**
+
+- Docker Hub: [kayvan/fabric](https://hub.docker.com/repository/docker/kayvan/fabric/general)
+- GHCR: [ksylvan/fabric](https://github.com/ksylvan/fabric/pkgs/container/fabric)
+
+See [scripts/docker/README.md](./scripts/docker/README.md) for building custom images and advanced configuration.
+
 ### Environment Variables
 
 You may need to set some environment variables in your `~/.bashrc` on linux or `~/.zshrc` file on mac to be able to run the `fabric` command. Here is an example of what you can add:
@@ -247,19 +354,66 @@ fabric --setup
 
 If everything works you are good to go.
 
+### Supported AI Providers
+
+Fabric supports a wide range of AI providers:
+
+**Native Integrations:**
+
+- OpenAI
+- Anthropic (Claude)
+- Google Gemini
+- Ollama (local models)
+- Azure OpenAI
+- Amazon Bedrock
+- Vertex AI
+- LM Studio
+- Perplexity
+
+**OpenAI-Compatible Providers:**
+
+- Abacus
+- AIML
+- Cerebras
+- DeepSeek
+- GitHub Models
+- GrokAI
+- Groq
+- Langdock
+- LiteLLM
+- MiniMax
+- Mistral
+- OpenRouter
+- SiliconCloud
+- Together
+- Venice AI
+- Z AI
+
+Run `fabric --setup` to configure your preferred provider(s), or use `fabric --listvendors` to see all available vendors.
+
+### Per-Pattern Model Mapping
+
+ You can configure specific models for individual patterns using environment variables
+ like `FABRIC_MODEL_PATTERN_NAME=vendor|model`
+
+ This makes it easy to maintain these per-pattern model mappings in your shell startup files.
+
 ### Add aliases for all patterns
 
-In order to add aliases for all your patterns and use them directly as commands ie. `summarize` instead of `fabric --pattern summarize`
-You can add the following to your `.zshrc` or `.bashrc` file.
+In order to add aliases for all your patterns and use them directly as commands, for example, `summarize` instead of `fabric --pattern summarize`
+You can add the following to your `.zshrc` or `.bashrc` file. You
+can also optionally set the `FABRIC_ALIAS_PREFIX` environment variable
+before, if you'd prefer all the fabric aliases to start with the same prefix.
 
 ```bash
 # Loop through all files in the ~/.config/fabric/patterns directory
 for pattern_file in $HOME/.config/fabric/patterns/*; do
     # Get the base name of the file (i.e., remove the directory path)
-    pattern_name=$(basename "$pattern_file")
+    pattern_name="$(basename "$pattern_file")"
+    alias_name="${FABRIC_ALIAS_PREFIX:-}${pattern_name}"
 
     # Create an alias in the form: alias pattern_name="fabric --pattern pattern_name"
-    alias_command="alias $pattern_name='fabric --pattern $pattern_name'"
+    alias_command="alias $alias_name='fabric --pattern $pattern_name'"
 
     # Evaluate the alias command to add it to the current shell
     eval "$alias_command"
@@ -288,11 +442,13 @@ You can add the below code for the equivalent aliases inside PowerShell by runni
 # Path to the patterns directory
 $patternsPath = Join-Path $HOME ".config/fabric/patterns"
 foreach ($patternDir in Get-ChildItem -Path $patternsPath -Directory) {
-    $patternName = $patternDir.Name
-
+    # Prepend FABRIC_ALIAS_PREFIX if set; otherwise use empty string
+    $prefix = $env:FABRIC_ALIAS_PREFIX ?? ''
+    $patternName = "$($patternDir.Name)"
+    $aliasName = "$prefix$patternName"
     # Dynamically define a function for each pattern
     $functionDefinition = @"
-function $patternName {
+function $aliasName {
     [CmdletBinding()]
     param(
         [Parameter(ValueFromPipeline = `$true)]
@@ -419,7 +575,7 @@ pipx uninstall fabric
 # Clear any old Fabric aliases
 (check your .bashrc, .zshrc, etc.)
 # Install the Go version
-go install github.com/danielmiessler/fabric@latest
+go install github.com/danielmiessler/fabric/cmd/fabric@latest
 # Run setup for the new version. Important because things have changed
 fabric --setup
 ```
@@ -431,7 +587,7 @@ Then [set your environmental variables](#environment-variables) as shown above.
 The great thing about Go is that it's super easy to upgrade. Just run the same command you used to install it in the first place and you'll always get the latest version.
 
 ```bash
-go install github.com/danielmiessler/fabric@latest
+go install github.com/danielmiessler/fabric/cmd/fabric@latest
 ```
 
 ### Shell Completions
@@ -440,6 +596,25 @@ Fabric provides shell completion scripts for Zsh, Bash, and Fish
 shells, making it easier to use the CLI by providing tab completion
 for commands and options.
 
+#### Quick install (no clone required)
+
+You can install completions directly via a one-liner:
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/danielmiessler/Fabric/refs/heads/main/completions/setup-completions.sh | sh
+```
+
+Optional variants:
+
+```bash
+# Dry-run (see actions without changing your system)
+curl -fsSL https://raw.githubusercontent.com/danielmiessler/Fabric/refs/heads/main/completions/setup-completions.sh | sh -s -- --dry-run
+
+# Override the download source (advanced)
+FABRIC_COMPLETIONS_BASE_URL="https://raw.githubusercontent.com/danielmiessler/Fabric/refs/heads/main/completions" \
+    sh -c "$(curl -fsSL https://raw.githubusercontent.com/danielmiessler/Fabric/refs/heads/main/completions/setup-completions.sh)"
+```
+
 #### Zsh Completion
 
 To enable Zsh completion:
@@ -485,7 +660,6 @@ fabric -h
 ```
 
 ```plaintext
-
 Usage:
   fabric [OPTIONS]
 
@@ -500,7 +674,10 @@ Application Options:
   -T, --topp=                       Set top P (default: 0.9)
   -s, --stream                      Stream
   -P, --presencepenalty=            Set presence penalty (default: 0.0)
-  -r, --raw                         Use the defaults of the model without sending chat options (like temperature etc.) and use the user role instead of the system role for patterns.
+  -r, --raw                         Use the defaults of the model without sending chat options
+                                    (temperature, top_p, etc.). Only affects OpenAI-compatible providers.
+                                    Anthropic models always use smart parameter selection to comply with
+                                    model-specific requirements.
   -F, --frequencypenalty=           Set frequency penalty (default: 0.0)
   -l, --listpatterns                List all patterns
   -L, --listmodels                  List all available models
@@ -509,14 +686,18 @@ Application Options:
   -U, --updatepatterns              Update patterns
   -c, --copy                        Copy to clipboard
   -m, --model=                      Choose model
+  -V, --vendor=                     Specify vendor for chosen model (e.g., -V "LM Studio" -m openai/gpt-oss-20b)
       --modelContextLength=         Model context length (only affects ollama)
   -o, --output=                     Output to file
       --output-session              Output the entire session (also a temporary one) to the output file
   -n, --latest=                     Number of latest patterns to list (default: 0)
   -d, --changeDefaultModel          Change default model
-  -y, --youtube=                    YouTube video or play list "URL" to grab transcript, comments from it and send to chat or print it put to the console and store it in the output file
+  -y, --youtube=                    YouTube video or play list "URL" to grab transcript, comments from it
+                                    and send to chat or print it put to the console and store it in the
+                                    output file
       --playlist                    Prefer playlist over video if both ids are present in the URL
-      --transcript                  Grab transcript from YouTube video and send to chat (it is used per default).
+      --transcript                  Grab transcript from YouTube video and send to chat (it is used per
+                                    default).
       --transcript-with-timestamps  Grab transcript from YouTube video with timestamps and send to chat
       --comments                    Grab comments from YouTube video and send to chat
       --metadata                    Output video metadata
@@ -530,6 +711,7 @@ Application Options:
       --printsession=               Print session
       --readability                 Convert HTML input into a clean, readable view
       --input-has-vars              Apply variables to user input
+      --no-variable-replacement     Disable pattern variable replacement
       --dry-run                     Show what would be sent to the model without actually sending it
       --serve                       Serve the Fabric Rest API
       --serveOllama                 Serve the Fabric Rest API with ollama endpoints
@@ -544,12 +726,93 @@ Application Options:
       --liststrategies              List all strategies
       --listvendors                 List all vendors
       --shell-complete-list         Output raw list without headers/formatting (for shell completion)
-
+      --search                      Enable web search tool for supported models (Anthropic, OpenAI, Gemini)
+      --search-location=            Set location for web search results (e.g., 'America/Los_Angeles')
+      --image-file=                 Save generated image to specified file path (e.g., 'output.png')
+      --image-size=                 Image dimensions: 1024x1024, 1536x1024, 1024x1536, auto (default: auto)
+      --image-quality=              Image quality: low, medium, high, auto (default: auto)
+      --image-compression=          Compression level 0-100 for JPEG/WebP formats (default: not set)
+      --image-background=           Background type: opaque, transparent (default: opaque, only for
+                                    PNG/WebP)
+      --suppress-think              Suppress text enclosed in thinking tags
+      --think-start-tag=            Start tag for thinking sections (default: <think>)
+      --think-end-tag=              End tag for thinking sections (default: </think>)
+      --disable-responses-api       Disable OpenAI Responses API (default: false)
+      --voice=                      TTS voice name for supported models (e.g., Kore, Charon, Puck)
+                                    (default: Kore)
+      --list-gemini-voices          List all available Gemini TTS voices
+      --notification                Send desktop notification when command completes
+      --notification-command=       Custom command to run for notifications (overrides built-in
+                                    notifications)
+      --yt-dlp-args=                Additional arguments to pass to yt-dlp (e.g. '--cookies-from-browser brave')
+      --thinking=                   Set reasoning/thinking level (e.g., off, low, medium, high, or
+                                    numeric tokens for Anthropic or Google Gemini)
+      --show-metadata               Print metadata (input/output tokens) to stderr
+      --debug=                     Set debug level (0: off, 1: basic, 2: detailed, 3: trace)
 Help Options:
   -h, --help                        Show this help message
-
 ```
 
+### Debug Levels
+
+Use the `--debug` flag to control runtime logging:
+
+- `0`: off (default)
+- `1`: basic debug info
+- `2`: detailed debugging
+- `3`: trace level
+
+### Dry Run Mode
+
+Use `--dry-run` to preview what would be sent to the AI model without making an API call:
+
+```bash
+echo "test input" | fabric --dry-run -p summarize
+```
+
+This is useful for debugging patterns, checking prompt construction, and verifying input formatting before using API credits.
+
+### Extensions
+
+Fabric supports extensions that can be called within patterns. See the [Extension Guide](internal/plugins/template/Examples/README.md) for complete documentation.
+
+**Important:** Extensions only work within pattern files, not via direct stdin. See the guide for details and examples.
+
+## REST API Server
+
+Fabric includes a built-in REST API server that exposes all core functionality over HTTP. Start the server with:
+
+```bash
+fabric --serve
+```
+
+The server provides endpoints for:
+
+- Chat completions with streaming responses
+- Pattern management (create, read, update, delete)
+- Context and session management
+- Model and vendor listing
+- YouTube transcript extraction
+- Configuration management
+
+For complete endpoint documentation, authentication setup, and usage examples, see [REST API Documentation](docs/rest-api.md).
+
+### Ollama Compatibility Mode
+
+Fabric can serve as a drop-in replacement for Ollama by exposing Ollama-compatible API endpoints. Start the server with:
+
+```bash
+fabric --serve --serveOllama
+```
+
+This enables the following Ollama-compatible endpoints:
+
+- `GET /api/tags` - List available patterns as models
+- `POST /api/chat` - Chat completions
+- `GET /api/version` - Server version
+
+Applications configured to use the Ollama API can point to your Fabric server instead, allowing you to use any of Fabric's supported AI providers through the Ollama interface. Patterns appear as models (e.g., `summarize:latest`).
+
 ## Our approach to prompting
 
 Fabric _Patterns_ are different than most prompts you'll see.
@@ -559,7 +822,7 @@ Fabric _Patterns_ are different than most prompts you'll see.
 Here's an example of a Fabric Pattern.
 
 ```bash
-https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/system.md
+https://github.com/danielmiessler/Fabric/blob/main/data/patterns/extract_wisdom/system.md
 ```
 
 <img width="1461" alt="pattern-example" src="https://github.com/danielmiessler/fabric/assets/50654/b910c551-9263-405f-9735-71ca69bbab6d">
@@ -607,7 +870,7 @@ Now let's look at some things you can do with Fabric.
 <br />
 <br />
 
-If you're not looking to do anything fancy, and you just want a lot of great prompts, you can navigate to the [`/patterns`](https://github.com/danielmiessler/fabric/tree/main/patterns) directory and start exploring!
+If you're not looking to do anything fancy, and you just want a lot of great prompts, you can navigate to the [`/patterns`](https://github.com/danielmiessler/fabric/tree/main/data/patterns) directory and start exploring!
 
 We hope that if you used nothing else from Fabric, the Patterns by themselves will make the project useful.
 
@@ -623,22 +886,87 @@ be used in addition to the basic patterns.
 See the [Thinking Faster by Writing Less](https://arxiv.org/pdf/2502.18600) paper and
 the [Thought Generation section of Learn Prompting](https://learnprompting.org/docs/advanced/thought_generation/introduction) for examples of prompt strategies.
 
-Each strategy is available as a small `json` file in the [`/strategies`](https://github.com/danielmiessler/fabric/tree/main/strategies) directory.
+Each strategy is available as a small `json` file in the [`/strategies`](https://github.com/danielmiessler/fabric/tree/main/data/strategies) directory.
 
 The prompt modification of the strategy is applied to the system prompt and passed on to the
 LLM in the chat session.
 
 Use `fabric -S` and select the option to install the strategies in your `~/.config/fabric` directory.
 
+#### Available Strategies
+
+Fabric includes several prompt strategies:
+
+- `cot` - Chain-of-Thought: Step-by-step reasoning
+- `cod` - Chain-of-Draft: Iterative drafting with minimal notes (5 words max per step)
+- `tot` - Tree-of-Thought: Generate multiple reasoning paths and select the best one
+- `aot` - Atom-of-Thought: Break problems into smallest independent atomic sub-problems
+- `ltm` - Least-to-Most: Solve problems from easiest to hardest sub-problems
+- `self-consistent` - Self-Consistency: Multiple reasoning paths with consensus
+- `self-refine` - Self-Refinement: Answer, critique, and refine
+- `reflexion` - Reflexion: Answer, critique briefly, and provide refined answer
+- `standard` - Standard: Direct answer without explanation
+
+Use the `--strategy` flag to apply a strategy:
+
+```bash
+echo "Analyze this code" | fabric --strategy cot -p analyze_code
+```
+
+List all available strategies with:
+
+```bash
+fabric --liststrategies
+```
+
+Strategies are stored as JSON files in `~/.config/fabric/strategies/`. See the default strategies for the format specification.
+
 ## Custom Patterns
 
 You may want to use Fabric to create your own custom Patterns—but not share them with others. No problem!
 
-Just make a directory in `~/.config/custompatterns/` (or wherever) and put your `.md` files in there.
+Fabric now supports a dedicated custom patterns directory that keeps your personal patterns separate from the built-in ones. This means your custom patterns won't be overwritten when you update Fabric's built-in patterns.
 
-When you're ready to use them, copy them into `~/.config/fabric/patterns/`
+### Setting Up Custom Patterns
 
-You can then use them like any other Patterns, but they won't be public unless you explicitly submit them as Pull Requests to the Fabric project. So don't worry—they're private to you.
+1. Run the Fabric setup:
+
+   ```bash
+   fabric --setup
+   ```
+
+2. Select the "Custom Patterns" option from the Tools menu and enter your desired directory path (e.g., `~/my-custom-patterns`)
+
+3. Fabric will automatically create the directory if it does not exist.
+
+### Using Custom Patterns
+
+1. Create your custom pattern directory structure:
+
+   ```bash
+   mkdir -p ~/my-custom-patterns/my-analyzer
+   ```
+
+2. Create your pattern file
+
+   ```bash
+   echo "You are an expert analyzer of ..." > ~/my-custom-patterns/my-analyzer/system.md
+   ```
+
+3. **Use your custom pattern:**
+
+   ```bash
+   fabric --pattern my-analyzer "analyze this text"
+   ```
+
+### How It Works
+
+- **Priority System**: Custom patterns take precedence over built-in patterns with the same name
+- **Seamless Integration**: Custom patterns appear in `fabric --listpatterns` alongside built-in ones
+- **Update Safe**: Your custom patterns are never affected by `fabric --updatepatterns`
+- **Private by Default**: Custom patterns remain private unless you explicitly share them
+
+Your custom patterns are completely private and won't be affected by Fabric updates!
 
 ## Helper Apps
 
@@ -667,25 +995,43 @@ This will create a PDF file named `output.pdf` in the current directory.
 To install `to_pdf`, install it the same way as you install Fabric, just with a different repo name.
 
 ```bash
-go install github.com/danielmiessler/fabric/plugins/tools/to_pdf@latest
+go install github.com/danielmiessler/fabric/cmd/to_pdf@latest
 ```
 
 Make sure you have a LaTeX distribution (like TeX Live or MiKTeX) installed on your system, as `to_pdf` requires `pdflatex` to be available in your system's PATH.
 
-### `code_helper`
+### `code2context`
 
-`code_helper` is used in conjunction with the `create_coding_feature` pattern.
+`code2context` is used in conjunction with the `create_coding_feature` pattern.
 It generates a `json` representation of a directory of code that can be fed into an AI model
 with instructions to create a new feature or edit the code in a specified way.
 
-See [the Create Coding Feature Pattern README](./patterns/create_coding_feature/README.md) for details.
+See [the Create Coding Feature Pattern README](./data/patterns/create_coding_feature/README.md) for details.
 
 Install it first using:
 
 ```bash
-go install github.com/danielmiessler/fabric/plugins/tools/code_helper@latest
+go install github.com/danielmiessler/fabric/cmd/code2context@latest
 ```
 
+### `generate_changelog`
+
+`generate_changelog` generates changelogs from git commit history and GitHub pull requests. It walks through your repository's git history, extracts PR information, and produces well-formatted markdown changelogs.
+
+```bash
+generate_changelog --help
+```
+
+Features include SQLite caching for fast incremental updates, GitHub GraphQL API integration for efficient PR fetching, and optional AI-enhanced summaries using Fabric.
+
+Install it using:
+
+```bash
+go install github.com/danielmiessler/fabric/cmd/generate_changelog@latest
+```
+
+See the [generate_changelog README](./cmd/generate_changelog/README.md) for detailed usage and options.
+
 ## pbpaste
 
 The [examples](#examples) use the macOS program `pbpaste` to paste content from the clipboard to pipe into `fabric` as the input. `pbpaste` is not available on Windows or Linux, but there are alternatives.
@@ -709,60 +1055,9 @@ You can also create an alias by editing `~/.bashrc` or `~/.zshrc` and adding the
 alias pbpaste='xclip -selection clipboard -o'
 ```
 
-## Web Interface
+## Web Interface (Fabric Web App)
 
-Fabric now includes a built-in web interface that provides a GUI alternative to the command-line interface and an out-of-the-box website for those who want to get started with web development or blogging.
-You can use this app as a GUI interface for Fabric, a ready to go blog-site, or a website template for your own projects.
-
-The `web/src/lib/content` directory includes starter `.obsidian/` and `templates/` directories, allowing you to open up the `web/src/lib/content/` directory as an [Obsidian.md](https://obsidian.md) vault. You can place your posts in the posts directory when you're ready to publish.
-
-### Installing
-
-The GUI can be installed by navigating to the `web` directory and using `npm install`, `pnpm install`, or your favorite package manager. Then simply run the development server to start the app.
-
-_You will need to run fabric in a separate terminal with the `fabric --serve` command._
-
-**From the fabric project `web/` directory:**
-
-```shell
-npm run dev
-
-## or ##
-
-pnpm run dev
-
-## or your equivalent
-```
-
-### Streamlit UI
-
-To run the Streamlit user interface:
-
-```bash
-# Install required dependencies
-pip install -r requirements.txt
-
-# Or manually install dependencies
-pip install streamlit pandas matplotlib seaborn numpy python-dotenv pyperclip
-
-# Run the Streamlit app
-streamlit run streamlit.py
-```
-
-The Streamlit UI provides a user-friendly interface for:
-
-- Running and chaining patterns
-- Managing pattern outputs
-- Creating and editing patterns
-- Analyzing pattern results
-
-#### Clipboard Support
-
-The Streamlit UI supports clipboard operations across different platforms:
-
-- **macOS**: Uses `pbcopy` and `pbpaste` (built-in)
-- **Windows**: Uses `pyperclip` library (install with `pip install pyperclip`)
-- **Linux**: Uses `xclip` (install with `sudo apt-get install xclip` or equivalent for your distro)
+Fabric now includes a built-in web interface that provides a GUI alternative to the command-line interface. Refer to [Web App README](/web/README.md) for installation instructions and an overview of features.
 
 ## Meta
 
@@ -780,15 +1075,15 @@ The Streamlit UI supports clipboard operations across different platforms:
 
 ### Primary contributors
 
-<a href="https://github.com/danielmiessler"><img src="https://avatars.githubusercontent.com/u/50654?v=4" title="Daniel Miessler" width="50" height="50"></a>
-<a href="https://github.com/xssdoctor"><img src="https://avatars.githubusercontent.com/u/9218431?v=4" title="Jonathan Dunn" width="50" height="50"></a>
-<a href="https://github.com/sbehrens"><img src="https://avatars.githubusercontent.com/u/688589?v=4" title="Scott Behrens" width="50" height="50"></a>
-<a href="https://github.com/agu3rra"><img src="https://avatars.githubusercontent.com/u/10410523?v=4" title="Andre Guerra" width="50" height="50"></a>
+<a href="https://github.com/danielmiessler"><img src="https://avatars.githubusercontent.com/u/50654?v=4" title="Daniel Miessler" width="50" height="50" alt="Daniel Miessler"></a>
+<a href="https://github.com/xssdoctor"><img src="https://avatars.githubusercontent.com/u/9218431?v=4" title="Jonathan Dunn" width="50" height="50" alt="Jonathan Dunn"></a>
+<a href="https://github.com/sbehrens"><img src="https://avatars.githubusercontent.com/u/688589?v=4" title="Scott Behrens" width="50" height="50" alt="Scott Behrens"></a>
+<a href="https://github.com/agu3rra"><img src="https://avatars.githubusercontent.com/u/10410523?v=4" title="Andre Guerra" width="50" height="50" alt="Andre Guerra"></a>
 
 ### Contributors
 
 <a href="https://github.com/danielmiessler/fabric/graphs/contributors">
-  <img src="https://contrib.rocks/image?repo=danielmiessler/fabric" />
+  <img src="https://contrib.rocks/image?repo=danielmiessler/fabric" alt="contrib.rocks" />
 </a>
 
 Made with [contrib.rocks](https://contrib.rocks).

cli/cli.go

@@ -1,357 +0,0 @@
-package cli
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"strconv"
-	"strings"
-
-	"github.com/danielmiessler/fabric/plugins/tools/youtube"
-
-	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/core"
-	"github.com/danielmiessler/fabric/plugins/ai"
-	"github.com/danielmiessler/fabric/plugins/db/fsdb"
-	"github.com/danielmiessler/fabric/plugins/tools/converter"
-	"github.com/danielmiessler/fabric/restapi"
-)
-
-// Cli Controls the cli. It takes in the flags and runs the appropriate functions
-func Cli(version string) (err error) {
-	var currentFlags *Flags
-	if currentFlags, err = Init(); err != nil {
-		return
-	}
-
-	if currentFlags.Version {
-		fmt.Println(version)
-		return
-	}
-
-	var homedir string
-	if homedir, err = os.UserHomeDir(); err != nil {
-		return
-	}
-
-	fabricDb := fsdb.NewDb(filepath.Join(homedir, ".config/fabric"))
-
-	if err = fabricDb.Configure(); err != nil {
-		if !currentFlags.Setup {
-			println(err.Error())
-			currentFlags.Setup = true
-		}
-	}
-
-	var registry *core.PluginRegistry
-	if registry, err = core.NewPluginRegistry(fabricDb); err != nil {
-		return
-	}
-
-	// if the setup flag is set, run the setup function
-	if currentFlags.Setup {
-		err = registry.Setup()
-		return
-	}
-
-	if currentFlags.Serve {
-		registry.ConfigureVendors()
-		err = restapi.Serve(registry, currentFlags.ServeAddress, currentFlags.ServeAPIKey)
-		return
-	}
-
-	if currentFlags.ServeOllama {
-		registry.ConfigureVendors()
-		err = restapi.ServeOllama(registry, currentFlags.ServeAddress, version)
-		return
-	}
-
-	if currentFlags.UpdatePatterns {
-		err = registry.PatternsLoader.PopulateDB()
-		return
-	}
-
-	if currentFlags.ChangeDefaultModel {
-		if err = registry.Defaults.Setup(); err != nil {
-			return
-		}
-		err = registry.SaveEnvFile()
-		return
-	}
-
-	if currentFlags.LatestPatterns != "0" {
-		var parsedToInt int
-		if parsedToInt, err = strconv.Atoi(currentFlags.LatestPatterns); err != nil {
-			return
-		}
-
-		if err = fabricDb.Patterns.PrintLatestPatterns(parsedToInt); err != nil {
-			return
-		}
-		return
-	}
-
-	if currentFlags.ListPatterns {
-		err = fabricDb.Patterns.ListNames(currentFlags.ShellCompleteOutput)
-		return
-	}
-
-	if currentFlags.ListAllModels {
-		var models *ai.VendorsModels
-		if models, err = registry.VendorManager.GetModels(); err != nil {
-			return
-		}
-		models.Print(currentFlags.ShellCompleteOutput)
-		return
-	}
-
-	if currentFlags.ListAllContexts {
-		err = fabricDb.Contexts.ListNames(currentFlags.ShellCompleteOutput)
-		return
-	}
-
-	if currentFlags.ListAllSessions {
-		err = fabricDb.Sessions.ListNames(currentFlags.ShellCompleteOutput)
-		return
-	}
-
-	if currentFlags.WipeContext != "" {
-		err = fabricDb.Contexts.Delete(currentFlags.WipeContext)
-		return
-	}
-
-	if currentFlags.WipeSession != "" {
-		err = fabricDb.Sessions.Delete(currentFlags.WipeSession)
-		return
-	}
-
-	if currentFlags.PrintSession != "" {
-		err = fabricDb.Sessions.PrintSession(currentFlags.PrintSession)
-		return
-	}
-
-	if currentFlags.PrintContext != "" {
-		err = fabricDb.Contexts.PrintContext(currentFlags.PrintContext)
-		return
-	}
-
-	if currentFlags.HtmlReadability {
-		if msg, cleanErr := converter.HtmlReadability(currentFlags.Message); cleanErr != nil {
-			fmt.Println("use original input, because can't apply html readability", err)
-		} else {
-			currentFlags.Message = msg
-		}
-	}
-
-	if currentFlags.ListExtensions {
-		err = registry.TemplateExtensions.ListExtensions()
-		return
-	}
-
-	if currentFlags.AddExtension != "" {
-		err = registry.TemplateExtensions.RegisterExtension(currentFlags.AddExtension)
-		return
-	}
-
-	if currentFlags.RemoveExtension != "" {
-		err = registry.TemplateExtensions.RemoveExtension(currentFlags.RemoveExtension)
-		return
-	}
-
-	if currentFlags.ListStrategies {
-		err = registry.Strategies.ListStrategies(currentFlags.ShellCompleteOutput)
-		return
-	}
-
-	if currentFlags.ListVendors {
-		err = registry.ListVendors(os.Stdout)
-		return
-	}
-
-	// if the interactive flag is set, run the interactive function
-	// if currentFlags.Interactive {
-	// 	interactive.Interactive()
-	// }
-
-	// if none of the above currentFlags are set, run the initiate chat function
-
-	var messageTools string
-
-	if currentFlags.YouTube != "" {
-		if !registry.YouTube.IsConfigured() {
-			err = fmt.Errorf("YouTube is not configured, please run the setup procedure")
-			return
-		}
-
-		var videoId string
-		var playlistId string
-		if videoId, playlistId, err = registry.YouTube.GetVideoOrPlaylistId(currentFlags.YouTube); err != nil {
-			return
-		} else if (videoId == "" || currentFlags.YouTubePlaylist) && playlistId != "" {
-			if currentFlags.Output != "" {
-				err = registry.YouTube.FetchAndSavePlaylist(playlistId, currentFlags.Output)
-			} else {
-				var videos []*youtube.VideoMeta
-				if videos, err = registry.YouTube.FetchPlaylistVideos(playlistId); err != nil {
-					err = fmt.Errorf("error fetching playlist videos: %v", err)
-					return
-				}
-
-				for _, video := range videos {
-					var message string
-					if message, err = processYoutubeVideo(currentFlags, registry, video.Id); err != nil {
-						return
-					}
-
-					if !currentFlags.IsChatRequest() {
-						if err = WriteOutput(message, fmt.Sprintf("%v.md", video.TitleNormalized)); err != nil {
-							return
-						}
-					} else {
-						messageTools = AppendMessage(messageTools, message)
-					}
-				}
-			}
-			return
-		}
-
-		if messageTools, err = processYoutubeVideo(currentFlags, registry, videoId); err != nil {
-			return
-		}
-		if !currentFlags.IsChatRequest() {
-			err = currentFlags.WriteOutput(messageTools)
-			return
-		}
-	}
-
-	if (currentFlags.ScrapeURL != "" || currentFlags.ScrapeQuestion != "") && registry.Jina.IsConfigured() {
-		// Check if the scrape_url flag is set and call ScrapeURL
-		if currentFlags.ScrapeURL != "" {
-			var website string
-			if website, err = registry.Jina.ScrapeURL(currentFlags.ScrapeURL); err != nil {
-				return
-			}
-			messageTools = AppendMessage(messageTools, website)
-		}
-
-		// Check if the scrape_question flag is set and call ScrapeQuestion
-		if currentFlags.ScrapeQuestion != "" {
-			var website string
-			if website, err = registry.Jina.ScrapeQuestion(currentFlags.ScrapeQuestion); err != nil {
-				return
-			}
-
-			messageTools = AppendMessage(messageTools, website)
-		}
-
-		if !currentFlags.IsChatRequest() {
-			err = currentFlags.WriteOutput(messageTools)
-			return
-		}
-	}
-
-	if messageTools != "" {
-		currentFlags.AppendMessage(messageTools)
-	}
-
-	var chatter *core.Chatter
-	if chatter, err = registry.GetChatter(currentFlags.Model, currentFlags.ModelContextLength,
-		currentFlags.Strategy, currentFlags.Stream, currentFlags.DryRun); err != nil {
-		return
-	}
-
-	var session *fsdb.Session
-	var chatReq *common.ChatRequest
-	if chatReq, err = currentFlags.BuildChatRequest(strings.Join(os.Args[1:], " ")); err != nil {
-		return
-	}
-
-	if chatReq.Language == "" {
-		chatReq.Language = registry.Language.DefaultLanguage.Value
-	}
-	if session, err = chatter.Send(chatReq, currentFlags.BuildChatOptions()); err != nil {
-		return
-	}
-
-	result := session.GetLastMessage().Content
-
-	if !currentFlags.Stream {
-		// print the result if it was not streamed already
-		fmt.Println(result)
-	}
-
-	// if the copy flag is set, copy the message to the clipboard
-	if currentFlags.Copy {
-		if err = CopyToClipboard(result); err != nil {
-			return
-		}
-	}
-
-	// if the output flag is set, create an output file
-	if currentFlags.Output != "" {
-		if currentFlags.OutputSession {
-			sessionAsString := session.String()
-			err = CreateOutputFile(sessionAsString, currentFlags.Output)
-		} else {
-			err = CreateOutputFile(result, currentFlags.Output)
-		}
-	}
-	return
-}
-
-func processYoutubeVideo(
-	flags *Flags, registry *core.PluginRegistry, videoId string) (message string, err error) {
-
-	if (!flags.YouTubeComments && !flags.YouTubeMetadata) || flags.YouTubeTranscript || flags.YouTubeTranscriptWithTimestamps {
-		var transcript string
-		var language = "en"
-		if flags.Language != "" || registry.Language.DefaultLanguage.Value != "" {
-			if flags.Language != "" {
-				language = flags.Language
-			} else {
-				language = registry.Language.DefaultLanguage.Value
-			}
-		}
-		if flags.YouTubeTranscriptWithTimestamps {
-			if transcript, err = registry.YouTube.GrabTranscriptWithTimestamps(videoId, language); err != nil {
-				return
-			}
-		} else {
-			if transcript, err = registry.YouTube.GrabTranscript(videoId, language); err != nil {
-				return
-			}
-		}
-		message = AppendMessage(message, transcript)
-	}
-
-	if flags.YouTubeComments {
-		var comments []string
-		if comments, err = registry.YouTube.GrabComments(videoId); err != nil {
-			return
-		}
-
-		commentsString := strings.Join(comments, "\n")
-
-		message = AppendMessage(message, commentsString)
-	}
-
-	if flags.YouTubeMetadata {
-		var metadata *youtube.VideoMetadata
-		if metadata, err = registry.YouTube.GrabMetadata(videoId); err != nil {
-			return
-		}
-		metadataJson, _ := json.MarshalIndent(metadata, "", "  ")
-		message = AppendMessage(message, string(metadataJson))
-	}
-
-	return
-}
-
-func WriteOutput(message string, outputFile string) (err error) {
-	fmt.Println(message)
-	if outputFile != "" {
-		err = CreateOutputFile(message, outputFile)
-	}
-	return
-}

cli/flags.go

@@ -1,361 +0,0 @@
-package cli
-
-import (
-	"bufio"
-	"errors"
-	"fmt"
-	"io"
-	"os"
-	"reflect"
-	"strconv"
-	"strings"
-
-	"github.com/danielmiessler/fabric/common"
-	"github.com/jessevdk/go-flags"
-	goopenai "github.com/sashabaranov/go-openai"
-	"golang.org/x/text/language"
-	"gopkg.in/yaml.v2"
-)
-
-// Flags create flags struct. the users flags go into this, this will be passed to the chat struct in cli
-type Flags struct {
-	Pattern                         string            `short:"p" long:"pattern" yaml:"pattern" description:"Choose a pattern from the available patterns" default:""`
-	PatternVariables                map[string]string `short:"v" long:"variable" description:"Values for pattern variables, e.g. -v=#role:expert -v=#points:30"`
-	Context                         string            `short:"C" long:"context" description:"Choose a context from the available contexts" default:""`
-	Session                         string            `long:"session" description:"Choose a session from the available sessions"`
-	Attachments                     []string          `short:"a" long:"attachment" description:"Attachment path or URL (e.g. for OpenAI image recognition messages)"`
-	Setup                           bool              `short:"S" long:"setup" description:"Run setup for all reconfigurable parts of fabric"`
-	Temperature                     float64           `short:"t" long:"temperature" yaml:"temperature" description:"Set temperature" default:"0.7"`
-	TopP                            float64           `short:"T" long:"topp" yaml:"topp" description:"Set top P" default:"0.9"`
-	Stream                          bool              `short:"s" long:"stream" yaml:"stream" description:"Stream"`
-	PresencePenalty                 float64           `short:"P" long:"presencepenalty" yaml:"presencepenalty" description:"Set presence penalty" default:"0.0"`
-	Raw                             bool              `short:"r" long:"raw" yaml:"raw" description:"Use the defaults of the model without sending chat options (like temperature etc.) and use the user role instead of the system role for patterns."`
-	FrequencyPenalty                float64           `short:"F" long:"frequencypenalty" yaml:"frequencypenalty" description:"Set frequency penalty" default:"0.0"`
-	ListPatterns                    bool              `short:"l" long:"listpatterns" description:"List all patterns"`
-	ListAllModels                   bool              `short:"L" long:"listmodels" description:"List all available models"`
-	ListAllContexts                 bool              `short:"x" long:"listcontexts" description:"List all contexts"`
-	ListAllSessions                 bool              `short:"X" long:"listsessions" description:"List all sessions"`
-	UpdatePatterns                  bool              `short:"U" long:"updatepatterns" description:"Update patterns"`
-	Message                         string            `hidden:"true" description:"Messages to send to chat"`
-	Copy                            bool              `short:"c" long:"copy" description:"Copy to clipboard"`
-	Model                           string            `short:"m" long:"model" yaml:"model" description:"Choose model"`
-	ModelContextLength              int               `long:"modelContextLength" yaml:"modelContextLength" description:"Model context length (only affects ollama)"`
-	Output                          string            `short:"o" long:"output" description:"Output to file" default:""`
-	OutputSession                   bool              `long:"output-session" description:"Output the entire session (also a temporary one) to the output file"`
-	LatestPatterns                  string            `short:"n" long:"latest" description:"Number of latest patterns to list" default:"0"`
-	ChangeDefaultModel              bool              `short:"d" long:"changeDefaultModel" description:"Change default model"`
-	YouTube                         string            `short:"y" long:"youtube" description:"YouTube video or play list \"URL\" to grab transcript, comments from it and send to chat or print it put to the console and store it in the output file"`
-	YouTubePlaylist                 bool              `long:"playlist" description:"Prefer playlist over video if both ids are present in the URL"`
-	YouTubeTranscript               bool              `long:"transcript" description:"Grab transcript from YouTube video and send to chat (it is used per default)."`
-	YouTubeTranscriptWithTimestamps bool              `long:"transcript-with-timestamps" description:"Grab transcript from YouTube video with timestamps and send to chat"`
-	YouTubeComments                 bool              `long:"comments" description:"Grab comments from YouTube video and send to chat"`
-	YouTubeMetadata                 bool              `long:"metadata" description:"Output video metadata"`
-	Language                        string            `short:"g" long:"language" description:"Specify the Language Code for the chat, e.g. -g=en -g=zh" default:""`
-	ScrapeURL                       string            `short:"u" long:"scrape_url" description:"Scrape website URL to markdown using Jina AI"`
-	ScrapeQuestion                  string            `short:"q" long:"scrape_question" description:"Search question using Jina AI"`
-	Seed                            int               `short:"e" long:"seed" yaml:"seed" description:"Seed to be used for LMM generation"`
-	WipeContext                     string            `short:"w" long:"wipecontext" description:"Wipe context"`
-	WipeSession                     string            `short:"W" long:"wipesession" description:"Wipe session"`
-	PrintContext                    string            `long:"printcontext" description:"Print context"`
-	PrintSession                    string            `long:"printsession" description:"Print session"`
-	HtmlReadability                 bool              `long:"readability" description:"Convert HTML input into a clean, readable view"`
-	InputHasVars                    bool              `long:"input-has-vars" description:"Apply variables to user input"`
-	DryRun                          bool              `long:"dry-run" description:"Show what would be sent to the model without actually sending it"`
-	Serve                           bool              `long:"serve" description:"Serve the Fabric Rest API"`
-	ServeOllama                     bool              `long:"serveOllama" description:"Serve the Fabric Rest API with ollama endpoints"`
-	ServeAddress                    string            `long:"address" description:"The address to bind the REST API" default:":8080"`
-	ServeAPIKey                     string            `long:"api-key" description:"API key used to secure server routes" default:""`
-	Config                          string            `long:"config" description:"Path to YAML config file"`
-	Version                         bool              `long:"version" description:"Print current version"`
-	ListExtensions                  bool              `long:"listextensions" description:"List all registered extensions"`
-	AddExtension                    string            `long:"addextension" description:"Register a new extension from config file path"`
-	RemoveExtension                 string            `long:"rmextension" description:"Remove a registered extension by name"`
-	Strategy                        string            `long:"strategy" description:"Choose a strategy from the available strategies" default:""`
-	ListStrategies                  bool              `long:"liststrategies" description:"List all strategies"`
-	ListVendors                     bool              `long:"listvendors" description:"List all vendors"`
-	ShellCompleteOutput             bool              `long:"shell-complete-list" description:"Output raw list without headers/formatting (for shell completion)"`
-}
-
-var debug = false
-
-func Debugf(format string, a ...interface{}) {
-	if debug {
-		fmt.Printf("DEBUG: "+format, a...)
-	}
-}
-
-// Init Initialize flags. returns a Flags struct and an error
-func Init() (ret *Flags, err error) {
-	// Track which yaml-configured flags were set on CLI
-	usedFlags := make(map[string]bool)
-	yamlArgsScan := os.Args[1:]
-
-	// Get list of fields that have yaml tags, could be in yaml config
-	yamlFields := make(map[string]bool)
-	t := reflect.TypeOf(Flags{})
-	for i := 0; i < t.NumField(); i++ {
-		if yamlTag := t.Field(i).Tag.Get("yaml"); yamlTag != "" {
-			yamlFields[yamlTag] = true
-			//Debugf("Found yaml-configured field: %s\n", yamlTag)
-		}
-	}
-
-	// Scan args for that are provided by cli and might be in yaml
-	for _, arg := range yamlArgsScan {
-		if strings.HasPrefix(arg, "--") {
-			flag := strings.TrimPrefix(arg, "--")
-			if i := strings.Index(flag, "="); i > 0 {
-				flag = flag[:i]
-			}
-			if yamlFields[flag] {
-				usedFlags[flag] = true
-				Debugf("CLI flag used: %s\n", flag)
-			}
-		}
-	}
-
-	// Parse CLI flags first
-	ret = &Flags{}
-	parser := flags.NewParser(ret, flags.Default)
-	var args []string
-	if args, err = parser.Parse(); err != nil {
-		return
-	}
-
-	// If config specified, load and apply YAML for unused flags
-	if ret.Config != "" {
-		var yamlFlags *Flags
-		if yamlFlags, err = loadYAMLConfig(ret.Config); err != nil {
-			return
-		}
-
-		// Apply YAML values where CLI flags weren't used
-		flagsVal := reflect.ValueOf(ret).Elem()
-		yamlVal := reflect.ValueOf(yamlFlags).Elem()
-		flagsType := flagsVal.Type()
-
-		for i := 0; i < flagsType.NumField(); i++ {
-			field := flagsType.Field(i)
-			if yamlTag := field.Tag.Get("yaml"); yamlTag != "" {
-				if !usedFlags[yamlTag] {
-					flagField := flagsVal.Field(i)
-					yamlField := yamlVal.Field(i)
-					if flagField.CanSet() {
-						if yamlField.Type() != flagField.Type() {
-							if err := assignWithConversion(flagField, yamlField); err != nil {
-								Debugf("Type conversion failed for %s: %v\n", yamlTag, err)
-								continue
-							}
-						} else {
-							flagField.Set(yamlField)
-						}
-						Debugf("Applied YAML value for %s: %v\n", yamlTag, yamlField.Interface())
-					}
-				}
-			}
-		}
-	}
-
-	// Handle stdin and messages
-	// Handle stdin and messages
-	info, _ := os.Stdin.Stat()
-	pipedToStdin := (info.Mode() & os.ModeCharDevice) == 0
-
-	// Append positional arguments to the message (custom message)
-	if len(args) > 0 {
-		ret.Message = AppendMessage(ret.Message, args[len(args)-1])
-	}
-
-	if pipedToStdin {
-		var pipedMessage string
-		if pipedMessage, err = readStdin(); err != nil {
-			return
-		}
-		ret.Message = AppendMessage(ret.Message, pipedMessage)
-	}
-	return
-}
-
-func assignWithConversion(targetField, sourceField reflect.Value) error {
-	// Handle string source values
-	if sourceField.Kind() == reflect.String {
-		str := sourceField.String()
-		switch targetField.Kind() {
-		case reflect.Int:
-			// Try parsing as float first to handle "42.9" -> 42
-			if val, err := strconv.ParseFloat(str, 64); err == nil {
-				targetField.SetInt(int64(val))
-				return nil
-			}
-			// Try direct int parse
-			if val, err := strconv.ParseInt(str, 10, 64); err == nil {
-				targetField.SetInt(val)
-				return nil
-			}
-		case reflect.Float64:
-			if val, err := strconv.ParseFloat(str, 64); err == nil {
-				targetField.SetFloat(val)
-				return nil
-			}
-		case reflect.Bool:
-			if val, err := strconv.ParseBool(str); err == nil {
-				targetField.SetBool(val)
-				return nil
-			}
-		}
-		return fmt.Errorf("cannot convert string %q to %v", str, targetField.Kind())
-	}
-
-	return fmt.Errorf("unsupported conversion from %v to %v", sourceField.Kind(), targetField.Kind())
-}
-
-func loadYAMLConfig(configPath string) (*Flags, error) {
-	absPath, err := common.GetAbsolutePath(configPath)
-	if err != nil {
-		return nil, fmt.Errorf("invalid config path: %w", err)
-	}
-
-	data, err := os.ReadFile(absPath)
-	if err != nil {
-		if os.IsNotExist(err) {
-			return nil, fmt.Errorf("config file not found: %s", absPath)
-		}
-		return nil, fmt.Errorf("error reading config file: %w", err)
-	}
-
-	// Use the existing Flags struct for YAML unmarshal
-	config := &Flags{}
-	if err := yaml.Unmarshal(data, config); err != nil {
-		return nil, fmt.Errorf("error parsing config file: %w", err)
-	}
-
-	Debugf("Config: %v\n", config)
-
-	return config, nil
-}
-
-// readStdin reads from stdin and returns the input as a string or an error
-func readStdin() (ret string, err error) {
-	reader := bufio.NewReader(os.Stdin)
-	var sb strings.Builder
-	for {
-		if line, readErr := reader.ReadString('\n'); readErr != nil {
-			if errors.Is(readErr, io.EOF) {
-				sb.WriteString(line)
-				break
-			}
-			err = fmt.Errorf("error reading piped message from stdin: %w", readErr)
-			return
-		} else {
-			sb.WriteString(line)
-		}
-	}
-	ret = sb.String()
-	return
-}
-
-func (o *Flags) BuildChatOptions() (ret *common.ChatOptions) {
-	ret = &common.ChatOptions{
-		Temperature:        o.Temperature,
-		TopP:               o.TopP,
-		PresencePenalty:    o.PresencePenalty,
-		FrequencyPenalty:   o.FrequencyPenalty,
-		Raw:                o.Raw,
-		Seed:               o.Seed,
-		ModelContextLength: o.ModelContextLength,
-	}
-	return
-}
-
-func (o *Flags) BuildChatRequest(Meta string) (ret *common.ChatRequest, err error) {
-	ret = &common.ChatRequest{
-		ContextName:      o.Context,
-		SessionName:      o.Session,
-		PatternName:      o.Pattern,
-		StrategyName:     o.Strategy,
-		PatternVariables: o.PatternVariables,
-		InputHasVars:     o.InputHasVars,
-		Meta:             Meta,
-	}
-
-	var message *goopenai.ChatCompletionMessage
-	if len(o.Attachments) == 0 {
-		if o.Message != "" {
-			message = &goopenai.ChatCompletionMessage{
-				Role:    goopenai.ChatMessageRoleUser,
-				Content: strings.TrimSpace(o.Message),
-			}
-		}
-	} else {
-		message = &goopenai.ChatCompletionMessage{
-			Role: goopenai.ChatMessageRoleUser,
-		}
-
-		if o.Message != "" {
-			message.MultiContent = append(message.MultiContent, goopenai.ChatMessagePart{
-				Type: goopenai.ChatMessagePartTypeText,
-				Text: strings.TrimSpace(o.Message),
-			})
-		}
-
-		for _, attachmentValue := range o.Attachments {
-			var attachment *common.Attachment
-			if attachment, err = common.NewAttachment(attachmentValue); err != nil {
-				return
-			}
-			url := attachment.URL
-			if url == nil {
-				var base64Image string
-				if base64Image, err = attachment.Base64Content(); err != nil {
-					return
-				}
-				var mimeType string
-				if mimeType, err = attachment.ResolveType(); err != nil {
-					return
-				}
-				dataURL := fmt.Sprintf("data:%s;base64,%s", mimeType, base64Image)
-				url = &dataURL
-			}
-			message.MultiContent = append(message.MultiContent, goopenai.ChatMessagePart{
-				Type: goopenai.ChatMessagePartTypeImageURL,
-				ImageURL: &goopenai.ChatMessageImageURL{
-					URL: *url,
-				},
-			})
-		}
-	}
-	ret.Message = message
-
-	if o.Language != "" {
-		if langTag, langErr := language.Parse(o.Language); langErr == nil {
-			ret.Language = langTag.String()
-		}
-	}
-	return
-}
-
-func (o *Flags) AppendMessage(message string) {
-	o.Message = AppendMessage(o.Message, message)
-}
-
-func (o *Flags) IsChatRequest() (ret bool) {
-	ret = o.Message != "" || len(o.Attachments) > 0 || o.Context != "" || o.Session != "" || o.Pattern != ""
-	return
-}
-
-func (o *Flags) WriteOutput(message string) (err error) {
-	fmt.Println(message)
-	if o.Output != "" {
-		err = CreateOutputFile(message, o.Output)
-	}
-	return
-}
-
-func AppendMessage(message string, newMessage string) (ret string) {
-	if message != "" {
-		ret = message + "\n" + newMessage
-	} else {
-		ret = newMessage
-	}
-	return
-}

cli/flags_test.go

@@ -1,166 +0,0 @@
-package cli
-
-import (
-	"bytes"
-	"io"
-	"os"
-	"strings"
-	"testing"
-
-	"github.com/danielmiessler/fabric/common"
-	"github.com/stretchr/testify/assert"
-)
-
-func TestInit(t *testing.T) {
-	args := []string{"--copy"}
-	expectedFlags := &Flags{Copy: true}
-	oldArgs := os.Args
-	defer func() { os.Args = oldArgs }()
-	os.Args = append([]string{"cmd"}, args...)
-
-	flags, err := Init()
-	assert.NoError(t, err)
-	assert.Equal(t, expectedFlags.Copy, flags.Copy)
-}
-
-func TestReadStdin(t *testing.T) {
-	input := "test input"
-	stdin := io.NopCloser(strings.NewReader(input))
-	// No need to cast stdin to *os.File, pass it as io.ReadCloser directly
-	content, err := ReadStdin(stdin)
-	if err != nil {
-		t.Fatalf("unexpected error: %v", err)
-	}
-	if content != input {
-		t.Fatalf("expected %q, got %q", input, content)
-	}
-}
-
-// ReadStdin function assuming it's part of `cli` package
-func ReadStdin(reader io.ReadCloser) (string, error) {
-	defer reader.Close()
-	buf := new(bytes.Buffer)
-	_, err := buf.ReadFrom(reader)
-	if err != nil {
-		return "", err
-	}
-	return buf.String(), nil
-}
-
-func TestBuildChatOptions(t *testing.T) {
-	flags := &Flags{
-		Temperature:      0.8,
-		TopP:             0.9,
-		PresencePenalty:  0.1,
-		FrequencyPenalty: 0.2,
-		Seed:             1,
-	}
-
-	expectedOptions := &common.ChatOptions{
-		Temperature:      0.8,
-		TopP:             0.9,
-		PresencePenalty:  0.1,
-		FrequencyPenalty: 0.2,
-		Raw:              false,
-		Seed:             1,
-	}
-	options := flags.BuildChatOptions()
-	assert.Equal(t, expectedOptions, options)
-}
-
-func TestBuildChatOptionsDefaultSeed(t *testing.T) {
-	flags := &Flags{
-		Temperature:      0.8,
-		TopP:             0.9,
-		PresencePenalty:  0.1,
-		FrequencyPenalty: 0.2,
-	}
-
-	expectedOptions := &common.ChatOptions{
-		Temperature:      0.8,
-		TopP:             0.9,
-		PresencePenalty:  0.1,
-		FrequencyPenalty: 0.2,
-		Raw:              false,
-		Seed:             0,
-	}
-	options := flags.BuildChatOptions()
-	assert.Equal(t, expectedOptions, options)
-}
-
-func TestInitWithYAMLConfig(t *testing.T) {
-	// Create a temporary YAML config file
-	configContent := `
-temperature: 0.9
-model: gpt-4
-pattern: analyze
-stream: true
-`
-	tmpfile, err := os.CreateTemp("", "config.*.yaml")
-	if err != nil {
-		t.Fatal(err)
-	}
-	defer os.Remove(tmpfile.Name())
-
-	if _, err := tmpfile.Write([]byte(configContent)); err != nil {
-		t.Fatal(err)
-	}
-	if err := tmpfile.Close(); err != nil {
-		t.Fatal(err)
-	}
-
-	// Test 1: Basic YAML loading
-	t.Run("Load YAML config", func(t *testing.T) {
-		oldArgs := os.Args
-		defer func() { os.Args = oldArgs }()
-		os.Args = []string{"cmd", "--config", tmpfile.Name()}
-
-		flags, err := Init()
-		assert.NoError(t, err)
-		assert.Equal(t, 0.9, flags.Temperature)
-		assert.Equal(t, "gpt-4", flags.Model)
-		assert.Equal(t, "analyze", flags.Pattern)
-		assert.True(t, flags.Stream)
-	})
-
-	// Test 2: CLI overrides YAML
-	t.Run("CLI overrides YAML", func(t *testing.T) {
-		oldArgs := os.Args
-		defer func() { os.Args = oldArgs }()
-		os.Args = []string{"cmd", "--config", tmpfile.Name(), "--temperature", "0.7", "--model", "gpt-3.5-turbo"}
-
-		flags, err := Init()
-		assert.NoError(t, err)
-		assert.Equal(t, 0.7, flags.Temperature)
-		assert.Equal(t, "gpt-3.5-turbo", flags.Model)
-		assert.Equal(t, "analyze", flags.Pattern) // unchanged from YAML
-		assert.True(t, flags.Stream)              // unchanged from YAML
-	})
-
-	// Test 3: Invalid YAML config
-	t.Run("Invalid YAML config", func(t *testing.T) {
-		badConfig := `
-temperature: "not a float"
-model: 123  # should be string
-`
-		badfile, err := os.CreateTemp("", "bad-config.*.yaml")
-		if err != nil {
-			t.Fatal(err)
-		}
-		defer os.Remove(badfile.Name())
-
-		if _, err := badfile.Write([]byte(badConfig)); err != nil {
-			t.Fatal(err)
-		}
-		if err := badfile.Close(); err != nil {
-			t.Fatal(err)
-		}
-
-		oldArgs := os.Args
-		defer func() { os.Args = oldArgs }()
-		os.Args = []string{"cmd", "--config", badfile.Name()}
-
-		_, err = Init()
-		assert.Error(t, err)
-	})
-}

cli/output.go

@@ -1,30 +0,0 @@
-package cli
-
-import (
-	"fmt"
-	"os"
-
-	"github.com/atotto/clipboard"
-)
-
-func CopyToClipboard(message string) (err error) {
-	if err = clipboard.WriteAll(message); err != nil {
-		err = fmt.Errorf("could not copy to clipboard: %v", err)
-	}
-	return
-}
-
-func CreateOutputFile(message string, fileName string) (err error) {
-	var file *os.File
-	if file, err = os.Create(fileName); err != nil {
-		err = fmt.Errorf("error creating file: %v", err)
-		return
-	}
-	defer file.Close()
-	if _, err = file.WriteString(message); err != nil {
-		err = fmt.Errorf("error writing to file: %v", err)
-	} else {
-		fmt.Printf("\n\n... written to %s\n", fileName)
-	}
-	return
-}

cli/output_test.go

@@ -1,28 +0,0 @@
-package cli
-
-import (
-	"os"
-	"testing"
-)
-
-func TestCopyToClipboard(t *testing.T) {
-	t.Skip("skipping test, because of docker env. in ci.")
-
-	message := "test message"
-	err := CopyToClipboard(message)
-	if err != nil {
-		t.Fatalf("CopyToClipboard() error = %v", err)
-	}
-}
-
-func TestCreateOutputFile(t *testing.T) {
-
-	fileName := "test_output.txt"
-	message := "test message"
-	err := CreateOutputFile(message, fileName)
-	if err != nil {
-		t.Fatalf("CreateOutputFile() error = %v", err)
-	}
-
-	defer os.Remove(fileName)
-}

cmd/code2context/code.go

@@ -109,11 +109,11 @@ func ScanDirectory(rootDir string, maxDepth int, instructions string, ignoreList
 	}
 
 	// Create final data structure
-	var data []interface{}
+	var data []any
 	data = append(data, rootItem)
 
 	// Add report
-	reportItem := map[string]interface{}{
+	reportItem := map[string]any{
 		"type":        "report",
 		"directories": dirCount,
 		"files":       fileCount,
@@ -121,7 +121,76 @@ func ScanDirectory(rootDir string, maxDepth int, instructions string, ignoreList
 	data = append(data, reportItem)
 
 	// Add instructions
-	instructionsItem := map[string]interface{}{
+	instructionsItem := map[string]any{
+		"type":    "instructions",
+		"name":    "code_change_instructions",
+		"details": instructions,
+	}
+	data = append(data, instructionsItem)
+
+	return json.MarshalIndent(data, "", "  ")
+}
+
+// ScanFiles scans specific files and returns a JSON representation
+func ScanFiles(files []string, instructions string) ([]byte, error) {
+	fileCount := 0
+	dirSet := make(map[string]bool)
+
+	// Create root directory item
+	rootItem := FileItem{
+		Type:     "directory",
+		Name:     ".",
+		Contents: []FileItem{},
+	}
+
+	for _, filePath := range files {
+		// Skip directories
+		info, err := os.Stat(filePath)
+		if err != nil {
+			return nil, fmt.Errorf("error accessing file %s: %v", filePath, err)
+		}
+		if info.IsDir() {
+			continue
+		}
+
+		// Track unique directories
+		dir := filepath.Dir(filePath)
+		if dir != "." {
+			dirSet[dir] = true
+		}
+
+		fileCount++
+
+		// Read file content
+		content, err := os.ReadFile(filePath)
+		if err != nil {
+			return nil, fmt.Errorf("error reading file %s: %v", filePath, err)
+		}
+
+		// Clean path for consistent handling
+		cleanPath := filepath.Clean(filePath)
+		if strings.HasPrefix(cleanPath, "./") {
+			cleanPath = cleanPath[2:]
+		}
+
+		// Add file to the structure
+		addFileToDirectory(&rootItem, cleanPath, string(content), ".")
+	}
+
+	// Create final data structure
+	var data []any
+	data = append(data, rootItem)
+
+	// Add report
+	reportItem := map[string]any{
+		"type":        "report",
+		"directories": len(dirSet) + 1,
+		"files":       fileCount,
+	}
+	data = append(data, reportItem)
+
+	// Add instructions
+	instructionsItem := map[string]any{
 		"type":    "instructions",
 		"name":    "code_change_instructions",
 		"details": instructions,

cmd/code2context/code_test.go

@@ -0,0 +1,100 @@
+package main
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestScanFiles(t *testing.T) {
+	// Create temp directory with test files
+	tmpDir := t.TempDir()
+
+	// Create test files
+	file1 := filepath.Join(tmpDir, "test1.go")
+	file2 := filepath.Join(tmpDir, "test2.go")
+	subDir := filepath.Join(tmpDir, "subdir")
+	file3 := filepath.Join(subDir, "test3.go")
+
+	require.NoError(t, os.WriteFile(file1, []byte("package main\n"), 0644))
+	require.NoError(t, os.WriteFile(file2, []byte("package main\n\nfunc main() {}\n"), 0644))
+	require.NoError(t, os.MkdirAll(subDir, 0755))
+	require.NoError(t, os.WriteFile(file3, []byte("package subdir\n"), 0644))
+
+	// Test scanning specific files
+	files := []string{file1, file3}
+	instructions := "Test instructions"
+
+	jsonData, err := ScanFiles(files, instructions)
+	require.NoError(t, err)
+
+	// Parse the JSON output
+	var result []any
+	err = json.Unmarshal(jsonData, &result)
+	require.NoError(t, err)
+	assert.Len(t, result, 3) // directory, report, instructions
+
+	// Check report
+	report := result[1].(map[string]any)
+	assert.Equal(t, "report", report["type"])
+	assert.Equal(t, float64(2), report["files"])
+
+	// Check instructions
+	instr := result[2].(map[string]any)
+	assert.Equal(t, "instructions", instr["type"])
+	assert.Equal(t, "Test instructions", instr["details"])
+}
+
+func TestScanFilesSkipsDirectories(t *testing.T) {
+	tmpDir := t.TempDir()
+
+	file1 := filepath.Join(tmpDir, "test.go")
+	subDir := filepath.Join(tmpDir, "subdir")
+
+	require.NoError(t, os.WriteFile(file1, []byte("package main\n"), 0644))
+	require.NoError(t, os.MkdirAll(subDir, 0755))
+
+	// Include a directory in the file list - should be skipped
+	files := []string{file1, subDir}
+
+	jsonData, err := ScanFiles(files, "test")
+	require.NoError(t, err)
+
+	var result []any
+	err = json.Unmarshal(jsonData, &result)
+	require.NoError(t, err)
+
+	// Check that only 1 file was counted (directory was skipped)
+	report := result[1].(map[string]any)
+	assert.Equal(t, float64(1), report["files"])
+}
+
+func TestScanFilesNonExistentFile(t *testing.T) {
+	files := []string{"/nonexistent/file.go"}
+	_, err := ScanFiles(files, "test")
+	assert.Error(t, err)
+	assert.Contains(t, err.Error(), "error accessing file")
+}
+
+func TestScanDirectory(t *testing.T) {
+	tmpDir := t.TempDir()
+
+	file1 := filepath.Join(tmpDir, "main.go")
+	require.NoError(t, os.WriteFile(file1, []byte("package main\n"), 0644))
+
+	jsonData, err := ScanDirectory(tmpDir, 3, "Test instructions", []string{})
+	require.NoError(t, err)
+
+	var result []any
+	err = json.Unmarshal(jsonData, &result)
+	require.NoError(t, err)
+	assert.Len(t, result, 3)
+
+	// Check instructions
+	instr := result[2].(map[string]any)
+	assert.Equal(t, "Test instructions", instr["details"])
+}

cmd/code2context/main.go

@@ -0,0 +1,109 @@
+package main
+
+import (
+	"bufio"
+	"flag"
+	"fmt"
+	"os"
+	"strings"
+)
+
+func main() {
+	// Command line flags
+	maxDepth := flag.Int("depth", 3, "Maximum directory depth to scan")
+	ignorePatterns := flag.String("ignore", ".git,node_modules,vendor", "Comma-separated patterns to ignore")
+	outputFile := flag.String("out", "", "Output file (default: stdout)")
+	flag.Usage = printUsage
+	flag.Parse()
+
+	// Check if stdin has data (is a pipe)
+	stdinInfo, _ := os.Stdin.Stat()
+	hasStdin := (stdinInfo.Mode() & os.ModeCharDevice) == 0
+
+	var jsonData []byte
+	var err error
+
+	if hasStdin {
+		// Stdin mode: read file list from stdin, instructions from argument
+		if flag.NArg() != 1 {
+			fmt.Fprintf(os.Stderr, "Error: When piping file list via stdin, provide exactly 1 argument: <instructions>\n")
+			fmt.Fprintf(os.Stderr, "Usage: find . -name '*.go' | code2context \"instructions\"\n")
+			os.Exit(1)
+		}
+
+		instructions := flag.Arg(0)
+
+		// Read file paths from stdin
+		var files []string
+		scanner := bufio.NewScanner(os.Stdin)
+		for scanner.Scan() {
+			line := strings.TrimSpace(scanner.Text())
+			if line != "" {
+				files = append(files, line)
+			}
+		}
+		if err := scanner.Err(); err != nil {
+			fmt.Fprintf(os.Stderr, "Error reading stdin: %v\n", err)
+			os.Exit(1)
+		}
+
+		if len(files) == 0 {
+			fmt.Fprintf(os.Stderr, "Error: No files provided via stdin\n")
+			os.Exit(1)
+		}
+
+		jsonData, err = ScanFiles(files, instructions)
+	} else {
+		// Directory mode: require directory and instructions arguments
+		if flag.NArg() != 2 {
+			printUsage()
+			os.Exit(1)
+		}
+
+		directory := flag.Arg(0)
+		instructions := flag.Arg(1)
+
+		// Validate directory
+		if info, err := os.Stat(directory); err != nil || !info.IsDir() {
+			fmt.Fprintf(os.Stderr, "Error: Directory '%s' does not exist or is not a directory\n", directory)
+			os.Exit(1)
+		}
+
+		// Parse ignore patterns and scan directory
+		jsonData, err = ScanDirectory(directory, *maxDepth, instructions, strings.Split(*ignorePatterns, ","))
+	}
+
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "Error scanning: %v\n", err)
+		os.Exit(1)
+	}
+
+	// Output result
+	if *outputFile != "" {
+		if err := os.WriteFile(*outputFile, jsonData, 0644); err != nil {
+			fmt.Fprintf(os.Stderr, "Error writing file: %v\n", err)
+			os.Exit(1)
+		}
+	} else {
+		fmt.Print(string(jsonData))
+	}
+}
+
+func printUsage() {
+	fmt.Fprintf(os.Stderr, `code2context - Code project scanner for use with Fabric AI
+
+Usage:
+  code2context [options] <directory> <instructions>
+  <file_list> | code2context [options] <instructions>
+
+Examples:
+  code2context . "Add input validation to all user inputs"
+  code2context -depth 4 ./my-project "Implement error handling"
+  code2context -out project.json ./src "Fix security issues"
+  find . -name '*.go' | code2context "Refactor error handling"
+  git ls-files '*.py' | code2context "Add type hints"
+
+Options:
+`)
+	flag.PrintDefaults()
+}

cmd/fabric/main.go

@@ -6,13 +6,13 @@ import (
 
 	"github.com/jessevdk/go-flags"
 
-	"github.com/danielmiessler/fabric/cli"
+	"github.com/danielmiessler/fabric/internal/cli"
 )
 
 func main() {
 	err := cli.Cli(version)
 	if err != nil && !flags.WroteHelp(err) {
-		fmt.Printf("%s\n", err)
+		fmt.Fprintf(os.Stderr, "%s\n", err)
 		os.Exit(1)
 	}
 }

cmd/fabric/version.go

@@ -0,0 +1,3 @@
+package main
+
+var version = "v1.4.375"

cmd/generate_changelog/PRD.md

@@ -0,0 +1,151 @@
+# Product Requirements Document: Changelog Generator
+
+## Overview
+
+The Changelog Generator is a high-performance Go tool that automatically generates comprehensive changelogs from git history and GitHub pull requests.
+
+## Goals
+
+1. **Performance**: Very fast. Efficient enough to be used in CI/CD as part of release process.
+2. **Completeness**: Capture ALL commits including unreleased changes
+3. **Efficiency**: Minimize API calls through caching and batch operations
+4. **Reliability**: Handle errors gracefully with proper Go error handling
+5. **Simplicity**: Single binary with no runtime dependencies
+
+## Key Features
+
+### 1. One-Pass Git History Algorithm
+
+- Walk git history once from newest to oldest
+- Start with "Unreleased" bucket for all new commits
+- Switch buckets when encountering version commits
+- No need to calculate ranges between versions
+
+### 2. Native Library Integration
+
+- **go-git**: Pure Go git implementation (no git binary required)
+- **go-github**: Official GitHub Go client library
+- Benefits: Type safety, better error handling, no subprocess overhead
+
+### 3. Smart Caching System
+
+- SQLite-based persistent cache
+- Stores: versions, commits, PR details, last processed commit
+- Enables incremental updates on subsequent runs
+- Instant changelog regeneration from cache
+
+### 4. Concurrent Processing
+
+- Parallel GitHub API calls (up to 10 concurrent)
+- Batch PR fetching with deduplication
+- Rate limiting awareness
+
+### 5. Enhanced Output
+
+- "Unreleased" section for commits since last version
+- Clean markdown formatting
+- Configurable version limiting
+- Direct commit tracking (non-PR commits)
+
+## Technical Architecture
+
+### Module Structure
+
+```text
+cmd/generate_changelog/
+├── main.go              # CLI entry point with cobra
+├── internal/
+│   ├── git/            # Git operations (go-git)
+│   ├── github/         # GitHub API client (go-github)
+│   ├── cache/          # SQLite caching layer
+│   ├── changelog/      # Core generation logic
+│   └── config/         # Configuration management
+└── changelog.db        # SQLite cache (generated)
+```
+
+### Data Flow
+
+1. Git walker collects all commits in one pass
+2. Commits bucketed by version (starting with "Unreleased")
+3. PR numbers extracted from merge commits
+4. GitHub API batch-fetches PR details
+5. Cache stores everything for future runs
+6. Formatter generates markdown output
+
+### Cache Schema
+
+- **metadata**: Last processed commit SHA
+- **versions**: Version names, dates, commit SHAs
+- **commits**: Full commit details with version associations
+- **pull_requests**: PR details including commits
+- Indexes on version and PR number for fast lookups
+
+### Features
+
+- **Unreleased section**: Shows all new commits
+- **Better caching**: SQLite vs JSON, incremental updates
+- **Smarter deduplication**: Removes consecutive duplicate commits
+- **Direct commit tracking**: Shows non-PR commits
+
+### Reliability
+
+- **No subprocess errors**: Direct library usage
+- **Type safety**: Compile-time checking
+- **Better error handling**: Go's explicit error returns
+
+### Deployment
+
+- **Single binary**: No Python/pip/dependencies
+- **Cross-platform**: Compile for any OS/architecture
+- **No git CLI required**: Uses go-git library
+
+## Configuration
+
+### Environment Variables
+
+- `GITHUB_TOKEN`: GitHub API authentication token
+
+### Command Line Flags
+
+- `--repo, -r`: Repository path (default: current directory)
+- `--output, -o`: Output file (default: stdout)
+- `--limit, -l`: Version limit (default: all)
+- `--version, -v`: Target specific version
+- `--save-data`: Export debug JSON
+- `--cache`: Cache file location
+- `--no-cache`: Disable caching
+- `--rebuild-cache`: Force cache rebuild
+- `--token`: GitHub token override
+
+## Success Metrics
+
+1. **Performance**: Generate full changelog in <5 seconds for fabric repo
+2. **Completeness**: 100% commit coverage including unreleased
+3. **Accuracy**: Correct PR associations and change extraction
+4. **Reliability**: Handle network failures gracefully
+5. **Usability**: Simple CLI with sensible defaults
+
+## Future Enhancements
+
+1. **Multiple output formats**: JSON, HTML, etc.
+2. **Custom version patterns**: Configurable regex
+3. **Change categorization**: feat/fix/docs auto-grouping
+4. **Conventional commits**: Full support for semantic versioning
+5. **GitLab/Bitbucket**: Support other platforms
+6. **Web UI**: Interactive changelog browser
+7. **Incremental updates**: Update existing CHANGELOG.md file
+8. **Breaking change detection**: Highlight breaking changes
+
+## Implementation Status
+
+- ✅ Core architecture and modules
+- ✅ One-pass git walking algorithm
+- ✅ GitHub API integration with concurrency
+- ✅ SQLite caching system
+- ✅ Changelog formatting and generation
+- ✅ CLI with all planned flags
+- ✅ Documentation (README and PRD)
+
+## Conclusion
+
+This Go implementation provides a modern, efficient, and feature-rich changelog generator.

cmd/generate_changelog/README.md

@@ -0,0 +1,264 @@
+# Changelog Generator
+
+A high-performance changelog generator for Git repositories that automatically creates comprehensive, well-formatted changelogs from your git history and GitHub pull requests.
+
+## Features
+
+- **One-pass git history walking**: Efficiently processes entire repository history in a single pass
+- **Automatic PR detection**: Extracts pull request information from merge commits
+- **GitHub API integration**: Fetches detailed PR information including commits, authors, and descriptions
+- **Smart caching**: SQLite-based caching for instant incremental updates
+- **Unreleased changes**: Tracks all commits since the last release
+- **Concurrent processing**: Parallel GitHub API calls for improved performance
+- **Flexible output**: Generate complete changelogs or target specific versions
+- **GraphQL optimization**: Ultra-fast PR fetching using GitHub GraphQL API (~5-10 calls vs 1000s)
+- **Intelligent sync**: Automatically syncs new PRs every 24 hours or when missing PRs are detected
+- **AI-powered summaries**: Optional Fabric integration for enhanced changelog summaries
+- **Advanced caching**: Content-based change detection for AI summaries with hash comparison
+- **Author type detection**: Distinguishes between users, bots, and organizations
+- **Lightning-fast incremental updates**: SHA→PR mapping for instant git operations
+
+## Installation
+
+```bash
+go install github.com/danielmiessler/fabric/cmd/generate_changelog@latest
+```
+
+## Usage
+
+### Basic usage (generate complete changelog)
+
+```bash
+generate_changelog
+```
+
+### Save to file
+
+```bash
+generate_changelog -o CHANGELOG.md
+```
+
+### Generate for specific version
+
+```bash
+generate_changelog -v v1.4.244
+```
+
+### Limit to recent versions
+
+```bash
+generate_changelog -l 10
+```
+
+### Using GitHub token for private repos or higher rate limits
+
+```bash
+export GITHUB_TOKEN=your_token_here
+generate_changelog
+
+# Or pass directly
+generate_changelog --token your_token_here
+```
+
+### AI-enhanced summaries
+
+```bash
+# Enable AI summaries using Fabric
+generate_changelog --ai-summarize
+
+# Use custom model for AI summaries
+FABRIC_CHANGELOG_SUMMARIZE_MODEL=claude-opus-4 generate_changelog --ai-summarize
+```
+
+### Cache management
+
+```bash
+# Rebuild cache from scratch
+generate_changelog --rebuild-cache
+
+# Force a full PR sync from GitHub
+generate_changelog --force-pr-sync
+
+# Disable cache usage
+generate_changelog --no-cache
+
+# Use custom cache location
+generate_changelog --cache /path/to/cache.db
+```
+
+## Command Line Options
+
+| Flag | Short | Description | Default |
+|------|-------|-------------|---------|
+| `--repo` | `-r` | Repository path | `.` (current directory) |
+| `--output` | `-o` | Output file | stdout |
+| `--limit` | `-l` | Limit number of versions | 0 (all) |
+| `--version` | `-v` | Generate for specific version | |
+| `--save-data` | | Save version data to JSON | false |
+| `--cache` | | Cache database file | `./cmd/generate_changelog/changelog.db` |
+| `--no-cache` | | Disable cache usage | false |
+| `--rebuild-cache` | | Rebuild cache from scratch | false |
+| `--force-pr-sync` | | Force a full PR sync from GitHub | false |
+| `--token` | | GitHub API token | `$GITHUB_TOKEN` |
+| `--ai-summarize` | | Generate AI-enhanced summaries using Fabric | false |
+| `--release` | | Update GitHub release description with AI summary for version | |
+
+## Output Format
+
+The generated changelog follows this structure:
+
+```markdown
+# Changelog
+
+## Unreleased
+
+### PR [#1601](url) by [author](profile): PR Title
+- Change description 1
+- Change description 2
+
+### Direct commits
+- Direct commit message 1
+- Direct commit message 2
+
+## v1.4.244 (2025-07-09)
+
+### PR [#1598](url) by [author](profile): PR Title
+- Change description
+...
+```
+
+## How It Works
+
+1. **Git History Walking**: The tool walks through your git history from newest to oldest commits
+2. **Version Detection**: Identifies version bump commits (pattern: "Update version to vX.Y.Z")
+3. **PR Extraction**: Detects merge commits and extracts PR numbers
+4. **GitHub API Calls**: Fetches detailed PR information in parallel batches
+5. **Change Extraction**: Extracts changes from PR commit messages or PR body
+6. **Formatting**: Generates clean, organized markdown output
+
+## Performance
+
+- **Native Go libraries**: Uses go-git and go-github for maximum performance
+- **Concurrent API calls**: Processes up to 10 GitHub API requests in parallel
+- **Smart caching**: SQLite cache eliminates redundant API calls
+- **Incremental updates**: Only processes new commits on subsequent runs
+- **GraphQL optimization**: Uses GitHub GraphQL API to fetch all PR data in ~5-10 calls
+- **AI-powered summaries**: Optional Fabric integration with intelligent caching
+- **Content-based change detection**: AI summaries only regenerated when content changes
+- **Lightning-fast git operations**: SHA→PR mapping stored in database for instant lookups
+
+### Major Optimization: GraphQL + Advanced Caching
+
+The tool has been optimized to drastically reduce GitHub API calls and improve performance:
+
+**Previous approach**: Individual API calls for each PR (2 API calls per PR)
+
+- For a repo with 500 PRs: 1,000 API calls
+
+**Current approach**: GraphQL batch fetching with intelligent caching
+
+- For a repo with 500 PRs: ~5-10 GraphQL calls (initial fetch) + 0 calls (subsequent runs with cache)
+- **99%+ reduction in API calls after initial run!**
+
+The optimization includes:
+
+1. **GraphQL Batch Fetch**: Uses GitHub's GraphQL API to fetch all merged PRs with commits in minimal calls
+2. **Smart Caching**: Stores complete PR data, commits, and SHA mappings in SQLite
+3. **Incremental Sync**: Only fetches PRs merged after the last sync timestamp
+4. **Automatic Refresh**: PRs are synced every 24 hours or when missing PRs are detected
+5. **AI Summary Caching**: Content-based change detection prevents unnecessary AI regeneration
+6. **Fallback Support**: If GraphQL fails, falls back to REST API batch fetching
+7. **Lightning Git Operations**: Pre-computed SHA→PR mappings for instant commit association
+
+## Requirements
+
+- Go 1.24+ (for installation from source)
+- Git repository
+- GitHub token (optional, for private repos or higher rate limits)
+- Fabric CLI (optional, for AI-enhanced summaries)
+
+## Authentication
+
+The tool supports GitHub authentication via:
+
+1. Environment variable: `export GITHUB_TOKEN=your_token`
+2. Command line flag: `--token your_token`
+3. `.env` file in the same directory as the binary
+
+### Environment File Support
+
+Create a `.env` file next to the `generate_changelog` binary:
+
+```bash
+GITHUB_TOKEN=your_github_token_here
+FABRIC_CHANGELOG_SUMMARIZE_MODEL=claude-sonnet-4-20250514
+```
+
+The tool automatically loads `.env` files for convenient configuration management.
+
+Without authentication, the tool is limited to 60 GitHub API requests per hour.
+
+## Caching
+
+The SQLite cache stores:
+
+- Version information and commit associations
+- Pull request details (title, body, commits, authors)
+- Last processed commit SHA for incremental updates
+- Last PR sync timestamp for intelligent refresh
+- AI summaries with content-based change detection
+- SHA→PR mappings for lightning-fast git operations
+
+Cache benefits:
+
+- Instant changelog regeneration
+- Drastically reduced GitHub API usage (99%+ reduction after initial run)
+- Offline changelog generation (after initial cache build)
+- Automatic PR data refresh every 24 hours
+- Batch database transactions for better performance
+- Content-aware AI summary regeneration
+
+## AI-Enhanced Summaries
+
+The tool can generate AI-powered summaries using Fabric for more polished, professional changelogs:
+
+```bash
+# Enable AI summarization
+generate_changelog --ai-summarize
+
+# Custom model (default: claude-sonnet-4-20250514)
+FABRIC_CHANGELOG_SUMMARIZE_MODEL=claude-opus-4 generate_changelog --ai-summarize
+```
+
+### AI Summary Features
+
+- **Content-based change detection**: AI summaries are only regenerated when version content changes
+- **Intelligent caching**: Preserves existing summaries and only processes changed versions
+- **Content hash comparison**: Uses SHA256 hashing to detect when "Unreleased" content changes
+- **Automatic fallback**: Falls back to raw content if AI processing fails
+- **Error detection**: Identifies and handles AI processing errors gracefully
+- **Minimum content filtering**: Skips AI processing for very brief content (< 256 characters)
+
+### AI Model Configuration
+
+Set the model via environment variable:
+
+```bash
+export FABRIC_CHANGELOG_SUMMARIZE_MODEL=claude-opus-4
+# or
+export FABRIC_CHANGELOG_SUMMARIZE_MODEL=gpt-4
+```
+
+AI summaries are cached and only regenerated when:
+
+- Version content changes (detected via hash comparison)
+- No existing AI summary exists for the version
+- Force rebuild is requested
+
+## Contributing
+
+This tool is part of the Fabric project. Contributions are welcome!
+
+## License
+
+The MIT License. Same as the Fabric project.

cmd/generate_changelog/internal/cache/cache.go

@@ -0,0 +1,485 @@
+package cache
+
+import (
+	"database/sql"
+	"encoding/json"
+	"fmt"
+	"os"
+	"time"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/git"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/github"
+	_ "github.com/mattn/go-sqlite3"
+)
+
+type Cache struct {
+	db *sql.DB
+}
+
+func New(dbPath string) (*Cache, error) {
+	db, err := sql.Open("sqlite3", dbPath)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open database: %w", err)
+	}
+
+	cache := &Cache{db: db}
+	if err := cache.createTables(); err != nil {
+		return nil, fmt.Errorf("failed to create tables: %w", err)
+	}
+
+	return cache, nil
+}
+
+func (c *Cache) Close() error {
+	return c.db.Close()
+}
+
+func (c *Cache) createTables() error {
+	queries := []string{
+		`CREATE TABLE IF NOT EXISTS metadata (
+			key TEXT PRIMARY KEY,
+			value TEXT NOT NULL,
+			updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		)`,
+		`CREATE TABLE IF NOT EXISTS versions (
+			name TEXT PRIMARY KEY,
+			date DATETIME,
+			commit_sha TEXT,
+			pr_numbers TEXT,
+			ai_summary TEXT,
+			created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		)`,
+		`CREATE TABLE IF NOT EXISTS commits (
+			sha TEXT PRIMARY KEY,
+			version TEXT NOT NULL,
+			message TEXT,
+			author TEXT,
+			email TEXT,
+			date DATETIME,
+			is_merge BOOLEAN,
+			pr_number INTEGER,
+			created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+			FOREIGN KEY (version) REFERENCES versions(name)
+		)`,
+		`CREATE TABLE IF NOT EXISTS pull_requests (
+			number INTEGER PRIMARY KEY,
+			title TEXT,
+			body TEXT,
+			author TEXT,
+			author_url TEXT,
+			author_type TEXT DEFAULT 'user',
+			url TEXT,
+			merged_at DATETIME,
+			merge_commit TEXT,
+			commits TEXT,
+			created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+		)`,
+		`CREATE INDEX IF NOT EXISTS idx_commits_version ON commits(version)`,
+		`CREATE INDEX IF NOT EXISTS idx_commits_pr_number ON commits(pr_number)`,
+		`CREATE TABLE IF NOT EXISTS commit_pr_mapping (
+			commit_sha TEXT PRIMARY KEY,
+			pr_number INTEGER NOT NULL,
+			created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+			FOREIGN KEY (pr_number) REFERENCES pull_requests(number)
+		)`,
+		`CREATE INDEX IF NOT EXISTS idx_commit_pr_mapping_sha ON commit_pr_mapping(commit_sha)`,
+	}
+
+	for _, query := range queries {
+		if _, err := c.db.Exec(query); err != nil {
+			return fmt.Errorf("failed to execute query: %w", err)
+		}
+	}
+
+	return nil
+}
+
+func (c *Cache) GetLastProcessedTag() (string, error) {
+	var tag string
+	err := c.db.QueryRow("SELECT value FROM metadata WHERE key = 'last_processed_tag'").Scan(&tag)
+	if err == sql.ErrNoRows {
+		return "", nil
+	}
+	return tag, err
+}
+
+func (c *Cache) SetLastProcessedTag(tag string) error {
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO metadata (key, value, updated_at)
+		VALUES ('last_processed_tag', ?, CURRENT_TIMESTAMP)
+	`, tag)
+	return err
+}
+
+func (c *Cache) SaveVersion(v *git.Version) error {
+	prNumbers, _ := json.Marshal(v.PRNumbers)
+
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO versions (name, date, commit_sha, pr_numbers, ai_summary)
+		VALUES (?, ?, ?, ?, ?)
+	`, v.Name, v.Date, v.CommitSHA, string(prNumbers), v.AISummary)
+
+	return err
+}
+
+// UpdateVersionAISummary updates only the AI summary for a specific version
+func (c *Cache) UpdateVersionAISummary(versionName, aiSummary string) error {
+	_, err := c.db.Exec(`
+		UPDATE versions SET ai_summary = ? WHERE name = ?
+	`, aiSummary, versionName)
+	return err
+}
+
+func (c *Cache) SaveCommit(commit *git.Commit, version string) error {
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO commits
+		(sha, version, message, author, email, date, is_merge, pr_number)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+	`, commit.SHA, version, commit.Message, commit.Author, commit.Email,
+		commit.Date, commit.IsMerge, commit.PRNumber)
+
+	return err
+}
+
+func (c *Cache) SavePR(pr *github.PR) error {
+	commits, _ := json.Marshal(pr.Commits)
+
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO pull_requests
+		(number, title, body, author, author_url, author_type, url, merged_at, merge_commit, commits)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+	`, pr.Number, pr.Title, pr.Body, pr.Author, pr.AuthorURL, pr.AuthorType,
+		pr.URL, pr.MergedAt, pr.MergeCommit, string(commits))
+
+	return err
+}
+
+func (c *Cache) GetPR(number int) (*github.PR, error) {
+	var pr github.PR
+	var commitsJSON string
+
+	err := c.db.QueryRow(`
+		SELECT number, title, body, author, author_url, COALESCE(author_type, 'user'), url, merged_at, merge_commit, commits
+		FROM pull_requests WHERE number = ?
+	`, number).Scan(
+		&pr.Number, &pr.Title, &pr.Body, &pr.Author, &pr.AuthorURL, &pr.AuthorType,
+		&pr.URL, &pr.MergedAt, &pr.MergeCommit, &commitsJSON,
+	)
+
+	if err == sql.ErrNoRows {
+		return nil, nil
+	}
+	if err != nil {
+		return nil, err
+	}
+
+	if err := json.Unmarshal([]byte(commitsJSON), &pr.Commits); err != nil {
+		return nil, fmt.Errorf("failed to unmarshal commits: %w", err)
+	}
+
+	return &pr, nil
+}
+
+func (c *Cache) GetVersions() (map[string]*git.Version, error) {
+	rows, err := c.db.Query(`
+		SELECT name, date, commit_sha, pr_numbers, ai_summary FROM versions
+	`)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	versions := make(map[string]*git.Version)
+
+	for rows.Next() {
+		var v git.Version
+		var dateStr sql.NullString
+		var prNumbersJSON string
+		var aiSummary sql.NullString
+
+		if err := rows.Scan(&v.Name, &dateStr, &v.CommitSHA, &prNumbersJSON, &aiSummary); err != nil {
+			return nil, err
+		}
+
+		if dateStr.Valid {
+			// Try multiple date formats: SQLite format, RFC3339Nano, and RFC3339
+			dateFormats := []string{
+				"2006-01-02 15:04:05-07:00",           // SQLite DATETIME format
+				"2006-01-02 15:04:05.999999999-07:00", // SQLite with fractional seconds
+				time.RFC3339Nano,
+				time.RFC3339,
+			}
+			var parseErr error
+			for _, format := range dateFormats {
+				v.Date, parseErr = time.Parse(format, dateStr.String)
+				if parseErr == nil {
+					break // Successfully parsed
+				}
+			}
+			if parseErr != nil {
+				fmt.Fprintf(os.Stderr, "Error parsing date '%s' for version '%s': %v\n", dateStr.String, v.Name, parseErr)
+			}
+		}
+
+		if prNumbersJSON != "" {
+			json.Unmarshal([]byte(prNumbersJSON), &v.PRNumbers)
+		}
+
+		if aiSummary.Valid {
+			v.AISummary = aiSummary.String
+		}
+
+		v.Commits, err = c.getCommitsForVersion(v.Name)
+		if err != nil {
+			return nil, err
+		}
+
+		versions[v.Name] = &v
+	}
+
+	return versions, rows.Err()
+}
+
+func (c *Cache) getCommitsForVersion(version string) ([]*git.Commit, error) {
+	rows, err := c.db.Query(`
+		SELECT sha, message, author, email, date, is_merge, pr_number
+		FROM commits WHERE version = ?
+		ORDER BY date DESC
+	`, version)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	var commits []*git.Commit
+
+	for rows.Next() {
+		var commit git.Commit
+		if err := rows.Scan(
+			&commit.SHA, &commit.Message, &commit.Author, &commit.Email,
+			&commit.Date, &commit.IsMerge, &commit.PRNumber,
+		); err != nil {
+			return nil, err
+		}
+		commits = append(commits, &commit)
+	}
+
+	return commits, rows.Err()
+}
+
+func (c *Cache) Clear() error {
+	tables := []string{"metadata", "versions", "commits", "pull_requests"}
+	for _, table := range tables {
+		if _, err := c.db.Exec("DELETE FROM " + table); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// VersionExists checks if a version already exists in the cache
+func (c *Cache) VersionExists(version string) (bool, error) {
+	var count int
+	err := c.db.QueryRow("SELECT COUNT(*) FROM versions WHERE name = ?", version).Scan(&count)
+	if err != nil {
+		return false, err
+	}
+	return count > 0, nil
+}
+
+// CommitExists checks if a commit already exists in the cache
+func (c *Cache) CommitExists(hash string) (bool, error) {
+	var count int
+	err := c.db.QueryRow("SELECT COUNT(*) FROM commits WHERE sha = ?", hash).Scan(&count)
+	if err != nil {
+		return false, err
+	}
+	return count > 0, nil
+}
+
+// GetLastPRSync returns the timestamp of the last PR sync
+func (c *Cache) GetLastPRSync() (time.Time, error) {
+	var timestamp string
+	err := c.db.QueryRow("SELECT value FROM metadata WHERE key = 'last_pr_sync'").Scan(&timestamp)
+	if err == sql.ErrNoRows {
+		return time.Time{}, nil
+	}
+	if err != nil {
+		return time.Time{}, err
+	}
+
+	return time.Parse(time.RFC3339, timestamp)
+}
+
+// SetLastPRSync updates the timestamp of the last PR sync
+func (c *Cache) SetLastPRSync(timestamp time.Time) error {
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO metadata (key, value, updated_at)
+		VALUES ('last_pr_sync', ?, CURRENT_TIMESTAMP)
+	`, timestamp.Format(time.RFC3339))
+	return err
+}
+
+// SavePRBatch saves multiple PRs in a single transaction for better performance
+func (c *Cache) SavePRBatch(prs []*github.PR) error {
+	tx, err := c.db.Begin()
+	if err != nil {
+		return fmt.Errorf("failed to begin transaction: %w", err)
+	}
+	defer tx.Rollback()
+
+	stmt, err := tx.Prepare(`
+		INSERT OR REPLACE INTO pull_requests
+		(number, title, body, author, author_url, author_type, url, merged_at, merge_commit, commits)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+	`)
+	if err != nil {
+		return fmt.Errorf("failed to prepare statement: %w", err)
+	}
+	defer stmt.Close()
+
+	for _, pr := range prs {
+		commits, _ := json.Marshal(pr.Commits)
+		_, err := stmt.Exec(
+			pr.Number, pr.Title, pr.Body, pr.Author, pr.AuthorURL, pr.AuthorType,
+			pr.URL, pr.MergedAt, pr.MergeCommit, string(commits),
+		)
+		if err != nil {
+			return fmt.Errorf("failed to save PR #%d: %w", pr.Number, err)
+		}
+	}
+
+	return tx.Commit()
+}
+
+// GetAllPRs returns all cached PRs
+func (c *Cache) GetAllPRs() (map[int]*github.PR, error) {
+	rows, err := c.db.Query(`
+		SELECT number, title, body, author, author_url, COALESCE(author_type, 'user'), url, merged_at, merge_commit, commits
+		FROM pull_requests
+	`)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	prs := make(map[int]*github.PR)
+
+	for rows.Next() {
+		var pr github.PR
+		var commitsJSON string
+
+		if err := rows.Scan(
+			&pr.Number, &pr.Title, &pr.Body, &pr.Author, &pr.AuthorURL, &pr.AuthorType,
+			&pr.URL, &pr.MergedAt, &pr.MergeCommit, &commitsJSON,
+		); err != nil {
+			return nil, err
+		}
+
+		if err := json.Unmarshal([]byte(commitsJSON), &pr.Commits); err != nil {
+			return nil, fmt.Errorf("failed to unmarshal commits for PR #%d: %w", pr.Number, err)
+		}
+
+		prs[pr.Number] = &pr
+	}
+
+	return prs, rows.Err()
+}
+
+// MarkPRAsNonExistent marks a PR number as non-existent to avoid future fetches
+func (c *Cache) MarkPRAsNonExistent(prNumber int) error {
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO metadata (key, value, updated_at)
+		VALUES (?, 'non_existent', CURRENT_TIMESTAMP)
+	`, fmt.Sprintf("pr_non_existent_%d", prNumber))
+	return err
+}
+
+// IsPRMarkedAsNonExistent checks if a PR is marked as non-existent
+func (c *Cache) IsPRMarkedAsNonExistent(prNumber int) bool {
+	var value string
+	err := c.db.QueryRow("SELECT value FROM metadata WHERE key = ?",
+		fmt.Sprintf("pr_non_existent_%d", prNumber)).Scan(&value)
+	return err == nil && value == "non_existent"
+}
+
+// SaveCommitPRMappings saves SHA→PR mappings for all commits in PRs
+func (c *Cache) SaveCommitPRMappings(prs []*github.PR) error {
+	tx, err := c.db.Begin()
+	if err != nil {
+		return fmt.Errorf("failed to begin transaction: %w", err)
+	}
+	defer tx.Rollback()
+
+	stmt, err := tx.Prepare(`
+		INSERT OR REPLACE INTO commit_pr_mapping (commit_sha, pr_number)
+		VALUES (?, ?)
+	`)
+	if err != nil {
+		return fmt.Errorf("failed to prepare statement: %w", err)
+	}
+	defer stmt.Close()
+
+	for _, pr := range prs {
+		for _, commit := range pr.Commits {
+			_, err := stmt.Exec(commit.SHA, pr.Number)
+			if err != nil {
+				return fmt.Errorf("failed to save commit mapping %s→%d: %w", commit.SHA, pr.Number, err)
+			}
+		}
+	}
+
+	return tx.Commit()
+}
+
+// GetPRNumberBySHA returns the PR number for a given commit SHA
+func (c *Cache) GetPRNumberBySHA(sha string) (int, bool) {
+	var prNumber int
+	err := c.db.QueryRow("SELECT pr_number FROM commit_pr_mapping WHERE commit_sha = ?", sha).Scan(&prNumber)
+	if err == sql.ErrNoRows {
+		return 0, false
+	}
+	if err != nil {
+		return 0, false
+	}
+	return prNumber, true
+}
+
+// GetCommitSHAsForPR returns all commit SHAs for a given PR number
+func (c *Cache) GetCommitSHAsForPR(prNumber int) ([]string, error) {
+	rows, err := c.db.Query("SELECT commit_sha FROM commit_pr_mapping WHERE pr_number = ?", prNumber)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	var shas []string
+	for rows.Next() {
+		var sha string
+		if err := rows.Scan(&sha); err != nil {
+			return nil, err
+		}
+		shas = append(shas, sha)
+	}
+
+	return shas, rows.Err()
+}
+
+// GetUnreleasedContentHash returns the cached content hash for Unreleased
+func (c *Cache) GetUnreleasedContentHash() (string, error) {
+	var hash string
+	err := c.db.QueryRow("SELECT value FROM metadata WHERE key = 'unreleased_content_hash'").Scan(&hash)
+	if err == sql.ErrNoRows {
+		return "", fmt.Errorf("no content hash found")
+	}
+	return hash, err
+}
+
+// SetUnreleasedContentHash stores the content hash for Unreleased
+func (c *Cache) SetUnreleasedContentHash(hash string) error {
+	_, err := c.db.Exec(`
+		INSERT OR REPLACE INTO metadata (key, value, updated_at)
+		VALUES ('unreleased_content_hash', ?, CURRENT_TIMESTAMP)
+	`, hash)
+	return err
+}

cmd/generate_changelog/internal/changelog/generator.go

@@ -0,0 +1,806 @@
+package changelog
+
+import (
+	"crypto/sha256"
+	"fmt"
+	"os"
+	"regexp"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/cache"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/config"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/git"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/github"
+)
+
+type Generator struct {
+	cfg       *config.Config
+	gitWalker *git.Walker
+	ghClient  *github.Client
+	cache     *cache.Cache
+	versions  map[string]*git.Version
+	prs       map[int]*github.PR
+}
+
+func New(cfg *config.Config) (*Generator, error) {
+	gitWalker, err := git.NewWalker(cfg.RepoPath)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create git walker: %w", err)
+	}
+
+	owner, repo, err := gitWalker.GetRepoInfo()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get repo info: %w", err)
+	}
+
+	ghClient := github.NewClient(cfg.GitHubToken, owner, repo)
+
+	var c *cache.Cache
+	if !cfg.NoCache {
+		c, err = cache.New(cfg.CacheFile)
+		if err != nil {
+			return nil, fmt.Errorf("failed to create cache: %w", err)
+		}
+
+		if cfg.RebuildCache {
+			if err := c.Clear(); err != nil {
+				return nil, fmt.Errorf("failed to clear cache: %w", err)
+			}
+		}
+	}
+
+	return &Generator{
+		cfg:       cfg,
+		gitWalker: gitWalker,
+		ghClient:  ghClient,
+		cache:     c,
+		prs:       make(map[int]*github.PR),
+	}, nil
+}
+
+func (g *Generator) Generate() (string, error) {
+	if err := g.collectData(); err != nil {
+		return "", fmt.Errorf("failed to collect data: %w", err)
+	}
+
+	if err := g.fetchPRs(g.cfg.ForcePRSync); err != nil {
+		return "", fmt.Errorf("failed to fetch PRs: %w", err)
+	}
+
+	return g.formatChangelog(), nil
+}
+
+func (g *Generator) collectData() error {
+	if g.cache != nil && !g.cfg.RebuildCache {
+		cachedTag, err := g.cache.GetLastProcessedTag()
+		if err != nil {
+			return fmt.Errorf("failed to get last processed tag: %w", err)
+		}
+
+		if cachedTag != "" {
+			// Get the current latest tag from git
+			currentTag, err := g.gitWalker.GetLatestTag()
+			if err == nil {
+				// Load cached data - we can use it even if there are new tags
+				cachedVersions, err := g.cache.GetVersions()
+				if err == nil && len(cachedVersions) > 0 {
+					g.versions = cachedVersions
+
+					// Load cached PRs
+					for _, version := range g.versions {
+						for _, prNum := range version.PRNumbers {
+							if pr, err := g.cache.GetPR(prNum); err == nil && pr != nil {
+								g.prs[prNum] = pr
+							}
+						}
+					}
+
+					// If we have new tags since cache, process the new versions only
+					if currentTag != cachedTag {
+						fmt.Fprintf(os.Stderr, "Processing new versions since %s...\n", cachedTag)
+						newVersions, err := g.gitWalker.WalkHistorySinceTag(cachedTag)
+						if err != nil {
+							fmt.Fprintf(os.Stderr, "Warning: Failed to walk history since tag %s: %v\n", cachedTag, err)
+						} else {
+							// Merge new versions into cached versions (only add if not already cached)
+							for name, version := range newVersions {
+								if name != "Unreleased" { // Handle Unreleased separately
+									if existingVersion, exists := g.versions[name]; !exists {
+										g.versions[name] = version
+									} else {
+										// Update existing version with new PR numbers if they're missing
+										if len(existingVersion.PRNumbers) == 0 && len(version.PRNumbers) > 0 {
+											existingVersion.PRNumbers = version.PRNumbers
+										}
+									}
+								}
+							}
+						}
+					}
+
+					// Always update Unreleased section with latest commits
+					unreleasedVersion, err := g.gitWalker.WalkCommitsSinceTag(currentTag)
+					if err != nil {
+						fmt.Fprintf(os.Stderr, "Warning: Failed to walk commits since tag %s: %v\n", currentTag, err)
+					} else if unreleasedVersion != nil {
+						// Preserve existing AI summary if available
+						if existingUnreleased, exists := g.versions["Unreleased"]; exists {
+							unreleasedVersion.AISummary = existingUnreleased.AISummary
+						}
+						// Replace or add the unreleased version
+						g.versions["Unreleased"] = unreleasedVersion
+					}
+
+					// Save any new versions to cache (after potential AI processing)
+					if currentTag != cachedTag {
+						for _, version := range g.versions {
+							// Skip versions that were already cached and Unreleased
+							if version.Name != "Unreleased" {
+								if err := g.cache.SaveVersion(version); err != nil {
+									fmt.Fprintf(os.Stderr, "Warning: Failed to save version to cache: %v\n", err)
+								}
+
+								for _, commit := range version.Commits {
+									if err := g.cache.SaveCommit(commit, version.Name); err != nil {
+										fmt.Fprintf(os.Stderr, "Warning: Failed to save commit to cache: %v\n", err)
+									}
+								}
+							}
+						}
+
+						// Update the last processed tag
+						if err := g.cache.SetLastProcessedTag(currentTag); err != nil {
+							fmt.Fprintf(os.Stderr, "Warning: Failed to update last processed tag: %v\n", err)
+						}
+					}
+
+					return nil
+				}
+			}
+		}
+	}
+
+	versions, err := g.gitWalker.WalkHistory()
+	if err != nil {
+		return fmt.Errorf("failed to walk history: %w", err)
+	}
+
+	g.versions = versions
+
+	if g.cache != nil {
+		for _, version := range versions {
+			if err := g.cache.SaveVersion(version); err != nil {
+				return fmt.Errorf("failed to save version to cache: %w", err)
+			}
+
+			for _, commit := range version.Commits {
+				if err := g.cache.SaveCommit(commit, version.Name); err != nil {
+					return fmt.Errorf("failed to save commit to cache: %w", err)
+				}
+			}
+		}
+
+		// Save the latest tag as our cache anchor point
+		if latestTag, err := g.gitWalker.GetLatestTag(); err == nil && latestTag != "" {
+			if err := g.cache.SetLastProcessedTag(latestTag); err != nil {
+				return fmt.Errorf("failed to save last processed tag: %w", err)
+			}
+		}
+	}
+
+	return nil
+}
+
+func (g *Generator) fetchPRs(forcePRSync bool) error {
+	// First, load all cached PRs
+	if g.cache != nil {
+		cachedPRs, err := g.cache.GetAllPRs()
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to load cached PRs: %v\n", err)
+		} else {
+			g.prs = cachedPRs
+		}
+	}
+
+	// Check if we need to fetch new PRs
+	var lastSync time.Time
+	if g.cache != nil {
+		lastSync, _ = g.cache.GetLastPRSync()
+	}
+
+	// Check if we need to sync for missing PRs
+	missingPRs := false
+	for _, version := range g.versions {
+		for _, prNum := range version.PRNumbers {
+			if _, exists := g.prs[prNum]; !exists {
+				missingPRs = true
+				break
+			}
+		}
+		if missingPRs {
+			break
+		}
+	}
+
+	if missingPRs {
+		fmt.Fprintf(os.Stderr, "Full sync triggered due to missing PRs in cache.\n")
+	}
+	// If we have never synced or it's been more than 24 hours, do a full sync
+	// Also sync if we have versions with PR numbers that aren't cached
+	needsSync := lastSync.IsZero() || time.Since(lastSync) > 24*time.Hour || forcePRSync || missingPRs
+
+	if !needsSync {
+		fmt.Fprintf(os.Stderr, "Using cached PR data (last sync: %s)\n", lastSync.Format("2006-01-02 15:04:05"))
+		return nil
+	}
+
+	fmt.Fprintf(os.Stderr, "Fetching merged PRs from GitHub using GraphQL...\n")
+
+	// Use GraphQL for ultimate performance - gets everything in ~5-10 calls
+	prs, err := g.ghClient.FetchAllMergedPRsGraphQL(lastSync)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "GraphQL fetch failed, falling back to REST API: %v\n", err)
+		// Fall back to REST API
+		prs, err = g.ghClient.FetchAllMergedPRs(lastSync)
+		if err != nil {
+			return fmt.Errorf("both GraphQL and REST API failed: %w", err)
+		}
+	}
+
+	// Update our PR map with new data
+	for _, pr := range prs {
+		g.prs[pr.Number] = pr
+	}
+
+	// Save all PRs to cache in a batch transaction
+	if g.cache != nil && len(prs) > 0 {
+		// Save PRs
+		if err := g.cache.SavePRBatch(prs); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to cache PRs: %v\n", err)
+		}
+
+		// Save SHA→PR mappings for lightning-fast git operations
+		if err := g.cache.SaveCommitPRMappings(prs); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to cache commit mappings: %v\n", err)
+		}
+
+		// Update last sync timestamp
+		if err := g.cache.SetLastPRSync(time.Now()); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to update last sync timestamp: %v\n", err)
+		}
+	}
+
+	if len(prs) > 0 {
+		fmt.Fprintf(os.Stderr, "Fetched %d PRs with commits (total cached: %d)\n", len(prs), len(g.prs))
+	}
+
+	return nil
+}
+
+func (g *Generator) formatChangelog() string {
+	var sb strings.Builder
+	sb.WriteString("# Changelog\n")
+
+	versionList := g.getSortedVersions()
+
+	for _, version := range versionList {
+		if g.cfg.Version != "" && version.Name != g.cfg.Version {
+			continue
+		}
+
+		versionText := g.formatVersion(version)
+		if versionText != "" {
+			sb.WriteString("\n")
+			sb.WriteString(versionText)
+		}
+	}
+
+	return sb.String()
+}
+
+func (g *Generator) getSortedVersions() []*git.Version {
+	var versions []*git.Version
+	var releasedVersions []*git.Version
+
+	// Collect all released versions (non-"Unreleased")
+	for name, version := range g.versions {
+		if name != "Unreleased" {
+			releasedVersions = append(releasedVersions, version)
+		}
+	}
+
+	// Sort released versions by date (newest first)
+	sort.Slice(releasedVersions, func(i, j int) bool {
+		return releasedVersions[i].Date.After(releasedVersions[j].Date)
+	})
+
+	// Add "Unreleased" first if it exists and has commits
+	if unreleased, exists := g.versions["Unreleased"]; exists && len(unreleased.Commits) > 0 {
+		versions = append(versions, unreleased)
+	}
+
+	// Add sorted released versions
+	versions = append(versions, releasedVersions...)
+
+	if g.cfg.Limit > 0 && len(versions) > g.cfg.Limit {
+		versions = versions[:g.cfg.Limit]
+	}
+
+	return versions
+}
+
+func (g *Generator) formatVersion(version *git.Version) string {
+	var sb strings.Builder
+
+	// Generate raw content
+	rawContent := g.generateRawVersionContent(version)
+	if rawContent == "" {
+		return ""
+	}
+
+	header := g.formatVersionHeader(version)
+	sb.WriteString(("\n"))
+	sb.WriteString(header)
+
+	// If AI summarization is enabled, enhance with AI
+	if g.cfg.EnableAISummary {
+		// For "Unreleased", check if content has changed since last AI summary
+		if version.Name == "Unreleased" && version.AISummary != "" && g.cache != nil {
+			// Get cached content hash
+			cachedHash, err := g.cache.GetUnreleasedContentHash()
+			if err == nil {
+				// Calculate current content hash
+				currentHash := hashContent(rawContent)
+				if cachedHash == currentHash {
+					// Content unchanged, use cached summary
+					fmt.Fprintf(os.Stderr, "✅ %s content unchanged (skipping AI)\n", version.Name)
+					sb.WriteString(version.AISummary)
+					return fixMarkdown(sb.String())
+				}
+			}
+		}
+
+		// For released versions, if we have cached AI summary, use it!
+		if version.Name != "Unreleased" && version.AISummary != "" {
+			fmt.Fprintf(os.Stderr, "✅ %s already summarized (skipping)\n", version.Name)
+			sb.WriteString(version.AISummary)
+			return fixMarkdown(sb.String())
+		}
+
+		fmt.Fprintf(os.Stderr, "🤖 AI summarizing %s...", version.Name)
+
+		aiSummary, err := SummarizeVersionContent(rawContent)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, " Failed: %v\n", err)
+			sb.WriteString((rawContent))
+			return fixMarkdown(sb.String())
+		}
+		if checkForAIError(aiSummary) {
+			fmt.Fprintf(os.Stderr, " AI error detected, using raw content instead\n")
+			sb.WriteString(rawContent)
+			fmt.Fprintf(os.Stderr, "Raw Content was: (%d bytes) %s \n", len(rawContent), rawContent)
+			fmt.Fprintf(os.Stderr, "AI Summary was: (%d bytes) %s\n", len(aiSummary), aiSummary)
+			return fixMarkdown(sb.String())
+		}
+
+		fmt.Fprintf(os.Stderr, " Done!\n")
+		aiSummary = strings.TrimSpace(aiSummary)
+
+		// Cache the AI summary and content hash
+		version.AISummary = aiSummary
+		if g.cache != nil {
+			if err := g.cache.UpdateVersionAISummary(version.Name, aiSummary); err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to cache AI summary: %v\n", err)
+			}
+			// Cache content hash for "Unreleased" to detect changes
+			if version.Name == "Unreleased" {
+				if err := g.cache.SetUnreleasedContentHash(hashContent(rawContent)); err != nil {
+					fmt.Fprintf(os.Stderr, "Warning: Failed to cache content hash: %v\n", err)
+				}
+			}
+		}
+
+		sb.WriteString(aiSummary)
+		return fixMarkdown(sb.String())
+	}
+
+	sb.WriteString(rawContent)
+	return fixMarkdown(sb.String())
+}
+
+func checkForAIError(summary string) bool {
+	// Check for common AI error patterns
+	errorPatterns := []string{
+		"I don't see any", "please provide",
+		"content you've provided appears to be incomplete",
+	}
+
+	for _, pattern := range errorPatterns {
+		if strings.Contains(summary, pattern) {
+			return true
+		}
+	}
+
+	return false
+}
+
+// formatVersionHeader formats just the version header (## ...)
+func (g *Generator) formatVersionHeader(version *git.Version) string {
+	if version.Name == "Unreleased" {
+		return "## Unreleased\n\n"
+	}
+	return fmt.Sprintf("\n## %s (%s)\n\n", version.Name, version.Date.Format("2006-01-02"))
+}
+
+// generateRawVersionContent generates the raw content (PRs + commits) for a version
+func (g *Generator) generateRawVersionContent(version *git.Version) string {
+	var sb strings.Builder
+
+	// Build a set of commit SHAs that are part of fetched PRs
+	prCommitSHAs := make(map[string]bool)
+	for _, prNum := range version.PRNumbers {
+		if pr, exists := g.prs[prNum]; exists {
+			for _, prCommit := range pr.Commits {
+				prCommitSHAs[prCommit.SHA] = true
+			}
+		}
+	}
+
+	prCommits := make(map[int][]*git.Commit)
+	directCommits := []*git.Commit{}
+
+	for _, commit := range version.Commits {
+		// Skip version bump commits from output
+		if commit.IsVersion {
+			continue
+		}
+
+		// If this commit is part of a fetched PR, don't include it in direct commits
+		if prCommitSHAs[commit.SHA] {
+			continue
+		}
+
+		if commit.PRNumber > 0 {
+			prCommits[commit.PRNumber] = append(prCommits[commit.PRNumber], commit)
+		} else {
+			directCommits = append(directCommits, commit)
+		}
+	}
+
+	// There are occasionally no PRs or direct commits other than version bumps, so we handle that gracefully
+	// However, don't return early if we have PRs to output from version.PRNumbers
+	if len(prCommits) == 0 && len(directCommits) == 0 && len(version.PRNumbers) == 0 {
+		return ""
+	}
+
+	prependNewline := ""
+	for _, prNum := range version.PRNumbers {
+		if pr, exists := g.prs[prNum]; exists {
+			sb.WriteString(prependNewline)
+			sb.WriteString(g.formatPR(pr))
+			prependNewline = "\n"
+		}
+	}
+
+	if len(directCommits) > 0 {
+		// Sort direct commits by date (newest first) for consistent ordering
+		sort.Slice(directCommits, func(i, j int) bool {
+			return directCommits[i].Date.After(directCommits[j].Date)
+		})
+
+		sb.WriteString(prependNewline + "### Direct commits\n\n")
+		for _, commit := range directCommits {
+			message := g.formatCommitMessage(strings.TrimSpace(commit.Message))
+			if message != "" && !g.isDuplicateMessage(message, directCommits) {
+				sb.WriteString(fmt.Sprintf("- %s\n", message))
+			}
+		}
+	}
+
+	return fixMarkdown(
+		strings.ReplaceAll(sb.String(), "\n-\n", "\n"), // Remove empty list items
+	)
+}
+
+func fixMarkdown(content string) string {
+
+	// Fix MD032/blank-around-lists: Lists should be surrounded by blank lines
+	lines := strings.Split(content, "\n")
+	inList := false
+	preListNewline := false
+	for i := range lines {
+		line := strings.TrimSpace(lines[i])
+		if strings.HasPrefix(line, "- ") || strings.HasPrefix(line, "* ") {
+			if !inList {
+				inList = true
+				// Ensure there's a blank line before the list starts
+				if !preListNewline && i > 0 && lines[i-1] != "" {
+					line = "\n" + line
+					preListNewline = true
+				}
+			}
+		} else {
+			if inList {
+				inList = false
+				preListNewline = false
+			}
+		}
+		lines[i] = strings.TrimRight(line, " \t")
+	}
+
+	fixedContent := strings.TrimSpace(strings.Join(lines, "\n"))
+
+	return fixedContent + "\n"
+}
+
+func (g *Generator) formatPR(pr *github.PR) string {
+	var sb strings.Builder
+
+	pr.Title = strings.TrimRight(strings.TrimSpace(pr.Title), ".")
+
+	// Add type indicator for non-users
+	authorName := pr.Author
+	switch pr.AuthorType {
+	case "bot":
+		authorName += "[bot]"
+	case "organization":
+		authorName += "[org]"
+	}
+
+	sb.WriteString(fmt.Sprintf("### PR [#%d](%s) by [%s](%s): %s\n\n",
+		pr.Number, pr.URL, authorName, pr.AuthorURL, strings.TrimSpace(pr.Title)))
+
+	changes := g.extractChanges(pr)
+	for _, change := range changes {
+		if change != "" {
+			sb.WriteString(fmt.Sprintf("- %s\n", change))
+		}
+	}
+
+	return sb.String()
+}
+
+func (g *Generator) extractChanges(pr *github.PR) []string {
+	var changes []string
+	seen := make(map[string]bool)
+
+	for _, commit := range pr.Commits {
+		message := g.formatCommitMessage(commit.Message)
+		if message != "" && !seen[message] {
+			seen[message] = true
+			changes = append(changes, message)
+		}
+	}
+
+	if len(changes) == 0 && pr.Body != "" {
+		lines := strings.SplitSeq(pr.Body, "\n")
+		for line := range lines {
+			line = strings.TrimSpace(line)
+			if strings.HasPrefix(line, "- ") || strings.HasPrefix(line, "* ") {
+				change := strings.TrimPrefix(strings.TrimPrefix(line, "- "), "* ")
+				if change != "" {
+					changes = append(changes, change)
+				}
+			}
+		}
+	}
+
+	return changes
+}
+
+func normalizeLineEndings(content string) string {
+	return strings.ReplaceAll(content, "\r\n", "\n")
+}
+
+func (g *Generator) formatCommitMessage(message string) string {
+	strings_to_remove := []string{
+		"### CHANGES\n", "## CHANGES\n", "# CHANGES\n",
+		"...\n", "---\n", "## Changes\n", "## Change",
+		"Update version to v..1 and commit\n",
+		"# What this Pull Request (PR) does\n",
+		"# Conflicts:",
+	}
+
+	message = normalizeLineEndings(message)
+	// No hard tabs
+	message = strings.ReplaceAll(message, "\t", " ")
+
+	if len(message) > 0 {
+		message = strings.ToUpper(message[:1]) + message[1:]
+	}
+
+	for _, str := range strings_to_remove {
+		if strings.Contains(message, str) {
+			message = strings.ReplaceAll(message, str, "")
+		}
+	}
+
+	message = fixFormatting(message)
+
+	return message
+}
+
+func fixFormatting(message string) string {
+	// Turn "*"" lists into "-" lists"
+	message = strings.ReplaceAll(message, "* ", "- ")
+	// Remove extra spaces around dashes
+	message = strings.ReplaceAll(message, "-   ", "- ")
+	message = strings.ReplaceAll(message, "-  ", "- ")
+	// turn bare URL into <URL>
+	if strings.Contains(message, "http://") || strings.Contains(message, "https://") {
+		// Use regex to wrap bare URLs with angle brackets
+		urlRegex := regexp.MustCompile(`\b(https?://[^\s<>]+)`)
+		message = urlRegex.ReplaceAllString(message, "<$1>")
+	}
+
+	// Replace "## LINKS\n" with "- "
+	message = strings.ReplaceAll(message, "## LINKS\n", "- ")
+	// Dependabot messages: "- [Commits]" should become "\n- [Commits]"
+	message = strings.TrimSpace(message)
+	// Turn multiple newlines into a single newline
+	message = strings.TrimSpace(strings.ReplaceAll(message, "\n\n", "\n"))
+	// Fix inline trailing spaces
+	message = strings.ReplaceAll(message, " \n", "\n")
+	// Fix weird indent before list,
+	message = strings.ReplaceAll(message, "\n - ", "\n- ")
+
+	// blanks-around-lists MD032 fix
+	// Use regex to ensure blank line before list items that don't already have one
+	listRegex := regexp.MustCompile(`(?m)([^\n-].*[^:\n])\n([-*] .*)`)
+	message = listRegex.ReplaceAllString(message, "$1\n\n$2")
+
+	// Change random first-level "#" to 4th level "####"
+	// This is a hack to fix spurious first-level headings that are not actual headings
+	// but rather just comments or notes in the commit message.
+	message = strings.ReplaceAll(message, "# ", "\n#### ")
+	message = strings.ReplaceAll(message, "\n\n\n", "\n\n")
+
+	// Wrap any non-wrapped Emails with angle brackets
+	emailRegex := regexp.MustCompile(`([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})`)
+	message = emailRegex.ReplaceAllString(message, "<$1>")
+
+	// Wrap any non-wrapped URLs with angle brackets
+	urlRegex := regexp.MustCompile(`(https?://[^\s<]+)`)
+	message = urlRegex.ReplaceAllString(message, "<$1>")
+
+	message = strings.ReplaceAll(message, "<<", "<")
+	message = strings.ReplaceAll(message, ">>", ">")
+
+	// Fix some spurious Issue/PR links at the beginning of a commit message line
+	prOrIssueLinkRegex := regexp.MustCompile("\n" + `(#\d+)`)
+	message = prOrIssueLinkRegex.ReplaceAllString(message, " $1")
+
+	// Remove leading/trailing whitespace
+	message = strings.TrimSpace(message)
+	return message
+}
+
+func (g *Generator) isDuplicateMessage(message string, commits []*git.Commit) bool {
+	if message == "." || strings.ToLower(message) == "fix" {
+		count := 0
+		for _, commit := range commits {
+			formatted := g.formatCommitMessage(commit.Message)
+			if formatted == message {
+				count++
+				if count > 1 {
+					return true
+				}
+			}
+		}
+	}
+	return false
+}
+
+// hashContent generates a SHA256 hash of the content for change detection
+func hashContent(content string) string {
+	hash := sha256.Sum256([]byte(content))
+	return fmt.Sprintf("%x", hash)
+}
+
+// SyncDatabase performs a comprehensive database synchronization and validation
+func (g *Generator) SyncDatabase() error {
+	if g.cache == nil {
+		return fmt.Errorf("cache is disabled, cannot sync database")
+	}
+
+	fmt.Fprintf(os.Stderr, "[SYNC] Starting database synchronization...\n")
+
+	// Step 1: Force PR sync (pass true explicitly)
+	fmt.Fprintf(os.Stderr, "[PR_SYNC] Forcing PR sync from GitHub...\n")
+	if err := g.fetchPRs(true); err != nil {
+		return fmt.Errorf("failed to sync PRs: %w", err)
+	}
+
+	// Step 2: Rebuild git history and verify versions/commits completeness
+	fmt.Fprintf(os.Stderr, "[VERIFY] Verifying git history and version completeness...\n")
+	if err := g.syncGitHistory(); err != nil {
+		return fmt.Errorf("failed to sync git history: %w", err)
+	}
+
+	// Step 3: Verify commit-PR mappings
+	fmt.Fprintf(os.Stderr, "[MAPPING] Verifying commit-PR mappings...\n")
+	if err := g.verifyCommitPRMappings(); err != nil {
+		return fmt.Errorf("failed to verify commit-PR mappings: %w", err)
+	}
+
+	fmt.Fprintf(os.Stderr, "[SUCCESS] Database synchronization completed successfully!\n")
+	return nil
+}
+
+// syncGitHistory walks the complete git history and ensures all versions and commits are cached
+func (g *Generator) syncGitHistory() error {
+	// Walk complete git history (reuse existing logic)
+	versions, err := g.gitWalker.WalkHistory()
+	if err != nil {
+		return fmt.Errorf("failed to walk git history: %w", err)
+	}
+
+	// Save only new versions and commits (preserve existing data)
+	var newVersions, newCommits int
+	for _, version := range versions {
+		// Only save version if it doesn't exist
+		exists, err := g.cache.VersionExists(version.Name)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to check existence of version %s: %v. This may affect the completeness of the sync operation.\n", version.Name, err)
+			continue
+		}
+		if !exists {
+			if err := g.cache.SaveVersion(version); err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to save version %s: %v\n", version.Name, err)
+			} else {
+				newVersions++
+			}
+		}
+
+		// Only save commits that don't exist
+		for _, commit := range version.Commits {
+			exists, err := g.cache.CommitExists(commit.SHA)
+			if err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to check commit %s existence: %v\n", commit.SHA, err)
+				continue
+			}
+			if !exists {
+				if err := g.cache.SaveCommit(commit, version.Name); err != nil {
+					fmt.Fprintf(os.Stderr, "Warning: Failed to save commit %s: %v\n", commit.SHA, err)
+				} else {
+					newCommits++
+				}
+			}
+		}
+	}
+
+	// Update last processed tag
+	if latestTag, err := g.gitWalker.GetLatestTag(); err == nil && latestTag != "" {
+		if err := g.cache.SetLastProcessedTag(latestTag); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to update last processed tag: %v\n", err)
+		}
+	}
+
+	fmt.Fprintf(os.Stderr, "   Added %d new versions and %d new commits (preserved existing data)\n", newVersions, newCommits)
+	return nil
+}
+
+// verifyCommitPRMappings ensures all PR commits have proper mappings
+func (g *Generator) verifyCommitPRMappings() error {
+	// Get all cached PRs
+	allPRs, err := g.cache.GetAllPRs()
+	if err != nil {
+		return fmt.Errorf("failed to get cached PRs: %w", err)
+	}
+
+	// Convert to slice for batch operations (reuse existing logic)
+	var prSlice []*github.PR
+	for _, pr := range allPRs {
+		prSlice = append(prSlice, pr)
+	}
+
+	// Save commit-PR mappings (reuse existing logic)
+	if err := g.cache.SaveCommitPRMappings(prSlice); err != nil {
+		return fmt.Errorf("failed to save commit-PR mappings: %w", err)
+	}
+
+	fmt.Fprintf(os.Stderr, "   Verified mappings for %d PRs\n", len(prSlice))
+	return nil
+}

cmd/generate_changelog/internal/changelog/generator_test.go

@@ -0,0 +1,115 @@
+package changelog
+
+import (
+	"os"
+	"path/filepath"
+	"regexp"
+	"testing"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/config"
+)
+
+func TestDetectVersionFromNix(t *testing.T) {
+	tempDir := t.TempDir()
+
+	t.Run("version.nix exists", func(t *testing.T) {
+		versionNixContent := `"1.2.3"`
+		versionNixPath := filepath.Join(tempDir, "version.nix")
+		err := os.WriteFile(versionNixPath, []byte(versionNixContent), 0644)
+		if err != nil {
+			t.Fatalf("Failed to write version.nix: %v", err)
+		}
+
+		data, err := os.ReadFile(versionNixPath)
+		if err != nil {
+			t.Fatalf("Failed to read version.nix: %v", err)
+		}
+
+		versionRegex := regexp.MustCompile(`"([^"]+)"`)
+		matches := versionRegex.FindStringSubmatch(string(data))
+
+		if len(matches) <= 1 {
+			t.Fatalf("No version found in version.nix")
+		}
+
+		version := matches[1]
+		if version != "1.2.3" {
+			t.Errorf("Expected version 1.2.3, got %s", version)
+		}
+	})
+}
+
+func TestEnsureIncomingDir(t *testing.T) {
+	tempDir := t.TempDir()
+	incomingDir := filepath.Join(tempDir, "incoming")
+
+	cfg := &config.Config{
+		IncomingDir: incomingDir,
+	}
+
+	g := &Generator{cfg: cfg}
+
+	err := g.ensureIncomingDir()
+	if err != nil {
+		t.Fatalf("ensureIncomingDir failed: %v", err)
+	}
+
+	if _, err := os.Stat(incomingDir); os.IsNotExist(err) {
+		t.Errorf("Incoming directory was not created")
+	}
+}
+
+func TestInsertVersionAtTop(t *testing.T) {
+	tempDir := t.TempDir()
+	changelogPath := filepath.Join(tempDir, "CHANGELOG.md")
+
+	cfg := &config.Config{
+		RepoPath: tempDir,
+	}
+
+	g := &Generator{cfg: cfg}
+
+	t.Run("new changelog", func(t *testing.T) {
+		entry := "## v1.0.0 (2025-01-01)\n\n- Initial release"
+
+		err := g.insertVersionAtTop(entry)
+		if err != nil {
+			t.Fatalf("insertVersionAtTop failed: %v", err)
+		}
+
+		content, err := os.ReadFile(changelogPath)
+		if err != nil {
+			t.Fatalf("Failed to read changelog: %v", err)
+		}
+
+		expected := "# Changelog\n\n## v1.0.0 (2025-01-01)\n\n- Initial release\n"
+		if string(content) != expected {
+			t.Errorf("Expected:\n%s\nGot:\n%s", expected, string(content))
+		}
+	})
+
+	t.Run("existing changelog", func(t *testing.T) {
+		existingContent := "# Changelog\n\n## v0.9.0 (2024-12-01)\n\n- Previous release"
+		err := os.WriteFile(changelogPath, []byte(existingContent), 0644)
+		if err != nil {
+			t.Fatalf("Failed to write existing changelog: %v", err)
+		}
+
+		entry := "## v1.0.0 (2025-01-01)\n\n- New release"
+
+		err = g.insertVersionAtTop(entry)
+		if err != nil {
+			t.Fatalf("insertVersionAtTop failed: %v", err)
+		}
+
+		content, err := os.ReadFile(changelogPath)
+		if err != nil {
+			t.Fatalf("Failed to read changelog: %v", err)
+		}
+
+		expected := "# Changelog\n\n## v1.0.0 (2025-01-01)\n\n- New release\n## v0.9.0 (2024-12-01)\n\n- Previous release"
+		if string(content) != expected {
+			t.Errorf("Expected:\n%s\nGot:\n%s", expected, string(content))
+		}
+	})
+}

cmd/generate_changelog/internal/changelog/merge_detection_test.go

@@ -0,0 +1,82 @@
+package changelog
+
+import (
+	"testing"
+	"time"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/github"
+)
+
+func TestIsMergeCommit(t *testing.T) {
+	tests := []struct {
+		name     string
+		commit   github.PRCommit
+		expected bool
+	}{
+		{
+			name: "Regular commit with single parent",
+			commit: github.PRCommit{
+				SHA:     "abc123",
+				Message: "Fix bug in user authentication",
+				Author:  "John Doe",
+				Date:    time.Now(),
+				Parents: []string{"def456"},
+			},
+			expected: false,
+		},
+		{
+			name: "Merge commit with multiple parents",
+			commit: github.PRCommit{
+				SHA:     "abc123",
+				Message: "Merge pull request #42 from feature/auth",
+				Author:  "GitHub",
+				Date:    time.Now(),
+				Parents: []string{"def456", "ghi789"},
+			},
+			expected: true,
+		},
+		{
+			name: "Merge commit detected by message pattern only",
+			commit: github.PRCommit{
+				SHA:     "abc123",
+				Message: "Merge pull request #123 from user/feature-branch",
+				Author:  "GitHub",
+				Date:    time.Now(),
+				Parents: []string{}, // Empty parents - fallback to message detection
+			},
+			expected: true,
+		},
+		{
+			name: "Merge branch commit pattern",
+			commit: github.PRCommit{
+				SHA:     "abc123",
+				Message: "Merge branch 'feature' into main",
+				Author:  "Developer",
+				Date:    time.Now(),
+				Parents: []string{"def456"}, // Single parent but merge pattern
+			},
+			expected: true,
+		},
+		{
+			name: "Regular commit with no merge patterns",
+			commit: github.PRCommit{
+				SHA:     "abc123",
+				Message: "Add new feature for user management",
+				Author:  "Jane Doe",
+				Date:    time.Now(),
+				Parents: []string{"def456"},
+			},
+			expected: false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isMergeCommit(tt.commit)
+			if result != tt.expected {
+				t.Errorf("isMergeCommit() = %v, expected %v for commit: %s",
+					result, tt.expected, tt.commit.Message)
+			}
+		})
+	}
+}

cmd/generate_changelog/internal/changelog/processing.go

@@ -0,0 +1,544 @@
+package changelog
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/git"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/github"
+)
+
+var (
+	mergePatterns     []*regexp.Regexp
+	mergePatternsOnce sync.Once
+)
+
+// getMergePatterns returns the compiled merge patterns, initializing them lazily
+func getMergePatterns() []*regexp.Regexp {
+	mergePatternsOnce.Do(func() {
+		mergePatterns = []*regexp.Regexp{
+			regexp.MustCompile(`^Merge pull request #\d+`),      // "Merge pull request #123 from..."
+			regexp.MustCompile(`^Merge branch '.*' into .*`),    // "Merge branch 'feature' into main"
+			regexp.MustCompile(`^Merge remote-tracking branch`), // "Merge remote-tracking branch..."
+			regexp.MustCompile(`^Merge '.*' into .*`),           // "Merge 'feature' into main"
+		}
+	})
+	return mergePatterns
+}
+
+// isMergeCommit determines if a commit is a merge commit based on its parents and message patterns.
+func isMergeCommit(commit github.PRCommit) bool {
+	// Primary method: Check parent count (merge commits have multiple parents)
+	if len(commit.Parents) > 1 {
+		return true
+	}
+
+	// Fallback method: Check commit message patterns
+	mergePatterns := getMergePatterns()
+	for _, pattern := range mergePatterns {
+		if pattern.MatchString(commit.Message) {
+			return true
+		}
+	}
+
+	return false
+}
+
+// calculateVersionDate determines the version date based on the most recent commit date from the provided PRs.
+//
+// If no valid commit dates are found, the function falls back to the current time.
+// The function iterates through the provided PRs and their associated commits, comparing commit dates
+// to identify the most recent one. If a valid date is found, it is returned; otherwise, the fallback is used.
+func calculateVersionDate(fetchedPRs []*github.PR) time.Time {
+	versionDate := time.Now() // fallback to current time
+	if len(fetchedPRs) > 0 {
+		var mostRecentCommitDate time.Time
+		for _, pr := range fetchedPRs {
+			for _, commit := range pr.Commits {
+				if commit.Date.After(mostRecentCommitDate) {
+					mostRecentCommitDate = commit.Date
+				}
+			}
+		}
+		if !mostRecentCommitDate.IsZero() {
+			versionDate = mostRecentCommitDate
+		}
+	}
+	return versionDate
+}
+
+// ProcessIncomingPR processes a single PR for changelog entry creation
+func (g *Generator) ProcessIncomingPR(prNumber int) error {
+	if err := g.validatePRState(prNumber); err != nil {
+		return fmt.Errorf("PR validation failed: %w", err)
+	}
+
+	if err := g.validateGitStatus(); err != nil {
+		return fmt.Errorf("git status validation failed: %w", err)
+	}
+
+	// Now fetch the full PR with commits for content generation
+	pr, err := g.ghClient.GetPRWithCommits(prNumber)
+	if err != nil {
+		return fmt.Errorf("failed to fetch PR %d: %w", prNumber, err)
+	}
+
+	content := g.formatPR(pr)
+
+	if g.cfg.EnableAISummary {
+		aiContent, err := SummarizeVersionContent(content)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: AI summarization failed: %v\n", err)
+		} else if !checkForAIError(aiContent) {
+			content = strings.TrimSpace(aiContent)
+		}
+	}
+
+	if err := g.ensureIncomingDir(); err != nil {
+		return fmt.Errorf("failed to create incoming directory: %w", err)
+	}
+
+	filename := filepath.Join(g.cfg.IncomingDir, fmt.Sprintf("%d.txt", prNumber))
+
+	// Ensure content ends with a single newline
+	content = strings.TrimSpace(content) + "\n"
+
+	if err := os.WriteFile(filename, []byte(content), 0644); err != nil {
+		return fmt.Errorf("failed to write incoming file: %w", err)
+	}
+
+	if err := g.commitAndPushIncoming(prNumber, filename); err != nil {
+		return fmt.Errorf("failed to commit and push: %w", err)
+	}
+
+	fmt.Printf("Successfully created incoming changelog entry: %s\n", filename)
+	return nil
+}
+
+// CreateNewChangelogEntry aggregates all incoming PR files for release and includes direct commits
+func (g *Generator) CreateNewChangelogEntry(version string) error {
+	files, err := filepath.Glob(filepath.Join(g.cfg.IncomingDir, "*.txt"))
+	if err != nil {
+		return fmt.Errorf("failed to scan incoming directory: %w", err)
+	}
+
+	var content strings.Builder
+	var processingErrors []string
+
+	// First, aggregate all incoming PR files
+	for i, file := range files {
+		data, err := os.ReadFile(file)
+		if err != nil {
+			processingErrors = append(processingErrors, fmt.Sprintf("failed to read %s: %v", file, err))
+			continue // Continue to attempt processing other files
+		}
+		content.WriteString(string(data))
+		// Add an extra newline between PR sections for proper spacing
+		if i < len(files)-1 {
+			content.WriteString("\n")
+		}
+	}
+
+	if len(processingErrors) > 0 {
+		return fmt.Errorf("encountered errors while processing incoming files: %s", strings.Join(processingErrors, "; "))
+	}
+
+	// Extract PR numbers and their commit SHAs from processed files to avoid including their commits as "direct"
+	processedPRs := make(map[int]bool)
+	processedCommitSHAs := make(map[string]bool)
+	var fetchedPRs []*github.PR
+	var prNumbers []int
+
+	for _, file := range files {
+		// Extract PR number from filename (e.g., "1640.txt" -> 1640)
+		filename := filepath.Base(file)
+		if prNumStr, ok := strings.CutSuffix(filename, ".txt"); ok {
+			if prNum, err := strconv.Atoi(prNumStr); err == nil {
+				processedPRs[prNum] = true
+				prNumbers = append(prNumbers, prNum)
+
+				// Fetch the PR to get its commit SHAs
+				if pr, err := g.ghClient.GetPRWithCommits(prNum); err == nil {
+					fetchedPRs = append(fetchedPRs, pr)
+					for _, commit := range pr.Commits {
+						processedCommitSHAs[commit.SHA] = true
+					}
+				}
+			}
+		}
+	}
+
+	// Now add direct commits since the last release, excluding commits from processed PRs
+	directCommitsContent, err := g.getDirectCommitsSinceLastRelease(processedPRs, processedCommitSHAs)
+	if err != nil {
+		return fmt.Errorf("failed to get direct commits since last release: %w", err)
+	}
+	if directCommitsContent != "" {
+		// Add spacing before direct commits section if we have PR content
+		if content.Len() > 0 {
+			content.WriteString("\n")
+		}
+		content.WriteString(directCommitsContent)
+	}
+
+	// Check if we have any content at all
+	if content.Len() == 0 {
+		if len(files) == 0 {
+			fmt.Fprintf(os.Stderr, "No incoming PR files found in %s and no direct commits since last release\n", g.cfg.IncomingDir)
+		} else {
+			fmt.Fprintf(os.Stderr, "No content found in incoming files and no direct commits since last release\n")
+		}
+		return nil
+	}
+
+	// Calculate the version date for the changelog entry as the most recent commit date from processed PRs
+	versionDate := calculateVersionDate(fetchedPRs)
+
+	entry := fmt.Sprintf("## %s (%s)\n\n%s",
+		version, versionDate.Format("2006-01-02"), strings.TrimLeft(content.String(), "\n"))
+
+	if err := g.insertVersionAtTop(entry); err != nil {
+		return fmt.Errorf("failed to update CHANGELOG.md: %w", err)
+	}
+
+	if g.cache != nil {
+		// Cache the fetched PRs using the same logic as normal changelog generation
+		if len(fetchedPRs) > 0 {
+			// Save PRs to cache
+			if err := g.cache.SavePRBatch(fetchedPRs); err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to save PR batch to cache: %v\n", err)
+			}
+
+			// Save SHA→PR mappings for lightning-fast git operations
+			if err := g.cache.SaveCommitPRMappings(fetchedPRs); err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to cache commit mappings: %v\n", err)
+			}
+
+			// Save individual commits to cache for each PR
+			for _, pr := range fetchedPRs {
+				for _, commit := range pr.Commits {
+					// Use actual commit timestamp, with fallback to current time if invalid
+					commitDate := commit.Date
+					if commitDate.IsZero() {
+						commitDate = time.Now()
+						fmt.Fprintf(os.Stderr, "Warning: Commit %s has invalid timestamp, using current time as fallback\n", commit.SHA)
+					}
+
+					// Convert github.PRCommit to git.Commit
+					gitCommit := &git.Commit{
+						SHA:      commit.SHA,
+						Message:  commit.Message,
+						Author:   commit.Author,
+						Email:    commit.Email,          // Use email from GitHub API
+						Date:     commitDate,            // Use actual commit timestamp from GitHub API
+						IsMerge:  isMergeCommit(commit), // Detect merge commits using parents and message patterns
+						PRNumber: pr.Number,
+					}
+					if err := g.cache.SaveCommit(gitCommit, version); err != nil {
+						fmt.Fprintf(os.Stderr, "Warning: Failed to save commit %s to cache: %v\n", commit.SHA, err)
+					}
+				}
+			}
+		}
+
+		// Create a proper new version entry for the database
+		newVersionEntry := &git.Version{
+			Name:      version,
+			Date:      versionDate, // Use most recent commit date instead of current time
+			CommitSHA: "",          // Will be set when the release commit is made
+			PRNumbers: prNumbers,   // Now we have the actual PR numbers
+			AISummary: content.String(),
+		}
+
+		if err := g.cache.SaveVersion(newVersionEntry); err != nil {
+			return fmt.Errorf("failed to save new version entry to database: %w", err)
+		}
+	}
+
+	for _, file := range files {
+		// Convert to relative path for git operations
+		relativeFile, err := filepath.Rel(g.cfg.RepoPath, file)
+		if err != nil {
+			relativeFile = file
+		}
+
+		// Use git remove to handle both filesystem and git index
+		if err := g.gitWalker.RemoveFile(relativeFile); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to remove %s from git index: %v\n", relativeFile, err)
+			// Fallback to filesystem-only removal
+			if err := os.Remove(file); err != nil {
+				fmt.Fprintf(os.Stderr, "Error: Failed to remove %s from the filesystem after failing to remove it from the git index.\n", relativeFile)
+				fmt.Fprintf(os.Stderr, "Filesystem error: %v\n", err)
+				fmt.Fprintf(os.Stderr, "Manual intervention required:\n")
+				fmt.Fprintf(os.Stderr, "  1. Remove the file %s manually (using the OS-specific command)\n", file)
+				fmt.Fprintf(os.Stderr, "  2. Remove from git index: git rm --cached %s\n", relativeFile)
+				fmt.Fprintf(os.Stderr, "  3. Or reset git index: git reset HEAD %s\n", relativeFile)
+			}
+		}
+	}
+
+	// Update metadata before staging changes so they get committed together
+	if g.cache != nil {
+		// Update last_processed_tag to the version we just processed
+		if err := g.cache.SetLastProcessedTag(version); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to update last_processed_tag: %v\n", err)
+		}
+
+		// Update last_pr_sync to the version date (not current time)
+		// This ensures future runs will fetch PRs merged after this version
+		if err := g.cache.SetLastPRSync(versionDate); err != nil {
+			fmt.Fprintf(os.Stderr, "Warning: Failed to update last_pr_sync: %v\n", err)
+		}
+	}
+
+	if err := g.stageChangesForRelease(); err != nil {
+		return fmt.Errorf("critical: failed to stage changes for release: %w", err)
+	}
+
+	fmt.Printf("Successfully processed %d incoming PR files for version %s\n", len(files), version)
+	return nil
+}
+
+// getDirectCommitsSinceLastRelease gets all direct commits (not part of PRs) since the last release
+func (g *Generator) getDirectCommitsSinceLastRelease(processedPRs map[int]bool, processedCommitSHAs map[string]bool) (string, error) {
+	// Get the latest tag to determine what commits are unreleased
+	latestTag, err := g.gitWalker.GetLatestTag()
+	if err != nil {
+		return "", fmt.Errorf("failed to get latest tag: %w", err)
+	}
+
+	// Get all commits since the latest tag
+	unreleasedVersion, err := g.gitWalker.WalkCommitsSinceTag(latestTag)
+	if err != nil {
+		return "", fmt.Errorf("failed to walk commits since tag %s: %w", latestTag, err)
+	}
+
+	if unreleasedVersion == nil || len(unreleasedVersion.Commits) == 0 {
+		return "", nil // No unreleased commits
+	}
+
+	// Filter out commits that are part of PRs (we already have those from incoming files)
+	// and format the direct commits
+	var directCommits []*git.Commit
+	for _, commit := range unreleasedVersion.Commits {
+		// Skip version bump commits
+		if commit.IsVersion {
+			continue
+		}
+
+		// Skip commits that belong to PRs we've already processed from incoming files (by PR number)
+		if commit.PRNumber > 0 && processedPRs[commit.PRNumber] {
+			continue
+		}
+
+		// Skip commits whose SHA is already included in processed PRs (this catches commits
+		// that might not have been detected as part of a PR but are actually in the PR)
+		if processedCommitSHAs[commit.SHA] {
+			continue
+		}
+
+		// Only include commits that are NOT part of any PR (direct commits)
+		if commit.PRNumber == 0 {
+			directCommits = append(directCommits, commit)
+		}
+	}
+
+	if len(directCommits) == 0 {
+		return "", nil // No direct commits
+	}
+
+	// Format the direct commits similar to how it's done in generateRawVersionContent
+	var sb strings.Builder
+	sb.WriteString("### Direct commits\n\n")
+
+	// Sort direct commits by date (newest first) for consistent ordering
+	sort.Slice(directCommits, func(i, j int) bool {
+		return directCommits[i].Date.After(directCommits[j].Date)
+	})
+
+	for _, commit := range directCommits {
+		message := g.formatCommitMessage(strings.TrimSpace(commit.Message))
+		if message != "" && !g.isDuplicateMessage(message, directCommits) {
+			sb.WriteString(fmt.Sprintf("- %s\n", message))
+		}
+	}
+
+	return sb.String(), nil
+}
+
+// validatePRState validates that a PR is in the correct state for processing
+func (g *Generator) validatePRState(prNumber int) error {
+	// Use lightweight validation call that doesn't fetch commits
+	details, err := g.ghClient.GetPRValidationDetails(prNumber)
+	if err != nil {
+		return fmt.Errorf("failed to fetch PR %d: %w", prNumber, err)
+	}
+
+	if details.State != "open" {
+		return fmt.Errorf("PR %d is not open (current state: %s)", prNumber, details.State)
+	}
+
+	if !details.Mergeable {
+		return fmt.Errorf("PR %d is not mergeable - please resolve conflicts first", prNumber)
+	}
+
+	return nil
+}
+
+// validateGitStatus ensures the working directory is clean
+func (g *Generator) validateGitStatus() error {
+	isClean, err := g.gitWalker.IsWorkingDirectoryClean()
+	if err != nil {
+		return fmt.Errorf("failed to check git status: %w", err)
+	}
+
+	if !isClean {
+		// Get detailed status for better error message
+		statusDetails, statusErr := g.gitWalker.GetStatusDetails()
+		if statusErr == nil && statusDetails != "" {
+			return fmt.Errorf("working directory is not clean - please commit or stash changes before proceeding:\n%s", statusDetails)
+		}
+		return fmt.Errorf("working directory is not clean - please commit or stash changes before proceeding")
+	}
+
+	return nil
+}
+
+// ensureIncomingDir creates the incoming directory if it doesn't exist
+func (g *Generator) ensureIncomingDir() error {
+	if err := os.MkdirAll(g.cfg.IncomingDir, 0755); err != nil {
+		return fmt.Errorf("failed to create directory %s: %w", g.cfg.IncomingDir, err)
+	}
+	return nil
+}
+
+// commitAndPushIncoming commits and optionally pushes the incoming changelog file
+func (g *Generator) commitAndPushIncoming(prNumber int, filename string) error {
+	relativeFilename, err := filepath.Rel(g.cfg.RepoPath, filename)
+	if err != nil {
+		relativeFilename = filename
+	}
+
+	// Add file to git index
+	if err := g.gitWalker.AddFile(relativeFilename); err != nil {
+		return fmt.Errorf("failed to add file %s: %w", relativeFilename, err)
+	}
+
+	// Commit changes
+	commitMessage := fmt.Sprintf("chore: incoming %d changelog entry", prNumber)
+	_, err = g.gitWalker.CommitChanges(commitMessage)
+	if err != nil {
+		return fmt.Errorf("failed to commit changes: %w", err)
+	}
+
+	// Push to remote if enabled
+	if g.cfg.Push {
+		if err := g.gitWalker.PushToRemote(); err != nil {
+			return fmt.Errorf("failed to push to remote: %w", err)
+		}
+	} else {
+		fmt.Println("Commit created successfully. Please review and push manually.")
+	}
+
+	return nil
+}
+
+// detectVersion detects the current version from version.nix or git tags
+func (g *Generator) detectVersion() (string, error) {
+	versionNixPath := filepath.Join(g.cfg.RepoPath, "version.nix")
+	if _, err := os.Stat(versionNixPath); err == nil {
+		data, err := os.ReadFile(versionNixPath)
+		if err != nil {
+			return "", fmt.Errorf("failed to read version.nix: %w", err)
+		}
+
+		versionRegex := regexp.MustCompile(`"([^"]+)"`)
+		matches := versionRegex.FindStringSubmatch(string(data))
+		if len(matches) > 1 {
+			return matches[1], nil
+		}
+	}
+
+	latestTag, err := g.gitWalker.GetLatestTag()
+	if err != nil {
+		return "", fmt.Errorf("failed to get latest tag: %w", err)
+	}
+
+	if latestTag == "" {
+		return "v1.0.0", nil
+	}
+
+	return latestTag, nil
+}
+
+// insertVersionAtTop inserts a new version entry at the top of CHANGELOG.md
+func (g *Generator) insertVersionAtTop(entry string) error {
+	changelogPath := filepath.Join(g.cfg.RepoPath, "CHANGELOG.md")
+	header := "# Changelog"
+	headerRegex := regexp.MustCompile(`(?m)^# Changelog\s*`)
+
+	existingContent, err := os.ReadFile(changelogPath)
+	if err != nil {
+		if !os.IsNotExist(err) {
+			return fmt.Errorf("failed to read existing CHANGELOG.md: %w", err)
+		}
+		// File doesn't exist, create it.
+		newContent := fmt.Sprintf("%s\n\n%s\n", header, entry)
+		return os.WriteFile(changelogPath, []byte(newContent), 0644)
+	}
+
+	contentStr := string(existingContent)
+	var newContent string
+
+	if loc := headerRegex.FindStringIndex(contentStr); loc != nil {
+		// Found the header, insert after it.
+		insertionPoint := loc[1]
+		// Skip any existing newlines after the header to avoid double spacing
+		for insertionPoint < len(contentStr) && (contentStr[insertionPoint] == '\n' || contentStr[insertionPoint] == '\r') {
+			insertionPoint++
+		}
+		// Insert with proper spacing: single newline after header, then entry, then newline before existing content
+		newContent = contentStr[:loc[1]] + entry + "\n" + contentStr[insertionPoint:]
+	} else {
+		// Header not found, prepend everything.
+		newContent = fmt.Sprintf("%s\n\n%s\n\n%s", header, entry, contentStr)
+	}
+
+	return os.WriteFile(changelogPath, []byte(newContent), 0644)
+}
+
+// stageChangesForRelease stages the modified files for the release commit
+func (g *Generator) stageChangesForRelease() error {
+	changelogPath := filepath.Join(g.cfg.RepoPath, "CHANGELOG.md")
+	relativeChangelog, err := filepath.Rel(g.cfg.RepoPath, changelogPath)
+	if err != nil {
+		relativeChangelog = "CHANGELOG.md"
+	}
+
+	relativeCacheFile, err := filepath.Rel(g.cfg.RepoPath, g.cfg.CacheFile)
+	if err != nil {
+		relativeCacheFile = g.cfg.CacheFile
+	}
+
+	// Add CHANGELOG.md to git index
+	if err := g.gitWalker.AddFile(relativeChangelog); err != nil {
+		return fmt.Errorf("failed to add %s: %w", relativeChangelog, err)
+	}
+
+	// Add cache file to git index
+	if err := g.gitWalker.AddFile(relativeCacheFile); err != nil {
+		return fmt.Errorf("failed to add %s: %w", relativeCacheFile, err)
+	}
+
+	// Note: Individual incoming files are now removed during the main processing loop
+	// No need to remove the entire directory here
+
+	return nil
+}

cmd/generate_changelog/internal/changelog/processing_test.go

@@ -0,0 +1,262 @@
+package changelog
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/config"
+)
+
+func TestDetectVersion(t *testing.T) {
+	tempDir := t.TempDir()
+
+	tests := []struct {
+		name              string
+		versionNixContent string
+		expectedVersion   string
+		shouldError       bool
+	}{
+		{
+			name:              "valid version.nix",
+			versionNixContent: `"1.2.3"`,
+			expectedVersion:   "1.2.3",
+			shouldError:       false,
+		},
+		{
+			name:              "version with extra whitespace",
+			versionNixContent: `"1.2.3"   `,
+			expectedVersion:   "1.2.3",
+			shouldError:       false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Create version.nix file
+			versionNixPath := filepath.Join(tempDir, "version.nix")
+			if err := os.WriteFile(versionNixPath, []byte(tt.versionNixContent), 0644); err != nil {
+				t.Fatalf("Failed to create version.nix: %v", err)
+			}
+
+			cfg := &config.Config{
+				RepoPath: tempDir,
+			}
+
+			g := &Generator{cfg: cfg}
+
+			version, err := g.detectVersion()
+			if tt.shouldError && err == nil {
+				t.Errorf("Expected error but got none")
+			}
+			if !tt.shouldError && err != nil {
+				t.Errorf("Unexpected error: %v", err)
+			}
+			if version != tt.expectedVersion {
+				t.Errorf("Expected version '%s', got '%s'", tt.expectedVersion, version)
+			}
+
+			// Clean up
+			os.Remove(versionNixPath)
+		})
+	}
+}
+
+func TestInsertVersionAtTop_ImprovedRobustness(t *testing.T) {
+	tempDir := t.TempDir()
+	changelogPath := filepath.Join(tempDir, "CHANGELOG.md")
+
+	cfg := &config.Config{
+		RepoPath: tempDir,
+	}
+
+	g := &Generator{cfg: cfg}
+
+	tests := []struct {
+		name            string
+		existingContent string
+		entry           string
+		expectedContent string
+	}{
+		{
+			name:            "header with trailing spaces",
+			existingContent: "# Changelog   \n\n## v1.0.0\n- Old content",
+			entry:           "## v2.0.0\n- New content",
+			expectedContent: "# Changelog   \n\n## v2.0.0\n- New content\n## v1.0.0\n- Old content",
+		},
+		{
+			name:            "header with different line endings",
+			existingContent: "# Changelog\r\n\r\n## v1.0.0\r\n- Old content",
+			entry:           "## v2.0.0\n- New content",
+			expectedContent: "# Changelog\r\n\r\n## v2.0.0\n- New content\n## v1.0.0\r\n- Old content",
+		},
+		{
+			name:            "no existing header",
+			existingContent: "Some existing content without header",
+			entry:           "## v1.0.0\n- New content",
+			expectedContent: "# Changelog\n\n## v1.0.0\n- New content\n\nSome existing content without header",
+		},
+		{
+			name:            "new file creation",
+			existingContent: "",
+			entry:           "## v1.0.0\n- Initial release",
+			expectedContent: "# Changelog\n\n## v1.0.0\n- Initial release\n",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Write existing content (or create empty file)
+			if tt.existingContent != "" {
+				if err := os.WriteFile(changelogPath, []byte(tt.existingContent), 0644); err != nil {
+					t.Fatalf("Failed to write existing content: %v", err)
+				}
+			} else {
+				// Remove file if it exists to test new file creation
+				os.Remove(changelogPath)
+			}
+
+			// Insert new version
+			if err := g.insertVersionAtTop(tt.entry); err != nil {
+				t.Fatalf("insertVersionAtTop failed: %v", err)
+			}
+
+			// Read result
+			result, err := os.ReadFile(changelogPath)
+			if err != nil {
+				t.Fatalf("Failed to read result: %v", err)
+			}
+
+			if string(result) != tt.expectedContent {
+				t.Errorf("Expected:\n%q\nGot:\n%q", tt.expectedContent, string(result))
+			}
+		})
+	}
+}
+
+func TestProcessIncomingPRs_FileAggregation(t *testing.T) {
+	tempDir := t.TempDir()
+	incomingDir := filepath.Join(tempDir, "incoming")
+
+	// Create incoming directory and files
+	if err := os.MkdirAll(incomingDir, 0755); err != nil {
+		t.Fatalf("Failed to create incoming dir: %v", err)
+	}
+
+	// Create test incoming files
+	file1Content := "## PR #1\n- Feature A"
+	file2Content := "## PR #2\n- Feature B"
+
+	if err := os.WriteFile(filepath.Join(incomingDir, "1.txt"), []byte(file1Content), 0644); err != nil {
+		t.Fatalf("Failed to create test file: %v", err)
+	}
+	if err := os.WriteFile(filepath.Join(incomingDir, "2.txt"), []byte(file2Content), 0644); err != nil {
+		t.Fatalf("Failed to create test file: %v", err)
+	}
+
+	// Test file aggregation logic by calling the internal functions
+	files, err := filepath.Glob(filepath.Join(incomingDir, "*.txt"))
+	if err != nil {
+		t.Fatalf("Failed to glob files: %v", err)
+	}
+
+	if len(files) != 2 {
+		t.Fatalf("Expected 2 files, got %d", len(files))
+	}
+
+	// Test content aggregation
+	var content strings.Builder
+	var processingErrors []string
+	for _, file := range files {
+		data, err := os.ReadFile(file)
+		if err != nil {
+			processingErrors = append(processingErrors, err.Error())
+			continue
+		}
+		content.WriteString(string(data))
+		content.WriteString("\n")
+	}
+
+	if len(processingErrors) > 0 {
+		t.Fatalf("Unexpected processing errors: %v", processingErrors)
+	}
+
+	aggregatedContent := content.String()
+	if !strings.Contains(aggregatedContent, "Feature A") {
+		t.Errorf("Aggregated content should contain 'Feature A'")
+	}
+	if !strings.Contains(aggregatedContent, "Feature B") {
+		t.Errorf("Aggregated content should contain 'Feature B'")
+	}
+}
+
+func TestFileProcessing_ErrorHandling(t *testing.T) {
+	tempDir := t.TempDir()
+	incomingDir := filepath.Join(tempDir, "incoming")
+
+	// Create incoming directory with one good file and one unreadable file
+	if err := os.MkdirAll(incomingDir, 0755); err != nil {
+		t.Fatalf("Failed to create incoming dir: %v", err)
+	}
+
+	// Create a good file
+	if err := os.WriteFile(filepath.Join(incomingDir, "1.txt"), []byte("content"), 0644); err != nil {
+		t.Fatalf("Failed to create test file: %v", err)
+	}
+
+	// Create an unreadable file (simulate permission error)
+	unreadableFile := filepath.Join(incomingDir, "2.txt")
+	if err := os.WriteFile(unreadableFile, []byte("content"), 0000); err != nil {
+		t.Fatalf("Failed to create unreadable file: %v", err)
+	}
+	defer os.Chmod(unreadableFile, 0644) // Clean up
+
+	// Test error aggregation logic
+	files, err := filepath.Glob(filepath.Join(incomingDir, "*.txt"))
+	if err != nil {
+		t.Fatalf("Failed to glob files: %v", err)
+	}
+
+	var content strings.Builder
+	var processingErrors []string
+	for _, file := range files {
+		data, err := os.ReadFile(file)
+		if err != nil {
+			processingErrors = append(processingErrors, err.Error())
+			continue
+		}
+		content.WriteString(string(data))
+		content.WriteString("\n")
+	}
+
+	if len(processingErrors) == 0 {
+		t.Errorf("Expected processing errors due to unreadable file")
+	}
+
+	// Verify error message format
+	errorMsg := strings.Join(processingErrors, "; ")
+	if !strings.Contains(errorMsg, "2.txt") {
+		t.Errorf("Error message should mention the problematic file")
+	}
+}
+
+func TestEnsureIncomingDirCreation(t *testing.T) {
+	tempDir := t.TempDir()
+	incomingDir := filepath.Join(tempDir, "incoming")
+
+	cfg := &config.Config{
+		IncomingDir: incomingDir,
+	}
+
+	g := &Generator{cfg: cfg}
+
+	err := g.ensureIncomingDir()
+	if err != nil {
+		t.Fatalf("ensureIncomingDir failed: %v", err)
+	}
+
+	if _, err := os.Stat(incomingDir); os.IsNotExist(err) {
+		t.Errorf("Incoming directory was not created")
+	}
+}

cmd/generate_changelog/internal/changelog/summarize.go

@@ -0,0 +1,79 @@
+package changelog
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"strings"
+)
+
+const DefaultSummarizeModel = "claude-sonnet-4-20250514"
+const MinContentLength = 256 // Minimum content length to consider for summarization
+
+const prompt = `# ROLE
+You are an expert Technical Writer specializing in creating clear, concise,
+and professional release notes from raw Git commit logs.
+
+# TASK
+Your goal is to transform a provided block of Git commit logs into a clean,
+human-readable changelog summary. You will identify the most important changes,
+format them as a bulleted list, and preserve the associated Pull Request (PR)
+information.
+
+# INSTRUCTIONS:
+Follow these steps in order:
+1. Deeply analyze the input. You will be given a block of text containing PR
+   information and commit log messages. Carefully read through the logs
+   to identify individual commits and their descriptions.
+2. Identify Key Changes: Focus on commits that represent significant changes,
+   such as new features ("feat"), bug fixes ("fix"), performance improvements ("perf"),
+   or breaking changes ("BREAKING CHANGE").
+3. Select the Top 5: From the identified key changes, select a maximum of the five
+   (5) most impactful ones to include in the summary.
+   If there are five or fewer total changes, include all of them.
+4. Format the Output:
+    - Where you see a PR header, include the PR header verbatim. NO CHANGES.
+	  **This is a critical rule: Do not modify the PR header, as it contains
+	  important links.** What follow the PR header are the related changes.
+	- Do not add any additional text or preamble. Begin directly with the output.
+	- Use bullet points for each key change. Starting each point with a hyphen ("-").
+	- Ensure that the summary is concise and focused on the main changes.
+	- The summary should be in American English (en-US), using proper grammar and punctuation.
+5. If the content is too brief or you do not see any PR headers, return the content as is.
+`
+
+// getSummarizeModel returns the model to use for AI summarization
+func getSummarizeModel() string {
+	if model := os.Getenv("FABRIC_CHANGELOG_SUMMARIZE_MODEL"); model != "" {
+		return model
+	}
+	return DefaultSummarizeModel
+}
+
+// SummarizeVersionContent takes raw version content and returns AI-enhanced summary
+func SummarizeVersionContent(content string) (string, error) {
+	if strings.TrimSpace(content) == "" {
+		return "", fmt.Errorf("no content to summarize")
+	}
+	if len(content) < MinContentLength {
+		// If content is too brief, return it as is
+		return content, nil
+	}
+
+	model := getSummarizeModel()
+
+	cmd := exec.Command("fabric", "-m", model, prompt)
+	cmd.Stdin = strings.NewReader(content)
+
+	output, err := cmd.Output()
+	if err != nil {
+		return "", fmt.Errorf("fabric command failed: %w", err)
+	}
+
+	summary := strings.TrimSpace(string(output))
+	if summary == "" {
+		return "", fmt.Errorf("fabric returned empty summary")
+	}
+
+	return summary, nil
+}

cmd/generate_changelog/internal/config/config.go

@@ -0,0 +1,21 @@
+package config
+
+type Config struct {
+	RepoPath          string
+	OutputFile        string
+	Limit             int
+	Version           string
+	SaveData          bool
+	CacheFile         string
+	NoCache           bool
+	RebuildCache      bool
+	GitHubToken       string
+	ForcePRSync       bool
+	EnableAISummary   bool
+	IncomingPR        int
+	ProcessPRsVersion string
+	IncomingDir       string
+	Push              bool
+	SyncDB            bool
+	Release           string
+}

cmd/generate_changelog/internal/git/types.go

@@ -0,0 +1,26 @@
+package git
+
+import (
+	"time"
+)
+
+type Commit struct {
+	SHA       string
+	Message   string
+	Author    string
+	Email     string
+	Date      time.Time
+	IsMerge   bool
+	PRNumber  int
+	IsVersion bool
+	Version   string
+}
+
+type Version struct {
+	Name      string
+	Date      time.Time
+	CommitSHA string
+	Commits   []*Commit
+	PRNumbers []int
+	AISummary string
+}

cmd/generate_changelog/internal/git/walker.go

@@ -0,0 +1,595 @@
+package git
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/util"
+	"github.com/go-git/go-git/v5"
+	"github.com/go-git/go-git/v5/plumbing"
+	"github.com/go-git/go-git/v5/plumbing/object"
+	"github.com/go-git/go-git/v5/plumbing/storer"
+	"github.com/go-git/go-git/v5/plumbing/transport/http"
+)
+
+var (
+	// The versionPattern matches version commit messages with or without the optional "chore(release): " prefix.
+	// Examples of matching commit messages:
+	//   - "chore(release): Update version to v1.2.3"
+	//   - "Update version to v1.2.3"
+	// Examples of non-matching commit messages:
+	//   - "fix: Update version to v1.2.3" (missing "chore(release): " or "Update version to")
+	//   - "chore(release): Update version to 1.2.3" (missing "v" prefix in version)
+	//   - "Update version to v1.2" (incomplete version number)
+	versionPattern = regexp.MustCompile(`(?:chore\(release\): )?Update version to (v\d+\.\d+\.\d+)`)
+	prPattern      = regexp.MustCompile(`Merge pull request #(\d+)`)
+)
+
+type Walker struct {
+	repo *git.Repository
+}
+
+func NewWalker(repoPath string) (*Walker, error) {
+	repo, err := git.PlainOpen(repoPath)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open repository: %w", err)
+	}
+
+	return &Walker{repo: repo}, nil
+}
+
+// GetLatestTag returns the name of the most recent tag by committer date
+func (w *Walker) GetLatestTag() (string, error) {
+	tagRefs, err := w.repo.Tags()
+	if err != nil {
+		return "", err
+	}
+
+	var latestTagCommit *object.Commit
+	var latestTagName string
+
+	err = tagRefs.ForEach(func(tagRef *plumbing.Reference) error {
+		revision := plumbing.Revision(tagRef.Name().String())
+		tagCommitHash, err := w.repo.ResolveRevision(revision)
+		if err != nil {
+			return err
+		}
+
+		commit, err := w.repo.CommitObject(*tagCommitHash)
+		if err != nil {
+			return err
+		}
+
+		if latestTagCommit == nil {
+			latestTagCommit = commit
+			latestTagName = tagRef.Name().Short() // Get short name like "v1.4.245"
+		}
+
+		if commit.Committer.When.After(latestTagCommit.Committer.When) {
+			latestTagCommit = commit
+			latestTagName = tagRef.Name().Short()
+		}
+
+		return nil
+	})
+	if err != nil {
+		return "", err
+	}
+
+	return latestTagName, nil
+}
+
+// WalkCommitsSinceTag walks commits from the specified tag to HEAD and returns only "Unreleased" version
+func (w *Walker) WalkCommitsSinceTag(tagName string) (*Version, error) {
+	// Get the tag reference
+	tagRef, err := w.repo.Tag(tagName)
+	if err != nil {
+		return nil, fmt.Errorf("failed to find tag %s: %w", tagName, err)
+	}
+
+	// Get the commit that the tag points to
+	tagCommit, err := w.repo.CommitObject(tagRef.Hash())
+	if err != nil {
+		return nil, fmt.Errorf("failed to get tag commit: %w", err)
+	}
+
+	// Get HEAD
+	headRef, err := w.repo.Head()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get HEAD: %w", err)
+	}
+
+	// Walk from HEAD back to the tag commit (exclusive)
+	commitIter, err := w.repo.Log(&git.LogOptions{
+		From:  headRef.Hash(),
+		Order: git.LogOrderCommitterTime,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("failed to get commit log: %w", err)
+	}
+
+	version := &Version{
+		Name:    "Unreleased",
+		Commits: []*Commit{},
+	}
+
+	prNumbers := []int{}
+
+	err = commitIter.ForEach(func(c *object.Commit) error {
+		// Stop when we reach the tag commit (don't include it)
+		if c.Hash == tagCommit.Hash {
+			return fmt.Errorf("reached tag commit") // Use error to break out of iteration
+		}
+
+		commit := &Commit{
+			SHA:     c.Hash.String(),
+			Message: strings.TrimSpace(c.Message),
+			Date:    c.Committer.When,
+		}
+
+		// Check for version patterns
+		if versionMatch := versionPattern.FindStringSubmatch(commit.Message); versionMatch != nil {
+			commit.IsVersion = true
+		}
+
+		// Check for PR merge patterns
+		if prMatch := prPattern.FindStringSubmatch(commit.Message); prMatch != nil {
+			if prNumber, err := strconv.Atoi(prMatch[1]); err == nil {
+				commit.PRNumber = prNumber
+				prNumbers = append(prNumbers, prNumber)
+			}
+		}
+
+		version.Commits = append(version.Commits, commit)
+		return nil
+	})
+
+	// Ignore the "reached tag commit" error - it's expected
+	if err != nil && !strings.Contains(err.Error(), "reached tag commit") {
+		return nil, fmt.Errorf("failed to walk commits: %w", err)
+	}
+
+	// Remove duplicates from prNumbers and set them
+	prNumbersMap := make(map[int]bool)
+	for _, prNum := range prNumbers {
+		prNumbersMap[prNum] = true
+	}
+
+	version.PRNumbers = make([]int, 0, len(prNumbersMap))
+	for prNum := range prNumbersMap {
+		version.PRNumbers = append(version.PRNumbers, prNum)
+	}
+
+	return version, nil
+}
+
+func (w *Walker) WalkHistory() (map[string]*Version, error) {
+	ref, err := w.repo.Head()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get HEAD: %w", err)
+	}
+
+	commitIter, err := w.repo.Log(&git.LogOptions{
+		From:  ref.Hash(),
+		Order: git.LogOrderCommitterTime,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("failed to get commit log: %w", err)
+	}
+
+	versions := make(map[string]*Version)
+	currentVersion := "Unreleased"
+	versions[currentVersion] = &Version{
+		Name:    currentVersion,
+		Commits: []*Commit{},
+	}
+
+	prNumbers := make(map[string][]int)
+
+	err = commitIter.ForEach(func(c *object.Commit) error {
+		// c.Message = Summarize(c.Message)
+		commit := &Commit{
+			SHA:     c.Hash.String(),
+			Message: strings.TrimSpace(c.Message),
+			Author:  c.Author.Name,
+			Email:   c.Author.Email,
+			Date:    c.Author.When,
+			IsMerge: len(c.ParentHashes) > 1,
+		}
+
+		if matches := versionPattern.FindStringSubmatch(commit.Message); len(matches) > 1 {
+			commit.IsVersion = true
+			commit.Version = matches[1]
+			currentVersion = commit.Version
+
+			if _, exists := versions[currentVersion]; !exists {
+				versions[currentVersion] = &Version{
+					Name:      currentVersion,
+					Date:      commit.Date,
+					CommitSHA: commit.SHA,
+					Commits:   []*Commit{},
+				}
+			}
+			return nil
+		}
+
+		if matches := prPattern.FindStringSubmatch(commit.Message); len(matches) > 1 {
+			prNumber := 0
+			fmt.Sscanf(matches[1], "%d", &prNumber)
+			commit.PRNumber = prNumber
+
+			prNumbers[currentVersion] = append(prNumbers[currentVersion], prNumber)
+		}
+
+		versions[currentVersion].Commits = append(versions[currentVersion].Commits, commit)
+
+		return nil
+	})
+
+	if err != nil {
+		return nil, fmt.Errorf("failed to walk commits: %w", err)
+	}
+
+	for version, prs := range prNumbers {
+		versions[version].PRNumbers = dedupInts(prs)
+	}
+
+	return versions, nil
+}
+
+func (w *Walker) GetRepoInfo() (owner string, name string, err error) {
+	remotes, err := w.repo.Remotes()
+	if err != nil {
+		return "", "", fmt.Errorf("failed to get remotes: %w", err)
+	}
+
+	// First try upstream (preferred for forks)
+	for _, remote := range remotes {
+		if remote.Config().Name == "upstream" {
+			urls := remote.Config().URLs
+			if len(urls) > 0 {
+				owner, name = parseGitHubURL(urls[0])
+				if owner != "" && name != "" {
+					return owner, name, nil
+				}
+			}
+		}
+	}
+
+	// Then try origin
+	for _, remote := range remotes {
+		if remote.Config().Name == "origin" {
+			urls := remote.Config().URLs
+			if len(urls) > 0 {
+				owner, name = parseGitHubURL(urls[0])
+				if owner != "" && name != "" {
+					return owner, name, nil
+				}
+			}
+		}
+	}
+
+	return "danielmiessler", "fabric", nil
+}
+
+func parseGitHubURL(url string) (owner, repo string) {
+	patterns := []string{
+		`github\.com[:/]([^/]+)/([^/.]+)`,
+		`github\.com[:/]([^/]+)/([^/]+)\.git$`,
+	}
+
+	for _, pattern := range patterns {
+		re := regexp.MustCompile(pattern)
+		matches := re.FindStringSubmatch(url)
+		if len(matches) > 2 {
+			return matches[1], matches[2]
+		}
+	}
+
+	return "", ""
+}
+
+// WalkHistorySinceTag walks git history from HEAD down to (but not including) the specified tag
+// and returns any version commits found along the way
+func (w *Walker) WalkHistorySinceTag(sinceTag string) (map[string]*Version, error) {
+	// Get the commit SHA for the sinceTag
+	tagRef, err := w.repo.Tag(sinceTag)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get tag %s: %w", sinceTag, err)
+	}
+
+	tagCommit, err := w.repo.CommitObject(tagRef.Hash())
+	if err != nil {
+		return nil, fmt.Errorf("failed to get commit for tag %s: %w", sinceTag, err)
+	}
+
+	// Get HEAD reference
+	ref, err := w.repo.Head()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get HEAD: %w", err)
+	}
+
+	// Walk from HEAD down to the tag commit (excluding it)
+	commitIter, err := w.repo.Log(&git.LogOptions{
+		From:  ref.Hash(),
+		Order: git.LogOrderCommitterTime,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("failed to create commit iterator: %w", err)
+	}
+	defer commitIter.Close()
+
+	versions := make(map[string]*Version)
+	currentVersion := "Unreleased"
+	prNumbers := make(map[string][]int)
+
+	err = commitIter.ForEach(func(c *object.Commit) error {
+		// Stop iteration when the hash of the current commit matches the hash of the specified sinceTag commit
+		if c.Hash == tagCommit.Hash {
+			return storer.ErrStop
+		}
+
+		commit := &Commit{
+			SHA:     c.Hash.String(),
+			Message: strings.TrimSpace(c.Message),
+			Author:  c.Author.Name,
+			Email:   c.Author.Email,
+			Date:    c.Author.When,
+			IsMerge: len(c.ParentHashes) > 1,
+		}
+
+		// Check for version pattern
+		if matches := versionPattern.FindStringSubmatch(commit.Message); len(matches) > 1 {
+			commit.IsVersion = true
+			commit.Version = matches[1]
+			currentVersion = commit.Version
+
+			if _, exists := versions[currentVersion]; !exists {
+				versions[currentVersion] = &Version{
+					Name:      currentVersion,
+					Date:      commit.Date,
+					CommitSHA: commit.SHA,
+					Commits:   []*Commit{},
+				}
+			}
+			return nil
+		}
+
+		// Check for PR merge pattern
+		if matches := prPattern.FindStringSubmatch(commit.Message); len(matches) > 1 {
+			prNumber, err := strconv.Atoi(matches[1])
+			if err != nil {
+				// Handle parsing error (e.g., log it or skip processing)
+				return fmt.Errorf("failed to parse PR number: %v", err)
+			}
+			commit.PRNumber = prNumber
+
+			prNumbers[currentVersion] = append(prNumbers[currentVersion], prNumber)
+		}
+
+		// Add commit to current version
+		if _, exists := versions[currentVersion]; !exists {
+			versions[currentVersion] = &Version{
+				Name:      currentVersion,
+				Date:      time.Time{}, // Zero value, will be set by version commit
+				CommitSHA: "",
+				Commits:   []*Commit{},
+			}
+		}
+
+		versions[currentVersion].Commits = append(versions[currentVersion].Commits, commit)
+		return nil
+	})
+
+	// Handle the stop condition - storer.ErrStop is expected
+	if err == storer.ErrStop {
+		err = nil
+	}
+
+	// Assign collected PR numbers to each version
+	for version, prs := range prNumbers {
+		versions[version].PRNumbers = dedupInts(prs)
+	}
+
+	return versions, err
+}
+
+func dedupInts(ints []int) []int {
+	seen := make(map[int]bool)
+	result := []int{}
+
+	for _, i := range ints {
+		if !seen[i] {
+			seen[i] = true
+			result = append(result, i)
+		}
+	}
+
+	return result
+}
+
+// Worktree returns the git worktree for performing git operations
+func (w *Walker) Worktree() (*git.Worktree, error) {
+	return w.repo.Worktree()
+}
+
+// Repository returns the underlying git repository
+func (w *Walker) Repository() *git.Repository {
+	return w.repo
+}
+
+// IsWorkingDirectoryClean checks if the working directory has any uncommitted changes
+func (w *Walker) IsWorkingDirectoryClean() (bool, error) {
+	worktree, err := w.repo.Worktree()
+	if err != nil {
+		return false, fmt.Errorf("failed to get worktree: %w", err)
+	}
+
+	status, err := worktree.Status()
+	if err != nil {
+		return false, fmt.Errorf("failed to get git status: %w", err)
+	}
+
+	worktreePath := worktree.Filesystem.Root()
+
+	// In worktrees, files staged in the main repo may appear in status but not exist in the worktree
+	// We need to check both the working directory status AND filesystem existence
+	for file, fileStatus := range status {
+		// Check if there are any changes in the working directory
+		if fileStatus.Worktree != git.Unmodified && fileStatus.Worktree != git.Untracked {
+			return false, nil
+		}
+
+		// For staged files (Added, Modified in index), verify they exist in this worktree's filesystem
+		// This handles the worktree case where the main repo has staged files that don't exist here
+		if fileStatus.Staging != git.Unmodified && fileStatus.Staging != git.Untracked {
+			filePath := filepath.Join(worktreePath, file)
+			if _, err := os.Stat(filePath); os.IsNotExist(err) {
+				// File is staged but doesn't exist in this worktree - ignore it
+				continue
+			}
+			// File is staged AND exists in this worktree - not clean
+			return false, nil
+		}
+	}
+
+	return true, nil
+}
+
+// GetStatusDetails returns a detailed status of the working directory
+func (w *Walker) GetStatusDetails() (string, error) {
+	worktree, err := w.repo.Worktree()
+	if err != nil {
+		return "", fmt.Errorf("failed to get worktree: %w", err)
+	}
+
+	status, err := worktree.Status()
+	if err != nil {
+		return "", fmt.Errorf("failed to get git status: %w", err)
+	}
+
+	var details strings.Builder
+	for file, fileStatus := range status {
+		// Only include files with actual working directory changes
+		if fileStatus.Worktree != git.Unmodified && fileStatus.Worktree != git.Untracked {
+			details.WriteString(fmt.Sprintf("  %c%c %s\n", fileStatus.Staging, fileStatus.Worktree, file))
+		}
+	}
+
+	return details.String(), nil
+}
+
+// AddFile adds a file to the git index
+// Uses native git CLI instead of go-git to properly handle worktree scenarios
+func (w *Walker) AddFile(filename string) error {
+	worktree, err := w.repo.Worktree()
+	if err != nil {
+		return fmt.Errorf("failed to get worktree: %w", err)
+	}
+
+	worktreePath := worktree.Filesystem.Root()
+
+	// Use native git add command to avoid go-git worktree issues
+	cmd := exec.Command("git", "add", filename)
+	cmd.Dir = worktreePath
+
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		return fmt.Errorf("failed to add file %s: %w (output: %s)", filename, err, string(output))
+	}
+
+	return nil
+}
+
+// CommitChanges creates a commit with the given message
+// Uses native git CLI instead of go-git to properly handle worktree scenarios
+func (w *Walker) CommitChanges(message string) (plumbing.Hash, error) {
+	worktree, err := w.repo.Worktree()
+	if err != nil {
+		return plumbing.ZeroHash, fmt.Errorf("failed to get worktree: %w", err)
+	}
+
+	worktreePath := worktree.Filesystem.Root()
+
+	// Use native git commit command to avoid go-git worktree issues
+	cmd := exec.Command("git", "commit", "-m", message)
+	cmd.Dir = worktreePath
+
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		return plumbing.ZeroHash, fmt.Errorf("failed to commit: %w (output: %s)", err, string(output))
+	}
+
+	// Get the commit hash from HEAD
+	ref, err := w.repo.Head()
+	if err != nil {
+		return plumbing.ZeroHash, fmt.Errorf("failed to get HEAD after commit: %w", err)
+	}
+
+	return ref.Hash(), nil
+}
+
+// PushToRemote pushes the current branch to the remote repository
+// It automatically detects GitHub repositories and uses token authentication when available
+func (w *Walker) PushToRemote() error {
+	pushOptions := &git.PushOptions{}
+
+	// Check if we have a GitHub token for authentication
+	if githubToken := util.GetTokenFromEnv(""); githubToken != "" {
+		// Get remote URL to check if it's a GitHub repository
+		remotes, err := w.repo.Remotes()
+		if err == nil && len(remotes) > 0 {
+			// Get the origin remote (or first remote if origin doesn't exist)
+			var remote *git.Remote
+			for _, r := range remotes {
+				if r.Config().Name == "origin" {
+					remote = r
+					break
+				}
+			}
+			if remote == nil {
+				remote = remotes[0]
+			}
+
+			// Check if this is a GitHub repository
+			urls := remote.Config().URLs
+			if len(urls) > 0 {
+				url := urls[0]
+				if strings.Contains(url, "github.com") {
+					// Use token authentication for GitHub repositories
+					pushOptions.Auth = &http.BasicAuth{
+						Username: "token", // GitHub expects "token" as username
+						Password: githubToken,
+					}
+				}
+			}
+		}
+	}
+
+	err := w.repo.Push(pushOptions)
+	if err != nil {
+		return fmt.Errorf("failed to push: %w", err)
+	}
+	return nil
+}
+
+// RemoveFile removes a file from both the working directory and git index
+func (w *Walker) RemoveFile(filename string) error {
+	worktree, err := w.repo.Worktree()
+	if err != nil {
+		return fmt.Errorf("failed to get worktree: %w", err)
+	}
+
+	_, err = worktree.Remove(filename)
+	if err != nil {
+		return fmt.Errorf("failed to remove file %s: %w", filename, err)
+	}
+
+	return nil
+}

cmd/generate_changelog/internal/github/client.go

@@ -0,0 +1,431 @@
+package github
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+	"os"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/google/go-github/v66/github"
+	"github.com/hasura/go-graphql-client"
+	"golang.org/x/oauth2"
+)
+
+type Client struct {
+	client        *github.Client
+	graphqlClient *graphql.Client
+	owner         string
+	repo          string
+	token         string
+}
+
+func NewClient(token, owner, repo string) *Client {
+	var githubClient *github.Client
+	var httpClient *http.Client
+	var gqlClient *graphql.Client
+
+	if token != "" {
+		ts := oauth2.StaticTokenSource(
+			&oauth2.Token{AccessToken: token},
+		)
+		httpClient = oauth2.NewClient(context.Background(), ts)
+		githubClient = github.NewClient(httpClient)
+		gqlClient = graphql.NewClient("https://api.github.com/graphql", httpClient)
+	} else {
+		httpClient = http.DefaultClient
+		githubClient = github.NewClient(nil)
+		gqlClient = graphql.NewClient("https://api.github.com/graphql", httpClient)
+	}
+
+	return &Client{
+		client:        githubClient,
+		graphqlClient: gqlClient,
+		owner:         owner,
+		repo:          repo,
+		token:         token,
+	}
+}
+
+func (c *Client) FetchPRs(prNumbers []int) ([]*PR, error) {
+	if len(prNumbers) == 0 {
+		return []*PR{}, nil
+	}
+
+	ctx := context.Background()
+	prs := make([]*PR, 0, len(prNumbers))
+	prsChan := make(chan *PR, len(prNumbers))
+	errChan := make(chan error, len(prNumbers))
+
+	var wg sync.WaitGroup
+	semaphore := make(chan struct{}, 10)
+
+	for _, prNumber := range prNumbers {
+		wg.Add(1)
+		go func(num int) {
+			defer wg.Done()
+
+			semaphore <- struct{}{}
+			defer func() { <-semaphore }()
+
+			pr, err := c.fetchSinglePR(ctx, num)
+			if err != nil {
+				errChan <- fmt.Errorf("failed to fetch PR #%d: %w", num, err)
+				return
+			}
+			prsChan <- pr
+		}(prNumber)
+	}
+
+	go func() {
+		wg.Wait()
+		close(prsChan)
+		close(errChan)
+	}()
+
+	var errors []error
+	for pr := range prsChan {
+		prs = append(prs, pr)
+	}
+	for err := range errChan {
+		errors = append(errors, err)
+	}
+
+	if len(errors) > 0 {
+		return prs, fmt.Errorf("some PRs failed to fetch: %v", errors)
+	}
+
+	return prs, nil
+}
+
+// GetPRValidationDetails fetches only the data needed for validation (lightweight).
+func (c *Client) GetPRValidationDetails(prNumber int) (*PRDetails, error) {
+	ctx := context.Background()
+	ghPR, _, err := c.client.PullRequests.Get(ctx, c.owner, c.repo, prNumber)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get PR %d: %w", prNumber, err)
+	}
+
+	// Only return validation data, no commits fetched
+	details := &PRDetails{
+		PR:        nil, // Will be populated later if needed
+		State:     getString(ghPR.State),
+		Mergeable: ghPR.Mergeable != nil && *ghPR.Mergeable,
+	}
+
+	return details, nil
+}
+
+// GetPRWithCommits fetches the full PR and its commits.
+func (c *Client) GetPRWithCommits(prNumber int) (*PR, error) {
+	ctx := context.Background()
+	ghPR, _, err := c.client.PullRequests.Get(ctx, c.owner, c.repo, prNumber)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get PR %d: %w", prNumber, err)
+	}
+
+	return c.buildPRWithCommits(ctx, ghPR)
+}
+
+// GetPRDetails fetches a comprehensive set of details for a single PR.
+// Deprecated: Use GetPRValidationDetails + GetPRWithCommits for better performance
+func (c *Client) GetPRDetails(prNumber int) (*PRDetails, error) {
+	ctx := context.Background()
+	ghPR, _, err := c.client.PullRequests.Get(ctx, c.owner, c.repo, prNumber)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get PR %d: %w", prNumber, err)
+	}
+
+	// Reuse the existing logic to build the base PR object
+	pr, err := c.buildPRWithCommits(ctx, ghPR)
+	if err != nil {
+		return nil, fmt.Errorf("failed to build PR details for %d: %w", prNumber, err)
+	}
+
+	details := &PRDetails{
+		PR:        pr,
+		State:     getString(ghPR.State),
+		Mergeable: ghPR.Mergeable != nil && *ghPR.Mergeable,
+	}
+
+	return details, nil
+}
+
+// buildPRWithCommits fetches commits and constructs a PR object from a GitHub API response
+func (c *Client) buildPRWithCommits(ctx context.Context, ghPR *github.PullRequest) (*PR, error) {
+	commits, _, err := c.client.PullRequests.ListCommits(ctx, c.owner, c.repo, *ghPR.Number, nil)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch commits for PR %d: %w", *ghPR.Number, err)
+	}
+
+	return c.convertGitHubPR(ghPR, commits), nil
+}
+
+// convertGitHubPR transforms GitHub API data into our internal PR struct (pure function)
+func (c *Client) convertGitHubPR(ghPR *github.PullRequest, commits []*github.RepositoryCommit) *PR {
+
+	result := &PR{
+		Number:  *ghPR.Number,
+		Title:   getString(ghPR.Title),
+		Body:    getString(ghPR.Body),
+		URL:     getString(ghPR.HTMLURL),
+		Commits: make([]PRCommit, 0, len(commits)),
+	}
+
+	if ghPR.MergedAt != nil {
+		result.MergedAt = ghPR.MergedAt.Time
+	}
+
+	if ghPR.User != nil {
+		result.Author = getString(ghPR.User.Login)
+		result.AuthorURL = getString(ghPR.User.HTMLURL)
+		userType := getString(ghPR.User.Type)
+
+		switch userType {
+		case "User":
+			result.AuthorType = "user"
+		case "Organization":
+			result.AuthorType = "organization"
+		case "Bot":
+			result.AuthorType = "bot"
+		default:
+			result.AuthorType = "user"
+		}
+	}
+
+	if ghPR.MergeCommitSHA != nil {
+		result.MergeCommit = *ghPR.MergeCommitSHA
+	}
+
+	for _, commit := range commits {
+		if commit.Commit != nil {
+			prCommit := PRCommit{
+				SHA:     getString(commit.SHA),
+				Message: strings.TrimSpace(getString(commit.Commit.Message)),
+			}
+			if commit.Commit.Author != nil {
+				prCommit.Author = getString(commit.Commit.Author.Name)
+				prCommit.Email = getString(commit.Commit.Author.Email) // Extract author email from GitHub API response
+				// Capture actual commit timestamp from GitHub API
+				if commit.Commit.Author.Date != nil {
+					prCommit.Date = commit.Commit.Author.Date.Time
+				}
+			}
+			// Capture parent commit SHAs for merge detection
+			if commit.Parents != nil {
+				for _, parent := range commit.Parents {
+					if parent.SHA != nil {
+						prCommit.Parents = append(prCommit.Parents, *parent.SHA)
+					}
+				}
+			}
+			result.Commits = append(result.Commits, prCommit)
+		}
+	}
+
+	return result
+}
+
+func (c *Client) fetchSinglePR(ctx context.Context, prNumber int) (*PR, error) {
+	ghPR, _, err := c.client.PullRequests.Get(ctx, c.owner, c.repo, prNumber)
+	if err != nil {
+		return nil, err
+	}
+
+	return c.buildPRWithCommits(ctx, ghPR)
+}
+
+func getString(s *string) string {
+	if s == nil {
+		return ""
+	}
+	return *s
+}
+
+// FetchAllMergedPRs fetches all merged PRs using GitHub's search API
+// This is much more efficient than fetching PRs individually
+func (c *Client) FetchAllMergedPRs(since time.Time) ([]*PR, error) {
+	ctx := context.Background()
+	var allPRs []*PR
+
+	// Build search query for merged PRs
+	query := fmt.Sprintf("repo:%s/%s is:pr is:merged", c.owner, c.repo)
+	if !since.IsZero() {
+		query += fmt.Sprintf(" merged:>=%s", since.Format("2006-01-02"))
+	}
+
+	opts := &github.SearchOptions{
+		Sort:  "created",
+		Order: "desc",
+		ListOptions: github.ListOptions{
+			PerPage: 100, // Maximum allowed
+		},
+	}
+
+	for {
+		result, resp, err := c.client.Search.Issues(ctx, query, opts)
+		if err != nil {
+			return allPRs, fmt.Errorf("failed to search PRs: %w", err)
+		}
+
+		// Process PRs in parallel
+		prsChan := make(chan *PR, len(result.Issues))
+		errChan := make(chan error, len(result.Issues))
+		var wg sync.WaitGroup
+		semaphore := make(chan struct{}, 10) // Limit concurrent requests
+
+		for _, issue := range result.Issues {
+			if issue.PullRequestLinks == nil {
+				continue // Not a PR
+			}
+
+			wg.Add(1)
+			go func(prNumber int) {
+				defer wg.Done()
+
+				semaphore <- struct{}{}
+				defer func() { <-semaphore }()
+
+				pr, err := c.fetchSinglePR(ctx, prNumber)
+				if err != nil {
+					errChan <- fmt.Errorf("failed to fetch PR #%d: %w", prNumber, err)
+					return
+				}
+				prsChan <- pr
+			}(*issue.Number)
+		}
+
+		go func() {
+			wg.Wait()
+			close(prsChan)
+			close(errChan)
+		}()
+
+		// Collect results
+		for pr := range prsChan {
+			allPRs = append(allPRs, pr)
+		}
+
+		// Check for errors
+		for err := range errChan {
+			// Log error but continue processing
+			fmt.Fprintf(os.Stderr, "Warning: %v\n", err)
+		}
+
+		if resp.NextPage == 0 {
+			break
+		}
+		opts.Page = resp.NextPage
+	}
+
+	return allPRs, nil
+}
+
+// FetchAllMergedPRsGraphQL fetches all merged PRs with their commits using GraphQL
+// This is the ultimate optimization - gets everything in ~5-10 API calls
+func (c *Client) FetchAllMergedPRsGraphQL(since time.Time) ([]*PR, error) {
+	ctx := context.Background()
+	var allPRs []*PR
+	var after *string
+	totalFetched := 0
+
+	for {
+		// Prepare variables
+		variables := map[string]any{
+			"owner": graphql.String(c.owner),
+			"repo":  graphql.String(c.repo),
+			"after": (*graphql.String)(after),
+		}
+
+		// Execute GraphQL query
+		var query PullRequestsQuery
+		err := c.graphqlClient.Query(ctx, &query, variables)
+		if err != nil {
+			return allPRs, fmt.Errorf("GraphQL query failed: %w", err)
+		}
+
+		prs := query.Repository.PullRequests.Nodes
+		fmt.Fprintf(os.Stderr, "Fetched %d PRs via GraphQL (page %d)\n", len(prs), (totalFetched/100)+1)
+
+		// Convert GraphQL PRs to our PR struct
+		for _, gqlPR := range prs {
+			// If we have a since filter, stop when we reach older PRs
+			if !since.IsZero() && gqlPR.MergedAt.Before(since) {
+				fmt.Fprintf(os.Stderr, "Reached PRs older than %s, stopping\n", since.Format("2006-01-02"))
+				return allPRs, nil
+			}
+
+			pr := &PR{
+				Number:   gqlPR.Number,
+				Title:    gqlPR.Title,
+				Body:     gqlPR.Body,
+				URL:      gqlPR.URL,
+				MergedAt: gqlPR.MergedAt,
+				Commits:  make([]PRCommit, 0, len(gqlPR.Commits.Nodes)),
+			}
+
+			// Handle author - check if it's nil first
+			if gqlPR.Author != nil {
+				pr.Author = gqlPR.Author.Login
+				pr.AuthorURL = gqlPR.Author.URL
+
+				switch gqlPR.Author.Typename {
+				case "Bot":
+					pr.AuthorType = "bot"
+				case "Organization":
+					pr.AuthorType = "organization"
+				case "User":
+					pr.AuthorType = "user"
+				default:
+					pr.AuthorType = "user" // fallback
+					if gqlPR.Author.Typename != "" {
+						fmt.Fprintf(os.Stderr, "PR #%d: Unknown author typename '%s'\n", gqlPR.Number, gqlPR.Author.Typename)
+					}
+				}
+			} else {
+				// Author is nil - try to fetch from REST API as fallback
+				fmt.Fprintf(os.Stderr, "PR #%d: Author is nil in GraphQL response, fetching from REST API\n", gqlPR.Number)
+
+				// Fetch this specific PR from REST API
+				restPR, err := c.fetchSinglePR(ctx, gqlPR.Number)
+				if err == nil && restPR != nil && restPR.Author != "" {
+					pr.Author = restPR.Author
+					pr.AuthorURL = restPR.AuthorURL
+					pr.AuthorType = restPR.AuthorType
+				} else {
+					// Fallback if REST API also fails
+					pr.Author = "[unknown]"
+					pr.AuthorURL = ""
+					pr.AuthorType = "user"
+				}
+			}
+
+			// Convert commits
+			for _, commitNode := range gqlPR.Commits.Nodes {
+				commit := PRCommit{
+					SHA:     commitNode.Commit.OID,
+					Message: strings.TrimSpace(commitNode.Commit.Message),
+					Author:  commitNode.Commit.Author.Name,
+					Date:    commitNode.Commit.AuthoredDate, // Use actual commit timestamp
+				}
+				pr.Commits = append(pr.Commits, commit)
+			}
+
+			allPRs = append(allPRs, pr)
+		}
+
+		totalFetched += len(prs)
+
+		// Check if we need to fetch more pages
+		if !query.Repository.PullRequests.PageInfo.HasNextPage {
+			break
+		}
+
+		after = &query.Repository.PullRequests.PageInfo.EndCursor
+	}
+
+	fmt.Fprintf(os.Stderr, "Total PRs fetched via GraphQL: %d\n", len(allPRs))
+	return allPRs, nil
+}

cmd/generate_changelog/internal/github/email_test.go

@@ -0,0 +1,59 @@
+package github
+
+import (
+	"testing"
+	"time"
+)
+
+func TestPRCommitEmailHandling(t *testing.T) {
+	tests := []struct {
+		name     string
+		commit   PRCommit
+		expected string
+	}{
+		{
+			name: "Valid email field",
+			commit: PRCommit{
+				SHA:     "abc123",
+				Message: "Fix bug in authentication",
+				Author:  "John Doe",
+				Email:   "john.doe@example.com",
+				Date:    time.Now(),
+				Parents: []string{"def456"},
+			},
+			expected: "john.doe@example.com",
+		},
+		{
+			name: "Empty email field",
+			commit: PRCommit{
+				SHA:     "abc123",
+				Message: "Fix bug in authentication",
+				Author:  "John Doe",
+				Email:   "",
+				Date:    time.Now(),
+				Parents: []string{"def456"},
+			},
+			expected: "",
+		},
+		{
+			name: "Email field with proper initialization",
+			commit: PRCommit{
+				SHA:     "def789",
+				Message: "Add new feature",
+				Author:  "Jane Smith",
+				Email:   "jane.smith@company.org",
+				Date:    time.Now(),
+				Parents: []string{"ghi012"},
+			},
+			expected: "jane.smith@company.org",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if tt.commit.Email != tt.expected {
+				t.Errorf("Expected email %q, got %q", tt.expected, tt.commit.Email)
+			}
+		})
+	}
+}

cmd/generate_changelog/internal/github/types.go

@@ -0,0 +1,68 @@
+package github
+
+import "time"
+
+type PR struct {
+	Number      int
+	Title       string
+	Body        string
+	Author      string
+	AuthorURL   string
+	AuthorType  string // "user", "organization", or "bot"
+	URL         string
+	MergedAt    time.Time
+	Commits     []PRCommit
+	MergeCommit string
+}
+
+// PRDetails encapsulates all relevant information about a Pull Request.
+type PRDetails struct {
+	*PR
+	State     string
+	Mergeable bool
+}
+
+type PRCommit struct {
+	SHA     string
+	Message string
+	Author  string
+	Email   string    // Author email from GitHub API, empty if not public
+	Date    time.Time // Timestamp field
+	Parents []string  // Parent commits (for merge detection)
+}
+
+// GraphQL query structures for hasura client
+type PullRequestsQuery struct {
+	Repository struct {
+		PullRequests struct {
+			PageInfo struct {
+				HasNextPage bool
+				EndCursor   string
+			}
+			Nodes []struct {
+				Number   int
+				Title    string
+				Body     string
+				URL      string
+				MergedAt time.Time
+				Author   *struct {
+					Typename string `graphql:"__typename"`
+					Login    string `graphql:"login"`
+					URL      string `graphql:"url"`
+				}
+				Commits struct {
+					Nodes []struct {
+						Commit struct {
+							OID          string `graphql:"oid"`
+							Message      string
+							AuthoredDate time.Time `graphql:"authoredDate"`
+							Author       struct {
+								Name string
+							}
+						}
+					}
+				} `graphql:"commits(first: 250)"`
+			}
+		} `graphql:"pullRequests(first: 100, after: $after, states: MERGED, orderBy: {field: UPDATED_AT, direction: DESC})"`
+	} `graphql:"repository(owner: $owner, name: $repo)"`
+}

cmd/generate_changelog/internal/release.go

@@ -0,0 +1,149 @@
+package internal
+
+import (
+	"context"
+	"fmt"
+	"os/exec"
+	"regexp"
+	"strings"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/cache"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/config"
+	"github.com/google/go-github/v66/github"
+	"golang.org/x/oauth2"
+)
+
+type ReleaseManager struct {
+	cache       *cache.Cache
+	githubToken string
+	owner       string
+	repo        string
+}
+
+// getGitHubInfo extracts owner and repo from git remote origin URL
+func getGitHubInfo() (owner, repo string, err error) {
+	cmd := exec.Command("git", "remote", "get-url", "origin")
+	output, err := cmd.Output()
+	if err != nil {
+		return "", "", fmt.Errorf("failed to get git remote URL: %w", err)
+	}
+
+	url := strings.TrimSpace(string(output))
+
+	// Handle both SSH and HTTPS URLs
+	// SSH: git@github.com:owner/repo.git
+	// HTTPS: https://github.com/owner/repo.git
+	var re *regexp.Regexp
+	if strings.HasPrefix(url, "git@") {
+		re = regexp.MustCompile(`git@github\.com:([^/]+)/([^/.]+)(?:\.git)?`)
+	} else {
+		re = regexp.MustCompile(`https://github\.com/([^/]+)/([^/.]+)(?:\.git)?`)
+	}
+
+	matches := re.FindStringSubmatch(url)
+	if len(matches) < 3 {
+		return "", "", fmt.Errorf("invalid GitHub URL format: %s", url)
+	}
+
+	return matches[1], matches[2], nil
+}
+
+func NewReleaseManager(cfg *config.Config) (*ReleaseManager, error) {
+	cache, err := cache.New(cfg.CacheFile)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create cache: %w", err)
+	}
+
+	owner, repo, err := getGitHubInfo()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get GitHub repository info: %w", err)
+	}
+
+	return &ReleaseManager{
+		cache:       cache,
+		githubToken: cfg.GitHubToken,
+		owner:       owner,
+		repo:        repo,
+	}, nil
+}
+
+func (rm *ReleaseManager) Close() error {
+	return rm.cache.Close()
+}
+
+func (rm *ReleaseManager) UpdateReleaseDescription(version string) error {
+	versions, err := rm.cache.GetVersions()
+	if err != nil {
+		return fmt.Errorf("failed to get versions from cache: %w", err)
+	}
+
+	versionData, exists := versions[version]
+	if !exists {
+		return fmt.Errorf("version %s not found in versions table", version)
+	}
+
+	if versionData.AISummary == "" {
+		return fmt.Errorf("ai_summary is empty for version %s", version)
+	}
+
+	releaseBody := fmt.Sprintf("## Changes\n\n%s", versionData.AISummary)
+
+	ctx := context.Background()
+	var client *github.Client
+
+	if rm.githubToken != "" {
+		ts := oauth2.StaticTokenSource(
+			&oauth2.Token{AccessToken: rm.githubToken},
+		)
+		tc := oauth2.NewClient(ctx, ts)
+		client = github.NewClient(tc)
+	} else {
+		client = github.NewClient(nil)
+	}
+
+	// Check if current repository is a fork by getting repo details
+	repo, _, err := client.Repositories.Get(ctx, rm.owner, rm.repo)
+	if err != nil {
+		return fmt.Errorf("failed to get repository info: %w", err)
+	}
+
+	// If repository is a fork, try updating the upstream (parent) repository first
+	if repo.Parent != nil {
+		parentOwner := repo.Parent.Owner.GetLogin()
+		parentRepo := repo.Parent.GetName()
+
+		fmt.Printf("Repository is a fork of %s/%s, attempting to update upstream release...\n", parentOwner, parentRepo)
+
+		err := rm.updateReleaseForRepo(ctx, client, parentOwner, parentRepo, version, releaseBody)
+		if err == nil {
+			fmt.Printf("Successfully updated release description for %s in upstream repository %s/%s\n", version, parentOwner, parentRepo)
+			return nil
+		}
+
+		fmt.Printf("Failed to update upstream repository: %v\nFalling back to current repository...\n", err)
+	}
+
+	// Update current repository (either not a fork or upstream update failed)
+	err = rm.updateReleaseForRepo(ctx, client, rm.owner, rm.repo, version, releaseBody)
+	if err != nil {
+		return fmt.Errorf("failed to update release description for version %s in repository %s/%s: %w", version, rm.owner, rm.repo, err)
+	}
+
+	fmt.Printf("Successfully updated release description for %s in repository %s/%s\n", version, rm.owner, rm.repo)
+	return nil
+}
+
+func (rm *ReleaseManager) updateReleaseForRepo(ctx context.Context, client *github.Client, owner, repo, version, releaseBody string) error {
+	release, _, err := client.Repositories.GetReleaseByTag(ctx, owner, repo, version)
+	if err != nil {
+		return fmt.Errorf("failed to get release for version %s: %w", version, err)
+	}
+
+	release.Body = &releaseBody
+	_, _, err = client.Repositories.EditRelease(ctx, owner, repo, *release.ID, release)
+	if err != nil {
+		return fmt.Errorf("failed to update release description for version %s: %w", version, err)
+	}
+
+	return nil
+}

cmd/generate_changelog/main.go

@@ -0,0 +1,117 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/changelog"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/internal/config"
+	"github.com/danielmiessler/fabric/cmd/generate_changelog/util"
+	"github.com/joho/godotenv"
+	"github.com/spf13/cobra"
+)
+
+var (
+	cfg = &config.Config{}
+)
+
+var rootCmd = &cobra.Command{
+	Use:   "generate_changelog",
+	Short: "Generate changelog from git history and GitHub PRs",
+	Long: `A high-performance changelog generator that walks git history,
+collects version information and pull requests, and generates a
+comprehensive changelog in markdown format.`,
+	RunE:         run,
+	SilenceUsage: true, // Don't show usage on runtime errors, only on flag errors
+}
+
+func init() {
+	rootCmd.Flags().StringVarP(&cfg.RepoPath, "repo", "r", ".", "Repository path")
+	rootCmd.Flags().StringVarP(&cfg.OutputFile, "output", "o", "", "Output file (default: stdout)")
+	rootCmd.Flags().IntVarP(&cfg.Limit, "limit", "l", 0, "Limit number of versions (0 = all)")
+	rootCmd.Flags().StringVarP(&cfg.Version, "version", "v", "", "Generate changelog for specific version")
+	rootCmd.Flags().BoolVar(&cfg.SaveData, "save-data", false, "Save version data to JSON for debugging")
+	rootCmd.Flags().StringVar(&cfg.CacheFile, "cache", "./cmd/generate_changelog/changelog.db", "Cache database file")
+	rootCmd.Flags().BoolVar(&cfg.NoCache, "no-cache", false, "Disable cache usage")
+	rootCmd.Flags().BoolVar(&cfg.RebuildCache, "rebuild-cache", false, "Rebuild cache from scratch")
+	rootCmd.Flags().StringVar(&cfg.GitHubToken, "token", "", "GitHub API token (or set GITHUB_TOKEN env var)")
+	rootCmd.Flags().BoolVar(&cfg.ForcePRSync, "force-pr-sync", false, "Force a full PR sync from GitHub (ignores cache age)")
+	rootCmd.Flags().BoolVar(&cfg.EnableAISummary, "ai-summarize", false, "Generate AI-enhanced summaries using Fabric")
+	rootCmd.Flags().IntVar(&cfg.IncomingPR, "incoming-pr", 0, "Pre-process PR for changelog (provide PR number)")
+	rootCmd.Flags().StringVar(&cfg.ProcessPRsVersion, "process-prs", "", "Process all incoming PR files for release (provide version like v1.4.262)")
+	rootCmd.Flags().StringVar(&cfg.IncomingDir, "incoming-dir", "./cmd/generate_changelog/incoming", "Directory for incoming PR files")
+	rootCmd.Flags().BoolVar(&cfg.Push, "push", false, "Enable automatic git push after creating an incoming entry")
+	rootCmd.Flags().BoolVar(&cfg.SyncDB, "sync-db", false, "Synchronize and validate database integrity with git history and GitHub PRs")
+	rootCmd.Flags().StringVar(&cfg.Release, "release", "", "Update GitHub release description with AI summary for version (e.g., v1.2.3)")
+}
+
+func run(cmd *cobra.Command, args []string) error {
+	if cfg.IncomingPR > 0 && cfg.ProcessPRsVersion != "" {
+		return fmt.Errorf("--incoming-pr and --process-prs are mutually exclusive flags")
+	}
+
+	if cfg.Release != "" && (cfg.IncomingPR > 0 || cfg.ProcessPRsVersion != "" || cfg.SyncDB) {
+		return fmt.Errorf("--release cannot be used with other processing flags")
+	}
+
+	cfg.GitHubToken = util.GetTokenFromEnv(cfg.GitHubToken)
+
+	generator, err := changelog.New(cfg)
+	if err != nil {
+		return fmt.Errorf("failed to create changelog generator: %w", err)
+	}
+
+	if cfg.IncomingPR > 0 {
+		return generator.ProcessIncomingPR(cfg.IncomingPR)
+	}
+
+	if cfg.ProcessPRsVersion != "" {
+		return generator.CreateNewChangelogEntry(cfg.ProcessPRsVersion)
+	}
+
+	if cfg.SyncDB {
+		return generator.SyncDatabase()
+	}
+
+	if cfg.Release != "" {
+		releaseManager, err := internal.NewReleaseManager(cfg)
+		if err != nil {
+			return fmt.Errorf("failed to create release manager: %w", err)
+		}
+		defer releaseManager.Close()
+		return releaseManager.UpdateReleaseDescription(cfg.Release)
+	}
+
+	output, err := generator.Generate()
+	if err != nil {
+		return fmt.Errorf("failed to generate changelog: %w", err)
+	}
+
+	if cfg.OutputFile != "" {
+		if err := os.WriteFile(cfg.OutputFile, []byte(output), 0644); err != nil {
+			return fmt.Errorf("failed to write output file: %w", err)
+		}
+		fmt.Printf("Changelog written to %s\n", cfg.OutputFile)
+	} else {
+		fmt.Print(output)
+	}
+
+	return nil
+}
+
+func main() {
+	// Load .env file from the same directory as the binary
+	if exePath, err := os.Executable(); err == nil {
+		envPath := filepath.Join(filepath.Dir(exePath), ".env")
+		if _, err := os.Stat(envPath); err == nil {
+			// .env file exists, load it
+			if err := godotenv.Load(envPath); err != nil {
+				fmt.Fprintf(os.Stderr, "Warning: Failed to load .env file: %v\n", err)
+			}
+		}
+	}
+
+	rootCmd.Execute()
+}

cmd/generate_changelog/util/token.go

@@ -0,0 +1,31 @@
+package util
+
+import (
+	"os"
+)
+
+// GetTokenFromEnv returns a GitHub token based on the following precedence order:
+//  1. If tokenValue is non-empty, it is returned.
+//  2. Otherwise, if the GITHUB_TOKEN environment variable is set, its value is returned.
+//  3. Otherwise, if the GH_TOKEN environment variable is set, its value is returned.
+//  4. If none of the above are set, an empty string is returned.
+//
+// Example:
+//
+//	os.Setenv("GITHUB_TOKEN", "abc")
+//	os.Setenv("GH_TOKEN", "def")
+//	GetTokenFromEnv("xyz") // returns "xyz"
+//	GetTokenFromEnv("")    // returns "abc"
+//	os.Unsetenv("GITHUB_TOKEN")
+//	GetTokenFromEnv("")    // returns "def"
+//	os.Unsetenv("GH_TOKEN")
+//	GetTokenFromEnv("")    // returns ""
+func GetTokenFromEnv(tokenValue string) string {
+	if tokenValue == "" {
+		tokenValue = os.Getenv("GITHUB_TOKEN")
+		if tokenValue == "" {
+			tokenValue = os.Getenv("GH_TOKEN")
+		}
+	}
+	return tokenValue
+}

common/domain.go

@@ -1,50 +0,0 @@
-package common
-
-import goopenai "github.com/sashabaranov/go-openai"
-
-const ChatMessageRoleMeta = "meta"
-
-type ChatRequest struct {
-	ContextName      string
-	SessionName      string
-	PatternName      string
-	PatternVariables map[string]string
-	Message          *goopenai.ChatCompletionMessage
-	Language         string
-	Meta             string
-	InputHasVars     bool
-	StrategyName     string
-}
-
-type ChatOptions struct {
-	Model              string
-	Temperature        float64
-	TopP               float64
-	PresencePenalty    float64
-	FrequencyPenalty   float64
-	Raw                bool
-	Seed               int
-	ModelContextLength int
-	MaxTokens          int
-}
-
-// NormalizeMessages remove empty messages and ensure messages order user-assist-user
-func NormalizeMessages(msgs []*goopenai.ChatCompletionMessage, defaultUserMessage string) (ret []*goopenai.ChatCompletionMessage) {
-	// Iterate over messages to enforce the odd position rule for user messages
-	fullMessageIndex := 0
-	for _, message := range msgs {
-		if message.Content == "" {
-			// Skip empty messages as the anthropic API doesn't accept them
-			continue
-		}
-
-		// Ensure, that each odd position shall be a user message
-		if fullMessageIndex%2 == 0 && message.Role != goopenai.ChatMessageRoleUser {
-			ret = append(ret, &goopenai.ChatCompletionMessage{Role: goopenai.ChatMessageRoleUser, Content: defaultUserMessage})
-			fullMessageIndex++
-		}
-		ret = append(ret, message)
-		fullMessageIndex++
-	}
-	return
-}

common/domain_test.go

@@ -1,27 +0,0 @@
-package common
-
-import (
-	"testing"
-
-	goopenai "github.com/sashabaranov/go-openai"
-	"github.com/stretchr/testify/assert"
-)
-
-func TestNormalizeMessages(t *testing.T) {
-	msgs := []*goopenai.ChatCompletionMessage{
-		{Role: goopenai.ChatMessageRoleUser, Content: "Hello"},
-		{Role: goopenai.ChatMessageRoleAssistant, Content: "Hi there!"},
-		{Role: goopenai.ChatMessageRoleUser, Content: ""},
-		{Role: goopenai.ChatMessageRoleUser, Content: ""},
-		{Role: goopenai.ChatMessageRoleUser, Content: "How are you?"},
-	}
-
-	expected := []*goopenai.ChatCompletionMessage{
-		{Role: goopenai.ChatMessageRoleUser, Content: "Hello"},
-		{Role: goopenai.ChatMessageRoleAssistant, Content: "Hi there!"},
-		{Role: goopenai.ChatMessageRoleUser, Content: "How are you?"},
-	}
-
-	actual := NormalizeMessages(msgs, "default")
-	assert.Equal(t, expected, actual)
-}

completions/_fabric

@@ -1,47 +1,69 @@
-#compdef fabric
+#compdef fabric fabric-ai
 
 # Zsh completion for fabric CLI
 # Place this file in a directory in your $fpath (e.g. /usr/local/share/zsh/site-functions)
 
 _fabric_patterns() {
   local -a patterns
-  patterns=(${(f)"$(fabric --listpatterns --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  patterns=(${(f)"$($cmd --listpatterns --shell-complete-list 2>/dev/null)"})
   compadd -X "Patterns:" ${patterns}
 }
 
 _fabric_models() {
   local -a models
-  models=(${(f)"$(fabric --listmodels --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  models=(${(f)"$($cmd --listmodels --shell-complete-list 2>/dev/null)"})
   compadd -X "Models:" ${models}
 }
+
+_fabric_vendors() {
+  local -a vendors
+  local cmd=${words[1]}
+  vendors=(${(f)"$($cmd --listvendors --shell-complete-list 2>/dev/null)"})
+  compadd -X "Vendors:" ${vendors}
+}
+
 _fabric_contexts() {
   local -a contexts
-  contexts=(${(f)"$(fabric --listcontexts --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  contexts=(${(f)"$($cmd --listcontexts --shell-complete-list 2>/dev/null)"})
   compadd -X "Contexts:" ${contexts}
 }
+
 _fabric_sessions() {
   local -a sessions
-  sessions=(${(f)"$(fabric --listsessions --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  sessions=(${(f)"$($cmd --listsessions --shell-complete-list 2>/dev/null)"})
   compadd -X "Sessions:" ${sessions}
 }
+
 _fabric_strategies() {
   local -a strategies
-  strategies=(${(f)"$(fabric --liststrategies --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  strategies=(${(f)"$($cmd --liststrategies --shell-complete-list 2>/dev/null)"})
   compadd -X "Strategies:" ${strategies}
 }
 
 _fabric_extensions() {
   local -a extensions
-  extensions=(${(f)"$(fabric --listextensions --shell-complete-list 2>/dev/null)"})
+  local cmd=${words[1]}
+  extensions=(${(f)"$($cmd --listextensions --shell-complete-list 2>/dev/null)"})
   compadd -X "Extensions:" ${extensions}
-    '(-L --listmodels)'{-L,--listmodels}'[List all available models]:list models:_fabric_models' \
-    '(-x --listcontexts)'{-x,--listcontexts}'[List all contexts]:list contexts:_fabric_contexts' \
-    '(-X --listsessions)'{-X,--listsessions}'[List all sessions]:list sessions:_fabric_sessions' \
-    '(--listextensions)--listextensions[List all registered extensions]' \
-    '(--liststrategies)--liststrategies[List all strategies]:list strategies:_fabric_strategies' \
-    '(--listvendors)--listvendors[List all vendors]' \
-  vendors=(${(f)"$(fabric --listvendors 2>/dev/null)"})
-  compadd -X "Vendors:" ${vendors}
+}
+
+_fabric_gemini_voices() {
+  local -a voices
+  local cmd=${words[1]}
+  voices=(${(f)"$($cmd --list-gemini-voices --shell-complete-list 2>/dev/null)"})
+  compadd -X "Gemini TTS Voices:" ${voices}
+}
+
+_fabric_transcription_models() {
+  local -a models
+  local cmd=${words[1]}
+  models=(${(f)"$($cmd --list-transcription-models --shell-complete-list 2>/dev/null)"})
+  compadd -X "Transcription Models:" ${models}
 }
 
 _fabric() {
@@ -59,7 +81,7 @@ _fabric() {
     '(-T --topp)'{-T,--topp}'[Set top P (default: 0.9)]:topp:' \
     '(-s --stream)'{-s,--stream}'[Stream]' \
     '(-P --presencepenalty)'{-P,--presencepenalty}'[Set presence penalty (default: 0.0)]:presence penalty:' \
-    '(-r --raw)'{-r,--raw}'[Use the defaults of the model without sending chat options]' \
+    '(-r --raw)'{-r,--raw}'[Use the defaults of the model without sending chat options. Only affects OpenAI-compatible providers. Anthropic models always use smart parameter selection to comply with model-specific requirements.]' \
     '(-F --frequencypenalty)'{-F,--frequencypenalty}'[Set frequency penalty (default: 0.0)]:frequency penalty:' \
     '(-l --listpatterns)'{-l,--listpatterns}'[List all patterns]' \
     '(-L --listmodels)'{-L,--listmodels}'[List all available models]' \
@@ -68,6 +90,7 @@ _fabric() {
     '(-U --updatepatterns)'{-U,--updatepatterns}'[Update patterns]' \
     '(-c --copy)'{-c,--copy}'[Copy to clipboard]' \
     '(-m --model)'{-m,--model}'[Choose model]:model:_fabric_models' \
+    '(-V --vendor)'{-V,--vendor}'[Specify vendor for chosen model (e.g., -V "LM Studio" -m openai/gpt-oss-20b)]:vendor:_fabric_vendors' \
     '(--modelContextLength)--modelContextLength[Model context length (only affects ollama)]:length:' \
     '(-o --output)'{-o,--output}'[Output to file]:file:_files' \
     '(--output-session)--output-session[Output the entire session to the output file]' \
@@ -79,16 +102,19 @@ _fabric() {
     '(--transcript-with-timestamps)--transcript-with-timestamps[Grab transcript from YouTube video with timestamps]' \
     '(--comments)--comments[Grab comments from YouTube video and send to chat]' \
     '(--metadata)--metadata[Output video metadata]' \
+    '(--yt-dlp-args)--yt-dlp-args[Additional arguments to pass to yt-dlp]:yt-dlp args:' \
     '(-g --language)'{-g,--language}'[Specify the Language Code for the chat, e.g. -g=en -g=zh]:language:' \
     '(-u --scrape_url)'{-u,--scrape_url}'[Scrape website URL to markdown using Jina AI]:url:' \
     '(-q --scrape_question)'{-q,--scrape_question}'[Search question using Jina AI]:question:' \
     '(-e --seed)'{-e,--seed}'[Seed to be used for LMM generation]:seed:' \
+    '(--thinking)--thinking[Set reasoning/thinking level]:level:(off low medium high)' \
     '(-w --wipecontext)'{-w,--wipecontext}'[Wipe context]:context:_fabric_contexts' \
     '(-W --wipesession)'{-W,--wipesession}'[Wipe session]:session:_fabric_sessions' \
     '(--printcontext)--printcontext[Print context]:context:_fabric_contexts' \
     '(--printsession)--printsession[Print session]:session:_fabric_sessions' \
     '(--readability)--readability[Convert HTML input into a clean, readable view]' \
     '(--input-has-vars)--input-has-vars[Apply variables to user input]' \
+    '(--no-variable-replacement)--no-variable-replacement[Disable pattern variable replacement]' \
     '(--dry-run)--dry-run[Show what would be sent to the model without actually sending it]' \
     '(--serve)--serve[Serve the Fabric Rest API]' \
     '(--serveOllama)--serveOllama[Serve the Fabric Rest API with ollama endpoints]' \
@@ -96,16 +122,34 @@ _fabric() {
     '(--api-key)--api-key[API key used to secure server routes]:api-key:' \
     '(--config)--config[Path to YAML config file]:config file:_files -g "*.yaml *.yml"' \
     '(--version)--version[Print current version]' \
+    '(--search)--search[Enable web search tool for supported models (Anthropic, OpenAI, Gemini)]' \
+    '(--search-location)--search-location[Set location for web search results]:location:' \
+    '(--image-file)--image-file[Save generated image to specified file path]:image file:_files -g "*.png *.webp *.jpeg *.jpg"' \
+    '(--image-size)--image-size[Image dimensions]:size:(1024x1024 1536x1024 1024x1536 auto)' \
+    '(--image-quality)--image-quality[Image quality]:quality:(low medium high auto)' \
+    '(--image-compression)--image-compression[Compression level 0-100 for JPEG/WebP formats]:compression:' \
+    '(--image-background)--image-background[Background type]:background:(opaque transparent)' \
     '(--listextensions)--listextensions[List all registered extensions]' \
     '(--addextension)--addextension[Register a new extension from config file path]:config file:_files -g "*.yaml *.yml"' \
     '(--rmextension)--rmextension[Remove a registered extension by name]:extension:_fabric_extensions' \
     '(--strategy)--strategy[Choose a strategy from the available strategies]:strategy:_fabric_strategies' \
     '(--liststrategies)--liststrategies[List all strategies]' \
     '(--listvendors)--listvendors[List all vendors]' \
+    '(--voice)--voice[TTS voice name for supported models]:voice:_fabric_gemini_voices' \
+    '(--list-gemini-voices)--list-gemini-voices[List all available Gemini TTS voices]' \
     '(--shell-complete-list)--shell-complete-list[Output raw list without headers/formatting (for shell completion)]' \
+    '(--suppress-think)--suppress-think[Suppress text enclosed in thinking tags]' \
+    '(--think-start-tag)--think-start-tag[Start tag for thinking sections (default: <think>)]:start tag:' \
+    '(--think-end-tag)--think-end-tag[End tag for thinking sections (default: </think>)]:end tag:' \
+    '(--disable-responses-api)--disable-responses-api[Disable OpenAI Responses API (default: false)]' \
+    '(--transcribe-file)--transcribe-file[Audio or video file to transcribe]:audio file:_files -g "*.mp3 *.mp4 *.mpeg *.mpga *.m4a *.wav *.webm"' \
+    '(--transcribe-model)--transcribe-model[Model to use for transcription (separate from chat model)]:transcribe model:_fabric_transcription_models' \
+    '(--split-media-file)--split-media-file[Split audio/video files larger than 25MB using ffmpeg]' \
+    '(--debug)--debug[Set debug level (0=off, 1=basic, 2=detailed, 3=trace)]:debug level:(0 1 2 3)' \
+    '(--notification)--notification[Send desktop notification when command completes]' \
+    '(--notification-command)--notification-command[Custom command to run for notifications]:notification command:' \
     '(-h --help)'{-h,--help}'[Show this help message]' \
     '*:arguments:'
 }
 
 _fabric "$@"
-

completions/fabric.bash

@@ -10,14 +10,18 @@
 
 _fabric() {
   local cur prev words cword
-  _get_comp_words_by_ref -n : cur prev words cword
+  if declare -F _comp_get_words &>/dev/null; then
+      _comp_get_words cur prev words cword
+   else
+      _get_comp_words_by_ref cur prev words cword
+   fi
 
   # Define all possible options/flags
-  local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --language -g --scrape_url -u --scrape_question -q --seed -e --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --dry-run --serve --serveOllama --address --api-key --config --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
+  local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --vendor -V --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --yt-dlp-args --language -g --scrape_url -u --scrape_question -q --seed -e --thinking --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --no-variable-replacement --dry-run --serve --serveOllama --address --api-key --config --search --search-location --image-file --image-size --image-quality --image-compression --image-background --suppress-think --think-start-tag --think-end-tag --disable-responses-api --transcribe-file --transcribe-model --split-media-file --voice --list-gemini-voices --notification --notification-command --debug --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
 
   # Helper function for dynamic completions
   _fabric_get_list() {
-    fabric "$1" --shell-complete-list 2>/dev/null
+    "${COMP_WORDS[0]}" "$1" --shell-complete-list 2>/dev/null
   }
 
   # Handle completions based on the previous word
@@ -38,6 +42,10 @@ _fabric() {
     COMPREPLY=($(compgen -W "$(_fabric_get_list --listmodels)" -- "${cur}"))
     return 0
     ;;
+  -V | --vendor)
+    COMPREPLY=($(compgen -W "$(_fabric_get_list --listvendors)" -- "${cur}"))
+    return 0
+    ;;
   -w | --wipecontext)
     COMPREPLY=($(compgen -W "$(_fabric_get_list --listcontexts)" -- "${cur}"))
     return 0
@@ -54,6 +62,10 @@ _fabric() {
     COMPREPLY=($(compgen -W "$(_fabric_get_list --listsessions)" -- "${cur}"))
     return 0
     ;;
+  --thinking)
+    COMPREPLY=($(compgen -W "off low medium high" -- "${cur}"))
+    return 0
+    ;;
   --rmextension)
     COMPREPLY=($(compgen -W "$(_fabric_get_list --listextensions)" -- "${cur}"))
     return 0
@@ -62,13 +74,38 @@ _fabric() {
     COMPREPLY=($(compgen -W "$(_fabric_get_list --liststrategies)" -- "${cur}"))
     return 0
     ;;
+  --voice)
+    COMPREPLY=($(compgen -W "$(_fabric_get_list --list-gemini-voices)" -- "${cur}"))
+    return 0
+    ;;
+  --transcribe-model)
+    COMPREPLY=($(compgen -W "$(_fabric_get_list --list-transcription-models)" -- "${cur}"))
+    return 0
+    ;;
+  --debug)
+    COMPREPLY=($(compgen -W "0 1 2 3" -- "${cur}"))
+    return 0
+    ;;
   # Options requiring file/directory paths
-  -a | --attachment | -o | --output | --config | --addextension)
+  -a | --attachment | -o | --output | --config | --addextension | --image-file | --transcribe-file)
     _filedir
     return 0
     ;;
+  # Image generation options with specific values
+  --image-size)
+    COMPREPLY=($(compgen -W "1024x1024 1536x1024 1024x1536 auto" -- "$cur"))
+    return 0
+    ;;
+  --image-quality)
+    COMPREPLY=($(compgen -W "low medium high auto" -- "$cur"))
+    return 0
+    ;;
+  --image-background)
+    COMPREPLY=($(compgen -W "opaque transparent" -- "$cur"))
+    return 0
+    ;;
   # Options requiring simple arguments (no specific completion logic here)
-  -v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key)
+  -v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | --yt-dlp-args | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key | --search-location | --image-compression | --think-start-tag | --think-end-tag | --notification-command)
     # No specific completion suggestions, user types the value
     return 0
     ;;
@@ -87,4 +124,4 @@ _fabric() {
 
 }
 
-complete -F _fabric fabric
+complete -F _fabric fabric fabric-ai

completions/fabric.fish

@@ -8,87 +8,137 @@
 
 # Helper functions for dynamic completions
 function __fabric_get_patterns
-	fabric --listpatterns --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --listpatterns --shell-complete-list 2>/dev/null
 end
 
 function __fabric_get_models
-	fabric --listmodels --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --listmodels --shell-complete-list 2>/dev/null
+end
+
+function __fabric_get_vendors
+        set cmd (commandline -opc)[1]
+        $cmd --listvendors --shell-complete-list 2>/dev/null
 end
 
 function __fabric_get_contexts
-	fabric --listcontexts --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --listcontexts --shell-complete-list 2>/dev/null
 end
 
 function __fabric_get_sessions
-	fabric --listsessions --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --listsessions --shell-complete-list 2>/dev/null
 end
 
 function __fabric_get_strategies
-	fabric --liststrategies --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --liststrategies --shell-complete-list 2>/dev/null
 end
 
 function __fabric_get_extensions
-	fabric --listextensions --shell-complete-list 2>/dev/null
+        set cmd (commandline -opc)[1]
+        $cmd --listextensions --shell-complete-list 2>/dev/null
+end
+
+function __fabric_get_gemini_voices
+        set cmd (commandline -opc)[1]
+        $cmd --list-gemini-voices --shell-complete-list 2>/dev/null
+end
+
+function __fabric_get_transcription_models
+        set cmd (commandline -opc)[1]
+        $cmd --list-transcription-models --shell-complete-list 2>/dev/null
 end
 
 # Main completion function
-complete -c fabric -f
+function __fabric_register_completions
+        set cmd $argv[1]
+        complete -c $cmd -f
 
-# Flag completions with arguments
-complete -c fabric -s p -l pattern -d "Choose a pattern from the available patterns" -a "(__fabric_get_patterns)"
-complete -c fabric -s v -l variable -d "Values for pattern variables, e.g. -v=#role:expert -v=#points:30"
-complete -c fabric -s C -l context -d "Choose a context from the available contexts" -a "(__fabric_get_contexts)"
-complete -c fabric -l session -d "Choose a session from the available sessions" -a "(__fabric_get_sessions)"
-complete -c fabric -s a -l attachment -d "Attachment path or URL (e.g. for OpenAI image recognition messages)" -r
-complete -c fabric -s t -l temperature -d "Set temperature (default: 0.7)"
-complete -c fabric -s T -l topp -d "Set top P (default: 0.9)"
-complete -c fabric -s P -l presencepenalty -d "Set presence penalty (default: 0.0)"
-complete -c fabric -s F -l frequencypenalty -d "Set frequency penalty (default: 0.0)"
-complete -c fabric -s m -l model -d "Choose model" -a "(__fabric_get_models)"
-complete -c fabric -l modelContextLength -d "Model context length (only affects ollama)"
-complete -c fabric -s o -l output -d "Output to file" -r
-complete -c fabric -s n -l latest -d "Number of latest patterns to list (default: 0)"
-complete -c fabric -s y -l youtube -d "YouTube video or play list URL to grab transcript, comments from it"
-complete -c fabric -s g -l language -d "Specify the Language Code for the chat, e.g. -g=en -g=zh"
-complete -c fabric -s u -l scrape_url -d "Scrape website URL to markdown using Jina AI"
-complete -c fabric -s q -l scrape_question -d "Search question using Jina AI"
-complete -c fabric -s e -l seed -d "Seed to be used for LMM generation"
-complete -c fabric -s w -l wipecontext -d "Wipe context" -a "(__fabric_get_contexts)"
-complete -c fabric -s W -l wipesession -d "Wipe session" -a "(__fabric_get_sessions)"
-complete -c fabric -l printcontext -d "Print context" -a "(__fabric_get_contexts)"
-complete -c fabric -l printsession -d "Print session" -a "(__fabric_get_sessions)"
-complete -c fabric -l address -d "The address to bind the REST API (default: :8080)"
-complete -c fabric -l api-key -d "API key used to secure server routes"
-complete -c fabric -l config -d "Path to YAML config file" -r -a "*.yaml *.yml"
-complete -c fabric -l addextension -d "Register a new extension from config file path" -r -a "*.yaml *.yml"
-complete -c fabric -l rmextension -d "Remove a registered extension by name" -a "(__fabric_get_extensions)"
-complete -c fabric -l strategy -d "Choose a strategy from the available strategies" -a "(__fabric_get_strategies)"
+        # Flag completions with arguments
+        complete -c $cmd -s p -l pattern -d "Choose a pattern from the available patterns" -a "(__fabric_get_patterns)"
+        complete -c $cmd -s v -l variable -d "Values for pattern variables, e.g. -v=#role:expert -v=#points:30"
+        complete -c $cmd -s C -l context -d "Choose a context from the available contexts" -a "(__fabric_get_contexts)"
+        complete -c $cmd -l session -d "Choose a session from the available sessions" -a "(__fabric_get_sessions)"
+        complete -c $cmd -s a -l attachment -d "Attachment path or URL (e.g. for OpenAI image recognition messages)" -r
+        complete -c $cmd -s t -l temperature -d "Set temperature (default: 0.7)"
+        complete -c $cmd -s T -l topp -d "Set top P (default: 0.9)"
+        complete -c $cmd -s P -l presencepenalty -d "Set presence penalty (default: 0.0)"
+        complete -c $cmd -s F -l frequencypenalty -d "Set frequency penalty (default: 0.0)"
+        complete -c $cmd -s m -l model -d "Choose model" -a "(__fabric_get_models)"
+        complete -c $cmd -s V -l vendor -d "Specify vendor for chosen model (e.g., -V \"LM Studio\" -m openai/gpt-oss-20b)" -a "(__fabric_get_vendors)"
+        complete -c $cmd -l modelContextLength -d "Model context length (only affects ollama)"
+        complete -c $cmd -s o -l output -d "Output to file" -r
+        complete -c $cmd -s n -l latest -d "Number of latest patterns to list (default: 0)"
+        complete -c $cmd -s y -l youtube -d "YouTube video or play list URL to grab transcript, comments from it"
+        complete -c $cmd -s g -l language -d "Specify the Language Code for the chat, e.g. -g=en -g=zh"
+        complete -c $cmd -s u -l scrape_url -d "Scrape website URL to markdown using Jina AI"
+        complete -c $cmd -s q -l scrape_question -d "Search question using Jina AI"
+        complete -c $cmd -s e -l seed -d "Seed to be used for LMM generation"
+        complete -c $cmd -l thinking -d "Set reasoning/thinking level" -a "off low medium high"
+        complete -c $cmd -s w -l wipecontext -d "Wipe context" -a "(__fabric_get_contexts)"
+        complete -c $cmd -s W -l wipesession -d "Wipe session" -a "(__fabric_get_sessions)"
+        complete -c $cmd -l printcontext -d "Print context" -a "(__fabric_get_contexts)"
+        complete -c $cmd -l printsession -d "Print session" -a "(__fabric_get_sessions)"
+        complete -c $cmd -l address -d "The address to bind the REST API (default: :8080)"
+        complete -c $cmd -l api-key -d "API key used to secure server routes"
+        complete -c $cmd -l config -d "Path to YAML config file" -r -a "*.yaml *.yml"
+        complete -c $cmd -l search-location -d "Set location for web search results (e.g., 'America/Los_Angeles')"
+        complete -c $cmd -l image-file -d "Save generated image to specified file path (e.g., 'output.png')" -r -a "*.png *.webp *.jpeg *.jpg"
+        complete -c $cmd -l image-size -d "Image dimensions: 1024x1024, 1536x1024, 1024x1536, auto (default: auto)" -a "1024x1024 1536x1024 1024x1536 auto"
+        complete -c $cmd -l image-quality -d "Image quality: low, medium, high, auto (default: auto)" -a "low medium high auto"
+        complete -c $cmd -l image-compression -d "Compression level 0-100 for JPEG/WebP formats (default: not set)" -r
+        complete -c $cmd -l image-background -d "Background type: opaque, transparent (default: opaque, only for PNG/WebP)" -a "opaque transparent"
+        complete -c $cmd -l addextension -d "Register a new extension from config file path" -r -a "*.yaml *.yml"
+        complete -c $cmd -l rmextension -d "Remove a registered extension by name" -a "(__fabric_get_extensions)"
+        complete -c $cmd -l strategy -d "Choose a strategy from the available strategies" -a "(__fabric_get_strategies)"
+        complete -c $cmd -l think-start-tag -d "Start tag for thinking sections (default: <think>)"
+        complete -c $cmd -l think-end-tag -d "End tag for thinking sections (default: </think>)"
+        complete -c $cmd -l voice -d "TTS voice name for supported models (e.g., Kore, Charon, Puck)" -a "(__fabric_get_gemini_voices)"
+        complete -c $cmd -l transcribe-file -d "Audio or video file to transcribe" -r -a "*.mp3 *.mp4 *.mpeg *.mpga *.m4a *.wav *.webm"
+        complete -c $cmd -l transcribe-model -d "Model to use for transcription (separate from chat model)" -a "(__fabric_get_transcription_models)"
+        complete -c $cmd -l debug -d "Set debug level (0=off, 1=basic, 2=detailed, 3=trace)" -a "0 1 2 3"
+        complete -c $cmd -l notification-command -d "Custom command to run for notifications (overrides built-in notifications)"
 
-# Boolean flags (no arguments)
-complete -c fabric -s S -l setup -d "Run setup for all reconfigurable parts of fabric"
-complete -c fabric -s s -l stream -d "Stream"
-complete -c fabric -s r -l raw -d "Use the defaults of the model without sending chat options"
-complete -c fabric -s l -l listpatterns -d "List all patterns"
-complete -c fabric -s L -l listmodels -d "List all available models"
-complete -c fabric -s x -l listcontexts -d "List all contexts"
-complete -c fabric -s X -l listsessions -d "List all sessions"
-complete -c fabric -s U -l updatepatterns -d "Update patterns"
-complete -c fabric -s c -l copy -d "Copy to clipboard"
-complete -c fabric -l output-session -d "Output the entire session to the output file"
-complete -c fabric -s d -l changeDefaultModel -d "Change default model"
-complete -c fabric -l playlist -d "Prefer playlist over video if both ids are present in the URL"
-complete -c fabric -l transcript -d "Grab transcript from YouTube video and send to chat"
-complete -c fabric -l transcript-with-timestamps -d "Grab transcript from YouTube video with timestamps"
-complete -c fabric -l comments -d "Grab comments from YouTube video and send to chat"
-complete -c fabric -l metadata -d "Output video metadata"
-complete -c fabric -l readability -d "Convert HTML input into a clean, readable view"
-complete -c fabric -l input-has-vars -d "Apply variables to user input"
-complete -c fabric -l dry-run -d "Show what would be sent to the model without actually sending it"
-complete -c fabric -l serve -d "Serve the Fabric Rest API"
-complete -c fabric -l serveOllama -d "Serve the Fabric Rest API with ollama endpoints"
-complete -c fabric -l version -d "Print current version"
-complete -c fabric -l listextensions -d "List all registered extensions"
-complete -c fabric -l liststrategies -d "List all strategies"
-complete -c fabric -l listvendors -d "List all vendors"
-complete -c fabric -l shell-complete-list -d "Output raw list without headers/formatting (for shell completion)"
-complete -c fabric -s h -l help -d "Show this help message"
+        # Boolean flags (no arguments)
+        complete -c $cmd -s S -l setup -d "Run setup for all reconfigurable parts of fabric"
+        complete -c $cmd -s s -l stream -d "Stream"
+        complete -c $cmd -s r -l raw -d "Use the defaults of the model without sending chat options. Only affects OpenAI-compatible providers. Anthropic models always use smart parameter selection to comply with model-specific requirements."
+        complete -c $cmd -s l -l listpatterns -d "List all patterns"
+        complete -c $cmd -s L -l listmodels -d "List all available models"
+        complete -c $cmd -s x -l listcontexts -d "List all contexts"
+        complete -c $cmd -s X -l listsessions -d "List all sessions"
+        complete -c $cmd -s U -l updatepatterns -d "Update patterns"
+        complete -c $cmd -s c -l copy -d "Copy to clipboard"
+        complete -c $cmd -l output-session -d "Output the entire session to the output file"
+        complete -c $cmd -s d -l changeDefaultModel -d "Change default model"
+        complete -c $cmd -l playlist -d "Prefer playlist over video if both ids are present in the URL"
+        complete -c $cmd -l transcript -d "Grab transcript from YouTube video and send to chat"
+        complete -c $cmd -l transcript-with-timestamps -d "Grab transcript from YouTube video with timestamps"
+        complete -c $cmd -l comments -d "Grab comments from YouTube video and send to chat"
+        complete -c $cmd -l metadata -d "Output video metadata"
+        complete -c $cmd -l yt-dlp-args -d "Additional arguments to pass to yt-dlp (e.g. '--cookies-from-browser brave')"
+        complete -c $cmd -l readability -d "Convert HTML input into a clean, readable view"
+       complete -c $cmd -l input-has-vars -d "Apply variables to user input"
+       complete -c $cmd -l no-variable-replacement -d "Disable pattern variable replacement"
+       complete -c $cmd -l dry-run -d "Show what would be sent to the model without actually sending it"
+        complete -c $cmd -l search -d "Enable web search tool for supported models (Anthropic, OpenAI, Gemini)"
+        complete -c $cmd -l serve -d "Serve the Fabric Rest API"
+        complete -c $cmd -l serveOllama -d "Serve the Fabric Rest API with ollama endpoints"
+        complete -c $cmd -l version -d "Print current version"
+        complete -c $cmd -l listextensions -d "List all registered extensions"
+        complete -c $cmd -l liststrategies -d "List all strategies"
+        complete -c $cmd -l listvendors -d "List all vendors"
+        complete -c $cmd -l list-gemini-voices -d "List all available Gemini TTS voices"
+        complete -c $cmd -l shell-complete-list -d "Output raw list without headers/formatting (for shell completion)"
+        complete -c $cmd -l suppress-think -d "Suppress text enclosed in thinking tags"
+        complete -c $cmd -l disable-responses-api -d "Disable OpenAI Responses API (default: false)"
+        complete -c $cmd -l split-media-file -d "Split audio/video files larger than 25MB using ffmpeg"
+        complete -c $cmd -l notification -d "Send desktop notification when command completes"
+        complete -c $cmd -s h -l help -d "Show this help message"
+end
+
+__fabric_register_completions fabric
+__fabric_register_completions fabric-ai

completions/setup-completions.sh

@@ -0,0 +1,503 @@
+#!/bin/sh
+
+# Fabric Shell Completions Setup Script
+# This script automatically installs shell completions for the fabric CLI
+# based on your current shell and the installed fabric command name.
+
+set -e
+
+# Global variables
+DRY_RUN=false
+# Base URL to fetch completion files when not available locally
+# Can be overridden via environment variable FABRIC_COMPLETIONS_BASE_URL
+FABRIC_COMPLETIONS_BASE_URL="${FABRIC_COMPLETIONS_BASE_URL:-https://raw.githubusercontent.com/danielmiessler/Fabric/refs/heads/main/completions}"
+TEMP_DIR=""
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+CYAN='\033[0;36m'
+NC='\033[0m' # No Color
+
+# Function to print colored output
+print_info() {
+    printf "${BLUE}[INFO]${NC} %s\n" "$1"
+}
+
+print_success() {
+    printf "${GREEN}[SUCCESS]${NC} %s\n" "$1"
+}
+
+print_warning() {
+    printf "${YELLOW}[WARNING]${NC} %s\n" "$1"
+}
+
+print_error() {
+    printf "${RED}[ERROR]${NC} %s\n" "$1"
+}
+
+print_dry_run() {
+    printf "${CYAN}[DRY-RUN]${NC} %s\n" "$1"
+}
+
+# Function to execute commands with dry-run support
+execute_command() {
+    cmd="$1"
+
+    if [ "$DRY_RUN" = true ]; then
+        print_dry_run "Would run: $cmd"
+        return 0
+    else
+        eval "$cmd" 2>/dev/null
+    fi
+}
+
+# Simple downloader that prefers curl, falls back to wget
+to_github_raw_url() {
+    in_url="$1"
+    case "$in_url" in
+        https://github.com/*/*/blob/*)
+            # Convert blob URL to raw
+            # https://github.com/{owner}/{repo}/blob/{ref}/path -> https://raw.githubusercontent.com/{owner}/{repo}/{ref}/path
+            echo "$in_url" | sed -E 's#https://github.com/([^/]+)/([^/]+)/blob/([^/]+)/#https://raw.githubusercontent.com/\1/\2/\3/#'
+            ;;
+        https://github.com/*/*/tree/*)
+            # Convert tree URL base + file path to raw
+            # https://github.com/{owner}/{repo}/tree/{ref}/path -> https://raw.githubusercontent.com/{owner}/{repo}/{ref}/path
+            echo "$in_url" | sed -E 's#https://github.com/([^/]+)/([^/]+)/tree/([^/]+)/#https://raw.githubusercontent.com/\1/\2/\3/#'
+            ;;
+        *)
+            echo "$in_url"
+            ;;
+    esac
+}
+
+# Simple downloader that prefers curl, falls back to wget
+download_file() {
+    url="$1"
+    dest="$2"
+
+    if [ "$DRY_RUN" = true ]; then
+        print_dry_run "Would download: $url -> $dest"
+        return 0
+    fi
+
+    eff_url="$(to_github_raw_url "$url")"
+
+    if command -v curl >/dev/null 2>&1; then
+        curl -fsSL "$eff_url" -o "$dest"
+        return $?
+    elif command -v wget >/dev/null 2>&1; then
+        wget -q "$eff_url" -O "$dest"
+        return $?
+    else
+        print_error "Neither 'curl' nor 'wget' is available to download: $url"
+        return 1
+    fi
+}
+
+# Attempt to obtain completion files. If local copies are missing,
+# download them into a temporary directory and return that directory path.
+obtain_completion_files() {
+    obf_script_dir="$1"
+    obf_need_download=false
+
+    if [ ! -f "$obf_script_dir/_fabric" ] || [ ! -f "$obf_script_dir/fabric.bash" ] || [ ! -f "$obf_script_dir/fabric.fish" ]; then
+        obf_need_download=true
+    fi
+
+    if [ "$obf_need_download" = false ]; then
+        echo "$obf_script_dir"
+        return 0
+    fi
+
+    # Note: write only to stderr in this function except for the final echo which returns the path
+    printf "%s\n" "[INFO] Local completion files not found; will download from GitHub." 1>&2
+    printf "%s\n" "[INFO] Source: $FABRIC_COMPLETIONS_BASE_URL" 1>&2
+
+    if [ "$DRY_RUN" = true ]; then
+    printf "%s\n" "[DRY-RUN] Would create temporary directory for downloads" 1>&2
+    echo "$obf_script_dir" # Keep using original for dry-run copies
+        return 0
+    fi
+
+    TEMP_DIR="$(mktemp -d 2>/dev/null || mktemp -d -t fabric-completions)"
+    if [ ! -d "$TEMP_DIR" ]; then
+    print_error "Failed to create temporary directory for downloads."
+        return 1
+    fi
+
+    if ! download_file "$FABRIC_COMPLETIONS_BASE_URL/_fabric" "$TEMP_DIR/_fabric"; then
+        print_error "Failed to download _fabric"
+        return 1
+    fi
+    if [ ! -s "$TEMP_DIR/_fabric" ] || head -n1 "$TEMP_DIR/_fabric" | grep -qi "^<!DOCTYPE\|^<html"; then
+        print_error "Downloaded _fabric appears invalid (empty or HTML). Check FABRIC_COMPLETIONS_BASE_URL."
+        return 1
+    fi
+    if ! download_file "$FABRIC_COMPLETIONS_BASE_URL/fabric.bash" "$TEMP_DIR/fabric.bash"; then
+        print_error "Failed to download fabric.bash"
+        return 1
+    fi
+    if [ ! -s "$TEMP_DIR/fabric.bash" ] || head -n1 "$TEMP_DIR/fabric.bash" | grep -qi "^<!DOCTYPE\|^<html"; then
+        print_error "Downloaded fabric.bash appears invalid (empty or HTML). Check FABRIC_COMPLETIONS_BASE_URL."
+        return 1
+    fi
+    if ! download_file "$FABRIC_COMPLETIONS_BASE_URL/fabric.fish" "$TEMP_DIR/fabric.fish"; then
+        print_error "Failed to download fabric.fish"
+        return 1
+    fi
+    if [ ! -s "$TEMP_DIR/fabric.fish" ] || head -n1 "$TEMP_DIR/fabric.fish" | grep -qi "^<!DOCTYPE\|^<html"; then
+        print_error "Downloaded fabric.fish appears invalid (empty or HTML). Check FABRIC_COMPLETIONS_BASE_URL."
+        return 1
+    fi
+
+    echo "$TEMP_DIR"
+}
+
+# Ensure directory exists, try sudo on permission failure
+ensure_dir() {
+    dir="$1"
+    # Expand ~ if present
+    case "$dir" in
+        ~/*)
+            dir="$HOME${dir#~}"
+            ;;
+    esac
+
+    if [ -d "$dir" ]; then
+        return 0
+    fi
+
+    if [ "$DRY_RUN" = true ]; then
+        print_dry_run "Would run: mkdir -p \"$dir\""
+        print_dry_run "If permission denied, would run: sudo mkdir -p \"$dir\""
+        return 0
+    fi
+
+    if mkdir -p "$dir" 2>/dev/null; then
+        return 0
+    fi
+    if command -v sudo >/dev/null 2>&1 && sudo mkdir -p "$dir" 2>/dev/null; then
+        return 0
+    fi
+    print_error "Failed to create directory: $dir"
+    return 1
+}
+
+# Copy file with sudo fallback on permission failure
+install_file() {
+    src="$1"
+    dest="$2"
+
+    if [ "$DRY_RUN" = true ]; then
+        print_dry_run "Would run: cp \"$src\" \"$dest\""
+        print_dry_run "If permission denied, would run: sudo cp \"$src\" \"$dest\""
+        return 0
+    fi
+
+    if cp "$src" "$dest" 2>/dev/null; then
+        return 0
+    fi
+    if command -v sudo >/dev/null 2>&1 && sudo cp "$src" "$dest" 2>/dev/null; then
+        return 0
+    fi
+    print_error "Failed to install file to: $dest"
+    return 1
+}
+
+# Function to detect fabric command name
+detect_fabric_command() {
+    if command -v fabric >/dev/null 2>&1; then
+        echo "fabric"
+    elif command -v fabric-ai >/dev/null 2>&1; then
+        echo "fabric-ai"
+    else
+        print_error "Neither 'fabric' nor 'fabric-ai' command found in PATH"
+        exit 1
+    fi
+}
+
+# Function to detect shell
+detect_shell() {
+    if [ -n "$SHELL" ]; then
+        basename "$SHELL"
+    else
+        print_warning "SHELL environment variable not set, defaulting to sh"
+        echo "sh"
+    fi
+}
+
+# Function to get script directory
+get_script_dir() {
+    # Get the directory where this script is located
+    script_path="$(readlink -f "$0" 2>/dev/null || realpath "$0" 2>/dev/null || echo "$0")"
+    dirname "$script_path"
+}
+
+# Function to setup Zsh completions
+setup_zsh_completions() {
+    fabric_cmd="$1"
+    script_dir="$2"
+    completion_file="_${fabric_cmd}"
+
+    print_info "Setting up Zsh completions for '$fabric_cmd'..."
+
+    # Try to use existing $fpath first, then fall back to default directories
+    zsh_dirs=""
+
+    # Check if user's shell is zsh and try to get fpath from it
+    if [ "$(basename "$SHELL")" = "zsh" ] && command -v zsh >/dev/null 2>&1; then
+        # Get fpath from zsh by sourcing user's .zshrc first
+        fpath_output=$(zsh -c "source \$HOME/.zshrc 2>/dev/null && print -l \$fpath" 2>/dev/null | head -5 | tr '\n' ' ')
+        if [ -n "$fpath_output" ] && [ "$fpath_output" != "" ]; then
+            print_info "Using directories from zsh \$fpath"
+            zsh_dirs="$fpath_output"
+        fi
+    fi
+
+    # If we couldn't get fpath or it's empty, use default directories
+    if [ -z "$zsh_dirs" ] || [ "$zsh_dirs" = "" ]; then
+        print_info "Using default zsh completion directories"
+        zsh_dirs="/usr/local/share/zsh/site-functions /opt/homebrew/share/zsh/site-functions /usr/share/zsh/site-functions ~/.local/share/zsh/site-functions"
+    fi
+
+    installed=false
+
+    for dir in $zsh_dirs; do
+        # Create directory (with sudo fallback if needed)
+        if ensure_dir "$dir"; then
+            if install_file "$script_dir/_fabric" "$dir/$completion_file"; then
+                if [ "$DRY_RUN" = true ]; then
+                    print_success "Would install Zsh completion to: $dir/$completion_file"
+                else
+                    print_success "Installed Zsh completion to: $dir/$completion_file"
+                fi
+                installed=true
+                break
+            fi
+        fi
+    done
+
+    if [ "$installed" = false ]; then
+        if [ "$DRY_RUN" = true ]; then
+            print_warning "Would attempt to install Zsh completions but no writable directory found."
+        else
+            print_error "Failed to install Zsh completions. Try running with sudo or check permissions."
+            return 1
+        fi
+    fi
+
+    if [ "$DRY_RUN" = true ]; then
+        print_info "Would suggest: Restart your shell or run 'autoload -U compinit && compinit' to enable completions."
+    else
+        print_info "Restart your shell or run 'autoload -U compinit && compinit' to enable completions."
+    fi
+}
+
+# Function to setup Bash completions
+setup_bash_completions() {
+    fabric_cmd="$1"
+    script_dir="$2"
+    completion_file="${fabric_cmd}.bash"
+
+    print_info "Setting up Bash completions for '$fabric_cmd'..."
+
+    # Try different completion directories
+    bash_dirs="/etc/bash_completion.d /usr/local/etc/bash_completion.d /opt/homebrew/etc/bash_completion.d ~/.local/share/bash-completion/completions"
+    installed=false
+
+    for dir in $bash_dirs; do
+        if ensure_dir "$dir"; then
+            if install_file "$script_dir/fabric.bash" "$dir/$completion_file"; then
+                if [ "$DRY_RUN" = true ]; then
+                    print_success "Would install Bash completion to: $dir/$completion_file"
+                else
+                    print_success "Installed Bash completion to: $dir/$completion_file"
+                fi
+                installed=true
+                break
+            fi
+        fi
+    done
+
+    if [ "$installed" = false ]; then
+        if [ "$DRY_RUN" = true ]; then
+            print_warning "Would attempt to install Bash completions but no writable directory found."
+        else
+            print_error "Failed to install Bash completions. Try running with sudo or check permissions."
+            return 1
+        fi
+    fi
+
+    if [ "$DRY_RUN" = true ]; then
+        print_info "Would suggest: Restart your shell or run 'source ~/.bashrc' to enable completions."
+    else
+        print_info "Restart your shell or run 'source ~/.bashrc' to enable completions."
+    fi
+}
+
+# Function to setup Fish completions
+setup_fish_completions() {
+    fabric_cmd="$1"
+    script_dir="$2"
+    completion_file="${fabric_cmd}.fish"
+
+    print_info "Setting up Fish completions for '$fabric_cmd'..."
+
+    # Fish completion directory
+    fish_dir="$HOME/.config/fish/completions"
+
+    if [ "$DRY_RUN" = true ]; then
+        print_dry_run "Would run: mkdir -p \"$fish_dir\""
+        print_dry_run "Would run: cp \"$script_dir/fabric.fish\" \"$fish_dir/$completion_file\""
+        print_success "Would install Fish completion to: $fish_dir/$completion_file"
+        print_info "Fish will automatically load the completions (no restart needed)."
+    elif mkdir -p "$fish_dir" 2>/dev/null; then
+        if cp "$script_dir/fabric.fish" "$fish_dir/$completion_file"; then
+            print_success "Installed Fish completion to: $fish_dir/$completion_file"
+            print_info "Fish will automatically load the completions (no restart needed)."
+        else
+            print_error "Failed to copy Fish completion file."
+            return 1
+        fi
+    else
+        print_error "Failed to create Fish completions directory: $fish_dir"
+        return 1
+    fi
+}
+
+# Function to setup completions for other shells
+setup_other_shell_completions() {
+    fabric_cmd="$1"
+    shell_name="$2"
+    script_dir="$3"
+
+    print_warning "Shell '$shell_name' is not directly supported."
+    print_info "You can manually source the completion files:"
+    print_info "  Bash-compatible: source $script_dir/fabric.bash"
+    print_info "  Zsh-compatible: source $script_dir/_fabric"
+}
+
+# Function to show help
+show_help() {
+    cat << EOF
+Fabric Shell Completions Setup Script
+
+USAGE:
+    setup-completions.sh [OPTIONS]
+
+OPTIONS:
+    --dry-run    Show what commands would be run without executing them
+    --help       Show this help message
+
+DESCRIPTION:
+    This script automatically installs shell completions for the fabric CLI
+    based on your current shell and the installed fabric command name.
+
+        The script will use completion files from the same directory as the script
+        when available. If they are not present (e.g., when running via curl), it
+        will download them from GitHub:
+
+            $FABRIC_COMPLETIONS_BASE_URL
+
+        You can override the download source by setting
+        FABRIC_COMPLETIONS_BASE_URL to your preferred location.
+
+    Supports: zsh, bash, fish
+
+    The script will:
+    1. Detect whether 'fabric' or 'fabric-ai' is installed
+    2. Detect your current shell from the SHELL environment variable
+    3. Install the appropriate completion file with the correct name
+    4. Try multiple standard completion directories
+
+EXAMPLES:
+        ./setup-completions.sh                  # Install completions
+        ./setup-completions.sh --dry-run        # Show what would be done
+        FABRIC_COMPLETIONS_BASE_URL="https://raw.githubusercontent.com/<owner>/<repo>/main/completions" \\
+            ./setup-completions.sh               # Override download source
+        ./setup-completions.sh --help           # Show this help
+
+EOF
+}
+
+# Main function
+main() {
+    # Parse command line arguments
+    while [ $# -gt 0 ]; do
+        case "$1" in
+            --dry-run)
+                DRY_RUN=true
+                shift
+                ;;
+            --help|-h)
+                show_help
+                exit 0
+                ;;
+            *)
+                print_error "Unknown option: $1"
+                print_info "Use --help for usage information."
+                exit 1
+                ;;
+        esac
+    done
+
+    print_info "Fabric Shell Completions Setup"
+    print_info "==============================="
+
+    if [ "$DRY_RUN" = true ]; then
+        print_info "DRY RUN MODE - Commands will be shown but not executed"
+        print_info ""
+    fi
+
+    # Get script directory and obtain completion files (local or downloaded)
+    script_dir="$(get_script_dir)"
+    script_dir="$(obtain_completion_files "$script_dir" || echo "")"
+    if [ -z "$script_dir" ]; then
+        print_error "Unable to obtain completion files. Aborting."
+        exit 1
+    fi
+
+    # If we downloaded into a temp dir, arrange cleanup at process exit
+    if [ -n "$TEMP_DIR" ] && [ -d "$TEMP_DIR" ]; then
+        trap 'if [ -n "$TEMP_DIR" ] && [ -d "$TEMP_DIR" ]; then rm -rf "$TEMP_DIR"; fi' EXIT INT TERM
+    fi
+
+    # Detect fabric command
+    fabric_cmd="$(detect_fabric_command)"
+    print_info "Detected fabric command: $fabric_cmd"
+
+    # Detect shell
+    shell_name="$(detect_shell)"
+    print_info "Detected shell: $shell_name"
+
+    # Setup completions based on shell
+    case "$shell_name" in
+        zsh)
+            setup_zsh_completions "$fabric_cmd" "$script_dir"
+            ;;
+        bash)
+            setup_bash_completions "$fabric_cmd" "$script_dir"
+            ;;
+        fish)
+            setup_fish_completions "$fabric_cmd" "$script_dir"
+            ;;
+        *)
+            setup_other_shell_completions "$fabric_cmd" "$shell_name" "$script_dir"
+            ;;
+    esac
+
+    if [ "$DRY_RUN" = true ]; then
+        print_success "Dry-run completed! The above commands would set up shell completions."
+        print_info "Run without --dry-run to actually install the completions."
+    else
+        print_success "Shell completion setup completed!"
+        print_info "You can now use tab completion with the '$fabric_cmd' command."
+    fi
+}
+
+# Run main function
+main "$@"

core/plugin_registry.go

@@ -1,321 +0,0 @@
-package core
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"os"
-	"path/filepath"
-	"sort"
-	"strconv"
-	"strings"
-
-	"github.com/danielmiessler/fabric/plugins/ai/bedrock"
-	"github.com/danielmiessler/fabric/plugins/ai/exolab"
-	"github.com/danielmiessler/fabric/plugins/ai/perplexity" // Added Perplexity plugin
-	"github.com/danielmiessler/fabric/plugins/strategy"
-
-	"github.com/samber/lo"
-
-	"github.com/danielmiessler/fabric/common"
-	"github.com/danielmiessler/fabric/plugins"
-	"github.com/danielmiessler/fabric/plugins/ai"
-	"github.com/danielmiessler/fabric/plugins/ai/anthropic"
-	"github.com/danielmiessler/fabric/plugins/ai/azure"
-	"github.com/danielmiessler/fabric/plugins/ai/dryrun"
-	"github.com/danielmiessler/fabric/plugins/ai/gemini"
-	"github.com/danielmiessler/fabric/plugins/ai/lmstudio"
-	"github.com/danielmiessler/fabric/plugins/ai/ollama"
-	"github.com/danielmiessler/fabric/plugins/ai/openai"
-	"github.com/danielmiessler/fabric/plugins/ai/openai_compatible"
-	"github.com/danielmiessler/fabric/plugins/db/fsdb"
-	"github.com/danielmiessler/fabric/plugins/template"
-	"github.com/danielmiessler/fabric/plugins/tools"
-	"github.com/danielmiessler/fabric/plugins/tools/jina"
-	"github.com/danielmiessler/fabric/plugins/tools/lang"
-	"github.com/danielmiessler/fabric/plugins/tools/youtube"
-)
-
-// hasAWSCredentials checks if any AWS credentials are present either in the
-// environment variables or in the default/shared credentials file. It doesn't
-// attempt to verify the validity of the credentials, but simply ensures that a
-// potential authentication source exists so we can safely initialize the
-// Bedrock client without causing the AWS SDK to search for credentials.
-func hasAWSCredentials() bool {
-	if os.Getenv("AWS_PROFILE") != "" ||
-		os.Getenv("AWS_ROLE_SESSION_NAME") != "" ||
-		(os.Getenv("AWS_ACCESS_KEY_ID") != "" && os.Getenv("AWS_SECRET_ACCESS_KEY") != "") {
-
-		return true
-	}
-
-	credFile := os.Getenv("AWS_SHARED_CREDENTIALS_FILE")
-	if credFile == "" {
-		if home, err := os.UserHomeDir(); err == nil {
-			credFile = filepath.Join(home, ".aws", "credentials")
-		}
-	}
-	if credFile != "" {
-		if _, err := os.Stat(credFile); err == nil {
-			return true
-		}
-	}
-	return false
-}
-
-func NewPluginRegistry(db *fsdb.Db) (ret *PluginRegistry, err error) {
-	ret = &PluginRegistry{
-		Db:             db,
-		VendorManager:  ai.NewVendorsManager(),
-		VendorsAll:     ai.NewVendorsManager(),
-		PatternsLoader: tools.NewPatternsLoader(db.Patterns),
-		YouTube:        youtube.NewYouTube(),
-		Language:       lang.NewLanguage(),
-		Jina:           jina.NewClient(),
-		Strategies:     strategy.NewStrategiesManager(),
-	}
-
-	var homedir string
-	if homedir, err = os.UserHomeDir(); err != nil {
-		return
-	}
-	ret.TemplateExtensions = template.NewExtensionManager(filepath.Join(homedir, ".config/fabric"))
-
-	ret.Defaults = tools.NeeDefaults(ret.GetModels)
-
-	// Create a vendors slice to hold all vendors (order doesn't matter initially)
-	vendors := []ai.Vendor{}
-
-	// Add non-OpenAI compatible clients
-	vendors = append(vendors,
-		openai.NewClient(),
-		ollama.NewClient(),
-		azure.NewClient(),
-		gemini.NewClient(),
-		anthropic.NewClient(),
-		lmstudio.NewClient(),
-		exolab.NewClient(),
-		perplexity.NewClient(), // Added Perplexity client
-	)
-
-	if hasAWSCredentials() {
-		vendors = append(vendors, bedrock.NewClient())
-	}
-
-	// Add all OpenAI-compatible providers
-	for providerName := range openai_compatible.ProviderMap {
-		provider, _ := openai_compatible.GetProviderByName(providerName)
-		vendors = append(vendors, openai_compatible.NewClient(provider))
-	}
-
-	// Sort vendors by name for consistent ordering (case-insensitive)
-	sort.Slice(vendors, func(i, j int) bool {
-		return strings.ToLower(vendors[i].GetName()) < strings.ToLower(vendors[j].GetName())
-	})
-
-	// Add all sorted vendors to VendorsAll
-	ret.VendorsAll.AddVendors(vendors...)
-	_ = ret.Configure()
-
-	return
-}
-
-func (o *PluginRegistry) ListVendors(out io.Writer) error {
-	vendors := lo.Map(o.VendorsAll.Vendors, func(vendor ai.Vendor, _ int) string {
-		return vendor.GetName()
-	})
-	fmt.Fprint(out, "Available Vendors:\n\n")
-	for _, vendor := range vendors {
-		fmt.Fprintf(out, "%s\n", vendor)
-	}
-	return nil
-}
-
-type PluginRegistry struct {
-	Db *fsdb.Db
-
-	VendorManager      *ai.VendorsManager
-	VendorsAll         *ai.VendorsManager
-	Defaults           *tools.Defaults
-	PatternsLoader     *tools.PatternsLoader
-	YouTube            *youtube.YouTube
-	Language           *lang.Language
-	Jina               *jina.Client
-	TemplateExtensions *template.ExtensionManager
-	Strategies         *strategy.StrategiesManager
-}
-
-func (o *PluginRegistry) SaveEnvFile() (err error) {
-	// Now create the .env with all configured VendorsController info
-	var envFileContent bytes.Buffer
-
-	o.Defaults.Settings.FillEnvFileContent(&envFileContent)
-	o.PatternsLoader.SetupFillEnvFileContent(&envFileContent)
-	o.Strategies.SetupFillEnvFileContent(&envFileContent)
-
-	for _, vendor := range o.VendorManager.Vendors {
-		vendor.SetupFillEnvFileContent(&envFileContent)
-	}
-
-	o.YouTube.SetupFillEnvFileContent(&envFileContent)
-	o.Jina.SetupFillEnvFileContent(&envFileContent)
-	o.Language.SetupFillEnvFileContent(&envFileContent)
-
-	err = o.Db.SaveEnv(envFileContent.String())
-	return
-}
-
-func (o *PluginRegistry) Setup() (err error) {
-	setupQuestion := plugins.NewSetupQuestion("Enter the number of the plugin to setup")
-	groupsPlugins := common.NewGroupsItemsSelector("Available plugins (please configure all required plugins):",
-		func(plugin plugins.Plugin) string {
-			var configuredLabel string
-			if plugin.IsConfigured() {
-				configuredLabel = " (configured)"
-			} else {
-				configuredLabel = ""
-			}
-			return fmt.Sprintf("%v%v", plugin.GetSetupDescription(), configuredLabel)
-		})
-
-	groupsPlugins.AddGroupItems("AI Vendors [at least one, required]", lo.Map(o.VendorsAll.Vendors,
-		func(vendor ai.Vendor, _ int) plugins.Plugin {
-			return vendor
-		})...)
-
-	groupsPlugins.AddGroupItems("Tools", o.Defaults, o.Jina, o.Language, o.PatternsLoader, o.Strategies, o.YouTube)
-
-	for {
-		groupsPlugins.Print(false)
-
-		if answerErr := setupQuestion.Ask("Plugin Number"); answerErr != nil {
-			break
-		}
-
-		if setupQuestion.Value == "" {
-			break
-		}
-		number, parseErr := strconv.Atoi(setupQuestion.Value)
-		setupQuestion.Value = ""
-
-		if parseErr == nil {
-			var plugin plugins.Plugin
-			if _, plugin, err = groupsPlugins.GetGroupAndItemByItemNumber(number); err != nil {
-				return
-			}
-
-			if pluginSetupErr := plugin.Setup(); pluginSetupErr != nil {
-				println(pluginSetupErr.Error())
-			} else {
-				if err = o.SaveEnvFile(); err != nil {
-					break
-				}
-			}
-
-			if _, ok := o.VendorManager.VendorsByName[plugin.GetName()]; !ok {
-				var vendor ai.Vendor
-				if vendor, ok = plugin.(ai.Vendor); ok {
-					o.VendorManager.AddVendors(vendor)
-				}
-			}
-		} else {
-			break
-		}
-	}
-
-	err = o.SaveEnvFile()
-
-	return
-}
-
-func (o *PluginRegistry) SetupVendor(vendorName string) (err error) {
-	if err = o.VendorsAll.SetupVendor(vendorName, o.VendorManager.VendorsByName); err != nil {
-		return
-	}
-	err = o.SaveEnvFile()
-	return
-}
-
-func (o *PluginRegistry) ConfigureVendors() {
-	o.VendorManager.Clear()
-	for _, vendor := range o.VendorsAll.Vendors {
-		if vendorErr := vendor.Configure(); vendorErr == nil {
-			o.VendorManager.AddVendors(vendor)
-		}
-	}
-}
-
-func (o *PluginRegistry) GetModels() (ret *ai.VendorsModels, err error) {
-	o.ConfigureVendors()
-	ret, err = o.VendorManager.GetModels()
-	return
-}
-
-// Configure buildClient VendorsController based on the environment variables
-func (o *PluginRegistry) Configure() (err error) {
-	o.ConfigureVendors()
-	_ = o.Defaults.Configure()
-	_ = o.PatternsLoader.Configure()
-
-	//YouTube and Jina are not mandatory, so ignore not configured error
-	_ = o.YouTube.Configure()
-	_ = o.Jina.Configure()
-	_ = o.Language.Configure()
-	return
-}
-
-func (o *PluginRegistry) GetChatter(model string, modelContextLength int, strategy string, stream bool, dryRun bool) (ret *Chatter, err error) {
-	ret = &Chatter{
-		db:     o.Db,
-		Stream: stream,
-		DryRun: dryRun,
-	}
-
-	defaultModel := o.Defaults.Model.Value
-	defaultModelContextLength, err := strconv.Atoi(o.Defaults.ModelContextLength.Value)
-	defaultVendor := o.Defaults.Vendor.Value
-	vendorManager := o.VendorManager
-
-	if err != nil {
-		defaultModelContextLength = 0
-		err = nil
-	}
-
-	ret.modelContextLength = modelContextLength
-	if ret.modelContextLength == 0 {
-		ret.modelContextLength = defaultModelContextLength
-	}
-
-	if dryRun {
-		ret.vendor = dryrun.NewClient()
-		ret.model = model
-		if ret.model == "" {
-			ret.model = defaultModel
-		}
-	} else if model == "" {
-		ret.vendor = vendorManager.FindByName(defaultVendor)
-		ret.model = defaultModel
-	} else {
-		var models *ai.VendorsModels
-		if models, err = vendorManager.GetModels(); err != nil {
-			return
-		}
-		ret.vendor = vendorManager.FindByName(models.FindGroupsByItemFirst(model))
-		ret.model = model
-	}
-
-	if ret.vendor == nil {
-		var errMsg string
-		if defaultModel == "" || defaultVendor == "" {
-			errMsg = "Please run, fabric --setup, and select default model and vendor."
-		} else {
-			errMsg = "could not find vendor."
-		}
-		err = fmt.Errorf(
-			" Requested Model = %s\n Default Model = %s\n Default Vendor = %s.\n\n%s",
-			model, defaultModel, defaultVendor, errMsg)
-		return
-	}
-	ret.strategy = strategy
-	return
-}

core/plugin_registry_test.go

@@ -1,21 +0,0 @@
-package core
-
-import (
-	"os"
-	"testing"
-
-	"github.com/danielmiessler/fabric/plugins/db/fsdb"
-)
-
-func TestSaveEnvFile(t *testing.T) {
-	db := fsdb.NewDb(os.TempDir())
-	registry, err := NewPluginRegistry(db)
-	if err != nil {
-		t.Fatalf("NewPluginRegistry() error = %v", err)
-	}
-
-	err = registry.SaveEnvFile()
-	if err != nil {
-		t.Fatalf("SaveEnvFile() error = %v", err)
-	}
-}

data/patterns/analyze_debate/system.md

@@ -22,19 +22,20 @@ Take a deep breath and think step by step about how to best accomplish this goal
     This must be under the heading "INSIGHTFULNESS SCORE (0 = not very interesting and insightful to 10 = very interesting and insightful)".
     - A rating of how emotional the debate was from 0 (very calm) to 5 (very emotional). This must be under the heading "EMOTIONALITY SCORE (0 (very calm) to 5 (very emotional))".
     - A list of the participants of the debate and a score of their emotionality from 0 (very calm) to 5 (very emotional). This must be under the heading "PARTICIPANTS".
-    - A list of arguments attributed to participants with names and quotes. If possible, this should include external references that disprove or back up their claims. 
+    - A list of arguments attributed to participants with names and quotes. Each argument summary must be EXACTLY 16 words. If possible, this should include external references that disprove or back up their claims. 
     It is IMPORTANT that these references are from trusted and verifiable sources that can be easily accessed. These sources have to BE REAL and NOT MADE UP. This must be under the heading "ARGUMENTS". 
     If possible, provide an objective assessment of the truth of these arguments. If you assess the truth of the argument, provide some sources that back up your assessment. The material you provide should be from reliable, verifiable, and trustworthy sources. DO NOT MAKE UP SOURCES.
-    - A list of agreements the participants have reached, attributed with names and quotes. This must be under the heading "AGREEMENTS".
-    - A list of disagreements the participants were unable to resolve and the reasons why they remained unresolved, attributed with names and quotes. This must be under the heading "DISAGREEMENTS".
-    - A list of possible misunderstandings and why they may have occurred, attributed with names and quotes. This must be under the heading "POSSIBLE MISUNDERSTANDINGS".
-    - A list of learnings from the debate. This must be under the heading "LEARNINGS".
-    - A list of takeaways that highlight ideas to think about, sources to explore, and actionable items. This must be under the heading "TAKEAWAYS".
+    - A list of agreements the participants have reached. Each agreement summary must be EXACTLY 16 words, followed by names and quotes. This must be under the heading "AGREEMENTS".
+    - A list of disagreements the participants were unable to resolve. Each disagreement summary must be EXACTLY 16 words, followed by names and quotes explaining why they remained unresolved. This must be under the heading "DISAGREEMENTS".
+    - A list of possible misunderstandings. Each misunderstanding summary must be EXACTLY 16 words, followed by names and quotes explaining why they may have occurred. This must be under the heading "POSSIBLE MISUNDERSTANDINGS".
+    - A list of learnings from the debate. Each learning must be EXACTLY 16 words. This must be under the heading "LEARNINGS".
+    - A list of takeaways that highlight ideas to think about, sources to explore, and actionable items. Each takeaway must be EXACTLY 16 words. This must be under the heading "TAKEAWAYS".
 
 # OUTPUT INSTRUCTIONS
 
 - Output all sections above.
-- Use Markdown to structure your output.
+- Do not use any markdown formatting (no asterisks, no bullet points, no headers).
+- Keep all agreements, arguments, recommendations, learnings, and takeaways to EXACTLY 16 words each.
 - When providing quotes, these quotes should clearly express the points you are using them for. If necessary, use multiple quotes.
 
 # INPUT: