Compare commits

...

31 Commits

Author SHA1 Message Date
github-actions[bot]
acf1be71ce Update version to v1.4.230 and commit 2025-07-05 07:08:05 +00:00
Kayvan Sylvan
236a3c5f38 Merge pull request #1575 from ksylvan/0704-advanced-image-output-options
Advanced image generation parameters for OpenAI models
2025-07-05 00:06:38 -07:00
Kayvan Sylvan
b2418984f8 feat: add advanced image generation parameters for OpenAI models
## CHANGES

- Add four new image generation CLI flags
- Implement validation for image parameter combinations
- Support size, quality, compression, and background controls
- Add comprehensive test coverage for new parameters
- Update shell completions for new image options
- Enhance README with detailed image generation examples
- Fix PowerShell code block formatting issues
2025-07-04 23:04:50 -07:00
github-actions[bot]
152d74d160 Update version to v1.4.229 and commit 2025-07-05 01:59:22 +00:00
Kayvan Sylvan
4e16bbccd8 Merge pull request #1574 from ksylvan/0704-image-tool-model-validation
Add Model Validation for Image Generation and Fix CLI Flag Mapping
2025-07-04 18:57:51 -07:00
Kayvan Sylvan
60174f41a4 refactor: extract supported models list to shared constant for image generation validation
## CHANGES

• Extract hardcoded model lists into shared constant
• Create ImageGenerationSupportedModels variable for reusability
• Update supportsImageGeneration function to use shared constant
• Refactor error messages to reference centralized model list
• Add documentation comment for supported models variable
• Import strings package in test file
• Consolidate duplicate model validation logic across files
2025-07-04 18:52:49 -07:00
Kayvan Sylvan
ad4683952e Merge branch 'main' into 0704-image-tool-model-validation 2025-07-04 18:45:08 -07:00
Kayvan Sylvan
86a044735b feat: add model validation for image generation support
### CHANGES

- Add model field to `BuildChatOptions` method
- Implement `supportsImageGeneration` function for model checks
- Validate model supports image generation in `sendResponses`
- Remove `mars-colony.png` from repository
- Add tests for `supportsImageGeneration` function
- Validate image file support in `TestModelValidationLogic`
2025-07-04 18:40:20 -07:00
github-actions[bot]
58583114cb Update version to v1.4.228 and commit 2025-07-05 01:13:09 +00:00
Kayvan Sylvan
36524cd2e4 Merge pull request #1573 from ksylvan/0704-image-dynamic-formats
Add Image File Validation and Dynamic Format Support
2025-07-04 18:11:36 -07:00
Kayvan Sylvan
e59156ac2b feat: add image file validation and format detection for image generation
## CHANGES

• Add image file path validation with extension checking
• Implement dynamic output format detection from file extensions
• Update BuildChatOptions method to return error for validation
• Add comprehensive test coverage for image file validation
• Upgrade YAML library from v2 to v3
• Update shell completions to reflect supported image formats
• Add error handling for existing file conflicts
• Support PNG, JPEG, JPG, and WEBP image formats
2025-07-04 17:56:59 -07:00
Daniel Miessler
1eac026e92 Addded tutorial as a tag. 2025-07-04 16:42:19 -07:00
github-actions[bot]
17d863fd57 Update version to v1.4.227 and commit 2025-07-04 22:40:01 +00:00
Daniel Miessler 🛡️
7c9dbfd343 Merge pull request #1572 from ksylvan/0704-OpenAI-Image-Generation-Tool
Add Image Generation Support to Fabric
2025-07-04 15:38:27 -07:00
Kayvan Sylvan
d9260bdf26 chore: refactor image generation constants for clarity and reuse
### CHANGES

- Define `ImageGenerationResponseType` constant for response handling
- Define `ImageGenerationToolType` constant for tool type usage
- Update `addImageGenerationTool` to use defined constants
- Refactor `extractAndSaveImages` to use response type constant
2025-07-04 15:14:46 -07:00
Kayvan Sylvan
63a0cfeb1e feat: add web search and image file support to fabric CLI
## CHANGES

- Add web search tool for Anthropic and OpenAI models
- Add search location parameter for web search results
- Add image file output option with format support
- Update zsh completion with new search and image flags
- Update bash completion with new option handling logic
- Update fish completion with search and image descriptions
- Support PNG, JPG, JPEG, GIF, BMP image formats
2025-07-04 14:49:40 -07:00
Kayvan Sylvan
12fc6e2000 feat: add image generation support with OpenAI image generation model
## CHANGES

- Add `--image-file` flag for saving generated images
- Implement image generation tool integration with OpenAI
- Support image editing with attachment input files
- Add comprehensive test coverage for image features
- Update documentation with image generation examples
- Fix HTML formatting issues in README
- Improve PowerShell code block indentation
- Clean up help text formatting and spacing
2025-07-04 14:36:55 -07:00
Daniel Miessler
fe5900a5dc Fixed ul tag applier. 2025-07-04 14:25:54 -07:00
Daniel Miessler
1b6b8e3d72 Updated ul tag prompt. 2025-07-04 14:21:25 -07:00
Daniel Miessler
c85301cb1f Added the UL tags pattern. 2025-07-04 14:17:43 -07:00
github-actions[bot]
7cc8226339 Update version to v1.4.226 and commit 2025-07-04 07:25:58 +00:00
Kayvan Sylvan
fc8c4babf8 Merge pull request #1569 from ksylvan/0703-openai-web-search
OpenAI Plugin Now Supports Web Search Functionality
2025-07-04 00:24:32 -07:00
Kayvan Sylvan
bd809a1f94 docs: update README with new web search feature details 2025-07-04 00:22:36 -07:00
Kayvan Sylvan
50aec6291b feat: add web search tool support for OpenAI models with citation formatting
## CHANGES

- Enable web search tool for OpenAI models
- Add location parameter support for search results
- Extract and format citations from search responses
- Implement citation deduplication to avoid duplicates
- Add comprehensive test coverage for search functionality
- Update CLI flag description to include OpenAI
- Format citations as markdown links with sources
2025-07-04 00:01:54 -07:00
github-actions[bot]
f927fdf40f Update version to v1.4.225 and commit 2025-07-04 06:58:02 +00:00
Kayvan Sylvan
918862ef57 Merge pull request #1568 from ksylvan/0703-enhanced-anthropic-search-tool
Runtime Web Search Control via Command-Line Flag
2025-07-03 23:56:34 -07:00
Kayvan Sylvan
d9b8bc3233 chore: refactor Send method to optimize string building
### CHANGES

- Add `sourcesHeader` constant for citation section title.
- Use `strings.Builder` to construct result efficiently.
- Append sources header and citations in result builder.
- Update `ret` to use constructed string from builder.
2025-07-03 23:52:12 -07:00
Kayvan Sylvan
da29b8e388 chore: remove unused web-search tool parameters for simplification
### CHANGES

- Remove unused `AllowedDomains` and `MaxUses` parameters
- Simplify `webTool` definition in `buildMessageParams` method
2025-07-03 23:41:22 -07:00
Kayvan Sylvan
5e6d4110fa refactor: extract web search tool constants in anthropic plugin
## CHANGES

- Add webSearchToolName constant for tool identification
- Add webSearchToolType constant for tool versioning
- Replace hardcoded string literals with named constants
- Improve code maintainability through constant extraction
2025-07-03 23:20:19 -07:00
Kayvan Sylvan
4bb090694b chore: update formatOptions to include search options display
### CHANGES

- Add search option status to `formatOptions`
- Include `SearchLocation` in formatted output if specified
2025-07-03 22:55:55 -07:00
Kayvan Sylvan
d232222787 feat: add web search tool support for Anthropic models
## CHANGES

- Add --search flag to enable web search
- Add --search-location for timezone-based results
- Pass search options through ChatOptions struct
- Implement web search tool in Anthropic client
- Format search citations with sources section
- Add comprehensive tests for search functionality
- Remove plugin-level web search configuration
2025-07-03 22:40:39 -07:00
22 changed files with 1537 additions and 49 deletions

View File

@@ -15,9 +15,7 @@ Fabric is graciously supported by…
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/danielmiessler/fabric)
<div align="center">
<p class="align center">
<h4><code>fabric</code> is an open-source framework for augmenting humans using AI.</h4>
</p>
</div>
[Updates](#updates) •
@@ -41,9 +39,9 @@ Since the start of modern AI in late 2022 we've seen an **_extraordinary_** numb
It's all really exciting and powerful, but _it's not easy to integrate this functionality into our lives._
<p class="align center">
<div class="align center">
<h4>In other words, AI doesn't have a capabilities problem—it has an <em>integration</em> problem.</h4>
</p>
</div>
**Fabric was created to address this by creating and organizing the fundamental units of AI—the prompts themselves!**
@@ -114,6 +112,14 @@ Keep in mind that many of these were recorded when Fabric was Python-based, so r
> [!NOTE]
>
> July 4, 2025
>
> - **Web Search**: Fabric now supports web search for Anthropic and OpenAI models using the `--search` and `--search-location` flags. This replaces the previous plugin-based search, so you may want to remove the old `ANTHROPIC_WEB_SEARCH_TOOL_*` variables from your `~/.config/fabric/.env` file.
> - **Image Generation**: Fabric now has powerful image generation capabilities with OpenAI.
> - Generate images from text prompts and save them using `--image-file`.
> - Edit existing images by providing an input image with `--attachment`.
> - Control image `size`, `quality`, `compression`, and `background` with the new `--image-*` flags.
>
>June 17, 2025
>
>- Fabric now supports Perplexity AI. Configure it by using `fabric -S` to add your Perplexity AI API Key,
@@ -485,7 +491,6 @@ fabric -h
```
```plaintext
Usage:
fabric [OPTIONS]
@@ -500,7 +505,9 @@ Application Options:
-T, --topp= Set top P (default: 0.9)
-s, --stream Stream
-P, --presencepenalty= Set presence penalty (default: 0.0)
-r, --raw Use the defaults of the model without sending chat options (like temperature etc.) and use the user role instead of the system role for patterns.
-r, --raw Use the defaults of the model without sending chat options (like
temperature etc.) and use the user role instead of the system role for
patterns.
-F, --frequencypenalty= Set frequency penalty (default: 0.0)
-l, --listpatterns List all patterns
-L, --listmodels List all available models
@@ -514,9 +521,12 @@ Application Options:
--output-session Output the entire session (also a temporary one) to the output file
-n, --latest= Number of latest patterns to list (default: 0)
-d, --changeDefaultModel Change default model
-y, --youtube= YouTube video or play list "URL" to grab transcript, comments from it and send to chat or print it put to the console and store it in the output file
-y, --youtube= YouTube video or play list "URL" to grab transcript, comments from it
and send to chat or print it put to the console and store it in the
output file
--playlist Prefer playlist over video if both ids are present in the URL
--transcript Grab transcript from YouTube video and send to chat (it is used per default).
--transcript Grab transcript from YouTube video and send to chat (it is used per
default).
--transcript-with-timestamps Grab transcript from YouTube video with timestamps and send to chat
--comments Grab comments from YouTube video and send to chat
--metadata Output video metadata
@@ -544,6 +554,14 @@ Application Options:
--liststrategies List all strategies
--listvendors List all vendors
--shell-complete-list Output raw list without headers/formatting (for shell completion)
--search Enable web search tool for supported models (Anthropic, OpenAI)
--search-location= Set location for web search results (e.g., 'America/Los_Angeles')
--image-file= Save generated image to specified file path (e.g., 'output.png')
--image-size= Image dimensions: 1024x1024, 1536x1024, 1024x1536, auto (default: auto)
--image-quality= Image quality: low, medium, high, auto (default: auto)
--image-compression= Compression level 0-100 for JPEG/WebP formats (default: not set)
--image-background= Background type: opaque, transparent (default: opaque, only for
PNG/WebP)
Help Options:
-h, --help Show this help message

View File

@@ -270,7 +270,11 @@ func Cli(version string) (err error) {
if chatReq.Language == "" {
chatReq.Language = registry.Language.DefaultLanguage.Value
}
if session, err = chatter.Send(chatReq, currentFlags.BuildChatOptions()); err != nil {
var chatOptions *common.ChatOptions
if chatOptions, err = currentFlags.BuildChatOptions(); err != nil {
return
}
if session, err = chatter.Send(chatReq, chatOptions); err != nil {
return
}

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"reflect"
"strconv"
"strings"
@@ -14,7 +15,7 @@ import (
"github.com/danielmiessler/fabric/common"
"github.com/jessevdk/go-flags"
"golang.org/x/text/language"
"gopkg.in/yaml.v2"
"gopkg.in/yaml.v3"
)
// Flags create flags struct. the users flags go into this, this will be passed to the chat struct in cli
@@ -74,6 +75,13 @@ type Flags struct {
ListStrategies bool `long:"liststrategies" description:"List all strategies"`
ListVendors bool `long:"listvendors" description:"List all vendors"`
ShellCompleteOutput bool `long:"shell-complete-list" description:"Output raw list without headers/formatting (for shell completion)"`
Search bool `long:"search" description:"Enable web search tool for supported models (Anthropic, OpenAI)"`
SearchLocation string `long:"search-location" description:"Set location for web search results (e.g., 'America/Los_Angeles')"`
ImageFile string `long:"image-file" description:"Save generated image to specified file path (e.g., 'output.png')"`
ImageSize string `long:"image-size" description:"Image dimensions: 1024x1024, 1536x1024, 1024x1536, auto (default: auto)"`
ImageQuality string `long:"image-quality" description:"Image quality: low, medium, high, auto (default: auto)"`
ImageCompression int `long:"image-compression" description:"Compression level 0-100 for JPEG/WebP formats (default: not set)"`
ImageBackground string `long:"image-background" description:"Background type: opaque, transparent (default: opaque, only for PNG/WebP)"`
}
var debug = false
@@ -254,8 +262,121 @@ func readStdin() (ret string, err error) {
return
}
func (o *Flags) BuildChatOptions() (ret *common.ChatOptions) {
// validateImageFile validates the image file path and extension
func validateImageFile(imagePath string) error {
if imagePath == "" {
return nil // No validation needed if no image file specified
}
// Check if file already exists
if _, err := os.Stat(imagePath); err == nil {
return fmt.Errorf("image file already exists: %s", imagePath)
}
// Check file extension
ext := strings.ToLower(filepath.Ext(imagePath))
validExtensions := []string{".png", ".jpeg", ".jpg", ".webp"}
for _, validExt := range validExtensions {
if ext == validExt {
return nil // Valid extension found
}
}
return fmt.Errorf("invalid image file extension '%s'. Supported formats: .png, .jpeg, .jpg, .webp", ext)
}
// validateImageParameters validates image generation parameters
func validateImageParameters(imagePath, size, quality, background string, compression int) error {
if imagePath == "" {
// Check if any image parameters are specified without --image-file
if size != "" || quality != "" || background != "" || compression != 0 {
return fmt.Errorf("image parameters (--image-size, --image-quality, --image-background, --image-compression) can only be used with --image-file")
}
return nil
}
// Validate size
if size != "" {
validSizes := []string{"1024x1024", "1536x1024", "1024x1536", "auto"}
valid := false
for _, validSize := range validSizes {
if size == validSize {
valid = true
break
}
}
if !valid {
return fmt.Errorf("invalid image size '%s'. Supported sizes: 1024x1024, 1536x1024, 1024x1536, auto", size)
}
}
// Validate quality
if quality != "" {
validQualities := []string{"low", "medium", "high", "auto"}
valid := false
for _, validQuality := range validQualities {
if quality == validQuality {
valid = true
break
}
}
if !valid {
return fmt.Errorf("invalid image quality '%s'. Supported qualities: low, medium, high, auto", quality)
}
}
// Validate background
if background != "" {
validBackgrounds := []string{"opaque", "transparent"}
valid := false
for _, validBackground := range validBackgrounds {
if background == validBackground {
valid = true
break
}
}
if !valid {
return fmt.Errorf("invalid image background '%s'. Supported backgrounds: opaque, transparent", background)
}
}
// Get file format for format-specific validations
ext := strings.ToLower(filepath.Ext(imagePath))
// Validate compression (only for jpeg/webp)
if compression != 0 { // 0 means not set
if ext != ".jpg" && ext != ".jpeg" && ext != ".webp" {
return fmt.Errorf("image compression can only be used with JPEG and WebP formats, not %s", ext)
}
if compression < 0 || compression > 100 {
return fmt.Errorf("image compression must be between 0 and 100, got %d", compression)
}
}
// Validate background transparency (only for png/webp)
if background == "transparent" {
if ext != ".png" && ext != ".webp" {
return fmt.Errorf("transparent background can only be used with PNG and WebP formats, not %s", ext)
}
}
return nil
}
func (o *Flags) BuildChatOptions() (ret *common.ChatOptions, err error) {
// Validate image file if specified
if err = validateImageFile(o.ImageFile); err != nil {
return nil, err
}
// Validate image parameters
if err = validateImageParameters(o.ImageFile, o.ImageSize, o.ImageQuality, o.ImageBackground, o.ImageCompression); err != nil {
return nil, err
}
ret = &common.ChatOptions{
Model: o.Model,
Temperature: o.Temperature,
TopP: o.TopP,
PresencePenalty: o.PresencePenalty,
@@ -263,6 +384,13 @@ func (o *Flags) BuildChatOptions() (ret *common.ChatOptions) {
Raw: o.Raw,
Seed: o.Seed,
ModelContextLength: o.ModelContextLength,
Search: o.Search,
SearchLocation: o.SearchLocation,
ImageFile: o.ImageFile,
ImageSize: o.ImageSize,
ImageQuality: o.ImageQuality,
ImageCompression: o.ImageCompression,
ImageBackground: o.ImageBackground,
}
return
}

View File

@@ -4,6 +4,7 @@ import (
"bytes"
"io"
"os"
"path/filepath"
"strings"
"testing"
@@ -64,7 +65,8 @@ func TestBuildChatOptions(t *testing.T) {
Raw: false,
Seed: 1,
}
options := flags.BuildChatOptions()
options, err := flags.BuildChatOptions()
assert.NoError(t, err)
assert.Equal(t, expectedOptions, options)
}
@@ -84,7 +86,8 @@ func TestBuildChatOptionsDefaultSeed(t *testing.T) {
Raw: false,
Seed: 0,
}
options := flags.BuildChatOptions()
options, err := flags.BuildChatOptions()
assert.NoError(t, err)
assert.Equal(t, expectedOptions, options)
}
@@ -164,3 +167,269 @@ model: 123 # should be string
assert.Error(t, err)
})
}
func TestValidateImageFile(t *testing.T) {
t.Run("Empty path should be valid", func(t *testing.T) {
err := validateImageFile("")
assert.NoError(t, err)
})
t.Run("Valid extensions should pass", func(t *testing.T) {
validExtensions := []string{".png", ".jpeg", ".jpg", ".webp"}
for _, ext := range validExtensions {
filename := "/tmp/test" + ext
err := validateImageFile(filename)
assert.NoError(t, err, "Extension %s should be valid", ext)
}
})
t.Run("Invalid extensions should fail", func(t *testing.T) {
invalidExtensions := []string{".gif", ".bmp", ".tiff", ".svg", ".txt", ""}
for _, ext := range invalidExtensions {
filename := "/tmp/test" + ext
err := validateImageFile(filename)
assert.Error(t, err, "Extension %s should be invalid", ext)
assert.Contains(t, err.Error(), "invalid image file extension")
}
})
t.Run("Existing file should fail", func(t *testing.T) {
// Create a temporary file
tempFile, err := os.CreateTemp("", "test*.png")
assert.NoError(t, err)
defer os.Remove(tempFile.Name())
tempFile.Close()
// Validation should fail because file exists
err = validateImageFile(tempFile.Name())
assert.Error(t, err)
assert.Contains(t, err.Error(), "image file already exists")
})
t.Run("Non-existing file with valid extension should pass", func(t *testing.T) {
nonExistentFile := filepath.Join(os.TempDir(), "non_existent_file.png")
// Make sure the file doesn't exist
os.Remove(nonExistentFile)
err := validateImageFile(nonExistentFile)
assert.NoError(t, err)
})
}
func TestBuildChatOptionsWithImageFileValidation(t *testing.T) {
t.Run("Valid image file should pass", func(t *testing.T) {
flags := &Flags{
ImageFile: "/tmp/output.png",
}
options, err := flags.BuildChatOptions()
assert.NoError(t, err)
assert.Equal(t, "/tmp/output.png", options.ImageFile)
})
t.Run("Invalid extension should fail", func(t *testing.T) {
flags := &Flags{
ImageFile: "/tmp/output.gif",
}
options, err := flags.BuildChatOptions()
assert.Error(t, err)
assert.Nil(t, options)
assert.Contains(t, err.Error(), "invalid image file extension")
})
t.Run("Existing file should fail", func(t *testing.T) {
// Create a temporary file
tempFile, err := os.CreateTemp("", "existing*.png")
assert.NoError(t, err)
defer os.Remove(tempFile.Name())
tempFile.Close()
flags := &Flags{
ImageFile: tempFile.Name(),
}
options, err := flags.BuildChatOptions()
assert.Error(t, err)
assert.Nil(t, options)
assert.Contains(t, err.Error(), "image file already exists")
})
}
func TestValidateImageParameters(t *testing.T) {
t.Run("No image file and no parameters should pass", func(t *testing.T) {
err := validateImageParameters("", "", "", "", 0)
assert.NoError(t, err)
})
t.Run("Image parameters without image file should fail", func(t *testing.T) {
// Test each parameter individually
err := validateImageParameters("", "1024x1024", "", "", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image parameters")
assert.Contains(t, err.Error(), "can only be used with --image-file")
err = validateImageParameters("", "", "high", "", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image parameters")
err = validateImageParameters("", "", "", "transparent", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image parameters")
err = validateImageParameters("", "", "", "", 50)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image parameters")
// Test multiple parameters
err = validateImageParameters("", "1024x1024", "high", "transparent", 50)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image parameters")
})
t.Run("Valid size values should pass", func(t *testing.T) {
validSizes := []string{"1024x1024", "1536x1024", "1024x1536", "auto"}
for _, size := range validSizes {
err := validateImageParameters("/tmp/test.png", size, "", "", 0)
assert.NoError(t, err, "Size %s should be valid", size)
}
})
t.Run("Invalid size should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.png", "invalid", "", "", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid image size")
})
t.Run("Valid quality values should pass", func(t *testing.T) {
validQualities := []string{"low", "medium", "high", "auto"}
for _, quality := range validQualities {
err := validateImageParameters("/tmp/test.png", "", quality, "", 0)
assert.NoError(t, err, "Quality %s should be valid", quality)
}
})
t.Run("Invalid quality should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.png", "", "invalid", "", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid image quality")
})
t.Run("Valid background values should pass", func(t *testing.T) {
validBackgrounds := []string{"opaque", "transparent"}
for _, background := range validBackgrounds {
err := validateImageParameters("/tmp/test.png", "", "", background, 0)
assert.NoError(t, err, "Background %s should be valid", background)
}
})
t.Run("Invalid background should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.png", "", "", "invalid", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "invalid image background")
})
t.Run("Compression for JPEG should pass", func(t *testing.T) {
err := validateImageParameters("/tmp/test.jpg", "", "", "", 75)
assert.NoError(t, err)
})
t.Run("Compression for WebP should pass", func(t *testing.T) {
err := validateImageParameters("/tmp/test.webp", "", "", "", 50)
assert.NoError(t, err)
})
t.Run("Compression for PNG should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.png", "", "", "", 75)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image compression can only be used with JPEG and WebP formats")
})
t.Run("Invalid compression range should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.jpg", "", "", "", 150)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image compression must be between 0 and 100")
err = validateImageParameters("/tmp/test.jpg", "", "", "", -10)
assert.Error(t, err)
assert.Contains(t, err.Error(), "image compression must be between 0 and 100")
})
t.Run("Transparent background for PNG should pass", func(t *testing.T) {
err := validateImageParameters("/tmp/test.png", "", "", "transparent", 0)
assert.NoError(t, err)
})
t.Run("Transparent background for WebP should pass", func(t *testing.T) {
err := validateImageParameters("/tmp/test.webp", "", "", "transparent", 0)
assert.NoError(t, err)
})
t.Run("Transparent background for JPEG should fail", func(t *testing.T) {
err := validateImageParameters("/tmp/test.jpg", "", "", "transparent", 0)
assert.Error(t, err)
assert.Contains(t, err.Error(), "transparent background can only be used with PNG and WebP formats")
})
}
func TestBuildChatOptionsWithImageParameters(t *testing.T) {
t.Run("Valid image parameters should pass", func(t *testing.T) {
flags := &Flags{
ImageFile: "/tmp/test.png",
ImageSize: "1024x1024",
ImageQuality: "high",
ImageBackground: "transparent",
ImageCompression: 0, // Not set for PNG
}
options, err := flags.BuildChatOptions()
assert.NoError(t, err)
assert.NotNil(t, options)
assert.Equal(t, "/tmp/test.png", options.ImageFile)
assert.Equal(t, "1024x1024", options.ImageSize)
assert.Equal(t, "high", options.ImageQuality)
assert.Equal(t, "transparent", options.ImageBackground)
assert.Equal(t, 0, options.ImageCompression)
})
t.Run("Invalid image parameters should fail", func(t *testing.T) {
flags := &Flags{
ImageFile: "/tmp/test.png",
ImageSize: "invalid",
ImageQuality: "high",
ImageBackground: "transparent",
}
options, err := flags.BuildChatOptions()
assert.Error(t, err)
assert.Nil(t, options)
assert.Contains(t, err.Error(), "invalid image size")
})
t.Run("JPEG with compression should pass", func(t *testing.T) {
flags := &Flags{
ImageFile: "/tmp/test.jpg",
ImageSize: "1536x1024",
ImageQuality: "medium",
ImageBackground: "opaque",
ImageCompression: 80,
}
options, err := flags.BuildChatOptions()
assert.NoError(t, err)
assert.NotNil(t, options)
assert.Equal(t, 80, options.ImageCompression)
})
t.Run("Image parameters without image file should fail in BuildChatOptions", func(t *testing.T) {
flags := &Flags{
ImageSize: "1024x1024", // Image parameter without ImageFile
}
options, err := flags.BuildChatOptions()
assert.Error(t, err)
assert.Nil(t, options)
assert.Contains(t, err.Error(), "image parameters")
assert.Contains(t, err.Error(), "can only be used with --image-file")
})
}

View File

@@ -26,6 +26,13 @@ type ChatOptions struct {
Seed int
ModelContextLength int
MaxTokens int
Search bool
SearchLocation string
ImageFile string
ImageSize string
ImageQuality string
ImageCompression int
ImageBackground string
}
// NormalizeMessages remove empty messages and ensure messages order user-assist-user

View File

@@ -96,6 +96,13 @@ _fabric() {
'(--api-key)--api-key[API key used to secure server routes]:api-key:' \
'(--config)--config[Path to YAML config file]:config file:_files -g "*.yaml *.yml"' \
'(--version)--version[Print current version]' \
'(--search)--search[Enable web search tool for supported models (Anthropic, OpenAI)]' \
'(--search-location)--search-location[Set location for web search results]:location:' \
'(--image-file)--image-file[Save generated image to specified file path]:image file:_files -g "*.png *.webp *.jpeg *.jpg"' \
'(--image-size)--image-size[Image dimensions]:size:(1024x1024 1536x1024 1024x1536 auto)' \
'(--image-quality)--image-quality[Image quality]:quality:(low medium high auto)' \
'(--image-compression)--image-compression[Compression level 0-100 for JPEG/WebP formats]:compression:' \
'(--image-background)--image-background[Background type]:background:(opaque transparent)' \
'(--listextensions)--listextensions[List all registered extensions]' \
'(--addextension)--addextension[Register a new extension from config file path]:config file:_files -g "*.yaml *.yml"' \
'(--rmextension)--rmextension[Remove a registered extension by name]:extension:_fabric_extensions' \

View File

@@ -13,7 +13,7 @@ _fabric() {
_get_comp_words_by_ref -n : cur prev words cword
# Define all possible options/flags
local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --language -g --scrape_url -u --scrape_question -q --seed -e --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --dry-run --serve --serveOllama --address --api-key --config --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
local opts="--pattern -p --variable -v --context -C --session --attachment -a --setup -S --temperature -t --topp -T --stream -s --presencepenalty -P --raw -r --frequencypenalty -F --listpatterns -l --listmodels -L --listcontexts -x --listsessions -X --updatepatterns -U --copy -c --model -m --modelContextLength --output -o --output-session --latest -n --changeDefaultModel -d --youtube -y --playlist --transcript --transcript-with-timestamps --comments --metadata --language -g --scrape_url -u --scrape_question -q --seed -e --wipecontext -w --wipesession -W --printcontext --printsession --readability --input-has-vars --dry-run --serve --serveOllama --address --api-key --config --search --search-location --image-file --image-size --image-quality --image-compression --image-background --version --listextensions --addextension --rmextension --strategy --liststrategies --listvendors --shell-complete-list --help -h"
# Helper function for dynamic completions
_fabric_get_list() {
@@ -63,12 +63,25 @@ _fabric() {
return 0
;;
# Options requiring file/directory paths
-a | --attachment | -o | --output | --config | --addextension)
-a | --attachment | -o | --output | --config | --addextension | --image-file)
_filedir
return 0
;;
# Image generation options with specific values
--image-size)
COMPREPLY=($(compgen -W "1024x1024 1536x1024 1024x1536 auto" -- "$cur"))
return 0
;;
--image-quality)
COMPREPLY=($(compgen -W "low medium high auto" -- "$cur"))
return 0
;;
--image-background)
COMPREPLY=($(compgen -W "opaque transparent" -- "$cur"))
return 0
;;
# Options requiring simple arguments (no specific completion logic here)
-v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key)
-v | --variable | -t | --temperature | -T | --topp | -P | --presencepenalty | -F | --frequencypenalty | --modelContextLength | -n | --latest | -y | --youtube | -g | --language | -u | --scrape_url | -q | --scrape_question | -e | --seed | --address | --api-key | --search-location | --image-compression)
# No specific completion suggestions, user types the value
return 0
;;

View File

@@ -60,6 +60,12 @@ complete -c fabric -l printsession -d "Print session" -a "(__fabric_get_sessions
complete -c fabric -l address -d "The address to bind the REST API (default: :8080)"
complete -c fabric -l api-key -d "API key used to secure server routes"
complete -c fabric -l config -d "Path to YAML config file" -r -a "*.yaml *.yml"
complete -c fabric -l search-location -d "Set location for web search results (e.g., 'America/Los_Angeles')"
complete -c fabric -l image-file -d "Save generated image to specified file path (e.g., 'output.png')" -r -a "*.png *.webp *.jpeg *.jpg"
complete -c fabric -l image-size -d "Image dimensions: 1024x1024, 1536x1024, 1024x1536, auto (default: auto)" -a "1024x1024 1536x1024 1024x1536 auto"
complete -c fabric -l image-quality -d "Image quality: low, medium, high, auto (default: auto)" -a "low medium high auto"
complete -c fabric -l image-compression -d "Compression level 0-100 for JPEG/WebP formats (default: not set)" -r
complete -c fabric -l image-background -d "Background type: opaque, transparent (default: opaque, only for PNG/WebP)" -a "opaque transparent"
complete -c fabric -l addextension -d "Register a new extension from config file path" -r -a "*.yaml *.yml"
complete -c fabric -l rmextension -d "Remove a registered extension by name" -a "(__fabric_get_extensions)"
complete -c fabric -l strategy -d "Choose a strategy from the available strategies" -a "(__fabric_get_strategies)"
@@ -84,6 +90,7 @@ complete -c fabric -l metadata -d "Output video metadata"
complete -c fabric -l readability -d "Convert HTML input into a clean, readable view"
complete -c fabric -l input-has-vars -d "Apply variables to user input"
complete -c fabric -l dry-run -d "Show what would be sent to the model without actually sending it"
complete -c fabric -l search -d "Enable web search tool for supported models (Anthropic, OpenAI)"
complete -c fabric -l serve -d "Serve the Fabric Rest API"
complete -c fabric -l serveOllama -d "Serve the Fabric Rest API with ollama endpoints"
complete -c fabric -l version -d "Print current version"

1
go.mod
View File

@@ -27,7 +27,6 @@ require (
github.com/stretchr/testify v1.10.0
golang.org/x/text v0.26.0
google.golang.org/api v0.236.0
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
)

1
go.sum
View File

@@ -354,7 +354,6 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EV
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

View File

@@ -325,9 +325,6 @@ schema = 3
[mod."gopkg.in/warnings.v0"]
version = "v0.1.2"
hash = "sha256-ATVL9yEmgYbkJ1DkltDGRn/auGAjqGOfjQyBYyUo8s8="
[mod."gopkg.in/yaml.v2"]
version = "v2.4.0"
hash = "sha256-uVEGglIedjOIGZzHW4YwN1VoRSTK8o0eGZqzd+TNdd0="
[mod."gopkg.in/yaml.v3"]
version = "v3.0.1"
hash = "sha256-FqL9TKYJ0XkNwJFnq9j0VvJ5ZUU1RvH/52h/f5bkYAU="

View File

@@ -1 +1 @@
"1.4.224"
"1.4.230"

View File

@@ -0,0 +1,49 @@
# IDENTITY
You are a superintelligent expert on content of all forms, with deep understanding of which topics, categories, themes, and tags apply to any piece of content.
# GOAL
Your goal is to output a JSON object called tags, with the following tags applied if the content is significantly about their topic.
- **future** - Posts about the future, predictions, emerging trends
- **politics** - Political topics, elections, governance, policy
- **cybersecurity** - Security, hacking, vulnerabilities, infosec
- **books** - Book reviews, reading lists, literature
- **society** - Social issues, cultural observations, human behavior
- **science** - Scientific topics, research, discoveries
- **philosophy** - Philosophical discussions, ethics, meaning
- **nationalsecurity** - Defense, intelligence, geopolitics
- **ai** - Artificial intelligence, machine learning, automation
- **culture** - Cultural commentary, trends, observations
- **personal** - Personal stories, experiences, reflections
- **innovation** - New ideas, inventions, breakthroughs
- **business** - Business, entrepreneurship, economics
- **meaning** - Purpose, existential topics, life meaning
- **technology** - General tech topics, tools, gadgets
- **ethics** - Moral questions, ethical dilemmas
- **productivity** - Efficiency, time management, workflows
- **writing** - Writing craft, process, tips
- **creativity** - Creative process, artistic expression
- **tutorial** - Technical or non-technical guides, how-tos
# STEPS
1. Deeply understand the content and its themes and categories and topics.
2. Evaluate the list of tags above.
3. Determine which tags apply to the content.
4. Output the "tags" JSON object.
# NOTES
- It's ok, and quite normal, for multiple tags to apply—which is why this is tags and not categories
- All AI posts should have the technology tag, and that's ok. But not all technology posts are about AI, and therefore the AI tag needs to be evaluated separately. That goes for all potentially nested or conflicted tags.
- Be a bit conservative in applying tags. If a piece of content is only tangentially related to a tag, don't include it.
# OUTPUT INSTRUCTIONS
- Output ONLY the JSON object, and nothing else.
- That means DO NOT OUTPUT the ```json format indicator. ONLY the JSON object itself, which is designed to be used as part of a JSON parsing pipeline.

View File

@@ -5,8 +5,6 @@ import (
"fmt"
"strings"
"github.com/samber/lo"
"github.com/anthropics/anthropic-sdk-go"
"github.com/anthropics/anthropic-sdk-go/option"
"github.com/danielmiessler/fabric/chat"
@@ -16,6 +14,10 @@ import (
const defaultBaseUrl = "https://api.anthropic.com/"
const webSearchToolName = "web_search"
const webSearchToolType = "web_search_20250305"
const sourcesHeader = "## Sources"
func NewClient() (ret *Client) {
vendorName := "Anthropic"
ret = &Client{}
@@ -29,9 +31,6 @@ func NewClient() (ret *Client) {
ret.ApiBaseURL = ret.AddSetupQuestion("API Base URL", false)
ret.ApiBaseURL.Value = defaultBaseUrl
ret.ApiKey = ret.PluginBase.AddSetupQuestion("API key", true)
ret.UseWebTool = ret.AddSetupQuestionBool("Web Search Tool Enabled", false)
ret.WebToolLocation = ret.AddSetupQuestionCustom("Web Search Tool Location", false,
"Enter your approximate timezone location for web search (e.g., 'America/Los_Angeles', see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).")
ret.maxTokens = 4096
ret.defaultRequiredUserMessage = "Hi"
@@ -49,10 +48,8 @@ func NewClient() (ret *Client) {
type Client struct {
*plugins.PluginBase
ApiBaseURL *plugins.SetupQuestion
ApiKey *plugins.SetupQuestion
UseWebTool *plugins.SetupQuestion
WebToolLocation *plugins.SetupQuestion
ApiBaseURL *plugins.SetupQuestion
ApiKey *plugins.SetupQuestion
maxTokens int
defaultRequiredUserMessage string
@@ -127,20 +124,17 @@ func (an *Client) buildMessageParams(msgs []anthropic.MessageParam, opts *common
Messages: msgs,
}
if plugins.ParseBoolElseFalse(an.UseWebTool.Value) {
if opts.Search {
// Build the web-search tool definition:
webTool := anthropic.WebSearchTool20250305Param{
Name: "web_search", // string literal instead of constant
Type: "web_search_20250305", // string literal instead of constant
Name: webSearchToolName,
Type: webSearchToolType,
CacheControl: anthropic.NewCacheControlEphemeralParam(),
// Optional: restrict domains or max uses
// AllowedDomains: []string{"wikipedia.org", "openai.com"},
// MaxUses: anthropic.Opt[int64](5),
}
if an.WebToolLocation.Value != "" {
if opts.SearchLocation != "" {
webTool.UserLocation.Type = "approximate"
webTool.UserLocation.Timezone = anthropic.Opt(an.WebToolLocation.Value)
webTool.UserLocation.Timezone = anthropic.Opt(opts.SearchLocation)
}
// Wrap it in the union:
@@ -165,13 +159,42 @@ func (an *Client) Send(ctx context.Context, msgs []*chat.ChatCompletionMessage,
return
}
texts := lo.FilterMap(message.Content, func(block anthropic.ContentBlockUnion, _ int) (ret string, ok bool) {
if ok = block.Type == "text" && block.Text != ""; ok {
ret = block.Text
var textParts []string
var citations []string
citationMap := make(map[string]bool) // To avoid duplicate citations
for _, block := range message.Content {
if block.Type == "text" && block.Text != "" {
textParts = append(textParts, block.Text)
// Extract citations from this text block
for _, citation := range block.Citations {
if citation.Type == "web_search_result_location" {
citationKey := citation.URL + "|" + citation.Title
if !citationMap[citationKey] {
citationMap[citationKey] = true
citationText := fmt.Sprintf("- [%s](%s)", citation.Title, citation.URL)
if citation.CitedText != "" {
citationText += fmt.Sprintf(" - \"%s\"", citation.CitedText)
}
citations = append(citations, citationText)
}
}
}
}
return
})
ret = strings.Join(texts, "")
}
var resultBuilder strings.Builder
resultBuilder.WriteString(strings.Join(textParts, ""))
// Append citations if any were found
if len(citations) > 0 {
resultBuilder.WriteString("\n\n")
resultBuilder.WriteString(sourcesHeader)
resultBuilder.WriteString("\n\n")
resultBuilder.WriteString(strings.Join(citations, "\n"))
}
ret = resultBuilder.String()
return
}

View File

@@ -1,7 +1,11 @@
package anthropic
import (
"strings"
"testing"
"github.com/anthropics/anthropic-sdk-go"
"github.com/danielmiessler/fabric/common"
)
// Test generated using Keploy
@@ -63,3 +67,192 @@ func TestClient_ListModels_ReturnsCorrectModels(t *testing.T) {
}
}
}
func TestBuildMessageParams_WithoutSearch(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "claude-3-5-sonnet-latest",
Temperature: 0.7,
Search: false,
}
messages := []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("Hello")),
}
params := client.buildMessageParams(messages, opts)
if params.Tools != nil {
t.Error("Expected no tools when search is disabled, got tools")
}
if params.Model != anthropic.Model(opts.Model) {
t.Errorf("Expected model %s, got %s", opts.Model, params.Model)
}
if params.Temperature.Value != opts.Temperature {
t.Errorf("Expected temperature %f, got %f", opts.Temperature, params.Temperature.Value)
}
}
func TestBuildMessageParams_WithSearch(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "claude-3-5-sonnet-latest",
Temperature: 0.7,
Search: true,
}
messages := []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("What's the weather today?")),
}
params := client.buildMessageParams(messages, opts)
if params.Tools == nil {
t.Fatal("Expected tools when search is enabled, got nil")
}
if len(params.Tools) != 1 {
t.Errorf("Expected 1 tool, got %d", len(params.Tools))
}
webTool := params.Tools[0].OfWebSearchTool20250305
if webTool == nil {
t.Fatal("Expected web search tool, got nil")
}
if webTool.Name != "web_search" {
t.Errorf("Expected tool name 'web_search', got %s", webTool.Name)
}
if webTool.Type != "web_search_20250305" {
t.Errorf("Expected tool type 'web_search_20250305', got %s", webTool.Type)
}
}
func TestBuildMessageParams_WithSearchAndLocation(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "claude-3-5-sonnet-latest",
Temperature: 0.7,
Search: true,
SearchLocation: "America/Los_Angeles",
}
messages := []anthropic.MessageParam{
anthropic.NewUserMessage(anthropic.NewTextBlock("What's the weather in San Francisco?")),
}
params := client.buildMessageParams(messages, opts)
if params.Tools == nil {
t.Fatal("Expected tools when search is enabled, got nil")
}
webTool := params.Tools[0].OfWebSearchTool20250305
if webTool == nil {
t.Fatal("Expected web search tool, got nil")
}
if webTool.UserLocation.Type != "approximate" {
t.Errorf("Expected location type 'approximate', got %s", webTool.UserLocation.Type)
}
if webTool.UserLocation.Timezone.Value != opts.SearchLocation {
t.Errorf("Expected timezone %s, got %s", opts.SearchLocation, webTool.UserLocation.Timezone.Value)
}
}
func TestCitationFormatting(t *testing.T) {
// Test the citation formatting logic by creating a mock message with citations
message := &anthropic.Message{
Content: []anthropic.ContentBlockUnion{
{
Type: "text",
Text: "Based on recent research, artificial intelligence is advancing rapidly.",
Citations: []anthropic.TextCitationUnion{
{
Type: "web_search_result_location",
URL: "https://example.com/ai-research",
Title: "AI Research Advances 2025",
CitedText: "artificial intelligence is advancing rapidly",
},
{
Type: "web_search_result_location",
URL: "https://another-source.com/tech-news",
Title: "Technology News Today",
CitedText: "recent developments in AI",
},
},
},
{
Type: "text",
Text: " Machine learning models are becoming more sophisticated.",
Citations: []anthropic.TextCitationUnion{
{
Type: "web_search_result_location",
URL: "https://example.com/ai-research", // Duplicate URL should be deduplicated
Title: "AI Research Advances 2025",
CitedText: "machine learning models",
},
},
},
},
}
// Extract text and citations using the same logic as the Send method
var textParts []string
var citations []string
citationMap := make(map[string]bool)
for _, block := range message.Content {
if block.Type == "text" && block.Text != "" {
textParts = append(textParts, block.Text)
for _, citation := range block.Citations {
if citation.Type == "web_search_result_location" {
citationKey := citation.URL + "|" + citation.Title
if !citationMap[citationKey] {
citationMap[citationKey] = true
citationText := "- [" + citation.Title + "](" + citation.URL + ")"
if citation.CitedText != "" {
citationText += " - \"" + citation.CitedText + "\""
}
citations = append(citations, citationText)
}
}
}
}
}
result := strings.Join(textParts, "")
if len(citations) > 0 {
result += "\n\n## Sources\n\n" + strings.Join(citations, "\n")
}
// Verify the result contains the expected text
expectedText := "Based on recent research, artificial intelligence is advancing rapidly. Machine learning models are becoming more sophisticated."
if !strings.Contains(result, expectedText) {
t.Errorf("Expected result to contain text: %s", expectedText)
}
// Verify citations are included
if !strings.Contains(result, "## Sources") {
t.Error("Expected result to contain Sources section")
}
if !strings.Contains(result, "[AI Research Advances 2025](https://example.com/ai-research)") {
t.Error("Expected result to contain first citation")
}
if !strings.Contains(result, "[Technology News Today](https://another-source.com/tech-news)") {
t.Error("Expected result to contain second citation")
}
// Verify deduplication - should only have 2 unique citations, not 3
citationCount := strings.Count(result, "- [")
if citationCount != 2 {
t.Errorf("Expected 2 unique citations, got %d", citationCount)
}
}

View File

@@ -76,6 +76,15 @@ func (c *Client) formatOptions(opts *common.ChatOptions) string {
if opts.ModelContextLength != 0 {
builder.WriteString(fmt.Sprintf("ModelContextLength: %d\n", opts.ModelContextLength))
}
if opts.Search {
builder.WriteString("Search: enabled\n")
if opts.SearchLocation != "" {
builder.WriteString(fmt.Sprintf("SearchLocation: %s\n", opts.SearchLocation))
}
}
if opts.ImageFile != "" {
builder.WriteString(fmt.Sprintf("ImageFile: %s\n", opts.ImageFile))
}
return builder.String()
}

View File

@@ -2,6 +2,7 @@ package openai
import (
"context"
"fmt"
"slices"
"strings"
@@ -127,12 +128,23 @@ func (o *Client) Send(ctx context.Context, msgs []*chat.ChatCompletionMessage, o
}
func (o *Client) sendResponses(ctx context.Context, msgs []*chat.ChatCompletionMessage, opts *common.ChatOptions) (ret string, err error) {
// Validate model supports image generation if image file is specified
if opts.ImageFile != "" && !supportsImageGeneration(opts.Model) {
return "", fmt.Errorf("model '%s' does not support image generation. Supported models: %s", opts.Model, strings.Join(ImageGenerationSupportedModels, ", "))
}
req := o.buildResponseParams(msgs, opts)
var resp *responses.Response
if resp, err = o.ApiClient.Responses.New(ctx, req); err != nil {
return
}
// Extract and save images if requested
if err = o.extractAndSaveImages(resp, opts); err != nil {
return
}
ret = o.extractText(resp)
return
}
@@ -182,6 +194,31 @@ func (o *Client) buildResponseParams(
},
}
// Add tools if enabled
var tools []responses.ToolUnionParam
// Add web search tool if enabled
if opts.Search {
webSearchTool := responses.ToolParamOfWebSearchPreview("web_search_preview")
// Add user location if provided
if opts.SearchLocation != "" {
webSearchTool.OfWebSearchPreview.UserLocation = responses.WebSearchToolUserLocationParam{
Type: "approximate",
Timezone: openai.String(opts.SearchLocation),
}
}
tools = append(tools, webSearchTool)
}
// Add image generation tool if needed
tools = o.addImageGenerationTool(opts, tools)
if len(tools) > 0 {
ret.Tools = tools
}
if !opts.Raw {
ret.Temperature = openai.Float(opts.Temperature)
ret.TopP = openai.Float(opts.TopP)
@@ -232,15 +269,41 @@ func convertMessage(msg chat.ChatCompletionMessage) responses.ResponseInputItemU
}
func (o *Client) extractText(resp *responses.Response) (ret string) {
var textParts []string
var citations []string
citationMap := make(map[string]bool) // To avoid duplicate citations
for _, item := range resp.Output {
if item.Type == "message" {
for _, c := range item.Content {
if c.Type == "output_text" {
ret += c.AsOutputText().Text
outputText := c.AsOutputText()
textParts = append(textParts, outputText.Text)
// Extract citations from annotations
for _, annotation := range outputText.Annotations {
if annotation.Type == "url_citation" {
urlCitation := annotation.AsURLCitation()
citationKey := urlCitation.URL + "|" + urlCitation.Title
if !citationMap[citationKey] {
citationMap[citationKey] = true
citationText := fmt.Sprintf("- [%s](%s)", urlCitation.Title, urlCitation.URL)
citations = append(citations, citationText)
}
}
}
}
}
break
}
}
ret = strings.Join(textParts, "")
// Append citations if any were found
if len(citations) > 0 {
ret += "\n\n## Sources\n\n" + strings.Join(citations, "\n")
}
return
}

View File

@@ -0,0 +1,146 @@
package openai
// This file contains helper methods for image generation and processing
// using OpenAI's Responses API and Image API.
import (
"encoding/base64"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/danielmiessler/fabric/common"
"github.com/openai/openai-go/packages/param"
"github.com/openai/openai-go/responses"
)
// ImageGenerationResponseType is the type used for image generation calls in responses
const ImageGenerationResponseType = "image_generation_call"
const ImageGenerationToolType = "image_generation"
// ImageGenerationSupportedModels lists all models that support image generation
var ImageGenerationSupportedModels = []string{
"gpt-4o",
"gpt-4o-mini",
"gpt-4.1",
"gpt-4.1-mini",
"gpt-4.1-nano",
"o3",
}
// supportsImageGeneration checks if the given model supports the image_generation tool
func supportsImageGeneration(model string) bool {
for _, supportedModel := range ImageGenerationSupportedModels {
if model == supportedModel {
return true
}
}
return false
}
// getOutputFormatFromExtension determines the API output format based on file extension
func getOutputFormatFromExtension(imagePath string) string {
if imagePath == "" {
return "png" // Default format
}
ext := strings.ToLower(filepath.Ext(imagePath))
switch ext {
case ".png":
return "png"
case ".webp":
return "webp"
case ".jpg":
return "jpeg"
case ".jpeg":
return "jpeg"
default:
return "png" // Default fallback
}
}
// addImageGenerationTool adds the image generation tool to the request if needed
func (o *Client) addImageGenerationTool(opts *common.ChatOptions, tools []responses.ToolUnionParam) []responses.ToolUnionParam {
// Check if the request seems to be asking for image generation
if o.shouldUseImageGeneration(opts) {
outputFormat := getOutputFormatFromExtension(opts.ImageFile)
// Build the image generation tool with user parameters
imageGenTool := responses.ToolUnionParam{
OfImageGeneration: &responses.ToolImageGenerationParam{
Type: ImageGenerationToolType,
Model: "gpt-image-1",
OutputFormat: outputFormat,
},
}
// Set quality if specified by user (otherwise let OpenAI use default)
if opts.ImageQuality != "" {
imageGenTool.OfImageGeneration.Quality = opts.ImageQuality
}
// Set size if specified by user (otherwise let OpenAI use default)
if opts.ImageSize != "" {
imageGenTool.OfImageGeneration.Size = opts.ImageSize
}
// Set background if specified by user (otherwise let OpenAI use default)
if opts.ImageBackground != "" {
imageGenTool.OfImageGeneration.Background = opts.ImageBackground
}
// Set compression if specified by user (only for jpeg/webp)
if opts.ImageCompression != 0 {
imageGenTool.OfImageGeneration.OutputCompression = param.NewOpt(int64(opts.ImageCompression))
}
tools = append(tools, imageGenTool)
}
return tools
}
// shouldUseImageGeneration determines if image generation should be enabled
// This is a heuristic based on the presence of --image-file flag
func (o *Client) shouldUseImageGeneration(opts *common.ChatOptions) bool {
return opts.ImageFile != ""
}
// extractAndSaveImages extracts generated images from the response and saves them
func (o *Client) extractAndSaveImages(resp *responses.Response, opts *common.ChatOptions) error {
if opts.ImageFile == "" {
return nil // No image file specified, skip saving
}
// Extract image data from response
for _, item := range resp.Output {
if item.Type == ImageGenerationResponseType {
imageCall := item.AsImageGenerationCall()
if imageCall.Status == "completed" && imageCall.Result != "" {
// Decode base64 image data
imageData, err := base64.StdEncoding.DecodeString(imageCall.Result)
if err != nil {
return fmt.Errorf("failed to decode image data: %w", err)
}
// Ensure directory exists
dir := filepath.Dir(opts.ImageFile)
if dir != "." {
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("failed to create directory %s: %w", dir, err)
}
}
// Save image to file
if err := os.WriteFile(opts.ImageFile, imageData, 0644); err != nil {
return fmt.Errorf("failed to save image to %s: %w", opts.ImageFile, err)
}
fmt.Printf("Image saved to: %s\n", opts.ImageFile)
return nil
}
}
}
return nil
}

View File

@@ -0,0 +1,444 @@
package openai
import (
"fmt"
"strings"
"testing"
"github.com/danielmiessler/fabric/chat"
"github.com/danielmiessler/fabric/common"
"github.com/openai/openai-go/responses"
"github.com/stretchr/testify/assert"
)
func TestShouldUseImageGeneration(t *testing.T) {
client := NewClient()
// Test with image file specified
opts := &common.ChatOptions{
ImageFile: "output.png",
}
assert.True(t, client.shouldUseImageGeneration(opts), "Should use image generation when image file is specified")
// Test without image file
opts = &common.ChatOptions{
ImageFile: "",
}
assert.False(t, client.shouldUseImageGeneration(opts), "Should not use image generation when no image file is specified")
}
func TestAddImageGenerationTool(t *testing.T) {
client := NewClient()
// Test with image generation enabled
opts := &common.ChatOptions{
ImageFile: "output.png",
}
tools := []responses.ToolUnionParam{}
result := client.addImageGenerationTool(opts, tools)
assert.Len(t, result, 1, "Should add one image generation tool")
assert.NotNil(t, result[0].OfImageGeneration, "Should have image generation tool")
assert.Equal(t, "image_generation", string(result[0].OfImageGeneration.Type))
assert.Equal(t, "gpt-image-1", result[0].OfImageGeneration.Model)
assert.Equal(t, "png", result[0].OfImageGeneration.OutputFormat)
// Test without image generation
opts = &common.ChatOptions{
ImageFile: "",
}
tools = []responses.ToolUnionParam{}
result = client.addImageGenerationTool(opts, tools)
assert.Len(t, result, 0, "Should not add image generation tool when not needed")
}
func TestBuildResponseParams_WithImageGeneration(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "gpt-image-1",
ImageFile: "output.png",
}
msgs := []*chat.ChatCompletionMessage{
{Role: "user", Content: "Generate an image of a cat"},
}
params := client.buildResponseParams(msgs, opts)
assert.NotNil(t, params.Tools, "Expected tools when image generation is enabled")
// Should have image generation tool
hasImageTool := false
for _, tool := range params.Tools {
if tool.OfImageGeneration != nil {
hasImageTool = true
assert.Equal(t, "image_generation", string(tool.OfImageGeneration.Type))
assert.Equal(t, "gpt-image-1", tool.OfImageGeneration.Model)
break
}
}
assert.True(t, hasImageTool, "Should have image generation tool")
}
func TestBuildResponseParams_WithBothSearchAndImage(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "gpt-image-1",
Search: true,
SearchLocation: "America/Los_Angeles",
ImageFile: "output.png",
}
msgs := []*chat.ChatCompletionMessage{
{Role: "user", Content: "Search for cat images and generate one"},
}
params := client.buildResponseParams(msgs, opts)
assert.NotNil(t, params.Tools, "Expected tools when both search and image generation are enabled")
assert.Len(t, params.Tools, 2, "Should have both search and image generation tools")
hasSearchTool := false
hasImageTool := false
for _, tool := range params.Tools {
if tool.OfWebSearchPreview != nil {
hasSearchTool = true
}
if tool.OfImageGeneration != nil {
hasImageTool = true
}
}
assert.True(t, hasSearchTool, "Should have web search tool")
assert.True(t, hasImageTool, "Should have image generation tool")
}
func TestGetOutputFormatFromExtension(t *testing.T) {
tests := []struct {
name string
imagePath string
expectedFormat string
}{
{
name: "PNG extension",
imagePath: "/tmp/output.png",
expectedFormat: "png",
},
{
name: "WEBP extension",
imagePath: "/tmp/output.webp",
expectedFormat: "webp",
},
{
name: "JPG extension",
imagePath: "/tmp/output.jpg",
expectedFormat: "jpeg",
},
{
name: "JPEG extension",
imagePath: "/tmp/output.jpeg",
expectedFormat: "jpeg",
},
{
name: "Uppercase PNG extension",
imagePath: "/tmp/output.PNG",
expectedFormat: "png",
},
{
name: "Mixed case JPEG extension",
imagePath: "/tmp/output.JpEg",
expectedFormat: "jpeg",
},
{
name: "Empty path",
imagePath: "",
expectedFormat: "png",
},
{
name: "No extension",
imagePath: "/tmp/output",
expectedFormat: "png",
},
{
name: "Unsupported extension",
imagePath: "/tmp/output.gif",
expectedFormat: "png",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := getOutputFormatFromExtension(tt.imagePath)
assert.Equal(t, tt.expectedFormat, result)
})
}
}
func TestAddImageGenerationToolWithDynamicFormat(t *testing.T) {
client := NewClient()
tests := []struct {
name string
imageFile string
expectedFormat string
}{
{
name: "PNG file",
imageFile: "/tmp/output.png",
expectedFormat: "png",
},
{
name: "WEBP file",
imageFile: "/tmp/output.webp",
expectedFormat: "webp",
},
{
name: "JPG file",
imageFile: "/tmp/output.jpg",
expectedFormat: "jpeg",
},
{
name: "JPEG file",
imageFile: "/tmp/output.jpeg",
expectedFormat: "jpeg",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
opts := &common.ChatOptions{
ImageFile: tt.imageFile,
}
tools := client.addImageGenerationTool(opts, []responses.ToolUnionParam{})
assert.Len(t, tools, 1, "Should have one tool")
assert.NotNil(t, tools[0].OfImageGeneration, "Should be image generation tool")
assert.Equal(t, tt.expectedFormat, tools[0].OfImageGeneration.OutputFormat, "Output format should match file extension")
})
}
}
func TestSupportsImageGeneration(t *testing.T) {
tests := []struct {
name string
model string
expected bool
}{
{
name: "gpt-4o supports image generation",
model: "gpt-4o",
expected: true,
},
{
name: "gpt-4o-mini supports image generation",
model: "gpt-4o-mini",
expected: true,
},
{
name: "gpt-4.1 supports image generation",
model: "gpt-4.1",
expected: true,
},
{
name: "gpt-4.1-mini supports image generation",
model: "gpt-4.1-mini",
expected: true,
},
{
name: "gpt-4.1-nano supports image generation",
model: "gpt-4.1-nano",
expected: true,
},
{
name: "o3 supports image generation",
model: "o3",
expected: true,
},
{
name: "o1 does not support image generation",
model: "o1",
expected: false,
},
{
name: "o1-mini does not support image generation",
model: "o1-mini",
expected: false,
},
{
name: "o3-mini does not support image generation",
model: "o3-mini",
expected: false,
},
{
name: "gpt-4 does not support image generation",
model: "gpt-4",
expected: false,
},
{
name: "gpt-3.5-turbo does not support image generation",
model: "gpt-3.5-turbo",
expected: false,
},
{
name: "empty model does not support image generation",
model: "",
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := supportsImageGeneration(tt.model)
assert.Equal(t, tt.expected, result)
})
}
}
func TestModelValidationLogic(t *testing.T) {
t.Run("Unsupported model with image file should return validation error", func(t *testing.T) {
opts := &common.ChatOptions{
Model: "o1-mini",
ImageFile: "/tmp/output.png",
}
// Test the validation logic directly
if opts.ImageFile != "" && !supportsImageGeneration(opts.Model) {
err := fmt.Errorf("model '%s' does not support image generation. Supported models: %s", opts.Model, strings.Join(ImageGenerationSupportedModels, ", "))
assert.Contains(t, err.Error(), "does not support image generation")
assert.Contains(t, err.Error(), "o1-mini")
assert.Contains(t, err.Error(), "Supported models:")
} else {
t.Error("Expected validation to trigger")
}
})
t.Run("Supported model with image file should not trigger validation", func(t *testing.T) {
opts := &common.ChatOptions{
Model: "gpt-4o",
ImageFile: "/tmp/output.png",
}
// Test the validation logic directly
shouldFail := opts.ImageFile != "" && !supportsImageGeneration(opts.Model)
assert.False(t, shouldFail, "Validation should not trigger for supported model")
})
t.Run("Unsupported model without image file should not trigger validation", func(t *testing.T) {
opts := &common.ChatOptions{
Model: "o1-mini",
ImageFile: "", // No image file
}
// Test the validation logic directly
shouldFail := opts.ImageFile != "" && !supportsImageGeneration(opts.Model)
assert.False(t, shouldFail, "Validation should not trigger when no image file is specified")
})
}
func TestAddImageGenerationToolWithUserParameters(t *testing.T) {
client := NewClient()
tests := []struct {
name string
opts *common.ChatOptions
expected map[string]interface{}
}{
{
name: "All parameters specified",
opts: &common.ChatOptions{
ImageFile: "/tmp/test.png",
ImageSize: "1536x1024",
ImageQuality: "high",
ImageBackground: "transparent",
ImageCompression: 0, // Not applicable for PNG
},
expected: map[string]interface{}{
"size": "1536x1024",
"quality": "high",
"background": "transparent",
"output_format": "png",
},
},
{
name: "JPEG with compression",
opts: &common.ChatOptions{
ImageFile: "/tmp/test.jpg",
ImageSize: "1024x1024",
ImageQuality: "medium",
ImageBackground: "opaque",
ImageCompression: 75,
},
expected: map[string]interface{}{
"size": "1024x1024",
"quality": "medium",
"background": "opaque",
"output_format": "jpeg",
"output_compression": int64(75),
},
},
{
name: "Only some parameters specified",
opts: &common.ChatOptions{
ImageFile: "/tmp/test.webp",
ImageQuality: "low",
},
expected: map[string]interface{}{
"quality": "low",
"output_format": "webp",
},
},
{
name: "No parameters specified (defaults)",
opts: &common.ChatOptions{
ImageFile: "/tmp/test.png",
},
expected: map[string]interface{}{
"output_format": "png",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tools := client.addImageGenerationTool(tt.opts, []responses.ToolUnionParam{})
assert.Len(t, tools, 1)
assert.NotNil(t, tools[0].OfImageGeneration)
tool := tools[0].OfImageGeneration
// Check required fields
assert.Equal(t, "gpt-image-1", tool.Model)
assert.Equal(t, tt.expected["output_format"], tool.OutputFormat)
// Check optional fields
if expectedSize, ok := tt.expected["size"]; ok {
assert.Equal(t, expectedSize, tool.Size)
} else {
assert.Empty(t, tool.Size, "Size should not be set when not specified")
}
if expectedQuality, ok := tt.expected["quality"]; ok {
assert.Equal(t, expectedQuality, tool.Quality)
} else {
assert.Empty(t, tool.Quality, "Quality should not be set when not specified")
}
if expectedBackground, ok := tt.expected["background"]; ok {
assert.Equal(t, expectedBackground, tool.Background)
} else {
assert.Empty(t, tool.Background, "Background should not be set when not specified")
}
if expectedCompression, ok := tt.expected["output_compression"]; ok {
assert.Equal(t, expectedCompression, tool.OutputCompression.Value)
} else {
assert.Equal(t, int64(0), tool.OutputCompression.Value, "Compression should not be set when not specified")
}
})
}
}

View File

@@ -1,11 +1,13 @@
package openai
import (
"strings"
"testing"
"github.com/danielmiessler/fabric/chat"
"github.com/danielmiessler/fabric/common"
openai "github.com/openai/openai-go"
"github.com/openai/openai-go/responses"
"github.com/openai/openai-go/shared"
"github.com/stretchr/testify/assert"
)
@@ -60,3 +62,116 @@ func TestBuildResponseRequestNoMaxTokens(t *testing.T) {
assert.Equal(t, openai.Float(opts.TopP), request.TopP)
assert.False(t, request.MaxOutputTokens.Valid())
}
func TestBuildResponseParams_WithoutSearch(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "gpt-4o",
Temperature: 0.7,
Search: false,
}
msgs := []*chat.ChatCompletionMessage{
{Role: "user", Content: "Hello"},
}
params := client.buildResponseParams(msgs, opts)
assert.Nil(t, params.Tools, "Expected no tools when search is disabled")
assert.Equal(t, shared.ResponsesModel(opts.Model), params.Model)
assert.Equal(t, openai.Float(opts.Temperature), params.Temperature)
}
func TestBuildResponseParams_WithSearch(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "gpt-4o",
Temperature: 0.7,
Search: true,
}
msgs := []*chat.ChatCompletionMessage{
{Role: "user", Content: "What's the weather today?"},
}
params := client.buildResponseParams(msgs, opts)
assert.NotNil(t, params.Tools, "Expected tools when search is enabled")
assert.Len(t, params.Tools, 1, "Expected exactly one tool")
tool := params.Tools[0]
assert.NotNil(t, tool.OfWebSearchPreview, "Expected web search tool")
assert.Equal(t, responses.WebSearchToolType("web_search_preview"), tool.OfWebSearchPreview.Type)
}
func TestBuildResponseParams_WithSearchAndLocation(t *testing.T) {
client := NewClient()
opts := &common.ChatOptions{
Model: "gpt-4o",
Temperature: 0.7,
Search: true,
SearchLocation: "America/Los_Angeles",
}
msgs := []*chat.ChatCompletionMessage{
{Role: "user", Content: "What's the weather in San Francisco?"},
}
params := client.buildResponseParams(msgs, opts)
assert.NotNil(t, params.Tools, "Expected tools when search is enabled")
tool := params.Tools[0]
assert.NotNil(t, tool.OfWebSearchPreview, "Expected web search tool")
userLocation := tool.OfWebSearchPreview.UserLocation
assert.Equal(t, "approximate", string(userLocation.Type))
assert.True(t, userLocation.Timezone.Valid(), "Expected timezone to be set")
assert.Equal(t, opts.SearchLocation, userLocation.Timezone.Value)
}
func TestCitationFormatting(t *testing.T) {
// Test the citation formatting logic by simulating the citation extraction
var textParts []string
var citations []string
citationMap := make(map[string]bool)
// Simulate text content
textParts = append(textParts, "Based on recent research, artificial intelligence is advancing rapidly.")
// Simulate citations (as they would be extracted from OpenAI response)
mockCitations := []struct {
URL string
Title string
}{
{"https://example.com/ai-research", "AI Research Advances 2025"},
{"https://another-source.com/tech-news", "Technology News Today"},
{"https://example.com/ai-research", "AI Research Advances 2025"}, // Duplicate to test deduplication
}
for _, citation := range mockCitations {
citationKey := citation.URL + "|" + citation.Title
if !citationMap[citationKey] {
citationMap[citationKey] = true
citationText := "- [" + citation.Title + "](" + citation.URL + ")"
citations = append(citations, citationText)
}
}
result := strings.Join(textParts, "")
if len(citations) > 0 {
result += "\n\n## Sources\n\n" + strings.Join(citations, "\n")
}
// Verify the result contains the expected text
expectedText := "Based on recent research, artificial intelligence is advancing rapidly."
assert.Contains(t, result, expectedText, "Expected result to contain original text")
// Verify citations are included
assert.Contains(t, result, "## Sources", "Expected result to contain Sources section")
assert.Contains(t, result, "[AI Research Advances 2025](https://example.com/ai-research)", "Expected result to contain first citation")
assert.Contains(t, result, "[Technology News Today](https://another-source.com/tech-news)", "Expected result to contain second citation")
// Verify deduplication - should only have 2 unique citations, not 3
citationCount := strings.Count(result, "- [")
assert.Equal(t, 2, citationCount, "Expected 2 unique citations")
}

View File

@@ -152,7 +152,6 @@ func (o *Setting) FillEnvFileContent(buffer *bytes.Buffer) {
}
buffer.WriteString("\n")
}
return
}
func ParseBoolElseFalse(val string) (ret bool) {
@@ -279,7 +278,6 @@ func (o Settings) FillEnvFileContent(buffer *bytes.Buffer) {
for _, setting := range o {
setting.FillEnvFileContent(buffer)
}
return
}
type SetupQuestions []*SetupQuestion

View File

@@ -1,3 +1,3 @@
package main
var version = "v1.4.224"
var version = "v1.4.230"