feat(hosted-key-services) Add hosted key for multiple services (#3461)

* feat(hosted keys): Implement serper hosted key

* Handle required fields correctly for hosted keys

* Add rate limiting (3 tries, exponential backoff)

* Add custom pricing, switch to exa as first hosted key

* Add telemetry

* Consolidate byok type definitions

* Add warning comment if default calculation is used

* Record usage to user stats table

* Fix unit tests, use cost property

* Include more metadata in cost output

* Fix disabled tests

* Fix spacing

* Fix lint

* Move knowledge cost restructuring away from generic block handler

* Migrate knowledge unit tests

* Lint

* Fix broken tests

* Add user based hosted key throttling

* Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules.

* Remove research as hosted key. Recommend BYOK if throtttling occurs

* Make adding api keys adjustable via env vars

* Remove vestigial fields from research

* Make billing actor id required for throttling

* Switch to round robin for api key distribution

* Add helper method for adding hosted key cost

* Strip leading double underscores to avoid breaking change

* Lint fix

* Remove falsy check in favor for explicit null check

* Add more detailed metrics for different throttling types

* Fix _costDollars field

* Handle hosted agent tool calls

* Fail loudly if cost field isn't found

* Remove any type

* Fix type error

* Fix lint

* Fix usage log double logging data

* Fix test

* Add browseruse hosted key

* Add firecrawl and serper hosted keys

* feat(hosted key): Add exa hosted key (#3221)

* feat(hosted keys): Implement serper hosted key

* Handle required fields correctly for hosted keys

* Add rate limiting (3 tries, exponential backoff)

* Add custom pricing, switch to exa as first hosted key

* Add telemetry

* Consolidate byok type definitions

* Add warning comment if default calculation is used

* Record usage to user stats table

* Fix unit tests, use cost property

* Include more metadata in cost output

* Fix disabled tests

* Fix spacing

* Fix lint

* Move knowledge cost restructuring away from generic block handler

* Migrate knowledge unit tests

* Lint

* Fix broken tests

* Add user based hosted key throttling

* Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules.

* Remove research as hosted key. Recommend BYOK if throtttling occurs

* Make adding api keys adjustable via env vars

* Remove vestigial fields from research

* Make billing actor id required for throttling

* Switch to round robin for api key distribution

* Add helper method for adding hosted key cost

* Strip leading double underscores to avoid breaking change

* Lint fix

* Remove falsy check in favor for explicit null check

* Add more detailed metrics for different throttling types

* Fix _costDollars field

* Handle hosted agent tool calls

* Fail loudly if cost field isn't found

* Remove any type

* Fix type error

* Fix lint

* Fix usage log double logging data

* Fix test

---------

Co-authored-by: Theodore Li <teddy@zenobiapay.com>

* Fail fast on cost data not being found

* Add hosted key for google services

* Add hosting configuration and pricing logic for ElevenLabs TTS tools

* Add linkup hosted key

* Add jina hosted key

* Add hugging face hosted key

* Add perplexity hosting

* Add broader metrics for throttling

* Add skill for adding hosted key

* Lint, remove vestigial hosted keys not implemented

* Revert agent changes

* fail fast

* Fix build issue

* Fix build issues

* Fix type error

* Remove byok types that aren't implemented

* Address feedback

* Use default model when model id isn't provided

* Fix cost default issues

* Remove firecrawl error suppression

* Restore original behavior for hugging face

* Add mistral hosted key

* Remove hugging face hosted key

* Fix pricing mismatch is mistral and perplexity

* Add hosted keys for parallel and brand fetch

* Add brandfetch hosted key

* Update types

* Change byok name to parallel_ai

* Add telemetry on unknown models

---------

Co-authored-by: Theodore Li <theo@sim.ai>
This commit is contained in:
Theodore Li
2026-03-09 19:56:45 -07:00
committed by GitHub
parent 9400df6085
commit 8fc75a6e9d
65 changed files with 1399 additions and 44 deletions

View File

@@ -0,0 +1,257 @@
---
name: add-hosted-key
description: Add hosted API key support to a tool so Sim provides the key when users don't bring their own. Use when adding hosted keys, BYOK support, hideWhenHosted, or hosted key pricing to a tool or block.
---
# Adding Hosted Key Support to a Tool
When a tool has hosted key support, Sim provides its own API key if the user hasn't configured one (via BYOK or env var). Usage is metered and billed to the workspace.
## Overview
| Step | What | Where |
|------|------|-------|
| 1 | Register BYOK provider ID | `tools/types.ts`, `app/api/workspaces/[id]/byok-keys/route.ts` |
| 2 | Research the API's pricing and rate limits | API docs / pricing page (before writing any code) |
| 3 | Add `hosting` config to the tool | `tools/{service}/{action}.ts` |
| 4 | Hide API key field when hosted | `blocks/blocks/{service}.ts` |
| 5 | Add to BYOK settings UI | BYOK settings component (`byok.tsx`) |
| 6 | Summarize pricing and throttling comparison | Output to user (after all code changes) |
## Step 1: Register the BYOK Provider ID
Add the new provider to the `BYOKProviderId` union in `tools/types.ts`:
```typescript
export type BYOKProviderId =
| 'openai'
| 'anthropic'
// ...existing providers
| 'your_service'
```
Then add it to `VALID_PROVIDERS` in `app/api/workspaces/[id]/byok-keys/route.ts`:
```typescript
const VALID_PROVIDERS = ['openai', 'anthropic', 'google', 'mistral', 'your_service'] as const
```
## Step 2: Research the API's Pricing Model and Rate Limits
**Before writing any `getCost` or `rateLimit` code**, look up the service's official documentation for both pricing and rate limits. You need to understand:
### Pricing
1. **How the API charges** — per request, per credit, per token, per step, per minute, etc.
2. **Whether the API reports cost in its response** — look for fields like `creditsUsed`, `costDollars`, `tokensUsed`, or similar in the response body or headers
3. **Whether cost varies by endpoint/options** — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. **The dollar-per-unit rate** — what each credit/token/unit costs in dollars on our plan
### Rate Limits
1. **What rate limits the API enforces** — requests per minute/second, tokens per minute, concurrent requests, etc.
2. **Whether limits vary by plan tier** — free vs paid vs enterprise often have different ceilings
3. **Whether limits are per-key or per-account** — determines whether adding more hosted keys actually increases total throughput
4. **What the API returns when rate limited** — HTTP 429, `Retry-After` header, error body format, etc.
5. **Whether there are multiple dimensions** — some APIs limit both requests/min AND tokens/min independently
Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in `getCost` so future maintainers know the source of truth.
### Setting Our Rate Limits
Our rate limiter (`lib/core/rate-limiter/hosted-key/`) uses a token-bucket algorithm applied **per billing actor** (workspace). It supports two modes:
- **`per_request`** — simple; just `requestsPerMinute`. Good when the API charges flat per-request or cost doesn't vary much.
- **`custom`** — `requestsPerMinute` plus additional `dimensions` (e.g., `tokens`, `search_units`). Each dimension has its own `limitPerMinute` and an `extractUsage` function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.
When choosing values for `requestsPerMinute` and any dimension limits:
- **Stay well below the API's per-key limit** — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
- **Account for key pooling** — our round-robin distributes requests across `N` hosted keys, so the effective API-side rate per key is `(total requests) / N`. But per-workspace limits are enforced *before* key selection, so they apply regardless of key count.
- **Prefer conservative defaults** — it's easy to raise limits later but hard to claw back after users depend on high throughput.
## Step 3: Add `hosting` Config to the Tool
Add a `hosting` object to the tool's `ToolConfig`. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.
```typescript
hosting: {
envKeyPrefix: 'YOUR_SERVICE_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'your_service',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001 // dollars per credit
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
```
### Hosted Key Env Var Convention
Keys use a numbered naming pattern driven by a count env var:
```
YOUR_SERVICE_API_KEY_COUNT=3
YOUR_SERVICE_API_KEY_1=sk-...
YOUR_SERVICE_API_KEY_2=sk-...
YOUR_SERVICE_API_KEY_3=sk-...
```
The `envKeyPrefix` value (`YOUR_SERVICE_API_KEY`) determines which env vars are read at runtime. Adding more keys only requires bumping the count and adding the new env var.
### Pricing: Prefer API-Reported Cost
Always prefer using cost data returned by the API (e.g., `creditsUsed`, `costDollars`). This is the most accurate because it accounts for variable pricing tiers, feature modifiers, and plan-level discounts.
**When the API reports cost** — use it directly and throw if missing:
```typescript
pricing: {
type: 'custom',
getCost: (params, output) => {
if (output.creditsUsed == null) {
throw new Error('Response missing creditsUsed field')
}
// $0.001 per credit — from https://example.com/pricing
const cost = (output.creditsUsed as number) * 0.001
return { cost, metadata: { creditsUsed: output.creditsUsed } }
},
},
```
**When the API does NOT report cost** — compute it from params/output based on the pricing docs, but still validate the data you depend on:
```typescript
pricing: {
type: 'custom',
getCost: (params, output) => {
if (!Array.isArray(output.searchResults)) {
throw new Error('Response missing searchResults, cannot determine cost')
}
// Serper: 1 credit for <=10 results, 2 credits for >10 — from https://serper.dev/pricing
const credits = Number(params.num) > 10 ? 2 : 1
return { cost: credits * 0.001, metadata: { credits } }
},
},
```
**`getCost` must always throw** if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.
### Capturing Cost Data from the API
If the API returns cost info, capture it in `transformResponse` so `getCost` can read it from the output:
```typescript
transformResponse: async (response: Response) => {
const data = await response.json()
return {
success: true,
output: {
results: data.results,
creditsUsed: data.creditsUsed, // pass through for getCost
},
}
},
```
For async/polling tools, capture it in `postProcess` when the job completes:
```typescript
if (jobData.status === 'completed') {
result.output = {
data: jobData.data,
creditsUsed: jobData.creditsUsed,
}
}
```
## Step 4: Hide the API Key Field When Hosted
In the block config (`blocks/blocks/{service}.ts`), add `hideWhenHosted: true` to the API key subblock. This hides the field on hosted Sim since the platform provides the key:
```typescript
{
id: 'apiKey',
title: 'API Key',
type: 'short-input',
placeholder: 'Enter your API key',
password: true,
required: true,
hideWhenHosted: true,
},
```
The visibility is controlled by `isSubBlockHiddenByHostedKey()` in `lib/workflows/subblocks/visibility.ts`, which checks the `isHosted` feature flag.
## Step 5: Add to the BYOK Settings UI
Add an entry to the `PROVIDERS` array in the BYOK settings component so users can bring their own key. You need the service icon from `components/icons.tsx`:
```typescript
{
id: 'your_service',
name: 'Your Service',
icon: YourServiceIcon,
description: 'What this service does',
placeholder: 'Enter your API key',
},
```
## Step 6: Summarize Pricing and Throttling Comparison
After all code changes are complete, output a detailed summary to the user covering:
### What to include
1. **API's pricing model** — how the service charges (per token, per credit, per request, etc.), the specific rates found in docs, and whether the API reports cost in responses.
2. **Our `getCost` approach** — how we calculate cost, what fields we depend on, and any assumptions or estimates (especially when the API doesn't report exact dollar cost).
3. **API's rate limits** — the documented limits (RPM, TPM, concurrent, etc.), which plan tier they apply to, and whether they're per-key or per-account.
4. **Our `rateLimit` config** — what we set for `requestsPerMinute` (and dimensions if custom mode), why we chose those values, and how they compare to the API's limits.
5. **Key pooling impact** — how many hosted keys we expect, and how round-robin distribution affects the effective per-key rate at the API.
6. **Gaps or risks** — anything the API charges for that we don't meter, rate limit dimensions we chose not to enforce, or pricing that may be inaccurate due to variable model/tier costs.
### Format
Present this as a structured summary with clear headings. Example:
```
### Pricing
- **API charges**: $X per 1M tokens (input), $Y per 1M tokens (output) — varies by model
- **Response reports cost?**: No — only token counts in `usage` field
- **Our getCost**: Estimates cost at $Z per 1M total tokens based on median model pricing
- **Risk**: Actual cost varies by model; our estimate may over/undercharge for cheap/expensive models
### Throttling
- **API limits**: 300 RPM per key (paid tier), 60 RPM (free tier)
- **Per-key or per-account**: Per key — more keys = more throughput
- **Our config**: 60 RPM per workspace (per_request mode)
- **With N keys**: Effective per-key rate is (total RPM across workspaces) / N
- **Headroom**: Comfortable — even 10 active workspaces at full rate = 600 RPM / 3 keys = 200 RPM per key, under the 300 RPM API limit
```
This summary helps reviewers verify that the pricing and rate limiting are well-calibrated and surfaces any risks that need monitoring.
## Checklist
- [ ] Provider added to `BYOKProviderId` in `tools/types.ts`
- [ ] Provider added to `VALID_PROVIDERS` in the BYOK keys API route
- [ ] API pricing docs researched — understand per-unit cost and whether the API reports cost in responses
- [ ] API rate limits researched — understand RPM/TPM limits, per-key vs per-account, and plan tiers
- [ ] `hosting` config added to the tool with `envKeyPrefix`, `apiKeyParam`, `byokProviderId`, `pricing`, and `rateLimit`
- [ ] `getCost` throws if required cost data is missing from the response
- [ ] Cost data captured in `transformResponse` or `postProcess` if API provides it
- [ ] `hideWhenHosted: true` added to the API key subblock in the block config
- [ ] Provider entry added to the BYOK settings UI with icon and description
- [ ] Env vars documented: `{PREFIX}_COUNT` and `{PREFIX}_1..N`
- [ ] Pricing and throttling summary provided to reviewer

View File

@@ -1,5 +1,5 @@
import { existsSync } from 'fs'
import { join, resolve, sep } from 'path'
import path from 'path'
import { createLogger } from '@sim/logger'
import { NextResponse } from 'next/server'
import { UPLOAD_DIR } from '@/lib/uploads/config'
@@ -156,7 +156,7 @@ function sanitizeFilename(filename: string): string {
return sanitized
})
return sanitizedSegments.join(sep)
return sanitizedSegments.join(path.sep)
}
export function findLocalFile(filename: string): string | null {
@@ -169,17 +169,18 @@ export function findLocalFile(filename: string): string | null {
}
const possiblePaths = [
join(UPLOAD_DIR, sanitizedFilename),
join(process.cwd(), 'uploads', sanitizedFilename),
path.join(UPLOAD_DIR, sanitizedFilename),
path.join(process.cwd(), 'uploads', sanitizedFilename),
]
for (const path of possiblePaths) {
const resolvedPath = resolve(path)
const allowedDirs = [resolve(UPLOAD_DIR), resolve(process.cwd(), 'uploads')]
for (const filePath of possiblePaths) {
const resolvedPath = path.resolve(filePath)
const allowedDirs = [path.resolve(UPLOAD_DIR), path.resolve(process.cwd(), 'uploads')]
// Must be within allowed directory but NOT the directory itself
const isWithinAllowedDir = allowedDirs.some(
(allowedDir) => resolvedPath.startsWith(allowedDir + sep) && resolvedPath !== allowedDir
(allowedDir) =>
resolvedPath.startsWith(allowedDir + path.sep) && resolvedPath !== allowedDir
)
if (!isWithinAllowedDir) {

View File

@@ -13,7 +13,23 @@ import { getUserEntityPermissions, getWorkspaceById } from '@/lib/workspaces/per
const logger = createLogger('WorkspaceBYOKKeysAPI')
const VALID_PROVIDERS = ['openai', 'anthropic', 'google', 'mistral', 'exa'] as const
const VALID_PROVIDERS = [
'openai',
'anthropic',
'google',
'mistral',
'exa',
'browser_use',
'serper',
'firecrawl',
'linkup',
'perplexity',
'jina',
'google_cloud',
'elevenlabs',
'parallel_ai',
'brandfetch',
] as const
const UpsertKeySchema = z.object({
providerId: z.enum(VALID_PROVIDERS),

View File

@@ -13,7 +13,23 @@ import {
ModalFooter,
ModalHeader,
} from '@/components/emcn'
import { AnthropicIcon, ExaAIIcon, GeminiIcon, MistralIcon, OpenAIIcon } from '@/components/icons'
import {
AnthropicIcon,
BrandfetchIcon,
BrowserUseIcon,
ElevenLabsIcon,
ExaAIIcon,
FirecrawlIcon,
GeminiIcon,
GoogleIcon,
JinaAIIcon,
LinkupIcon,
MistralIcon,
OpenAIIcon,
ParallelIcon,
PerplexityIcon,
SerperIcon,
} from '@/components/icons'
import { Skeleton } from '@/components/ui'
import {
type BYOKKey,
@@ -67,6 +83,76 @@ const PROVIDERS: {
description: 'AI-powered search and research',
placeholder: 'Enter your Exa API key',
},
{
id: 'browser_use',
name: 'Browser Use',
icon: BrowserUseIcon,
description: 'Browser automation tasks',
placeholder: 'Enter your Browser Use API key',
},
{
id: 'serper',
name: 'Serper',
icon: SerperIcon,
description: 'Google search API',
placeholder: 'Enter your Serper API key',
},
{
id: 'firecrawl',
name: 'Firecrawl',
icon: FirecrawlIcon,
description: 'Web scraping, crawling, and data extraction',
placeholder: 'Enter your Firecrawl API key',
},
{
id: 'linkup',
name: 'Linkup',
icon: LinkupIcon,
description: 'Web search and content retrieval',
placeholder: 'Enter your Linkup API key',
},
{
id: 'parallel_ai',
name: 'Parallel AI',
icon: ParallelIcon,
description: 'Web search, extraction, and deep research',
placeholder: 'Enter your Parallel AI API key',
},
{
id: 'perplexity',
name: 'Perplexity',
icon: PerplexityIcon,
description: 'AI-powered chat and web search',
placeholder: 'pplx-...',
},
{
id: 'jina',
name: 'Jina AI',
icon: JinaAIIcon,
description: 'Web reading and search',
placeholder: 'jina_...',
},
{
id: 'elevenlabs',
name: 'ElevenLabs',
icon: ElevenLabsIcon,
description: 'Text-to-speech generation',
placeholder: 'Enter your ElevenLabs API key',
},
{
id: 'google_cloud',
name: 'Google Cloud',
icon: GoogleIcon,
description: 'Translate, Maps, PageSpeed, and Books APIs',
placeholder: 'Enter your Google Cloud API key',
},
{
id: 'brandfetch',
name: 'Brandfetch',
icon: BrandfetchIcon,
description: 'Brand assets, logos, colors, and company info',
placeholder: 'Enter your Brandfetch API key',
},
]
function BYOKKeySkeleton() {

View File

@@ -49,6 +49,7 @@ export const BrandfetchBlock: BlockConfig<BrandfetchGetBrandResponse | Brandfetc
placeholder: 'Enter your Brandfetch API key',
required: true,
password: true,
hideWhenHosted: true,
},
],

View File

@@ -72,6 +72,7 @@ export const BrowserUseBlock: BlockConfig<BrowserUseResponse> = {
password: true,
placeholder: 'Enter your BrowserUse API key',
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -48,6 +48,7 @@ export const ElevenLabsBlock: BlockConfig<ElevenLabsBlockResponse> = {
placeholder: 'Enter your ElevenLabs API key',
password: true,
required: true,
hideWhenHosted: true,
},
],

View File

@@ -248,6 +248,7 @@ Example 2 - Product Data:
placeholder: 'Enter your Firecrawl API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -32,6 +32,7 @@ export const GoogleBooksBlock: BlockConfig = {
password: true,
placeholder: 'Enter your Google Books API key',
required: true,
hideWhenHosted: true,
},
{
id: 'query',

View File

@@ -44,6 +44,7 @@ export const GoogleMapsBlock: BlockConfig = {
password: true,
placeholder: 'Enter your Google Maps API key',
required: true,
hideWhenHosted: true,
},
// ========== Geocode ==========

View File

@@ -58,6 +58,7 @@ export const GooglePagespeedBlock: BlockConfig<GooglePagespeedAnalyzeResponse> =
required: true,
placeholder: 'Enter your Google PageSpeed API key',
password: true,
hideWhenHosted: true,
},
],

View File

@@ -192,6 +192,7 @@ export const GoogleTranslateBlock: BlockConfig = {
placeholder: 'Enter your Google Cloud API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -144,6 +144,7 @@ export const JinaBlock: BlockConfig<ReadUrlResponse | SearchResponse> = {
required: true,
placeholder: 'Enter your Jina API key',
password: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -120,6 +120,7 @@ Return ONLY the date string in YYYY-MM-DD format - no explanations, no quotes, n
placeholder: 'Enter your Linkup API key',
password: true,
required: true,
hideWhenHosted: true,
},
],

View File

@@ -327,6 +327,7 @@ export const MistralParseV3Block: BlockConfig<MistralParserOutput> = {
placeholder: 'Enter your Mistral API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {
@@ -334,13 +335,12 @@ export const MistralParseV3Block: BlockConfig<MistralParserOutput> = {
config: {
tool: () => 'mistral_parser_v3',
params: (params) => {
if (!params || !params.apiKey || params.apiKey.trim() === '') {
throw new Error('Mistral API key is required')
const parameters: Record<string, unknown> = {
resultType: params.resultType || 'markdown',
}
const parameters: Record<string, unknown> = {
apiKey: params.apiKey.trim(),
resultType: params.resultType || 'markdown',
if (params.apiKey?.trim()) {
parameters.apiKey = params.apiKey.trim()
}
// V3 pattern: use canonical document param directly

View File

@@ -143,6 +143,7 @@ export const ParallelBlock: BlockConfig<ToolResponse> = {
placeholder: 'Enter your Parallel AI API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -171,6 +171,7 @@ Return ONLY the date string in MM/DD/YYYY format - no explanations, no quotes, n
placeholder: 'Enter your Perplexity API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -78,6 +78,7 @@ export const SerperBlock: BlockConfig<SearchResponse> = {
placeholder: 'Enter your Serper API key',
password: true,
required: true,
hideWhenHosted: true,
},
],
tools: {

View File

@@ -21,9 +21,9 @@ export const isTest = env.NODE_ENV === 'test'
/**
* Is this the hosted version of the application
*/
export const isHosted =
getEnv('NEXT_PUBLIC_APP_URL') === 'https://www.sim.ai' ||
getEnv('NEXT_PUBLIC_APP_URL') === 'https://www.staging.sim.ai'
export const isHosted = true
// getEnv('NEXT_PUBLIC_APP_URL') === 'https://www.sim.ai' ||
// getEnv('NEXT_PUBLIC_APP_URL') === 'https://www.staging.sim.ai'
/**
* Is billing enforcement enabled

View File

@@ -938,7 +938,7 @@ export const PlatformEvents = {
* Track when a rate limit error is surfaced to the end user (not retried/absorbed).
* Fires for both billing-actor limits and exhausted upstream retries.
*/
userThrottled: (attrs: {
hostedKeyUserThrottled: (attrs: {
toolId: string
reason: 'billing_actor_limit' | 'upstream_retries_exhausted'
provider?: string
@@ -947,7 +947,7 @@ export const PlatformEvents = {
workspaceId?: string
workflowId?: string
}) => {
trackPlatformEvent('platform.user.throttled', {
trackPlatformEvent('platform.hosted_key.user_throttled', {
'tool.id': attrs.toolId,
'throttle.reason': attrs.reason,
...(attrs.provider && { 'provider.id': attrs.provider }),
@@ -983,6 +983,18 @@ export const PlatformEvents = {
})
},
hostedKeyUnknownModelCost: (attrs: {
toolId: string
modelName: string
defaultCost: number
}) => {
trackPlatformEvent('platform.hosted_key.unknown_model_cost', {
'tool.id': attrs.toolId,
'model.name': attrs.modelName,
'cost.default_cost': attrs.defaultCost,
})
},
/**
* Track chat deployed (workflow deployed as chat interface)
*/

View File

@@ -11,6 +11,21 @@ export const brandfetchGetBrandTool: ToolConfig<
'Retrieve brand assets including logos, colors, fonts, and company info by domain, ticker, ISIN, or crypto symbol',
version: '1.0.0',
hosting: {
envKeyPrefix: 'BRANDFETCH_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'brandfetch',
pricing: {
type: 'per_request',
// Brand API: $99/month for 2,500 calls = $0.0396/request — https://docs.brandfetch.com/brand-api/quotas-and-usage
cost: 0.04,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
apiKey: {
type: 'string',

View File

@@ -11,6 +11,21 @@ export const brandfetchSearchTool: ToolConfig<BrandfetchSearchParams, Brandfetch
description: 'Search for brands by name and find their domains and logos',
version: '1.0.0',
hosting: {
envKeyPrefix: 'BRANDFETCH_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'brandfetch',
pricing: {
type: 'per_request',
// Brand Search API is free (fair-use 500K/month) — https://docs.brandfetch.com/brand-search-api/rate-limits
cost: 0,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
apiKey: {
type: 'string',

View File

@@ -1,5 +1,6 @@
import { createLogger } from '@sim/logger'
import { getMaxExecutionTimeout } from '@/lib/core/execution-limits'
import { PlatformEvents } from '@/lib/core/telemetry'
import type { BrowserUseRunTaskParams, BrowserUseRunTaskResponse } from '@/tools/browser_use/types'
import type { ToolConfig, ToolResponse } from '@/tools/types'
@@ -256,6 +257,62 @@ export const runTaskTool: ToolConfig<BrowserUseRunTaskParams, BrowserUseRunTaskR
},
},
hosting: {
envKeyPrefix: 'BROWSER_USE_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'browser_use',
pricing: {
type: 'custom',
getCost: (params, output) => {
if (!Array.isArray(output.steps)) {
throw new Error('Browser Use response missing steps array, cannot determine cost')
}
const INIT_COST = 0.01
const STEP_COSTS: Record<string, number> = {
'browser-use-llm': 0.002,
'browser-use-2.0': 0.006,
o3: 0.03,
'o4-mini': 0.03,
'gemini-3-pro-preview': 0.03,
'gemini-3-flash-preview': 0.015,
'gemini-flash-latest': 0.0075,
'gemini-flash-lite-latest': 0.005,
'gemini-2.5-flash': 0.0075,
'gemini-2.5-pro': 0.03,
'claude-sonnet-4-5-20250929': 0.05,
'claude-opus-4-5-20251101': 0.05,
'claude-3-7-sonnet-20250219': 0.05,
'gpt-4o': 0.006,
'gpt-4o-mini': 0.006,
'gpt-4.1': 0.006,
'gpt-4.1-mini': 0.006,
'llama-4-maverick-17b-128e-instruct': 0.006,
}
const DEFAULT_STEP_COST = 0.006
const model = (params.model as string) || 'browser-use-2.0'
const knownCost = STEP_COSTS[model]
if (!knownCost) {
logger.warn(
`Unknown Browser Use model "${model}", using default step cost $${DEFAULT_STEP_COST}`
)
PlatformEvents.hostedKeyUnknownModelCost({
toolId: 'browser_use_run_task',
modelName: model,
defaultCost: DEFAULT_STEP_COST,
})
}
const stepCost = knownCost ?? DEFAULT_STEP_COST
const stepCount = output.steps.length
const total = INIT_COST + stepCount * stepCost
return { cost: total, metadata: { model, stepCount, stepCost, initCost: INIT_COST } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
url: 'https://api.browser-use.com/api/v2/tasks',
method: 'POST',

View File

@@ -0,0 +1,5 @@
export const FLASH_TURBO_MODELS = new Set([
'eleven_turbo_v2',
'eleven_turbo_v2_5',
'eleven_flash_v2_5',
])

View File

@@ -1,3 +1,4 @@
import { FLASH_TURBO_MODELS } from '@/tools/elevenlabs/constants'
import type { ElevenLabsTtsParams, ElevenLabsTtsResponse } from '@/tools/elevenlabs/types'
import type { ToolConfig } from '@/tools/types'
@@ -7,6 +8,32 @@ export const elevenLabsTtsTool: ToolConfig<ElevenLabsTtsParams, ElevenLabsTtsRes
description: 'Convert TTS using ElevenLabs voices',
version: '1.0.0',
hosting: {
envKeyPrefix: 'ELEVENLABS_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'elevenlabs',
pricing: {
type: 'custom',
getCost: (params, _output) => {
const text = params.text as string | undefined
if (!text) {
throw new Error('Missing text parameter, cannot determine character cost')
}
const characterCount = text.length
const modelId = (params.modelId as string) || 'eleven_monolingual_v1'
// Flash/Turbo: $0.08/1K chars, Standard/Multilingual/v3: $0.18/1K chars
// Scale tier additional character rates — https://elevenlabs.io/pricing/api
const costPer1KChars = FLASH_TURBO_MODELS.has(modelId) ? 0.08 : 0.18
const cost = (characterCount / 1000) * costPer1KChars
return { cost, metadata: { characterCount, modelId, costPer1KChars } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
text: {
type: 'string',

View File

@@ -55,6 +55,27 @@ export const agentTool: ToolConfig<AgentParams, AgentResponse> = {
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl agent response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
method: 'POST',
url: 'https://api.firecrawl.dev/v2/agent',

View File

@@ -68,6 +68,28 @@ export const crawlTool: ToolConfig<FirecrawlCrawlParams, FirecrawlCrawlResponse>
description: 'Firecrawl API Key',
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl crawl response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
url: 'https://api.firecrawl.dev/v2/crawl',
method: 'POST',

View File

@@ -79,6 +79,27 @@ export const extractTool: ToolConfig<ExtractParams, ExtractResponse> = {
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl extract response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
method: 'POST',
url: 'https://api.firecrawl.dev/v2/extract',
@@ -164,6 +185,7 @@ export const extractTool: ToolConfig<ExtractParams, ExtractResponse> = {
jobId,
success: true,
data: extractData.data || {},
creditsUsed: extractData.creditsUsed,
}
return result
}
@@ -211,5 +233,9 @@ export const extractTool: ToolConfig<ExtractParams, ExtractResponse> = {
type: 'object',
description: 'Extracted structured data according to the schema or prompt',
},
creditsUsed: {
type: 'number',
description: 'Number of Firecrawl credits consumed by the extraction',
},
},
}

View File

@@ -66,6 +66,27 @@ export const mapTool: ToolConfig<MapParams, MapResponse> = {
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl map response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
method: 'POST',
url: 'https://api.firecrawl.dev/v2/map',
@@ -100,6 +121,7 @@ export const mapTool: ToolConfig<MapParams, MapResponse> = {
output: {
success: data.success,
links: data.links || [],
creditsUsed: data.creditsUsed,
},
}
},
@@ -116,5 +138,6 @@ export const mapTool: ToolConfig<MapParams, MapResponse> = {
type: 'string',
},
},
creditsUsed: { type: 'number', description: 'Number of Firecrawl credits consumed' },
},
}

View File

@@ -31,6 +31,27 @@ export const scrapeTool: ToolConfig<ScrapeParams, ScrapeResponse> = {
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl scrape response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
method: 'POST',
url: 'https://api.firecrawl.dev/v2/scrape',
@@ -82,6 +103,7 @@ export const scrapeTool: ToolConfig<ScrapeParams, ScrapeResponse> = {
markdown: data.data.markdown,
html: data.data.html,
metadata: data.data.metadata,
creditsUsed: data.creditsUsed,
},
}
},
@@ -94,5 +116,6 @@ export const scrapeTool: ToolConfig<ScrapeParams, ScrapeResponse> = {
description: 'Page metadata including SEO and Open Graph information',
properties: PAGE_METADATA_OUTPUT_PROPERTIES,
},
creditsUsed: { type: 'number', description: 'Number of Firecrawl credits consumed' },
},
}

View File

@@ -23,6 +23,27 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
},
},
hosting: {
envKeyPrefix: 'FIRECRAWL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'firecrawl',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Firecrawl search response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
method: 'POST',
url: 'https://api.firecrawl.dev/v2/search',
@@ -58,6 +79,7 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
success: true,
output: {
data: data.data,
creditsUsed: data.creditsUsed,
},
}
},
@@ -71,5 +93,6 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
properties: SEARCH_RESULT_OUTPUT_PROPERTIES,
},
},
creditsUsed: { type: 'number', description: 'Number of Firecrawl credits consumed' },
},
}

View File

@@ -448,6 +448,7 @@ export interface ScrapeResponse extends ToolResponse {
statusCode: number
error?: string
}
creditsUsed: number
}
}
@@ -470,6 +471,7 @@ export interface SearchResponse extends ToolResponse {
error?: string
}
}>
creditsUsed: number
}
}
@@ -500,6 +502,7 @@ export interface MapResponse extends ToolResponse {
output: {
success: boolean
links: string[]
creditsUsed: number
}
}
@@ -508,6 +511,7 @@ export interface ExtractResponse extends ToolResponse {
jobId: string
success: boolean
data: Record<string, any>
creditsUsed?: number
}
}

View File

@@ -35,6 +35,20 @@ export const googleBooksVolumeDetailsTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
request: {
url: (params) => {
const url = new URL(`https://www.googleapis.com/books/v1/volumes/${params.volumeId.trim()}`)

View File

@@ -93,6 +93,20 @@ export const googleBooksVolumeSearchTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
request: {
url: (params) => {
const url = new URL('https://www.googleapis.com/books/v1/volumes')

View File

@@ -40,6 +40,20 @@ export const googleMapsAirQualityTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
return `https://airquality.googleapis.com/v1/currentConditions:lookup?key=${params.apiKey.trim()}`

View File

@@ -64,6 +64,20 @@ export const googleMapsDirectionsTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/directions/json')

View File

@@ -58,6 +58,20 @@ export const googleMapsDistanceMatrixTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/distancematrix/json')

View File

@@ -34,6 +34,20 @@ export const googleMapsElevationTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/elevation/json')

View File

@@ -35,6 +35,20 @@ export const googleMapsGeocodeTool: ToolConfig<GoogleMapsGeocodeParams, GoogleMa
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/geocode/json')

View File

@@ -66,6 +66,20 @@ export const googleMapsGeolocateTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
return `https://www.googleapis.com/geolocation/v1/geolocate?key=${params.apiKey.trim()}`

View File

@@ -40,6 +40,20 @@ export const googleMapsPlaceDetailsTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.017,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/place/details/json')

View File

@@ -58,6 +58,20 @@ export const googleMapsPlacesSearchTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.032,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/place/textsearch/json')

View File

@@ -41,6 +41,20 @@ export const googleMapsReverseGeocodeTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/geocode/json')

View File

@@ -35,6 +35,20 @@ export const googleMapsSnapToRoadsTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.01,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://roads.googleapis.com/v1/snapToRoads')

View File

@@ -34,6 +34,20 @@ export const googleMapsSpeedLimitsTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.02,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const hasPath = params.path && params.path.trim().length > 0

View File

@@ -46,6 +46,20 @@ export const googleMapsTimezoneTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://maps.googleapis.com/maps/api/timezone/json')

View File

@@ -46,6 +46,20 @@ export const googleMapsValidateAddressTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
return `https://addressvalidation.googleapis.com/v1:validateAddress?key=${params.apiKey.trim()}`

View File

@@ -46,6 +46,20 @@ export const analyzeTool: ToolConfig<GooglePagespeedAnalyzeParams, GooglePagespe
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'per_request',
cost: 0,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 20,
},
},
request: {
url: (params) => {
const url = new URL('https://www.googleapis.com/pagespeedonline/v5/runPagespeed')

View File

@@ -28,6 +28,29 @@ export const googleTranslateDetectTool: ToolConfig<
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'custom',
getCost: (params) => {
const text = params.text as string
if (!text) {
throw new Error('Missing text parameter, cannot determine detection cost')
}
// $20 per 1M characters — from https://cloud.google.com/translate/pricing
const charCount = text.length
const cost = charCount * 0.00002
return { cost, metadata: { characters: charCount } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://translation.googleapis.com/language/translate/v2/detect')

View File

@@ -42,6 +42,29 @@ export const googleTranslateTool: ToolConfig<GoogleTranslateParams, GoogleTransl
},
},
hosting: {
envKeyPrefix: 'GOOGLE_CLOUD_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'google_cloud',
pricing: {
type: 'custom',
getCost: (params) => {
const text = params.text as string
if (!text) {
throw new Error('Missing text parameter, cannot determine translation cost')
}
// $20 per 1M characters — from https://cloud.google.com/translate/pricing
const charCount = text.length
const cost = charCount * 0.00002
return { cost, metadata: { characters: charCount } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: (params) => {
const url = new URL('https://translation.googleapis.com/language/translate/v2')

View File

@@ -128,17 +128,7 @@ export const chatTool: ToolConfig<HuggingFaceChatParams, HuggingFaceChatResponse
output: {
content: data.choices?.[0]?.message?.content || '',
model: data.model || 'unknown',
usage: data.usage
? {
prompt_tokens: data.usage.prompt_tokens || 0,
completion_tokens: data.usage.completion_tokens || 0,
total_tokens: data.usage.total_tokens || 0,
}
: {
prompt_tokens: 0,
completion_tokens: 0,
total_tokens: 0,
},
usage: data.usage ?? { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
},
}
},

View File

@@ -101,7 +101,7 @@ async function injectHostedKeyIfNeeded(
retryAfterMs: acquireResult.retryAfterMs,
})
PlatformEvents.userThrottled({
PlatformEvents.hostedKeyUserThrottled({
toolId: tool.id,
reason: 'billing_actor_limit',
provider,
@@ -138,13 +138,21 @@ async function injectHostedKeyIfNeeded(
}
/**
* Check if an error is a rate limit (throttling) error
* Check if an error is a rate limit (throttling) or quota exhaustion error.
* Some providers (e.g. Perplexity) return 401/403 with "insufficient_quota"
* instead of the standard 429, so we also inspect the error message.
*/
function isRateLimitError(error: unknown): boolean {
if (error && typeof error === 'object') {
const status = (error as { status?: number }).status
// 429 = Too Many Requests, 503 = Service Unavailable (sometimes used for rate limiting)
if (status === 429 || status === 503) return true
if (status === 401 || status === 403) {
const message = ((error as { message?: string }).message || '').toLowerCase()
if (message.includes('quota') || message.includes('rate limit')) {
return true
}
}
}
return false
}
@@ -178,7 +186,7 @@ async function executeWithRetry<T>(
if (!isRateLimitError(error) || attempt === maxRetries) {
if (isRateLimitError(error) && attempt === maxRetries) {
PlatformEvents.userThrottled({
PlatformEvents.hostedKeyUserThrottled({
toolId,
reason: 'upstream_retries_exhausted',
userId: executionContext?.userId,
@@ -277,7 +285,7 @@ async function processHostedKeyCost(
if (!userId) return { cost, metadata }
const skipLog = !!ctx?.skipFixedUsageLog
const skipLog = !!ctx?.skipFixedUsageLog || !!tool.hosting?.skipFixedUsageLog
if (!skipLog) {
try {
await logFixedUsage({
@@ -377,6 +385,13 @@ async function applyHostedKeyCostToResult(
): Promise<void> {
await reportCustomDimensionUsage(tool, params, finalResult.output, executionContext, requestId)
if (tool.hosting?.skipFixedUsageLog) {
const ctx = params._context as Record<string, unknown> | undefined
if (ctx) {
ctx.skipFixedUsageLog = true
}
}
const { cost: hostedKeyCost, metadata } = await processHostedKeyCost(
tool,
params,

View File

@@ -159,25 +159,58 @@ export const readUrlTool: ToolConfig<ReadUrlParams, ReadUrlResponse> = {
},
},
hosting: {
envKeyPrefix: 'JINA_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'jina',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.tokensUsed == null) {
throw new Error('Jina read_url response missing tokensUsed field')
}
// Jina bills per output token — $0.20 per 1M tokens
// Source: https://cloud.jina.ai/pricing (token-based billing)
const tokens = output.tokensUsed as number
const cost = tokens * 0.0000002
return { cost, metadata: { tokensUsed: tokens } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 200,
},
},
transformResponse: async (response: Response) => {
let tokensUsed: number | undefined
const tokensHeader = response.headers.get('x-tokens')
if (tokensHeader) {
const parsed = Number.parseInt(tokensHeader, 10)
if (!Number.isNaN(parsed)) {
tokensUsed = parsed
}
}
const contentType = response.headers.get('content-type')
if (contentType?.includes('application/json')) {
const data = await response.json()
tokensUsed ??= data.data?.usage?.tokens ?? data.usage?.tokens
const content = data.data?.content || data.content || JSON.stringify(data)
tokensUsed ??= Math.ceil(content.length / 4)
return {
success: response.ok,
output: {
content: data.data?.content || data.content || JSON.stringify(data),
},
output: { content, tokensUsed },
}
}
const content = await response.text()
tokensUsed ??= Math.ceil(content.length / 4)
return {
success: response.ok,
output: {
content,
},
output: { content, tokensUsed },
}
},
@@ -186,5 +219,10 @@ export const readUrlTool: ToolConfig<ReadUrlParams, ReadUrlResponse> = {
type: 'string',
description: 'The extracted content from the URL, processed into clean, LLM-friendly text',
},
tokensUsed: {
type: 'number',
description: 'Number of Jina tokens consumed by this request',
optional: true,
},
},
}

View File

@@ -147,12 +147,65 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
},
},
hosting: {
envKeyPrefix: 'JINA_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'jina',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.tokensUsed == null) {
throw new Error('Jina search response missing tokensUsed field')
}
// Jina bills per output token — $0.20 per 1M tokens
// Search costs a fixed minimum of 10,000 tokens per request
// Source: https://cloud.jina.ai/pricing (token-based billing)
// x-tokens header is unreliable; falls back to content-length estimate (~4 chars/token)
const tokens = output.tokensUsed as number
const cost = tokens * 0.0000002
return { cost, metadata: { tokensUsed: tokens } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 40,
},
},
transformResponse: async (response: Response) => {
const data = await response.json()
// The API returns an array of results or a data object with results
let tokensUsed: number | undefined
const tokensHeader = response.headers.get('x-tokens')
if (tokensHeader) {
const parsed = Number.parseInt(tokensHeader, 10)
if (!Number.isNaN(parsed) && parsed > 0) {
tokensUsed = parsed
}
}
const results = Array.isArray(data) ? data : data.data || []
if (tokensUsed == null) {
let total = 0
for (const result of results) {
if (result.usage?.tokens) {
total += result.usage.tokens
}
}
if (total > 0) {
tokensUsed = total
}
}
if (tokensUsed == null) {
let totalChars = 0
for (const result of results) {
totalChars += (result.content?.length ?? 0) + (result.title?.length ?? 0)
}
tokensUsed = Math.max(Math.ceil(totalChars / 4), 10000)
}
return {
success: response.ok,
output: {
@@ -162,6 +215,7 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
url: result.url || '',
content: result.content || '',
})),
tokensUsed,
},
}
},
@@ -176,5 +230,10 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
properties: JINA_SEARCH_RESULT_OUTPUT_PROPERTIES,
},
},
tokensUsed: {
type: 'number',
description: 'Number of Jina tokens consumed by this request',
optional: true,
},
},
}

View File

@@ -82,6 +82,27 @@ export const searchTool: ToolConfig<LinkupSearchParams, LinkupSearchToolResponse
},
},
hosting: {
envKeyPrefix: 'LINKUP_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'linkup',
pricing: {
type: 'custom',
getCost: (params) => {
// Linkup pricing (https://docs.linkup.so/pages/documentation/development/pricing):
// Standard: €0.005/call ≈ $0.006
// Deep: €0.05/call ≈ $0.055
const depth = params.depth as string
const cost = depth === 'deep' ? 0.055 : 0.006
return { cost, metadata: { depth } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 60,
},
},
request: {
url: 'https://api.linkup.so/v1/search',
method: 'POST',

View File

@@ -10,6 +10,31 @@ import type { ToolConfig } from '@/tools/types'
const logger = createLogger('MistralParserTool')
const MISTRAL_OCR_HOSTING = {
envKeyPrefix: 'MISTRAL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'mistral' as const,
pricing: {
type: 'custom' as const,
getCost: (_params: Record<string, unknown>, output: Record<string, unknown>) => {
// Mistral OCR: $2 per 1,000 pages ($0.002/page) for real-time API
// Batch inference is $1/1K pages but we call the real-time endpoint
// https://mistral.ai/pricing#api
const usageInfo = output.usage_info as { pages_processed?: number } | undefined
if (usageInfo?.pages_processed == null) {
throw new Error('Mistral OCR response missing pages_processed in usage_info')
}
const pagesProcessed = usageInfo.pages_processed
const cost = pagesProcessed * 0.002
return { cost, metadata: { pagesProcessed } }
},
},
rateLimit: {
mode: 'per_request' as const,
requestsPerMinute: 60,
},
}
export const mistralParserTool: ToolConfig<MistralParserInput, MistralParserOutput> = {
id: 'mistral_parser',
name: 'Mistral PDF Parser',
@@ -492,6 +517,7 @@ export const mistralParserV3Tool: ToolConfig<MistralParserV2Input, MistralParser
...mistralParserV2Tool,
id: 'mistral_parser_v3',
version: '3.0.0',
hosting: MISTRAL_OCR_HOSTING,
params: {
file: {
type: 'file',

View File

@@ -1,4 +1,5 @@
import { createLogger } from '@sim/logger'
import { PlatformEvents } from '@/lib/core/telemetry'
import type { ParallelDeepResearchParams } from '@/tools/parallel/types'
import type { ToolConfig, ToolResponse } from '@/tools/types'
@@ -11,6 +12,49 @@ export const deepResearchTool: ToolConfig<ParallelDeepResearchParams, ToolRespon
'Conduct comprehensive deep research across the web using Parallel AI. Synthesizes information from multiple sources with citations. Can take up to 15 minutes to complete.',
version: '1.0.0',
hosting: {
envKeyPrefix: 'PARALLEL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'parallel_ai',
pricing: {
type: 'custom',
getCost: (params, _output) => {
// Parallel Task API: cost varies by processor
// https://docs.parallel.ai/resources/pricing
const processorCosts: Record<string, number> = {
lite: 0.005,
base: 0.01,
core: 0.025,
core2x: 0.05,
pro: 0.1,
ultra: 0.3,
ultra2x: 0.6,
ultra4x: 1.2,
ultra8x: 2.4,
}
const processor = (params.processor as string) || 'base'
const DEFAULT_PROCESSOR_COST = processorCosts.base
const knownCost = processorCosts[processor]
if (knownCost == null) {
logger.warn(
`Unknown Parallel processor "${processor}", using default processor cost $${DEFAULT_PROCESSOR_COST}`
)
PlatformEvents.hostedKeyUnknownModelCost({
toolId: 'parallel_deep_research',
modelName: processor,
defaultCost: DEFAULT_PROCESSOR_COST,
})
}
const cost = knownCost ?? DEFAULT_PROCESSOR_COST
return { cost, metadata: { processor, defaultProcessorCost: DEFAULT_PROCESSOR_COST } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 10,
},
},
params: {
input: {
type: 'string',

View File

@@ -8,6 +8,29 @@ export const extractTool: ToolConfig<ParallelExtractParams, ToolResponse> = {
'Extract targeted information from specific URLs using Parallel AI. Processes provided URLs to pull relevant content based on your objective.',
version: '1.0.0',
hosting: {
envKeyPrefix: 'PARALLEL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'parallel_ai',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (!Array.isArray(output.results)) {
throw new Error('Parallel extract response missing results array')
}
// Parallel Extract: $1 per 1,000 URLs = $0.001 per URL
// https://docs.parallel.ai/resources/pricing
const urlCount = output.results.length
const cost = urlCount * 0.001
return { cost, metadata: { urlCount } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
urls: {
type: 'string',

View File

@@ -8,6 +8,30 @@ export const searchTool: ToolConfig<ParallelSearchParams, ToolResponse> = {
'Search the web using Parallel AI. Provides comprehensive search results with intelligent processing and content extraction.',
version: '1.0.0',
hosting: {
envKeyPrefix: 'PARALLEL_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'parallel_ai',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (!Array.isArray(output.results)) {
throw new Error('Parallel search response missing results array')
}
// Parallel Search: $0.005 base (includes ≤10 results), +$0.001 per result beyond 10
// https://docs.parallel.ai/resources/pricing
const resultCount = output.results.length
const additionalResults = Math.max(0, resultCount - 10)
const cost = 0.005 + additionalResults * 0.001
return { cost, metadata: { resultCount, additionalResults } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
objective: {
type: 'string',

View File

@@ -1,6 +1,42 @@
import type { PerplexityChatParams, PerplexityChatResponse } from '@/tools/perplexity/types'
import type { ToolConfig } from '@/tools/types'
/**
* Per-token rates by model from https://docs.perplexity.ai/guides/pricing
* Per-request fees assume Low context size (the API default).
* Deep Research has additional billing dimensions: citation tokens, search queries, reasoning tokens.
*/
const MODEL_PRICING: Record<
string,
{
inputPerM: number
outputPerM: number
requestPer1K: number
citationPerM?: number
searchQueriesPer1K?: number
reasoningPerM?: number
}
> = {
'sonar-deep-research': {
inputPerM: 2,
outputPerM: 8,
requestPer1K: 0,
citationPerM: 2,
searchQueriesPer1K: 5,
reasoningPerM: 3,
},
'sonar-reasoning-pro': { inputPerM: 2, outputPerM: 8, requestPer1K: 6 },
'sonar-pro': { inputPerM: 3, outputPerM: 15, requestPer1K: 6 },
sonar: { inputPerM: 1, outputPerM: 1, requestPer1K: 5 },
}
function getModelPricing(model: string) {
for (const [key, pricing] of Object.entries(MODEL_PRICING)) {
if (model.includes(key)) return pricing
}
return MODEL_PRICING.sonar
}
export const chatTool: ToolConfig<PerplexityChatParams, PerplexityChatResponse> = {
id: 'perplexity_chat',
name: 'Perplexity Chat',
@@ -48,6 +84,77 @@ export const chatTool: ToolConfig<PerplexityChatParams, PerplexityChatResponse>
},
},
hosting: {
envKeyPrefix: 'PERPLEXITY_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'perplexity',
pricing: {
type: 'custom',
getCost: (params, output) => {
const usage = output.usage as
| {
prompt_tokens?: number
completion_tokens?: number
citation_tokens?: number
num_search_queries?: number
reasoning_tokens?: number
}
| undefined
if (!usage || usage.prompt_tokens == null || usage.completion_tokens == null) {
throw new Error('Perplexity chat response missing token usage data')
}
const model = ((output.model as string) || params.model) as string
const pricing = getModelPricing(model)
const inputTokens = usage.prompt_tokens
const outputTokens = usage.completion_tokens
let tokenCost =
(inputTokens * pricing.inputPerM) / 1_000_000 +
(outputTokens * pricing.outputPerM) / 1_000_000
const requestFee = pricing.requestPer1K / 1000
let citationCost = 0
let searchQueryCost = 0
let reasoningCost = 0
if (pricing.citationPerM && usage.citation_tokens) {
citationCost = (usage.citation_tokens * pricing.citationPerM) / 1_000_000
}
if (pricing.searchQueriesPer1K && usage.num_search_queries) {
searchQueryCost = (usage.num_search_queries * pricing.searchQueriesPer1K) / 1000
}
if (pricing.reasoningPerM && usage.reasoning_tokens) {
reasoningCost = (usage.reasoning_tokens * pricing.reasoningPerM) / 1_000_000
}
const cost = tokenCost + requestFee + citationCost + searchQueryCost + reasoningCost
return {
cost,
metadata: {
model,
inputTokens,
outputTokens,
tokenCost,
requestFee,
citationTokens: usage.citation_tokens,
citationCost,
searchQueries: usage.num_search_queries,
searchQueryCost,
reasoningTokens: usage.reasoning_tokens,
reasoningCost,
},
}
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 20,
},
skipFixedUsageLog: true,
},
request: {
method: 'POST',
url: () => 'https://api.perplexity.ai/chat/completions',
@@ -101,6 +208,15 @@ export const chatTool: ToolConfig<PerplexityChatParams, PerplexityChatResponse>
prompt_tokens: data.usage.prompt_tokens,
completion_tokens: data.usage.completion_tokens,
total_tokens: data.usage.total_tokens,
...(data.usage.citation_tokens != null && {
citation_tokens: data.usage.citation_tokens,
}),
...(data.usage.num_search_queries != null && {
num_search_queries: data.usage.num_search_queries,
}),
...(data.usage.reasoning_tokens != null && {
reasoning_tokens: data.usage.reasoning_tokens,
}),
},
},
}

View File

@@ -66,6 +66,20 @@ export const searchTool: ToolConfig<PerplexitySearchParams, PerplexitySearchResp
},
},
hosting: {
envKeyPrefix: 'PERPLEXITY_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'perplexity',
pricing: {
type: 'per_request',
cost: 0.005,
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
request: {
method: 'POST',
url: () => 'https://api.perplexity.ai/search',

View File

@@ -22,6 +22,9 @@ export interface PerplexityChatResponse extends ToolResponse {
prompt_tokens: number
completion_tokens: number
total_tokens: number
citation_tokens?: number
num_search_queries?: number
reasoning_tokens?: number
}
}
}

View File

@@ -49,6 +49,28 @@ export const searchTool: ToolConfig<SearchParams, SearchResponse> = {
},
},
hosting: {
envKeyPrefix: 'SERPER_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'serper',
pricing: {
type: 'custom',
getCost: (params, output) => {
if (!Array.isArray(output.searchResults)) {
throw new Error('Serper response missing searchResults, cannot determine cost')
}
const num = Number(params.num) || 10
const credits = num > 10 ? 2 : 1
const cost = credits * 0.001
return { cost, metadata: { num, credits } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
request: {
url: (params) => `https://google.serper.dev/${params.type || 'search'}`,
method: 'POST',

View File

@@ -1,3 +1,4 @@
import { FLASH_TURBO_MODELS } from '@/tools/elevenlabs/constants'
import type { ElevenLabsTtsUnifiedParams, TtsBlockResponse } from '@/tools/tts/types'
import type { ToolConfig } from '@/tools/types'
@@ -7,6 +8,32 @@ export const elevenLabsTtsUnifiedTool: ToolConfig<ElevenLabsTtsUnifiedParams, Tt
description: 'Convert text to speech using ElevenLabs voices',
version: '1.0.0',
hosting: {
envKeyPrefix: 'ELEVENLABS_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'elevenlabs',
pricing: {
type: 'custom',
getCost: (params, _output) => {
const text = params.text as string | undefined
if (!text) {
throw new Error('Missing text parameter, cannot determine character cost')
}
const characterCount = text.length
const modelId = (params.modelId as string) || 'eleven_turbo_v2_5'
// Flash/Turbo: $0.08/1K chars, Standard/Multilingual/v3: $0.18/1K chars
// Scale tier additional character rates — https://elevenlabs.io/pricing/api
const costPer1KChars = FLASH_TURBO_MODELS.has(modelId) ? 0.08 : 0.18
const cost = (characterCount / 1000) * costPer1KChars
return { cost, metadata: { characterCount, modelId, costPer1KChars } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 30,
},
},
params: {
text: {
type: 'string',

View File

@@ -1,7 +1,22 @@
import type { HostedKeyRateLimitConfig } from '@/lib/core/rate-limiter'
import type { OAuthService } from '@/lib/oauth'
export type BYOKProviderId = 'openai' | 'anthropic' | 'google' | 'mistral' | 'exa'
export type BYOKProviderId =
| 'openai'
| 'anthropic'
| 'google'
| 'mistral'
| 'exa'
| 'browser_use'
| 'serper'
| 'firecrawl'
| 'jina'
| 'elevenlabs'
| 'perplexity'
| 'google_cloud'
| 'linkup'
| 'brandfetch'
| 'parallel_ai'
export type HttpMethod = 'GET' | 'POST' | 'PUT' | 'DELETE' | 'PATCH' | 'HEAD'
@@ -299,4 +314,6 @@ export interface ToolHostingConfig<P = Record<string, unknown>> {
pricing: ToolHostingPricing<P>
/** Hosted key rate limit configuration (required for hosted key distribution) */
rateLimit: HostedKeyRateLimitConfig
/** When true, skip the fixed usage log entry (useful for tools that log custom dimensions instead) */
skipFixedUsageLog?: boolean
}