Compare commits

...

14 Commits

Author SHA1 Message Date
zach
4a532aa58c cleanup(kernel): undo some kernel changes 2025-06-27 13:26:16 -07:00
zach
3300fed506 chore: fmt 2025-06-26 13:35:48 -07:00
zach
b026d30865 chore: clippy 2025-06-26 13:34:04 -07:00
zach
e8d0397754 test(kernel): add block re-use quickcheck test 2025-06-26 13:26:50 -07:00
zach
39685a9038 cleanup(kernel): improve re-use of freed blocks 2025-06-25 10:32:29 -07:00
Nutomic
d1ba15484e Remove key param for PluginPool (#859)
As mentioned in
https://github.com/extism/extism/pull/696/files#r1993086470, the key
parameter doesnt seem very useful. You can achieve the same effect by
doing `HashMap<Key, PluginPool>` in the client code.

---------

Co-authored-by: zach <zach@dylibso.com>
2025-06-18 12:28:39 -07:00
nu-wa
dedd81d90f docs: add more information about wasmtime caching (#863)
Hello,

I encountered an error when trying to enable caching in my project, I
could not compile any Plugins due to a bad configuration file used by
the wasmtime crate (`failed to parse config file`). I managed to find
understand the error after seeing [some changes made by
wasmtime](https://github.com/bytecodealliance/wasmtime/pull/10859) and
its documentation. I was simply missing the `enabled` key from the
configuration, that has been removed from `wasmtime` (and its [cache
sister
crate](https://github.com/bytecodealliance/wasmtime/tree/main/crates/cache)).

So I added more information in the README regarding this specific issue,
and some extra behaviour that I noticed would happen regarding caching
configuration.

Hopefully this is helpful!
2025-06-18 11:09:23 -07:00
zach
2732ca198d feat: add Pool type for pooling plugin instances (#696) 2025-06-06 10:14:22 -07:00
Muhammad Azeez
30b4a7d2d3 fix: use gh release download instead of downloading from github action artifacts in dotnet workflow (#857)
It seems like at some point we changed the `release.yml` workflow to
create multiple artifacts instead of one `release-artifacts` tarball. I
changed the .NET Nuget workflow to be more like Python
2025-05-29 22:09:52 +03:00
zach
b6e5684461 v1.11.0 2025-05-27 15:37:19 -07:00
zach
36e05e8668 chore: add ToBytes/FromBytes implementations for bool (#855)
Fixes https://github.com/extism/extism/issues/838
2025-05-27 15:35:22 -07:00
dependabot[bot]
9415aaa08a chore(deps): Update cbindgen requirement from 0.28 to 0.29 (#854)
Updates the requirements on
[cbindgen](https://github.com/mozilla/cbindgen) to permit the latest
version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/mozilla/cbindgen/releases">cbindgen's
releases</a>.</em></p>
<blockquote>
<h1>0.29.0</h1>
<ul>
<li>Support no-export annotation for statics and functions.</li>
<li>Fixed conditional fields of constexpr literal structs</li>
<li>Add rename rule for generated associated constant</li>
<li>Upgrade heck to 0.5</li>
<li>Add support for an optional nullable attribute</li>
<li>docs.md: Fix deprecated_with_note and deprecated_variant_with_note
being spelled as 'notes'</li>
<li>Fix generic with &quot;void&quot; default</li>
<li>Fixed error generation of structures using the keyword as inside
arrays</li>
<li>Added test for unsafe(no_mangle) attribute</li>
<li>Fixed handling of trait methods containing the unsafe attribute</li>
<li>Rename -Zparse-only</li>
</ul>
<h1>0.28.0</h1>
<ul>
<li>Parse unsafe attributes in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1020">mozilla/cbindgen#1020</a></li>
<li>Fix local override of enum prefix-with-name by jsgf in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1006">mozilla/cbindgen#1006</a></li>
<li>Add rename-all=prefix in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1021">mozilla/cbindgen#1021</a></li>
<li>ir: add support for UnsafeCell and SyncUnsafeCell by alekitto in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1003">mozilla/cbindgen#1003</a></li>
<li>Implement mangling for arrays in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1022">mozilla/cbindgen#1022</a></li>
<li>Fix: Ignore <code>CARGO_BUILD_TARGET</code> in tests by bryango in
<a
href="https://redirect.github.com/mozilla/cbindgen/pull/1010">mozilla/cbindgen#1010</a></li>
<li>Newline for each field for constexpr field constants by youknowone
in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/988">mozilla/cbindgen#988</a></li>
<li>Fix clippy warnings by youknowone in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1026">mozilla/cbindgen#1026</a></li>
<li>Add aarch64/arm64 to CI by NickeZ in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1036">mozilla/cbindgen#1036</a></li>
<li>Add <code>unstable_ir</code> feature flag that makes the ir pub by
heesooy in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/1011">mozilla/cbindgen#1011</a></li>
<li>Support generated a symbols file by TheElectronWill in <a
href="https://redirect.github.com/mozilla/cbindgen/pull/916">mozilla/cbindgen#916</a></li>
</ul>
<h1>0.27.0</h1>
<ul>
<li>Revert: The <code>Config</code> struct now has a private
member.</li>
<li>Allow users to specify a crate version for bindings generation (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/901">#901</a>).</li>
<li>Update MSRV to 1.74 (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/912">#912</a>,
<a
href="https://redirect.github.com/mozilla/cbindgen/issues/987">#987</a>).</li>
<li>Support #[deprecated] on enum variants (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/933">#933</a>).</li>
<li>Support integrating the package_version information in a header file
comment (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/939">#939</a>).</li>
<li>Add a language backend (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/942">#942</a>).</li>
<li>Support generics with defaulted args (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/959">#959</a>).</li>
<li>Add <code>VaList</code> compatibility (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/970">#970</a>).</li>
</ul>
<h1>0.26.0</h1>
<ul>
<li>Fix swapping of <code>&gt;&gt;=</code> and <code>&lt;&lt;=</code> in
constants.</li>
<li>Add support for #[deprecated] (<a
href="https://redirect.github.com/mozilla/cbindgen/issues/860">#860</a>).</li>
<li>Built-in support for bitflags 2.0.</li>
<li>Support for &quot;C-unwind&quot; ABI.</li>
<li>Generate bindings for non-public extern items if they are
#[no_mangle].</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/mozilla/cbindgen/blob/master/CHANGES">cbindgen's
changelog</a>.</em></p>
<blockquote>
<h1>0.29.0</h1>
<pre><code>  * Support no-export annotation for statics and functions.
  * Fixed conditional fields of constexpr literal structs
  * Add rename rule for generated associated constant
  * Upgrade heck to 0.5
  * Add support for an optional nullable attribute
* docs.md: Fix deprecated_with_note and deprecated_variant_with_note
being spelled as 'notes'
  * Fix generic with &quot;void&quot; default
* Fixed error generation of structures using the keyword as inside
arrays
  * Added test for unsafe(no_mangle) attribute
  * Fixed handling of trait methods containing the unsafe attribute
  * Rename -Zparse-only
</code></pre>
<h1>0.28.0</h1>
<pre><code> * Parse unsafe attributes in
https://github.com/mozilla/cbindgen/pull/1020
* Fix local override of enum prefix-with-name by jsgf in
https://github.com/mozilla/cbindgen/pull/1006
* Add rename-all=prefix in https://github.com/mozilla/cbindgen/pull/1021
* ir: add support for UnsafeCell and SyncUnsafeCell by alekitto in
https://github.com/mozilla/cbindgen/pull/1003
* Implement mangling for arrays in
https://github.com/mozilla/cbindgen/pull/1022
* Fix: Ignore `CARGO_BUILD_TARGET` in tests by bryango in
https://github.com/mozilla/cbindgen/pull/1010
* Newline for each field for constexpr field constants by youknowone in
https://github.com/mozilla/cbindgen/pull/988
* Fix clippy warnings by youknowone in
https://github.com/mozilla/cbindgen/pull/1026
* Add aarch64/arm64 to CI by NickeZ in
https://github.com/mozilla/cbindgen/pull/1036
* Add `unstable_ir` feature flag that makes the ir pub by heesooy in
https://github.com/mozilla/cbindgen/pull/1011
* Support generated a symbols file by TheElectronWill in
https://github.com/mozilla/cbindgen/pull/916
</code></pre>
<h1>0.27.0</h1>
<pre><code>  * Revert: The `Config` struct now has a private member.
* Allow users to specify a crate version for bindings generation
([#901](https://github.com/mozilla/cbindgen/issues/901)).
* Update MSRV to 1.74
([#912](https://github.com/mozilla/cbindgen/issues/912),
[#987](https://github.com/mozilla/cbindgen/issues/987)).
* Support #[deprecated] on enum variants
([#933](https://github.com/mozilla/cbindgen/issues/933)).
* Support integrating the package_version information in a header file
comment ([#939](https://github.com/mozilla/cbindgen/issues/939)).
* Add a language backend
([#942](https://github.com/mozilla/cbindgen/issues/942)).
* Support generics with defaulted args
([#959](https://github.com/mozilla/cbindgen/issues/959)).
* Add `VaList` compatibility
([#970](https://github.com/mozilla/cbindgen/issues/970)).
</code></pre>
<h1>0.26.0</h1>
<pre><code>  * Fix swapping of `&gt;&gt;=` and `&lt;&lt;=` in constants.
* Add support for #[deprecated]
([#860](https://github.com/mozilla/cbindgen/issues/860)).
  * Built-in support for bitflags 2.0.
  * Support for &quot;C-unwind&quot; ABI.
* Generate bindings for non-public extern items if they are
#[no_mangle].
</code></pre>
<h2>0.25.0</h2>
<pre><code>  * Re-release of yanked 0.24.6 as a major release
</code></pre>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="802154245e"><code>8021542</code></a>
Release 0.29.0</li>
<li><a
href="f3a0ecd2a9"><code>f3a0ecd</code></a>
Support no-export annotation for statics and functions.</li>
<li><a
href="14fa2d0669"><code>14fa2d0</code></a>
conditional fields of constexpr literal structs</li>
<li><a
href="021f3f3a42"><code>021f3f3</code></a>
Add LiteralStructField</li>
<li><a
href="4b2396dabf"><code>4b2396d</code></a>
Github action: Add aarch64 to deploy</li>
<li><a
href="2320ac4296"><code>2320ac4</code></a>
Add rename rule for generated associated constant</li>
<li><a
href="5ac9dc046b"><code>5ac9dc0</code></a>
Upgrade heck to 0.5</li>
<li><a
href="9f9da30c60"><code>9f9da30</code></a>
Add support for an optional nullable attribute</li>
<li><a
href="36b9f0df42"><code>36b9f0d</code></a>
docs.md: Fix deprecated_with_note and deprecated_variant_with_note being
spel...</li>
<li><a
href="0328601dc1"><code>0328601</code></a>
Fix generic with &quot;void&quot; default</li>
<li>Additional commits viewable in <a
href="https://github.com/mozilla/cbindgen/compare/0.28.0...0.29.0">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-27 12:36:33 -07:00
dependabot[bot]
168459be0f chore(deps): Update criterion requirement from 0.5.1 to 0.6.0 (#852)
Updates the requirements on
[criterion](https://github.com/bheisler/criterion.rs) to permit the
latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md">criterion's
changelog</a>.</em></p>
<blockquote>
<h2>[0.6.0] - 2025-05-17</h2>
<h3>Changed</h3>
<ul>
<li>MSRV bumped to 1.80</li>
<li>The <code>real_blackbox</code> feature no longer has any impact.
Criterion always uses <code>std::hint::black_box()</code> now.
Users of <code>criterion::black_box()</code> should switch to
<code>std::hint::black_box()</code>.</li>
<li><code>clap</code> dependency unpinned.</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>gnuplot version is now correctly detected when using certain Windows
binaries/configurations that used to fail</li>
</ul>
<h3>Added</h3>
<ul>
<li>Async benchmarking with Tokio may be done via a
<code>tokio::runtime::Handle</code>, not only a
<code>tokio::runtime::Runtime</code></li>
</ul>
<h2>[0.5.1] - 2023-05-26</h2>
<h3>Fixed</h3>
<ul>
<li>Quick mode (--quick) no longer crashes with measured times over 5
seconds when --noplot is not active</li>
</ul>
<h2>[0.5.0] - 2023-05-23</h2>
<h3>Changed</h3>
<ul>
<li>Replaced lazy_static dependency with once_cell</li>
<li>Improved documentation of the <code>html_reports</code> feature</li>
<li>Replaced atty dependency with is-terminal</li>
<li>MSRV bumped to 1.64</li>
<li>Upgraded clap dependency to v4</li>
<li>Upgraded tempfile dependency to v3.5.0</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Quick mode (<code>--quick</code>) no longer outputs 1ms for measured
times over 5 seconds</li>
<li>Documentation updates</li>
</ul>
<h2>[0.4.0] - 2022-09-10</h2>
<h3>Removed</h3>
<ul>
<li>The <code>Criterion::can_plot</code> function has been removed.</li>
<li>The <code>Criterion::bench_function_over_inputs</code> function has
been removed.</li>
<li>The <code>Criterion::bench_functions</code> function has been
removed.</li>
<li>The <code>Criterion::bench</code> function has been removed.</li>
</ul>
<h3>Changed</h3>
<ul>
<li>HTML report hidden behind non-default feature flag:
'html_reports'</li>
<li>Standalone support (ie without cargo-criterion) feature flag:
'cargo_bench_support'</li>
<li>MSRV bumped to 1.57</li>
<li><code>rayon</code> and <code>plotters</code> are optional (and
default) dependencies.</li>
<li>Status messages ('warming up', 'analyzing', etc) are printed to
stderr, benchmark results are printed to stdout.</li>
<li>Accept subsecond durations for <code>--warm-up-time</code>,
<code>--measurement-time</code> and <code>--profile-time</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="43bf90a64b"><code>43bf90a</code></a>
release version 0.6.0 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/860">#860</a>)</li>
<li><a
href="92696e45c5"><code>92696e4</code></a>
deps: unpin clap (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/858">#858</a>)</li>
<li><a
href="5756a5d526"><code>5756a5d</code></a>
chore: bump MSRV to 1.80 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/859">#859</a>)</li>
<li><a
href="9d887c0145"><code>9d887c0</code></a>
Fixed typo in faq.md (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/852">#852</a>)</li>
<li><a
href="59b791a587"><code>59b791a</code></a>
ci: test against MSRV and 1.87 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/857">#857</a>)</li>
<li><a
href="ace1cc93ef"><code>ace1cc9</code></a>
Fix warnings from clippy (rust 1.87.0) (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/856">#856</a>)</li>
<li><a
href="7afab6ef7c"><code>7afab6e</code></a>
Commit Cargo.lock to make CI and local debugging more stable (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/855">#855</a>)</li>
<li><a
href="260e2f1c78"><code>260e2f1</code></a>
Update Cargo features' comments (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/825">#825</a>)</li>
<li><a
href="58130ff859"><code>58130ff</code></a>
Update to async-std v1.13 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/812">#812</a>)</li>
<li><a
href="d2e705b855"><code>d2e705b</code></a>
Add rust-version 1.70 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/813">#813</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/bheisler/criterion.rs/compare/0.5.1...0.6.0">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: zach <zach@dylibso.com>
2025-05-19 10:10:58 -07:00
AlessandroRuggiero
1ea498b5d9 fix: better suggestion when encoding is not implemented (#843)
Here is a code snippet that demonstrates the issue:
```rust
use extism_pdk::{encoding, plugin_fn, FnResult, FromBytes, Json, ToBytes};
use serde::{Serialize, Deserialize};

#[derive(Deserialize,FromBytes)]
pub struct Input {
    value:String
}

#[derive(Serialize, ToBytes)]
pub struct Output {
    value: f64,
}

#[plugin_fn]
pub fn convert(in_value: Input) -> FnResult<Output> {
    Ok(Output { value: in_value.value.parse().expect("Conversion failed") })
}
```

Because the structs don't derive `encoding` the rust-analyzer gives this
suggestion:
```
encoding needs to be specified

  = try: `#[encoding(ToJson)]`
```

I don't think this suggestion is correct because i could not find
`ToJson` anywhere. Adding `#[encoding(Json)]` solves the issue (as
described in the docs).
The final code would look as follows:
```rust
use extism_pdk::{encoding, plugin_fn, FnResult, FromBytes, Json, ToBytes};
use serde::{Serialize, Deserialize};

#[derive(Deserialize,FromBytes)]
#[encoding(Json)]
pub struct Input {
    value:String
}

#[derive(Serialize, ToBytes)]
#[encoding(Json)]
pub struct Output {
    value: f64,
}

#[plugin_fn]
pub fn convert(in_value: Input) -> FnResult<Output> {
    Ok(Output { value: in_value.value.parse().expect("Conversion failed") })
}
```

And compiles perfectly.

I think the suggestion given by the rust-analyzer is incorrect so i am
proposing a fix for this.

Since I'm new to the project, any feedback is appreciated.
Thanks!
_Alessandro Ruggiero_
2025-03-31 08:15:23 -07:00
28 changed files with 389 additions and 53 deletions

View File

@@ -9,12 +9,8 @@ updates:
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
groups:
# This is the name of your group, it will be used in PR titles and branch names
wasmtime-deps:
# A pattern can be...
patterns:
- "wasmtime"
- "wasi-common"
- "wiggle"
ignore:
- dependency-name: "wasmtime"
- dependency-name: "wasi-common"
- dependency-name: "wiggle"

View File

@@ -20,10 +20,13 @@ jobs:
uses: actions/setup-dotnet@v3.0.3
with:
dotnet-version: 7.x
- uses: dawidd6/action-download-artifact@v6
with:
workflow: release.yml
name: release-artifacts
- name: download release
run: |
tag='${{ github.ref }}'
tag="${tag/refs\/tags\//}"
gh release download "$tag" -p 'libextism-*.tar.gz'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Extract Archive
run: |
extract_archive() {

View File

@@ -34,11 +34,11 @@ fn extract_encoding(attrs: &[Attribute]) -> Result<Path> {
.iter()
.filter(|attr| attr.path().is_ident("encoding"))
.collect();
ensure!(!encodings.is_empty(), "encoding needs to be specified"; try = "`#[encoding(ToJson)]`");
ensure!(!encodings.is_empty(), "encoding needs to be specified"; try = "`#[encoding(Json)]`");
ensure!(encodings.len() < 2, encodings[1], "only one encoding can be specified"; try = "remove `{}`", encodings[1].to_token_stream());
Ok(encodings[0].parse_args().map_err(
|e| error_message!(e.span(), "{e}"; note= "expects a path"; try = "`#[encoding(ToJson)]`"),
|e| error_message!(e.span(), "{e}"; note= "expects a path"; try = "`#[encoding(Json)]`"),
)?)
}

View File

@@ -1,6 +1,6 @@
error: encoding needs to be specified
= try: `#[encoding(ToJson)]`
= try: `#[encoding(Json)]`
--> tests/ui/invalid-encoding.rs:3:10
|
3 | #[derive(ToBytes)]
@@ -11,7 +11,7 @@ error: encoding needs to be specified
error: expected attribute arguments in parentheses: #[encoding(...)]
= note: expects a path
= try: `#[encoding(ToJson)]`
= try: `#[encoding(Json)]`
--> tests/ui/invalid-encoding.rs:7:3
|
7 | #[encoding]
@@ -20,7 +20,7 @@ error: expected attribute arguments in parentheses: #[encoding(...)]
error: expected parentheses: #[encoding(...)]
= note: expects a path
= try: `#[encoding(ToJson)]`
= try: `#[encoding(Json)]`
--> tests/ui/invalid-encoding.rs:11:12
|
11 | #[encoding = "string"]
@@ -29,7 +29,7 @@ error: expected parentheses: #[encoding(...)]
error: unexpected token
= note: expects a path
= try: `#[encoding(ToJson)]`
= try: `#[encoding(Json)]`
--> tests/ui/invalid-encoding.rs:15:21
|
15 | #[encoding(something, else)]

View File

@@ -141,6 +141,16 @@ impl FromBytesOwned for u32 {
}
}
impl FromBytesOwned for bool {
fn from_bytes_owned(data: &[u8]) -> Result<Self, Error> {
if let Some(x) = data.first() {
Ok(*x != 0)
} else {
Err(Error::msg("Expected one byte to read boolean value"))
}
}
}
impl FromBytesOwned for () {
fn from_bytes_owned(_: &[u8]) -> Result<Self, Error> {
Ok(())

View File

@@ -61,6 +61,17 @@ fn rountrip_option() {
assert_eq!(y.unwrap().0, z.unwrap().0);
}
#[test]
fn check_bool() {
// `None` case
let a = true.to_bytes().unwrap();
let b = false.to_bytes().unwrap();
assert_ne!(a, b);
assert_eq!(a, [1]);
assert_eq!(b, [0]);
}
#[cfg(all(feature = "raw", target_endian = "little"))]
mod raw_tests {
use crate::*;

View File

@@ -144,6 +144,14 @@ impl ToBytes<'_> for u32 {
}
}
impl ToBytes<'_> for bool {
type Bytes = [u8; 1];
fn to_bytes(&self) -> Result<Self::Bytes, Error> {
Ok([*self as u8])
}
}
impl<'a, T: ToBytes<'a>> ToBytes<'a> for &'a T {
type Bytes = T::Bytes;

View File

@@ -34,10 +34,10 @@ register-filesystem = [] # enables wasm to be loaded from disk
http = ["ureq"] # enables extism_http_request
[build-dependencies]
cbindgen = { version = "0.28", default-features = false }
cbindgen = { version = "0.29", default-features = false }
[dev-dependencies]
criterion = "0.5.1"
criterion = "0.6.0"
quickcheck = "1"
rand = "0.9.0"

View File

@@ -24,7 +24,7 @@ There are a few environment variables that can be used for debugging purposes:
- `EXTISM_COREDUMP=extism.core`: write [coredump](https://github.com/WebAssembly/tool-conventions/blob/main/Coredump.md) to a file when a WebAssembly function traps
- `EXTISM_DEBUG=1`: generate debug information
- `EXTISM_PROFILE=perf|jitdump|vtune`: enable Wasmtime profiling
- `EXTISM_CACHE_CONFIG=path/to/config.toml`: enable Wasmtime cache, see [the docs](https://docs.wasmtime.dev/cli-cache.html) for details about configuration. Setting this to an empty string will disable caching.
- `EXTISM_CACHE_CONFIG=path/to/config.toml`: enable Wasmtime cache, details [here](#wasmtime-caching)
> *Note*: The debug and coredump info will only be written if the plug-in has an error.
@@ -229,3 +229,42 @@ Inside your host application, the rust-sdk emits these as [tracing](https://gith
tracing_subscriber::fmt::init();
```
### Wasmtime Caching
To enable or disable caching for plugin compilation, you need to provide a configuration file that will be used by the [wasmtime crate](https://github.com/bytecodealliance/wasmtime).
For more information and values that can be used for configuring caching, take a look at [the docs](https://docs.wasmtime.dev/cli-cache.html).
> *Note*: As of now extism uses wasmtime [`version = ">= 27.0.0, < 31.0.0"`](https://github.com/extism/extism/blob/v1.11.1/runtime/Cargo.toml#L12), but the `enabled` key requirement [was removed](https://github.com/bytecodealliance/wasmtime/pull/10859) from `wasmtime` and its documentation, this could explain the `failed to parse config file` error you might encounter without it.
An example configuration for caching would be:
```toml
[cache]
enabled = true # This value is required
directory = "/some/path"
```
You can :
- [Create a global `wasmtime` configuration file](#using-a-configuration-file) in `$HOME/.config/wasmtime/config.toml`.
- [Set the `EXTISM_CACHE_CONFIG` environment variable](#using-an-environment-variable)
- [Set the configuration file path using `PluginBuilder`](#using-pluginbuilder)
#### Using a configuration file
The [wasmtime](https://github.com/bytecodealliance/wasmtime) crate, by default, will look for a configuration file in your systems' default configuration directory (for example on UNIX systems: `$HOME/.config/wasmtime/config.toml`),
for more [information on this behaviour](`https://docs.rs/wasmtime/31.0.0/wasmtime/struct.Config.html#method.cache_config_load_default`).
#### Using an environment variable
You can set the `EXTISM_CACHE_CONFIG=path/to/config.toml` environment variable to set the path of the configuration file used by [wasmtime](https://github.com/bytecodealliance/wasmtime).
Setting the variable to an empty string will disable caching (it won't load any configuration file).
> *Note*: If the environment variable is not set, `wasmtime` will still try to read from a configuration file that may exist in your system's default configuration folder (e.g. `$HOME/.config/wasmtime/config.toml`).
The environment variable does not override the path you might have set using `PluginBuilder`. will only be checked for if you did not specify a cache configuration path in `PluginBuilder`.
#### Using PluginBuilder
If you use a [PluginBuilder](https://docs.rs/extism/latest/extism/struct.PluginBuilder.html), you can set the `wasmtime` configuration path using the [with_cache_config](https://docs.rs/extism/latest/extism/struct.PluginBuilder.html#method.with_cache_config) method.
This will override the `EXTISM_CACHE_CONFIG` environment variable if it's set, so you could have a "global" and per plugin configuration if needed.

View File

@@ -16,7 +16,7 @@ fn main() {
let res = plugin.call::<&str, &str>("try_read", "").unwrap();
println!("{:?}", res);
println!("{res:?}");
println!("-----------------------------------------------------");
@@ -30,7 +30,7 @@ fn main() {
);
let res2 = plugin.call::<&str, &str>("try_write", &line).unwrap();
println!("{:?}", res2);
println!("{res2:?}");
println!("done!");
}

View File

@@ -30,6 +30,6 @@ fn main() {
let res = plugin
.call::<&str, &str>("reflect", "Hello, world!")
.unwrap();
println!("{}", res);
println!("{res}");
}
}

View File

@@ -25,6 +25,6 @@ fn main() {
println!("Dumping logs");
for line in LOGS.lock().unwrap().iter() {
print!("{}", line);
print!("{line}");
}
}

View File

@@ -52,6 +52,6 @@ fn main() {
let res = plugin
.call::<&str, &str>("count_vowels", "Hello, world!")
.unwrap();
println!("{}", res);
println!("{res}");
}
}

View File

@@ -156,7 +156,7 @@ void extism_current_plugin_memory_free(ExtismCurrentPlugin *plugin, ExtismMemory
* - `n_outputs`: number of return types
* - `func`: the function to call
* - `user_data`: a pointer that will be passed to the function when it's called
* this value should live as long as the function exists
* this value should live as long as the function exists
* - `free_user_data`: a callback to release the `user_data` value when the resulting
* `ExtismFunction` is freed.
*

View File

@@ -477,10 +477,7 @@ impl CurrentPlugin {
offset: offs,
length,
});
match s {
Ok(s) => Some(s),
Err(_) => None,
}
s.ok()
}
#[doc(hidden)]

Binary file not shown.

View File

@@ -284,8 +284,8 @@ impl Function {
/// A few things worth noting:
/// - The function always returns a `Result` that wraps the specified return type
/// - If a first parameter and type are passed (`_user_data` above) followed by a semicolon it will be
/// the name of the `UserData` parameter and can be used from inside the function
// definition.
/// the name of the `UserData` parameter and can be used from inside the function
// definition.
#[macro_export]
macro_rules! host_fn {
($pub:vis $name: ident ($($arg:ident : $argty:ty),*) $(-> $ret:ty)? $b:block) => {

View File

@@ -29,6 +29,7 @@ pub(crate) mod manifest;
pub(crate) mod pdk;
mod plugin;
mod plugin_builder;
mod pool;
mod readonly_dir;
mod timer;
@@ -43,6 +44,7 @@ pub use plugin::{
CancelHandle, CompiledPlugin, Plugin, WasmInput, EXTISM_ENV_MODULE, EXTISM_USER_MODULE,
};
pub use plugin_builder::{DebugOptions, PluginBuilder};
pub use pool::{Pool, PoolBuilder, PoolPlugin};
pub(crate) use internal::{Internal, Wasi};
pub(crate) use timer::{Timer, TimerAction};
@@ -96,7 +98,7 @@ pub fn set_log_callback<F: 'static + Clone + Fn(&str)>(
let x = tracing_subscriber::EnvFilter::builder()
.with_default_directive(tracing::Level::ERROR.into());
if is_level {
x.parse_lossy(format!("extism={}", filter))
x.parse_lossy(format!("extism={filter}"))
} else {
x.parse_lossy(filter)
}

View File

@@ -10,7 +10,7 @@ use crate::*;
fn hex(data: &[u8]) -> String {
let mut s = String::new();
for &byte in data {
write!(&mut s, "{:02x}", byte).unwrap();
write!(&mut s, "{byte:02x}").unwrap();
}
s
}

View File

@@ -191,6 +191,7 @@ pub(crate) fn profiling_strategy() -> ProfilingStrategy {
/// Defines an input type for Wasm data.
///
/// Types that implement `Into<WasmInput>` can be passed directly into `Plugin::new`
#[derive(Clone)]
pub enum WasmInput<'a> {
/// Raw Wasm module
Data(std::borrow::Cow<'a, [u8]>),
@@ -962,8 +963,7 @@ impl Plugin {
}
Err(msg) => {
res = Err(Error::msg(format!(
"unable to load error message from memory: {}",
msg,
"unable to load error message from memory: {msg}",
)));
}
}

View File

@@ -33,6 +33,7 @@ impl Default for DebugOptions {
}
/// PluginBuilder is used to configure and create `Plugin` instances
#[derive(Clone)]
pub struct PluginBuilder<'a> {
pub(crate) source: WasmInput<'a>,
pub(crate) config: Option<wasmtime::Config>,

174
runtime/src/pool.rs Normal file
View File

@@ -0,0 +1,174 @@
use crate::{Error, FromBytesOwned, Plugin, ToBytes};
// `PoolBuilder` is used to configure and create `Pool`s
#[derive(Debug, Clone)]
pub struct PoolBuilder {
/// Max number of concurrent instances for a plugin - by default this is set to
/// the output of `std::thread::available_parallelism`
pub max_instances: usize,
}
impl PoolBuilder {
/// Create a `PoolBuilder` with default values
pub fn new() -> Self {
Self::default()
}
/// Set the max number of parallel instances
pub fn with_max_instances(mut self, n: usize) -> Self {
self.max_instances = n;
self
}
/// Create a new `Pool` with the given configuration
pub fn build<F: 'static + Fn() -> Result<Plugin, Error>>(self, source: F) -> Pool {
Pool::new_from_builder(source, self)
}
}
impl Default for PoolBuilder {
fn default() -> Self {
PoolBuilder {
max_instances: std::thread::available_parallelism()
.expect("available parallelism")
.into(),
}
}
}
/// `PoolPlugin` is used by the pool to track the number of live instances of a particular plugin
#[derive(Clone, Debug)]
pub struct PoolPlugin(std::rc::Rc<std::cell::RefCell<Plugin>>);
impl PoolPlugin {
fn new(plugin: Plugin) -> Self {
Self(std::rc::Rc::new(std::cell::RefCell::new(plugin)))
}
/// Access the underlying plugin
pub fn plugin(&self) -> std::cell::RefMut<Plugin> {
self.0.borrow_mut()
}
/// Helper to call a plugin function on the underlying plugin
pub fn call<'a, Input: ToBytes<'a>, Output: FromBytesOwned>(
&self,
name: impl AsRef<str>,
input: Input,
) -> Result<Output, Error> {
self.plugin().call(name.as_ref(), input)
}
/// Helper to get the underlying plugin's ID
pub fn id(&self) -> uuid::Uuid {
self.plugin().id
}
}
type PluginSource = dyn Fn() -> Result<Plugin, Error>;
struct PoolInner {
plugin_source: Box<PluginSource>,
instances: Vec<PoolPlugin>,
}
unsafe impl Send for PoolInner {}
unsafe impl Sync for PoolInner {}
/// `Pool` manages threadsafe access to a limited number of instances of multiple plugins
#[derive(Clone)]
pub struct Pool {
config: PoolBuilder,
inner: std::sync::Arc<std::sync::Mutex<PoolInner>>,
}
unsafe impl Send for Pool {}
unsafe impl Sync for Pool {}
impl Pool {
/// Create a new pool with the default configuration
pub fn new<F: 'static + Fn() -> Result<Plugin, Error>>(source: F) -> Self {
Pool {
config: Default::default(),
inner: std::sync::Arc::new(std::sync::Mutex::new(PoolInner {
plugin_source: Box::new(source),
instances: Default::default(),
})),
}
}
/// Create a new pool configured using a `PoolBuilder`
pub fn new_from_builder<F: 'static + Fn() -> Result<Plugin, Error>>(
source: F,
builder: PoolBuilder,
) -> Self {
Pool {
config: builder,
inner: std::sync::Arc::new(std::sync::Mutex::new(PoolInner {
plugin_source: Box::new(source),
instances: Default::default(),
})),
}
}
fn find_available(&self) -> Result<Option<PoolPlugin>, Error> {
let pool = self.inner.lock().unwrap();
for instance in pool.instances.iter() {
if std::rc::Rc::strong_count(&instance.0) == 1 {
return Ok(Some(instance.clone()));
}
}
Ok(None)
}
/// Get the number of live instances for a plugin
pub fn count(&self) -> usize {
self.inner.lock().unwrap().instances.len()
}
/// Get access to a plugin, this will create a new instance if needed (and allowed by the specified
/// max_instances). `Ok(None)` is returned if the timeout is reached before an available plugin could be
/// acquired
pub fn get(&self, timeout: std::time::Duration) -> Result<Option<PoolPlugin>, Error> {
let start = std::time::Instant::now();
let max = self.config.max_instances;
if let Some(avail) = self.find_available()? {
return Ok(Some(avail));
}
{
let mut pool = self.inner.lock().unwrap();
if pool.instances.len() < max {
let plugin = (*pool.plugin_source)()?;
let instance = PoolPlugin::new(plugin);
pool.instances.push(instance);
return Ok(Some(pool.instances.last().unwrap().clone()));
}
}
loop {
if let Ok(Some(x)) = self.find_available() {
return Ok(Some(x));
}
if std::time::Instant::now() - start > timeout {
return Ok(None);
}
std::thread::sleep(std::time::Duration::from_millis(100));
}
}
/// Access a plugin in a callback function. This calls `Pool::get` then the provided
/// callback. `Ok(None)` is returned if the timeout is reached before an available
/// plugin could be acquired
pub fn with_plugin<T>(
&self,
timeout: std::time::Duration,
f: impl FnOnce(&mut Plugin) -> Result<T, Error>,
) -> Result<Option<T>, Error> {
if let Some(plugin) = self.get(timeout)? {
return f(&mut plugin.plugin()).map(Some);
}
Ok(None)
}
}

View File

@@ -180,7 +180,7 @@ pub unsafe extern "C" fn extism_current_plugin_memory_free(
/// - `n_outputs`: number of return types
/// - `func`: the function to call
/// - `user_data`: a pointer that will be passed to the function when it's called
/// this value should live as long as the function exists
/// this value should live as long as the function exists
/// - `free_user_data`: a callback to release the `user_data` value when the resulting
/// `ExtismFunction` is freed.
///
@@ -871,7 +871,7 @@ fn set_log_file(log_file: impl Into<std::path::PathBuf>, filter: &str) -> Result
let x = tracing_subscriber::EnvFilter::builder()
.with_default_directive(tracing::Level::ERROR.into());
if is_level {
x.parse_lossy(format!("extism={}", filter))
x.parse_lossy(format!("extism={filter}"))
} else {
x.parse_lossy(filter)
}
@@ -926,7 +926,7 @@ unsafe fn set_log_buffer(filter: &str) -> Result<(), Error> {
let x = tracing_subscriber::EnvFilter::builder()
.with_default_directive(tracing::Level::ERROR.into());
if is_level {
x.parse_lossy(format!("extism={}", filter))
x.parse_lossy(format!("extism={filter}"))
} else {
x.parse_lossy(filter)
}

View File

@@ -16,7 +16,7 @@ fn test_issue_620() {
// Call test method, this does not work
let p = plugin.call::<(), String>("test", ()).unwrap();
println!("{}", p);
println!("{p}");
}
// https://github.com/extism/extism/issues/619
@@ -53,5 +53,5 @@ fn test_issue_775() {
Ok(code) => Err(code),
}
.unwrap();
println!("{}", p);
println!("{p}");
}

View File

@@ -225,6 +225,26 @@ fn test_kernel_allocations() {
extism_free(&mut store, instance, q);
}
#[test]
fn test_kernel_page_allocations() {
let (mut store, mut instance) = init_kernel_test();
let instance = &mut instance;
let a = extism_alloc(&mut store, instance, 65500 * 4);
let a_size = extism_length(&mut store, instance, a);
let b = extism_alloc(&mut store, instance, 65539);
extism_free(&mut store, instance, a);
let c = extism_alloc(&mut store, instance, 65500 * 2);
let c_size = extism_length(&mut store, instance, c);
let d = extism_alloc(&mut store, instance, 65536);
assert_eq!(a + (a_size - c_size), c);
assert!(c < b);
assert!(d < b);
}
#[test]
fn test_kernel_error() {
let (mut store, mut instance) = init_kernel_test();
@@ -436,3 +456,32 @@ quickcheck! {
true
}
}
quickcheck! {
fn check_block_reuse(allocs: Vec<u16>) -> bool {
let (mut store, mut instance) = init_kernel_test();
let instance = &mut instance;
let init = extism_alloc(&mut store, instance, allocs.iter().map(|x| *x as u64).sum::<u64>() + (allocs.len() as u64 * (65535 + 128)));
let bounds = init + extism_length(&mut store, instance, init);
extism_free(&mut store, instance, init);
for a in allocs {
let ptr = extism_alloc(&mut store, instance, a as u64 + 65535);
if ptr == 0 {
continue
}
if extism_length(&mut store, instance, ptr) != a as u64 + 65535 {
println!("FAILED ALLOC");
return false
}
extism_free(&mut store, instance, ptr);
if ptr > bounds {
println!("ptr={ptr}, bounds={bounds}");
return false
}
}
true
}
}

View File

@@ -1,3 +1,4 @@
mod issues;
mod kernel;
mod pool;
mod runtime;

48
runtime/src/tests/pool.rs Normal file
View File

@@ -0,0 +1,48 @@
use crate::*;
fn run_thread(p: Pool, i: u64) -> std::thread::JoinHandle<()> {
std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(i));
let s: String = p
.get(std::time::Duration::from_secs(1))
.unwrap()
.unwrap()
.call("count_vowels", "abc")
.unwrap();
println!("{s}");
})
}
#[test]
fn test_threads() {
for i in 1..=3 {
let data = include_bytes!("../../../wasm/code.wasm");
let plugin_builder =
extism::PluginBuilder::new(extism::Manifest::new([extism::Wasm::data(data)]))
.with_wasi(true);
let pool: Pool = PoolBuilder::new()
.with_max_instances(i)
.build(move || plugin_builder.clone().build());
let threads = vec![
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 1000),
run_thread(pool.clone(), 500),
run_thread(pool.clone(), 500),
run_thread(pool.clone(), 500),
run_thread(pool.clone(), 500),
run_thread(pool.clone(), 500),
run_thread(pool.clone(), 0),
];
for t in threads {
t.join().unwrap();
}
assert!(pool.count() <= i);
}
}

View File

@@ -133,10 +133,7 @@ fn it_works() {
.unwrap();
let native_avg: std::time::Duration = native_sum / native_num_tests as u32;
println!(
"native function call (avg, N = {}): {:?}",
native_num_tests, native_avg
);
println!("native function call (avg, N = {native_num_tests}): {native_avg:?}");
let num_tests = test_times.len();
let sum: std::time::Duration = test_times
@@ -145,7 +142,7 @@ fn it_works() {
.unwrap();
let avg: std::time::Duration = sum / num_tests as u32;
println!("wasm function call (avg, N = {}): {:?}", num_tests, avg);
println!("wasm function call (avg, N = {num_tests}): {avg:?}");
// Check that log file was written to
if log {
@@ -212,7 +209,7 @@ fn test_cancel() {
let _output: Result<&[u8], Error> = plugin.call("loop_forever", "abc123");
let end = std::time::Instant::now();
let time = end - start;
println!("Cancelled plugin ran for {:?}", time);
println!("Cancelled plugin ran for {time:?}");
}
}
@@ -271,7 +268,7 @@ fn test_fuel_consumption() {
assert!(output.is_err());
let fuel_consumed = plugin.fuel_consumed().unwrap();
println!("Fuel consumed: {}", fuel_consumed);
println!("Fuel consumed: {fuel_consumed}");
assert!(fuel_consumed > 0);
}
@@ -440,7 +437,7 @@ fn test_memory_max() {
assert!(output.is_err());
let err = output.unwrap_err().root_cause().to_string();
println!("{:?}", err);
println!("{err:?}");
assert_eq!(err, "oom");
// Should pass with memory.max set to a large enough number
@@ -503,7 +500,7 @@ fn test_extism_error() {
let mut plugin = Plugin::new(&manifest, [f], true).unwrap();
let output: Result<String, Error> = plugin.call("count_vowels", "a".repeat(1024));
assert!(output.is_err());
println!("{:?}", output);
println!("{output:?}");
assert_eq!(output.unwrap_err().root_cause().to_string(), "TEST");
}
@@ -823,7 +820,7 @@ fn test_http_response_headers() {
.unwrap();
let req = HttpRequest::new("https://extism.org");
let Json(res): Json<HashMap<String, String>> = plugin.call("http_get", Json(req)).unwrap();
println!("{:?}", res);
println!("{res:?}");
assert_eq!(res["content-type"], "text/html; charset=utf-8");
}
@@ -838,6 +835,6 @@ fn test_http_response_headers_disabled() {
.unwrap();
let req = HttpRequest::new("https://extism.org");
let Json(res): Json<HashMap<String, String>> = plugin.call("http_get", Json(req)).unwrap();
println!("{:?}", res);
println!("{res:?}");
assert!(res.is_empty());
}