- Adds `extism:host/env::http_headers` to access HTTP response headers
- Adds `PluginBuilder::with_http_response_headers` to enable response
headers from Rust
- Adds `extism_plugin_allow_http_response_headers` to enable response
headers using the C API
TODO:
- [x] Update a PDK to use `extism:host/env::http_headers` so I can test
this
Updates the kernel to only scan for free blocks before a `memory.grow`
operation, this should improve performance of `alloc` by not iterating
over every block for each allocation.
An alternative to #763, this PR allows an initial `wasmtime::Config` to
be passed in when building a plugin. Some of these values may be
overwritten by the Extism runtime, but it allows for things like static
memory size and other low-level details to be handled directly instead
of us having to wrap every option ourselves.
- Checks error results to determine if a plugin ran out of fuel, if the
fuel is 0 after we get the error, then we return an out of fuel error
message instead.
- Updates `extism_plugin_error` to check `Plugin::error_msg` regardless
of the kernel's error state
Both [as_ptr][as_ptr] and [as_mut_ptr][as_mut_ptr] are allowed to return
a dangling raw pointer when the Vec size is 0.
The idea is that you should guard that read checking the size. This
probably works well in most cases, but at the very least in the
java-sdk, the JNA machinery tries to be helpful and it dereferences the
pointer, causing a SIGSEGV.
The solution is to check if the resulting vector is empty and return
null instead. A new, empty vector would be better, but I think that
would not solve the problem, because the problem is caused by a new,
empty vector in the first place.
Caveat: this might break consumers downstream.
On the other hand: consumers that do not check for the nInput, nOutput
counts are just waiting to explode, like JNA.
This addresses https://github.com/extism/java-sdk/issues/27
[as_ptr]:
https://doc.rust-lang.org/std/vec/struct.Vec.html#method.as_ptr
[as_mut_ptr]:
https://doc.rust-lang.org/std/vec/struct.Vec.html#method.as_mut_ptr
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Updates plugins to allocate a single `ExternRef` for the host context up
front, to avoid running into the `failed to allocate externref` error
from Wasmtime
Updates the requirements on
[cbindgen](https://github.com/mozilla/cbindgen) to permit the latest
version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/mozilla/cbindgen/blob/master/CHANGES">cbindgen's
changelog</a>.</em></p>
<blockquote>
<h1>0.27.0</h1>
<pre><code> * Revert: The `Config` struct now has a private member.
* Allow users to specify a crate version for bindings generation
([#901](https://github.com/mozilla/cbindgen/issues/901)).
* Update MSRV to 1.74
([#912](https://github.com/mozilla/cbindgen/issues/912),
[#987](https://github.com/mozilla/cbindgen/issues/987)).
* Support #[deprecated] on enum variants
([#933](https://github.com/mozilla/cbindgen/issues/933)).
* Support integrating the package_version information in a header file
comment ([#939](https://github.com/mozilla/cbindgen/issues/939)).
* Add a language backend
([#942](https://github.com/mozilla/cbindgen/issues/942)).
* Support generics with defaulted args
([#959](https://github.com/mozilla/cbindgen/issues/959)).
* Add `VaList` compatibility
([#970](https://github.com/mozilla/cbindgen/issues/970)).
</code></pre>
<h1>0.26.0</h1>
<pre><code> * Fix swapping of `>>=` and `<<=` in constants.
* Add support for #[deprecated]
([#860](https://github.com/mozilla/cbindgen/issues/860)).
* Built-in support for bitflags 2.0.
* Support for "C-unwind" ABI.
* Generate bindings for non-public extern items if they are
#[no_mangle].
</code></pre>
<h2>0.25.0</h2>
<pre><code> * Re-release of yanked 0.24.6 as a major release
* Update MSRV to 1.57
* Support variadic arguments (`...`)
([#805](https://github.com/mozilla/cbindgen/issues/805))
* Add --depfile option
([#820](https://github.com/mozilla/cbindgen/issues/820))
* Breaking changes: The `Config` struct now has a private member.
</code></pre>
<h2>0.24.6 (YANKED: depfile option was breaking, see <a
href="https://redirect.github.com/mozilla/cbindgen/issues/841">#841</a>)</h2>
<pre><code> * Update MSRV to 1.57
* Support variadic arguments (`...`)
([#805](https://github.com/mozilla/cbindgen/issues/805))
* Add --depfile option
([#820](https://github.com/mozilla/cbindgen/issues/820))
</code></pre>
<h2>0.24.5</h2>
<pre><code> * Don't enforce tempfile version.
</code></pre>
<h2>0.24.4</h2>
<pre><code> * Move expand infinite recursion fix
([#799](https://github.com/mozilla/cbindgen/issues/799))
* Add with_cpp_compat to the builder
([#796](https://github.com/mozilla/cbindgen/issues/796))
* Handle never type in return position consistently
([#780](https://github.com/mozilla/cbindgen/issues/780))
* Fix warnings ([#816](https://github.com/mozilla/cbindgen/issues/816),
[#819](https://github.com/mozilla/cbindgen/issues/819))
* Updated documentation
([#788](https://github.com/mozilla/cbindgen/issues/788),
[#791](https://github.com/mozilla/cbindgen/issues/791),
[#792](https://github.com/mozilla/cbindgen/issues/792),
[#810](https://github.com/mozilla/cbindgen/issues/810),
[#823](https://github.com/mozilla/cbindgen/issues/823))
</code></pre>
<h2>0.24.3</h2>
<pre><code> * Make struct expressions correctly generated through
typedefs ([#768](https://github.com/mozilla/cbindgen/issues/768)).
</code></pre>
<h2>0.24.2</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="58c6156b0d"><code>58c6156</code></a>
Bump MSRV again for clap.</li>
<li><a
href="103a8de0ac"><code>103a8de</code></a>
Cargo update and version bump.</li>
<li><a
href="67cb560430"><code>67cb560</code></a>
Update CHANGES since v0.26.0</li>
<li><a
href="316298182e"><code>3162981</code></a>
Output condition for globals.</li>
<li><a
href="e469e44c00"><code>e469e44</code></a>
Add missing parens to log message</li>
<li><a
href="3cbb637bbf"><code>3cbb637</code></a>
Update cython expectations from the previous patch.</li>
<li><a
href="785e066e03"><code>785e066</code></a>
Fix variadic arguments when used in function pointer</li>
<li><a
href="aa8ea654e1"><code>aa8ea65</code></a>
deps: Update syn to 2.0</li>
<li><a
href="9f632843b8"><code>9f63284</code></a>
Generalize Item to expose documentation and generic params</li>
<li><a
href="1dda6162e5"><code>1dda616</code></a>
Used documented method of specifying variadic arguments</li>
<li>Additional commits viewable in <a
href="https://github.com/mozilla/cbindgen/compare/0.26.0...v0.27.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This is a rough POC for allowing people to whitelist a dir as readonly.
When the source path is prefixed with `ro:`, the dir is considered as
readonly. This preserved backward compatibility. This suggestion came up
in https://github.com/extism/go-sdk/pull/1#discussion_r1276700587
Readonly:
```rs
let manifest = Manifest::new([url])
.with_allowed_path("ro:D:/x/rust/fs/data".to_string(), "/data")
.with_config_key("path", "/data/data.txt");
```
```
trying to read file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\n"
-----------------------------------------------------
trying to write file:
thread '<unnamed>' panicked at src\lib.rs:24:34:
called `Result::unwrap()` on an `Err` value: Os { code: 58, kind: Unsupported, message: "Not supported" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at runtime\examples\fs.rs:27:6:
called `Result::unwrap()` on an `Err` value: error while executing at wasm backtrace:
0: 0x234d2 - fs.wasm!__rust_start_panic
1: 0x232a1 - fs.wasm!rust_panic
2: 0x231da - fs.wasm!std::panicking::rust_panic_with_hook::hd3fb69bc0aea298a
3: 0x22467 - fs.wasm!std::panicking::begin_panic_handler::{{closure}}::h4d99b90b43f79472
4: 0x223ca - fs.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::h5691573a73161cb1
5: 0x22bca - fs.wasm!rust_begin_unwind
6: 0x303e9 - fs.wasm!core::panicking::panic_fmt::hdb62f5cdb45533e4
7: 0x3234d - fs.wasm!core::result::unwrap_failed::h30d23efcc9e41efc
8: 0x36c2 - fs.wasm!fs::try_write::inner::h0b3b0df8e129f5cc
9: 0x29cd - fs.wasm!try_write
10: 0x35e4a - fs.wasm!try_write.command_export
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
Caused by:
wasm trap: wasm `unreachable` instruction executed
Stack backtrace:
0: std::backtrace_rs::backtrace::dbghelp64::trace
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\..\..\backtrace\src\backtrace\dbghelp64.rs:99
1: std::backtrace_rs::backtrace::trace_unsynchronized
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
2: std::backtrace::Backtrace::create
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\backtrace.rs:331
3: std::backtrace::Backtrace::capture
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\backtrace.rs:296
4: anyhow::error::impl$1::from<wasmtime_environ::trap_encoding::Trap>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\anyhow-1.0.86\src\error.rs:565
5: core::convert::impl$3::into<wasmtime_environ::trap_encoding::Trap,anyhow::Error>
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\convert\mod.rs:759
6: wasmtime_environ::impl$1::into_anyhow<wasmtime_environ::trap_encoding::Trap>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-environ-22.0.0\src\lib.rs:90
7: wasmtime::runtime::trap::from_runtime_box
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\trap.rs:118
8: wasmtime::runtime::func::invoke_wasm_and_catch_traps::closure$0<extism::current_plugin::CurrentPlugin,wasmtime::runtime::func::impl$1::call_unchecked_raw::closure_env$0<extism::current_plugin::CurrentPlugin> >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1597
9: enum2$<core::result::Result<tuple$<>,alloc::boxed::Box<wasmtime::runtime::vm::traphandlers::Trap,alloc::alloc::Global> > >::map_err<tuple$<>,alloc::boxed::Box<wasmtime::runtime::vm::traphandlers::Trap,alloc::alloc::Global>,anyhow::Error,wasmtime::runtime:
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\result.rs:829
10: wasmtime::runtime::func::invoke_wasm_and_catch_traps<extism::current_plugin::CurrentPlugin,wasmtime::runtime::func::impl$1::call_unchecked_raw::closure_env$0<extism::current_plugin::CurrentPlugin> >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1597
11: wasmtime::runtime::func::Func::call_unchecked_raw<extism::current_plugin::CurrentPlugin>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1063
12: wasmtime::runtime::func::Func::call_unchecked<ref_mut$<wasmtime::runtime::store::context::StoreContextMut<extism::current_plugin::CurrentPlugin> > >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1049
13: wasmtime::runtime::func::Func::call_impl_do_call<extism::current_plugin::CurrentPlugin>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1243
14: wasmtime::runtime::func::Func::call<ref_mut$<wasmtime::runtime::store::Store<extism::current_plugin::CurrentPlugin> > >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1002
15: extism::plugin::Plugin::raw_call<ref$<str$>,ref$<str$> >
at .\src\plugin.rs:753
16: extism::plugin::Plugin::call<ref$<str$>,ref$<str$>,ref$<str$> >
at .\src\plugin.rs:900
17: fs::main
at .\examples\fs.rs:25
18: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\ops\function.rs:250
19: core::hint::black_box
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\hint.rs:337
20: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\sys_common\backtrace.rs:155
21: std::rt::lang_start::closure$0<tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\rt.rs:166
22: std::rt::lang_start_internal
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\rt.rs:148
23: std::rt::lang_start<tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\rt.rs:165
24: main
25: invoke_main
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
26: __scrt_common_main_seh
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
27: BaseThreadInitThunk
28: RtlUserThreadStart
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\panicking.rs:645
1: core::panicking::panic_fmt
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:72
2: core::result::unwrap_failed
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\result.rs:1654
3: enum2$<core::result::Result<ref$<str$>,anyhow::Error> >::unwrap
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\result.rs:1077
4: fs::main
at .\examples\fs.rs:25
5: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\ops\function.rs:250
6: core::hint::black_box
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\hint.rs:337
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `D:\dylibso\extism\target\debug\examples\fs.exe` (exit code: 101)
```
Writable:
```rs
let manifest = Manifest::new([url])
.with_allowed_path("D:/x/rust/fs/data".to_string(), "/data")
.with_config_key("path", "/data/data.txt");
```
```
trying to read file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\n"
-----------------------------------------------------
trying to write file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\nHello World at 1719851500.7803263s\n"
done!
```
- Breaking: No longer copies Extism config values into WASI environment
variables because the new interface doesn't allow for the environment to
be updated - these should be accessed using the Extism config functions
instead
- Requires wasmtime 20 or greater
- Enables wasm-gc
- Similar to https://github.com/extism/extism/pull/697 without sockets
or additional support for command modules
Currently HTTP requests can extend beyond the configured timeout for a
plugin - this PR sets the timeout of the HTTP request to the remaining
time left if a timeout is set in the manifest.
Add `plugin.call_with_host_context` and `current_plugin.host_context`
methods, enabling per-call context to be looped from the guest invocation
to any host functions it calls. In an HTTP server environment, this enables
re-using a plugin across multiple requests while switching out backing
connections and user information in host functions. (Imagine a host
function, `update_user` -- previously the plugin would have to have been
aware of the user to pass to the host function. Now that information is
ambient.)
This is a backwards-compatible change and requires no changes to
existing plugins.
Implement by adding a global, mutable externref to the extism kernel.
Since most programming languages, including Rust, don't let you define
these natively, we accomplish this by using `wasm-merge` to combine the
kernel Wasm with Wasm generated by a WAT file containing only the
global.
(This pattern might be useful for other Wasm constructs we can't use
directly from Rust, like `v128` in argument parameters.)
Wasmtime requires extern refs to be `Any + Send + Sync + 'static`; we
additionally add `Clone`. I haven't tried this with an `Arc` directly,
but it should work at least for container structs that hold `Arc`'s
themselves.
Some of the old constants conflicted with Perl headers used for build
Perl extensions such the
[perl-sdk](002e4437ac/Extism/lib/Extism/XS.xs).
This fix would allow removing the copy of extism.h (inlined in that file
linked) out of the perl-sdk.
These constants especially `I32` likely have conflicts with many other
large C codebases.
A new extism release with this fix should not be made until the affected
sdks (cpp-sdk) have an update ready.
- Adds `memory.max_var_bytes` to the manifest to limit the number of
bytes allowed to be stored in Extism vars - if `max_var_bytes` is set to
0 then vars are disabled.
- Adds some builder functions to `MemoryOptions` struct
- Sets the default var store size to 1mb
- Includes a test to make sure `var_set` returns an error when the limit
is reached
It looks like when a module is added to the linker, all of its imports
must already be present. This PR updates the linking process to take
that into consideration and adds a test with a reproduction of the issue
@chrisdickinson shared with me.
- Adds quickcheck tests for alloc/free/load/store from the kernel, I
wasn't able to include these in the new wasm-bindgen tests because
`getrandom` isn't available on wasm32-unknown-unknown
- Fixes a bug in the calculation of how much memory is needed to
allocate the next block in the kernel. This bug is triggered when
allocating at the end of a page, the size of the MemoryBlock value
wasn't being taken in consideration when determining whether or not to
call memory.grow
base64 support is retained. Now `data` can instead be `{"ptr":
ptr_value, "len": len_value}` where `ptr_value` points to a wasm module
and `len_value` is the size of bytes of it.
