This is a rough POC for allowing people to whitelist a dir as readonly.
When the source path is prefixed with `ro:`, the dir is considered as
readonly. This preserved backward compatibility. This suggestion came up
in https://github.com/extism/go-sdk/pull/1#discussion_r1276700587
Readonly:
```rs
let manifest = Manifest::new([url])
.with_allowed_path("ro:D:/x/rust/fs/data".to_string(), "/data")
.with_config_key("path", "/data/data.txt");
```
```
trying to read file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\n"
-----------------------------------------------------
trying to write file:
thread '<unnamed>' panicked at src\lib.rs:24:34:
called `Result::unwrap()` on an `Err` value: Os { code: 58, kind: Unsupported, message: "Not supported" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at runtime\examples\fs.rs:27:6:
called `Result::unwrap()` on an `Err` value: error while executing at wasm backtrace:
0: 0x234d2 - fs.wasm!__rust_start_panic
1: 0x232a1 - fs.wasm!rust_panic
2: 0x231da - fs.wasm!std::panicking::rust_panic_with_hook::hd3fb69bc0aea298a
3: 0x22467 - fs.wasm!std::panicking::begin_panic_handler::{{closure}}::h4d99b90b43f79472
4: 0x223ca - fs.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::h5691573a73161cb1
5: 0x22bca - fs.wasm!rust_begin_unwind
6: 0x303e9 - fs.wasm!core::panicking::panic_fmt::hdb62f5cdb45533e4
7: 0x3234d - fs.wasm!core::result::unwrap_failed::h30d23efcc9e41efc
8: 0x36c2 - fs.wasm!fs::try_write::inner::h0b3b0df8e129f5cc
9: 0x29cd - fs.wasm!try_write
10: 0x35e4a - fs.wasm!try_write.command_export
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
Caused by:
wasm trap: wasm `unreachable` instruction executed
Stack backtrace:
0: std::backtrace_rs::backtrace::dbghelp64::trace
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\..\..\backtrace\src\backtrace\dbghelp64.rs:99
1: std::backtrace_rs::backtrace::trace_unsynchronized
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
2: std::backtrace::Backtrace::create
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\backtrace.rs:331
3: std::backtrace::Backtrace::capture
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\backtrace.rs:296
4: anyhow::error::impl$1::from<wasmtime_environ::trap_encoding::Trap>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\anyhow-1.0.86\src\error.rs:565
5: core::convert::impl$3::into<wasmtime_environ::trap_encoding::Trap,anyhow::Error>
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\convert\mod.rs:759
6: wasmtime_environ::impl$1::into_anyhow<wasmtime_environ::trap_encoding::Trap>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-environ-22.0.0\src\lib.rs:90
7: wasmtime::runtime::trap::from_runtime_box
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\trap.rs:118
8: wasmtime::runtime::func::invoke_wasm_and_catch_traps::closure$0<extism::current_plugin::CurrentPlugin,wasmtime::runtime::func::impl$1::call_unchecked_raw::closure_env$0<extism::current_plugin::CurrentPlugin> >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1597
9: enum2$<core::result::Result<tuple$<>,alloc::boxed::Box<wasmtime::runtime::vm::traphandlers::Trap,alloc::alloc::Global> > >::map_err<tuple$<>,alloc::boxed::Box<wasmtime::runtime::vm::traphandlers::Trap,alloc::alloc::Global>,anyhow::Error,wasmtime::runtime:
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\result.rs:829
10: wasmtime::runtime::func::invoke_wasm_and_catch_traps<extism::current_plugin::CurrentPlugin,wasmtime::runtime::func::impl$1::call_unchecked_raw::closure_env$0<extism::current_plugin::CurrentPlugin> >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1597
11: wasmtime::runtime::func::Func::call_unchecked_raw<extism::current_plugin::CurrentPlugin>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1063
12: wasmtime::runtime::func::Func::call_unchecked<ref_mut$<wasmtime::runtime::store::context::StoreContextMut<extism::current_plugin::CurrentPlugin> > >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1049
13: wasmtime::runtime::func::Func::call_impl_do_call<extism::current_plugin::CurrentPlugin>
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1243
14: wasmtime::runtime::func::Func::call<ref_mut$<wasmtime::runtime::store::Store<extism::current_plugin::CurrentPlugin> > >
at C:\Users\muham\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-22.0.0\src\runtime\func.rs:1002
15: extism::plugin::Plugin::raw_call<ref$<str$>,ref$<str$> >
at .\src\plugin.rs:753
16: extism::plugin::Plugin::call<ref$<str$>,ref$<str$>,ref$<str$> >
at .\src\plugin.rs:900
17: fs::main
at .\examples\fs.rs:25
18: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\ops\function.rs:250
19: core::hint::black_box
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\hint.rs:337
20: std::sys_common::backtrace::__rust_begin_short_backtrace<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\sys_common\backtrace.rs:155
21: std::rt::lang_start::closure$0<tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\rt.rs:166
22: std::rt::lang_start_internal
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\rt.rs:148
23: std::rt::lang_start<tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\rt.rs:165
24: main
25: invoke_main
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
26: __scrt_common_main_seh
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
27: BaseThreadInitThunk
28: RtlUserThreadStart
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\panicking.rs:645
1: core::panicking::panic_fmt
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:72
2: core::result::unwrap_failed
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\result.rs:1654
3: enum2$<core::result::Result<ref$<str$>,anyhow::Error> >::unwrap
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\result.rs:1077
4: fs::main
at .\examples\fs.rs:25
5: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\ops\function.rs:250
6: core::hint::black_box
at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\hint.rs:337
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `D:\dylibso\extism\target\debug\examples\fs.exe` (exit code: 101)
```
Writable:
```rs
let manifest = Manifest::new([url])
.with_allowed_path("D:/x/rust/fs/data".to_string(), "/data")
.with_config_key("path", "/data/data.txt");
```
```
trying to read file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\n"
-----------------------------------------------------
trying to write file:
"Hello World at 1719851282.5109031sHello World at 1719851299.0819795sHello World at 1719851317.8934608s\nHello World at 1719851500.7803263s\n"
done!
```
- Breaking: No longer copies Extism config values into WASI environment
variables because the new interface doesn't allow for the environment to
be updated - these should be accessed using the Extism config functions
instead
- Requires wasmtime 20 or greater
- Enables wasm-gc
- Similar to https://github.com/extism/extism/pull/697 without sockets
or additional support for command modules
https://github.com/extism/dotnet-pdk/pull/84 brought to my attention
that the Rust SDK supports both `headers` and `header` while the go sdk
only supports `headers`. After talking to Zach, we decided to remove the
`header` alias from the Rust SDK too, since it's obsolete and we want
people to use `headers`
Currently HTTP requests can extend beyond the configured timeout for a
plugin - this PR sets the timeout of the HTTP request to the remaining
time left if a timeout is set in the manifest.
Add `plugin.call_with_host_context` and `current_plugin.host_context`
methods, enabling per-call context to be looped from the guest invocation
to any host functions it calls. In an HTTP server environment, this enables
re-using a plugin across multiple requests while switching out backing
connections and user information in host functions. (Imagine a host
function, `update_user` -- previously the plugin would have to have been
aware of the user to pass to the host function. Now that information is
ambient.)
This is a backwards-compatible change and requires no changes to
existing plugins.
Implement by adding a global, mutable externref to the extism kernel.
Since most programming languages, including Rust, don't let you define
these natively, we accomplish this by using `wasm-merge` to combine the
kernel Wasm with Wasm generated by a WAT file containing only the
global.
(This pattern might be useful for other Wasm constructs we can't use
directly from Rust, like `v128` in argument parameters.)
Wasmtime requires extern refs to be `Any + Send + Sync + 'static`; we
additionally add `Clone`. I haven't tried this with an `Arc` directly,
but it should work at least for container structs that hold `Arc`'s
themselves.
Some of the old constants conflicted with Perl headers used for build
Perl extensions such the
[perl-sdk](002e4437ac/Extism/lib/Extism/XS.xs).
This fix would allow removing the copy of extism.h (inlined in that file
linked) out of the perl-sdk.
These constants especially `I32` likely have conflicts with many other
large C codebases.
A new extism release with this fix should not be made until the affected
sdks (cpp-sdk) have an update ready.
- Adds `memory.max_var_bytes` to the manifest to limit the number of
bytes allowed to be stored in Extism vars - if `max_var_bytes` is set to
0 then vars are disabled.
- Adds some builder functions to `MemoryOptions` struct
- Sets the default var store size to 1mb
- Includes a test to make sure `var_set` returns an error when the limit
is reached
Updates the requirements on
[base64](https://github.com/marshallpierce/rust-base64) to permit the
latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md">base64's
changelog</a>.</em></p>
<blockquote>
<h1>0.22.0</h1>
<ul>
<li><code>DecodeSliceError::OutputSliceTooSmall</code> is now
conservative rather than precise. That is, the error will only occur if
the decoded output <em>cannot</em> fit, meaning that
<code>Engine::decode_slice</code> can now be used with exactly-sized
output slices. As part of this, <code>Engine::internal_decode</code> now
returns <code>DecodeSliceError</code> instead of
<code>DecodeError</code>, but that is not expected to affect any
external callers.</li>
<li><code>DecodeError::InvalidLength</code> now refers specifically to
the <em>number of valid symbols</em> being invalid (i.e. <code>len % 4
== 1</code>), rather than just the number of input bytes. This avoids
confusing scenarios when based on interpretation you could make a case
for either <code>InvalidLength</code> or <code>InvalidByte</code> being
appropriate.</li>
<li>Decoding is somewhat faster (5-10%)</li>
</ul>
<h1>0.21.7</h1>
<ul>
<li>Support getting an alphabet's contents as a str via
<code>Alphabet::as_str()</code></li>
</ul>
<h1>0.21.6</h1>
<ul>
<li>Improved introductory documentation and example</li>
</ul>
<h1>0.21.5</h1>
<ul>
<li>Add <code>Debug</code> and <code>Clone</code> impls for the general
purpose Engine</li>
</ul>
<h1>0.21.4</h1>
<ul>
<li>Make <code>encoded_len</code> <code>const</code>, allowing the
creation of arrays sized to encode compile-time-known data lengths</li>
</ul>
<h1>0.21.3</h1>
<ul>
<li>Implement <code>source</code> instead of <code>cause</code> on Error
types</li>
<li>Roll back MSRV to 1.48.0 so Debian can continue to live in a time
warp</li>
<li>Slightly faster chunked encoding for short inputs</li>
<li>Decrease binary size</li>
</ul>
<h1>0.21.2</h1>
<ul>
<li>Rollback MSRV to 1.57.0 -- only dev dependencies need 1.60, not the
main code</li>
</ul>
<h1>0.21.1</h1>
<ul>
<li>Remove the possibility of panicking during decoded length
calculations</li>
<li><code>DecoderReader</code> no longer sometimes erroneously ignores
padding <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/226">#226</a></li>
</ul>
<h2>Breaking changes</h2>
<ul>
<li><code>Engine.internal_decode</code> return type changed</li>
<li>Update MSRV to 1.60.0</li>
</ul>
<h1>0.21.0</h1>
<h2>Migration</h2>
<h3>Functions</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5d70ba7576"><code>5d70ba7</code></a>
Merge pull request <a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/269">#269</a>
from marshallpierce/mp/decode-precisely</li>
<li><a
href="efb6c006c7"><code>efb6c00</code></a>
Release notes</li>
<li><a
href="2b91084a31"><code>2b91084</code></a>
Add some tests to boost coverage</li>
<li><a
href="9e9c7abe65"><code>9e9c7ab</code></a>
Engine::internal_decode now returns DecodeSliceError</li>
<li><a
href="a8a60f43c5"><code>a8a60f4</code></a>
Decode main loop improvements</li>
<li><a
href="a25be0667c"><code>a25be06</code></a>
Simplify leftover output writes</li>
<li><a
href="9979cc33bb"><code>9979cc3</code></a>
Keep morsels as separate bytes</li>
<li><a
href="37670c5ec2"><code>37670c5</code></a>
Bump dev toolchain version (<a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/268">#268</a>)</li>
<li><a
href="9652c78773"><code>9652c78</code></a>
v0.21.7</li>
<li><a
href="08deccf703"><code>08deccf</code></a>
provide as_str() method to return the alphabet characters (<a
href="https://redirect.github.com/marshallpierce/rust-base64/issues/264">#264</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/marshallpierce/rust-base64/compare/v0.21.0...v0.22.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
It looks like when a module is added to the linker, all of its imports
must already be present. This PR updates the linking process to take
that into consideration and adds a test with a reproduction of the issue
@chrisdickinson shared with me.
