This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
Unsure if now is the best time to do the `extism` crate version bumps,
but I figured they'd need to happen at some point in the near future
anyways. Happy to revert if there is any opposition. Also, I added back
the local `path` property to the Elixir NIF crate manifest. I want to
see if this breaks again in CI, or if we can leave it.
---------
Co-authored-by: zach <zach@dylib.so>
Let's get the last changes in this week for a release. Will hold this PR
until all changes we want are in.
Release checklist:
- [x] test: updates across CI to test for Host Function output /
integration (#219)
- this should probably look something like a grep for some kind of
output proving guest/host interop?
- [x] Fix for userData pointer issue in Go host functions (#220)
- [x] docs: Host Functions in SDKs
- [ ] sdk: C
- [ ] sdk: C++
- [ ] sdk: Python
- [ ] sdk: Node
- [ ] sdk: Go
- [ ] sdk: Rust
- [x] docs: Manifest property names (http `headers` & memory
`max_pages`)
- [ ] blog: announcing v0.2.0, including host functions, Zig SDK/PDK,
Java SDK, .NET SDK, + ...
Updates the requirements on
[base64](https://github.com/marshallpierce/rust-base64) to permit the
latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md">base64's
changelog</a>.</em></p>
<blockquote>
<h1>0.21.0</h1>
<p>(not yet released)</p>
<h2>Migration</h2>
<h3>Functions</h3>
<table>
<thead>
<tr>
<th>< 0.20 function</th>
<th>0.21 equivalent</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>encode()</code></td>
<td><code>engine::general_purpose::STANDARD.encode()</code> or
<code>prelude::BASE64_STANDARD.encode()</code></td>
</tr>
<tr>
<td><code>encode_config()</code></td>
<td><code>engine.encode()</code></td>
</tr>
<tr>
<td><code>encode_config_buf()</code></td>
<td><code>engine.encode_string()</code></td>
</tr>
<tr>
<td><code>encode_config_slice()</code></td>
<td><code>engine.encode_slice()</code></td>
</tr>
<tr>
<td><code>decode()</code></td>
<td><code>engine::general_purpose::STANDARD.decode()</code> or
<code>prelude::BASE64_STANDARD.decode()</code></td>
</tr>
<tr>
<td><code>decode_config()</code></td>
<td><code>engine.decode()</code></td>
</tr>
<tr>
<td><code>decode_config_buf()</code></td>
<td><code>engine.decode_vec()</code></td>
</tr>
<tr>
<td><code>decode_config_slice()</code></td>
<td><code>engine.decode_slice()</code></td>
</tr>
</tbody>
</table>
<p>The short-lived 0.20 functions were the 0.13 functions with
<code>config</code> replaced with <code>engine</code>.</p>
<h3>Padding</h3>
<p>If applicable, use the preset engines <code>engine::STANDARD</code>,
<code>engine::STANDARD_NO_PAD</code>, <code>engine::URL_SAFE</code>,
or <code>engine::URL_SAFE_NO_PAD</code>.
The <code>NO_PAD</code> ones require that padding is absent when
decoding, and the others require that
canonical padding is present .</p>
<p>If you need the < 0.20 behavior that did not care about padding,
or want to recreate < 0.20.0's predefined <code>Config</code>s
precisely, see the following table.</p>
<table>
<thead>
<tr>
<th>0.13.1 Config</th>
<th>0.20.0+ alphabet</th>
<th><code>encode_padding</code></th>
<th><code>decode_padding_mode</code></th>
</tr>
</thead>
<tbody>
<tr>
<td>STANDARD</td>
<td>STANDARD</td>
<td>true</td>
<td>Indifferent</td>
</tr>
<tr>
<td>STANDARD_NO_PAD</td>
<td>STANDARD</td>
<td>false</td>
<td>Indifferent</td>
</tr>
<tr>
<td>URL_SAFE</td>
<td>URL_SAFE</td>
<td>true</td>
<td>Indifferent</td>
</tr>
<tr>
<td>URL_SAFE_NO_PAD</td>
<td>URL_SAFE</td>
<td>false</td>
<td>Indifferent</td>
</tr>
</tbody>
</table>
<h1>0.21.0-rc.1</h1>
<ul>
<li>Restore the ability to decode into a slice of precisely the correct
length with <code>Engine.decode_slice_unchecked</code>.</li>
<li>Add <code>Engine</code> as a <code>pub use</code> in
<code>prelude</code>.</li>
</ul>
<h1>0.21.0-beta.2</h1>
<h2>Breaking changes</h2>
<ul>
<li>Re-exports of preconfigured engines in <code>engine</code> are
removed in favor of <code>base64::prelude::...</code> that are better
suited to those who wish to <code>use</code> the entire path to a
name.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d7fb31c4ad"><code>d7fb31c</code></a>
v0.21.0</li>
<li><a
href="83503761e9"><code>8350376</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/marshallpierce/rust-base64/issues/207">#207</a>
from marshallpierce/mp/api-rework</li>
<li><a
href="726f7841b7"><code>726f784</code></a>
v0.21.0-rc.1</li>
<li><a
href="b29ab0196d"><code>b29ab01</code></a>
Add <code>Engine</code> in <code>prelude</code></li>
<li><a
href="64bbcc02df"><code>64bbcc0</code></a>
Remove no longer needed test helpers</li>
<li><a
href="0f981bd04f"><code>0f981bd</code></a>
Add decode_slice_unchecked to restore ability to decode into a precisely
size...</li>
<li><a
href="a51e822ae9"><code>a51e822</code></a>
v0.21.0-beta.2</li>
<li><a
href="936569a658"><code>936569a</code></a>
Move re-exports from <code>engine</code> to <code>prelude</code></li>
<li><a
href="53e1091e1b"><code>53e1091</code></a>
Fix release notes typo</li>
<li><a
href="b03eb5a84d"><code>b03eb5a</code></a>
v0.21.0-beta.1</li>
<li>Additional commits viewable in <a
href="https://github.com/marshallpierce/rust-base64/compare/v0.20.0-alpha.1...v0.21.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Adds `Manifest::timeout_ms` field to specify the plugin timeout in
milliseconds
- Sets a 30 second default timeout
Co-authored-by: Steve Manuel <steve@dylib.so>
- Adds the ability to initialize plugins with additional functions
created by the host (only supported by the Rust SDK right now)
- Adds `PluginBuilder` to the Rust SDK to make it easier to configure all the options
## Hello, world
The following example uses `println` from the host to print to stdout
from inside the plugin.
Host:
```rust
use extism::*;
fn main() -> Result<(), Error> {
let context = Context::new();
let manifest = Manifest::new([std::path::PathBuf::from("code.wasm")]);
let say_hello = Function::new("say_hello", [ValType::I64], [], |caller, input, _out| {
let internal = caller.data();
let r = internal.memory().get_str(input[0].unwrap_i64() as usize)?;
println!("Hello, {r}!");
Ok(())
});
let mut plugin = PluginBuilder::new(manifest)
.with_function(say_hello)
.with_wasi(true)
.build(&context)?;
plugin.call("test", b"world").unwrap();
Ok(())
}
```
Plugin:
```rust
use extism_pdk::*;
extern "C" {
fn say_hello(offs: u64);
}
#[plugin_fn]
pub fn test(input: String) -> FnResult<()> {
let memory = Memory::from_bytes(&input);
unsafe {
say_hello(memory.offset);
}
Ok(())
}
```
Adds ability to make local files available to plugins. Using the Rust
SDK:
```rust
let wasm = include_bytes!("code.wasm");
let mut context = Context::new();
let manifest = Manifest::new(vec![Wasm::data(wasm)]).with_allowed_path("/path/on/disk", "/plugin/path");
let plugin = Plugin::new_with_manifest(&mut context, &manifest, false)?;
...
```
- Adds some helper functions for creating a manifest, mostly helpful for
the Rust SDK
- Renames `ManifestWasm` to `Wasm`
- Renames `ManifestMemory` to `MemoryOptions`
- `ManifestWasm` and `ManifestMemory` have been marked as deprecated
- Adds an alias from `MemoryOptions::max_pages` to `MemoryOptions::max`
in serde specification
Co-authored-by: Steve Manuel <steve@dylib.so>
- Adds `allowed_hosts` to the manifest
- If there are any `allowed_hosts` then `extism_http_request` checks to
make sure a URL's host is listed before performing the reques
- Adds `ExtismContext` instead of global `PLUGINS` registry
- Adds `extism_context_new`, `extism_context_free` and
`extism_context_reset`
- Requires updating nearly every SDK function to add context parameter
- Renames some SDK functions to follow better naming conventions
- `extism_plugin_register` -> `extism_plugin_new`
- `extism_output_get` -> `extism_plugin_output_data`
- `extism_output_length` -> `extism_plugin_output_length`
- `extism_call` -> `extism_plugin_call`
- Updates `extism_error` to return the context error when -1 issued for
the plug-in ID
- Adds `extism_plugin_free` to remove an existing plugin
- Updates SDKs to include these functions
- Updates SDK examples and comments
Co-authored-by: Steve Manuel <steve@dylib.so>
