Okay, phew.
The main bug from the [last
PR](https://github.com/extism/extism/pull/610) was that the rust
releases scramble to publish, but we have to publish them in a
particular order for the release to work since they depend on each
other.
There's a secondary bug: `cargo publish` is prone to 502'ing on publish.
I lean towards seeing how painful this is in practice; we should be able
to safely re-run the release flow on failure.
Follow-up to f7d297f98f.
Update the release workflows for the dependent crates. The Cargo
workflows run on GitHub release publish to match the Python package
release workflow. This means all of our crates are versioned in lockstep
by the Git tag.
There are four changes:
1. Trigger the workflows on GitHub release publish
2. Add the `set version` step from `release.yml` to modify the version
tags in `Cargo.toml`.
3. Publish with `--allow-dirty` (to account for the edit in step 2)
4. In `extism-maturin`, do not inherit `authors` from the workspace
since [PyPI rejects the value we have
set](https://github.com/extism/extism/actions/runs/7012534645/job/19077216730#step:7:23).
- Adds `extism-convert` crate with `ToBytes`, `FromBytes` and
`FromBytesOwned` traits
- This serves as a single interface for reading/writing rich types from
WebAssembly linear memory.
- Supports `Json` and `Msgpack` and `Protobuf` encodings out-of-the-box
- Updates `Plugin::call` to take `ToBytes` as the input argument and
return a `FromBytes` value
- Adds `host_fn!` macro to simplify host function creation
- Cleans up generated documentation a little
- PR for the Rust PDK: https://github.com/extism/rust-pdk/pull/31
- Adds a `typed_plugin!` macro to implement type-safe wrappers around
`Plugin`
- After this we should focus on adding similar type-conversion helpers
to the SDKs and other PDKs to make it easier to use across languages.
For example, a Python host running a Rust plugin using Msgpack encoded
types.
## Examples
### Calling a function
Instead of the untyped, bytes-only `call` function:
```rust
let output = plugin.call("func_name", "my data").unwrap();
let output: MyType = serde_json::from_slice(&output).unwrap();
```
We can now use richer types to encode/decode our values directly when
using `call`:
```rust
let Json(output) = plugin.call::<_, Json<MyType>>("func_name", "my data").unwrap();
```
### Allocating inside of a host function
The same interface works for host functions, so instead of:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let handle = plugin.memory_handle_val(&inputs[0])?;
let input = plugin.memory_read_str(handle)?;
let output = plugin.memory_alloc_bytes(&input).unwrap();
outputs[0] = output.into();
Ok(())
}
```
Becomes:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let my_value: String = plugin.memory_get_val(&inputs[0])?;
let output = plugin.memory_new(&my_value)?;
outputs[0] = plugin.memory_to_val(output);
Ok(())
}
```
Although this isn't much of an improvement, using the `host_fn` macro,
we can really begin to see how the above function is really just an
identity function:
```rust
host_fn!(hello_world(a: String) -> String {
a
});
```
### typed_plugin!
`typed_plugin!` is used to make a typed wrapper around a Plugin:
```rust
/// Create the typed plugin
typed_plugin!(Testing {
count_vowels(&str) -> Json<Count>
});
/// Create the `Plugin` and convert it to `Testing` wrapper
let mut plugin: Testing = Plugin::new(WASM, [f], true).unwrap().into();
/// Call the `count_vowels` function:
let Json(output0): Json<Count> = plugin.count_vowels("abc123")?;
```
It could make sense to convert `host_fn` and/or `typed_plugin` to
proc-macros at some point, but for now they work and provide some
flexibility in experimenting with the interfaces. Another future update
could be to figure out a nice way to make it so input can be written in
multiple chunks, so the entire input doesn't have to get copied into
memory at once.
- Removes the `ExtismContext` type from runtime and all SDKs
- Removed SDK functions: `extism_context_new`, `extism_context_reset`,
`extism_context_free`
- All SDKs have been updated, but there are still some TODOs below
- Removes `extism_plugin_update`
- Plugins can no longer be updated - a new plugin should be created
instead
- Adds `extism_plugin_id` to uniquely identify plugins
- Merges the `extism-runtime` and `extism` crates (there is no longer an
`extism-runtime` crate)
- Makes `extism::Manifest` an alias for `extism_manifest::Manifest`
instead of a distinct type
- Adds `MemoryHandle` type to SDKs to refer to blocks of Extism memory
that can be accessed in host functions
- Improves thread-safety of Plugins, adds C++ test to call a single
plugin from multiple threads.
- Expands wasmtime bounds to include 12.0
