Attempts to address an error seen when upgrading both Rust PDK & SDK to
latest:
```
Updating git repository `https://github.com/extism/extism.git`
Updating git repository `https://github.com/extism/rust-pdk.git`
Updating crates.io index
error: failed to select a version for `base64`.
... required by package `extism-convert v0.2.0 (https://github.com/extism/extism.git?branch=main#7636c873)`
... which satisfies git dependency `extism-convert` of package `extism v1.0.0-alpha.0 (https://github.com/extism/extism.git?branch=main#7636c873)`
... which satisfies git dependency `extism` of package `proto_core v0.22.4 (/Users/miles/Projects/proto/crates/core)`
... which satisfies path dependency `proto_core` (locked to 0.22.4) of package `proto_cli v0.22.0 (/Users/miles/Projects/proto/crates/cli)`
versions that meet the requirements `^0.21.3` are: 0.21.5, 0.21.4, 0.21.3
all possible versions conflict with previously selected packages.
previously selected package `base64 v0.21.0`
... which satisfies dependency `base64 = "^0.21.0"` (locked to 0.21.0) of package `extism-manifest v0.5.0`
... which satisfies dependency `extism-manifest = "^0.5.0"` (locked to 0.5.0) of package `extism-pdk v1.0.0-beta.0 (https://github.com/extism/rust-pdk.git?branch=main#009bf808)`
... which satisfies git dependency `extism-pdk` of package `proto_pdk v0.10.2 (/Users/miles/Projects/proto/crates/pdk)`
failed to select a version for `base64` which could resolve this conflict
```
```
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
```
Build the python native library along with libextism since they change
at roughly the same rate; we can pull the resulting wheels into
python-sdk as needed.
Reduce the release job into a single matrix of `OS x RUST TARGET` – this
unifies the macos, windows, and linux builds into a single job.
---
**BREAKING CHANGE**: This changes the trigger for the release workflow.
Instead of being triggered by the creation of a release, it is triggered
by pushing new git tags. It will catch `v*` – `v0.5.0`, `v200.0.1-dev`
for example. The workflow creates a draft release.
- Adds `extism-convert` crate with `ToBytes`, `FromBytes` and
`FromBytesOwned` traits
- This serves as a single interface for reading/writing rich types from
WebAssembly linear memory.
- Supports `Json` and `Msgpack` and `Protobuf` encodings out-of-the-box
- Updates `Plugin::call` to take `ToBytes` as the input argument and
return a `FromBytes` value
- Adds `host_fn!` macro to simplify host function creation
- Cleans up generated documentation a little
- PR for the Rust PDK: https://github.com/extism/rust-pdk/pull/31
- Adds a `typed_plugin!` macro to implement type-safe wrappers around
`Plugin`
- After this we should focus on adding similar type-conversion helpers
to the SDKs and other PDKs to make it easier to use across languages.
For example, a Python host running a Rust plugin using Msgpack encoded
types.
## Examples
### Calling a function
Instead of the untyped, bytes-only `call` function:
```rust
let output = plugin.call("func_name", "my data").unwrap();
let output: MyType = serde_json::from_slice(&output).unwrap();
```
We can now use richer types to encode/decode our values directly when
using `call`:
```rust
let Json(output) = plugin.call::<_, Json<MyType>>("func_name", "my data").unwrap();
```
### Allocating inside of a host function
The same interface works for host functions, so instead of:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let handle = plugin.memory_handle_val(&inputs[0])?;
let input = plugin.memory_read_str(handle)?;
let output = plugin.memory_alloc_bytes(&input).unwrap();
outputs[0] = output.into();
Ok(())
}
```
Becomes:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let my_value: String = plugin.memory_get_val(&inputs[0])?;
let output = plugin.memory_new(&my_value)?;
outputs[0] = plugin.memory_to_val(output);
Ok(())
}
```
Although this isn't much of an improvement, using the `host_fn` macro,
we can really begin to see how the above function is really just an
identity function:
```rust
host_fn!(hello_world(a: String) -> String {
a
});
```
### typed_plugin!
`typed_plugin!` is used to make a typed wrapper around a Plugin:
```rust
/// Create the typed plugin
typed_plugin!(Testing {
count_vowels(&str) -> Json<Count>
});
/// Create the `Plugin` and convert it to `Testing` wrapper
let mut plugin: Testing = Plugin::new(WASM, [f], true).unwrap().into();
/// Call the `count_vowels` function:
let Json(output0): Json<Count> = plugin.count_vowels("abc123")?;
```
It could make sense to convert `host_fn` and/or `typed_plugin` to
proc-macros at some point, but for now they work and provide some
flexibility in experimenting with the interfaces. Another future update
could be to figure out a nice way to make it so input can be written in
multiple chunks, so the entire input doesn't have to get copied into
memory at once.
- Removes the `ExtismContext` type from runtime and all SDKs
- Removed SDK functions: `extism_context_new`, `extism_context_reset`,
`extism_context_free`
- All SDKs have been updated, but there are still some TODOs below
- Removes `extism_plugin_update`
- Plugins can no longer be updated - a new plugin should be created
instead
- Adds `extism_plugin_id` to uniquely identify plugins
- Merges the `extism-runtime` and `extism` crates (there is no longer an
`extism-runtime` crate)
- Makes `extism::Manifest` an alias for `extism_manifest::Manifest`
instead of a distinct type
- Adds `MemoryHandle` type to SDKs to refer to blocks of Extism memory
that can be accessed in host functions
- Improves thread-safety of Plugins, adds C++ test to call a single
plugin from multiple threads.
- Expands wasmtime bounds to include 12.0
This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
Published this fix as 0.3.2: https://github.com/extism/extism/issues/343
going to disconnect this rust project for the time being. If we want to
publish a new elixir client with the new runtime then we must wait until
the runtime hit's crates.io
