* libextism is now built to crate-type `staticlib` as recommend by
@zshipko
* updated `Makefile` to install and uninstall the static library.
* updated c example to have `static` target to link libextism statically
* Compiler is no longer hardcoded to `clang`. Set `CC` to use a compiler
other than the system compiler. Tested with `gcc` and `clang`.
* added `static-artifact` to actions release. Untested as it's only ran
on push with tag.
---------
Co-authored-by: zach <zach@dylibso.com>
- Updates to wasmtime 13.0.0 as the lowest supported version
- Removes `wasmtime-wasi-nn` support because it now takes parameters to
specify particular backends and registries during initialization. Since
I don't think anyone is using the `nn` feature I chose to remove it for
now, but we could also expand the manifest to add these options.
- Adds `extism-convert` crate with `ToBytes`, `FromBytes` and
`FromBytesOwned` traits
- This serves as a single interface for reading/writing rich types from
WebAssembly linear memory.
- Supports `Json` and `Msgpack` and `Protobuf` encodings out-of-the-box
- Updates `Plugin::call` to take `ToBytes` as the input argument and
return a `FromBytes` value
- Adds `host_fn!` macro to simplify host function creation
- Cleans up generated documentation a little
- PR for the Rust PDK: https://github.com/extism/rust-pdk/pull/31
- Adds a `typed_plugin!` macro to implement type-safe wrappers around
`Plugin`
- After this we should focus on adding similar type-conversion helpers
to the SDKs and other PDKs to make it easier to use across languages.
For example, a Python host running a Rust plugin using Msgpack encoded
types.
## Examples
### Calling a function
Instead of the untyped, bytes-only `call` function:
```rust
let output = plugin.call("func_name", "my data").unwrap();
let output: MyType = serde_json::from_slice(&output).unwrap();
```
We can now use richer types to encode/decode our values directly when
using `call`:
```rust
let Json(output) = plugin.call::<_, Json<MyType>>("func_name", "my data").unwrap();
```
### Allocating inside of a host function
The same interface works for host functions, so instead of:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let handle = plugin.memory_handle_val(&inputs[0])?;
let input = plugin.memory_read_str(handle)?;
let output = plugin.memory_alloc_bytes(&input).unwrap();
outputs[0] = output.into();
Ok(())
}
```
Becomes:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let my_value: String = plugin.memory_get_val(&inputs[0])?;
let output = plugin.memory_new(&my_value)?;
outputs[0] = plugin.memory_to_val(output);
Ok(())
}
```
Although this isn't much of an improvement, using the `host_fn` macro,
we can really begin to see how the above function is really just an
identity function:
```rust
host_fn!(hello_world(a: String) -> String {
a
});
```
### typed_plugin!
`typed_plugin!` is used to make a typed wrapper around a Plugin:
```rust
/// Create the typed plugin
typed_plugin!(Testing {
count_vowels(&str) -> Json<Count>
});
/// Create the `Plugin` and convert it to `Testing` wrapper
let mut plugin: Testing = Plugin::new(WASM, [f], true).unwrap().into();
/// Call the `count_vowels` function:
let Json(output0): Json<Count> = plugin.count_vowels("abc123")?;
```
It could make sense to convert `host_fn` and/or `typed_plugin` to
proc-macros at some point, but for now they work and provide some
flexibility in experimenting with the interfaces. Another future update
could be to figure out a nice way to make it so input can be written in
multiple chunks, so the entire input doesn't have to get copied into
memory at once.
- Removes the `ExtismContext` type from runtime and all SDKs
- Removed SDK functions: `extism_context_new`, `extism_context_reset`,
`extism_context_free`
- All SDKs have been updated, but there are still some TODOs below
- Removes `extism_plugin_update`
- Plugins can no longer be updated - a new plugin should be created
instead
- Adds `extism_plugin_id` to uniquely identify plugins
- Merges the `extism-runtime` and `extism` crates (there is no longer an
`extism-runtime` crate)
- Makes `extism::Manifest` an alias for `extism_manifest::Manifest`
instead of a distinct type
- Adds `MemoryHandle` type to SDKs to refer to blocks of Extism memory
that can be accessed in host functions
- Improves thread-safety of Plugins, adds C++ test to call a single
plugin from multiple threads.
- Expands wasmtime bounds to include 12.0
This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
Unsure if now is the best time to do the `extism` crate version bumps,
but I figured they'd need to happen at some point in the near future
anyways. Happy to revert if there is any opposition. Also, I added back
the local `path` property to the Elixir NIF crate manifest. I want to
see if this breaks again in CI, or if we can leave it.
---------
Co-authored-by: zach <zach@dylib.so>
Let's get the last changes in this week for a release. Will hold this PR
until all changes we want are in.
Release checklist:
- [x] test: updates across CI to test for Host Function output /
integration (#219)
- this should probably look something like a grep for some kind of
output proving guest/host interop?
- [x] Fix for userData pointer issue in Go host functions (#220)
- [x] docs: Host Functions in SDKs
- [ ] sdk: C
- [ ] sdk: C++
- [ ] sdk: Python
- [ ] sdk: Node
- [ ] sdk: Go
- [ ] sdk: Rust
- [x] docs: Manifest property names (http `headers` & memory
`max_pages`)
- [ ] blog: announcing v0.2.0, including host functions, Zig SDK/PDK,
Java SDK, .NET SDK, + ...
