See https://github.com/extism/cpp-sdk/issues/15
- Limits a call to memset in the kernel to the size of the current
memory offset instead of the total size of memory.
- Adds `extism_plugin_reset` to the C API and `extism::Plugin::reset` to
Rust
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
- Uses `tracing` instead of `log` crate
- Uses `tracing-subscriber` instead of `fern`
- This allows us to automatically capture `log` events using
`tracing-subscriber`
- Breaking: Makes `extism::set_log_file` private and only used through
the C API, Rust users should use `tracing-subscriber` to determine which
filters/levels to log.
- Adds `extism::set_log_callback` function to set a callback that can be
used for custom logging from Rust.
- Adds `bool extism_log_custom(const char *level)` and
`extism_log_drain(void (*fn)(const char *s, size_t length)` to the C API
to enable custom sinks in other SDKs
I realized the glibc static build does actually work (though prints
warnings at compile time), so I made it no longer hard-coded to the
compiler. It's still a bad idea, so I added notes for why you shouldn't
do it with glibc.
The `x86_64-unknown-linux-musl` target does not build dynamic libraries
for now:
```
warning: dropping unsupported crate type `cdylib` for target `x86_64-unknown-linux-musl`
```
A target may now be optionally specified to the root `Makefile`,
example:
`make RUST_TARGET=x86_64-unknown-linux-musl && sudo make
RUST_TARGET=x86_64-unknown-linux-musl install`
The new musl targets may be used to create entirely static binaries with
libextism. See the new `musl-static` target in `libextism/Makefile`
Adds `pkg-config` configs for dynamic and static builds of libextism.
Example usage is shown in `libextism/Makefile`.
Added the `.in` versions of the `.pc` files to non-MSVC releases.
See #504
Removes CI for the in-repo SDKs, we can remove the actual code closer to
the 1.0 release.
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
* libextism is now built to crate-type `staticlib` as recommend by
@zshipko
* updated `Makefile` to install and uninstall the static library.
* updated c example to have `static` target to link libextism statically
* Compiler is no longer hardcoded to `clang`. Set `CC` to use a compiler
other than the system compiler. Tested with `gcc` and `clang`.
* added `static-artifact` to actions release. Untested as it's only ran
on push with tag.
---------
Co-authored-by: zach <zach@dylibso.com>
- Updates to wasmtime 13.0.0 as the lowest supported version
- Removes `wasmtime-wasi-nn` support because it now takes parameters to
specify particular backends and registries during initialization. Since
I don't think anyone is using the `nn` feature I chose to remove it for
now, but we could also expand the manifest to add these options.
- Adds `extism-convert` crate with `ToBytes`, `FromBytes` and
`FromBytesOwned` traits
- This serves as a single interface for reading/writing rich types from
WebAssembly linear memory.
- Supports `Json` and `Msgpack` and `Protobuf` encodings out-of-the-box
- Updates `Plugin::call` to take `ToBytes` as the input argument and
return a `FromBytes` value
- Adds `host_fn!` macro to simplify host function creation
- Cleans up generated documentation a little
- PR for the Rust PDK: https://github.com/extism/rust-pdk/pull/31
- Adds a `typed_plugin!` macro to implement type-safe wrappers around
`Plugin`
- After this we should focus on adding similar type-conversion helpers
to the SDKs and other PDKs to make it easier to use across languages.
For example, a Python host running a Rust plugin using Msgpack encoded
types.
## Examples
### Calling a function
Instead of the untyped, bytes-only `call` function:
```rust
let output = plugin.call("func_name", "my data").unwrap();
let output: MyType = serde_json::from_slice(&output).unwrap();
```
We can now use richer types to encode/decode our values directly when
using `call`:
```rust
let Json(output) = plugin.call::<_, Json<MyType>>("func_name", "my data").unwrap();
```
### Allocating inside of a host function
The same interface works for host functions, so instead of:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let handle = plugin.memory_handle_val(&inputs[0])?;
let input = plugin.memory_read_str(handle)?;
let output = plugin.memory_alloc_bytes(&input).unwrap();
outputs[0] = output.into();
Ok(())
}
```
Becomes:
```rust
fn hello_world(
plugin: &mut CurrentPlugin,
inputs: &[Val],
outputs: &mut [Val],
_user_data: UserData,
) -> Result<(), Error> {
let my_value: String = plugin.memory_get_val(&inputs[0])?;
let output = plugin.memory_new(&my_value)?;
outputs[0] = plugin.memory_to_val(output);
Ok(())
}
```
Although this isn't much of an improvement, using the `host_fn` macro,
we can really begin to see how the above function is really just an
identity function:
```rust
host_fn!(hello_world(a: String) -> String {
a
});
```
### typed_plugin!
`typed_plugin!` is used to make a typed wrapper around a Plugin:
```rust
/// Create the typed plugin
typed_plugin!(Testing {
count_vowels(&str) -> Json<Count>
});
/// Create the `Plugin` and convert it to `Testing` wrapper
let mut plugin: Testing = Plugin::new(WASM, [f], true).unwrap().into();
/// Call the `count_vowels` function:
let Json(output0): Json<Count> = plugin.count_vowels("abc123")?;
```
It could make sense to convert `host_fn` and/or `typed_plugin` to
proc-macros at some point, but for now they work and provide some
flexibility in experimenting with the interfaces. Another future update
could be to figure out a nice way to make it so input can be written in
multiple chunks, so the entire input doesn't have to get copied into
memory at once.
- Removes the `ExtismContext` type from runtime and all SDKs
- Removed SDK functions: `extism_context_new`, `extism_context_reset`,
`extism_context_free`
- All SDKs have been updated, but there are still some TODOs below
- Removes `extism_plugin_update`
- Plugins can no longer be updated - a new plugin should be created
instead
- Adds `extism_plugin_id` to uniquely identify plugins
- Merges the `extism-runtime` and `extism` crates (there is no longer an
`extism-runtime` crate)
- Makes `extism::Manifest` an alias for `extism_manifest::Manifest`
instead of a distinct type
- Adds `MemoryHandle` type to SDKs to refer to blocks of Extism memory
that can be accessed in host functions
- Improves thread-safety of Plugins, adds C++ test to call a single
plugin from multiple threads.
- Expands wasmtime bounds to include 12.0
This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
Unsure if now is the best time to do the `extism` crate version bumps,
but I figured they'd need to happen at some point in the near future
anyways. Happy to revert if there is any opposition. Also, I added back
the local `path` property to the Elixir NIF crate manifest. I want to
see if this breaks again in CI, or if we can leave it.
---------
Co-authored-by: zach <zach@dylib.so>
Let's get the last changes in this week for a release. Will hold this PR
until all changes we want are in.
Release checklist:
- [x] test: updates across CI to test for Host Function output /
integration (#219)
- this should probably look something like a grep for some kind of
output proving guest/host interop?
- [x] Fix for userData pointer issue in Go host functions (#220)
- [x] docs: Host Functions in SDKs
- [ ] sdk: C
- [ ] sdk: C++
- [ ] sdk: Python
- [ ] sdk: Node
- [ ] sdk: Go
- [ ] sdk: Rust
- [x] docs: Manifest property names (http `headers` & memory
`max_pages`)
- [ ] blog: announcing v0.2.0, including host functions, Zig SDK/PDK,
Java SDK, .NET SDK, + ...
