Add an example of dynamically linking plugins and a benchmark that does
an apples-to-apples comparison of `reflect` using host functions vs.
`reflect` using a linked wasm module. (To my surprise, the host
functions are a _little bit faster_!)
Further flesh out our runtime benchmarks: add benchmarks for sending
bytes one-way into Wasm linear memory, as well as testing copy-in then
copy-out. This compliments our `reflect` benchmark; we can get a sense
of how expensive the sub-operations are via `echo` and `consume`.
(Notably, we're missing a `produce` or `emit` that purely generates
output and sends it to the host!)
The source of these plugins is available in `extism/plugins` as
`echo.wat` and `consume.wat` [1].
[1]: https://github.com/extism/plugins/pull/4/files
See #504
Removes CI for the in-repo SDKs, we can remove the actual code closer to
the 1.0 release.
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
Same as #471 for `main` branch
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
Adds support for host functions and cleans up some of the FFI code.
## API
To make a host function, you can pass a proc to `Function::new`:
```ruby
func = proc do |current_plugin, inputs, outputs, user_data|
input = current_plugin.input_as_bytes(inputs.first)
current_plugin.return_string(outputs.first, "#{input} #{user_data}")
end
f = Extism::Function.new('transform_string', [Extism::ValType::I64], [Extism::ValType::I64], func, 'My User Data')
plugin = Extism::Plugin.new(host_manifest, [f], true)
result = plugin.call('reflect_string', 'Hello, World!')
assert_equal result, 'Hello, World! My User Data'
```
If your function is in a module or a class, you can use
`method(name).to_proc`. Example:
```ruby
module Test
def self.my_function(current_plugin, inputs, outputs, user_data)
input = current_plugin.input_as_bytes(inputs.first)
current_plugin.return_string(outputs.first, "#{input} #{user_data}")
end
end
func = Test.method(:my_function).to_proc
f = Extism::Function.new('my_function', [Extism::ValType::I64], [Extism::ValType::I64], func, 'My User Data')
```
`current_plugin` is of the type CurrentPlugin which has some helpful
methods:
* `CurrentPlugin#memory_at_offset(int)` returns a `Memory` object given
a memory pointer
* `CurrentPlugin#free(Memory)` frees the memory
* `CurrentPlugin#alloc(int)` allocates new memory and returns a `Memory`
* `CurrentPlugin#input_as_bytes(Value)` returns the bytes for the given
input param
* `CurrentPlugin#return_bytes(Value, Array)` Sets the array of bytes to
the return for the given output value
* `CurrentPlugin#input_as_bytes(Value, String)` Sets the string to the
return for the given output value
It seems like our runtime initialization process is a little too
aggressive, this PR scales it back to initialize the runtime once if it
exists and only reinitializes when `_start` is called. This prevents
globals from being wiped out between plugin calls.
- Removes Runtime::cleanup
- Only initializes runtime once, or if `_start` is called
- Improves Haskell reactor support
See https://github.com/extism/go-sdk/pull/11 for the go-sdk PR.
This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
- New types:
- `ExtismValType` - Enum of WebAssembly types
- `ExtismValUnion` - A union of the possible WebAssembly types
- `ExtismVal` - A struct with `ExtismValType` and `ExtismValUnion`
- `ExtismFunction` - The host function wrapper type
- `ExtismFunctionType` - The type of the host function callback
- `ExtismCurrentPlugin` - Provides access to the currently running
plugin from inside a host function
- New functions:
- `extism_function_new` - Create a new `ExtismFunction`
- `extism_function_free` - Free an `ExtismFunction`
- `extism_current_plugin_memory`, `extism_current_plugin_memory_alloc`,
`extism_current_plugin_memory_free`,
`extism_current_plugin_memory_length` - Manage plugin memory from inside
a host functions
- Updated functions
- `extism_plugin_new` and `extsim_plugin_update` - now accept two extra
parameters for `ExtismFunction*` array and length of that array
## Notes
- Host functions take a user-data argument, which is owned by the
resulting `ExtismFunction` and will be cleaned up when
`extism_function_free` is called (if a cleanup function was passed in
with the user data)
- Host functions in every SDK require working with `ExtismVal` arguments
directly, this is pretty low-level for what is kind of a high-level
feature. We could work on adding some types to the SDKs that make
working with pointers to plugin data more accessible, maybe something
similar to how the Rust PDK handes input/output data.
- In each language the host functions more-or-less share a signature:
`(CurrentPlugin plugin, Val inputs[], Val outputs[], userData)`
- C, C++, OCaml and Go take a single userData argument but Python and
Node take a "rest" argument which allows passing any number of user-data
values
- Go requires the host function to be exported:
f9eb5ed839/go/main.go (L13-L26)
- Zig and Ruby should be relatively simple to add host functions to next
but I haven't really looked into Elixir, .NET or Java yet.
- Also closes#20
This PR updates `extism_output_get` to return an actual pointer to the
output value (`const uint8_t* extism_output_get(PluginIndex plugin)`
instead of `void extism_output_get(PluginIndex plugin, uint8_t *buffer,
uint64_t length)`), this pointer will only be valid until the next call,
but it makes it possible to access the output data without copying.
The input buffer is also not copied and the same issue applies:
the input buffer must not change during `call`.
Co-authored-by: Steve Manuel <steve@dylib.so>
