- Requires modules compiled to run with manifests that set `max_memory`
to have an exported memory with lower and upper bounds
- Includes the size of memory exported from modules when calculating
available memory for plugins
## How to compile a module with bounded memory
You will need to pass `--max-memory=$NUM_BYTES` to wasm-ld. `$NUM_BYTES`
must be a multiple of the page size. Here are some examples for
supported PDK languages:
**C**
Pass `-Wl,--max-memory=65536` to your C compiler
**Rust**:
In a `.cargo/config` file:
```toml
[target.wasm32-unknown-unknown]
rustflags = ["-Clink-args=--max-memory=65536"]
```
**Haskell**
Add the following to the cabal file entry for your `cabal.project` file:
```
package myproject
ghc-options:
-optl -Wl,--max-memory=65536
```
**AssemblyScript**
Pass `--maximumMemory 65536` to the assemblyscropt compiler
**TinyGo**:
Create a `target.json` file:
```json
{
"inherits": [ "wasm" ],
"ldflags": [
"--max-memory=65536",
]
}
```
and build using `tinygo -target ./target.json`
EIP: https://github.com/extism/proposals/pull/8
This PR makes minor breaking changes to several SDKs, but not to runtime
C API. The threadsafety updates in the Rust SDK are kind of specific to
Rust, I'm not sure if it makes sense to add the locks to all the other
SDKs at this point. For the most part the `Context` and `Plugin` types
in the SDKs should be safe to use protected by a mutex but they aren't
inherently threadsafe. That kind of locking should probably be done by
the user.
- Runtime
- improve thread safety
- reinstantiates less
- fixes a potential resource exhaustion bug from re-instantiating using
the same store too many times
- Rust SDK
- adds `Send` and `Sync` implementations for `Context`
- adds test sharing a context between threads
- adds `Plugin::call_map` to call a plugin and handle the output with
the lock held
- adds testing sharing an `Arc<Mutex<Plugin>>` between threads
- adds `Plugin::create` and `Plugin::create_from_manifest` to create a
plugin without a `Context`
- Python
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `Plugin(data, context=context)` instead
- Ruby
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `Plugin.new(data, context=context)` instead
- Go
- adds `NewPlugin` and `NewPluginFromManifest` functions
- Node
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `new Plugin(data, wasi, config, host, context)`
instead of `new Plugin(context, data, wasi, functions, config)` (most
people are probably using `context.plugin` instead of the Plugin
constructor anyway)
- OCaml
- BREAKING
- changes `Plugin.create` and `Plugin.of_manifest` to take `context` as
an optional named argument, to update `Plugin.create ~context data` and
`Plugin.of_manifest ~context data` instead
- Haskell
- adds `createPlugin` and `createPluginFromManifest` functions
- Elixir
- adds `Plugin.new` to make a plugin without going through
`Context.new_plugin`
- Java
- adds new `Plugin` constructors without a `Context` argument
- C++
- BREAKING
- Updates `Plugin` constructor to take an optional context as the last
argument, instead of requiring it to be the first argument
- Use `Plugin(wasm, wasi, functions, ctx)` instead of `Plugin(ctx, wasm,
wasi, functions)`
- Zig
- Adds `Plugin.create` and `Plugin.createWithManifest` to create plugins
in their own context.
---------
Co-authored-by: zach <zach@dylib.so>
Co-authored-by: Benjamin Eckel <bhelx@simst.im>
This PR adds the ability to cancel running plugins from another thread
in the host.
- All SDKs have been updated except .NET
- Adds a new SDK type: `ExtismCancelHandle`
- Adds 2 new SDK functions: `extism_plugin_cancel_handle` and
`extism_plugin_cancel`
- `extism_plugin_cancel_handle` returns a pointer to
`ExtismCancelHandle`, which can be passed to `extism_plugin_cancel` to
stop plugin execution.
- One thing that's worth noting is when plugin is executing a host
function it cannot be cancelled until after the host function returns.
---------
Co-authored-by: Etienne ANNE <etienne.anne@icloud.com>
Let's get the last changes in this week for a release. Will hold this PR
until all changes we want are in.
Release checklist:
- [x] test: updates across CI to test for Host Function output /
integration (#219)
- this should probably look something like a grep for some kind of
output proving guest/host interop?
- [x] Fix for userData pointer issue in Go host functions (#220)
- [x] docs: Host Functions in SDKs
- [ ] sdk: C
- [ ] sdk: C++
- [ ] sdk: Python
- [ ] sdk: Node
- [ ] sdk: Go
- [ ] sdk: Rust
- [x] docs: Manifest property names (http `headers` & memory
`max_pages`)
- [ ] blog: announcing v0.2.0, including host functions, Zig SDK/PDK,
Java SDK, .NET SDK, + ...
Hi,
We want to use Extism on our project
[Otoroshi](https://github.com/MAIF/otoroshi) but we need to run it on
jdk11
This pull request makes everything run smoothly on jdk11
If you have any suggestion about this pull request, i'm open to it
Thanks for your time
- New types:
- `ExtismValType` - Enum of WebAssembly types
- `ExtismValUnion` - A union of the possible WebAssembly types
- `ExtismVal` - A struct with `ExtismValType` and `ExtismValUnion`
- `ExtismFunction` - The host function wrapper type
- `ExtismFunctionType` - The type of the host function callback
- `ExtismCurrentPlugin` - Provides access to the currently running
plugin from inside a host function
- New functions:
- `extism_function_new` - Create a new `ExtismFunction`
- `extism_function_free` - Free an `ExtismFunction`
- `extism_current_plugin_memory`, `extism_current_plugin_memory_alloc`,
`extism_current_plugin_memory_free`,
`extism_current_plugin_memory_length` - Manage plugin memory from inside
a host functions
- Updated functions
- `extism_plugin_new` and `extsim_plugin_update` - now accept two extra
parameters for `ExtismFunction*` array and length of that array
## Notes
- Host functions take a user-data argument, which is owned by the
resulting `ExtismFunction` and will be cleaned up when
`extism_function_free` is called (if a cleanup function was passed in
with the user data)
- Host functions in every SDK require working with `ExtismVal` arguments
directly, this is pretty low-level for what is kind of a high-level
feature. We could work on adding some types to the SDKs that make
working with pointers to plugin data more accessible, maybe something
similar to how the Rust PDK handes input/output data.
- In each language the host functions more-or-less share a signature:
`(CurrentPlugin plugin, Val inputs[], Val outputs[], userData)`
- C, C++, OCaml and Go take a single userData argument but Python and
Node take a "rest" argument which allows passing any number of user-data
values
- Go requires the host function to be exported:
f9eb5ed839/go/main.go (L13-L26)
- Zig and Ruby should be relatively simple to add host functions to next
but I haven't really looked into Elixir, .NET or Java yet.
- Also closes#20
