EIP: https://github.com/extism/proposals/pull/8
This PR makes minor breaking changes to several SDKs, but not to runtime
C API. The threadsafety updates in the Rust SDK are kind of specific to
Rust, I'm not sure if it makes sense to add the locks to all the other
SDKs at this point. For the most part the `Context` and `Plugin` types
in the SDKs should be safe to use protected by a mutex but they aren't
inherently threadsafe. That kind of locking should probably be done by
the user.
- Runtime
- improve thread safety
- reinstantiates less
- fixes a potential resource exhaustion bug from re-instantiating using
the same store too many times
- Rust SDK
- adds `Send` and `Sync` implementations for `Context`
- adds test sharing a context between threads
- adds `Plugin::call_map` to call a plugin and handle the output with
the lock held
- adds testing sharing an `Arc<Mutex<Plugin>>` between threads
- adds `Plugin::create` and `Plugin::create_from_manifest` to create a
plugin without a `Context`
- Python
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `Plugin(data, context=context)` instead
- Ruby
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `Plugin.new(data, context=context)` instead
- Go
- adds `NewPlugin` and `NewPluginFromManifest` functions
- Node
- BREAKING
- changes `Plugin` constructor to take `context` as an optional named
argument, to update use `new Plugin(data, wasi, config, host, context)`
instead of `new Plugin(context, data, wasi, functions, config)` (most
people are probably using `context.plugin` instead of the Plugin
constructor anyway)
- OCaml
- BREAKING
- changes `Plugin.create` and `Plugin.of_manifest` to take `context` as
an optional named argument, to update `Plugin.create ~context data` and
`Plugin.of_manifest ~context data` instead
- Haskell
- adds `createPlugin` and `createPluginFromManifest` functions
- Elixir
- adds `Plugin.new` to make a plugin without going through
`Context.new_plugin`
- Java
- adds new `Plugin` constructors without a `Context` argument
- C++
- BREAKING
- Updates `Plugin` constructor to take an optional context as the last
argument, instead of requiring it to be the first argument
- Use `Plugin(wasm, wasi, functions, ctx)` instead of `Plugin(ctx, wasm,
wasi, functions)`
- Zig
- Adds `Plugin.create` and `Plugin.createWithManifest` to create plugins
in their own context.
---------
Co-authored-by: zach <zach@dylib.so>
Co-authored-by: Benjamin Eckel <bhelx@simst.im>
This PR adds the ability to cancel running plugins from another thread
in the host.
- All SDKs have been updated except .NET
- Adds a new SDK type: `ExtismCancelHandle`
- Adds 2 new SDK functions: `extism_plugin_cancel_handle` and
`extism_plugin_cancel`
- `extism_plugin_cancel_handle` returns a pointer to
`ExtismCancelHandle`, which can be passed to `extism_plugin_cancel` to
stop plugin execution.
- One thing that's worth noting is when plugin is executing a host
function it cannot be cancelled until after the host function returns.
---------
Co-authored-by: Etienne ANNE <etienne.anne@icloud.com>
Updates the requirements on
[minitest](https://github.com/seattlerb/minitest) to permit the latest
version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/minitest/minitest/blob/master/History.rdoc">minitest's
changelog</a>.</em></p>
<blockquote>
<p>=== 5.18.0 / 2023-03-04</p>
<ul>
<li>
<p>2 major enhancements:</p>
<ul>
<li>Added assert_pattern & refute_pattern for pattern matching.
(flavorjones)</li>
<li>Added matching must_pattern_match & wont_pattern_match to
minitest/spec.</li>
</ul>
</li>
<li>
<p>1 bug fix:</p>
<ul>
<li>Support the new message format of NameError in Ruby 3.3 (mame)</li>
</ul>
</li>
</ul>
<p>=== 5.17.0 / 2022-12-31</p>
<ul>
<li>
<p>1 minor enhancement:</p>
<ul>
<li>Refactor setup hooks into a SETUP_METHODS constant. (MSP-Greg)</li>
</ul>
</li>
<li>
<p>3 bug fixes:</p>
<ul>
<li>Fix kwargs for Mock calls to delegator. (blowmage)</li>
<li>Fix kwargs for expectations. (bobmazanec, blowmage)</li>
<li>Remove check for .b method. (tenderlove)</li>
</ul>
</li>
</ul>
<p>=== 5.16.3 / 2022-08-17</p>
<ul>
<li>
<p>2 bug fixes:</p>
<ul>
<li>Fixed exception sanitization by removing TypeError restriction on
rescue.</li>
<li>Use A instead of deprecated TESTOPTS in rake test:slow.
(davidstosik)</li>
</ul>
</li>
</ul>
<p>=== 5.16.2 / 2022-07-03</p>
<ul>
<li>
<p>4 bug fixes:</p>
<ul>
<li>Added MT_KWARGS_HACK kludge for stub to deal with ruby 2.7 kwargs
nastiness. (tsugimoto)</li>
<li>In #expect, pop Hash class from args if $MT_KWARGS_HACK.
(casperisfine)</li>
<li>In above scenario, set expected kwargs (as Objects) based on actual
kwargs.</li>
<li>Nuke ivars if exception fails to marshal twice (eg better_errors).
(irphilli)</li>
</ul>
</li>
</ul>
<p>=== 5.16.1 / 2022-06-20</p>
<ul>
<li>
<p>2 bug fixes:</p>
<ul>
<li>Apparently adding real kwarg support to mocks/stubs broke some code.
Fixed.
<ul>
<li>Use <code>MT_KWARGS_HACK=1</code> to activate the kludgy kwargs
support w/ caveats.</li>
</ul>
</li>
<li>Clarified some doco wrt the block on #stub.</li>
</ul>
</li>
</ul>
<p>=== 5.16.0 / 2022-06-14</p>
<ul>
<li>2 major enhancements:</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="506ce83b45"><code>506ce83</code></a>
prepped for release</li>
<li><a
href="0c44f4ea32"><code>0c44f4e</code></a>
! Added assert_pattern & refute_pattern for pattern matching.
(flavorjones)</li>
<li><a
href="899b420aa0"><code>899b420</code></a>
Fixed typo in doco. (ahangarha)</li>
<li><a
href="ebd8a673a1"><code>ebd8a67</code></a>
- Support the new message format of NameError in Ruby 3.3 (mame)</li>
<li><a
href="0984e29995"><code>0984e29</code></a>
Add 2.6 to matrix... stays until some rails versions expire, sadly.</li>
<li><a
href="d5e68e6ef4"><code>d5e68e6</code></a>
Adds Ruby 3.2 to the CI matrix. Also updates checkout action version.
(peterg...</li>
<li><a
href="69bc4b0f5b"><code>69bc4b0</code></a>
Minor tweak to Rakefile to fix CI on older rubies</li>
<li>See full diff in <a
href="https://github.com/seattlerb/minitest/compare/v5.17.0...v5.18.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Let's get the last changes in this week for a release. Will hold this PR
until all changes we want are in.
Release checklist:
- [x] test: updates across CI to test for Host Function output /
integration (#219)
- this should probably look something like a grep for some kind of
output proving guest/host interop?
- [x] Fix for userData pointer issue in Go host functions (#220)
- [x] docs: Host Functions in SDKs
- [ ] sdk: C
- [ ] sdk: C++
- [ ] sdk: Python
- [ ] sdk: Node
- [ ] sdk: Go
- [ ] sdk: Rust
- [x] docs: Manifest property names (http `headers` & memory
`max_pages`)
- [ ] blog: announcing v0.2.0, including host functions, Zig SDK/PDK,
Java SDK, .NET SDK, + ...
- New types:
- `ExtismValType` - Enum of WebAssembly types
- `ExtismValUnion` - A union of the possible WebAssembly types
- `ExtismVal` - A struct with `ExtismValType` and `ExtismValUnion`
- `ExtismFunction` - The host function wrapper type
- `ExtismFunctionType` - The type of the host function callback
- `ExtismCurrentPlugin` - Provides access to the currently running
plugin from inside a host function
- New functions:
- `extism_function_new` - Create a new `ExtismFunction`
- `extism_function_free` - Free an `ExtismFunction`
- `extism_current_plugin_memory`, `extism_current_plugin_memory_alloc`,
`extism_current_plugin_memory_free`,
`extism_current_plugin_memory_length` - Manage plugin memory from inside
a host functions
- Updated functions
- `extism_plugin_new` and `extsim_plugin_update` - now accept two extra
parameters for `ExtismFunction*` array and length of that array
## Notes
- Host functions take a user-data argument, which is owned by the
resulting `ExtismFunction` and will be cleaned up when
`extism_function_free` is called (if a cleanup function was passed in
with the user data)
- Host functions in every SDK require working with `ExtismVal` arguments
directly, this is pretty low-level for what is kind of a high-level
feature. We could work on adding some types to the SDKs that make
working with pointers to plugin data more accessible, maybe something
similar to how the Rust PDK handes input/output data.
- In each language the host functions more-or-less share a signature:
`(CurrentPlugin plugin, Val inputs[], Val outputs[], userData)`
- C, C++, OCaml and Go take a single userData argument but Python and
Node take a "rest" argument which allows passing any number of user-data
values
- Go requires the host function to be exported:
f9eb5ed839/go/main.go (L13-L26)
- Zig and Ruby should be relatively simple to add host functions to next
but I haven't really looked into Elixir, .NET or Java yet.
- Also closes#20
Updates the requirements on
[minitest](https://github.com/seattlerb/minitest) to permit the latest
version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/minitest/minitest/blob/master/History.rdoc">minitest's
changelog</a>.</em></p>
<blockquote>
<p>=== 5.17.0 / 2022-12-31</p>
<ul>
<li>
<p>1 minor enhancement:</p>
<ul>
<li>Refactor setup hooks into a SETUP_METHODS constant. (MSP-Greg)</li>
</ul>
</li>
<li>
<p>3 bug fixes:</p>
<ul>
<li>Fix kwargs for Mock calls to delegator. (blowmage)</li>
<li>Fix kwargs for expectations. (bobmazanec, blowmage)</li>
<li>Remove check for .b method. (tenderlove)</li>
</ul>
</li>
</ul>
<p>=== 5.16.3 / 2022-08-17</p>
<ul>
<li>
<p>2 bug fixes:</p>
<ul>
<li>Fixed exception sanitization by removing TypeError restriction on
rescue.</li>
<li>Use A instead of deprecated TESTOPTS in rake test:slow.
(davidstosik)</li>
</ul>
</li>
</ul>
<p>=== 5.16.2 / 2022-07-03</p>
<ul>
<li>
<p>4 bug fixes:</p>
<ul>
<li>Added MT_KWARGS_HACK kludge for stub to deal with ruby 2.7 kwargs
nastiness. (tsugimoto)</li>
<li>In #expect, pop Hash class from args if $MT_KWARGS_HACK.
(casperisfine)</li>
<li>In above scenario, set expected kwargs (as Objects) based on actual
kwargs.</li>
<li>Nuke ivars if exception fails to marshal twice (eg better_errors).
(irphilli)</li>
</ul>
</li>
</ul>
<p>=== 5.16.1 / 2022-06-20</p>
<ul>
<li>
<p>2 bug fixes:</p>
<ul>
<li>Apparently adding real kwarg support to mocks/stubs broke some code.
Fixed.
<ul>
<li>Use <code>MT_KWARGS_HACK=1</code> to activate the kludgy kwargs
support w/ caveats.</li>
</ul>
</li>
<li>Clarified some doco wrt the block on #stub.</li>
</ul>
</li>
</ul>
<p>=== 5.16.0 / 2022-06-14</p>
<ul>
<li>
<p>2 major enhancements:</p>
<ul>
<li>Added Minitest::TestTask.</li>
<li>Dropping ruby 2.2 - 2.5. 2.6 is DTM soon too.</li>
</ul>
</li>
<li>
<p>11 minor enhancements:</p>
<ul>
<li>Added --show-skips option to show skips at end of run but not
require --verbose. (MSP-Greg)</li>
<li>Added Minitest.seed, the random seed used by the run.</li>
<li>Calling <code>srand Minitest.seed</code> before all shuffles to
ensure determinism.</li>
<li>Extended #stub to handle kwargs for both block and call args.
(SampsonCrowley)</li>
<li>Extended Mock#__call to display kwargs.</li>
</ul>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="abdde9d03b"><code>abdde9d</code></a>
prepped for release</li>
<li><a
href="c0be030603"><code>c0be030</code></a>
- Fix kwargs for Mock calls to delegator. (blowmage)</li>
<li><a
href="87604fca4d"><code>87604fc</code></a>
- Fix kwargs for expectations. (bobmazanec, blowmage)</li>
<li><a
href="0b816d303b"><code>0b816d3</code></a>
Add EOL date to rails matrix</li>
<li><a
href="2f7ed237f1"><code>2f7ed23</code></a>
cleaned up rails version</li>
<li><a
href="ae54abfb23"><code>ae54abf</code></a>
Updated README for rails/ruby compatibilty matrix</li>
<li><a
href="4f31487068"><code>4f31487</code></a>
Fixed race condition causing flaky tests. (XrXr)</li>
<li><a
href="dcdd882fbe"><code>dcdd882</code></a>
get rake dcov back to 100%</li>
<li><a
href="3a77687e8b"><code>3a77687</code></a>
+ Refactor setup hooks into a SETUP_METHODS constant. (MSP-Greg)</li>
<li><a
href="b5565c0c7a"><code>b5565c0</code></a>
- Remove check for .b method. (tenderlove)</li>
<li>See full diff in <a
href="https://github.com/seattlerb/minitest/compare/v5.16.3...v5.17.0">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Work in progress
* Adds some inline docs which generates a yarddoc site
* Adds linting and code formatting abilities using rufo
* Adds a Makefile with common operations
Co-authored-by: Steve Manuel <steve@dylib.so>
- Adds `ExtismContext` instead of global `PLUGINS` registry
- Adds `extism_context_new`, `extism_context_free` and
`extism_context_reset`
- Requires updating nearly every SDK function to add context parameter
- Renames some SDK functions to follow better naming conventions
- `extism_plugin_register` -> `extism_plugin_new`
- `extism_output_get` -> `extism_plugin_output_data`
- `extism_output_length` -> `extism_plugin_output_length`
- `extism_call` -> `extism_plugin_call`
- Updates `extism_error` to return the context error when -1 issued for
the plug-in ID
- Adds `extism_plugin_free` to remove an existing plugin
- Updates SDKs to include these functions
- Updates SDK examples and comments
Co-authored-by: Steve Manuel <steve@dylib.so>
This PR updates `extism_output_get` to return an actual pointer to the
output value (`const uint8_t* extism_output_get(PluginIndex plugin)`
instead of `void extism_output_get(PluginIndex plugin, uint8_t *buffer,
uint64_t length)`), this pointer will only be valid until the next call,
but it makes it possible to access the output data without copying.
The input buffer is also not copied and the same issue applies:
the input buffer must not change during `call`.
Co-authored-by: Steve Manuel <steve@dylib.so>
This gives host the ability to re-use plugin descriptors instead of
loading a new plugin each time. The plugin memory and everything is
reset, so no state is shared with the newly loaded plugin.
Co-authored-by: Steve Manuel <steve@dylib.so>
I noticed when calling set_log_file that it's on the top-level namespace.
And so is the rest of the gem. It's not a good idea to do this as it
pollutes the user's application. Best practice is to wrap all code in a
module which by convention is the gem name.
I was also getting `Uninitialized constant FFI:NIL` and in the
set_log_file method and I could not tell why it was needed. I changed it
to a regular ruby nil. If it's needed for some reason I'll change it
back or move to another PR.
