Updates the kernel to only scan for free blocks before a `memory.grow`
operation, this should improve performance of `alloc` by not iterating
over every block for each allocation.
Add `plugin.call_with_host_context` and `current_plugin.host_context`
methods, enabling per-call context to be looped from the guest invocation
to any host functions it calls. In an HTTP server environment, this enables
re-using a plugin across multiple requests while switching out backing
connections and user information in host functions. (Imagine a host
function, `update_user` -- previously the plugin would have to have been
aware of the user to pass to the host function. Now that information is
ambient.)
This is a backwards-compatible change and requires no changes to
existing plugins.
Implement by adding a global, mutable externref to the extism kernel.
Since most programming languages, including Rust, don't let you define
these natively, we accomplish this by using `wasm-merge` to combine the
kernel Wasm with Wasm generated by a WAT file containing only the
global.
(This pattern might be useful for other Wasm constructs we can't use
directly from Rust, like `v128` in argument parameters.)
Wasmtime requires extern refs to be `Any + Send + Sync + 'static`; we
additionally add `Clone`. I haven't tried this with an `Arc` directly,
but it should work at least for container structs that hold `Arc`'s
themselves.
- Adds quickcheck tests for alloc/free/load/store from the kernel, I
wasn't able to include these in the new wasm-bindgen tests because
`getrandom` isn't available on wasm32-unknown-unknown
- Fixes a bug in the calculation of how much memory is needed to
allocate the next block in the kernel. This bug is triggered when
allocating at the end of a page, the size of the MemoryBlock value
wasn't being taken in consideration when determining whether or not to
call memory.grow
This cosmetic change to the kernel adds two `u64` types for usage where
`Pointer` was used and it is actually something else or is more
restrictive (a pointer to the start of a data block (handle) or a
relative offset).
Instead of adding `Offset` to match `Length` I'd prefer to use `u64`
directly when referring to those scalars, but not sure if the those
types are preferred?
- Adds `length_unsafe` function to the extism kernel, a more performant
`length` for known-valid memory handles
After this is merged I will update go-sdk and js-sdk too.
Closes#643
Fixes#634
- Updates `extism_length` to walks the allocation list to determine
valid offsets instead of assuming the provided offset is valid
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
See https://github.com/extism/cpp-sdk/issues/15
- Limits a call to memset in the kernel to the size of the current
memory offset instead of the total size of memory.
- Adds `extism_plugin_reset` to the C API and `extism::Plugin::reset` to
Rust
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
See #504
Removes CI for the in-repo SDKs, we can remove the actual code closer to
the 1.0 release.
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
Fixes https://github.com/extism/extism/issues/537
- Requires wasmtime 14.0.0 or greater for coredump serialization
- Adds `EXTISM_COREDUMP` environment variable to write wasmtime coredump
to a file when an error occurs
- This will create a coredump from the main wasm module, which means it
doesn't give us too much insight into the kernel
- Adds `EXTISM_MEMDUMP` environment variable to write extism linear
memory to a file when an error occurs
- This gives us access to the kernel memory
- Adds some missing profiling options
- Converts timeouts to a Trap instead of a plain error, this helps us
get better information about where the timeout occured
- Some small improvements to error handling after a plugin call
- Adds a test for coredump and memdump generation
- Adds the ability to configure debug options using `PluginBuilder`
- Fixes memory layout and a wasted page of memory in the kernel, found
while debugging a memory dump
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
`MemoryBlock::next_ptr` used to calculate the offset to the data of the
next block, but it should be returning the start of the `MemoryBlock`
structure in memory. Also updates the tests for this case.
This was discovered when trying to update the kernel in the go-sdk:
https://github.com/extism/go-sdk/pull/32
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
- Adds benchmarking, run with `cargo bench` or `cargo criterion`
- Adds `MemoryRoot::pointer_in_bounds_fast` to do less precise bounds
checking in loads/stores
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
- Kernel-specific tests
- Add bounds checking to the kernel, which can toggled using the
`bounds-checking` feature
- Next week I will do some benchmarking to determine the runtime
performance impact of bounds checking and can consider if it should be
relaxed for certain operations (mainly stores/loads)
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
Same as #471 for `main` branch
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
Closes#469
This PR adds a github workflow that is triggered by updates to the
`kernel/` directory or `runtime/src/extism-kernel.wasm` - it builds the
kernel, including using `wasm-strip` and makes a PR against any PR that
has a kernel module that doesn't match the expected output.
I had considered making this run when the PR is merged into main, but
this approach gives us a chance to run CI with the generated wasm file.
I think automatically adding a commit would be simpler, but this way
seems more transparent.
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zshipko <zshipko@users.noreply.github.com>
This PR adds the `kernel` directory which contains a port of the Extism
memory allocator compiled to WebAssembly and removes
`runtime/src/memory.rs` completely.
Being able to re-use memory functions as a WASM module allows us to
begin to experiment with porting Extism to new runtimes!
This is in a draft state while I'm verifying some of these changes.
