currently this is not necessary because context (WASINNContext) is
local to instance. (wasm_module_instance_t)
i plan to make a context shared among instances in a cluster when
fixing https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313.
this is a preparation for that direction.
an obvious alternative is to tweak the module instance context APIs
to allow declaring some kind of contexts instance-local. but i feel,
in this particular case, it's more natural to make "wasi-nn handles"
shared among threads within a "process".
note that, spec-wise, how wasi-nn behaves wrt threads is not defined
at all because wasi officially doesn't have threads yet. i suppose, at
this point, that how wasi-nn interacts with wasi-threads is something
we need to define by ourselves, especially when we are using an outdated
wasi-nn version.
with this change, if a thread attempts to access a context while
another thread is using it, we simply make the operation fail with
the "busy" error. this is intended for the mimimum serialization to
avoid problems like crashes/leaks/etc. this is not intended to allow
parallelism or such.
no functional changes are intended at this point yet.
cf.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/4313https://github.com/bytecodealliance/wasm-micro-runtime/issues/2430
as wasi-nn doesn't have these concepts, the best we can do without
risking breaking certain applications here is to pass through tensors
as they are.
this matches wasmtime's behavior.
tested with:
* wasmtime classification-example
(with this change, this example fails on tensor size mismatch
instead of implicitly resizing it.)
* license-plate-recognition-barrier-0007, a converted version
with non-fp32 output. [1]
(with this change, this model outputs integers as expected.)
[1] cd7ebe313b/models/public/license-plate-recognition-barrier-0007
sync with a more appropriate version of the definitions.
as we use the "wasi_ephemeral_nn", which is p1-based, it seems
more appropriate to use definitions from witx, not wit.
it's a bit unfortunate p2-based wasi-nn made gratuitous changes
like this from p1.
note: this is an ABI change.
it seems meaningless and quite confusing to access a table with
two aliases ("lookup" and "backends") within a function.
no functional changes are intended.
- All files under *core/iwasm/libraries/wasi-nn* are compiled as shared libraries
- *wasi-nn.c* is shared between backends
- Every backend has a separated shared library
- If wasi-nn feature is enabled, iwasm will depend on shared library libiwasm.so
instead of linking static library libvmlib.a
## Context
Currently, WAMR supports compiling iwasm with flag `WAMR_BUILD_WASI_NN`.
However, there are scenarios where the user might prefer having it as a shared library.
## Proposed Changes
Decouple wasi-nn context management by internally managing the context given
a module instance reference.
Add VX delegation as an external delegation of TFLite, so that several NPU/GPU
(from VeriSilicon, NXP, Amlogic) can be controlled via WASI-NN.
Test Code can work with the X86 simulator.
Remove restrictions:
- Only 1 WASM app at a time
- Only 1 model at a time
- `graph` and `graph-execution-context` are ignored
Refer to previous document:
e8d718096d/core/iwasm/libraries/wasi-nn/README.md
- Split logic in several dockers
- runtime: wasi-nn-cpu and wasi-nn- Nvidia-gpu.
- compilation: wasi-nn-compile. Prepare the testing wasm and generates the TFLites.
- Implement GPU support for TFLite with Opencl.
- Reorganize the library structure
- Use the latest version of `wasi-nn` wit (Oct 25, 2022):
0f77c48ec1/wasi-nn.wit.md
- Split logic that converts WASM structs to native structs in a separate file
- Simplify addition of new frameworks
