Commit Graph

20 Commits

Author SHA1 Message Date
2372a472aa wasi-nn: make the host use the wasi_ephemeral_nn version of tensor_data (#4411)
the motivations:

* make the actual input size available to the backends.
  (currently the backends have to make a guess from shape/type.)

* make the host logic look a bit similar to wasi_ephemeral_nn.

this is a backend api/abi change.
2025-06-27 07:41:42 +08:00
71c07f3e4e deprecate legacy WAMR-specific "wasi_nn" module (#4382)
wasi_nn.h: deprecate legacy "wasi_nn"

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4326
2025-06-19 14:32:26 +08:00
aa53d648fa wasi-nn: fix tensor_data abi for wasi_ephemeral_nn (#4379)
it's "(list u8)" in the witx definition.

the new definition matches both of our own host definition
(struct tensor_wasm) and wasmtime.

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4352
2025-06-19 14:18:36 +08:00
20be1d33fe wasi_ephemeral_nn.h: prefix identfiers to avoid too generic names (#4358) 2025-06-17 11:15:01 +08:00
3a087c4244 wamr-wasi-extensions: add a cmake package to provide our wasi extension (#4344)
* wasi_ephemeral_nn.h: add a convenience wrapper header
* wamr-wasi-extensions: add a cmake package to provide our wasi extension

the sample app was tested with:
* wasmtime
* iwasm with https://github.com/bytecodealliance/wasm-micro-runtime/pull/4308

currently only contains wasi-nn.
maybe it makes sense to add lib-socket things as well.

cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4288
2025-06-12 09:33:25 +08:00
c932597057 wasi_nn_types.h: remove a seemingly stale comment (#4348) 2025-06-12 09:29:59 +08:00
4d6b8dcd5d wasi_nn.h: make this compatible with wasi_ephemeral_nn (#4330)
- wasi_nn.h: make this compatible with wasi_ephemeral_nn
cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4323

- fix WASM_ENABLE_WASI_EPHEMERAL_NN build
this structure is used by host logic as well.
ideally definitions for wasm and host should be separated.
until it happens, check __wasm__ to avoid the breakage.
2025-06-09 11:36:05 +08:00
933f8124b0 wasi-nn: fix the size of tensor->type (#4333)
* this enum is (@witx tag u8) in witx
* it seems that some wasm modules actually use non-zero padding
  and cause errors
* it's a bad practice to use C enum for ABI description anyway
2025-06-06 15:08:18 +08:00
769d16eaab wasi-nn: move some host-only things out of wasi_nn_types.h (#4334)
cf. https://github.com/bytecodealliance/wasm-micro-runtime/issues/4324
2025-06-06 15:07:29 +08:00
79cb4366ae wasi-nn: remove unused wasi_nn_dump_tensor_dimension prototype (#4325) 2025-06-05 09:48:28 +08:00
b20ebc2724 wasi_nn.h: add import_name attribute (#4328)
this would fix undefined symbol errors by making it clear
these functions are imported.

references:
e2c698c7e8/llvm/lib/MC/WasmObjectWriter.cpp (L1798-L1799)
e2c698c7e8/llvm/lib/Object/WasmObjectFile.cpp (L749-L752)
e2c698c7e8/lld/wasm/Symbols.cpp (L203)
e2c698c7e8/lld/wasm/Relocations.cpp (L36-L40)
2025-06-05 09:48:00 +08:00
ae6e490ad5 fix wasi-nn abi definitions (#4307)
sync with a more appropriate version of the definitions.

as we use the "wasi_ephemeral_nn", which is p1-based, it seems
more appropriate to use definitions from witx, not wit.

it's a bit unfortunate p2-based wasi-nn made gratuitous changes
like this from p1.

note: this is an ABI change.
2025-06-03 13:22:48 +08:00
aa1ff778b9 add load_by_name in wasi-nn (#4298) 2025-06-03 06:26:58 +08:00
0599351262 wasi-nn: Add a new target for llama.cpp as a wasi-nn backend (#3709)
Minimum support:
- [x] accept (WasmEdge) customized model parameters. metadata.
- [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml)
  - [x] basic
  - [x] chatml
  - [x] gemma
  - [x] llama
  - [x] qwen

---

In the future, to support if required:
- [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml)
  - [ ] command-r. (>70G memory requirement)
  - [ ] embedding. (embedding mode)
  - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output)
  - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`)
  - [ ] llava. (image representation)
  - [ ] llava-base64-stream. (image representation)
  - [ ] multimodel. (image representation)
- [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)
2024-09-10 08:45:18 +08:00
140ff25d46 wasi-nn: Apply new architecture (#3692)
ps.
https://github.com/bytecodealliance/wasm-micro-runtime/issues/3677
2024-08-13 09:14:52 +08:00
058bc47102 [wasi-nn] Add a new wasi-nn backend openvino (#3603) 2024-07-22 17:16:41 +08:00
db025e457a sync up with latest wasi-nn spec (#3530) 2024-06-17 14:58:09 +08:00
f844b33b2d Make wasi-nn backends as separated shared libraries (#3509)
- All files under *core/iwasm/libraries/wasi-nn* are compiled as shared libraries
- *wasi-nn.c* is shared between backends
- Every backend has a separated shared library
- If wasi-nn feature is enabled, iwasm will depend on shared library libiwasm.so
  instead of linking static library libvmlib.a
2024-06-14 12:06:56 +08:00
028f43bc18 Fix compilation warnings of wasi-nn (#3497) 2024-06-07 10:49:44 +08:00
ac9e789951 wasi-nn: Simplify cmake and headers' location (#2308)
Major changes:
- Public headers inside `wasi-nn/include`
- Put cmake files in `cmake` folder
- Make linux iwasm link with `${WASI_NN_LIBS}` so iwasm can enable wasi-nn
2023-06-26 09:29:05 +08:00