wasi-nn: Add a new target for llama.cpp as a wasi-nn backend (#3709)

Minimum support:
- [x] accept (WasmEdge) customized model parameters. metadata.
- [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml)
  - [x] basic
  - [x] chatml
  - [x] gemma
  - [x] llama
  - [x] qwen

---

In the future, to support if required:
- [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml)
  - [ ] command-r. (>70G memory requirement)
  - [ ] embedding. (embedding mode)
  - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output)
  - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`)
  - [ ] llava. (image representation)
  - [ ] llava-base64-stream. (image representation)
  - [ ] multimodel. (image representation)
- [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)
This commit is contained in:
liang.he
2024-09-10 08:45:18 +08:00
committed by GitHub
parent cb71ca5822
commit 0599351262
11 changed files with 949 additions and 122 deletions

View File

@ -442,7 +442,9 @@ if (WAMR_BUILD_WASI_NN EQUAL 1)
message (" WASI-NN enabled")
add_definitions (-DWASM_ENABLE_WASI_NN=1)
# Variant backends
if (NOT WAMR_BUILD_WASI_NN_TFLITE EQUAL 1 AND NOT WAMR_BUILD_WASI_NN_OPENVINO EQUAL 1)
if (NOT WAMR_BUILD_WASI_NN_TFLITE EQUAL 1 AND
NOT WAMR_BUILD_WASI_NN_OPENVINO EQUAL 1 AND
NOT WAMR_BUILD_WASI_NN_LLAMACPP EQUAL 1)
message (FATAL_ERROR " Need to select a backend for WASI-NN")
endif ()
@ -454,6 +456,10 @@ if (WAMR_BUILD_WASI_NN EQUAL 1)
message (" WASI-NN: backend openvino enabled")
add_definitions (-DWASM_ENABLE_WASI_NN_OPENVINO)
endif ()
if (WAMR_BUILD_WASI_NN_LLAMACPP EQUAL 1)
message (" WASI-NN: backend llamacpp enabled")
add_definitions (-DWASM_ENABLE_WASI_NN_LLAMACPP)
endif ()
# Variant devices
if (WAMR_BUILD_WASI_NN_ENABLE_GPU EQUAL 1)
message (" WASI-NN: GPU enabled")