wasi-nn: Add a new target for llama.cpp as a wasi-nn backend (#3709)
Minimum support: - [x] accept (WasmEdge) customized model parameters. metadata. - [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [x] basic - [x] chatml - [x] gemma - [x] llama - [x] qwen --- In the future, to support if required: - [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [ ] command-r. (>70G memory requirement) - [ ] embedding. (embedding mode) - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output) - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`) - [ ] llava. (image representation) - [ ] llava-base64-stream. (image representation) - [ ] multimodel. (image representation) - [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)
This commit is contained in:
@ -442,7 +442,9 @@ if (WAMR_BUILD_WASI_NN EQUAL 1)
|
||||
message (" WASI-NN enabled")
|
||||
add_definitions (-DWASM_ENABLE_WASI_NN=1)
|
||||
# Variant backends
|
||||
if (NOT WAMR_BUILD_WASI_NN_TFLITE EQUAL 1 AND NOT WAMR_BUILD_WASI_NN_OPENVINO EQUAL 1)
|
||||
if (NOT WAMR_BUILD_WASI_NN_TFLITE EQUAL 1 AND
|
||||
NOT WAMR_BUILD_WASI_NN_OPENVINO EQUAL 1 AND
|
||||
NOT WAMR_BUILD_WASI_NN_LLAMACPP EQUAL 1)
|
||||
message (FATAL_ERROR " Need to select a backend for WASI-NN")
|
||||
endif ()
|
||||
|
||||
@ -454,6 +456,10 @@ if (WAMR_BUILD_WASI_NN EQUAL 1)
|
||||
message (" WASI-NN: backend openvino enabled")
|
||||
add_definitions (-DWASM_ENABLE_WASI_NN_OPENVINO)
|
||||
endif ()
|
||||
if (WAMR_BUILD_WASI_NN_LLAMACPP EQUAL 1)
|
||||
message (" WASI-NN: backend llamacpp enabled")
|
||||
add_definitions (-DWASM_ENABLE_WASI_NN_LLAMACPP)
|
||||
endif ()
|
||||
# Variant devices
|
||||
if (WAMR_BUILD_WASI_NN_ENABLE_GPU EQUAL 1)
|
||||
message (" WASI-NN: GPU enabled")
|
||||
|
||||
Reference in New Issue
Block a user