Commit Graph

332 Commits

Author SHA1 Message Date
e170c355a2 Merge branch main into dev/wasi_threads 2023-02-17 08:46:12 +08:00
50650e4634 Modify poll_oneoff in libc-wasi to make it interruptible (#1951) 2023-02-16 21:03:33 +08:00
1c17665f68 Fix libc-wasi build failure when using clang (#1950)
Resolves #1949
2023-02-14 18:28:00 +08:00
531771eb11 Don't terminate other threads when create thread failed (#1955) 2023-02-13 16:23:24 +08:00
3698f2279b Improve wasm-c-api instantiation-time linking (#1902)
Add APIs to help prepare the imports for the wasm-c-api `wasm_instance_new`:
- wasm_importtype_is_linked
- wasm_runtime_is_import_func_linked
- wasm_runtime_is_import_global_linked
- wasm_extern_new_empty

For wasm-c-api, developer may use `wasm_module_imports` to get the import
types info, check whether an import func/global is linked with the above API,
and ignore the linking of an import func/global with `wasm_extern_new_empty`.

Sample `wasm-c-api-import` is added and document is updated.
2023-02-13 15:06:04 +08:00
e2c754f0a6 Fix data race when terminating or waiting for a thread (#1924)
There is a possibility that while iterating through a list of threads, the threads
finish by themselves, making a list of exec_env invalid.
2023-02-06 19:08:45 +08:00
17f3375d53 Enable gcc-4.8 compilation (#1928) 2023-02-02 19:34:20 +08:00
1614ce12fa wasi-nn: Enable GPU support (#1922)
- Split logic in several dockers
  - runtime: wasi-nn-cpu and wasi-nn- Nvidia-gpu.
  - compilation: wasi-nn-compile. Prepare the testing wasm and generates the TFLites.
- Implement GPU support for TFLite with Opencl.
2023-02-02 08:09:46 +08:00
58316635d4 ci: Enable libc-wasi compilation test on NuttX (#1918) 2023-01-28 12:39:14 +08:00
2eed50b303 Document how to use WASI threads in AOT mode (#1905)
Describe how to use WASI threads in AOT mode, following the discussion below:
  https://github.com/bytecodealliance/wasm-micro-runtime/pull/1867#discussion_r1070268062

Make aux stack boundary checks of wasi-threads always successful by setting
`exec_env->aux_stack_bottom` to UINT32_MAX and `exec_env->aux_stack_boundary` to 0
2023-01-27 18:32:33 +08:00
42f8fed20e Rename thread_spawn import (#1907)
Following the wit-defined ABI:
https://github.com/WebAssembly/wasi-threads/pull/26

cf. https://github.com/WebAssembly/wasi-libc/pull/387
2023-01-27 06:45:34 +08:00
9eed6686df Refactor WASI-NN to simplify the support for multiple frameworks (#1834)
- Reorganize the library structure
- Use the latest version of `wasi-nn` wit (Oct 25, 2022):
    0f77c48ec1/wasi-nn.wit.md
- Split logic that converts WASM structs to native structs in a separate file
- Simplify addition of new frameworks
2023-01-25 18:32:40 +08:00
c7141894fb Merge branch main into dev/wasi_threads 2023-01-21 13:18:55 +08:00
cadf9d0ad3 Main thread spread exception when thread-mgr is enabled (#1889)
And refactor clear_wasi_proc_exit_exception, refer to
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1869
2023-01-20 08:54:27 +08:00
e696ac36d7 Fix potential block issue in source debugger (#1887)
Fix issue reported in #1860
2023-01-17 08:45:29 +08:00
0e2382a959 Disable aux stack allocations for threads spawned by wasi_thread_start (#1867)
This syscall doesn't need allocating stack or TLS and it's expected from the application
to do that instead. E.g. WASI-libc already does this for `pthread_create`.

Also fix some of the examples to allocate memory for stack and not use stack before
the stack pointer is set to a correct value.
2023-01-09 20:36:34 +08:00
2615646c20 Fix on-heap aux stack allocation (#1865)
Because stack grows from high address towards low address, the value
returned by malloc is the end of the stack, not top of the stack. The top
of the stack is the end of the allocated space (i.e. address returned by
malloc + cluster size).

Refer to #1790.
2023-01-06 18:21:28 +08:00
4e5529f21f Reserve TID values for WASI threads (#1862)
According to the [WASI thread specification](https://github.com/WebAssembly/wasi-threads/pull/16),
some thread identifiers are reserved and should not be used. In fact, only IDs between `1` and
`0x1FFFFFFF` are valid.

The thread ID allocator has been moved to a separate class to avoid polluting the
`lib_wasi_threads_wrapper` logic.
2023-01-06 10:28:25 +08:00
3403f0a7f4 Merge main into dev/wasi_threads 2023-01-03 18:15:23 +08:00
d04f3988c3 libc-wasi: Fix spurious poll timeout (#1824)
This reverts the "Fix libc-wasi poll_oneoff hang issue" change.
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1300
2022-12-21 17:29:29 +08:00
7db49db777 Update release notes and fix issues reported by Coverity (#1813) 2022-12-20 14:37:07 +08:00
ead6478285 Add memory watchpoint support for source debugger (#1762)
Allow to add watchpoints to variables for source debugging. For instance:
`breakpoint set variable var`
will pause WAMR execution when the address at var is written to.
Can also set read/write watchpoints by passing r/w flags. This will pause
execution when the address at var is read:
`watchpoint set variable -w read var`

Add two linked lists for read/write watchpoints. When the debug message
handler receives a watchpoint request, it adds/removes to one/both of these
lists. In the interpreter, when an address is read or stored to, check whether
the address is in these lists. If so, throw a sigtrap and suspend the process.
2022-12-20 14:37:07 +08:00
0c85cb1fe6 feat(wasi-threads): Improve thread id allocator to reuse identifiers (#1809)
This PR allows reusing thread ids once they are released. That is done by using
a stack data structure to keep track of the used ids.
When a thread is created, it takes an available identifier from the stack. When
the thread exits, it returns the id to the stack of available identifiers.
2022-12-19 19:55:01 +08:00
0f637cff07 Enable aux stack allocations on application heap (#1799)
This is necessary for WASI threads as the aux stack should be managed by the application.
See https://github.com/bytecodealliance/wasm-micro-runtime/issues/1790 for details.
2022-12-19 10:58:12 +08:00
fb8727ba68 Update release notes and fix issues reported by Coverity (#1813) 2022-12-16 14:16:58 +08:00
929d5942b9 Add implementation for wasi_thread_spawn() (#1786)
For now this implementation uses thread manager.

Not sure whether thread manager is needed in that case. In the future there'll be likely another syscall added (for pthread_exit) and for that we might need some kind of thread management - with that in mind, we keep thread manager for now and will refactor this later if needed.
2022-12-13 21:33:27 +08:00
c3d66f916e Add memory watchpoint support for source debugger (#1762)
Allow to add watchpoints to variables for source debugging. For instance:
`breakpoint set variable var`
will pause WAMR execution when the address at var is written to.
Can also set read/write watchpoints by passing r/w flags. This will pause
execution when the address at var is read:
`watchpoint set variable -w read var`

Add two linked lists for read/write watchpoints. When the debug message
handler receives a watchpoint request, it adds/removes to one/both of these
lists. In the interpreter, when an address is read or stored to, check whether
the address is in these lists. If so, throw a sigtrap and suspend the process.
2022-12-07 17:18:28 +08:00
684ae6554d Create a placeholder for WASI threads implementation (#1783)
This a simpler version of the PR: https://github.com/bytecodealliance/wasm-micro-runtime/pull/1638
2022-12-06 21:11:27 +08:00
1652f22a77 Fix issues reported by Coverity (#1775)
Fix some issues reported by Coverity and fix windows exception
check with guard page issue
2022-12-01 19:24:13 +08:00
be7a4abee2 Fix source debugger error handling: continue executing when detached (#1725)
Change main thread hangs when encounter debugger encounters error to
main thread exits when debugger encounters error
Change main thread blocks when debugger detaches to
main thread continues executing when debugger detaches, and main thread
exits normally when finishing executing
2022-11-28 22:12:46 +08:00
93d3d09aa1 Clear some warnings and enable -Werror for NuttX (#1756) 2022-11-28 09:44:33 +08:00
29b76dd275 Create module hash for each module in SGX lib-rats (#1745)
Current SGX lib-rats wasm module hash is stored in a global buffer,
which may be overwritten if there are multiple wasm module loadings.
We move the module hash into the enclave module to resolve the issue.

And rename the SGX_IPFS macro/variable in Makefile and Enclave.edl to
make the code more consistent.

And refine the sgx-ra sample document.
2022-11-24 21:48:50 +08:00
1032aac60b Add wasm_runtime_get_wasi_exit_code (#1748)
Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/1738
2022-11-24 20:26:18 +08:00
2c5e7bc650 Bump tensorflow from 2.10.0 to 2.10.1 in /core/iwasm/libraries/wasi-nn/test (#1735)
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.10.0 to 2.10.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.10.0...v2.10.1)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 16:21:06 +08:00
656a8427e6 linux-sgx: Improve the remote attestation (#1695)
The current implementation of remote attestation does not take into
account the integrity of the wasm module. The SHA256 of the wasm
module has been put into user_data to generate the quote, and more
parameters are exposed for further verification.
2022-11-22 14:45:03 +08:00
e278861206 Avoid raising exception when debugging with VSCode (#1686) 2022-11-07 11:57:53 +08:00
52ebd98ff4 Fix a build failure error on nuttx (#1675)
NuttX doesn't have SO_REUSEPORT.
2022-11-03 16:29:58 +08:00
9ba8c8957d esp-idf: Add socket support for esp-idf platform (#1364)
Support to get/set recv_buf_size/send_buf_size/reuse_port/reuse_addr for wasm app
Add socket APIs for esp-idf platform
Add setsockopt for linux-sgx platform
2022-11-02 18:02:58 +08:00
3b4033aceb Allow source debugger reconnection (#1667)
Allow to wait for a new debugger connection once the previous one
is disconnected:
- when receiving a detach command
- when the client socket is closed (for example, lldb process is killed)
2022-11-02 17:12:34 +08:00
101ea0ea3c Update libuv version number (#1666)
Update the libuv version to work around a mingw64 bug, refer to:
  https://github.com/libuv/libuv/pull/3345
2022-11-01 11:07:35 +08:00
a182926a73 Refactor interpreter/AOT module instance layout (#1559)
Refactor the layout of interpreter and AOT module instance:
- Unify the interp/AOT module instance, use the same WASMModuleInstance/
  WASMMemoryInstance/WASMTableInstance data structures for both interpreter
  and AOT
- Make the offset of most fields the same in module instance for both interpreter
  and AOT, append memory instance structure, global data and table instances to
  the end of module instance for interpreter mode (like AOT mode)
- For extra fields in WASM module instance, use WASMModuleInstanceExtra to
  create a field `e` for interpreter
- Change the LLVM JIT module instance creating process, LLVM JIT uses the WASM
  module and module instance same as interpreter/Fast-JIT mode. So that Fast JIT
  and LLVM JIT can access the same data structures, and make it possible to
  implement the Multi-tier JIT (tier-up from Fast JIT to LLVM JIT) in the future
- Unify some APIs: merge some APIs for module instance and memory instance's
  related operations (only implement one copy)

Note that the AOT ABI is same, the AOT file format, AOT relocation types, how AOT
code accesses the AOT module instance and so on are kept unchanged.

Refer to:
https://github.com/bytecodealliance/wasm-micro-runtime/issues/1384
2022-10-18 10:59:28 +08:00
7cc7b56f88 Fix build socket api error (#1610)
Fix build socket api error introduced by #1590
2022-10-17 15:54:03 +08:00
a75295ec14 Minor fix in assembly script library (#1597)
And fix a code format issue introduced by last commit.
2022-10-14 12:54:25 +08:00
84e88e4f85 Small change to make init of vars buildable in clang and clang++ (#1590) 2022-10-14 10:22:25 +08:00
f28a15bf2f Wrap wasi_socket_ext api with extern "C" (#1575)
Fix wasi_socket_ext library failed to link with cxx project
2022-10-12 15:51:07 +08:00
e53ab91439 Integrate WASI-NN into WAMR (#1521)
Initial integration of WASI-NN based on #1225:
- Implement the library core/iwasm/libraries/wasi-nn
- Support TensorFlow, CPU, F32 at the first stage
- Add cmake variable `-DWAMR_BUILD_WASI_NN`
- Add test case based on Docker image and update document

Refer to #1573
2022-10-12 12:09:29 +08:00
5ddc335a7f Add timeout send/recv and multicast client/server socket examples (#1519)
Add a couple of socket examples that can be used with WAMR:
- The `timeout_client` and `timeout_server` examples demonstrate socket
  send and receive timeouts using the socket options
- The `multicast_client` and `multicast_server` examples demonstrate receiving
  multicast packets in WASM

And add several macro controls for `socket_opts` example.
2022-10-07 10:31:21 +08:00
5b10d4e630 socket: Explicit narrowing type cast and add missing static keywords (#1539)
While compiling the file wasi_socket_ext.c with pedantic options (typically
`-Wimplicit-int-conversion` and `-Wmissing-prototypes`), some warnings are raised.

This PR addresses those warnings by adding missing static statements before
functions and explicitly casting a narrowing conversion.

And fix the error handling after calling getpeername.
2022-09-29 22:44:57 +08:00
3fad613ea2 Enable build wasi_socket_ext.c with both clang and clang++ (#1527)
Enable to run WolfSSL into wasm and need some features from C++:
https://github.com/JamesMenetrey/wolfssl-examples/tree/wasm/Wasm
2022-09-29 20:52:11 +08:00
c505da7464 Update __wasi_sock_accept signature to match wasi_snapshot_preview1 (#1531)
The function was introduced to WASI about half a year ago after it already
existed in WAMR.

It caused problems with compiling `wasi_socket_ext.c` with the wasi-sdk
that already had this hostcall exported (wasi-sdk >= 15).

The approach we take is the following:
- we update WASI interface to be compatible with the wasi_snapshot_preview1
- compilation with `wasi_socket_ext.c` supports both wasi_sdk >= 15 and wasi_sdk < 15
  (although we intend to drop support for < 15 at one point of time)
- we override `accept()` from wasi-libc - we do that because `accept()` in `wasi-libc`
  doesn't support returning address (as it doesn't have `getpeername()` implemented),
  so `wasi_socket_ext.c` offers more functionality right now

Resolves #1167 and #1528.

[1] https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/witx/wasi_snapshot_preview1.witx
2022-09-29 09:29:54 +08:00