Commit Graph

414 Commits

Author SHA1 Message Date
d8ee771e28 Add Cosmopolitan Libc Platform (#2598)
This PR adds the Cosmopolitan Libc platform enabling compatibility with multiple
x86_64 operating systems with the same binary. The platform is similar to the
Linux platform, but for now only x86_64 with interpreter modes are supported.

The only major change to the core is `posix.c/convert_errno()` was rewritten to use
a switch statement. With Cosmopolitan errno values depend on the currently
running operating system, and so they are non-constant and cannot be used in array
designators. However, the `cosmocc` compiler allows non-constant case labels in
switch statements, enabling the new version.

And updated wamr-test-suites script to add `-j <platform>` option. The spec tests
can be ran via `CC=cosmocc ./test_wamr.sh -j cosmopolitan -t classic-interp`
or `CC=cosmocc ./test_wamr.sh -j cosmopolitan -t fast-interp`.
2023-10-04 21:55:37 +08:00
8987432f36 libc-wasi: Conditionally support SYNC flags (#2581)
To make it clearer to users when synchronization behaviour is not
supported, return ENOTSUP when O_RSYNC, O_DSYNC or O_SYNC are
respectively not defined. Linux also doesn't support O_RSYNC despite the
O_RSYNC flag being defined.
2023-10-04 21:02:25 +08:00
28ebd57400 Refactor stress tests to make them runnable in reactor mode (#2614) 2023-10-04 08:10:10 +08:00
1a88104160 Remove module instance from hashmap in wasi_nn_destroy (#2613)
When destroying wasi-nn context, module instance should be also
removed from hashmap to avoid memory leak.
2023-10-03 08:33:11 +08:00
913a1414ad Add support for closing/renumbering preopen fds (#2578)
There doesn't appear to be a clear reason not to support this behavior.
It seems it was disallowed previously as a precaution. See
67e2e57b02
for more context.
2023-09-26 09:31:32 +08:00
f474f3d668 Return __WASI_EINVAL from fd_prestat_dir_name (#2580)
Return a WASI error code (rather than a host POSIX one). In addition,
there is no need to return an error in the case that the provided buffer
is too large.
2023-09-25 18:40:12 +08:00
06527f724e Remove provision of unnecessary fd rights (#2579)
The WASI docs allow for fewer rights to be applied to an fd than requested but
not more. This behavior is also asserted in the rust WASI tests, so it's necessary
for those to pass as well.
2023-09-22 09:11:25 +08:00
444b159963 Implement async termination of blocking thread (#2516)
Send a signal whose handler is no-op to a blocking thread to wake up
the blocking syscall with either EINTR equivalent or partial success.

Unlike the approach taken in the `dev/interrupt_block_insn` branch (that is,
signal + longjmp similarly to `OS_ENABLE_HW_BOUND_CHECK`), this PR
does not use longjmp because:
* longjmp from signal handler doesn't work on nuttx
  refer to https://github.com/apache/nuttx/issues/10326
* the singal+longjmp approach may be too difficult for average programmers
  who might implement host functions to deal with

See also https://github.com/bytecodealliance/wasm-micro-runtime/issues/1910
2023-09-20 18:11:52 +08:00
a6fda9b7ab Add an API to terminate instance (#2538)
Add API wasm_runtime_terminate to terminate a module instance
by setting "terminated by user" exception to the module instance.

And update the product-mini of posix platforms.

Note: this doesn't work for some situations like blocking system calls.
2023-09-13 17:09:31 +08:00
f128150d43 fd_object_release: Preserve errno (#2535)
Preserve errno because this function is often used like
the following. The caller wants to report the error from the main
operation (`lseek` in this example), not from fd_object_release.

```
    off_t ret = lseek(fd_number(fo), offset, nwhence);
    fd_object_release(fo);
    if (ret < 0)
        return convert_errno(errno);
```
2023-09-08 12:26:32 +08:00
6c846acc59 Implement module instance context APIs (#2436)
Introduce module instance context APIs which can set one or more contexts created
by the embedder for a wasm module instance:
```C
    wasm_runtime_create_context_key
    wasm_runtime_destroy_context_key
    wasm_runtime_set_context
    wasm_runtime_set_context_spread
    wasm_runtime_get_context
```

And make libc-wasi use it and set wasi context as the first context bound to the wasm
module instance.

Also add samples.

Refer to https://github.com/bytecodealliance/wasm-micro-runtime/issues/2460.
2023-09-07 14:54:11 +08:00
af2f3c8759 Fix typo for IP address buffer (#2532) 2023-09-07 14:29:52 +08:00
151dfae49c Unifdef -U WASMTIME_SSP_STATIC_CURFDS (#2533)
The macro WASMTIME_SSP_STATIC_CURFDS isn't used in the source code,
remove the related macro control code.
2023-09-06 15:25:58 +08:00
48b71a05fb libc-wasi: Remove unused code (#2528) 2023-09-04 19:14:56 +08:00
9e3904337e Revert "Return error when exception was raised after main thread finishes" (#2524)
This reverts commit 71d43f3ca1.
2023-09-04 18:01:09 +08:00
2e78230e06 set_exception_visitor: Remove the special case for wasi proc exit (#2525)
While wasi proc exit is not a real trap, what the runtime does on it is mostly same as
real traps. That is, kill the siblings threads and represent the exit/trap as the result of
the "process" to the user api. There seems no reason to distinguish it from real traps
here.

Note that:
- The target thread either doesn't care the specific exception type or ignore wasi
  proc exit by themselves. (clear_wasi_proc_exit_exception)
- clear_wasi_proc_exit_exception only clears local exception.
2023-09-04 16:52:19 +08:00
8c2dc1d011 Add unit tests for the tid allocator (#2519)
Add simple infrastructure to add more unit tests in the future. At the moment tests
are only executed on Linux, but can be extended to other platforms if needed.

Use https://github.com/google/googletest/ as a framework.
2023-09-04 13:21:10 +08:00
382d52fc05 Stop abusing shared memory lock to protect exception (#2509)
Use a separate global lock instead.

Fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/2407
2023-08-31 20:39:08 +08:00
53d7027de0 Implement strict validation of thread IDs according to the specification (#2521) 2023-08-31 20:23:54 +08:00
411b903cee Add mutex stress test (#2472)
As a part of stress-testing we want to ensure that mutex implementation is working
correctly and protecting shared resource to be allocated from other threads when
mutex is locked.

This test covers the most common situations that happen when some program uses
mutexes like locks from various threads, locks from the same thread etc.
2023-08-30 19:01:44 +08:00
ff151fb7ba sgx-ra: Disable the building of samples (#2507) 2023-08-28 09:42:16 +08:00
b7a9da1620 Merge branch main into dev/wasi-libc-windows 2023-08-23 17:05:19 +08:00
e2f8721ec9 Fix issues reported by Coverity and clear windows warnings (#2467) 2023-08-17 10:54:02 +08:00
4ce675aacd Add another wamr test (#2411)
Follows up #2364 where we discussed that we might want to have a test
which has really short thread function and creates many threads.
2023-08-17 10:31:05 +08:00
a9bd6af0ac Build wasi-libc from source in WAMR CI (#2465)
We need to apply some bug fixes that were merged to wasi-libc because wasi-sdk-20
is about half a year old.

It is a temporary solution and the code will be removed when wasi-sdk 21 is released.
2023-08-15 10:36:10 +08:00
365cdfeb71 Upgrade SGX-RA integration for 0.1.2 and Ubuntu 20.04 (#2454) 2023-08-15 10:12:43 +08:00
b45d014112 wasi-nn: Improve TPU support (#2447)
1. Allow TPU and GPU support at the same time.
2. Add Dockerfile to run example with [Coral USB](https://coral.ai/products/accelerator/).
2023-08-14 20:03:56 +08:00
0b0af1b3df wasi-nn: Support uint8 quantized networks (#2433)
Support (non-full) uint8 quantized networks.
Inputs and outputs are still required to be `float`. The (de)quantization is done internally by wasi-nn.

Example generated from `quantized_model.py`:
![Screenshot from 2023-08-07 17-57-05](https://github.com/bytecodealliance/wasm-micro-runtime/assets/80318361/91f12ff6-870c-427a-b1dc-e307f7d1f5ee)

Visualization with [netron](https://netron.app/).
2023-08-11 07:55:40 +08:00
51714c41c0 Introduce WASMModuleInstanceExtraCommon (#2429)
Move the common parts of WASMModuleInstanceExtra and
AOTModuleInstanceExtra into the new structure.
2023-08-08 09:35:29 +08:00
a07d8160f9 Merge pull request #2426 from bytecodealliance/main
Merge branch main into dev/wasi-libc-windows
2023-08-06 08:50:21 +08:00
91592429f4 Fix memory sharing (#2415)
- Inherit shared memory from the parent instance, instead of
  trying to look it up by the underlying module. The old method
  works correctly only when every cluster uses different module.
- Use reference count in WASMMemoryInstance/AOTMemoryInstance
  to mark whether the memory is shared or not
- Retire WASMSharedMemNode
- For atomic opcode implementations in the interpreters, use
  a global lock for now
- Update the internal API users
  (wasi-threads, lib-pthread, wasm_runtime_spawn_thread)

Fixes https://github.com/bytecodealliance/wasm-micro-runtime/issues/1962
2023-08-04 10:18:13 +08:00
cb6d85069e Fix lib-pthread issues (#2410)
- Avoid destroying module instance repeatedly in pthread_exit_wrapper and
  wasm_thread_cluster_exit.
- Wait enough time in pthread_join_wrapper for target thread to exit and
  destroy its resources.
2023-08-01 18:11:58 +08:00
b88f2c06c6 Add initial stress test (#2364)
We need to make a test that runs longer than the tests we had before to check
some problems that might happen after running for some time (e.g. memory
corruption or something else).
2023-08-01 17:38:37 +08:00
84ecf15322 Update libuv version to v1.46.0 (#2405) 2023-07-31 18:32:20 +08:00
151600fef2 Fix wasi-sockets tests (#2389)
Tests were failing because the right permissions were not provided to iwasm.
Also, test failures didn't trigger build failure due to typo - also fixed in this change.

In addition to that, this PR fixes a few issues with the test itself:
* the `server_init_complete` was not reset early enough causing the client to occasionally
  assume the server started even though it didn't yet
* set `SO_REUSEADDR` on the server socket so the port can be reused shortly after
  closing the previous socket
* defined receive-send-receive sequence from server to make sure server is alive at the
  time of sending message
2023-07-30 19:34:09 +08:00
5463ea4a20 Temporarily disable code that does not compile on windows (#2386)
And return ENOSYS. We do that so we can at least compile the code on CI.
We'll be gradually enabling more and more functions.

Also, enabled `proc_raise()` for windows.
2023-07-25 10:35:05 +08:00
fe4ee37122 Add more fixes for wasi libc on windows (#2380)
* disable translations of errno codes that aren't defined on Windows
* undef `min()` macro if it is defined to not conflict with the `min()` function we define
* implement `shed_yield` wasi call
* disable some of the features in the config for windows by default
2023-07-24 15:14:52 +08:00
0f4edf9735 Implement suspend flags as atomic variable (#2361)
We have observed a significant performance degradation after merging
https://github.com/bytecodealliance/wasm-micro-runtime/pull/1991
Instead of protecting suspend flags with a mutex, we implement the flags
as atomic variable and only use mutex when atomics are not available
on a given platform.
2023-07-21 08:27:09 +08:00
662e38e9b0 [Partial] Build wasi-libc for Windows (#2338)
Build wasi-libc library on Windows since libuv may be not supported. This PR is a first step
to make it working, but there's still a number of changes to get it fully working.
2023-07-13 21:02:29 +08:00
a9b76228b0 Allow defining hints without exact socket type or address family (#2337) 2023-07-04 21:21:21 +08:00
6ed4c9c908 Increase default native stack size (#2332)
Calling `__wasi_sock_addr_resolve` syscall causes native stack overflow.
Given this is a standard function available in WAMR, we should have at least
the default stack size large enough to handle this case.

The socket tests were updated so they also run in separate thread, but
the simple retro program is:

```C
void *th(void *p)
{
    struct addrinfo *res;
    getaddrinfo("amazon.com", NULL, NULL, &res);
    return NULL;
}

int main(int argc, char **argv)
{
    pthread_t pt;
    pthread_create(&pt, NULL, th, NULL);
    pthread_join(pt, NULL);
    return 0;
}
```
2023-07-03 21:02:10 +08:00
ab96e01f5e wasi-nn: Add support of wasi-nn as shared lib (#2310)
## Context

Currently, WAMR supports compiling iwasm with flag `WAMR_BUILD_WASI_NN`.
However, there are scenarios where the user might prefer having it as a shared library.

## Proposed Changes

Decouple wasi-nn context management by internally managing the context given
a module instance reference.
2023-06-27 18:18:26 +08:00
f5c5a83331 wasi-nn: Improve tests paths for local dev (#2309)
## Context

Path to models use `/assets` for testing inside docker. While testing directly from
the repo we are forced to use soft-links or modify the paths.

## Proposed Changes

Use relative path and adjust docker volumes in docs.
2023-06-27 08:07:30 +08:00
ac9e789951 wasi-nn: Simplify cmake and headers' location (#2308)
Major changes:
- Public headers inside `wasi-nn/include`
- Put cmake files in `cmake` folder
- Make linux iwasm link with `${WASI_NN_LIBS}` so iwasm can enable wasi-nn
2023-06-26 09:29:05 +08:00
f51d98f850 Use system libuv if available (#1861)
This PR attempts to search for the system libuv and use it if found instead of
downloading it. As reported in #1831, this is needed because some tools
build in a sandbox and clear the extra sources.
2023-06-26 08:37:10 +08:00
8f82b1e8a2 Fix dockerfile linter warnings (#2291) 2023-06-15 16:52:48 +08:00
bd96696236 Appease unused warning on min_uint64 (#2277) 2023-06-09 08:40:21 +08:00
6e3c3fe9ec Fix build error with LLVM 16 (#2259) 2023-06-06 13:45:18 +08:00
1e5f206464 Fix compile warnings on windows platform (#2208) 2023-05-15 13:48:48 +08:00
0899638ece Use a manual flag to disable clock_nanosleep on the unsupported platforms (#2176)
For some platforms WAMR gets compiled with `CONFIG_HAS_CLOCK_NANOSLEEP=1`,
while `clock_nanosleep` is not present at the platform, which causes compilation error.

Add check for macro `DISABLE_CLOCK_NANOSLEEP` to resolve the issue, only when
the macro isn't defined can the macro `CONFIG_HAS_CLOCK_NANOSLEEP` take effect.
2023-05-12 07:49:21 +08:00