If the user enabled multiple experiments, it does not make sense to
execute them concurrently. Therefore, we now only register all enabled
experiments and choose one experiment on fail-client startup. For this
selection, the user can either set the FAIL_EXPERIMENT environment
variable or invoke the tool with a specific argv[0]. For the
bin/-folder, we create symlinks for this mechanism.
With this change, the generic-tracing and generic-experiment can be
compiled into the same executable and we could distribute a "standard"
version of FAIL*.
When prefixing a symbol name with '?', the GenericExperiment does not abort
in case the symbol is not found in the provided ELF binary:
fail-client -Wf,--detected-marker=?eddiErrorHandler
[...]
[GenericExperiment] ELF Symbol not found, ignoring: eddiErrorHandler
Change-Id: Iec12416ce8e38ff0ee1704e3a725c2cadc97b756
The generic-experiment now learned to record and compare output on an
arbitrary serial port. Using Bochs' port 0xe9 hack (parameter
--e9-file) is kept for compatibility reasons.
Change-Id: I5b1aa02d244e8b474919e1bdf043e523ea0e4f45
Instead of using assert() (which only does something in a Debug
build), explicitly fail when a user-specified symbol is not found.
Change-Id: I33ac59ca4483ee65ba70c264b5153a7766a919d2
I did this mainly so server and client use a common networking API
IMO, using Boost::asio results in nicer name-lookup code.
Since no longer needed, I removed the SocketComm stuff.
The client is still synchronous; I see no benefit in having it
asynchronous.
I'm not super happy with the random backoff by the clients, if they
can't connect to the server. It makes the code really messy, 3 retries
is totally arbitrary, as is the backup windows. I believe launching
the server and clients in the correct order should be handled by a
launch script
Change-Id: Ifea64919fc228aa530c90449686f51bf63eb70e7
- search for libdwarf.h in new locations (e.g., /usr/include/libdwarf/)
- build Bochs with -std=gnu++98 (gnu++14 is default since GCC 6.1)
- specify "proto2" syntax for protobuf messages
- minor build-system and C++ namespace fixes
Change-Id: I16dbc622c797ef8e936fe3c0fb9b03029d27529d
Before this change, the GenericExperiment only recorded port 0xe9 output
*after* the fault was injected. When a fault was injected during the
workload's output loop, the output data before that point in time was
missing, and the experiment outcome was wrongly classified as SDC.
This change moves the logging activation to before the fast-forwarding
step (DatabaseExperiment::cb_before_fast_forward). It also makes sure the
DatabaseExperiment only clears its own listeners instead of also touching
the SerialOutputLogger's one.
Change-Id: I66bda4ee318d271ddda6f7ade4e817bf9d14cf46
Limit the serial-output logger buffer to prevent overly large memory
consumption in case the target system ends up, e.g., in an endless loop.
The buffer is limited to (golden-run output size)+1 to be able to detect
the case when the target system makes a correct output but faultily adds
extra characters afterwards.
Change-Id: I50c082f8fb09a702d87ab83732ca3e3463c46597
This change prevents an integer overflow in the memory-access listener
for WRITE_OUTERSPACE. Instead of matching all addresses above
maxima_data, l_mem_outerspace never matched in the
generic-experiment's "--catch-write-outerspace" mode.
Change-Id: I8f4ee4515af3998b7c2a8e83c7a18306c26d8d66
This change adds detection of SDCs to GenericTracing and
GenericExperiment via Bochs's I/O port E9.
Change-Id: Ie036aa97468b45cad94b6c8f73d1ef2d227547b2
Up until now only generic-tracing had the feature to directly
pass an ELF file to the experiment. generic-experiment lacked
that functionality and resorted to using the $FAIL_ELF_PATH
environment variable.
This change introduces the "--elf-file" command line argument
to generic-experiment.
Change-Id: Ie74de9e1781275ab247786856e13e412bac39224
ERIKA Enterprise is a OSEK conforming embedded RTOS. The supplied tracer
and experiment are similar to the cored-{tracing,tester} experiment, but
checks the integrity of the RTOS application in a different
manner. Stacks and stackpointers are located differently in ERIKA. This
experiment was used in RTAS'15 Hoffmann et al.
Change-Id: Idc8d874eb4d4ef15837f903270cfa521bc9514a2
With the instantiate-indirect.ah method, we can choose between different
experiment flows at runtime. By this, we can combine tracing and actual
injection into one fail-client binary. A -Wf,--mode={tester,tracer}
switch does hand the control to different experiment flows.
Change-Id: Ia268489ff6bc74dffea745b7aedcb36e262e8079
For redoing the bench-coptermock-isorc experiment, we have to change the
timeout settings. We now use a soft timeout setting. A soft timeout is
resetted after each checkpoint event. If a hard timeout (2 seconds) is
reached although the soft timeout was resetted, we also abort the injection.
Change-Id: Ib7c2b1ad201641f47434a11d3273dde797e0012e
This options performs a restore to the saved state of the machine immediately
after saving (default: off). This option is needed when the state is used by
other experiments that depend on the trace, which slighty differs without a
restore.
Change-Id: I4fdf4c5e03779bb9c6e0a0fa335ceae3e20608a5
The generic-tracing experiment now supports logging of I/O port access to file.
Therefore, the serialoutput plugin needs to be included in the experiment
configuration. Without the --serial-file option specified, logging is disabled.
Change-Id: I9e60d8ffd598ee04a50b4d92fc283f75382d478a
* Removed all command-line options.
* Read all required information from *-traceinfo.txt file or kernel elf file.
* Record error_corrected (but only in the 'OK' case).
* Add support for multiple variants (similar to the ecos experiment).
Change-Id: I933e52881fc6bee0750d8aaef813fe2539166b06
Due to a bug (most likely a copy and paste issue), the detected-marker
group was defined to point to the "FAIL_marker"-set, which would be
redundant. This commit will correctly map it to the "DETECTED_marker"
group.
Change-Id: I7de688357006ced1adf2423e213ae6633629cb81
The color_assert_port symbol does not exist in all dOSEK variant,
therefore we add the listener only if the symbol exists. Otherwise the
invalid handler will trigger on INV_ADDR
Change-Id: I7b81940a8413850527efb9e4bae86248794c622c
This change adapts several experiments, including the
DatabaseExperiment framework, to the restore() behavior update from
the previous change. Existing traces should continue to be usable.
This is not tested yet, mainly because I don't have access to most of
the experiment targets / guest systems necessary for testing. Please
test your own experiments if possible, or at least leave me a note
that you couldn't test it!
Especially the cored-voter/experiment.cc update may be broken, but
maybe the "FISHY" +2 in there was not OK in the first place.
Change-Id: I0c5daeabc8fe6ce0c3ce3e7e13d02195f41340ad
The GenericExperiment is a standard campaign+experiment pair. It is
derived from the DatabaseCampaign+DatabaseExperiment. Its experiment
endpoints are set on the command line, therefore it can be used to give
users a first impression of FAIL*. Currently it supports different
endpoints:
--trap: Catch all traps that occur and end the experiment
--timeout <N>: kill the experiment after N microseconds
--catch-write-text: detect writes on the text segment
--catch-write-outerspace: detect writes into nirvana
--{ok,fail,detected}-marker: groups of ELF symbols that are used as
execution breakpoints
Change-Id: Idc7fcf8875953f1007e1a37bacb086eddd29cd10
The inline configuration mechanism did work properly for iterated
invocation of cmake. But since the configuration items were set after
the decision was made whether a tool was build or not, they had only
influence on the second run of cmake.
The experiment configuration is no saved in a config.cmake file, which
populates the CMakeCache.txt before any other CMakeLists.txt is read.
Change-Id: I6eca1c6e462af3a241bd9c5b9a27a71a1f5d2829
The configuration from weather-monitor was also stripped down to those
values which do not represent the default settings.
Change-Id: I207306e555067156a4ed80edcbcd524137bc8e27
The configuration needed to build this experiment is no noted within the
experiment's CMakeLists.txt. This allows the easy building of this
experiment.
Change-Id: Ifddc4e6a8fa4e63c4656058166f9ef42e46e1edb
Campaign to perform fault injection experiments on the Fiasco microkernel. Required arguments:
- Variant (-v): The actual experiment variant (e.g. BASELINE)
- Benchmark (-b): The actual experiment benchmark
- Golden-Run (-g) : Specify whether the golden-run or the actual fault-injection experiment should be executed (only for fail-client)
- Stop address (-E): Address where the experiment should finish
- Timer-ticks (-T): Number of timer ticks from the golden run experiment
- Total-instructions (-t): Number of total instructions from the golden run experiment
- Ecc-panic-function-address (-p): Address of the Ecc-panic function in order to detect failures (if any fault detection is included)
- Errors_corrected variable address (-c): Address of the errors_corrected variable in order to determain if any error was corrected
Important: First run the generic-tracing experiment so there is a "state" folder where the actual experiment can start from and import/prune the resulting trace.
Change-Id: I151428ecc21f5e714cc923674ebbca9d84435704
This is a copy of the generic-tracing plugin for CoRedOS with the following
additions:
- random generator plugin is added if a "random_source" symbol is found
- checkpoint plugin is added if a "fail_trace" symbol is found. If stack
limits are defined by "_sstack" and "_estack" symbols this memory region
is checksummed on each checkpoint.
Change-Id: I00403ed917ad941d87ac2aeabd00c441135d9af4
This change renames the logger object used in weather-monitor from
"log" to "LOG" in order to circumvent potential naming conflicts
with e.g. the log() function from e.g. math.h/cmath.
Change-Id: I73758374f76ea5c29cb636ae3e0685a4a4cceacb
This commit modifies the weather-monitor experiment so that it
uses util/ElfReader instead of "nm -C" in a wrapper-script to
determine the various symbol addresses. experimentInfo.hpp now
only contains (rudimentary) configuration data for the experiment.
Furthermore this commit modularizes the experiment's code so that
there no longer is a humongous run() method.
Change-Id: I42461e1dabb5050af372810bb9fb61a6fccd0d99
