Commit Graph

491 Commits

Author SHA1 Message Date
c5e0825c6f Database: reduce varchar cols to fit MyISAM indexes
MyISAM indexes are limited to 1000 bytes per index.  Recently, Linux
distros (e.g. Debian 9) started to default MariaDB installations to
utf8mb4, which can use up to 4 bytes per character.  Hence, two
varchar columns indexed in a single key have a total maximum length of
250.  Instead, we use some lower, round numbers.

Change-Id: I4b53bc217912bc7070102a0af4938763e61b041d
2018-07-24 09:16:33 +02:00
ff3a5fb498 move to LLVM 3.9
This change removes support for earlier LLVM versions; making them
work as well is simply too tedious.

Change-Id: I372a151279ceb2bfd6de101c9e0c15f0a4b18c03
2018-07-24 09:15:33 +02:00
baaa6c3ce8 JobClient/Server fixes
- Retain original CLIENT_RETRY_COUNT semantics after Boost::Asio
  switch
- JobClient is C++11 now, too
- Message reception copy/paste error fixes

Change-Id: I19c474b2a79cd2ac8657e8d58d6170202d096fb0
2018-05-09 17:43:28 +02:00
9272c5cbed Move JobClient to Boost::asio as well
I did this mainly so server and client use a common networking API
IMO, using Boost::asio results in nicer name-lookup code.
Since no longer needed, I removed the SocketComm stuff.
The client is still synchronous; I see no benefit in having it
asynchronous.

I'm not super happy with the random backoff by the clients, if they
can't connect to the server. It makes the code really messy, 3 retries
is totally arbitrary, as is the backup windows. I believe launching
the server and clients in the correct order should be handled by a
launch script
Change-Id: Ifea64919fc228aa530c90449686f51bf63eb70e7
2018-05-09 17:41:52 +02:00
9ae8123433 JobServer: fix C++14 dependency
The recent Boost.Asio overhaul requires C++14 features, not only C++11.

Change-Id: I6decf0e6532956f7061d8a9021ec2c8406679266
2018-05-03 16:28:26 +02:00
6f41ad73d3 util: MemoryMap test failure more verbose
Change-Id: Ie42e1983d8cc5658b7e88d59cdbe689e6aefe9f2
2018-05-03 15:24:52 +02:00
6c120004eb Use boost-asio to improve FAIL* server performance
This patch overhauls the FAIL* server code to leverage Boost asio to be able to
handle a large number of clients (>4000). In this implementation the server is
now single threaded. I've not encountered any problems with this for up to
about 10k clients. Boost ASIO can also be used multithreaded, but I assume the
FAIL* internal data structures (Synchronized*) will become a bottleneck first.

The code now additionally depends on Boost Coro and Boost Context, as well as
a C++ 14 compiler, although the only C++14 feature required is a lambda capture
with initializer, such as [ x = std::move(x) ]. gcc-4.9.2 does this.

The code could (and probably should) be cleaned up more. Comments are wordy,
code is unnecessary now (multiple server threads), code is not self-contained
(headers spread dependencies), many ifdef's (server performance measuring
should be runtime rather than a compile time option), and much more. But for
this patch I was going for a minimal changeset the get the functionality in,
to have an easier review. Alas, FAIL* has no Unit-test suite to run the changes
against.

To handle such a large number of clients more changes were necessary, for
example server status output is now performed every 1s, instead for every
request.

The class Minion was removed completely; the only thing it was doing was
encapsulate an int.

The server has now a runtime-configurable port, or it can select a free port on
its own if none is specified. This requires the CampaignManager to add a port
argument and instantiate the JobServer dynamically.

Change-Id: Iad9238972161f95f5802bd2251116f8aeee14884
2017-09-15 06:26:14 +02:00
3ad42e270c fixes for Debian 9
- search for libdwarf.h in new locations (e.g., /usr/include/libdwarf/)
- build Bochs with -std=gnu++98 (gnu++14 is default since GCC 6.1)
- specify "proto2" syntax for protobuf messages
- minor build-system and C++ namespace fixes

Change-Id: I16dbc622c797ef8e936fe3c0fb9b03029d27529d
2017-08-01 14:12:03 +02:00
d0d62de3f4 sal: remove perf dependency to watchpoints/breakpoints
This change removes the hard compile-time dependency from the
performance-improving dedicated listener-list implementation
(core/sal/perf/) to basic watchpoints / breakpoints being enabled in
the cmake config.  This allows to keep the CONFIG_FAST_* switches
enabled in practically every experiment.

The primary reason for this change was the recent insight that enabled
breakpoints with disabled CONFIG_FAST_BREAKPOINTS can massively slow
down an experiment even if the latter does not use a single breakpoint
itself.

Change-Id: I5e3f5c1632ed1ee98a3ec887f18b174fa0e15773
2016-12-03 17:49:07 +01:00
d3d2faf680 globally rename Fail* to FAIL*
Change-Id: Ief2cb687cc69dd92c2e04f9314f0f1347e0a84ed
2016-07-26 17:41:32 +02:00
449ac1a692 DatabaseExperiment: local debug helper code
Change-Id: Ibf42c93df26f6123edc867147621a011665e9c43
2016-03-11 20:59:01 +01:00
39b120f7ca GenericExperiment: record output during complete runtime
Before this change, the GenericExperiment only recorded port 0xe9 output
*after* the fault was injected.  When a fault was injected during the
workload's output loop, the output data before that point in time was
missing, and the experiment outcome was wrongly classified as SDC.

This change moves the logging activation to before the fast-forwarding
step (DatabaseExperiment::cb_before_fast_forward).  It also makes sure the
DatabaseExperiment only clears its own listeners instead of also touching
the SerialOutputLogger's one.

Change-Id: I66bda4ee318d271ddda6f7ade4e817bf9d14cf46
2016-03-11 20:59:01 +01:00
ad558abeb6 DatabaseCampaign/-Experiment: add burst faults
This change introduces the ability to inject burst faults to
the DatabaseCampaign/-Experiment and thus to all derived
campaigns/experiments.

Change-Id: I491d021ed3953562bd7c908e9de50d448bc8ef33
2016-03-11 19:01:17 +01:00
748b0aea09 MemWriteListener: Set accesstype correctly
One construction of MemWriteListener did not set the MemAccessEvent type
correctly.

Change-Id: I34a34a34c1c23b2081d4749ee5e5372461c21717
2015-09-18 12:51:55 +02:00
f7c9917f7e database-experiment: no abort on injection_instr_absolute==NULL
The injection_instr_absolute can be NULL, if the trace was imported by
--faultspace-rightmargin R. The database-experiment then aborted the
injection, since a non present injection instruction is encoded as 0,
which is != 0.

Change-Id: I0abcbf102e8b26678ea574d6f73741c2cfac6781
2015-09-18 12:51:03 +02:00
d71db9211c DatabaseExperiment: fix headers
-  Add missing iomanip header: Without this one, Fail/gem5 does not
    compile.

 -  Remove unnecessary sal/bochs header: This seems to be a relic from
    when the DatabaseExperiment was Bochs-specific.

Change-Id: I91c991795c2c2e76359e9d11415f5119d225a4ab
2015-08-06 16:33:59 +02:00
1d9dae0e21 bochs: translate virtual to linear addresses
This change makes MemoryAccessListeners deliver linear addresses
instead of virtual ones deprived of their segment selector.  Even in
modern operating systems, segment selectors are still used for, e.g.,
thread-local storage.

The hooks within MemAccess.ah could maybe be implemented in a simpler
and less fragile way using the BX_INSTR_LIN_ACCESS instrumentation
hook, but this needs more investigation.

Change-Id: I0cee6271d6812d0a29b3a24f34d605a327ced7da
2015-07-31 12:46:06 +02:00
d38218f0eb DatabaseExperiment: remove Bochs dependency
Use the newly introduced SimulatorController::getCPUCount() instead of
BX_SMP_PROCESSORS to figure out the number of CPUs the back end provides.

Change-Id: I6d6521ae508154366ab5d0c23ddcb6f2de99aa04
2015-04-10 16:44:41 +02:00
ae15ac704d add missing headers
This change adds some missing headers needed for compiling the
PandaBoard variant, which seems to not have seen a compiler for a
while.

Change-Id: Ifb54abb4dc676fafc29ecbae97bafaa547fcfc80
2015-04-10 16:43:13 +02:00
96fae94b1f DatabaseExperiment: fix wrong variable scope
This fixes a wrong variable scope introduced in commit 193e5b7,
breaking compilation.

Change-Id: I74194e9ea6e726bc0a7ce2ee5ad5439b7de87fba
2015-04-10 15:07:45 +02:00
193e5b757e adapt experiments to new restore() behavior
This change adapts several experiments, including the
DatabaseExperiment framework, to the restore() behavior update from
the previous change.  Existing traces should continue to be usable.

This is not tested yet, mainly because I don't have access to most of
the experiment targets / guest systems necessary for testing.  Please
test your own experiments if possible, or at least leave me a note
that you couldn't test it!

Especially the cored-voter/experiment.cc update may be broken, but
maybe the "FISHY" +2 in there was not OK in the first place.

Change-Id: I0c5daeabc8fe6ce0c3ce3e7e13d02195f41340ad
2015-03-18 18:22:21 +01:00
91a9c6f688 core/sal: restore() more reliable for bochs
BochsController::restore() now recreates a state more expectable from
the experiment.  The state is now the same that save() leaves behind
in its most prominent use case after hitting a breakpoint.  This
change breaks backwards compatibility with some experiments, see
below!

Right after a breakpoint on a specific address fired and
BochsController::save() was called, another breakpoint on that
specific address would not fire again (unless that instruction is
executed again later on).

Up to this change, the situation after calling
BochsController::restore() was different:  A breakpoint on that
specific address would fire twice.  This difference led to the problem
that running the tracing plugin after save() would work fine
(recording the current instruction once, since 3dc752c "tracing: fix
loss of first dynamic instruction"), but running it after restore()
would record the current instruction *twice*.

This change aligns restore()'s behavior to that of save().  The
implications for existing experiments, traces and results are:

 -  Existing result data should be not affected at all, as
    trace.time1/time2 were correct before this change.  Nevertheless,
    the assumption time2-time1 >= instr2-instr1 does not hold for
    equivalence classes including the first instruction, if the latter
    was faultily recorded twice (see below).

 -  Existing traces that were recorded after a restore() (with a
    tracing plugin including the aforementioned commit 3dc752c)
    contain the first instruction twice.  An affected trace can be
    corrected with this command line:

      dump-trace old.tc | tail -n +2 | convert-trace -f dump -t new.tc

 -  For experiments that record traces after a restore() (such as
    ecos_kernel_test), nothing changes, as both the tracing and the
    fast-forwarding before the fault injection now see one instruction
    event less.

 -  Experiments that record traces after a save(), especially those
    that rely on the generic-tracing experiment for tracing, now see
    one instruction event less, before they need to inject their
    fault.  These experiments need to be adjusted, for example
    dciao-kernelstructs now should use bp.setCounter(injection_instr)
    instead of bp.setCounter(injection_instr+1).

Change-Id: I913bed9f1cad91ed3025f610024d62cfc2b9b11b
2015-03-06 08:38:40 +01:00
bd5802e5d7 core/sal: allow repeating BochsController::save
BochsController::save() now can in principle be called multiple times
in a row.  Not that this would really make sense, but the results are
consistent now.

Change-Id: Ib4c6eb571a364b0f7ea6142c8cfec004a12f98b3
2015-03-06 08:38:40 +01:00
d2899e8db7 core/sal: silence "unused function" warning
BochsHelpers.hpp is included by some aspect headers, which are implicitly
included into many (all?) translation units.  As in most TUs the "static
inline" defined getCPU function is not used, every time a "unused function"
warning was generated.

Change-Id: Ibb903fe7a11aaf1f455a626c8bf8b86f50857645
2015-02-09 11:02:40 +01:00
8973f65a50 util: don't leak resources from SumTree
This fixes the resource-leaking "should never happen" case when no
element is found by returning a notfound member.  Found by Coverity
Scan, CID 25555.

Change-Id: I9055ae0a3b31e61f3a8e3b098ec5613c3b5535f6
2015-02-07 18:20:40 +01:00
6a0214b132 ProtoStream: member variable -> local var
The contained state is not used over function boundaries anyways.
Found by Coverity Scan, CID 25689.

Change-Id: I34e42c227710be4859f6d62de9311c4201ed29b0
2015-02-07 18:20:39 +01:00
e99e4aafa8 JobServer: initialize sockaddr_in
This most probably is not a real problem, but does not take much work
to fix.  Found by Coverity Scan, in several reports.

Change-Id: I8bd12e3f7afeb4b1c4e1b057bdbd95da9aa9211c
2015-02-07 18:20:39 +01:00
8c2b6cf028 JobServer: fix socket leaks
Found by Coverity Scan, CID 25600.

Change-Id: Ic0c549928ce8058c145d178ed06b41b543676460
2015-02-07 18:20:30 +01:00
2c6aa6cd37 ElfReader: fix conditional free
buff cannot be zero.  Found by Coverity Scan, CID 25712.

Change-Id: Ibb411be376aab13832e3e5593fe001df1cbfc051
2015-02-07 17:29:48 +01:00
a19eb6db74 DwarfReader: fix premature fd close
When Richard decided we need not yet give up when dwarf_srclines()
fails, he left a -- now premature -- close(fd) behind.  Found by
Coverity Scan, CID 25806.

Change-Id: I0bc0cb6796225c9efaf5290e2799b6814f88e5b4
2015-02-07 17:29:44 +01:00
0e305034e9 Disassembler: restore ostream state after changing it
Found by Coverity Scan, CID 25679.

Change-Id: Ia0c175cd79ddc42ab9154c667db0d647618e41e2
2015-02-07 17:29:42 +01:00
141cefbaf4 DatabaseProtobufAdapter: remove unused struct field
Found by Coverity Scan, CID 25685.

Change-Id: Id7c9d756a7f824af41d1315740f7415f021eaa91
2015-02-07 17:29:41 +01:00
c499292e2f DatabaseProtobufAdapter: init top-level type bridge
Found by Coverity Scan, CID 25724.

Change-Id: I30ab9251bd0b6c60a64075a5cf934f40874a7c6c
2015-02-07 17:29:40 +01:00
26076d070c DatabaseProtobufAdapter: fix uninitialized members
Found by Coverity Scan, CID 25694.

Change-Id: I8fbb1434b73dcecb16675da870e73f1ad39391e4
2015-02-07 17:29:38 +01:00
0047919644 Database: fix tmp array deletion
Found by Coverity Scan, CID 25544.

Change-Id: Iaf0866ebd4a458fabfe79336d8c8b639e42147c5
2015-02-07 17:29:37 +01:00
cf6b481d5e DatabaseExperiment: fix uninitialized variable
Found by Coverity Scan, CID 25741.

Change-Id: I6105fe76afbfe0a6ec76b59778214683d7d754e2
2015-02-07 17:29:35 +01:00
a7184af37b core/sal: uninitialized BochsController member
False positive, nevertheless worth fixing.  Found by Coverity Scan,
CID 25723.

Change-Id: Ia4f1eb033d7cc8c20889280a59d8973ecf768933
2015-02-07 17:29:34 +01:00
61038c2578 core/sal: fix CPUState bracing error
This is why our coding style suggests braces even for single-line "if"
or loop bodies.  Found by Coverity Scan, CID 25767.

Change-Id: I53062286accba7c0fc9795ecea0a5e2795443321
2015-02-07 17:29:32 +01:00
fe9e25374a CampaignManager: initialize campaign member
Found by Coverity Scan, CID 25798.

Change-Id: Ib310ca3198c78a8e01d044d90ada1cd0c22b26d6
2015-02-07 17:29:29 +01:00
412ecbba63 dbcampaign: skip existing pilots with wrong fspmethod
Loading existing pilots with a different fspmethod_id is a waste of
time.

Change-Id: I3519a14822029999fa2ed854daff9853c0cbeec1
2015-01-21 14:53:33 +01:00
d58694521c dbcampaign: don't include fspmethod/variant ID in job msg
These IDs don't make sense by themselves but only after a lookup in the
database, which clients usually don't have (and don't need) access to.

Conflicts:
	src/core/comm/DatabaseCampaignMessage.proto.in

Change-Id: Ice739463552039b7fb48581722ea2e05984cea47
2015-01-21 14:53:32 +01:00
c422911741 dbcampaign: allow wildcard for prune method
Using mixed pruning methods now does not require to run the campaign
server twice anymore.

Change-Id: I3f62c269166b750892bb0e659ad0c180425d1479
2015-01-21 14:53:32 +01:00
336ffd6453 Merge branch 'sampling'
Change-Id: Id6be7954b0bafcbfd64ba702e69e016bd2810115
2015-01-21 14:53:28 +01:00
39df0a979e Merge branch 'listener-perf-bug'
Change-Id: Ia863115ec31011ad2b7a954543c8ee64188d3d32
2015-01-21 14:52:33 +01:00
99a923b11e Merge remote-tracking branch 'origin/master' into sampling-wip
Change-Id: Iae5c02be5801d75e8adc55222ccb35c559f7ebf4
2015-01-21 00:22:28 +01:00
f8e0f1bb3f util: add SumTree::iterator
Change-Id: I8304b64634fa3ab92a126fe5d942674b26334b3d
2015-01-21 00:17:48 +01:00
2f70e05db6 util: rename SumTree::get -> remove, add r/o get
SumTree::get now non-intrusively picks an element and returns a
reference to it, SumTree::remove removes and returns a copy.  The
former is needed for sampling with replacement.

Change-Id: Iefef2fdf0b7df6ea7a9949f2588528ec9e86bb7a
2015-01-21 00:17:48 +01:00
b0c58bab78 util: SumTree::add() documentation
This function copies the reference argument 'element' internally.

Change-Id: I33d94f224bc2b1b89057b90258d500eaa364ab85
2015-01-21 00:17:48 +01:00
a00c4d9a69 util/DwarfReader: plug file-descriptor leaks
This change fixes several file-descriptor leaks in the DwarfReader
implementation.  The patch is taken from Richard Hellwig's not yet
merged change I161f626d12ca7f2b7b9d13ba9cbc254eb55692f1.  I did not
apply any white-space cleanups to prevent conflicts when Richard's
change will be merged later.

Change-Id: Icd9c1bdeeab39e77900e2ce88b756a8cf7ade96a
2015-01-14 10:15:29 +01:00
d68ea990ca sal: catch listener removal corner case
This change makes sure an active but not-yet fired listener does not fire
anymore if it gets removed.  In this case, the listener already has an
invalid index, but still needs to be copied to the delete list.

This issue has not been observed in the wild, and is unlikely to have
caused real problems in the past.

Change-Id: I8be8a5b1d4cdc783a092b93f34f33b969894e5da
2015-01-13 19:22:49 +01:00