Commit Graph

1157 Commits

Author SHA1 Message Date
5ee96032c9 jobserver: gracefully handle thread creation failures
Due to the previous DatabaseCampaign fix, this may not be necessary
anymore, but it's nevertheless a good idea to handle thread creation
failures properly.

Change-Id: I8317a77dd5338509727e737040944320e7755ae3
2014-02-25 13:32:56 +01:00
25a390970a DatabaseCampaign: avoid table locking
It is necessary to copy pilot IDs of existing results to a temporary table
before fetching undone jobs from the DB: Otherwise, due to MyISAMs
table-level locking, collect_result_thread() will block in INSERT (SHOW
PROCESSLIST state "Waiting for table level lock") until the (streamed)
pilot query finishes.  As one pilot query follows after the other,
collect_result_thread() may even starve until the memory for the
JobServer's "done" queue runs out, resulting in a crash and the loss of all
queued results.

Change-Id: Ib0ec5fa84db466844b1e9aa0e94142b4d336b022
2014-02-25 13:32:55 +01:00
b094753fde doc: missing libraries
Change-Id: Ife89a0b3cc74433e4fb711580c5eb3cd82467081
2014-02-25 13:32:55 +01:00
bc2103c527 sal/bochs: don't show errors in non-verbose mode
The patched eCos variant we analyze intentionally overflows the 16550
UART FIFOs, flooding the terminal with Bochs error messages.  Enabling
CONFIG_BOCHS_NON_VERBOSE now also enforces ignoring error messages,
regardless of log verbosity settings in the bochsrc.

Change-Id: If14e2532234e61bf60720a45150ef4973e8d508b
2014-02-25 13:32:55 +01:00
953fbe2156 serialoutput: cleanup
Change-Id: I255a3dd44fcf075a441461a883f564ee2d626ee1
2014-02-25 13:32:55 +01:00
455c088cd9 serialoutput: optional character limit
This prevents unlimited memory consumption in case the guest system
enters an endless loop.

Change-Id: Ia1bb178f7d8cb8ad8bf958210d90f6d7c2e11359
2014-02-25 13:32:55 +01:00
c319f3458c serialoutput: consistent plugin class naming
Change-Id: I8abe0cfdebecb0adc7229e29bd241da65b27105a
2014-02-25 13:32:55 +01:00
36ae6fd6c3 prune-trace: use none/none only without any parameters
Before this change, running prune-trace with, e.g.
"prune-trace -d fsp_mibench -v bitmap% --benchmark-exclude clockcnv"
resulted in an implied "--benchmark none", rendering --benchmark-exclude
ineffective and resulting in nothing being pruned.  Now, the "none" default
only applies when neither --benchmark nor --benchmark-exclude (analogously
for --variant / --variant-exclude) is provided.

Change-Id: Ic7c88919d7cfde1261749a745dc6a679472ff348
2014-02-25 13:32:55 +01:00
1df43e9726 import-trace: major speedup
Using Database::insert_multiple() instead of prepared statements
speeds up trace import by a factor of 3-4.  While being there, we now
properly deal with nonexistent extended trace values (i.e., put NULLs
into the DB).

Side note: The ElfImporter should switch to insert_multiple(), too.

Change-Id: I96785e9775e3ef4f242fd50720d5c34adb4e88a1
2014-02-25 13:32:55 +01:00
76bda55c5e Merge "plugins/randomgenerator: add deterministic PRNG plugin" 2014-02-20 11:26:35 +01:00
69ba9e0f94 dump-trace: properly deal with empty extended trace entries
As dereferencing register contents is not always possible, extended trace
entries may be empty.

Change-Id: I603fcef2eb2b0429a9d6ed0469441bc314e365fd
2014-02-19 19:08:46 +01:00
b6fc98abae generic-tracing: remove --save-symbol
At least for the Bochs backend there might be side effects when saving
the simulator state while tracing, which therefore should be avoided.
As there is no known use-case for using a --save-symbol different to
--start-symbol, this change disables the semantics behind
--save-symbol completely and only keeps the command-line switch for
backward compatibility reasons (existing automatic test scripts etc.).
The generic-tracing experiment now complains and aborts if a
--save-symbol different to --start-symbol is given.

Change-Id: I6072d846be96e016534cc83db375a400cfc25303
2014-02-19 19:08:46 +01:00
836325e74b generic-tracing: cleanups
Change-Id: I5c3d1131248910228cb4fee44cf107c750c01e21
2014-02-19 19:08:46 +01:00
85152238da tracing: fix endless loop when only tracing mem accesses
With m_tracetype=TRACE_MEM, bool first was never reset to false in the
tracing plugin's main loop.  This bug was most probably never
triggered, though, as nobody only traces memory accesses.

This change also slightly simplifies the internal logic in the tracing
plugin.

Change-Id: I65d7df6a3781ec552cfb892bbf3394b421e227f1
2014-02-19 19:08:46 +01:00
b82e547b53 plugins/randomgenerator: add deterministic PRNG plugin
A simple plugin which deterministically returns a new random value
each time the specified symbol is read.

Change-Id: I6ccac421fc064f02a88e8b126f8a26044d1f51c6
2014-02-18 16:40:34 +01:00
01c1321b48 tracing: bugfix for mem dereferences at mapping boundary
As we copy a 32-bit word from the dereferenced address, we also need to
check whether address+3 is also mapped.  (Yes, I've seen this in the
wild.)

Change-Id: I43f891c56e077333670c9cb48c0ee8e9342fa41d
2014-02-17 23:24:16 +01:00
58fa4c59cc sal/bochs: fix handling of unmapped memory
Up to now, BochsMemory::isMapped() always returned true in 32-bit protected
mode with a 4GB linear address space (as used by, e.g., eCos), even for
addresses greater than the configured memory size.  This led to lots of
bogus memory dereferences in the (extended) tracing plugin.

This change (a follow-up to commit 5171645) additionally checks the return
value of getHostMemAddr(), and announces BX_RW (read/write access) instead
of BX_READ as the intended type of memory access.  In the aforementioned
scenario, memory addresses greater than the memory size are now correctly
detected as "not mapped".

Change-Id: Ic2fa7554c869cb90191164535a601bae4dbb49b6
2014-02-17 23:24:16 +01:00
e4bf980b97 Fail* result browser for pruning experiments.
Based on the database layout given by the pruner.
Run ./run.py -c <path to mysql.cnf>
(Default config ~/.my.cnf)

- Checks if objdump table exists
- Added view for results per instruction
- Added config file support for table details
- Overview data loaded at server startup
- Result type mapping configurable via config file

Based on Flask and MySQLdb

Change-Id: Ib49eac8f5c1e0ab23921aedb5bc53c34d0cde14d
2014-02-17 10:17:25 +01:00
b4f144745a Revert "import-trace: emit warning for malformed traces"
Memory accesses that don't belong to the preceding IP event in the
trace *do* have a use case:  a hardware interrupt causes the CPU to
push its state onto the (kernel) stack.  At the moment we cannot
distinguish this case from a malformed trace (as we don't record the
occurrence of interrupts), hence this warning needs to be disabled for
now.

This reverts commit 84edd02b6f.
2014-02-11 14:57:29 +01:00
8942740d9a Merge branch 'memorymap' 2014-02-09 14:20:53 +01:00
44a0359087 Adds KESO garbage collector experiment
Change-Id: I429b58912b46b2ab7213e18651f14f7aed303419
2014-02-07 12:55:32 +01:00
26308dc9ea Merge "Removes serial.out from bochs config for massive performance boost on cluster runs." 2014-01-31 13:52:45 +01:00
4ba028e740 Removes serial.out from bochs config for massive performance boost on
cluster runs.

If this output file is enabled, all running processes try to write to the
same file on the shared filesystem. They block each other which leads to
massive I/O wait time and CPU idle time.

This change reduces the runtime e.g. from several hours (12+) to few minutes
(20).

Change-Id: I028628af31c845fc517e5daca5b4f981eade3cf4
2014-01-31 12:51:58 +01:00
4b921a5fe3 util: MemoryMap test
Change-Id: I54680685326a85cfd723a47e8aef8d71662c9aeb
2014-01-30 15:26:20 +01:00
4bcce14659 util: space-efficient MemoryMap
We now use boost::icl::interval_set internally, consuming extremely
lower amounts of memory.  boost::icl was introduced with Boost 1.46;
Debian 7.0 comes with 1.49, so this dependency should be no problem
anymore.

Both the class interface and the memory-map file format stay the same.

Change-Id: I38e8148384c90aa493984d0f6280817df00f1702
2014-01-30 15:26:12 +01:00
119ae40be9 util/Database: added a wrapper function for mysql_real_escape_string()
Change-Id: I999aad3c35c5f389fa3acfe8d7a11c417c478787
2014-01-28 11:07:34 +01:00
13175c259b import-trace: import debug info
If the --debug option is set, the line number table of the elf binary will
be imported into the database. The information will be stored in the
"dbg_mapping" table.

If the --sources option is set, the source files will be imported
into the database. Only the files that were actually used in the
elf binary will be imported.

Change-Id: I0e9de6b456bc42b329c1700c25e5839d9552cdbb
2014-01-28 11:07:34 +01:00
d307dd2ecb dciao-kernelstructs: reuse sobres experiment for ISORC2014
Differences:

- the task activation order is determined in the faulty experiment as
  well as in the golden run (which is now done by
  fail-generic-tracing) by observing a variable fail_virtual_port.
- There is a panic value read from the fail_virtual_port
- The golden run task activation is determined by giving an extended
  trace to task_activation.py. The script collects all writes to
  fail_virtual_port, and determines the activation from this.

Change-Id: Id401b78933b45a4b2cf031fc0a8b5ac90151ec24
2014-01-27 10:32:09 +01:00
c48c7296fb util/WallclockTimer: bugfix: include ostream
This only compiled everywhere because all users included (i)ostream.

Change-Id: I29b0fb13a01606fdffd8ebdb9701eff652065916
2014-01-24 20:33:32 +01:00
85e3911202 Merge branch 'ubuntu-saucy-fixes' 2014-01-24 17:02:44 +01:00
5df364dea2 compute-hops: re-enabled memory usage lookup
Change-Id: I51c7dcac5e6077364da63f4189fea9a90c43e059
2014-01-23 18:53:21 +01:00
5079cd7e31 doc: documentation of failpanda
Change-Id: I05a57942daaa66d146db754fc3ddc72707e55dd9
2014-01-23 18:53:21 +01:00
e1aae23c64 openocd: straighten up wrapper code
Change-Id: I3cec9af50ea9e3f10385839890b84d4e71d1fddb
2014-01-23 18:53:21 +01:00
0e7ee03937 compute-hops: bugfix rollback-threshold
Rollback threshold was not set by command line argument.

Change-Id: I68ba63cc5cd31e3af8408f83caec8b3d7bb10c78
2014-01-23 18:53:21 +01:00
99bcd93ffb openocd: reordering main loop
Main loop reordered, as before some buggy states could be reached.

Change-Id: If798724da5e7bc98b036058f2247e0ad17135723
2014-01-23 18:53:20 +01:00
bcb1e6b7a4 openocd: trace format fix
The output format of the tracing plugin needs to be as follows:
For a position in execution, first the PC-Event needs to be sent up
to the simulator controller, afterwards potential MEM-Events can be
signaled.

This order requirement is fixed with this commit.

Change-Id: I8412dd7a5a85a58c50a0d0d9b9d0369c838bc0a3
2014-01-23 18:53:20 +01:00
b97998829a openocd: some minor fixes in wrapper
Change-Id: I94ecde881d5eaeea280e5c6de3c3abc16c2903f4
2014-01-23 18:53:20 +01:00
55d17ba439 openocd: always try to aggregate mmu config writes
As memory writes have high fix costs, we want to minimize the number
of writes in consequence of the mmu configuration.

This commit aggregates writes as far as possible.

Change-Id: I4ba6618fc9023ae4a353585710e2933d3d7a6051
2014-01-23 18:53:20 +01:00
98777e1ec3 openocd: get activated watchpoint from openocd
Decoding the current instruction sometimes failes, so we need
to extract the currently activated watchpoint from the openocd
internal list of watchpoints. If we have more than one watchpoint,
we are in trouble.

Change-Id: I14b172e4f933ca894cf66685c6d2ca8e573a3151
2014-01-23 18:53:20 +01:00
eac600e1b6 openocd: silence debug-messages
Change-Id: I10d1ad484871cda5dba47f75b0d310ed441f701a
2014-01-23 18:53:20 +01:00
9f334a02bd openocd: arm read pc -> offset of 8
On ARM we need to add a offset to the pc, if we read it
(ARM ARM (DDI 0100I) page A2-9). This fixes bugs in memory
trace generation and watchpoint recognition.

Change-Id: I1cfcb84af2abae7971869d2ce29d602648e2f020
2014-01-23 18:53:20 +01:00
83ac40dfb5 comm: use unsigned trace positions in protobuf
Change-Id: I91927ce2c54989c724ea537a6bd344de8c467a16
2014-01-23 18:53:19 +01:00
eab469192c cpn: regard equivalence classes of length 1
Previously the code did not handle equivalence classes, which consist
only of one instruction (length 1). As these classes for example
come up at two consecutive read instructions, we have to handle them.

Change-Id: Ib9e475a782828a380dfc79f5b390ca9192f4b8e3
2014-01-23 18:53:19 +01:00
99bc6e7c03 lra-simple-panda: Experiment updated for evaluation
Change-Id: I7905bda6aab7825c28b750683f2712161add325b
2014-01-23 18:53:19 +01:00
286a28e7ae smarthops: calculate target instruction & allow zero input
As we might need information of target instruction (in case of
checkpoint, etc.) this information is now added to the output
protobuf message.

Trace-Events are generated also for position zero, so this case is
also regarded.

Change-Id: I69ff4818e7f8d6771923802f65bf0aa1b81883c5
2014-01-23 18:53:19 +01:00
ba765c16c2 cpn: pruning-aware injection points
As we gain some degrees of freedom in choice of the specific
injection instruction offset, this can be used to minimize
navigational costs. This is a first approach towards pruning-aware
injection points.

To do so, we need to modify the sql query, which gets the pilots,
so we additionally join with the trace table to get begin and
end information for equivalence classes, which are feeded into
the creation of InjectionPoints.

Change-Id: I343b712dfcbed1299121f02eee9ce1b136a7ff15
2014-01-23 18:53:19 +01:00
d7a9a2811d cpn: Not every InjectionPointHops calcs smart-hops
As the InjectionPoint is considered to be a container for abstract
"points in time" which can be navigated to, not every object of
a InjectionPointHops needs a smart-hopping calculator.

Change-Id: I150a46cf79a2b9d8ddb2d24a6d89dc3d4246cdb3
2014-01-23 18:53:19 +01:00
e824e7a0fa cpn: Parsing of unsigned int fixed
As atoi caps the value of a unsigned int bigger than (2^31 - 1) other
than just letting it overflow to the corresponding negative value on
32Bit-integer machines, it must not be used for parsing to unsigned int.

TODO: Also apply this fix to all other unsigned values (in database)
which get parsed by atoi.

Change-Id: I96e29b14d36479ab6e567c527a40feb0b5fb14e5
2014-01-23 18:53:18 +01:00
8b5098abdd tools: added compute-hops and dump-hops tools
As these tools work closely together with fail components, its
easiest, to build them in this context. As these tools don't
really matter for fail use, they might never be pushed to the
master branch.

Change-Id: I8c8bd80376d0475f08a531a995d829e85032371b
2014-01-23 18:53:11 +01:00
17e76c140b cpn: needs comm and MySQL at link time
The dependency on fail-comm exists not only at compile time (the
latter is due to protobuf header generation).

Change-Id: I2bae51e763d9a385bda94e77df3e88619fa28a30
2014-01-23 14:31:24 +01:00