Commit Graph

1026 Commits

Author SHA1 Message Date
5378620b63 core/sal: fix warning for 32-bit Bochs
warning: unused variable ‘regdata’

Change-Id: I2b5b0ab7d920dc060ec814494e9817f1f49496a9
2014-03-26 17:47:42 +01:00
ba60ecc4e6 core/sal: comment typos
Change-Id: I9ac66d7d1afa22dc6645433736e1c1d38a4e23fa
2014-03-26 17:47:37 +01:00
136d397a52 core/sal: identical register IDs for 32 and 64 bit
With this change, x86 special-purpose registers (e.g., EFLAGS) get the
same register ID in both 32 bit and 64 bit configurations.

Change-Id: I69db9397481414f99ca05ecb0ea9dc8ab7d989c9
2014-03-26 17:47:25 +01:00
1ad2eb5110 core/sal: fix x86 register IDs for x86_64
This bug mapped EFLAGS to the same Fail* register ID as R9 on x86_64.  This
probably has not had consequences yet, as most FailBochs users use 32-bit
code.

Change-Id: I00a680675bb9e73c2781276f3ef651162c8e4445
2014-03-26 17:27:18 +01:00
84a03b55ff import-trace: limit to general-purpose registers
This change limits fault injection to general-purpose registers, instead of
relying on the LLVM/Fail* bridge to only recognize the status register
(EFLAGS on x86) and general-purpose registers.  Since this bridge just
learned to translate x86's control and segment registers, and these
registers need special handling for fault injection (def/use pruning does
not work here), only import register accesses from the RT_GP subset.

Status register and instruction pointer injection remain functional, and
import-trace now should work architecture independently.

Change-Id: Id8ad2f0a9dab1861bf16ea9443c3bdfe7213d3fa
2014-03-26 17:18:18 +01:00
fb788ce33e Merge "util/llvmdisassembler: map registers by names" 2014-03-26 16:45:19 +01:00
48002c81f8 Merge "core/sal: add x86 control and segment registers" 2014-03-26 16:45:11 +01:00
0acfe5b2e2 TracingPlugin: cap size of memory access to 8
There are instruction in the real world, that seem to read more than 8 bytes.
Therefore we cap the accesses memory to 8 and don't abort with an assertion.

Change-Id: Ib2bfd0ac6dc0c030cd6a22940e31f8463a3337cf
2014-03-26 14:54:17 +01:00
0799e52fde util/llvmdisassembler: map registers by names
Internal LLVM register IDs can and did change between LLVM versions.
These magic integers are replaced by iterating over all LLVM registers
and mapping them to FAIL* registers by name.
As this iteration requires a LLVM object created from a binary, a static
convenience function is added to LLVMtoFailTranslator which creates a
translator given the binary filename. Building this functionality inside
libfail-llvmdisassembler prevents experiments from needing to add LLVM
includes and library definitions.

Change-Id: I27927f40d5cb6d9a22bb2caf21ca2450f6bcb0b8
2014-03-24 15:01:09 +01:00
21f5f681e0 core/sal: add x86 control and segment registers
new register type RT_CONTROL:
CR0, CR2, CR3, CR4

new register type RT_SEGMENT:
CS, DS, ES, FS, GS, SS

Reading/writing is mostly untested except for CR3...

Change-Id: I0d0fba4a1669153ab2577e82ab64a04cf2bbfb94
2014-03-24 14:32:54 +01:00
396e00ce59 cmake: static library dependencies
CMake does not support linker groups, which were used to "automatically"
fix circular dependencies between different static FAIL* libraries and
the ordering of dynamic external libraries broke linking.
CMake can however correctly invoke the linker if dependencies are decribed
correctly (even if circular). This required changing all add_dependencies
calls between libraries to target_link_libraries (which creates a link-time
dependency) and linking all experiments to fail-sal.

Change-Id: I3a0d5dddb9b3d963ef538814e20d6b3de85d4ec5
2014-03-24 11:47:46 +01:00
3cf2fa082f Merge "DatabaseCampaign: experiment completion checks" 2014-03-24 11:45:20 +01:00
858afe6e6f Merge "DatabaseCampaign: fix finished experiments SQL" 2014-03-24 11:45:10 +01:00
5567c595fb DatabaseCampaign: experiment completion checks
If the queue for outbound jobs is not unlimited, experiment rows are fetched
from the DB server continuously as experiments finish. When this takes too
long the connection to the DB server can be lost. The code did not check for
a mysql_error and assumed the result set was fetched completely, thus skipping
a potentially large amount of experiments (in our case only ~20000 of 400000+
experiments were run).

This change adds checks to determine if the result fetch loop was finished due
to an error and checks the sent pilot count to the unfinished experiment count.
Additionally, the mysql result object is correctly freed.

The underlying problem of MySQL connection loss can hopefully be prevented by
increasing timeouts in the MySQL config as described in doc/how-to-build.txt.
To prevent the problem from occurring when this is forgotten, this change
reverts the default job queue length to be unlimited (SERVER_OUT_QUEUE_SIZE=0),
at the cost of increased memory usage.

Change-Id: I09d9faddd8190c6dd5fbe733a0679a733d5837ec
2014-03-21 11:36:38 +01:00
b7ff284d54 generic-tracing: add configuration dependencies
Added hint: Use /dev/null if no state is required

Change-Id: I76a935f88e6a7a8f707970797d508dad3a961ab5
2014-03-20 10:59:45 +01:00
010d4a892d DatabaseCampaign: fix finished experiments SQL
The database queries to fetch all unfinished experiments were broken.
The server tried to insert all finished pilot_ids into the temporary
result_ids table and then discard all experiments which have the correct
(finished) count of IDs in this table. This cannot work as the pilot_id
is the only column of result_ids and must be a unique primary key.

As a fix, the count of results is stored as a second field in result_ids
and the result table is now joined against result_ids to check this field.

Change-Id: I6a9fb774825f0cc4ce104c6e51d7b2fe16957aec
2014-03-18 11:18:27 +01:00
f775c92d72 dump-trace: keep track of read/written bytes
This change adds access-size tracking to dump-trace and output thereof in mode "-s".

Change-Id: I5647d7b16c89499b7813faaf8c3844f275bc552a
2014-03-05 11:55:11 +01:00
9df6d983bf util/llvmdisassembler: compile with -fno-rtti
For some reason, this is required even when LLVM is not built using
-fno-rtti.

Change-Id: I992799c8b54135a0a87b2de7c4a3d57f2d3670d9
2014-02-26 14:46:23 +01:00
6e03753b6e ecos: modifications for mibench benchmarks
Change-Id: Ifecbf24912dbffa814b189aed9336a5420ec6392
2014-02-25 13:32:56 +01:00
cbf9daea97 ecos: rewrite for DatabaseCampaign
+ modified resulttypes

Change-Id: I463759e66e7497c80eeee9a065fc95e058ec3dc1
2014-02-25 13:32:56 +01:00
0009a95e62 fail-env updated
Change-Id: Idf605dddc4026c9b796d0ef7174e430d9aed1236
2014-02-25 13:32:56 +01:00
5ccc6e3525 comm: ExperimentData needs a virtual destructor
Classes deriving from ExperimentData usually contain the
experiment-specific Protobuf message, which needs to be properly
destroyed.  This is particularly a problem in the generic
DatabaseCampaign, as it never downcasts ExperimentData objects
retrieved from JobServer::getDone().  As the embedded
DatabaseCampaignMessage (usually named "fsppilot") is allocated on the
heap (this happens in the campaign's cb_send_pilot() function, asking
for a mutable_fsppilot()), the lack of a virtual destructor in
ExperimentData led to a memory leak, rendering the campaign server
inoperable after handling ~1E7 messages (with a 4GiB / 32-bit process
memory limit).

Change-Id: I4cb8a26d5a702e03189c4aae340051ce62a9c9ce
2014-02-25 13:32:56 +01:00
5ee96032c9 jobserver: gracefully handle thread creation failures
Due to the previous DatabaseCampaign fix, this may not be necessary
anymore, but it's nevertheless a good idea to handle thread creation
failures properly.

Change-Id: I8317a77dd5338509727e737040944320e7755ae3
2014-02-25 13:32:56 +01:00
25a390970a DatabaseCampaign: avoid table locking
It is necessary to copy pilot IDs of existing results to a temporary table
before fetching undone jobs from the DB: Otherwise, due to MyISAMs
table-level locking, collect_result_thread() will block in INSERT (SHOW
PROCESSLIST state "Waiting for table level lock") until the (streamed)
pilot query finishes.  As one pilot query follows after the other,
collect_result_thread() may even starve until the memory for the
JobServer's "done" queue runs out, resulting in a crash and the loss of all
queued results.

Change-Id: Ib0ec5fa84db466844b1e9aa0e94142b4d336b022
2014-02-25 13:32:55 +01:00
b094753fde doc: missing libraries
Change-Id: Ife89a0b3cc74433e4fb711580c5eb3cd82467081
2014-02-25 13:32:55 +01:00
bc2103c527 sal/bochs: don't show errors in non-verbose mode
The patched eCos variant we analyze intentionally overflows the 16550
UART FIFOs, flooding the terminal with Bochs error messages.  Enabling
CONFIG_BOCHS_NON_VERBOSE now also enforces ignoring error messages,
regardless of log verbosity settings in the bochsrc.

Change-Id: If14e2532234e61bf60720a45150ef4973e8d508b
2014-02-25 13:32:55 +01:00
953fbe2156 serialoutput: cleanup
Change-Id: I255a3dd44fcf075a441461a883f564ee2d626ee1
2014-02-25 13:32:55 +01:00
455c088cd9 serialoutput: optional character limit
This prevents unlimited memory consumption in case the guest system
enters an endless loop.

Change-Id: Ia1bb178f7d8cb8ad8bf958210d90f6d7c2e11359
2014-02-25 13:32:55 +01:00
c319f3458c serialoutput: consistent plugin class naming
Change-Id: I8abe0cfdebecb0adc7229e29bd241da65b27105a
2014-02-25 13:32:55 +01:00
36ae6fd6c3 prune-trace: use none/none only without any parameters
Before this change, running prune-trace with, e.g.
"prune-trace -d fsp_mibench -v bitmap% --benchmark-exclude clockcnv"
resulted in an implied "--benchmark none", rendering --benchmark-exclude
ineffective and resulting in nothing being pruned.  Now, the "none" default
only applies when neither --benchmark nor --benchmark-exclude (analogously
for --variant / --variant-exclude) is provided.

Change-Id: Ic7c88919d7cfde1261749a745dc6a679472ff348
2014-02-25 13:32:55 +01:00
1df43e9726 import-trace: major speedup
Using Database::insert_multiple() instead of prepared statements
speeds up trace import by a factor of 3-4.  While being there, we now
properly deal with nonexistent extended trace values (i.e., put NULLs
into the DB).

Side note: The ElfImporter should switch to insert_multiple(), too.

Change-Id: I96785e9775e3ef4f242fd50720d5c34adb4e88a1
2014-02-25 13:32:55 +01:00
76bda55c5e Merge "plugins/randomgenerator: add deterministic PRNG plugin" 2014-02-20 11:26:35 +01:00
69ba9e0f94 dump-trace: properly deal with empty extended trace entries
As dereferencing register contents is not always possible, extended trace
entries may be empty.

Change-Id: I603fcef2eb2b0429a9d6ed0469441bc314e365fd
2014-02-19 19:08:46 +01:00
b6fc98abae generic-tracing: remove --save-symbol
At least for the Bochs backend there might be side effects when saving
the simulator state while tracing, which therefore should be avoided.
As there is no known use-case for using a --save-symbol different to
--start-symbol, this change disables the semantics behind
--save-symbol completely and only keeps the command-line switch for
backward compatibility reasons (existing automatic test scripts etc.).
The generic-tracing experiment now complains and aborts if a
--save-symbol different to --start-symbol is given.

Change-Id: I6072d846be96e016534cc83db375a400cfc25303
2014-02-19 19:08:46 +01:00
836325e74b generic-tracing: cleanups
Change-Id: I5c3d1131248910228cb4fee44cf107c750c01e21
2014-02-19 19:08:46 +01:00
85152238da tracing: fix endless loop when only tracing mem accesses
With m_tracetype=TRACE_MEM, bool first was never reset to false in the
tracing plugin's main loop.  This bug was most probably never
triggered, though, as nobody only traces memory accesses.

This change also slightly simplifies the internal logic in the tracing
plugin.

Change-Id: I65d7df6a3781ec552cfb892bbf3394b421e227f1
2014-02-19 19:08:46 +01:00
b82e547b53 plugins/randomgenerator: add deterministic PRNG plugin
A simple plugin which deterministically returns a new random value
each time the specified symbol is read.

Change-Id: I6ccac421fc064f02a88e8b126f8a26044d1f51c6
2014-02-18 16:40:34 +01:00
01c1321b48 tracing: bugfix for mem dereferences at mapping boundary
As we copy a 32-bit word from the dereferenced address, we also need to
check whether address+3 is also mapped.  (Yes, I've seen this in the
wild.)

Change-Id: I43f891c56e077333670c9cb48c0ee8e9342fa41d
2014-02-17 23:24:16 +01:00
58fa4c59cc sal/bochs: fix handling of unmapped memory
Up to now, BochsMemory::isMapped() always returned true in 32-bit protected
mode with a 4GB linear address space (as used by, e.g., eCos), even for
addresses greater than the configured memory size.  This led to lots of
bogus memory dereferences in the (extended) tracing plugin.

This change (a follow-up to commit 5171645) additionally checks the return
value of getHostMemAddr(), and announces BX_RW (read/write access) instead
of BX_READ as the intended type of memory access.  In the aforementioned
scenario, memory addresses greater than the memory size are now correctly
detected as "not mapped".

Change-Id: Ic2fa7554c869cb90191164535a601bae4dbb49b6
2014-02-17 23:24:16 +01:00
e4bf980b97 Fail* result browser for pruning experiments.
Based on the database layout given by the pruner.
Run ./run.py -c <path to mysql.cnf>
(Default config ~/.my.cnf)

- Checks if objdump table exists
- Added view for results per instruction
- Added config file support for table details
- Overview data loaded at server startup
- Result type mapping configurable via config file

Based on Flask and MySQLdb

Change-Id: Ib49eac8f5c1e0ab23921aedb5bc53c34d0cde14d
2014-02-17 10:17:25 +01:00
b4f144745a Revert "import-trace: emit warning for malformed traces"
Memory accesses that don't belong to the preceding IP event in the
trace *do* have a use case:  a hardware interrupt causes the CPU to
push its state onto the (kernel) stack.  At the moment we cannot
distinguish this case from a malformed trace (as we don't record the
occurrence of interrupts), hence this warning needs to be disabled for
now.

This reverts commit 84edd02b6f.
2014-02-11 14:57:29 +01:00
8942740d9a Merge branch 'memorymap' 2014-02-09 14:20:53 +01:00
26308dc9ea Merge "Removes serial.out from bochs config for massive performance boost on cluster runs." 2014-01-31 13:52:45 +01:00
4ba028e740 Removes serial.out from bochs config for massive performance boost on
cluster runs.

If this output file is enabled, all running processes try to write to the
same file on the shared filesystem. They block each other which leads to
massive I/O wait time and CPU idle time.

This change reduces the runtime e.g. from several hours (12+) to few minutes
(20).

Change-Id: I028628af31c845fc517e5daca5b4f981eade3cf4
2014-01-31 12:51:58 +01:00
4b921a5fe3 util: MemoryMap test
Change-Id: I54680685326a85cfd723a47e8aef8d71662c9aeb
2014-01-30 15:26:20 +01:00
4bcce14659 util: space-efficient MemoryMap
We now use boost::icl::interval_set internally, consuming extremely
lower amounts of memory.  boost::icl was introduced with Boost 1.46;
Debian 7.0 comes with 1.49, so this dependency should be no problem
anymore.

Both the class interface and the memory-map file format stay the same.

Change-Id: I38e8148384c90aa493984d0f6280817df00f1702
2014-01-30 15:26:12 +01:00
119ae40be9 util/Database: added a wrapper function for mysql_real_escape_string()
Change-Id: I999aad3c35c5f389fa3acfe8d7a11c417c478787
2014-01-28 11:07:34 +01:00
13175c259b import-trace: import debug info
If the --debug option is set, the line number table of the elf binary will
be imported into the database. The information will be stored in the
"dbg_mapping" table.

If the --sources option is set, the source files will be imported
into the database. Only the files that were actually used in the
elf binary will be imported.

Change-Id: I0e9de6b456bc42b329c1700c25e5839d9552cdbb
2014-01-28 11:07:34 +01:00
d307dd2ecb dciao-kernelstructs: reuse sobres experiment for ISORC2014
Differences:

- the task activation order is determined in the faulty experiment as
  well as in the golden run (which is now done by
  fail-generic-tracing) by observing a variable fail_virtual_port.
- There is a panic value read from the fail_virtual_port
- The golden run task activation is determined by giving an extended
  trace to task_activation.py. The script collects all writes to
  fail_virtual_port, and determines the activation from this.

Change-Id: Id401b78933b45a4b2cf031fc0a8b5ac90151ec24
2014-01-27 10:32:09 +01:00
c48c7296fb util/WallclockTimer: bugfix: include ostream
This only compiled everywhere because all users included (i)ostream.

Change-Id: I29b0fb13a01606fdffd8ebdb9701eff652065916
2014-01-24 20:33:32 +01:00