Commit Graph

110 Commits

Author SHA1 Message Date
42182591e5 fix compiler warnings
(DatabaseCampaign; llvmdisassembler)

Change-Id: Ic31758018a0a1ff0ceac81f781eecfc5f8060f89
2014-08-28 12:12:38 +02:00
84cfa2ae42 Merge commit '0da8ba0dec111d78292455bb5f17c6045820db25' 2014-08-28 12:04:34 +02:00
0426970dff llvmdisas.: Do not put side-effect code into assert(...)
assert(...) can be optimized away, therefore side-effect code should not
be placed inside an assertion.

Change-Id: I28aee42e53cb105333094d0042a3f6e2cc5b5a30
2014-08-25 12:48:10 +02:00
87008a05b6 util: LLVM test code compilation fix
This change removes an unnecessary "#ifndef __puma" from the LLVM
disassembler test code and fixes compilation with the latest AspectC++
binaries.

Change-Id: Ibe835a4a6df69255555c668985f15b9cf8fb82b4
2014-07-13 18:43:39 +02:00
9a81ab4222 Database: more flexible get_variants()
This change moves prune-trace's --variants-exclude / --benchmarks-exclude
capabilities to Database::get_variants() to make it available to all users.

Change-Id: Icbc6bb1a3ae7c846d2de40b881f47a9cc1ed7bbf
2014-07-03 15:42:25 +02:00
6300a2b364 util: SumTree implementation
The SumTree implements an efficient tree data structure for
"roulette-wheel" sampling, or "sampling with fault expansion", i.e.,
sampling of trace entries / pilots without replacement and with a
picking probability proportional to the entries' sizes.

For every sample, the naive approach picks a random number between 0
and the sum of all entry sizes minus one.  It then iterates over all
entries and sums their sizes until the sum exceeds the random number.
The current entry gets picked.  The main disadvantage is the linear
complexity, which gets unpleasant for millions of entries.

The core idea behind the SumTree implementation is to maintain the
size sum of groups of entries, kept in "buckets".  Thereby, a bucket
can be quickly jumped over.  To keep bucket sizes (and thereby linear
search times) bounded, more bucket hierarchy levels are introduced
when a defined bucket size limit is reached.

Note that the current implementation is built for a pure growth phase
(when the tree gets filled with pilots from the database), followed by
a sampling phase when the tree gets emptied.  It does not handle a
mixed add/remove case very smartly, although it should remain
functional.

Change-Id: If05e9700bc84761b5bc31006402641e7112b3a72
2014-07-03 15:42:25 +02:00
b92df6592f util: BlackholeLogger, a quiet Logger drop-in replacement
The compiler should be able to completely optimize away side-effect
free usage of this logger.  Can be used as a drop-in replacement for
Loggers to silence logging output for known-good code without having
to remove the corresponding "LOG << ..." code.

Change-Id: Ifb276223f61686773dd6108aafd567e99c88b223
2014-07-03 15:42:25 +02:00
960f0881a2 util/smarthops: missing linker dependency
Change-Id: I3017a8824b3bd7ae9547b6c91db6847e5f5b8694
2014-06-27 13:16:25 +02:00
e73ac88d1b util: LLVM disassembler off-by-one
The disassembled memory region's end (variable "End") is exclusive
now.  Up to now, the two branches defining this variable disagreed on
inclusiveness, leading to an infinite loop in one case.

Change-Id: I055fc240f6ec2f4a1d1937e48617c86612cff5c5
2014-06-25 16:43:28 +02:00
c827750090 Merge branch 'failpanda'
Conflicts:
	src/core/comm/DatabaseCampaignMessage.proto.in
	src/core/cpn/CMakeLists.txt
	src/core/cpn/DatabaseCampaign.cc
	src/core/sal/ConcreteCPU.hpp
	src/core/sal/SALConfig.hpp
	src/core/util/CMakeLists.txt

Change-Id: Id86b93d0e3ea4d9963fcc88605eec0603575ec83
2014-06-03 12:24:49 +02:00
c6655c0dbb util/smarthops: don't redefine address_t
Change-Id: I702c2fe8824dfe13c4e99f9400d6dacb3ae63f53
2014-06-03 11:47:21 +02:00
277958b31b cleanups
Change-Id: I8022d937477668253c613e97c3a579ae65084b1e
2014-06-03 11:47:20 +02:00
af92a751d9 import-trace: alias-based (importer) registry
This change implements a generic registry in order to clean up import-trace's
code - it's possible (and reasonable) to use the registry for pruners as well.
Importer now extends AliasedRegisterable; all importers have been adapted
to suit the interface/abstract methods.
Each AliasedRegisterable should have at least one alias (the class' name
is a sensible choice) but can have several. The first specified alias is
the class' prime alias which can be used e.g. to list all registered objects.

Change-Id: If6daa34edce35a3b0194e4ba67ed3b44b74a49b0
2014-04-25 08:34:01 +02:00
b500873bbb Merge branch 'ubuntu-trusty-fixes'
Conflicts:
	src/core/util/CMakeLists.txt

Change-Id: I94811612bcd31406c0e275f72058c331a99c8943
2014-04-11 15:05:32 +02:00
9113d7c2fa cmake: find libiberty headers via cmake module
The libiberty headers moved from /usr/include (and a part of binutils-dev)
to /usr/include/libiberty (libiberty-dev) between Ubuntu 13.10 and 14.04,
which made a proper cmake search module necessary.  Searching still
continues working well on Debian 7.

Change-Id: I324e5ccb847e4664442d6fa7d7a027705a4f0587
2014-04-11 14:42:00 +02:00
799bdd4c64 Merge branch 'database_config_file' 2014-04-11 13:51:48 +02:00
c6344af189 Database: commandline option for alt. config file
This change adds an optional command line argument "--database-option-file",
which can be used to override the default database configuration file ~/.my.cnf

Change-Id: I5c71523e1c31dead26f3fedb0ca7354ca99892d4
2014-04-11 13:51:37 +02:00
c384894a4e Merge branch 'find_libdwarf_libelf_via_cmake'
Conflicts:
	src/core/util/CMakeLists.txt

Change-Id: I510578482e30a88981fba0774b66dfcf4812e07f
2014-04-11 13:24:23 +02:00
ba774a258c util/llvmdisassembler: fix section end symbols
Somehow, while iterating symbols in a section, it can happen that the last
symbol start address is equal to the section size, which means it is beyond
the section end.
In this case the LLVM getInstruction() method does not return a failure, but a
zero-size instruction, resulting in an infinite loop.

Now, if beyond section limits, the iteration is aborted.
Additionally, an assertion checks for disassembled zero-size instructions.

Change-Id: Id8a355475161150d3ee919cd6cf603d4ff26b228
2014-04-03 15:02:57 +02:00
da839c9765 util: find libdwarf/libelf via cmake itself
Instead of hardcoded -lelf/-ldwarf use cmake modules FindLibElf/-Dwarf
and link the found libraries accordingly. This enables the user to have those
libs in arbitrary locations and link them from there.

Change-Id: I2cea3ef648a46f11b0d49d2fe0b006f76a9d4140
2014-03-30 18:20:49 +02:00
0799e52fde util/llvmdisassembler: map registers by names
Internal LLVM register IDs can and did change between LLVM versions.
These magic integers are replaced by iterating over all LLVM registers
and mapping them to FAIL* registers by name.
As this iteration requires a LLVM object created from a binary, a static
convenience function is added to LLVMtoFailTranslator which creates a
translator given the binary filename. Building this functionality inside
libfail-llvmdisassembler prevents experiments from needing to add LLVM
includes and library definitions.

Change-Id: I27927f40d5cb6d9a22bb2caf21ca2450f6bcb0b8
2014-03-24 15:01:09 +01:00
396e00ce59 cmake: static library dependencies
CMake does not support linker groups, which were used to "automatically"
fix circular dependencies between different static FAIL* libraries and
the ordering of dynamic external libraries broke linking.
CMake can however correctly invoke the linker if dependencies are decribed
correctly (even if circular). This required changing all add_dependencies
calls between libraries to target_link_libraries (which creates a link-time
dependency) and linking all experiments to fail-sal.

Change-Id: I3a0d5dddb9b3d963ef538814e20d6b3de85d4ec5
2014-03-24 11:47:46 +01:00
7b12442117 util: find libdwarf/libelf via cmake itself
Instead of hardcoded -lelf/-ldwarf use cmake modules FindLibElf/-Dwarf
and link the found libraries accordingly. This enables the user to have those
libs in arbitrary locations and link them from there.

Change-Id: I2cea3ef648a46f11b0d49d2fe0b006f76a9d4140
2014-03-19 13:56:59 +01:00
9df6d983bf util/llvmdisassembler: compile with -fno-rtti
For some reason, this is required even when LLVM is not built using
-fno-rtti.

Change-Id: I992799c8b54135a0a87b2de7c4a3d57f2d3670d9
2014-02-26 14:46:23 +01:00
1df43e9726 import-trace: major speedup
Using Database::insert_multiple() instead of prepared statements
speeds up trace import by a factor of 3-4.  While being there, we now
properly deal with nonexistent extended trace values (i.e., put NULLs
into the DB).

Side note: The ElfImporter should switch to insert_multiple(), too.

Change-Id: I96785e9775e3ef4f242fd50720d5c34adb4e88a1
2014-02-25 13:32:55 +01:00
4b921a5fe3 util: MemoryMap test
Change-Id: I54680685326a85cfd723a47e8aef8d71662c9aeb
2014-01-30 15:26:20 +01:00
4bcce14659 util: space-efficient MemoryMap
We now use boost::icl::interval_set internally, consuming extremely
lower amounts of memory.  boost::icl was introduced with Boost 1.46;
Debian 7.0 comes with 1.49, so this dependency should be no problem
anymore.

Both the class interface and the memory-map file format stay the same.

Change-Id: I38e8148384c90aa493984d0f6280817df00f1702
2014-01-30 15:26:12 +01:00
119ae40be9 util/Database: added a wrapper function for mysql_real_escape_string()
Change-Id: I999aad3c35c5f389fa3acfe8d7a11c417c478787
2014-01-28 11:07:34 +01:00
13175c259b import-trace: import debug info
If the --debug option is set, the line number table of the elf binary will
be imported into the database. The information will be stored in the
"dbg_mapping" table.

If the --sources option is set, the source files will be imported
into the database. Only the files that were actually used in the
elf binary will be imported.

Change-Id: I0e9de6b456bc42b329c1700c25e5839d9552cdbb
2014-01-28 11:07:34 +01:00
c48c7296fb util/WallclockTimer: bugfix: include ostream
This only compiled everywhere because all users included (i)ostream.

Change-Id: I29b0fb13a01606fdffd8ebdb9701eff652065916
2014-01-24 20:33:32 +01:00
85e3911202 Merge branch 'ubuntu-saucy-fixes' 2014-01-24 17:02:44 +01:00
286a28e7ae smarthops: calculate target instruction & allow zero input
As we might need information of target instruction (in case of
checkpoint, etc.) this information is now added to the output
protobuf message.

Trace-Events are generated also for position zero, so this case is
also regarded.

Change-Id: I69ff4818e7f8d6771923802f65bf0aa1b81883c5
2014-01-23 18:53:19 +01:00
8b5098abdd tools: added compute-hops and dump-hops tools
As these tools work closely together with fail components, its
easiest, to build them in this context. As these tools don't
really matter for fail use, they might never be pushed to the
master branch.

Change-Id: I8c8bd80376d0475f08a531a995d829e85032371b
2014-01-23 18:53:11 +01:00
c142818325 cpn: Generic wrapper for injection point
As for the pandaboard to navigate fast to the injection
instruction we need to deliver a hop chain to the fail-client,
this commit adds a generic wrapper for a injection point.
For now we have only the two options hop chain and instruction
offset, so it is activated via a cmake ON/OFF switch.

Change-Id: Ic01a07a30ac386d4316e6d6d271baf1549db966a
2014-01-22 18:04:29 +01:00
4cb97a7fa5 formatting, typos, comments, details
Change-Id: Iae5f1acb653a694622e9ac2bad93efcfca588f3a
2014-01-22 13:08:13 +01:00
813414984c util: boost::thread 1.53 depends on boost::system
Unfortunately this implicit dependency is currently not resolved anywhere
else (e.g., FindBoost.cmake), although the 'net heavily discusses this
issue.

Change-Id: I8a7c8518394cdba27e591fed250623011d988067
2014-01-21 00:29:34 +01:00
8505ddbb04 jobserver: synchronization cleanup
This change cleans up in/out queue synchronization in the job server.
End-of-jobs conditions are now properly signaled through the
SynchronizedQueue, allowing to resume and abort blocked readers when
no more input is expected.

Change-Id: I3eaf37115ccf8c5b5afe3d971c7109cd62b68906
2014-01-20 22:48:08 +01:00
33b63651ae DatabaseCampaign: MySQL / concurrency fixes
According to
<http://dev.mysql.com/doc/refman/5.5/en/c-api-threaded-clients.html>,
a MySQL connection handle must not be used concurrently with an open
result set and mysql_use_result() in one thread
(DatabaseCampaign::run()), and mysql_query() in another
(DatabaseCampaign::collect_result_thread()).  This indeed leads to
crashes when bounding the outgoing job queue (SERVER_OUT_QUEUE_SIZE),
and maybe even more insidous effects in other cases.  The solution is
to create separate connections for both threads.

Additionally, call mysql_library_init() before spawning any threads.

Change-Id: I2981f2fdc67c9a2cbe8781f1a21654418f621aeb
2014-01-20 18:34:51 +01:00
ec969603d5 Merge commit 'dcd2c021a5ac91d38187d397914e5f51e2fc8819'
Conflicts:
	tools/import-trace/RegisterImporter.cc

Change-Id: I4f49c976bd60badba73c15746aa03c420cb9f77b
2013-09-11 14:38:55 +02:00
d26fc28fa4 cpn/database: include data_width in the fsppilot during prune step
During the prune step the data_width of the injected location was not
propagated before. It is now stored in fsppilot (database layout change!) and
sent in the fsppilot protobuf message.

Change-Id: I0562f6fc8957adea0f8a9fb63469ca5e3f4b7b2d
2013-09-11 10:27:04 +02:00
dcd2c021a5 util: global lock for certain MySQL operations
Even the reentrant libmysqlclient_r has some non-threadsafe operations,
which need to be protected by a global mutex.
<http://dev.mysql.com/doc/refman/5.5/en/c-api-threaded-clients.html>

Change-Id: I444e42f82cf982a6c8f8f2596e8991d0a5009b28
2013-09-10 18:35:44 +02:00
ba7c663551 import-trace: introduce AdvancedMemoryImporter
A MemoryImporter that additionally imports Relyzer-style conditional
branch history, instruction opcodes, and a virtual
duration=time2-time1+1 column (MariaDB 5.2+ only) for fault-space
pruning purposes.

Change-Id: I6764a26fa8aae21655be44134b88fdee85e67ff6
2013-09-10 17:37:26 +02:00
12b539ff75 misc cleanups
This change touches several subsystems, tools and experiments
(sal, util, cmake, import-trace, generic-tracing, nanojpeg), and
changes details not worth separate commits.

Change-Id: Icd1d664d1be5cfc2212dbf77801c271183214d08
2013-09-10 17:37:25 +02:00
96f2f56d5e Merge branch 'register-mapping-fixes' 2013-09-10 11:46:58 +02:00
11513ef78d util: handle missing register mapping gracefully
It's OK if we cannot map every register LLVM knows to a Fail register
ID, but we need to explicitly skip these cases in the
RegisterImporter.

Change-Id: I2152f819fb94aa4de5720c5798b229b66988d382
2013-09-09 16:14:35 +02:00
e4a5a7a592 util: gzstream needs zlib
This change is needed to build on Ubuntu 13.04.

Change-Id: I683ed4427044264f58bc8f7c94cb5fbbff89cd95
2013-09-08 22:15:14 +02:00
203ec6c5cc remove #ifndef __puma from code using LLVM
Contemporary AspectC++ versions can deal with the LLVM headers very
well, and #ifdef __puma stuff in Fail* headers results in
unmaintainable #ifdef __puma blocks in other parts of Fail* (e.g., the
trace importer).

Make sure you're using a 64-bit ac++ when living in a 64-bit userland
(the 32-bit version doesn't know about __int128), and be aware that
AspectC++ r325 introduced a regression that has not been fixed yet.

Change-Id: I5bb759b08995a74b020d44a2b40e9d7a6e18111c
2013-09-04 10:13:48 +02:00
89aff7df65 util: Bochs/Fail register mapping fix
CL and DL are 8 bits wide, not 255.

Change-Id: Ic454215344d08ba2f8374db251792f82d106305b
2013-09-02 12:07:27 +02:00
2108c8932f util: disassembler register/address mapping fix
For def/use pruning, the linear address mapping of the x86
sub-registers (e.g., AX represents the lower 16 bits of EAX) must
overlap.  If it doesn't, e.g., AX and EAX are considered separate
registers by def/use pruning, resulting in a failure to correlate an
EAX def with a subsequent AX use.

The only user of this mapping up to now, RegisterImporter, forced all
register widths to 8 bits, thereby fortunately canceling out this
problem.  Nevertheless it makes no sense to continue encoding a
constant width in these virtual addresses.

Existing trace, fspgroup, fsppilot and result tables may be converted
to the new encoding by using this query:

UPDATE <tablename> SET data_address = ((data_address >> 4) & ~0xf) | data_address & 0xf;

Change-Id: I7a942b78c34f6140803a86af639eeedef3550f34
2013-09-02 09:50:55 +02:00
ff1ceeddac util: silence const warnings with optionparser
We don't use the warned-about part of optionparser's interface
anyways.

Change-Id: I42c4714e4dce4112ea5b6cfb5159ed8b62d4c418
2013-08-27 13:38:40 +02:00