Commit Graph

451 Commits

Author SHA1 Message Date
d58694521c dbcampaign: don't include fspmethod/variant ID in job msg
These IDs don't make sense by themselves but only after a lookup in the
database, which clients usually don't have (and don't need) access to.

Conflicts:
	src/core/comm/DatabaseCampaignMessage.proto.in

Change-Id: Ice739463552039b7fb48581722ea2e05984cea47
2015-01-21 14:53:32 +01:00
c422911741 dbcampaign: allow wildcard for prune method
Using mixed pruning methods now does not require to run the campaign
server twice anymore.

Change-Id: I3f62c269166b750892bb0e659ad0c180425d1479
2015-01-21 14:53:32 +01:00
336ffd6453 Merge branch 'sampling'
Change-Id: Id6be7954b0bafcbfd64ba702e69e016bd2810115
2015-01-21 14:53:28 +01:00
39df0a979e Merge branch 'listener-perf-bug'
Change-Id: Ia863115ec31011ad2b7a954543c8ee64188d3d32
2015-01-21 14:52:33 +01:00
99a923b11e Merge remote-tracking branch 'origin/master' into sampling-wip
Change-Id: Iae5c02be5801d75e8adc55222ccb35c559f7ebf4
2015-01-21 00:22:28 +01:00
f8e0f1bb3f util: add SumTree::iterator
Change-Id: I8304b64634fa3ab92a126fe5d942674b26334b3d
2015-01-21 00:17:48 +01:00
2f70e05db6 util: rename SumTree::get -> remove, add r/o get
SumTree::get now non-intrusively picks an element and returns a
reference to it, SumTree::remove removes and returns a copy.  The
former is needed for sampling with replacement.

Change-Id: Iefef2fdf0b7df6ea7a9949f2588528ec9e86bb7a
2015-01-21 00:17:48 +01:00
b0c58bab78 util: SumTree::add() documentation
This function copies the reference argument 'element' internally.

Change-Id: I33d94f224bc2b1b89057b90258d500eaa364ab85
2015-01-21 00:17:48 +01:00
a00c4d9a69 util/DwarfReader: plug file-descriptor leaks
This change fixes several file-descriptor leaks in the DwarfReader
implementation.  The patch is taken from Richard Hellwig's not yet
merged change I161f626d12ca7f2b7b9d13ba9cbc254eb55692f1.  I did not
apply any white-space cleanups to prevent conflicts when Richard's
change will be merged later.

Change-Id: Icd9c1bdeeab39e77900e2ce88b756a8cf7ade96a
2015-01-14 10:15:29 +01:00
d68ea990ca sal: catch listener removal corner case
This change makes sure an active but not-yet fired listener does not fire
anymore if it gets removed.  In this case, the listener already has an
invalid index, but still needs to be copied to the delete list.

This issue has not been observed in the wild, and is unlikely to have
caused real problems in the past.

Change-Id: I8be8a5b1d4cdc783a092b93f34f33b969894e5da
2015-01-13 19:22:49 +01:00
c4d44aeb0c sal: fix watch/breakpoint perf implementation
This commit changes the entities in a ResultSet from listener indexes
(into ListenerManagers m_BufferList) to BaseListener pointers, as
ListenerManager::makeActive(BaseListener*) (called from
FastBreakpoints/Watchpoints.ah) may invalidate indexes temporarily
stored in a ResultSet.  The issue occurs when multiple breakpoints
(watchpoints) fire at the same time, and one added more recently than
the others occupies the largest index position in m_BufferList.

Although this issue is extremely rare and was only observed in a few
corner cases up to now, it may have falsified results in the past.

Change-Id: I7b788a06d412f15700ca75f56f2be5d3b78465fa
2015-01-13 19:22:49 +01:00
e83e87ecc7 DwarfReader: fixed linetable mapping
This change alters/corrects DwarfReader's way of determining the size of
linetable entries (i.e. their "range size"); the interface (e.g. to
ElfReader) stays the same. Prior to this, it was assumed that static
instructions within "the linetable" were sorted in ascending order. This
assumption turned out to be false, as every compilation unit's  header has
its own linetable and the compilation unit headers are not sorted by their
static instructions.
Furthermore this change implements normalization of file names.

Change-Id: Ia4beb7bf9cfb6f1a499aeebd01228335b70ab52d
2014-12-03 19:29:12 +01:00
c00268242a Merge changes I7fe63611,I84ebbb50
* changes:
  util/DwarfReader, ElfImporter: import address ranges
  util/DwarfReader, ElfImporter: use unsigned addresses
2014-10-23 07:53:46 +02:00
d53c83d7c8 TimerListener: make timeout field changeable
An accessor for the m_Timeout field was missing.

Change-Id: Idf86a3da16a328c783613ed8502446d561f6287c
2014-10-22 18:31:57 +02:00
6ebd9b003a DatabaseExperiment: base class for distributed fail experiments
The DatabaseExperiment is a class a concrete experiment can inherit
from. It handles the communication with the campaign server. Does the
fast forward to the fault location, injects the fault and gives the
result over experiment outcome to the child class.

Change-Id: I1fb676da6c704cd570a638f0dfaadd4f1a9845e4
2014-10-22 17:30:07 +02:00
0ba03ca97e cmake: Set PROTOBUF_IMPORT_DIRS in toplevel dir
The variable PROTOBUF_IMPORT_DIRS has to be set in the toplevel
CMakeLists.txt, since the import path has to be available for all .proto
files within all subdirectories. Without this addition, the
GenericExperiment will fail to compile.

Change-Id: I676e0abd83bd1c5d247afcd33e7522e72da3dc2f
2014-10-22 16:35:17 +02:00
5378573b1d util/DwarfReader, ElfImporter: import address ranges
This change implements the following:
-DwarfReader now exports the address range of linetable-entries instead of
	only the first address
-ElfImporter saves this range alongside the mapping

Change-Id: I7fe6361178f761a8f605a44bb0183c56a236cc95
2014-10-22 15:42:05 +02:00
d94b005be2 util/DwarfReader, ElfImporter: use unsigned addresses
This change alters DwarfReader and import-trace's ElfImporter so that they use
unsigned int for static address and line numbers instead of signed int.

Change-Id: I84ebbb500afd7cd4d93b137a35dcf736dc679fab
2014-10-22 14:30:43 +02:00
7ee105016c cmake/bochs: more strict dependency handling for libfailboch_external
The configure step for libfailbochs_external could be executed parallel
to other build steps, which required the files produced by the configure
step. Therefore a race-condition occurred. By giving the configure step
an explicit target name, more correct dependencies could be modeled
within bochs.cmake.

Change-Id: If2d7dafdace23b0eba6efcdff3ed0bfca2423048
2014-10-21 12:39:42 +02:00
66408e83ce Merge branch 'llvm-rtti' 2014-10-01 15:02:00 +02:00
ebbf35c509 Merge branch 'tools-report-unknown-parameters' 2014-10-01 13:58:57 +02:00
d69ac98654 util: MemoryMap test case for hex/octal support
+ abort() on test failure

Change-Id: I3bd4f19615b1234dae8854950e9d3e1b5810c101
2014-09-30 15:12:31 +02:00
2d1b19ea0f Merge "util: hex and octal number support for MemoryMap" 2014-09-29 16:07:31 +02:00
744ae1b7d7 util: hex and octal number support for MemoryMap
The import-trace tool supports a memorymap file as argument. Currently the
import tool accepts only decimal values, but the hexadecimal system is more
common to specify a memory address or range. To avoid a manual translation
between hex and dec values the patch extends the import tool to handle both
types according to the prefix of a value.

Change-Id: I79d0bc03ecf296dfbced8fb33518e8f5a5790366
2014-09-25 19:40:46 +02:00
3fd94abcd3 cmdline: parse parameters in GNU mode
This change sets the option parser to GNU mode, that is, allows
further option parameters after non-option parameters, e.g.:

dump-trace foo.tc -s

instead of

dump-trace -s foo.tc

As Fail* currently works on GNU platforms only, this behavior is the
one presumably expected from users.

Change-Id: I9c55eaf4560cde81ebd0b94214201c8ad02c2b74
2014-09-25 17:14:08 +02:00
6d3ab14365 DatabaseProtobufAdapter: fix translation of unsigned fields
Independent of the protocol specification all integer fields in the
result table are signed. For instance, if I store a kernel IP (e.g.
0xf1....) the value in the database is 0x7fffffff because of the wrong
type.

Therefore, the data fields of the result table should have the same
types as specified in the protocol.

Change-Id: I9154251e4ad67ba70fe86155ebda378c4a9982c2
2014-09-24 16:16:21 +02:00
d962a322ea DatabaseProtobufAdapter: fix string insertion
This change fixes the Protobuf->MySQL bridge for strings, which were
corrupted in rare cases, especially with debug builds of the
DatabaseCampaign.  String columns in result tables from any campaign
up to this point may contain corrupted data.

The core reason for the corruption was that the TypeBridge_string
bound a temporary (a nameless local variable) to the prepared
statement.  This temporary is destroyed before the subsequent call to
mysql_stmt_bind_param(), and the string within can only be referenced
successfully if it has not been overwritten yet.  The solution is to
copy the string to a bridge-internal variable.

Although it might seem that TypeBridge_enum has the same problem, the
protobuf library seems to return references to internal string
constants when retrieving the enum values.

Change-Id: Id127e6b3333d7c304d688e45de9bea44bbc610b0
2014-09-18 19:40:59 +02:00
0208e80dbb Merge branch 'sampling'
Conflicts:
	src/core/cpn/DatabaseCampaign.cc

Change-Id: Ic11d9ce26546bccba11768383a8fda6a3458530f
2014-09-08 15:36:21 +02:00
e87ace65ef util: SumTree test build fix
This fixes a "funny" conflict between the SumTreeTest and
weather-monitor's experiment.hpp.

Note to self: Preprocessor macros are evil.

Change-Id: I3f8c95fe086357db77110c0c53d3120ca839f30a
2014-08-29 11:35:45 +02:00
42182591e5 fix compiler warnings
(DatabaseCampaign; llvmdisassembler)

Change-Id: Ic31758018a0a1ff0ceac81f781eecfc5f8060f89
2014-08-28 12:12:38 +02:00
84cfa2ae42 Merge commit '0da8ba0dec111d78292455bb5f17c6045820db25' 2014-08-28 12:04:34 +02:00
61050b3760 db-campaign: load completed pilots in one query
Instead of issuing a query for every variant, we assemble a set of
variant ids and query `WHERE variant_id in (1,...)'. This has not only
the effect of higher optimization potential for the database, but also
the query is issued before any result can come back. This will avoid an
overfull receive queue within the job server.

Change-Id: I5b1c60f92b97741ce26d9e50760b601929cef44f
2014-08-25 13:50:20 +02:00
0426970dff llvmdisas.: Do not put side-effect code into assert(...)
assert(...) can be optimized away, therefore side-effect code should not
be placed inside an assertion.

Change-Id: I28aee42e53cb105333094d0042a3f6e2cc5b5a30
2014-08-25 12:48:10 +02:00
268d9d4658 db-campaign: Do only load completed pilots from variant
Since we know for which variant we want to have the completed pilots, we
do not have to catch all pilot_ids but only those who of pilots that are
finished and have the correct variant_id. This speeds the startup of the
campaign server enormously when having many completed campaigns in the
database.

Change-Id: I8be584a2dd6d8d7315f30dcb5bff89647353001e
2014-08-25 12:48:10 +02:00
84b7a1d08a Merge branch 'bochs-catchall-io' 2014-07-18 15:00:16 +02:00
87008a05b6 util: LLVM test code compilation fix
This change removes an unnecessary "#ifndef __puma" from the LLVM
disassembler test code and fixes compilation with the latest AspectC++
binaries.

Change-Id: Ibe835a4a6df69255555c668985f15b9cf8fb82b4
2014-07-13 18:43:39 +02:00
2a68a08313 Merge "config: enable commonly used features by default" 2014-07-04 13:45:06 +02:00
c2eb718886 sal/bochs: catch all I/O instructions
Up to now, the IOPortListener only caught invocations of INB/OUTB with
the port number in DX and 8-bit input/output (from/to AL), but none of
the various other methods to access the I/O port range (e.g., by 8-bit
immediate port numbers).  This change is supposed to catch all other
cases, too.

Change-Id: I1180cd9c1d59df600067739817adab684b18a608
2014-07-04 13:43:23 +02:00
2100001497 DatabaseCampaign: use more flexible get_variants()
This change allows DatabaseCampaign users to take advantage of the
improved variant selection methods in the Database class (multiple
uses of --variant/--benchmark possible, plus
--exclude-variant/--exclude-benchmark switches).

Change-Id: Idb1ca04538ff7601b3648cd9ba766aa8690fff6b
2014-07-03 15:49:05 +02:00
0e1ed1feab prune-trace+DBCampaign: default to variant/benchmark %
If no --variant / --benchmark is specified, it's more reasonable to
prune or run *all* variants/benchmarks (using the wildcard "%")
instead of defaulting to "none"/"none".  The trivial case with only
one single variant/benchmark (which may still be "none"/"none" if
import-trace's default is used) is still covered by this new default
behavior.

Change-Id: I0e9001137d5e052183dd74211e2edbcfab749528
2014-07-03 15:42:25 +02:00
9a81ab4222 Database: more flexible get_variants()
This change moves prune-trace's --variants-exclude / --benchmarks-exclude
capabilities to Database::get_variants() to make it available to all users.

Change-Id: Icbc6bb1a3ae7c846d2de40b881f47a9cc1ed7bbf
2014-07-03 15:42:25 +02:00
6300a2b364 util: SumTree implementation
The SumTree implements an efficient tree data structure for
"roulette-wheel" sampling, or "sampling with fault expansion", i.e.,
sampling of trace entries / pilots without replacement and with a
picking probability proportional to the entries' sizes.

For every sample, the naive approach picks a random number between 0
and the sum of all entry sizes minus one.  It then iterates over all
entries and sums their sizes until the sum exceeds the random number.
The current entry gets picked.  The main disadvantage is the linear
complexity, which gets unpleasant for millions of entries.

The core idea behind the SumTree implementation is to maintain the
size sum of groups of entries, kept in "buckets".  Thereby, a bucket
can be quickly jumped over.  To keep bucket sizes (and thereby linear
search times) bounded, more bucket hierarchy levels are introduced
when a defined bucket size limit is reached.

Note that the current implementation is built for a pure growth phase
(when the tree gets filled with pilots from the database), followed by
a sampling phase when the tree gets emptied.  It does not handle a
mixed add/remove case very smartly, although it should remain
functional.

Change-Id: If05e9700bc84761b5bc31006402641e7112b3a72
2014-07-03 15:42:25 +02:00
b92df6592f util: BlackholeLogger, a quiet Logger drop-in replacement
The compiler should be able to completely optimize away side-effect
free usage of this logger.  Can be used as a drop-in replacement for
Loggers to silence logging output for known-good code without having
to remove the corresponding "LOG << ..." code.

Change-Id: Ifb276223f61686773dd6108aafd567e99c88b223
2014-07-03 15:42:25 +02:00
a07a6b3ed8 config: enable commonly used features by default
This enables commonly used event sources and other backend features to
avoid newcomers not getting their first experiments to work.  The
performance impact is not really a problem, but for optimal throughput
unused features can still be disabled.

Change-Id: Id507474eab4647137ce82cb8674fe23f7102975a
2014-07-03 15:05:30 +02:00
9d5737a37d jobclient: fix error reporting
gethostbyname() doesn't set errno but h_errno.  Thus, we need to call
herror() instead of perror() to print an appropriate error message.
(Thanks, Björn.)

Change-Id: I8fd4bdd4af41774dd290151c5ad37090d006f423
2014-06-30 12:21:42 +02:00
960f0881a2 util/smarthops: missing linker dependency
Change-Id: I3017a8824b3bd7ae9547b6c91db6847e5f5b8694
2014-06-27 13:16:25 +02:00
e73ac88d1b util: LLVM disassembler off-by-one
The disassembled memory region's end (variable "End") is exclusive
now.  Up to now, the two branches defining this variable disagreed on
inclusiveness, leading to an infinite loop in one case.

Change-Id: I055fc240f6ec2f4a1d1937e48617c86612cff5c5
2014-06-25 16:43:28 +02:00
c827750090 Merge branch 'failpanda'
Conflicts:
	src/core/comm/DatabaseCampaignMessage.proto.in
	src/core/cpn/CMakeLists.txt
	src/core/cpn/DatabaseCampaign.cc
	src/core/sal/ConcreteCPU.hpp
	src/core/sal/SALConfig.hpp
	src/core/util/CMakeLists.txt

Change-Id: Id86b93d0e3ea4d9963fcc88605eec0603575ec83
2014-06-03 12:24:49 +02:00
1947382c6e sal: avoid include cycle
Enabling both CONFIG_FAST_BREAKPOINTS / CONFIG_FAST_WATCHPOINTS and
CONFIG_EVENT_BREAKPOINTS / CONFIG_EVENT_BREAKPOINTS_RANGE /
CONFIG_EVENT_MEMREAD / CONFIG_EVENT_MEMWRITE led to an (unnecessary)
include cycle.

Change-Id: I8144e3e72da69b98e21a844a4bfded1b77bdce07
2014-06-03 11:47:21 +02:00
0487604c09 sal/panda: reboot is implemented
Change-Id: I2297eb60cd8f17a74a7ede122c8576d74f1251fe
2014-06-03 11:47:21 +02:00