Commit Graph

75 Commits

Author SHA1 Message Date
4cb97a7fa5 formatting, typos, comments, details
Change-Id: Iae5f1acb653a694622e9ac2bad93efcfca588f3a
2014-01-22 13:08:13 +01:00
8505ddbb04 jobserver: synchronization cleanup
This change cleans up in/out queue synchronization in the job server.
End-of-jobs conditions are now properly signaled through the
SynchronizedQueue, allowing to resume and abort blocked readers when
no more input is expected.

Change-Id: I3eaf37115ccf8c5b5afe3d971c7109cd62b68906
2014-01-20 22:48:08 +01:00
33b63651ae DatabaseCampaign: MySQL / concurrency fixes
According to
<http://dev.mysql.com/doc/refman/5.5/en/c-api-threaded-clients.html>,
a MySQL connection handle must not be used concurrently with an open
result set and mysql_use_result() in one thread
(DatabaseCampaign::run()), and mysql_query() in another
(DatabaseCampaign::collect_result_thread()).  This indeed leads to
crashes when bounding the outgoing job queue (SERVER_OUT_QUEUE_SIZE),
and maybe even more insidous effects in other cases.  The solution is
to create separate connections for both threads.

Additionally, call mysql_library_init() before spawning any threads.

Change-Id: I2981f2fdc67c9a2cbe8781f1a21654418f621aeb
2014-01-20 18:34:51 +01:00
ec969603d5 Merge commit 'dcd2c021a5ac91d38187d397914e5f51e2fc8819'
Conflicts:
	tools/import-trace/RegisterImporter.cc

Change-Id: I4f49c976bd60badba73c15746aa03c420cb9f77b
2013-09-11 14:38:55 +02:00
d26fc28fa4 cpn/database: include data_width in the fsppilot during prune step
During the prune step the data_width of the injected location was not
propagated before. It is now stored in fsppilot (database layout change!) and
sent in the fsppilot protobuf message.

Change-Id: I0562f6fc8957adea0f8a9fb63469ca5e3f4b7b2d
2013-09-11 10:27:04 +02:00
dcd2c021a5 util: global lock for certain MySQL operations
Even the reentrant libmysqlclient_r has some non-threadsafe operations,
which need to be protected by a global mutex.
<http://dev.mysql.com/doc/refman/5.5/en/c-api-threaded-clients.html>

Change-Id: I444e42f82cf982a6c8f8f2596e8991d0a5009b28
2013-09-10 18:35:44 +02:00
ba7c663551 import-trace: introduce AdvancedMemoryImporter
A MemoryImporter that additionally imports Relyzer-style conditional
branch history, instruction opcodes, and a virtual
duration=time2-time1+1 column (MariaDB 5.2+ only) for fault-space
pruning purposes.

Change-Id: I6764a26fa8aae21655be44134b88fdee85e67ff6
2013-09-10 17:37:26 +02:00
12b539ff75 misc cleanups
This change touches several subsystems, tools and experiments
(sal, util, cmake, import-trace, generic-tracing, nanojpeg), and
changes details not worth separate commits.

Change-Id: Icd1d664d1be5cfc2212dbf77801c271183214d08
2013-09-10 17:37:25 +02:00
96f2f56d5e Merge branch 'register-mapping-fixes' 2013-09-10 11:46:58 +02:00
11513ef78d util: handle missing register mapping gracefully
It's OK if we cannot map every register LLVM knows to a Fail register
ID, but we need to explicitly skip these cases in the
RegisterImporter.

Change-Id: I2152f819fb94aa4de5720c5798b229b66988d382
2013-09-09 16:14:35 +02:00
e4a5a7a592 util: gzstream needs zlib
This change is needed to build on Ubuntu 13.04.

Change-Id: I683ed4427044264f58bc8f7c94cb5fbbff89cd95
2013-09-08 22:15:14 +02:00
203ec6c5cc remove #ifndef __puma from code using LLVM
Contemporary AspectC++ versions can deal with the LLVM headers very
well, and #ifdef __puma stuff in Fail* headers results in
unmaintainable #ifdef __puma blocks in other parts of Fail* (e.g., the
trace importer).

Make sure you're using a 64-bit ac++ when living in a 64-bit userland
(the 32-bit version doesn't know about __int128), and be aware that
AspectC++ r325 introduced a regression that has not been fixed yet.

Change-Id: I5bb759b08995a74b020d44a2b40e9d7a6e18111c
2013-09-04 10:13:48 +02:00
89aff7df65 util: Bochs/Fail register mapping fix
CL and DL are 8 bits wide, not 255.

Change-Id: Ic454215344d08ba2f8374db251792f82d106305b
2013-09-02 12:07:27 +02:00
2108c8932f util: disassembler register/address mapping fix
For def/use pruning, the linear address mapping of the x86
sub-registers (e.g., AX represents the lower 16 bits of EAX) must
overlap.  If it doesn't, e.g., AX and EAX are considered separate
registers by def/use pruning, resulting in a failure to correlate an
EAX def with a subsequent AX use.

The only user of this mapping up to now, RegisterImporter, forced all
register widths to 8 bits, thereby fortunately canceling out this
problem.  Nevertheless it makes no sense to continue encoding a
constant width in these virtual addresses.

Existing trace, fspgroup, fsppilot and result tables may be converted
to the new encoding by using this query:

UPDATE <tablename> SET data_address = ((data_address >> 4) & ~0xf) | data_address & 0xf;

Change-Id: I7a942b78c34f6140803a86af639eeedef3550f34
2013-09-02 09:50:55 +02:00
ff1ceeddac util: silence const warnings with optionparser
We don't use the warned-about part of optionparser's interface
anyways.

Change-Id: I42c4714e4dce4112ea5b6cfb5159ed8b62d4c418
2013-08-27 13:38:40 +02:00
79ff417e9b doc+cmake: LLVM 3.3 is preferred
Change-Id: Ie480efbe5ffd2f0a59da7f185bba80a5355251f9
2013-08-27 13:38:40 +02:00
470d455e34 db: handle multi-row INSERTs
Several database implementations, including, e.g., MySQL and PostgreSQL,
allow multiple rows to be INSERTed at once.  At least for MySQL this
improves INSERT speed significantly.

Change-Id: Ic223f7915053f30b0cb00c567310f3e780c9fa59
2013-08-22 15:11:30 +02:00
cbbd0ec764 db: added insert_id()
Database::insert_id() retrieves the auto-generated ID from the last INSERT
or UPDATE on a table containing an AUTO_INCREMENT column.

Change-Id: I6ec94d1820bb7e77bd37df33145caa93016fd328
2013-08-22 15:11:30 +02:00
760353fb97 util: LLVM disassembler bugfix
Start/End is both inclusive; otherwise we may miss a trailing single-byte
instruction.

Change-Id: Ifb25593886864f7f92b08f13875a12e0a5a0d345
2013-07-11 10:38:53 +02:00
8622c1de12 db: explicitly use MyISAM engine
InnoDB is the default on some setups.

Change-Id: I5cc59854cb88cbec0e7bb7f6aab946252d0bd8e5
2013-07-11 10:38:53 +02:00
bedb9c2eb2 import-trace: introduce InstructionImporter
The InstructionImporter does disassemble the binary and generate read
traces for every instruction byte executed.

Change-Id: I6b8697c711c009e106ed733c74c6ff8f9bbf8ac5
2013-07-05 10:19:58 +02:00
9843b520c1 dbcampaign: select multiple variants/benchmark pairs
The variant/benchmark selection now can use SQL LIKE syntax, all unfinished
pilots from all selected variants are sent to the clients. E.g.:

./cored-voter-server  -v x86-cored-voter -b simple-% -p basic

Will select the fsppilots in the variants:

- x86-cored-voter/simple-ip/basic
- x86-cored-voter/simple-instr/basic

The variant and benchmark information is now sent within the
fsppilot.

Change-Id: I287bfcddc478d0b79d89e156d6f5bf8188674532
2013-07-05 10:19:58 +02:00
40f610b536 util: integrate LLVM-based Disassembler
The LLVM Disassembler infrastructure can be used to analyze many kinds
of ELF Binaries. For every instruction the used and defined registers
is available as well as information about the instruction itself.

Change-Id: I9cc89b6c116ceff7b5143a6f179ae31c4e994d2d
2013-07-05 10:19:57 +02:00
c74c8a2e30 dbutils: enable variant and benchmark selection with LIKE
This does enable % patterns in variant and benchmark selections, like
it is already implemented in the prune-tool.

Change-Id: I7497bfd588d6ffdf851d767bafc660f4872ae92f
2013-07-03 13:43:54 +02:00
d937be90ea util/CommandLine: make non-option args accessible
The argv list must not be temporary for this, and became a class
member.

Change-Id: I159dd2a0d43768a7926b639f93a4ff9f2bfb7c9a
2013-05-29 16:29:09 +02:00
d9c9b43102 dciao-kernelstructs: several experiment fixes.
The previous fault injection experiment was kind of bullshit. This one
is better in several ways:

- sanity check at injection time (correct IP)
- correct counting of kernel_transistions
- copy whole activation scheme

Change-Id: I014eea4d6fe103bc02ffd7bbca95dc56a1a4d9ea
2013-05-29 16:18:22 +02:00
6789a313a9 DCiAOKernelImporter: different injection semantic.
Is now very similar to normal importer, and may be deleted in the future, but
at the moment, this should be merged, since it is the importer used in the
sobres-2013 paper.

This changes the MySQL Schema. instr1_absolute was introduced.

Change-Id: I1bc2919bd14c335beca6d586b7cc0f80767ad7d5
2013-05-29 16:17:03 +02:00
578e7defd0 gem5: compiles again
Change-Id: Ia95f64953af86003c4be4cbeb01a927d6638f1be
2013-05-07 14:41:52 +02:00
2753a165ba ElfReader: Add support for getting a section iterator
Change-Id: I6241ea2de9da1a1e709fae6374df4fc06ef26aaa
2013-04-28 12:51:59 +02:00
0f16f18d75 cosmetics
Change-Id: Ifae805ae1e2dac95324e054af09a7b70f5d5b60c
2013-04-22 14:24:02 +02:00
4d03b7ce5c core/util: MemoryMap learns to materialize in files
Change-Id: I003168d3718491524db91b9f7d855d6b3428961c
2013-04-10 13:01:03 +02:00
331257492e Merge "core/sal: pass commandline parameters by reference" 2013-04-03 11:39:36 +02:00
87b11968c7 DatabaseProtobufAdapter: remove invalid include
The include was an development artifact and caused compilation errors in most
cases.

Change-Id: I6ecf8c1faa52cf6f854112e19aa1f8556cf09930
2013-04-03 10:37:13 +02:00
3fb58de4a5 core/sal: pass commandline parameters by reference
This allows the commandline parameter parser to modify argc, as it finds
arguments for the Fail* client.  Additionally argv is correctly null
terminated when removing arguments.

This fixes a bug introduced in eb17e9ef82.

Change-Id: Iabe84530790ecb7c587b0af139127015aad868d5
2013-04-02 21:47:39 +02:00
f18cddc63c DatabaseCampaign: abstract campain for interaction with MySQL Database
The DatabaseCampaign interacts with the MySQL tables that are created
by the import-trace and prune-trace tools. It does offer all
unfinished experiment pilots from the database to the
fail-clients. Those clients send back a (by the experiment) defined
protobuf message as a result. The custom protobuf message does have to
need the form:

   import "DatabaseCampaignMessage.proto";

   message ExperimentMsg {
       required DatabaseCampaignMessage fsppilot = 1;

       repeated group Result = 2 {
          // custom fields
          required int32 bitoffset = 1;
          optional int32 result = 2;
       }
   }

The DatabaseCampaignMessage is the pilot identifier from the
database. For each of the repeated result entries a row in a table is
allocated. The structure of this table is constructed (by protobuf
reflection) from the description of the message. Each field in the
Result group becomes a column in the result table. For the given
example it would be:

    CREATE TABLE result_ExperimentMessage(
           pilot_id INT,
           bitoffset INT NOT NULL,
           result INT,
           PRIMARY_KEY(pilot_id)
    )

Change-Id: I28fb5488e739d4098b823b42426c5760331027f8
2013-04-02 09:52:42 +02:00
20a137d8d9 cmake/bochs: build --with-all-libs again
Without --with-all-libs (as documented in doc/how-to-build.txt), at least
the "nogui" GUI does not work anymore -- the default for headless
experiments.

(Note the UNDEFINED->UNDEF enum rename; this was necessary due to a vga.h
include side-effect in bochs/gui/svga.cc.)

Change-Id: I1bc3208e905783505a35bbc48ff29f00eef599d6
2013-03-27 17:52:31 +01:00
72a021be38 misc cleanup, comments, whitespace, gitignore
Change-Id: I6250339ddc5807879c98da2d204418e7b4898a73
2013-03-27 17:37:03 +01:00
1d25372283 cosmetics, whitespace
Change-Id: I1212dfeff33d7bcb8b699233279fbf05014ea3bd
2013-03-23 18:06:27 +01:00
95402cee5d util/Database: fail gracefully if db selection didn't work
Change-Id: If4195353859ef4fb845eda5c5ee251a6adf9c5f5
2013-03-23 18:06:27 +01:00
14b7d4e6f4 util/Database: MySQL connection parameters may all come from my.cnf
It doesn't make sense to override database, hostname or username parameters
coming from ~/.my.cnf with arbitrary own defaults.  Instead allow
libmysqlclient to use the builtin default parameter mechanism.

See <https://dev.mysql.com/doc/refman/5.1/en/option-files.html> for what a
~/.my.cnf may look like; example:

[client]
user=hsc
password=secret42
database=fail_demo

Change-Id: I90aadc04375c624d5e906557c8bd4049e5caa17c
2013-03-23 18:06:02 +01:00
28fc41da65 Merge branch 'master' of ssh://vamos.informatik.uni-erlangen.de:29418/fail
Change-Id: I35e86d4b367be395121451a86f2731606f43eeff
2013-03-22 16:51:29 +01:00
3c349e06f3 Documentation: Cleanup
Doxygen docu is now build in ${PROJECT_BINARY_DIR}/doc.
(and other small changes.)

Change-Id: I3fef910307e104f120c4f770526c800cfd01a41f
2013-03-22 16:18:58 +01:00
e3c633c248 tools/import-trace: new tool to import traces into MySQL database
The import tool does support the following import strategies:

- BasicImporter: generates def-use equivalence classes for read and
  write memory accesses
- DCiAOKernelImporter: generates equivalence classes for read access in
  the ciao kernel space.

Change-Id: I8960561d3e14dcf5dffa3ff7a59b61a5e8f7e719
2013-03-22 11:41:25 +01:00
a2554bfc49 cmake: more "advanced" library variables hidden in configuration tool 2013-03-19 22:31:18 +01:00
ad3c185b61 core/util: Added CommandLine interface (for bochs) 2013-03-11 15:29:26 +01:00
c1f32f5a98 ElfReader: add some convenience functions 2013-03-11 15:29:25 +01:00
f586351e79 T32: Dissassembler to evaluate memory instructions.
For the T32 variant we have to evaluate the memory
access instruction to find out, which memory address
was accessed.

Dissassmbly by OpenOCDs arm_disassembler.hpp/.cc:
- fine for ARM / Thumb1
- needs fixes for Thumb2 :( (currently doing that..)
2013-03-11 12:17:53 +01:00
1fe1dbb3ed util: Added disassembler using objdump tool.
The disassembler disassembles an elf file with
an external objdump tool.
The architecture specific objdump must be configured
via cmake (ARCH_TOOL_PREFIX), e.g. arm-none-eabi- for
arm-none-eabi-objdump.
2013-03-05 21:20:25 +01:00
4686c27d3d ElfReader: Support for Section and Symbol size.
- getSection/getSymbol now returns an ElfSymbol reference.

Searching by address now searches if address is within
symbol address and symbol address + size.
So we can test, if we are *within* a function, object or
section and not only at the start address.
2013-03-04 15:18:52 +01:00
0303f00d72 ElfReader::getSectionSize(): return type of definition should equal the return type of the declaration (i.e., guest_address_t)
This caused a compilation error on ARM (+gem5).

git-svn-id: https://www4.informatik.uni-erlangen.de/i4svn/danceos/trunk/devel/fail@2076 8c4709b5-6ec9-48aa-a5cd-a96041d1645a
2013-02-07 11:54:27 +00:00