christoph/fail
1
Fork 0
Code Releases Activity
Files
1cbd5acab0be07711fabccf161201df6817591d4
fail/doc/how-to-build.txt
Horst Schirmeier eef19b80a0 FAIL* works with LLVM 3.9, 4.0, 5.0 or 6.0 
Change-Id: I5480c3451daac7c8ea6160a9afe5ce557b73afb1
2018-07-27 08:55:09 +02:00

307 lines
14 KiB
Plaintext
Raw Blame History

=========================================================================================
Additional libraries/packages/tools needed for FAIL*:
=========================================================================================
Required for FAIL*:
**********************************************************************
- build-essential (e.g. g++)
- libmysqlclient-dev or libmariadbclient-dev
- protobuf-compiler libprotobuf-dev
- libpcl1-dev
- libboost-thread-dev libboost-system-dev libboost-regex-dev libboost-coroutine-dev libboost-context-dev
- libdwarf-dev libelf-dev
- cmake >=2.8.2
- libfontconfig1-dev
- zlib1g-dev
- binutils-dev libiberty-dev
- AspectC++ (ag++, ac++):
* AspectC++ 1.1 or 1.2 as well as the clang-based >1.2 versions are known to
work and can be obtained from <http://www.aspectc.org>. Make sure you use
the 64-bit version if running in a 64-bit environment.
* In some environments, some AspectC++ versions have issues ingesting
specific system headers. Workaround: build with CMAKE_AGPP_FLAGS
"-D__NO_MATH_INLINES -D__STRICT_ANSI__" or "--c_compiler clang++" (the
latter requires the clang++ compiler).
- optional:
* LLVM 3.9, 4.0, 5.0, or 6.0 (needed for several importers in
tools/import-trace): llvm-3.9-dev, llvm-4.0-dev, llvm-5.0-dev or
llvm-6.0-dev
- built with "make REQUIRES_RTTI=1" (the Debian/Ubuntu packages already
come built this way)
- details below
* a MySQL 5.0+ or MariaDB 5.1+ (MariaDB 5.5 recommended) server
* doxygen
* cmake-curses-gui
* python-numpy and python-matplotlib for the faultspaceplot tool
Required for the Bochs simulator backend:
**********************************************************************
- libpthread
- Probably more, depending on, e.g., the GUI you configure
(bochs_configure_params in the cmake configuration):
SDL/X11 (--with-sdl;--with-x11) -> libsdl1.2-dev
wx (--with-wx) -> libgtk2.0-dev libwxgtk3.0-dev libwxbase3.0-dev
Required for the gem5 simulator backend:
**********************************************************************
- SCons
- SWIG
- zlib
- m4
- python-dev
- optional: libgoogle-perftools-dev
Required for the pandaboard (openocd) backend:
**********************************************************************
- libftdi-dev
- libjim-dev
Required for the compute-hops tool
**********************************************************************
- libprocps0-dev
For distribution/parallelization:
**********************************************************************
- rsync
- tmux
32-bit FailBochs on x86_64 Linux machines:
**********************************************************************
- Create a "bochslibs" directory and fill it with all necessary libraries from
your build machine:
$ mkdir bochslibs
$ cp -v $(ldd fail-client|awk '{print $3}'|egrep -v '\(|lib(pthread|selinux|c.so.)|^$') bochslibs/
- Copy this directory to ~/bochslibs on all machines lacking these libraries
(this may also be the case for i386 machines you cannot install library
packages on yourself). client.sh will add ~/bochslibs to LD_LIBRARY_PATH if
it exists.
=========================================================================================
Compiling, building and modifying: Simulators and FAIL*
=========================================================================================
Building FAIL*:
**********************************************************************
For the first time:
------------------------------------------------------------
1. Enter the "fail/" directory (${FAIL_DIR}, see also "fail-structure.txt"):
$ cd fail/
2. (Optional) Cleanup previous CMake remnants:
$ find -name CMakeCache.txt | xargs rm
3. Create out-of-source build directory (${BUILD_DIR}, see also "fail-structure.txt"):
$ mkdir build
4. Enter out-of-source build directory. All generated files end up there.
$ cd build
5. Generate CMake environment.
$ cmake ..
6. Setup build configuration by opening the CMake configuration tool
$ ccmake .
Select "BUILD_BOCHS" or "BUILD_GEM5". Select an experiment to enable by
naming its "experiments/" subdirectory under "EXPERIMENTS_ACTIVATED".
Configure FAIL* features you need for this experiment by enabling
"CONFIG_*" options. Press 'c', 'g' to regenerate the build system.
(Alternatively use
$ cmake-gui .
for a Qt GUI.) To enable a Debug build, choose "Debug" as the build type,
otherwise choose "Release".
If building FailBochs, optionally set up the bochs configure flags
for your need.
After changes to FAIL* code:
------------------------------------------------------------
Compile (in ${BUILD_DIR}, optionally "add -jN" for parallel building):
$ make
CMake will build all FAIL* libraries and link them with the simulator backend
library to a binary called "fail-client". You may use the shell script
$ ${FAIL_DIR}/scripts/rebuild-bochs.sh [-]
to speed up repetitive tasks regarding Fail/Bochs builds. This script contains
a concise documentation on itself.
Add new FAIL* sources to build chain:
------------------------------------------------------------
To add new source files to the build, see CMakeLists.txt in the subdirectory of the
corresponding component in "${FAIL_DIR}/src/core/", and probably consultate the
CMake docs: http://cmake.org/cmake/help/documentation.html
Add new experiment/plugin/campaign:
------------------------------------------------------------
Look at "${FAIL_DIR}/src/experiments/coolchecksum/" as a template. After creating a
new subdirectory in "experiments/", activate it in the CMake configuration step (see
above).
Generating the Doxygen documentation for FAIL*:
**********************************************************************
To generate the Doxygen documentation, type:
$ make doc
The documentation files (HTML and LaTeX) are located in "${BUILD_DIR}/src/core/doc/".
To open the HTML documentation, type:
$ firefox src/core/doc/html/index.html
(You may want to replace "firefox" with your favourite browser.) The LaTeX docs need
to be compiled previously:
$ cd src/core/doc/latex; make
FailBochs: Bochs configuration features
**********************************************************************
The autotools-based bochs is configured within the FAIL* build run.
The configuration flags can be set within the ccmake configuration (ccmake ${FAIL_DIR}/build)
- Sufficient:
--enable-cpu-level=6;--enable-ne2000;--enable-trace-cache;--enable-gdb-stub;--disable-docbook;--with-nogui
- More simulator features and GUIs (FailBochs default configuration):
--enable-a20-pin;--enable-x86-64;--enable-cpu-level=6;--enable-ne2000;--enable-acpi;--enable-pci;--enable-usb;--enable-trace-cache;--enable-fast-function-calls;--enable-host-specific-asms;--enable-disasm;--enable-readline;--enable-clgd54xx;--enable-fpu;--enable-vmx=2;--enable-monitor-mwait;--enable-cdrom;--enable-sb16=linux;--enable-gdb-stub;--disable-docbook;--with-nogui;--with-x11;--with-wx;--with-sdl
The --with-nogui flag suffices for for "headless" experiments, with the
advantage of removing lots of library dependencies (thus reducing startup
overhead). --with-x11 enables only the "x" (X11), --with-wx only the "wx"
(wxWidgets) GUI. Note that "wx" does not play well together with FAIL*'s
"restore" feature (FailBochs will fall back to "x" if available, or die
trying.)
Once you know everything works as it should, you may want to add the
following flags before running larger campaigns:
--disable-logging --disable-assert-checks
Make sure --enable-repeat-speedups (--enable-all-optimizations implicitly
enables this) is not enabled; you will otherwise encounter funny effects
when tracing REP-prefixed instructions.
FIXME: Remove more redundant flags/libraries
After changes to Bochs code:
------------------------------------------------------------
- Just re-run "make" in ${BUILD_DIR}, or call "scripts/rebuild-bochs.sh -".
The latter automatically runs "make install" after rebuilding fail-client
(and probably the experiment's campaign server).
- Cleaning up (forcing a complete rebuild of libfailbochs.a next time):
$ make bochsallclean
This is especially necessary if you changed a Bochs-affecting aspect header
(.ah), as the build system does not know about Bochs sources depending on
certain aspects.
Debug build:
------------------------------------------------------------
Configure Bochs to use debugging-related compiler flags (expects to be in ${BUILD_DIR}):
$ cd ../simulator/bochs
$ CFLAGS="-g -O0" CXXFLAGS="-g -O0" ./configure --prefix=... ... (see above)
You might additionally want to configure the rest of FAIL* into debug mode by
setting CMAKE_BUILD_TYPE to "Debug" (ccmake, see above).
FIXME: Does this still work?
Profiling-based optimization build:
------------------------------------------------------------
FIXME: ag++ needs to be run with --keep_woven
Configure Bochs to use compiler flags to enable profiling:
$ cd ../simulator/bochs
$ CFLAGS="-fprofile-generate" CXXFLAGS="-fprofile-generate" LDFLAGS="-fprofile-generate" ./configure --prefix=... ... (see above)
Build Bochs normally, and run it. Configure Bochs to use compiler flags to enable
optimizations based on profiling results:
$ CFLAGS="-fprofile-use -Wcoverage-mismatch" CXXFLAGS="-fprofile-use -Wcoverage-mismatch" LDFLAGS="-fprofile-use" ./configure --prefix=... ... (see above)
Benchmarking:
------------------------------------------------------------
Simple instructions/second measurement:
- Configure Bochs with --enable-show-ips (see above)
- Modify the bochsrc: print_timestamps: enabled=1
Comparison IPS numbers are shown in the default "bochsrc". Headless bochsrc configuration,
all aspects disabled, guest system executes endless loop, host CPU Xeon X5470 (3.33GHz):
IPS: 66124283 average = 66964789
Building gem5:
**********************************************************************
For the first time (incl. selecting an experiment):
------------------------------------------------------------
1. Change to the FAIL* directory (expects to be in ${FAIL_DIR}) and
create a new build directory.
$ cd ${FAIL_DIR}; mkdir build; cd build
3. Start the configuration by typing
$ cmake ..
4. Modify the generated configuration according to your needs using
$ ccmake .
At least, you should set an experiment and turn on the configuration flags
BUILD_GEM5 and BUILD_ARM. Additionally, all config options specific for
other back-ends and architectures should be turned OFF, especially
BUILD_BOCHS and BUILD_X86.
5. Typing
$ make (or nice make -jN)
will start the build process of FAIL* and gem5. This automatically
builds the debug variant for ARM targets. (Note that this build will
automatically include the ${FAIL_DIR}/src/core/sal/gem5 as an
"EXTRAS" feature, see gem5/scons build system documentation for
further information. That means, adding another directory containing
gem5-specific code requires to modify the gem5 build command, see
${FAIL_DIR}/cmake/gem5.cmake.) The build process for gem5 currently
uses 9 cores for a parallel build regardless whether you specify a
different core count in the -jN option. This and various other stuff
(e.g. a release build, i.e. using "gem5.fast" instead of "gem5.debug")
can be changed in the aforementioned gem5.cmake file.
NOTE: gem5 currently does not compile with the newer clang-based daily builds
of AspectC++ (ac++). You may need to use the ac++ 1.2 release instead.
After changes to FAIL*/gem5 code (incl. aspect headers):
------------------------------------------------------------
1. Clean the current build by typing
$ make gem5-allclean
(in your build dir ${FAIL_DIR}/build). This cleans the current
FAIL* and gem5 build directories. Note that "make clean" only cleans
the FAIL* build directory. Furthermore, all remaining CMake remnants
should be deleted:
$ find -name CMakeCache.txt | xargs rm
2. Rebuild by typing
$ make (or nice make -jN)
(as in the last step of the previous section).
=========================================================================================
Database backend setup: MySQL / MariaDB
=========================================================================================
1. Install MySQL or MariaDB via your favourite package manager. (MariaDB
offers repositories for all major distributions.)
2. Increase network timeouts in /etc/mysql/my.cnf to a reasonably high value,
campaigns tend to timeout on a regular basis with the default settings:
net_write_timeout = 3600
net_read_timeout = 3600
You may also want to tune other settings, especially increase memory usage.
There's lots of tuning knowledge on the web, and also automatic tuning
scripts such as the "Tuning Primer" <https://launchpad.net/mysql-tuning-primer>
or "MySQL Tuner" <http://mysqltuner.com>.
3. Setup a non-root DB user. <http://dev.mysql.com/doc/refman/5.5/en/adding-users.html>
Create a ~/.my.cnf (mode 0600) to avoid having to pass user name and
password to every program contacting the DB:
[client]
user=XXX
password=XXX
4. Create a database for each FI campaign you want to conduct, and grant all
permissions to your non-root user.
<http://dev.mysql.com/doc/refman/5.5/en/create-database.html>
<http://dev.mysql.com/doc/refman/5.5/en/grant.html>
<http://dev.mysql.com/doc/refman/5.5/en/adding-users.html>
=========================================================================================
Building LLVM from sources
=========================================================================================
If your Linux distribution does not provide a library package for LLVM 3.9 or
newer (the following steps work for LLVM 3.9), and you need LLVM support in
FAIL*, you may need to build LLVM from the sources and install it, e.g.,
locally in your home.
1. Download the source tarball of LLVM 3.9 from http://llvm.org (or use the git
repository http://llvm.org/git/llvm.git and checkout release_39)
2. Configure as needed. On mixed 32/64-bit systems (userland/kernel),
prefixing with "linux32" may be necessary:
$ linux32 ./configure --prefix=$(echo ~/localroot/usr) --enable-optimized --disable-assertions --disable-werror
If you have an old version of clang installed, you may need to convince the
configure script to use gcc/g++ instead:
$ CC=gcc CXX=g++ ./configure --prefix=...
3. Build with "make REQUIRES_RTTI=1", install.
View Git Blame Copy Permalink