If the queue for outbound jobs is not unlimited, experiment rows are fetched
from the DB server continuously as experiments finish. When this takes too
long the connection to the DB server can be lost. The code did not check for
a mysql_error and assumed the result set was fetched completely, thus skipping
a potentially large amount of experiments (in our case only ~20000 of 400000+
experiments were run).
This change adds checks to determine if the result fetch loop was finished due
to an error and checks the sent pilot count to the unfinished experiment count.
Additionally, the mysql result object is correctly freed.
The underlying problem of MySQL connection loss can hopefully be prevented by
increasing timeouts in the MySQL config as described in doc/how-to-build.txt.
To prevent the problem from occurring when this is forgotten, this change
reverts the default job queue length to be unlimited (SERVER_OUT_QUEUE_SIZE=0),
at the cost of increased memory usage.
Change-Id: I09d9faddd8190c6dd5fbe733a0679a733d5837ec
The database queries to fetch all unfinished experiments were broken.
The server tried to insert all finished pilot_ids into the temporary
result_ids table and then discard all experiments which have the correct
(finished) count of IDs in this table. This cannot work as the pilot_id
is the only column of result_ids and must be a unique primary key.
As a fix, the count of results is stored as a second field in result_ids
and the result table is now joined against result_ids to check this field.
Change-Id: I6a9fb774825f0cc4ce104c6e51d7b2fe16957aec
A simple plugin which deterministically returns a new random value
each time the specified symbol is read.
Change-Id: I6ccac421fc064f02a88e8b126f8a26044d1f51c6
Change the aspect weaver project path to all source paths (src, simulators,
debuggers, tools) and the <build>/src path.
Without this change "external" builds do work but do not weave in any
experiments. This change also allows multiple build directories within the
project root, as the weaver will no longer use aspects from all builds.
When further source directories are added to the project root, these
definitions will need to be updated.
Change-Id: If3d3d9fe61427fcba8b815171c8acdcdeb06cb69