This chapter details a fresh installation of DOMjudge. The first section is a Quick Installation Reference, but that should only be used by those already acquainted with the system. A detailed guide follows after that.
Note: this is not a replacement for the thorough installation instructions below, but more a cheat-sheet for those who've already installed DOMjudge before and need a few hints. When in doubt, always consult the full installation instruction.
External software:
DOMjudge:
./configure
[--enable-fhs] --prefix=<basepath>
.make domserver judgehost docs
or just those
targets you want installed on the current host.make install-{domserver,judgehost,docs}
as root to install the system.bin/dj-setup-database -u root -r
install
on the domserver host.etc/apache.conf
to your Apache configuration, edit
it to your needs, reload web server:
sudo ln -s .../domserver/etc/apache.conf /etc/apache2/conf.d/domjudge.conf &&
sudo apache2ctl graceful
etc/htpasswd-{jury,plugin}
.useradd -d /nonexistent -g nobody -M -n -s /bin/false domjudge-run
useradd -d /nonexistent -g nogroup -s /bin/false domjudge-run
etc/dbpasswords.secret
from the
domserver to all judgehosts to synchronise database passwords.bin/judgedaemon
cd tests make check ./check-judgings
check-judgings
script automatically verifies most of the
test sources, except for a few with multiple possible outcomes; these
have to be verified by hand. Read the sources for a description of
what should (not) happen.
You may want to change the AUTH_METHOD
to FIXED
and set the
environment variable SUBMITBASEURL
to your DOMjudge base URL, e.g.
http://domjudge.example.com/
.
Optionally:
cd bin; ./balloons
;
or use the balloon web interface.bin/dj_make_chroot <chrootdir> <architecture>
$EDITOR judge/chroot-startstop.sh
chroot-startstop.sh
script in etc/judgehost-config.php
and add the following lines to /etc/sudoers
:domjudge ALL=(root) NOPASSWD: /bin/mount -n -t proc --bind /proc proc
domjudge ALL=(root) NOPASSWD: /bin/umount /*/proc
domjudge ALL=(root) NOPASSWD: /bin/mount --bind <chrootdir>/*
domjudge ALL=(root) NOPASSWD: /bin/umount JUDGEDIR/*
This manual assumes you are aware of some of the concepts used within DOMjudge. Here's an overview.
DOMjudge discerns three different kinds of hosts:
Workstation for a team, where they develop their solutions and from which they submit them to the jury system. The only part of DOMjudge that runs here is the optional command line submit client; all other interaction by teams is done with a browser via the web interface.
A host that receives the submissions, runs the database and serves the web pages. This host will run Apache, and MySQL. Optionally these tasks can be further split out to separate machines, but that's normally not necessary and not supported out of the box.
A number of hosts, at least one, that will retrieve submitted solutions from the DOMjudge server, compile and run them and send the results back to the server. Since this is computationally intensive, there should ideally be at least a couple of these. They will run the judgedaemon from DOMjudge. For security and performance reasons it is highly recommended not to use the server as a judgehost.
Note that the judges (persons) are not required and not recommended to work on any of the DOMjudge server or judgehosts. They can just access the system via the jury web interface and working e.g. on judgehosts can interfere with system stability.
The requirements for the deployment of DOMjudge are:
The following software is required for running DOMjudge.
/bin/sh
(e.g. bash or ash)
lib/judge/sh-static
(dash is included for Linux IA32)
The following items are optional, but may be required to use certain functionality.
alert
script.Software required for building DOMjudge from distributed sources.
checktestdata
program for advanced checking of input/output data correctness.Additional software required for building DOMjudge from a Subversion checkout.
checktestdata
script.
In the most basic setup the team workstations only need (next to the tools needed for program development) a web browser. The web interface fully works with any known browser, with the exception of notification of new clarifications in the menu bar. That can be updated without reloading the page by using AJAX. This is supported by any reasonably current browser with JavaScript enabled.
For your convenience, the following command will install needed software on the DOMjudge server as mentioned above when using Debian GNU/Linux, or one of its derivate distributions. Most systems will have the bulk of these packages installed already.
apt-get install gcc g++ make libcurl4-gnutls-dev mysql-server \ apache2 php5 php5-cli libapache2-mod-php5 php5-mysql php-geshi \ ntp sudo procps sharutils \ phpmyadmin xsltproc libboost-regex-dev libgmp3-dev \ linuxdoc-tools transfig texlive-latex-recommended texlive-latex-extra
This is for Debian 5.0 "Lenny", for Debian 4.0 "Etch",
replace libcurl4-gnutls-dev
with libcurl3-dev
.
On a judgehost, the following should be sufficient. The last line shows some example compilers to install for C, C++, Java (GNU), Java (Sun), Haskell and Pascal; change the list as appropriate.
apt-get install make sudo php5-cli php5-mysql ntp xsltproc procps sharutils \ gcc g++ gcj openjdk-6-jre-headless ghc fp-compiler
The DOMjudge build/install system consists of a configure
script and makefiles, but when installing it, some more care has to be
taken than simply running './configure && make &&
make install
'. DOMjudge needs to be installed both on the server
and on the judgehosts. These require different parts of the complete
system to be present and can be installed separately. Within the build
system these parts are referred to as domserver, judgehost
and additionally docs
for all documentation.
When installing from a Subversion checkout, the configure/build system
first has to be bootstrapped. This can be done by running make
dist
, which creates the configure
script and generates
documentation from SGML/LaTeX sources. Note that this requires
additional software as specified in the
software requirements.
There are three different methods for installing DOMjudge:
With this method all DOMjudge related files and programs are installed in a single directory tree which is specified by the prefix option of configure, like
./configure --prefix=$HOME/domjudge
domserver, judgehost,
docs
parts in a subdirectory
$HOME/domjudge/domserver
etc. Note that these
subdirectories can be overridden from the defaults with options
like --with-domserver_root=DIR
, see configure
--help
for a complete list. The prefix defaults to
/opt/domjudge
.
Besides the installed files, there will also be directories for logging, temporary files, submitted sources and judging data:
log
contains all log files.
tmp
contains temporary files.
submissions
(optionally) on the domserver contains all correctly submitted files: as backup only, the database is the authoritative source. Note that this directory must be writable by the web server for this feature to work.
judgings
location on judgehosts where submissions are tested, each in its own subdirectory. The system needs root access to this directory! (for chroot and mounting of proc-fs).
This method of installation is the default and probably most practical for normal purposes as it keeps all files together, hence easily found.
This method installs DOMjudge in directories according to the
Filesystem Hierarchy Standard. It can be enabled by
passing the option --enable-fhs
to configure
and in this case the prefix defaults to /usr/local
.
Files will be placed e.g. in PREFIX/share/domjudge,
PREFIX/bin, /var/log, /tmp, /etc/domjudge
.
The last installation method is meant for developers/maintainers of DOMjudge and does an in-place installation within the source tree. This allows one to immediately see effects when modifying code.
This method requires some special steps which can most easily be run via makefile rules as follows:
make maintainer-conf [CONFIGURE_FLAGS=<extra options for ./configure>] make maintainer-install
su
.
After running the configure
script, the system can be built
and installed. Each of the domserver, judgehost, docs
parts
can be built and installed separately, respectively by:
make domserver && sudo make install-domserver make judgehost && sudo make install-judgehost make docs && make install-docs
runguard
have to be
set. One should not run DOMjudge programs under the root user
however, but under a normal user: runguard
is specifically
installed setuid-root to make this unnecessary and running as root
will give rise to problems, see
runguard: root privileges not dropped in the common problems
section.
For a list of basic make targets, run make
in the source root
directory without arguments.
The following information is meant for developers or other people who want to make changes to the sources.
The Makefiles in the source tree use a recursion mechanism to run make
targets within the relevant subdirectories. The recursion is handled
by the REC_TARGETS
and SUBDIRS
variables and the
recursion step is executed in Makefile.global
. Any target
added to the REC_TARGETS
list will be recursively called in
all directories in SUBDIRS
. Moreover, a local variant of the
target with -l
appended is called after recursing into the
subdirectories, so recursion is depth-first.
The targets dist, clean, distclean, maintainer-clean
are
recursive by default, which means that these call their local
-l
variants in all directories containing a Makefile. This
allows for true depth-first traversal, which is necessary to correctly
run the *clean
targets: otherwise e.g. paths.mk
will
be deleted before subdirectory *clean
targets are called that
depend on information in it.
Configuration of the judge system is mostly done by editing the
configuration file(s) in etc
: domserver-config.php,
judgehost-config.php, common-config.php
for the configuration
options of the domserver, judgehost and shared configuration options
respectively. The latter should be synchronised between domserver and
judgehosts. Descriptions of settings are included in these files.
Besides these settings, there are a few other places where changes can be made to the system, see other configurable scripts.
Configuration of the compilers of the supported languages should be
done separately. For each supported language a shell-script named
compile_<lang>.sh
should be created and placed in
lib/judge
on the judgehosts, where <lang> is the ID of
the language as specified in the database. For more information, see
for example compile_c.sh
, and compile.sh
in
lib/judge
for syntax. Note that compile scripts are included
for the most common languages already.
Interpreted languages and non-statically linked binaries can in
principle also be used, but then the option USE_CHROOT
should
be disabled (or all dependencies be added to the chroot environment).
Interpreted languages do not generate an executable and in principle
do not need a compilation step. However, to be able to use interpreted
languages (also Sun's Java), a script must be generated during the
compilation step, which will function as the executable: the script
must run the interpreter on the source. See compile_perl.sh
and compile_java_javac.sh
in lib/judge
for
examples.
DOMjudge supports the use of Sun Java within a chroot environment. For
this, a chroot environment which includes the Sun Java libraries must
first be built. This can be accomplished with the included script
dj_make_chroot
: run this as root and pass as arguments
the target directory to build the chroot environment in and as second
argument the target machine architecture. Start the script without
arguments for usage information. See also sections
Installation of a judgehost
and
Problems: Java & chroot.
To allow for problems that do not fit within the standard scheme of fixed input and/or output, DOMjudge has the possibility to change the way submissions are run and checked for correctness.
The back-end scripts (compile.sh
, testcase_run.sh
)
that handle the
compilation, running and checking of submissions, call separate
programs for running and comparison of the results. These can be
specialised and adapted to the requirements per problem. For this, one
has to create programs or scripts run_<some-tag>
and/or
compare_<some-tag>
in the lib/judge
directory (see run
and compare
for examples and
usage information). Then one must specify this
<some-tag>
in the special_run and/or special_compare
fields of the problem entry in the MySQL database (empty means that
the default script should be used).
Implementing a special compare script, also called a
validator, can be done in two ways: either write a program
that is called directly (by testcase_run.sh
) or use (a copy
of) the compare_program.sh
script. The latter generates the
XML result file and handles redirection of input/output for you. When
using this wrapper (the easiest method), the jury should write a
checker program which can be called as
$CHECK_PROGRAM <testdata.in> <program.out> <testdata.out>
compare_program.sh
as
shipped is configured to call check_float
, which compares
floating point numbers.
For example, to compare output while ignoring DOS/UNIX newline
differences, one can copy compare_program.sh
to
compare_dos_newline_OK
and in that file set the variable
CHECK_PROGRAM="`which diff`"
and replace the line
"$CHECK_PROGRAM" $CHECK_OPTIONS "$TESTIN" "$PROGRAM" "$TESTOUT" > "$DIFFOUT"
sed -i 's/\r$//' "$TESTOUT" sed 's/\r$//' "$PROGRAM" | diff -a - "$TESTOUT" > "$DIFFOUT"
special_compare
field to dos_newline_OK
. As an
alternative to this modified validator script, one can accept
presentation errors as correct answers by uncommenting the line
'presentation-error' => 'correct',
RESULTS_REMAP
array in the file
etc/judgehost-config.php
.
For more details on modifying validator scripts, see the comments at the top
of the files testcase_run.sh
, compare_program.sh
and (when not using the wrapper) the appendix on the
ICPC validator interface.
DOMjudge supports a presentation-error
result. The default
compare
program returns this result when output only differs
by whitespace; this is counted as an incorrect submission. The wrapper
script compare_program.sh
does not support presentation error
results however.
DOMjudge includes an alerting system. This allows the administrator to receive alerts when important system events happen, e.g. an error occurs, or a submission or judging is made.
These alerts are passed to a plugin script alert
which can
easily be adapted to fit your needs. The default script emits
different beeping sounds for the different messages when the
beep
program is available, but it could for example also be
modified to send a mail on specific issues, connect to monitoring
software like Nagios, etc. For more details, see the script
lib/alert
.
There are a few more places where some configuration of the system can be made. These are sometimes needed in non-standard environments.
bin/dj_make_chroot
on a judgehost some changes to
variables can be made, most notably DEBMIRROR
to
select a Debian mirror site near you.submit/submit_copy.sh
and
lib/judge/chroot-startstop.sh
can be modified
to suit your local environment. See comments in those files for
more information.DOMjudge supports two submission methods: via the command line submit program and via the web interface. From experience, both methods have users that prefer the one above the other.
The command line submit client can send submissions by either using the web interface internally (http protocol, the default), or using a special command line submit protocol, called Dolstra. The latter has some special features but is not usually needed. See Submitdaemon and the Dolstra protocol for details on this.
Using the http protocol with the submit client requires the libcURL library development files at compile time (the submit client is statically linked to libcURL to avoid a runtime dependency).
The database is the authoritative version for submission sources;
file system storage is available as an easy way
to access the source files and as backup. The program
bin/restore_sources2db
is available to recover the submission
table in the database from these files. The command line daemon will
automatically store sources on the file system; the web server needs
write permissions on SUBMITDIR
and ignores file system storage
if these permissions are not set.
DOMjudge uses a MySQL database server for information storage.
The database structure and privileges are included in MySQL
dump files in the sql subdirectory. The default database name is
domjudge
. This can be changed manually in the
etc/dbpasswords.secret
file: the database name as specified
for the jury
user will be used when installing.
Installation of the database is done with bin/dj-setup-database
.
For this, you need an installed and configured MySQL server and
administrator access to it. Run
dj-setup-database [-u <admin_user>] [-p <password>|-r] install
-r
will prompt for a
password; when no user is specified, the mysql client will try to read
credentials from $HOME/.my.cnf
as usual. The command
uninstall
can be passed to dj-setup-database
to
remove the DOMjudge database and users; this deletes all data!
The domjudge database contains a number of tables, some of which need to be manually filled with data before the contest can be run. See the database section of Contest setup for details.
It may be desirable or even necessary to fine tune some MySQL default settings:
max_connections
: The default 100 is too low, because of the
connection caching by Apache threads. 1000 is more appropriate.max_allowed_packet
: The default of 16MB might be too
low when using large testcases. This should be changed both in the
mysql server and client configuration.skip-networking
or bind-address
: By default MySQL
only listens on a local socket, but judgehosts need to connect remotely to
it. When enabling remote connections, you may want to limit it to only the
IP's of judgehosts in the MySQL user configuration (or with firewall rules).The MySQL server is the central place of information storage for DOMjudge. Think well about what to do if the MySQL host fails or loses your data.
A very robust solution is to set up a replicating MySQL server on another host. This will be a hot copy of all data up to the second, and can take over immediately in the event of failure. The MySQL manual has more information about setting this up.
Alternatively, you can make regular backups of your data to another host,
for example with mysqldump
, or use a RAID based system.
Replication can also be used to improve performance, by directing all select-queries to one or more replicated slave servers, while updates will still be done to the master. This is not supported out of the box, and will require making changes to the DOMjudge source.
For the web interface, you need to have a web server (e.g. Apache) installed on the jury system and made sure that PHP correctly works with it. Refer to the documentation of your web server and PHP for details.
You should turn PHP's magic_quotes_*
options off. We
also recommend to turn off register_globals
.
If you want to upload large testcases in the jury web interface,
it may be necessary to raise some PHP limits or you'll get
"memory exhausted" errors. Make sure that the parameters
memory_limit
, upload_max_filesize
and post_max_size
in php.ini
are all well above the size of your largest testcase.
To configure the web server for DOMjudge, use the Apache configuration
snippet from etc/apache.conf
. It contains examples for
configuring the DOMjudge pages with an alias directive, or as a
virtualhost, optionally with SSL; it also contains PHP and security
settings. The Apache configuration snippet by default includes HTTP
basic-auth authentication to the jury and plugin interfaces. A default
user domjudge_jury
with password equal to that in
etc/dbpasswords.secret
is set for the jury interface.
Additional users can be added with the htpasswd
program to
either file etc/htpasswd-{jury,plugin}
.
Reload the web server for changes to take effect.
See also section
Security: webserver privileges for some details on file permissions for the
etc/dbpasswords.secret
and
etc/htpasswd-{jury,plugin}
files.
All DOMjudge daemons and web interface scripts support logging and
debugging in a uniform manner via functions in lib.error.*
.
There are three ways in which information is logged:
stderr
for daemons or to the web page for
web interface scripts (the latter only on serious issues).LOGFILE
, which is set
in each program. Unsetting this variable disables this method.SYSLOG
configuration variable in etc/common-config.php
. This
option gives the flexibility of syslog, such as remote logging.
See the syslog(daemon) documentation for more information.
Unsetting this variable disables this method.VERBOSE
: defaults to LOG_INFO
in
daemons and LOG_ERROR
in the web interface) and for
log file/syslog (LOGLEVEL
: defaults to LOG_DEBUG
).
In case of problems, it is advisable to check the logs for clues.
Extra debugging information can be obtained by setting the config
option DEBUG
to a bitwise-or of the available
DEBUG_*
flags in etc/common-config.php
, to e.g.
generate extra SQL query and timing information in the web interface.
A few extra steps might need to be taken to completely install and configure a judgehost.
For running solution programs under a non-privileged user, a user has to be added to the system(s) that act as judgehost. This user does not need a home-directory or password, so the following command would suffice to add a user `domjudge-run' with minimal privileges.
On RedHat:
useradd -d /nonexistent -g nobody -M -n -s /bin/false domjudge-runOn Debian:
useradd -d /nonexistent -g nogroup -s /bin/false domjudge-run
For other systems check the specifics of your useradd command.
This user must also be configured as the user under which programs run
via configure --enable-runuser=USER
; the default is
domjudge-run
.
When the chroot setting is enabled (default), a static POSIX shell has
to be available for copying it to the chroot environment. For Linux
i386, a static Dash shell is included, which works out of the box. For
other architectures or operating systems, a shell has to be added
manually. Then simply point the lib/sh-static
symlink to this
file.
If you use the default chroot-startstop.sh
script, then
the following lines must be added to /etc/sudoers
:
domjudge ALL=(root) NOPASSWD: /bin/mount -n -t proc --bind /proc proc domjudge ALL=(root) NOPASSWD: /bin/umount /*/proc domjudge ALL=(root) NOPASSWD: /bin/mount --bind <chrootdir>/* domjudge ALL=(root) NOPASSWD: /bin/umount JUDGEDIR/*Here the user
domjudge
must be replaced by the user you
intend to run the judgedaemon as, <chrootdir>
by the path
the chroot environment was installed to and JUDGEDIR
by the
value of judgehost_judgedir
specified by configure
.
Note that <chrootdir>
is different from
CHROOTDIR
as specified in configure
; the first is
the tree from which bind-mounts are made when Sun Java is used, the
latter the directory under which judgings are allowed to be executed
in a chroot environment, and this path is by default set to
judgehost_judgedir
.
The submit client can be built with make submitclient
. There
is no make target to install the submit client, as its location will
very much depend on the environment. You might e.g. want to copy it to
all team computers or make it available on a network filesystem. Note
that if the team computers run a different (version of the) operating
system than the jury systems, then you need to build the submit
client for that OS.
The submit client needs to know the address of the domserver. This
can be passed as a command line option or environment variable. The
latter option makes for easier usage. A sample script
submit_wrapper.sh
is included, which sets this variable.
See that script for more details on how to set this up.
The submit client can also be built under Windows when the Cygwin environment is installed. First the Cygwin setup.exe program must be downloaded and installed with GCC, curl-devel and maybe some more packages included.
When Cygwin is correctly installed with all necessary development
tools, the submit binary can be created by running configure
followed by make submit.exe
in the submit
directory.
There are three sets of documentation available under the doc
directory in DOMjudge:
for administrators of the system (this document),
for judges, describing the jury web interface and giving some general information about this system,
for teams, explaining how to use the system and what restrictions there are.
The team manual is only available in PDF format and must be built from
the LaTeX sources in doc/team
after configuration of the
system. A prebuilt team manual is included, but note that it contains
default/example values for site-specific configuration settings such
as the team web interface URL and judging settings such as the memory
limit. We strongly recommend rebuilding the team manual to include
site-specific settings and also to revise it to reflect your contest
specific environment and rules.
Besides a standard LaTeX installation, the team manual
requires the svn
and expdlist
packages. These are
available in TeX Live in the texlive-latex-extra
package in
any modern Linux distribution. Alternatively, you can download and
install them manually from their respective subdirectories in
http://mirror.ctan.org/macros/latex/contrib.
When the docs
part of DOMjudge is installed and site-specific
configuration set, the team manual can be generated with the command
genteammanual
found under docs/team
. The PDF
document will be placed in the current
directory or a directory given as argument. The option -w WEBBASEURI
can be passed to set the base URI of the DOMjudge webinterface; it
should end with a slash and defaults to http://example.com/domjudge/
.
The following should do it on a Debian-like system:
sudo apt-get install make transfig texlive-latex-extra texlive-latex-recommended cd .../docs/team ./genteammanual [-w http://your.location.example.com/domjudge/] [targetdir]
The team manual is currently available in two languages: English and Dutch. We welcome any translations to other languages.
The administrator's and judge's manuals are available in PDF and HTML format and prebuilt from SGML sources. Rebuilding these is not normally necessary. To rebuild them on a Debian-like system, the following commands should do it:
sudo apt-get install linuxdoc-tools make transfig texlive-latex-recommended make -C doc/admin docs make -C doc/judge docs
To support coloured display of submitted source code in the jury interface, two external classes of syntax highlighters are supported: GeSHi and the PEAR Text_Highlighter class. DOMjudge tries to find either of those in your PHP include path. When none are found, DOMjudge falls back to source code display without highlighting.
If you run a Debian-like system, you can simply install the
php-geshi
package. If not, download GeSHi and place
geshi.php
and the geshi/
directory in
DOMjudge's LIBWWWDIR
(see domserver-static.php for the exact path).
You can install the Text Highlighter system wide with the
PEAR-provided tools, like this: pear install Text_Highlighter
.
Alternatively you can download the source code from the
Text_Highlighter website and unpack that under the LIBWWWDIR
directory (see domserver-static.php for the exact path). Rename the
resulting Text_Highlighter-x.y.z
directory to just Text
.
We advise to install an NTP-daemon (Network Time Protocol) to make sure the time between jury computer and judgehost (and team computers) is in sync.
Next to the public, team and jury web interfaces, DOMjudge also provides a plugin web interface. This web interface is still in development so subject to change. The interface provides contest data from DOMjudge in XML format and is meant to provide external programs (plugins) with data on the contest. This allows for all kinds of extensions beyond the core functionality of DOMjudge such as providing a fancy scoreboard with more statistics, aggregation of scoreboard data for a final presentation during the prize ceremony.
As we are still thinking about possible uses and thus the data to be provided, the exact specification of this interface may change. Also, we are especially interested in feedback and ideas.
There are currently two data-sets provided within the plugin
subdirectory of the DOMjudge web interface, both in XML format:
scoreboard.php
This page provides a representation of the scoreboard. Additionally it includes legend tables for problems, languages, affiliations and team categories. It does not accept any arguments.
event.php
This page provides a representation of events that happened
during the contest, including submissions, judgings, contest
state changes and general clarifications. This page accepts
two arguments fromid
and toid
to limit the
output to events with event ID in that range.
xsd
-files for the exact
structure.
There is some support to upgrade DOMjudge to newer versions. Note that
this functionality is not extensively tested, so when you plan to
upgrade, you are strongly advised to backup the DOMjudge database
and other data before continuing. We also advise to check the
ChangeLog
file for important changes.
Upgrading the filesystem installation is probably best done by installing the new version of DOMjudge in a separate place and transferring the configuration settings from the old version.
There are SQL upgrade scripts to transform the database including its
data to the layout of a newer version. The scripts can be found under
sql/upgrade
and each script applies changes between two
consecutive DOMjudge versions. At the beginning of each script, a check
is performed which will let MySQL bail out with an error if it should
not be applied anymore. Note that the scripts must be applied in order
(sorted by release). These scripts can be applied by running
dj-database-setup upgrade
.