Next Previous Contents

2. Installation and configuration

This chapter details a fresh installation of DOMjudge. The first section is a Quick Installation Reference, but that should only be used by those already acquainted with the system. A detailed guide follows after that.

2.1 Quick installation

Note: this is not a replacement for the thorough installation instructions below, but more a cheat-sheet for those who've already installed DOMjudge before and need a few hints. When in doubt, always consult the full installation instruction.

External software:

DOMjudge:

On the domserver host: On the judgehosts: It should be done by now. As a check that (almost) everything works, the set of test sources can be submitted:
cd tests
make check

Then, in the main jury web interface, select the admin link judging verifier to automatically verify most of the test sources, except for a few with multiple possible outcomes; these have to be verified by hand. Read the test sources for a description of what should (not) happen. You may want to change the AUTH_METHOD to FIXED and set the environment variable SUBMITBASEURL to your DOMjudge base URL, e.g. http://domjudge.example.com/.

Optionally:

2.2 Concepts

This manual assumes you are aware of some of the concepts used within DOMjudge. Here's an overview.

DOMjudge discerns three different kinds of hosts:

Team computer

Workstation for a team, where they develop their solutions and from which they submit them to the jury system. The only part of DOMjudge that runs here is the optional command line submit client; all other interaction by teams is done with a browser via the web interface.

DOMjudge server

A host that receives the submissions, runs the database and serves the web pages. This host will run Apache, and MySQL. Optionally these tasks can be further split out to separate machines, but that's normally not necessary and not supported out of the box. Also called domserver for brevity.

Judgehosts

A number of hosts, at least one, that will retrieve submitted solutions from the DOMjudge server, compile and run them and send the results back to the server. Since this is computationally intensive, there should ideally be at least a couple of these. They will run the judgedaemon from DOMjudge. For security and performance reasons it is highly recommended not to use the server as a judgehost.

Note that the judges (persons) are not required and not recommended to work on any of the DOMjudge server or judgehosts. They can just access the system via the jury web interface and working e.g. on judgehosts can interfere with system stability.

2.3 Requirements

System requirements

The requirements for the deployment of DOMjudge are:

Software requirements

The following software is required for running DOMjudge.

The following items are optional, but may be required to use certain functionality.

Software required for building DOMjudge:

Requirements for team workstations

In the most basic setup the team workstations only need (next to the tools needed for program development) a web browser. The web interface fully works with any known browser. With JavaScript disabled, all basic functionality remains working, with the notable exception of multiple file submissions and some convenience functions.

Debian installation command

For your convenience, the following command will install needed software on the DOMjudge server as mentioned above when using Debian GNU/Linux, or one of its derivate distributions like Ubuntu.


apt-get install gcc g++ make libcurl4-gnutls-dev mysql-server \
        apache2 php5 php5-cli libapache2-mod-php5 php5-mysql php-geshi \
        ntp sudo procps phpmyadmin xsltproc \
        libboost-regex-dev libgmp3-dev linuxdoc-tools linuxdoc-tools-text \
        transfig groff texlive-latex-recommended texlive-latex-extra \
        texlive-fonts-recommended

On a judgehost, the following should be sufficient. The last line shows some example compilers to install for C, C++, Java (GNU), Java (Oracle/Sun), Haskell and Pascal; change the list as appropriate.


apt-get install make sudo php5-cli php5-mysql ntp xsltproc procps \
        gcc g++ gcj openjdk-6-jre-headless openjdk-6-jdk ghc fp-compiler

2.4 Installation system

The DOMjudge build/install system consists of a configure script and makefiles, but when installing it, some more care has to be taken than simply running './configure && make && make install'. DOMjudge needs to be installed both on the server and on the judgehosts. These require different parts of the complete system to be present and can be installed separately. Within the build system these parts are referred to as domserver, judgehost and additionally docs for all documentation.

There are three different methods for installing DOMjudge:

Single directory tree

With this method all DOMjudge related files and programs are installed in a single directory tree which is specified by the prefix option of configure, like


./configure --prefix=$HOME/domjudge

This will install each of the domserver, judgehost, docs parts in a subdirectory $HOME/domjudge/domserver etc. These subdirectories can be overridden from the defaults with options like --with-domserver_root=DIR, see configure --help for a complete list. The prefix defaults to /opt/domjudge.

Besides the installed files, there will also be directories for logging, temporary files, submitted sources and judging data:

log

contains all log files.

tmp

contains temporary files.

submissions

(optionally) on the domserver contains all correctly submitted files: as backup only, the database is the authoritative source. Note that this directory must be writable by the web server for this feature to work.

judgings

location on judgehosts where submissions are tested, each in its own subdirectory.

This method of installation is the default and probably most practical for normal purposes as it keeps all files together, hence easily found.

FHS compliant

This method installs DOMjudge in directories according to the Filesystem Hierarchy Standard. It can be enabled by passing the option --enable-fhs to configure and in this case the prefix defaults to /usr/local. Files will be placed e.g. in PREFIX/share/domjudge, PREFIX/bin, /var/log, /tmp, /etc/domjudge.

Maintainer install

Meant for those wishing to do development on the DOMjudge source code. See the appendix with developer information.

After running the configure script, the system can be built and installed. Each of the domserver, judgehost, docs parts can be built and installed separately, respectively by:


make domserver && sudo make install-domserver
make judgehost && sudo make install-judgehost
make docs && make install-docs

Note that even when installing e.g. in your own home directory, root privileges are still required for domserver and judgehost installation, because user and group ownership of password files, some directories and to give sudo access to runguard. One should not run DOMjudge programs and daemons under the root user however, but under a normal user: runguard is specifically designed to be the only part invoked as root (through sudo) to make this unnecessary and running as root will give rise to problems, see runguard: root privileges not dropped in the common problems section.

For a list of basic make targets, run make in the source root directory without arguments.

2.5 Configuration

Configuration of the judge system is mostly done by editing the configuration variables on the page Configuration settings available in the administrator interface. Changes take effect immediately.

Some settings that are tightly coupled to the filesystem can be configured in the files in etc: domserver-config.php, judgehost-config.php, common-config.php for the configuration options of the domserver, judgehost and shared configuration options respectively. The latter should be synchronised between domserver and judgehosts. Descriptions of settings are included in these files. The judgedaemon must be restarted for changes to take effect, while these are directly picked up by the webinterfaces.

Besides these settings, there are a few other places where changes can be made to the system, see other configurable scripts.

2.6 Configuration of languages

Configuration of the compilers of the supported languages should be done separately. For each supported language a shell-script named compile_<lang>.sh should be created and placed in lib/judge on the judgehosts, where <lang> is the ID of the language as specified in the database. For more information, see for example compile_c.sh, and compile.sh in lib/judge for syntax. Note that compile scripts are included for the most common languages already.

Interpreted languages and non-statically linked binaries can in principle also be used, but then the option USE_CHROOT should be disabled (or all dependencies be added to the chroot environment). Interpreted languages do not generate an executable and in principle do not need a compilation step. However, to be able to use interpreted languages (also Oracle's (Sun) Java), a script must be generated during the compilation step, which will function as the executable: the script must run the interpreter on the source. See compile_perl.sh and compile_java_javac.sh in lib/judge for examples.

DOMjudge supports the use of Oracle (Sun) Java within a chroot environment. For this, a chroot environment which includes the Java libraries must first be built. This can be accomplished with the included script dj_make_chroot: run this as root and pass as arguments the target directory to build the chroot environment in and as second argument the target machine architecture. Start the script without arguments for usage information. See also sections Installation of a judgehost and Problems: Java & chroot.

2.7 Configuration of special run and compare programs

To allow for problems that do not fit within the standard scheme of fixed input and/or output, DOMjudge has the possibility to change the way submissions are run and checked for correctness.

The back end script testcase_run.sh that handles the running and checking of submissions, calls separate programs for running and comparison of the results. These can be specialised and adapted to the requirements per problem. For this, one has to create programs or scripts named run_<tag> and/or compare_<tag> in the lib/judge directory, see run and compare for examples and usage information. Then the <tag> must be specified in the special_run and/or special_compare fields of the problem (empty means that the default script run and compare scripts should be used). To simplify the use of custom run and compare programs, DOMjudge comes with wrapper scripts that handle the tedious, standard part. In most cases it will probably be convenient to use these, see run_wrapper and compare_wrapper for details, and the usage explanations below.

Compare programs

Implementing a special compare program, also called a validator, can be done in two ways: either write a program that is called directly (by testcase_run.sh) or use a copy of the compare_wrapper script. In the first case, the compare program must adhere to the ICPC validator interface. In the latter case, the wrapper generates the XML result file and handles redirection of input/output for you. Use this wrapper by copying it to compare_<tag> and let the jury write a checker program which can be called as


check_<tag> <testdata.in> <program.out> <testdata.out>

This program should write some kind of difference to stdout. No output results in a correct verdict and a nonzero exitcode in an internal (system) error. See as an example the included program check_float, which compares floating point numbers. The name of the check program and any parameters can also be modified in the compare_wrapper script.

For example, to compare output while ignoring DOS/UNIX newline differences, one can copy compare_wrapper to compare_dos_newline_OK and in that file set the variable CHECK_PROGRAM="`which diff`" and replace the line


"$CHECK_PROGRAM" $CHECK_OPTIONS "$TESTIN" "$PROGRAM" "$TESTOUT" > "$DIFFOUT"

by the lines
sed -i 's/\r$//' "$TESTOUT"
sed 's/\r$//' "$PROGRAM" | $CHECK_PROGRAM -a - "$TESTOUT" > "$DIFFOUT"

Note that these commands will modify the local copy of the jury testdata, but the original output generated by the team's solution is retained, and a plain diff output is generated. Next, for each problem that you want to use this validator for, set the special_compare field to dos_newline_OK. As an alternative to this modified validator script, one can accept presentation errors as correct answers by adding the mapping
        'presentation-error' => 'correct',

to the results_remap configuration variable.

For more details on modifying validator scripts, see the comments at the top of the files testcase_run.sh, compare_wrapper and (when not using the wrapper) the appendix on the ICPC validator interface.

DOMjudge supports a presentation-error result. The default compare program returns this result when output only differs by whitespace; this is counted as an incorrect submission. The script compare_wrapper does not support presentation error results however. By default presentation errors are remapped to wrong answer; this can be changed with results_remap.

Run programs

Special run programs can be used, for example, to create an interactive problem, where the contestants' program exchanges information with a jury program and receives data depending on its own output. The problem boolfind is included as an example interactive problem, see docs/examples/boolfind.pdf for the description.

Usage is similar to compare programs: you can either create a program run_<tag> yourself, or use the provided wrapper script, which handles bi-directional communication between a jury program and the contestants' program on stdin/stdout.

For the first case, the calling syntax that the program must accept is equal to the calling syntax of run_wrapper, which is documented in that file. When using run_wrapper, you should copy or symlink it to another name run_<tag> and the jury must write a program named exactly runjury_<tag>, accepting the calling syntax


runjury_<tag> <testdata.in> <program.out>

where the arguments are files to read input testdata from and write program output to, respectively. This program will communicate via stdin/stdout with the contestants' program. A special compare program must probably also be created, so the exact data written to <program.out> is not important, as long as the correctness of the contestants' program can be deduced from the contents by the compare program.

2.8 Alerting system

DOMjudge includes an alerting system. This allows the administrator to receive alerts when important system events happen, e.g. an error occurs, or a submission or judging is made.

These alerts are passed to a plugin script alert which can easily be adapted to fit your needs. The default script emits different beeping sounds for the different messages when the beep program is available, but it could for example also be modified to send a mail on specific issues, connect to monitoring software like Nagios, etc. For more details, see the script lib/alert.

2.9 Other configurable scripts

There are a few more places where some configuration of the system can be made. These are sometimes needed in non-standard environments.

2.10 Submission methods

DOMjudge supports two submission methods: via the command line submit program and via the web interface. From experience, both methods have users that prefer the one above the other.

The command line submit client can send submissions by either using the web interface internally (http protocol, the default), or using a special command line submit protocol, called Dolstra. The latter has some special features but is not usually needed. See Submitdaemon and the Dolstra protocol for details on this.

Using the http protocol with the submit client requires the libcURL library development files at compile time (the submit client is statically linked to libcURL to avoid a runtime dependency).

The database is the authoritative version for submission sources; file system storage is available as an easy way to access the source files and as backup. The program bin/restore_sources2db is available to recover the submission table in the database from these files. The command line daemon will automatically store sources on the file system; the web server needs write permissions on <domjudge_submitdir> and ignores file system storage if these permissions are not set.

2.11 Database installation

DOMjudge uses a MySQL database server for information storage.

The database structure and privileges are included in MySQL dump files in the sql subdirectory. The default database name is domjudge. This can be changed manually in the etc/dbpasswords.secret file: the database name as specified in this file will be used when installing.

Installation of the database is done with bin/dj-setup-database. For this, you need an installed and configured MySQL server and administrator access to it. Run


dj-setup-database genpass
dj-setup-database [-u <admin_user>] [-p <password>|-r] install

This first creates the DOMjudge database credentials file etc/dbpasswords.secret (optionally change the random generated password, although it is not needed for normal operation). Then it creates the database and users and inserts some default/example data into the domjudge database. The option -r will prompt for a password for mysql; when no user is specified, the mysql client will try to read credentials from $HOME/.my.cnf as usual. The command uninstall can be passed to dj-setup-database to remove the DOMjudge database and users; this deletes all data!

The domjudge database contains a number of tables, some of which need to be manually filled with data before the contest can be run. See the database section of Contest setup for details.

Fine tuning settings

For Apache, there are countless documents on how to maximise performance. Of particular importance is to ensure that the MaxClients setting is high enough to receive the number of parallel requests you expect, but not higher than your amount of RAM allows.

As for PHP, the use of an opcode cache like the Alternative PHP Cache (Debian package: php-apc) is beneficial for performance. For uploading large testcases, see the section about memory limits.

It may be desirable or even necessary to fine tune some MySQL default settings:

Setting up replication or backups

The MySQL server is the central place of information storage for DOMjudge. Think well about what to do if the MySQL host fails or loses your data.

A very robust solution is to set up a replicating MySQL server on another host. This will be a hot copy of all data up to the second, and can take over immediately in the event of failure. The MySQL manual has more information about setting this up.

Alternatively, you can make regular backups of your data to another host, for example with mysqldump, or use a RAID based system.

Replication can also be used to improve performance, by directing all select-queries to one or more replicated slave servers, while updates will still be done to the master. This is not supported out of the box, and will require making changes to the DOMjudge source.

2.12 Web server configuration

For the web interface, you need to have a web server (e.g. Apache) installed on the domserver and made sure that PHP correctly works with it. Refer to the documentation of your web server and PHP for details.

You should turn PHP's magic_quotes_* options off. We also recommend to turn off register_globals.

To configure the web server for DOMjudge, use the Apache configuration snippet from etc/apache.conf. It contains examples for configuring the DOMjudge pages with an alias directive, or as a virtualhost, optionally with SSL; it also contains PHP and security settings. Reload the web server for changes to take effect.

Jury authentication

Protection of the jury (and plugin) interface happens through HTTP basic-auth configured in Apache. A default user domjudge_jury with password equal to that in etc/dbpasswords.secret is set at installation. You should add accounts for the individual users (admins, judges) that will access the jury interface. These users can be added with the htpasswd program to etc/htpasswd-jury:


htpasswd [<path to etc>]/htpasswd-jury <username>

Individual judge accounts are needed because actions in the jury interface, e.g. who claimed or verified a submission, are tied to this user.

Apache supports many types of authentication backends, so it's also possible to use LDAP, CAS, SAML (Shibboleth) or any other means, as long as this results in a username being presented to DOMjudge. The included apache.conf has examples.

For team authentication, see the relevant section in Contest Setup.

See also section Security: webserver privileges for some details on file permissions for the etc/dbpasswords.secret and etc/htpasswd-{jury,plugin} files.

2.13 Logging & debugging

All DOMjudge daemons and web interface scripts support logging and debugging in a uniform manner via functions in lib.error.*. There are three ways in which information is logged:

Each script also defines a default threshold level for messages to be logged to stderr (VERBOSE: defaults to LOG_INFO in daemons and LOG_ERR in the web interface) and for log file/syslog (LOGLEVEL: defaults to LOG_DEBUG).

In case of problems, it is advisable to check the logs for clues. Extra debugging information can be obtained by setting the config option DEBUG to a bitwise-or of the available DEBUG_* flags in etc/common-config.php, to e.g. generate extra SQL query and timing information in the web interface.

2.14 Installation of a judgehost

A few extra steps might need to be taken to completely install and configure a judgehost.

For running solution programs under a non-privileged user, a user has to be added to the system(s) that act as judgehost. This user does not need a home-directory or password, so the following command would suffice to add a user `domjudge-run' with minimal privileges.

On RedHat:

useradd -d /nonexistent -g nobody -M -n -s /bin/false domjudge-run
On Debian:
useradd -d /nonexistent -g nogroup -s /bin/false domjudge-run

For other systems check the specifics of your useradd command. This user must also be configured as the user under which programs run via configure --enable-runuser=USER; the default is domjudge-run.

Runguard needs to be able to become root for certain operations like changing to the runuser and performing a chroot. The following lines must be added to the end of /etc/sudoers, replacing <domjudge> with the user you intend to run the judgedaemon as, and the full path to runguard.

domjudge ALL=(root) NOPASSWD: <path-to-runguard>/runguard *
domjudge ALL=(root) NOPASSWD: /bin/cp -pR /dev/null ../dev/null
There's a pregenerated /etc/sudoers.d/ snippet in etc/sudoers-domjudge.

When the chroot setting is enabled (default), a static POSIX shell has to be available for copying it to the chroot environment. For Linux i386, a static Dash shell is included, which works out of the box. For other architectures or operating systems, a shell has to be added manually. Then simply point the lib/sh-static symlink to this file.

If you use the default chroot-startstop.sh script, then the following lines must be added to the end of /etc/sudoers:

domjudge ALL=(root) NOPASSWD: /bin/mount -n -t proc --bind /proc proc
domjudge ALL=(root) NOPASSWD: /bin/umount /*/proc
domjudge ALL=(root) NOPASSWD: /bin/mount --bind <chrootdir>/*
domjudge ALL=(root) NOPASSWD: /bin/mount -o remount\,ro\,bind <judgehost_judgedir>/*
domjudge ALL=(root) NOPASSWD: /bin/umount <judgehost_judgedir>/*
Here the user domjudge must be replaced by the user you intend to run the judgedaemon as, <chrootdir> by the path the chroot environment was installed to and <judgehost_judgedir> by the value specified in configure. This is also in the pregenerated /etc/sudoers.d/ snippet in etc/sudoers-domjudge.

2.15 Building and installing the submit client

The submit client can be built with make submitclient. There is no make target to install the submit client, as its location will very much depend on the environment. You might e.g. want to copy it to all team computers or make it available on a network filesystem. Note that if the team computers run a different (version of the) operating system than the jury systems, then you need to build the submit client for that OS.

The submit client needs to know the address of the domserver. This can be passed as a command line option or environment variable. The latter option makes for easier usage. A sample script submit_wrapper.sh is included, which sets this variable. See that script for more details on how to set this up.

The submit client under Windows/Cygwin

The submit client can also be built under Windows when the Cygwin environment is installed. First the Cygwin setup.exe program must be downloaded and installed with GCC, curl-devel and maybe some more packages included.

When Cygwin is correctly installed with all necessary development tools, the submit binary can be created by running configure followed by make submit.exe in the submit directory.

2.16 (Re)generating documentation and the team manual

There are three sets of documentation available under the doc directory in DOMjudge:

the admin-manual

for administrators of the system (this document),

the judge-manual

for judges, describing the jury web interface and giving some general information about this system,

the team-manual

for teams, explaining how to use the system and what restrictions there are.

The team manual is only available in PDF format and must be built from the LaTeX sources in doc/team after configuration of the system. A prebuilt team manual is included, but note that it contains default/example values for site-specific configuration settings such as the team web interface URL and judging settings such as the memory limit. We strongly recommend rebuilding the team manual to include site-specific settings and also to revise it to reflect your contest specific environment and rules.

Besides a standard LaTeX installation, the team manual requires the svn and expdlist packages. These are available in TeX Live in the texlive-latex-extra package in any modern Linux distribution. Alternatively, you can download and install them manually from their respective subdirectories in http://mirror.ctan.org/macros/latex/contrib.

When the docs part of DOMjudge is installed and site-specific configuration set, the team manual can be generated with the command genteammanual found under docs/team. The PDF document will be placed in the current directory or a directory given as argument. The option -w WEBBASEURI can be passed to set the base URI of the DOMjudge webinterface; it should end with a slash and defaults to http://example.com/domjudge/. The following should do it on a Debian-like system:


sudo apt-get install make transfig texlive-latex-extra texlive-latex-recommended
cd .../docs/team
./genteammanual [-w http://your.location.example.com/domjudge/] [targetdir]

The team manual is currently available in two languages: English and Dutch. We welcome any translations to other languages.

The administrator's and judge's manuals are available in PDF and HTML format and prebuilt from SGML sources. Rebuilding these is not normally necessary. To rebuild them on a Debian-like system, the following commands should do it:


sudo apt-get install linuxdoc-tools make transfig groff texlive-latex-recommended
make -C doc/admin docs
make -C doc/judge docs

2.17 Optional features

Source code syntax highlighting

To support coloured display of submitted source code in the jury interface, two external classes of syntax highlighters are supported: GeSHi and the PEAR Text_Highlighter class. DOMjudge includes a copy of GeSHi under the lib/ext/ dir, but tries to find either of those in your PHP include path. When none are found, DOMjudge falls back to source code display without highlighting.

GeSHi

GeSHi is included by default under the lib/ext/ dir.

PEAR Text Highlighter

If you prefer the PEAR Text Highlighter, first move away the lib/ext/geshi directory. You can install the Text Highlighter system wide with the PEAR-provided tools, like this: pear install Text_Highlighter. Alternatively you can download the source code from the Text_Highlighter website and unpack that under the lib/ext/ directory on the domserver. Rename the resulting Text_Highlighter-x.y.z directory to just Text.

NTP time synchronisation

We advise to install an NTP-daemon (Network Time Protocol) to make sure the time between domserver and judgehost (and team computers) is in sync.

The plugin web interface

Next to the public, team and jury web interfaces, DOMjudge also provides a plugin web interface. This web interface is still in beta/development so subject to change. The interface provides contest data from DOMjudge in XML format and is meant to provide external programs (plugins) with data on the contest. This allows for all kinds of extensions beyond the core functionality of DOMjudge such as providing a fancy scoreboard with more statistics, aggregation of scoreboard data for a final presentation during the prize ceremony.

As we are still thinking about possible uses and thus the data to be provided, the exact specification of this interface may change. Also, we are especially interested in feedback and ideas.

There are currently two data-sets provided within the plugin subdirectory of the DOMjudge web interface, both in XML format:

scoreboard.php

This page provides a representation of the scoreboard. Additionally it includes legend tables for problems, languages, affiliations and team categories. It does not accept any arguments.

event.php

This page provides a representation of events that happened during the contest, including submissions, judgings, contest state changes and general clarifications. This page accepts two arguments fromid and toid to limit the output to events with event ID in that range.

See these pages or the accompanying xsd-files for the exact structure.

A nice example plugin is DOMjura by Nicky Gerritsen. This provides a graphical resolver of the scoreboard from the freeze time until end of contest and can be used during the final prize ceremony. It is a reimplementation of the resolver made by Tim deBoer for the ICPC World Finals.

2.18 Upgrading

There is some support to upgrade DOMjudge to newer versions. Note that this functionality is not extensively tested, so when you plan to upgrade, you are strongly advised to backup the DOMjudge database and other data before continuing. We also advise to check the ChangeLog file for important changes.

Upgrading the filesystem installation is probably best done by installing the new version of DOMjudge in a separate place and transferring the configuration settings from the old version.

There are SQL upgrade scripts to transform the database including its data to the layout of a newer version. The scripts can be found under sql/upgrade and each script applies changes between two consecutive DOMjudge versions. At the beginning of each script, a check is performed which will let MySQL bail out with an error if it should not be applied anymore. Note that the scripts must be applied in order (sorted by release). These scripts can be applied by running dj-setup-database upgrade.


Next Previous Contents