Next Previous Contents

2. Installation and configuration

This chapter details a fresh installation of DOMjudge. The first section is a Quick Installation Reference, but that should only be used by those already acquainted with the system. A detailed guide follows after that.

2.1 Quick installation

Note: this is not a replacement for the thorough installation instructions below, but more a cheat-sheet for those who've already installed DOMjudge before and need a few hints. When in doubt, always consult the full installation instruction.

External software:

DOMjudge:

On the domserver host: On the judgehosts: It should be done by now. As a check that (almost) everything works, the set of test sources can be submitted:
cd tests
make check
./check-judgings

The check-judgings script automatically verifies most of the test sources, except for a few with multiple possible outcomes; these have to be verified by hand. Read the sources for a description of what should (not) happen. You may want to change the AUTH_METHOD to FIXED and set the environment variable SUBMITBASEURL to your DOMjudge base URL, e.g. http://domjudge.example.com/.

Optionally:

2.2 Concepts

This manual assumes you are aware of some of the concepts used within DOMjudge. Here's an overview.

DOMjudge discerns three different kinds of hosts:

Team computer

Workstation for a team, where they develop their solutions and from which they submit them to the jury system. The only part of DOMjudge that runs here is the optional command line submit client; all other interaction by teams is done with a browser via the web interface.

DOMjudge server

A host that receives the submissions, runs the database and serves the web pages. This host will run Apache, and MySQL. Optionally these tasks can be further split out to separate machines, but that's normally not necessary and not supported out of the box.

Judgehosts

A number of hosts, at least one, that will retrieve submitted solutions from the DOMjudge server, compile and run them and send the results back to the server. Since this is computationally intensive, there should ideally be at least a couple of these. They will run the judgedaemon from DOMjudge. For security and performance reasons it is highly recommended not to use the server as a judgehost.

Note that the judges (persons) are not required and not recommended to work on any of the DOMjudge server or judgehosts. They can just access the system via the jury web interface and working e.g. on judgehosts can interfere with system stability.

2.3 Requirements

System requirements

The requirements for the deployment of DOMjudge are:

Software requirements

The following software is required for running DOMjudge.

The following items are optional, but may be required to use certain functionality.

Software required for building DOMjudge from distributed sources.

Additional software required for building DOMjudge from a Subversion checkout.

Requirements for team workstations

In the most basic setup the team workstations only need (next to the tools needed for program development) a web browser. The web interface fully works with any known browser, with the exception of notification of new clarifications in the menu bar. That can be updated without reloading the page by using AJAX. This is supported by any reasonably current browser with JavaScript enabled.

Debian installation command

For your convenience, the following command will install needed software on the DOMjudge server as mentioned above when using Debian GNU/Linux, or one of its derivate distributions. Most systems will have the bulk of these packages installed already.


apt-get install gcc g++ make libcurl4-gnutls-dev mysql-server \
        apache2 php5 php5-cli libapache2-mod-php5 php5-mysql php-geshi \
        ntp sudo procps sharutils \
        phpmyadmin xsltproc libboost-regex-dev libgmp3-dev \
        linuxdoc-tools transfig texlive-latex-recommended texlive-latex-extra

This is for Debian 5.0 "Lenny", for Debian 4.0 "Etch", replace libcurl4-gnutls-dev with libcurl3-dev.

On a judgehost, the following should be sufficient. The last line shows some example compilers to install for C, C++, Java (GNU), Java (Sun), Haskell and Pascal; change the list as appropriate.


apt-get install make sudo php5-cli php5-mysql ntp xsltproc procps sharutils \
        gcc g++ gcj openjdk-6-jre-headless ghc fp-compiler

2.4 Installation system

The DOMjudge build/install system consists of a configure script and makefiles, but when installing it, some more care has to be taken than simply running './configure && make && make install'. DOMjudge needs to be installed both on the server and on the judgehosts. These require different parts of the complete system to be present and can be installed separately. Within the build system these parts are referred to as domserver, judgehost and additionally docs for all documentation.

When installing from a Subversion checkout, the configure/build system first has to be bootstrapped. This can be done by running make dist, which creates the configure script and generates documentation from SGML/LaTeX sources. Note that this requires additional software as specified in the software requirements.

There are three different methods for installing DOMjudge:

Single directory tree

With this method all DOMjudge related files and programs are installed in a single directory tree which is specified by the prefix option of configure, like


./configure --prefix=$HOME/domjudge

This will install each of the domserver, judgehost, docs parts in a subdirectory $HOME/domjudge/domserver etc. Note that these subdirectories can be overridden from the defaults with options like --with-domserver_root=DIR, see configure --help for a complete list. The prefix defaults to /opt/domjudge.

Besides the installed files, there will also be directories for logging, temporary files, submitted sources and judging data:

log

contains all log files.

tmp

contains temporary files.

submissions

(optionally) on the domserver contains all correctly submitted files: as backup only, the database is the authoritative source. Note that this directory must be writable by the web server for this feature to work.

judgings

location on judgehosts where submissions are tested, each in its own subdirectory. The system needs root access to this directory! (for chroot and mounting of proc-fs).

This method of installation is the default and probably most practical for normal purposes as it keeps all files together, hence easily found.

FHS compliant

This method installs DOMjudge in directories according to the Filesystem Hierarchy Standard. It can be enabled by passing the option --enable-fhs to configure and in this case the prefix defaults to /usr/local. Files will be placed e.g. in PREFIX/share/domjudge, PREFIX/bin, /var/log, /tmp, /etc/domjudge.

Maintainer install

The last installation method is meant for developers/maintainers of DOMjudge and does an in-place installation within the source tree. This allows one to immediately see effects when modifying code.

This method requires some special steps which can most easily be run via makefile rules as follows:


make maintainer-conf [CONFIGURE_FLAGS=<extra options for ./configure>]
make maintainer-install

Note that these targets have to be executed separately and the latter requests root privileges via su.

After running the configure script, the system can be built and installed. Each of the domserver, judgehost, docs parts can be built and installed separately, respectively by:


make domserver && sudo make install-domserver
make judgehost && sudo make install-judgehost
make docs && make install-docs

Note that even when installing e.g. in your own home directory, root privileges are still required for domserver and judgehost installation, because user and group ownership of password files, some directories and the setuid-root program runguard have to be set. One should not run DOMjudge programs under the root user however, but under a normal user: runguard is specifically installed setuid-root to make this unnecessary and running as root will give rise to problems, see runguard: root privileges not dropped in the common problems section.

For a list of basic make targets, run make in the source root directory without arguments.

Makefile structure

The following information is meant for developers or other people who want to make changes to the sources.

The Makefiles in the source tree use a recursion mechanism to run make targets within the relevant subdirectories. The recursion is handled by the REC_TARGETS and SUBDIRS variables and the recursion step is executed in Makefile.global. Any target added to the REC_TARGETS list will be recursively called in all directories in SUBDIRS. Moreover, a local variant of the target with -l appended is called after recursing into the subdirectories, so recursion is depth-first.

The targets dist, clean, distclean, maintainer-clean are recursive by default, which means that these call their local -l variants in all directories containing a Makefile. This allows for true depth-first traversal, which is necessary to correctly run the *clean targets: otherwise e.g. paths.mk will be deleted before subdirectory *clean targets are called that depend on information in it.

2.5 Configuration

Configuration of the judge system is mostly done by editing the configuration file(s) in etc: domserver-config.php, judgehost-config.php, common-config.php for the configuration options of the domserver, judgehost and shared configuration options respectively. The latter should be synchronised between domserver and judgehosts. Descriptions of settings are included in these files.

Besides these settings, there are a few other places where changes can be made to the system, see other configurable scripts.

2.6 Configuration of languages

Configuration of the compilers of the supported languages should be done separately. For each supported language a shell-script named compile_<lang>.sh should be created and placed in lib/judge on the judgehosts, where <lang> is the ID of the language as specified in the database. For more information, see for example compile_c.sh, and compile.sh in lib/judge for syntax. Note that compile scripts are included for the most common languages already.

Interpreted languages and non-statically linked binaries can in principle also be used, but then the option USE_CHROOT should be disabled (or all dependencies be added to the chroot environment). Interpreted languages do not generate an executable and in principle do not need a compilation step. However, to be able to use interpreted languages (also Sun's Java), a script must be generated during the compilation step, which will function as the executable: the script must run the interpreter on the source. See compile_perl.sh and compile_java_javac.sh in lib/judge for examples.

DOMjudge supports the use of Sun Java within a chroot environment. For this, a chroot environment which includes the Sun Java libraries must first be built. This can be accomplished with the included script dj_make_chroot: run this as root and pass as arguments the target directory to build the chroot environment in and as second argument the target machine architecture. Start the script without arguments for usage information. See also sections Installation of a judgehost and Problems: Java & chroot.

2.7 Configuration of special run and compare programs

To allow for problems that do not fit within the standard scheme of fixed input and/or output, DOMjudge has the possibility to change the way submissions are run and checked for correctness.

The back-end scripts (compile.sh, testcase_run.sh) that handle the compilation, running and checking of submissions, call separate programs for running and comparison of the results. These can be specialised and adapted to the requirements per problem. For this, one has to create programs or scripts run_<some-tag> and/or compare_<some-tag> in the lib/judge directory (see run and compare for examples and usage information). Then one must specify this <some-tag> in the special_run and/or special_compare fields of the problem entry in the MySQL database (empty means that the default script should be used).

Implementing a special compare script, also called a validator, can be done in two ways: either write a program that is called directly (by testcase_run.sh) or use (a copy of) the compare_program.sh script. The latter generates the XML result file and handles redirection of input/output for you. When using this wrapper (the easiest method), the jury should write a checker program which can be called as


$CHECK_PROGRAM <testdata.in> <program.out> <testdata.out>

and this program should write some kind of difference to stdout. No output results in a correct verdict and a nonzero exitcode in an internal (system) error. The script compare_program.sh as shipped is configured to call check_float, which compares floating point numbers.

For example, to compare output while ignoring DOS/UNIX newline differences, one can copy compare_program.sh to compare_dos_newline_OK and in that file set the variable CHECK_PROGRAM="`which diff`" and replace the line


"$CHECK_PROGRAM" $CHECK_OPTIONS "$TESTIN" "$PROGRAM" "$TESTOUT" > "$DIFFOUT"

by the lines
sed -i 's/\r$//' "$TESTOUT"
sed 's/\r$//' "$PROGRAM" | diff -a - "$TESTOUT" > "$DIFFOUT"

Note that these commands will modify the local copy of the jury testdata, but the original output generated by the team's solution is retained, and a plain diff output is generated. Next, for each problem that you want to use this validator for, set the special_compare field to dos_newline_OK. As an alternative to this modified validator script, one can accept presentation errors as correct answers by uncommenting the line
        'presentation-error' => 'correct',

in the RESULTS_REMAP array in the file etc/judgehost-config.php.

For more details on modifying validator scripts, see the comments at the top of the files testcase_run.sh, compare_program.sh and (when not using the wrapper) the appendix on the ICPC validator interface.

DOMjudge supports a presentation-error result. The default compare program returns this result when output only differs by whitespace; this is counted as an incorrect submission. The wrapper script compare_program.sh does not support presentation error results however.

2.8 Alerting system

DOMjudge includes an alerting system. This allows the administrator to receive alerts when important system events happen, e.g. an error occurs, or a submission or judging is made.

These alerts are passed to a plugin script alert which can easily be adapted to fit your needs. The default script emits different beeping sounds for the different messages when the beep program is available, but it could for example also be modified to send a mail on specific issues, connect to monitoring software like Nagios, etc. For more details, see the script lib/alert.

2.9 Other configurable scripts

There are a few more places where some configuration of the system can be made. These are sometimes needed in non-standard environments.

2.10 Submission methods

DOMjudge supports two submission methods: via the command line submit program and via the web interface. From experience, both methods have users that prefer the one above the other.

The command line submit client can send submissions by either using the web interface internally (http protocol, the default), or using a special command line submit protocol, called Dolstra. The latter has some special features but is not usually needed. See Submitdaemon and the Dolstra protocol for details on this.

Using the http protocol with the submit client requires the libcURL library development files at compile time (the submit client is statically linked to libcURL to avoid a runtime dependency).

The database is the authoritative version for submission sources; file system storage is available as an easy way to access the source files and as backup. The program bin/restore_sources2db is available to recover the submission table in the database from these files. The command line daemon will automatically store sources on the file system; the web server needs write permissions on SUBMITDIR and ignores file system storage if these permissions are not set.

2.11 Database installation

DOMjudge uses a MySQL database server for information storage.

The database structure and privileges are included in MySQL dump files in the sql subdirectory. The default database name is domjudge. This can be changed manually in the etc/dbpasswords.secret file: the database name as specified for the jury user will be used when installing.

Installation of the database is done with bin/dj-setup-database. For this, you need an installed and configured MySQL server and administrator access to it. Run


dj-setup-database [-u <admin_user>] [-p <password>|-r] install

to create the database, users and insert some default/example data into the domjudge database. The option -r will prompt for a password; when no user is specified, the mysql client will try to read credentials from $HOME/.my.cnf as usual. The command uninstall can be passed to dj-setup-database to remove the DOMjudge database and users; this deletes all data!

The domjudge database contains a number of tables, some of which need to be manually filled with data before the contest can be run. See the database section of Contest setup for details.

Fine tuning settings

It may be desirable or even necessary to fine tune some MySQL default settings:

Setting up replication or backups

The MySQL server is the central place of information storage for DOMjudge. Think well about what to do if the MySQL host fails or loses your data.

A very robust solution is to set up a replicating MySQL server on another host. This will be a hot copy of all data up to the second, and can take over immediately in the event of failure. The MySQL manual has more information about setting this up.

Alternatively, you can make regular backups of your data to another host, for example with mysqldump, or use a RAID based system.

Replication can also be used to improve performance, by directing all select-queries to one or more replicated slave servers, while updates will still be done to the master. This is not supported out of the box, and will require making changes to the DOMjudge source.

2.12 Web server configuration

For the web interface, you need to have a web server (e.g. Apache) installed on the jury system and made sure that PHP correctly works with it. Refer to the documentation of your web server and PHP for details.

You should turn PHP's magic_quotes_* options off. We also recommend to turn off register_globals. If you want to upload large testcases in the jury web interface, it may be necessary to raise some PHP limits or you'll get "memory exhausted" errors. Make sure that the parameters memory_limit, upload_max_filesize and post_max_size in php.ini are all well above the size of your largest testcase.

To configure the web server for DOMjudge, use the Apache configuration snippet from etc/apache.conf. It contains examples for configuring the DOMjudge pages with an alias directive, or as a virtualhost, optionally with SSL; it also contains PHP and security settings. The Apache configuration snippet by default includes HTTP basic-auth authentication to the jury and plugin interfaces. A default user domjudge_jury with password equal to that in etc/dbpasswords.secret is set for the jury interface. Additional users can be added with the htpasswd program to either file etc/htpasswd-{jury,plugin}. Reload the web server for changes to take effect.

See also section Security: webserver privileges for some details on file permissions for the etc/dbpasswords.secret and etc/htpasswd-{jury,plugin} files.

2.13 Logging & debugging

All DOMjudge daemons and web interface scripts support logging and debugging in a uniform manner via functions in lib.error.*. There are three ways in which information is logged:

Each script also defines a default threshold level for messages to be logged to stderr (VERBOSE: defaults to LOG_INFO in daemons and LOG_ERROR in the web interface) and for log file/syslog (LOGLEVEL: defaults to LOG_DEBUG).

In case of problems, it is advisable to check the logs for clues. Extra debugging information can be obtained by setting the config option DEBUG to a bitwise-or of the available DEBUG_* flags in etc/common-config.php, to e.g. generate extra SQL query and timing information in the web interface.

2.14 Installation of a judgehost

A few extra steps might need to be taken to completely install and configure a judgehost.

For running solution programs under a non-privileged user, a user has to be added to the system(s) that act as judgehost. This user does not need a home-directory or password, so the following command would suffice to add a user `domjudge-run' with minimal privileges.

On RedHat:

useradd -d /nonexistent -g nobody -M -n -s /bin/false domjudge-run
On Debian:
useradd -d /nonexistent -g nogroup -s /bin/false domjudge-run

For other systems check the specifics of your useradd command. This user must also be configured as the user under which programs run via configure --enable-runuser=USER; the default is domjudge-run.

When the chroot setting is enabled (default), a static POSIX shell has to be available for copying it to the chroot environment. For Linux i386, a static Dash shell is included, which works out of the box. For other architectures or operating systems, a shell has to be added manually. Then simply point the lib/sh-static symlink to this file.

If you use the default chroot-startstop.sh script, then the following lines must be added to /etc/sudoers:

domjudge ALL=(root) NOPASSWD: /bin/mount -n -t proc --bind /proc proc
domjudge ALL=(root) NOPASSWD: /bin/umount /*/proc
domjudge ALL=(root) NOPASSWD: /bin/mount --bind <chrootdir>/*
domjudge ALL=(root) NOPASSWD: /bin/umount JUDGEDIR/*
Here the user domjudge must be replaced by the user you intend to run the judgedaemon as, <chrootdir> by the path the chroot environment was installed to and JUDGEDIR by the value of judgehost_judgedir specified by configure. Note that <chrootdir> is different from CHROOTDIR as specified in configure; the first is the tree from which bind-mounts are made when Sun Java is used, the latter the directory under which judgings are allowed to be executed in a chroot environment, and this path is by default set to judgehost_judgedir.

2.15 Building and installing the submit client

The submit client can be built with make submitclient. There is no make target to install the submit client, as its location will very much depend on the environment. You might e.g. want to copy it to all team computers or make it available on a network filesystem. Note that if the team computers run a different (version of the) operating system than the jury systems, then you need to build the submit client for that OS.

The submit client needs to know the address of the domserver. This can be passed as a command line option or environment variable. The latter option makes for easier usage. A sample script submit_wrapper.sh is included, which sets this variable. See that script for more details on how to set this up.

The submit client under Windows/Cygwin

The submit client can also be built under Windows when the Cygwin environment is installed. First the Cygwin setup.exe program must be downloaded and installed with GCC, curl-devel and maybe some more packages included.

When Cygwin is correctly installed with all necessary development tools, the submit binary can be created by running configure followed by make submit.exe in the submit directory.

2.16 (Re)generating documentation and the team manual

There are three sets of documentation available under the doc directory in DOMjudge:

the admin-manual

for administrators of the system (this document),

the judge-manual

for judges, describing the jury web interface and giving some general information about this system,

the team-manual

for teams, explaining how to use the system and what restrictions there are.

The team manual is only available in PDF format and must be built from the LaTeX sources in doc/team after configuration of the system. A prebuilt team manual is included, but note that it contains default/example values for site-specific configuration settings such as the team web interface URL and judging settings such as the memory limit. We strongly recommend rebuilding the team manual to include site-specific settings and also to revise it to reflect your contest specific environment and rules.

Besides a standard LaTeX installation, the team manual requires the svn and expdlist packages. These are available in TeX Live in the texlive-latex-extra package in any modern Linux distribution. Alternatively, you can download and install them manually from their respective subdirectories in http://mirror.ctan.org/macros/latex/contrib.

When the docs part of DOMjudge is installed and site-specific configuration set, the team manual can be generated with the command genteammanual found under docs/team. The PDF document will be placed in the current directory or a directory given as argument. The option -w WEBBASEURI can be passed to set the base URI of the DOMjudge webinterface; it should end with a slash and defaults to http://example.com/domjudge/. The following should do it on a Debian-like system:


sudo apt-get install make transfig texlive-latex-extra texlive-latex-recommended
cd .../docs/team
./genteammanual [-w http://your.location.example.com/domjudge/] [targetdir]

The team manual is currently available in two languages: English and Dutch. We welcome any translations to other languages.

The administrator's and judge's manuals are available in PDF and HTML format and prebuilt from SGML sources. Rebuilding these is not normally necessary. To rebuild them on a Debian-like system, the following commands should do it:


sudo apt-get install linuxdoc-tools make transfig texlive-latex-recommended
make -C doc/admin docs
make -C doc/judge docs

2.17 Optional features

Source code syntax highlighting

To support coloured display of submitted source code in the jury interface, two external classes of syntax highlighters are supported: GeSHi and the PEAR Text_Highlighter class. DOMjudge tries to find either of those in your PHP include path. When none are found, DOMjudge falls back to source code display without highlighting.

GeSHi

If you run a Debian-like system, you can simply install the php-geshi package. If not, download GeSHi and place geshi.php and the geshi/ directory in DOMjudge's LIBWWWDIR (see domserver-static.php for the exact path).

PEAR Text Highlighter

You can install the Text Highlighter system wide with the PEAR-provided tools, like this: pear install Text_Highlighter.

Alternatively you can download the source code from the Text_Highlighter website and unpack that under the LIBWWWDIR directory (see domserver-static.php for the exact path). Rename the resulting Text_Highlighter-x.y.z directory to just Text.

NTP time synchronisation

We advise to install an NTP-daemon (Network Time Protocol) to make sure the time between jury computer and judgehost (and team computers) is in sync.

The plugin web interface

Next to the public, team and jury web interfaces, DOMjudge also provides a plugin web interface. This web interface is still in development so subject to change. The interface provides contest data from DOMjudge in XML format and is meant to provide external programs (plugins) with data on the contest. This allows for all kinds of extensions beyond the core functionality of DOMjudge such as providing a fancy scoreboard with more statistics, aggregation of scoreboard data for a final presentation during the prize ceremony.

As we are still thinking about possible uses and thus the data to be provided, the exact specification of this interface may change. Also, we are especially interested in feedback and ideas.

There are currently two data-sets provided within the plugin subdirectory of the DOMjudge web interface, both in XML format:

scoreboard.php

This page provides a representation of the scoreboard. Additionally it includes legend tables for problems, languages, affiliations and team categories. It does not accept any arguments.

event.php

This page provides a representation of events that happened during the contest, including submissions, judgings, contest state changes and general clarifications. This page accepts two arguments fromid and toid to limit the output to events with event ID in that range.

See these pages or the accompanying xsd-files for the exact structure.

2.18 Upgrading

There is some support to upgrade DOMjudge to newer versions. Note that this functionality is not extensively tested, so when you plan to upgrade, you are strongly advised to backup the DOMjudge database and other data before continuing. We also advise to check the ChangeLog file for important changes.

Upgrading the filesystem installation is probably best done by installing the new version of DOMjudge in a separate place and transferring the configuration settings from the old version.

There are SQL upgrade scripts to transform the database including its data to the layout of a newer version. The scripts can be found under sql/upgrade and each script applies changes between two consecutive DOMjudge versions. At the beginning of each script, a check is performed which will let MySQL bail out with an error if it should not be applied anymore. Note that the scripts must be applied in order (sorted by release). These scripts can be applied by running dj-database-setup upgrade.


Next Previous Contents