Executors

This package provides several SIO Workers jobs, which can be used to run arbitrary binary programs under a supervisor program in a secure way.

These jobs also allows to check the correctness of the output produced by the program, either using the default compare program distributed in an appropriate sandbox, or by using an external checker provided by the user.

Job parameters

The following parameters are recognized in environ:

exe_file
Filetracker path of the binary to run.
in_file
Filetracker path of the file to be passed to program’s stdin.
out_file
(optional) Filetracker path where the output produced by the program will be saved.
upload_out

Should the out_file be uploaded to the Filetracker remote store? If False, then the file is only stored in the Filetracker local cache.

Ignored if no out_file is specified.

Default: False

exec_mem_limit

Memory limit in kB.

Default: 66000 kB

exec_time_limit

Time limit in milliseconds.

Default: 30 sec.

exec_out_limit

Program output limit in bytes.

Default: 50 MB

check_output

Should the program output be checked?

The output is checked only if no error was reported by the supervisor, i.e. the program terminated with exit code 0, obeying the resource limits.

Default: False

hint_file
Filetracker path of the model output file for the checker. Mandatory if check_output is True.
chk_file

Filetracker path of the output checker binary. See Custom output checker.

Default: compare program from the sandbox.

untrusted_checker
Pass True to run chk_file in sandbox.
checker_mem_limit, checker_time_limit, checker_out_limit
Just like for executing program, but for checker. Only difference is default memory limit raised to 256MiB

Parameters added to the environment:

result_code
A short code describing the result: OK, OLE (Output Limit Exceeded), RV (Rule Violation), MLE (Memory Limit Exceeded), TLE (Time Limit Exceeded).
result_string
A slightly more elaborate description of the status. Only present if available. The string may come either from the supervisor of the output checker.
result_percentage
What percent of maximum score for test should this solution get on this test? This value is returned only if output checking is active. Custom output checkers can control this value. The default one returns either 100 or 0.
time_used
CPU time used, in milliseconds (only user time is counted, not system).
mem_used
Maximum amount of virtual memory used, in kB. Lower-bound estimate.
num_syscalls
Number of system calls performed.

Custom output checker

The output checker can be any binary program. It is recommended to compile it statically in 32-bit mode. This program is run by SIO Workers in the following way:

# ./checker in_file out_file hint_file

The exit code of the checker is ignored. It should output up to three lines in the following format:

OK or WRONG
[one-line comment]
[float value --- percentage of full score, only if OK]

for example:

OK

or:

WRONG
not enough edges, expected 15, read 25

or:

OK
program scored 40 points, max. was 50
80

Anything different than OK in the first line (including nothing) is treated as WRONG.

Builtin jobs

Name Secure Prerequisites Info
unsafe-exec No None This job provides simple resource management relying on ulimit.
cpu-exec Yes exec-sandbox Executes programs in a dedicated, secure sandbox. Because time used by real cpu is returned, no other job will be executed simultaneously.
vcpu-exec Yes vcpu_exec-sandbox This is machine-independent execution job, which uses instruction counting for meansuring “runtime” of programs. It uses a secure sandbox as well.

Shell scripts

The package provides a convenience shell script sio-compile which mimicks SIO1’s compile.sh script. It expects three arguments: input file name, output file name and programming language source file extension (optionally).

Defining new executors

  1. (Optional) Create new executing environment: Executors (environment)
  2. Copy-and-paste code from sio/workers/common.py, adjust accordingly.
  3. Add to entry_points in setup.py.