HULA
Documentation

Code submissions

In a code competition you submit the program, not the predictions. We run it in an isolated sandbox against test inputs you never download, then score what it writes. Each competition's Overview and Rules list its exact environment, hardware and time limit — they take precedence over the defaults described here.

01

How it works

  1. 01
    Upload

    You upload a bundle (zip, tar.gz, a single .py or a .ipynb) on the Submit page.

  2. 02
    Sandbox

    We start an isolated container with the competition's environment, extract your bundle into /workspace and mount the test inputs read-only at /data.

  3. 03
    Run

    Your entrypoint runs with the competition's CPU/GPU, memory and time limit. Output is streamed into the run log.

  4. 04
    Score

    The file your code wrote (normally /output/submission.csv) is copied back and scored exactly like an uploaded prediction file.

Code runs count towards your team's daily submission limit (some competitions set a separate limit for code runs). Runs that fail — a crash, a timeout, a missing output file — still count, so test locally first. You can follow a run live on My submissions.

02

Your bundle

A bundle is one of:

  • a .zip or .tar.gz / .tgz archive of your project,
  • a single .py script — it is run directly with python your_script.py,
  • a single .ipynb notebook — it is executed top to bottom, headless.

Archives are extracted into /workspace, which is also the working directory. If everything sits inside one top-level folder (as when you zip a folder), we step into that folder automatically. Junk like __MACOSX/, .DS_Store, .git/ and __pycache__/ is skipped.

bundle layout
my-solution.zip
├── main.py            ← entrypoint (python main.py)
├── model.py
├── requirements.txt   ← optional, install it yourself (see below)
└── weights/
    └── model.pt       ← small enough? ship it. Large? download at run time.

For safety, bundles may only contain regular files: symlinks, absolute paths and .. paths are rejected. Bundles are limited to 10 GB (20 GB extracted) and 50,000 files — but the competition's upload limit is usually much lower.

03

Entrypoint

The competition defines the command that starts your code — by default python main.py. It runs with bash in /workspace, so it must match a file at the top level of your bundle (we check this before starting and tell you which files we found if it doesn't). If the entrypoint is a notebook (e.g. submission.ipynb), it is executed with jupyter nbconvert --to notebook --execute; Jupyter is installed for you, and cell output and errors show up in the log.

04

Inputs & output

  • Inputs: the competition's test inputs are mounted read-only at /data, with the same folder layout as on the Data tab. Writes there fail — use /tmp or /workspace for scratch files.
  • Output: write your predictions to /output/submission.csv (or the path the competition specifies), in the same format as a normal submission file. Create the parent folder if needed. If the file is missing or empty when your code exits, the run fails.
  • Both paths are also available as environment variables — HULA_DATA_DIR and HULA_OUTPUT_PATH — so the same script works locally.
The answers never leave the platform
Only inputs are mounted into the sandbox. The solution file stays on our side, and scoring happens after your code has finished.
05

Environment, network & GPUs

  • Software: a Debian image with the competition's Python version (default 3.11) and pre-installed packages chosen by the hosts (default: numpy, pandas, scikit-learn). Check the competition page for the exact list and versions.
  • Network access is enabled by default — unlike most platforms. You can pip install extra packages, clone a repository or download model weights from the Hugging Face Hub at run time. Hosts can disable network access for a competition; then all outbound connections fail and everything you need must be in the bundle or the base image.
  • Hardware: CPU cores, memory and GPU type (e.g. T4, L4, A10G, A100, H100 — or none) are fixed per competition. The default is 2 CPUs and 8 GB of memory without a GPU. Use torch.cuda.is_available() to check for the GPU and fall back gracefully when testing locally.
  • The sandbox has no credentials or secrets, and each run starts from a clean container — nothing persists between runs.
python
# main.py — install extras at run time (network is on unless the competition disables it)
import subprocess, sys
subprocess.check_call([sys.executable, "-m", "pip", "install", "-q", "-r", "requirements.txt"])

# Download weights from the Hugging Face Hub instead of shipping them
from huggingface_hub import hf_hub_download
ckpt = hf_hub_download("your-org/your-model", "model.pt", revision="3f2c1e9")  # pin a commit!
06

Time limits & failures

Your code gets the competition's time limit (default 30 minutes) measured from when it starts — building the environment and downloading inputs don't count, but run-time installs and downloads do. A run fails with a clear message when:

  • it exceeds the time limit (it is stopped),
  • it runs out of memory (it is killed, usually with exit code 137),
  • it exits with a non-zero exit code — the message includes the last lines of the log,
  • it finishes but didn't write the output file, or the output can't be parsed (then you get the same validation errors as for uploaded files).
07

Logs

Everything your code prints to stdout and stderr appears in the run log on My submissions, live while it runs. Lines starting with [hula] come from the platform (setup, timings, the final status). Logs are capped at about 64 KB: for very chatty runs we keep the beginning and the end and mark the cut, so print progress sparingly (e.g. every N batches) and make sure errors are printed last.

08

Starter example

A complete minimal submission reads the inputs, predicts and writes the CSV. The same example (plus a notebook version) lives in the platform repository under evaluator/examples/.

main.py
"""Minimal HULA code submission: read /data, write /output/submission.csv."""
import csv, os
from pathlib import Path

DATA_DIR = Path(os.environ.get("HULA_DATA_DIR", "/data"))          # read-only test inputs
OUTPUT_PATH = Path(os.environ.get("HULA_OUTPUT_PATH", "/output/submission.csv"))

def predict(features):
    return sum(features) / len(features)   # replace with your model

OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(DATA_DIR / "test.csv") as fin, open(OUTPUT_PATH, "w", newline="") as fout:
    reader, writer = csv.DictReader(fin), csv.writer(fout)
    writer.writerow(["id", "prediction"])
    for row in reader:
        feats = [float(v) for k, v in row.items() if k != "id"]
        writer.writerow([row["id"], predict(feats)])

print("wrote", OUTPUT_PATH)
shell
# Run it locally exactly like the sandbox does (paths via environment variables):
HULA_DATA_DIR=./sample_data HULA_OUTPUT_PATH=./submission.csv python main.py

# Then bundle it (zip the folder contents, or the folder itself — both work):
zip -r my-solution.zip main.py model.py weights/
09

Reproducibility tips

  • Pin versions of everything you install at run time (torch==2.5.1, not torch) and pin Hugging Face downloads to a revision — a submission that worked last week should work the same during verification.
  • Set seeds (Python, NumPy, PyTorch) and prefer deterministic kernels where it's cheap.
  • Don't rely on row order or file names beyond what the competition documents — inputs can be shuffled.
  • Test with a dry run: run your bundle locally against the sample data, from a fresh virtual environment with the competition's package versions, before spending a submission.
  • Write the output early and atomically: write to a temporary file and rename it at the end, so a crash never leaves a half-written CSV.
Questions?
Ask in the competition's Discussion tab, or see the participant guide for submitting, final selection and verification.