scaling_law

GitHub

mediumperplexityPython

results

Measured by test-set perplexity — lower is better.

95.023.0

sota

Claude-Opus-4.6

reward 0.850

Usage

Run the reference answer to verify your environment is set up correctly

$ harbor run -p tasks/scaling_law

Test model

$ harbor run -p tasks/scaling_law \
  -a claude-code -m claude-opus-4-6

Description

Train a language model from scratch on WikiText-103 to achieve the lowest possible test perplexity within a fixed compute budget on a single H100 GPU. You can change the model architecture, size, training hyperparameters, precision, and any other aspect of the training script. The checkpoint must be compatible with the fixed evaluation pipeline (block_size >= 1024).

Files

path

permission

/app/train.py✎ Edit

/app/train.shRead-only

/app/evaluate_local.pyRead-only

/data/wikitext103/train.ptRead-only

Rules

01Edit only /app/train.py.
02No external network access.
03Single GPU (H100), 8 CPUs, 32GB RAM. Time budget: 12 hours.
04Checkpoint must contain model_state_dict and model_config; block_size >= 1024.