lora_finetune

GitHub
mediumruntimePython
results

Measured by wall-clock runtime in seconds — lower is better.

45.0s3.5s

Usage

Run the reference answer to verify your environment is set up correctly

$ harbor run -p tasks/lora_finetune

Test model

$ harbor run -p tasks/lora_finetune \
  -a claude-code -m claude-opus-4-6

Description

Run 100 training steps of LoRA on a single transformer block: Q, K, V, O projections (d_model=4096, rank=16). The baseline computes W_eff = W + A @ B, materializing a 4096×4096 matrix per projection. The reference keeps W frozen, computes x@W + x@A@B separately, and only backprops through A and B. Memory and compute scale with rank, not full dimension.

Files

path
permission
/app/solve.py✎ Edit
/app/main.pyRead-only

Rules

  • 01Edit /app/solve.py only.
  • 02Allowed imports: numpy only. No PyTorch or compiled extensions.
  • 03Gradient checksum must match reference within 1e-5. Wrong results score 0.
  • 04Frozen weights W must not be modified.

Tags

LoRAlow-rankfine-tuningPEFTtransformermemory
<- all tasks