flash_attention

GitHub

hardruntimeC

results

Measured by wall-clock runtime in seconds — lower is better.

0.75s0.10s

sota

Claude-Opus-4.6

reward 0.850

Usage

Run the reference answer to verify your environment is set up correctly

$ harbor run -p tasks/flash_attention

Test model

$ harbor run -p tasks/flash_attention \
  -a claude-code -m claude-opus-4-6

Description

Implement an efficient scaled dot-product attention computation (output = softmax(QK^T / sqrt(d)) V) for sequence length 4096 and head dimension 64 in float32. The naive approach materializes a full 128 MB score matrix that blows out the CPU cache, so the key challenge is computing attention in a tiled, memory-efficient manner.