data_select_ifeval

GitHub
mediumscorePython
results

Measured by task-specific score — higher is better.

0.380.42
sota
Claude-Opus-4.6
reward 0.640

Usage

Run the reference answer to verify your environment is set up correctly

$ harbor run -p tasks/data_select_ifeval

Test model

$ harbor run -p tasks/data_select_ifeval \
  -a claude-code -m claude-opus-4-6

Description

Select up to 5,000 training samples from a pool of 50,000 samples spanning 19 data sources (instruction-following, math, code, multilingual, safety, conversation) to maximize instruction-following performance after LoRA fine-tuning. The challenge is figuring out which data sources and samples best improve IFEval prompt-level strict accuracy on Qwen2.5-3B-Instruct.

Files

path
permission
/app/select_data.py✎ Edit
/data/pool.jsonRead-only
/data/pool_metadata.jsonRead-only
/app/train.shRead-only

Rules

  • 01Edit only /app/select_data.py.
  • 02Do NOT modify train_config.yaml, train.sh, convert_to_sharegpt.py.
  • 03Maximum 5,000 selected samples.
  • 04No external network access. Single GPU.

Tags

data-selectioninstruction-followingifevallorasftqwen2.5

Model Results

Click a row to view its trajectory in Live Lab

model
reward
score
Claude-Opus-4.6
0.640
Gemini-3.1-Pro
0.490
Grok-4-20
0.490
Kimi-K2.6
0.480
DeepSeek-V4-Pro
0.460
MiniMax-M2.7
0.430
MiMo-V2.5-Pro
0.380
GLM-5
0.280
Qwen-3.6-Plus
0.280
Hunyuan-3-Preview
0.230
GPT-5.4
0.090
<- all tasks