addition
addition
¶
AdditionEval()
¶
Bases: RolloutEvaluation
Exact-match rollout evaluation at the addition <mid> boundary.
Given the dataset sequence::
<bos> 2 3 7 + 6 8 2 = <mid> 0 9 1 9 <eos>
get returns the numeric IDs through <mid> as the prompt and the
four digit IDs as the label. RolloutEvaluation removes generated token
0 (<eos>) before exact comparison, so prediction and label have the
same space-separated integer representation.