stream
stream
¶
Deterministic sample plans and sequential, node-clocked prefetching.
A plan fixes both mixture membership and sample locations before any batch is read. Plans are independently seeded, so restoring a sequence skips directly to its plan without replaying previous batches. Workers only advance a private cursor; the training node controls which prefetched batch is visible.
AsyncStrategy(datasets: Sequence[Dataset], rates: Sequence[float], batch_size: int, split: str = 'train', node: Node | None = None, seed: int = 0)
¶
Cache the current node's batch while prefetching future steps in order.
Without a node, each read advances an owned clock for legacy sequential callers. An explicitly supplied node is advanced only by its owner.
batches(datasets: Sequence[Dataset], rates: Sequence[float], batch_size: int, split: str = 'train', seed: int = 0, start: int = 0, validate: bool = True) -> Iterator[dict[str, np.ndarray]]
¶
Read a host slice of a shared, rank- and nonce-independent sample plan.
Sequence s covers global positions [s * G, (s + 1) * G), where G = batch_size * process_count. Host r owns its contiguous batch_size positions. Keeping G, seed, and dataset contents fixed preserves the samples when host count changes; sampling itself can still repeat dataset rows.