JSONL K-Fold Split
Split a JSONL dataset into k cross-validation folds. The shuffle is seeded, so the same seed always produces the same folds — which is what makes a cross-validation result reproducible. Optionally stratify on a field so every fold gets the same class mix.
Split
How the folds are built
Records are shuffled with a seeded generator, then dealt round-robin into k folds — so fold sizes differ by at most one record, no matter how the total divides. With stratification on, the dealing happens within each class, which is what keeps the class ratio the same in every fold.
For fold i, the validation set is fold i and the training set is everything else. Pick Training set for one fold or Validation set for one fold and step Which fold from 1 to k to export all 2k files.
K-fold or a single held-out split?
- Single split (train / val / test) — cheap, and the right default for large datasets and for LLM fine-tuning where each run is expensive.
- K-fold — every record gets used for both training and validation across the k runs, so the estimate has much less variance. Worth it when data is scarce, which is exactly when a single 10% validation slice is too small to trust.
The cost is k training runs instead of one. For fine-tuning a large model that is rarely justified; for a classifier or an embedding-based approach it usually is.
Two ways to get this wrong
- Splitting after duplication. If the same example appears twice and lands in two folds, every fold leaks into every other. Deduplicate first, then split, then confirm with the leakage detector.
- Splitting related records apart. Multiple turns of one conversation, or several records from one document, must stay together — otherwise the model sees part of a test item during training. This tool assigns per record; for grouped data, stratify on the group id, or shuffle by group with Shuffle and split sequentially.
Privacy
Nothing is uploaded. The whole thing runs in this tab, in your own browser. That matters here more than on most tool sites — training data is usually the most sensitive file a team owns.
Frequently asked questions
What is the __fold field?
A 1-based fold number added to each record in annotated mode, so you can filter to any fold later with Filter or in your training script. It is added at the top level and does not touch your existing keys unless you already have a __fold key.
Can I stratify on a nested field?
Yes — use a dotted path such as meta.label. Records missing the field are grouped under __missing and still distributed evenly.
Why is the same seed giving different folds?
Because something else changed: k, the stratification field, the shuffle toggle, or the input itself. All of those feed the assignment. The seed alone only fixes the random order.
Is 5 or 10 folds better?
10 gives a lower-variance estimate and costs twice as much. 5 is the usual compromise. Below 5, the training sets differ enough between folds that the estimate gets noisy again.