Split JSONL into Train, Validation, Test
Train, validation and test splitter. Split a JSONL (JSON Lines) dataset into train, val and test sets. Random with seed, optional stratify by key, configurable ratios. Three downloads in one click. Browser-only.
100% client-side. No upload.
Split
Train / Val / Test Splitter
The reproducible split every ML pipeline needs. Drop in a JSONL file, set the ratios (80/10/10 by default), choose a seed so your results are repeatable, and optionally stratify by a label key to keep the class distribution identical across all three splits. Three named files come out: train.jsonl, val.jsonl, test.jsonl. 100% in-browser.
Random split
Shuffles the input with a seeded PRNG (so the same input + same seed always gives the same three files), then takes the first train% for train, the next val% for validation, and the rest for test. The seed defaults to 42 — change it if you want to try a different shuffle.
Stratified split
Set a key (typically the label or class field — label, category,
intent) and the splitter keeps each class's proportion the same in all three
files. Critical when classes are imbalanced: a pure random split can put 0 examples of a
rare class into val/test and silently destroy your evaluation.
When a class is too small to reach every split — three examples cannot be divided 80/10/10 — the summary says so by name rather than leaving you to notice that val has no examples of it. The per-class counts for all three files are printed underneath the download links.
Stratifying builds each split one class at a time, which would leave the output ordered by label; each file is shuffled again at the end so a trainer that does not shuffle for you is not fed all of one class and then all of another.
Leakage: the thing that makes a split worth doing
A split exists so the number you get at the end means something. It stops meaning anything the moment an example in the eval set also appears in training — the model has seen the answer, the score goes up, and nothing tells you. It is the commonest way a fine-tune looks better than it is.
Two shapes of it, and both are handled here rather than left to you:
- The same example twice. Exports overlap, scrapes re-fetch, and augmentation copies. Records whose content matches — not whose JSON text matches, since key order and whitespace differ between two copies of the same conversation — are treated as one unit and assigned to one split together. Set Duplicates to exact matches only for a strict byte-level comparison, or to drop them to keep just the first copy.
- Several examples from one source. Twenty turns of one conversation, or forty paragraphs of one document, are not twenty or forty independent examples. Split them at random and the model trains on half a conversation and is evaluated on the other half. Name the field they share in Group by key and the whole group travels together.
Because whole units are assigned rather than individual rows, the split percentages are targets rather than guarantees — units are placed largest-first into whichever split is furthest below its share, and the achieved percentages are reported next to the ones you asked for. If they are far apart, one group is a large fraction of your data, which is worth knowing on its own.
Why a separate test set?
Standard ML hygiene: use train for fitting, val for hyperparameter tuning and early-stopping, test for the final unbiased evaluation. If you tune on test, your reported metrics will be optimistic and your model will underperform in production.
Ratios that aren't 80/10/10
- Tiny datasets (< 500 rows): 70/15/15 or even 60/20/20 to keep eval sets statistically meaningful.
- Large datasets (> 1M): 98/1/1 or 90/5/5 — when val and test are big enough in absolute terms, give the train set more.
- Fine-tune only (no eval): 90/10/0 — set test to 0 and you get only train and val files.
Tips & common pitfalls
- Sums must be 100. The splitter complains if your three percentages don't add up.
- Duplicates are handled here, not beforehand. Records with the same content are kept in one split by default, so nothing you evaluate on was also trained on. Set Duplicates to drop them to remove the copies entirely, or use the deduplicator if you want to see them first.
- Group leakage is handled too. Put the field that identifies the source —
conversation_id,user_id,doc_id— in Group by key and every record sharing it goes to the same split. - Check the achieved percentages. They are printed next to what you asked for. Grouping makes exact ratios impossible: if one conversation is 30% of the file, some split is getting 30% in one lump.
Before you start
You need a single JSONL file representing your full dataset. The splitter shuffles it (using your seed) and produces three files: train, val and test.
How to use it
- Drop your JSONL or paste it.
- Set the ratios — default 80/10/10 (train/val/test). Any three numbers that sum to 100 work.
- Pin a Seed for reproducible splits.
- Optional: enable Stratify by key to preserve a label distribution across splits.
- Click Split, then download each file.
Stratified split
For classification or labelled data, stratify by the label key (e.g. label, category). The split keeps the proportion of each label roughly equal in all three sets — important when a class is rare.
Tips & common pitfalls
- Dedupe first. Run the prompt deduper before splitting; otherwise duplicate prompts leak from train into val/test and inflate eval numbers.
- Pin the seed for every paper or experiment. Random splits between runs make results unreplicable.
- Don't split, then shuffle each part. The shuffle happens before splitting; reshuffling per-split changes the contract.
Frequently asked questions
Can I do a 90/5/5 or 70/15/15?
Yes — any three positive numbers summing to 100.
Can I skip the test set?
Set test ratio to 0. The tool will produce just train and val.
k-fold cross-validation?
Not yet — on the roadmap.