Learn how to correctly do a train test split in machine learning with Python. This guide shows how to use scikit-learn, how to apply stratification, and how to prevent data leakage.
Quick answer
A train test split in machine learning Python example uses scikit-learn's train_test_split function to divide the data into a training set and a test set. This keeps the model from overfitting because you check its performance on data it hasn't seen. To do a typical split, import train_test_split from sklearn.model_selection, give it the feature matrix X and the target vector y, and set the test_size (usually 0.2). Use the stratify argument so the class ratios stay the same, and run any preprocessing steps only after the split to avoid data leakage.
The Mechanics of Data Partitioning in Machine Learning
Testing a machine learning model on the same data it was trained on causes a big problem called overfitting. The model can look like it works perfectly because it simply memorizes the training examples, but it will fail on new, unseen data in real use. To get a fair estimate of how well the model will generalize, we split the data into separate parts: one part for training the model and another part for testing its performance. In most projects, we use a holdout split. Typically we assign 70% to 80% of the data for training and keep the remaining 20% to 30% for testing.
This approach assumes both sets are representative samples drawn from the same underlying distribution. If that assumption is broken, or if any information from the test set leaks into the training step, the evaluation results become unreliable.
For a closely related example, see A Fast Loss Model for Cascode GaN-FETs and Real-Time Degradation-Sensitive Control of Solid-State Transformers..
Data Preprocessing and Splitting Order
The order of operations when splitting and scaling data determines whether your model evaluation is valid or corrupted by data leakage.
- 1Incorrect: Scale then Split
Calculating mean and variance on the entire dataset before splitting.
Causes data leakage. Test set statistics influence training, leading to overly optimistic evaluation metrics.
- 2Correct: Split then Scale
Splitting the raw data first, fitting the scaler on the training set, then transforming both sets.
Prevents data leakage. Keeps the test set completely unseen and preserves evaluation integrity.
Step-by-Step Implementation: Train Test Split in Machine Learning Python Example
To implement a clean data partition, we use the scikit-learn library. Rather than generating independent random noise features which contain no learnable patterns, we will use a synthetic classification dataset with controlled, informative features. This ensures the model can actually learn mathematical relationships during training.
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# 1. Generate a synthetic dataset with informative relationships
X, y = make_classification(
n_samples=1000,
n_features=10,
n_informative=8,
n_redundant=2,
random_state=42
)
# 2. Perform the train-test split
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.20,
random_state=42,
stratify=y
)
# 3. Fit preprocessing transformers ONLY on the training set
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
# 4. Transform the test set using the fitted parameters
X_test_scaled = scaler.transform(X_test)
# 5. Train the model
model = LogisticRegression(random_state=42)
model.fit(X_train_scaled, y_train)
# 6. Generate predictions and evaluate performance
train_preds = model.predict(X_train_scaled)
test_preds = model.predict(X_test_scaled)
print(f"Training Accuracy: {accuracy_score(y_train, train_preds):.4f}")
print(f"Test Accuracy: {accuracy_score(y_test, test_preds):.4f}")The code above demonstrates three critical parameters inside the train_test_split function. The test_size parameter defines the exact proportion of the test partition. The random_state parameter seeds the internal pseudo-random number generator, ensuring that anyone running this code receives the exact same split. The stratify parameter maintains the proportion of target classes across both sets.
For a closely related example, see A Novel Fault Diagnosis Method for Train Real‐Time Ethernet Network Based on Physical Layer Electrical Signal Features.
The Critical Danger of Data Leakage during Preprocessing
A common mistake in machine learning engineering is to scale, impute, or select features on the whole data set before doing the train-test split. If you compute the mean and standard deviation of all the data to scale your features, the training set ends up with information about the test set's distribution. This creates a serious form of data leakage. When data leakage happens, the test scores look too good and hide the fact that the model will not generalize well. To avoid this, always split the data first. Fit any preprocessing tools, like StandardScaler or SimpleImputer, using only the training portion.
Then apply those fitted parameters to both the training and test sets. Feature scaling is required for distance-based methods such as Support Vector Machines or for gradient-descent models like Logistic Regression. It is not needed for tree-based algorithms such as Random Forests.
For a closely related example, see Leveraging Information Technology and Data Analytics for Sustainable Corporate Practices: A Case Study on the Impact of Green Finance Policies.
Holdout Split vs. K-Fold Cross-Validation
Choose the optimal evaluation strategy based on dataset size, computational budget, and variance tolerance.
| Factor | Engineering view | Why it matters |
|---|---|---|
| Computational Cost | Holdout: Low (Trains a single model) | K-Fold: High (Trains K separate models) |
| Variance of Performance Estimate | Holdout: High (Highly dependent on the random seed) | K-Fold: Low (Averages performance across multiple iterations) |
| Dataset Size Recommendation | Holdout: Large datasets and deep learning systems | K-Fold: Small-to-medium tabular datasets |
| Hyperparameter Tuning Suitability | Holdout: Requires a third validation set (train-val-test) | K-Fold: Highly compatible with grid or random search |
Stratification: Preserving Class Distributions in Imbalanced Datasets
When dealing with imbalanced data sets, like fraud detection or rare disease diagnosis, a plain random split can cause huge validation errors. If only 1% of your data belong to the positive class, a random split might put all positive samples into the training set and leave the test set with none. Then you cannot measure recall or precision for the minority class.
Adding the argument stratify=y fixes this. It tells the splitter to use stratified random sampling, which keeps the original class distribution in both training and test sets. If your raw data have a 99:1 class ratio, both splits will preserve that exact 99:1 ratio, giving a statistically sound evaluation.
Train-Test Split vs. K-Fold Cross-Validation
The single holdout train-test split is computationally efficient and highly suitable for very large datasets or deep learning models where training times are measured in hours or days. However, for small-to-medium datasets, a single split can introduce high variance; your model's performance metrics might change significantly depending on which random seed you select for the split. K-Fold cross-validation mitigates this variance by dividing the dataset into K equally sized folds. The model is trained K times, each time using a different fold as the test set and the remaining K-1 folds as the training set. This provides a more stable estimate of performance but requires K times more computational resources.
In practice, engineers often use a single train-test split for initial prototyping and deep learning, and K-Fold cross-validation for hyperparameter tuning on tabular datasets.
Selecting the Right Splitting Strategy
Match your data structure to the correct partitioning mechanism to avoid biased evaluations.
| Factor | Engineering view | Why it matters |
|---|---|---|
| Standard Tabular Data | Random holdout or standard K-Fold split. | Assumes observations are independent and classes are balanced. |
| Imbalanced Target Classes | Stratified train-test split or Stratified K-Fold. | Preserves class proportions across training and testing sets. |
| Time-Series / Temporal Data | Chronological split (e.g., TimeSeriesSplit). | Prevents look-ahead bias by keeping training data strictly in the past. |
| Clustered / Grouped Data | Group-based split (e.g., GroupKFold). | Ensures entire groups or subjects are kept together in one partition. |
Advanced Splitting Scenarios: Group and Temporal Splits
Standard random splitting works only when every data point is independent and follows the same distribution. In many engineering problems this isn't true. Two common cases are time-series data and grouped observations. With time-series data, a random split creates look-ahead bias because the model would be trained on data that actually occurs later in time and then asked to predict earlier points. To avoid this, use a temporal split: put all older records in the training set and all newer records in the test set.
For grouped data, such as several measurements taken from the same patient, a random split can put some of a patient's readings in the training set and other readings in the test set. The model may then simply memorize that patient's quirks instead of learning patterns that apply to everyone. In this case, use a group-aware splitter like GroupKFold so that no patient's data appears in both the training and test sets.
Useful research starting points
Key takeaways
- A train-test split is essential to evaluate how well a machine learning model generalizes to unseen data.
- Data leakage occurs when test set statistics are exposed to the model during preprocessing; always fit transformers on the training set only.
- For classification tasks, always use stratification to preserve class distributions across both splits, especially on imbalanced datasets.
- The random_state parameter is critical for reproducibility, ensuring the exact same data split is generated on subsequent runs.
- Time-series and grouped datasets require specialized splitting techniques to prevent chronological look-ahead bias and group leakage.
Questions engineers often ask
Why is random_state used in train_test_split?
The random_state parameter works like a seed for the pseudo-random number generator that shuffles the data before the split. When you set it to a specific integer, the split becomes reproducible. That means every time you run the code you'll get exactly the same training and testing subsets. This consistency is important for debugging and for comparing different model designs.
What is the difference between a validation set and a test set?
During training, we use a validation set to fine-tune hyperparameters, compare different models, and keep the model from overfitting. The test set stays completely hidden until the very end of the project. Then it provides an unbiased estimate of how well the final model will work on completely new data.
Should I apply SMOTE or oversampling before or after splitting?
Apply SMOTE or any other resampling method only after you have done the train-test split, and only to the training set. Resampling before the split creates data leakage because synthetic points made from the test data end up in the training data, which falsely boosts evaluation scores.
What happens if my test_size is too small or too large?
What happens if my test_size is too small or too large?
When the test_size is too small, the evaluation metrics become highly variable and don't reliably show the model's true performance. When the test_size is too large, there isn't enough data left for training, which causes underfitting and lowers the model's performance.
Continue learning: train test split in machine learning python example
Explore our technical guides to build production-grade machine learning pipelines and MLOps workflows.
