🎓 A1Learner
🔍
0 of 87 lessons completed

PyTorch Data Validation

Data validation checks a batch against the rules the model expects, before training starts. Bad rows fail loudly here, with a printed reason.

A range check

The pixel rule here says values run from 0 to 1. One planted pixel breaks the rule.

images = torch.zeros(2, 1, 16, 16)
images[0, 0, 0, 0] = 1.7
print("Min:", float(images.min()), "Max:", round(float(images.max()), 2))
print("Inside 0 to 1:", bool(images.min() >= 0.0 and images.max() <= 1.0))

A label set check

Validation compares what the data holds against what the model expects. Set difference finds the labels that should not exist.

labels = torch.tensor([0, 1, 2, 5])
bad = sorted(set(labels.tolist()) - {0, 1, 2})
print("Unknown labels:", bad)

A dtype check

One comparison checks the dtype. A long batch fails the float32 rule.

print("Float batch dtype ok:", torch.zeros(2, 1, 16, 16).dtype == torch.float32)
print("Long batch dtype ok:", torch.zeros(2, 1, 16, 16, dtype=torch.long).dtype == torch.float32)

Example

import torch

torch.manual_seed(0)


def validate(images, labels):
    problems = []
    if images.dtype != torch.float32:
        problems.append(f"dtype is {images.dtype}, expected torch.float32")
    if images.dim() != 4 or images.shape[1:] != torch.Size([1, 16, 16]):
        problems.append(f"image shape is {tuple(images.shape)}, expected (N, 1, 16, 16)")
    if images.numel() and (float(images.min()) < 0.0 or float(images.max()) > 1.0):
        problems.append(f"pixels run {float(images.min()):.2f} to {float(images.max()):.2f}, expected 0 to 1")
    bad = sorted(set(labels.tolist()) - {0, 1, 2})
    if bad:
        problems.append(f"labels hold {bad}, expected only 0, 1, 2")
    return problems


good_images = torch.zeros(4, 1, 16, 16)
good_labels = torch.tensor([0, 1, 2, 1])
print("Good batch problems:", validate(good_images, good_labels))

# Defect 1: a pixel outside the 0 to 1 range.
bad_range = good_images.clone()
bad_range[0, 0, 3, 3] = 1.7
print("Range defect:", validate(bad_range, good_labels))

# Defect 2: images stored as integers, not floats.
bad_dtype = torch.zeros(4, 1, 16, 16, dtype=torch.long)
print("Dtype defect:", validate(bad_dtype, good_labels))

# Defect 3: a label outside the known set.
bad_labels = torch.tensor([0, 1, 2, 5])
print("Label defect:", validate(good_images, bad_labels))

Key points

  • Validate shapes, dtypes, value ranges, and label sets when data loads
  • Use validation to catch bad rows before they reach the loss
  • A loud failure at load time is cheaper than a silent bad training run

In this example, the validate function checks four rules. The dtype must be float32. The image shape must be (N, 1, 16, 16). Pixels must run from 0 to 1. Labels must come from the set 0, 1, 2. A good batch returns an empty problem list. Then three batches carry one planted defect each. The first plants a pixel of 1.7, and the range message prints. The second stores the images as integers, and the dtype message prints. The third plants a label 5, and the label message prints. Each defect surfaces at load time, with its reason in plain words.

Note

Validation checks the data, not the model, and a valid batch can still train badly.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

A batch holds a label 5, but the model has 3 classes. When should that surface?