🎓 A1Learner
🔍
0 of 79 lessons completed

PyTorch What Is a Loss

A loss is one number that says how wrong a prediction is. Training changes the model so this number goes down.

One guess, one loss

The loss here is the distance between the guess and the target. Distance means how far apart the two numbers are.

import torch

target = torch.tensor(10.0)
guess = torch.tensor(7.0)
loss = (guess - target).abs()
print(loss)

A perfect guess

A guess that hits the target exactly earns the smallest possible loss.

import torch

target = torch.tensor(10.0)
guess = torch.tensor(10.0)
loss = (guess - target).abs()
print(loss)

Many guesses at once

One loss per guess turns being wrong into numbers that can be compared.

import torch

target = torch.tensor(10.0)
guesses = torch.tensor([6.0, 9.0, 12.0])
losses = (guesses - target).abs()
print(losses)

Example

import torch

target = torch.tensor(10.0)
guesses = torch.tensor([6.0, 9.0, 12.0])
losses = (guesses - target).abs()

for guess, loss in zip(guesses, losses):
    print(f"Guess {guess.item():.0f}: loss {loss.item():.0f}")

best = torch.argmin(losses)
print(f"Best guess: {guesses[best].item():.0f}")

Key points

  • Loss: one number that scores how wrong the predictions are
  • Target: the correct value the model should predict
  • A lower loss means better predictions
  • A loss of zero means the predictions match the targets exactly
  • Use torch.argmin to find the position of the smallest loss
  • One number stands in for all the mistakes, and training shrinks that number

In this example, the target is 10. The program scores three guesses with the same distance rule. Guess 9 sits one away from the target, so its loss is 1. Guess 6 and guess 12 sit further away, so their losses are bigger. The program then finds the smallest loss with argmin. It prints guess 9 as the best guess.

Note

PyTorch loss classes return a tensor, so a printed loss looks like tensor(3.)

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

One guess has loss 0.2 and another guess has loss 3.1 for the same target. Which guess is better?