Lua Processing Files

File processing means reading a whole file and turning its lines into a result. A log file is a file where each line records one event with a level word such as INFO.

Count every line in a log

Write a small log file first, then read it back. One counter grows by one for each line read.

local writer = io.open("log.txt", "w")
writer:write("INFO start\n")
writer:write("WARN low memory\n")
writer:write("ERROR crash\n")
writer:write("INFO done\n")
writer:close()
local reader = io.open("log.txt", "r")
local count = 0
local line = reader:read("*l")
while line do
  count = count + 1
  line = reader:read("*l")
end
reader:close()
print(count)

Count only one level

A line counts only when it starts with the wanted level word. The anchor keeps the match at the start of the line.

local writer = io.open("log.txt", "w")
writer:write("INFO start\n")
writer:write("ERROR first\n")
writer:write("WARN low\n")
writer:write("ERROR second\n")
writer:close()
local reader = io.open("log.txt", "r")
local count = 0
local line = reader:read("*l")
while line do
  if string.match(line, "^ERROR") then
    count = count + 1
  end
  line = reader:read("*l")
end
reader:close()
print(count)

Count every level at once

Each line adds one to the counter for its own level word. Printing the counters by name gives the totals in a fixed order.

local writer = io.open("log.txt", "w")
writer:write("INFO start\n")
writer:write("WARN low\n")
writer:write("ERROR crash\n")
writer:write("INFO done\n")
writer:write("ERROR timeout\n")
writer:close()
local counts = { INFO = 0, WARN = 0, ERROR = 0 }
local reader = io.open("log.txt", "r")
local line = reader:read("*l")
while line do
  local level = string.match(line, "^(%a+)")
  counts[level] = counts[level] + 1
  line = reader:read("*l")
end
reader:close()
print(counts.INFO)
print(counts.WARN)
print(counts.ERROR)

Example

local writer = io.open("app.log", "w")
writer:write("INFO start\n")
writer:write("INFO loaded\n")
writer:write("WARN low memory\n")
writer:write("ERROR failed\n")
writer:write("INFO done\n")
writer:write("ERROR timeout\n")
writer:close()
local counts = { INFO = 0, WARN = 0, ERROR = 0 }
local total = 0
local reader = io.open("app.log", "r")
local line = reader:read("*l")
while line do
  local level = string.match(line, "^(%a+)")
  counts[level] = counts[level] + 1
  total = total + 1
  line = reader:read("*l")
end
reader:close()
print("INFO", counts.INFO)
print("WARN", counts.WARN)
print("ERROR", counts.ERROR)
print("TOTAL", total)

Key points

Log line: each log line starts with a level word that says how serious the event is.
Level word: string.match can pull the first word from the start of a line.
Counter table: a table with one counter per level can hold the running totals.
Count step: each line adds one to the counter for its own level.
Print in order: printing each counter by name gives a fixed order for the summary.

In this example, the program writes six log lines to a file and closes it. It then reads each line, pulls out the level word, and adds one to the matching counter. Finally it prints each level total and the total line count.

Note

Printing counters by name keeps the summary order fixed on every run.

Try it Yourself

Edit the code, then press Run.

Running is simulated

local writer = io.open("app.log", "w")
writer:write("INFO start\n")
writer:write("INFO loaded\n")
writer:write("WARN low memory\n")
writer:write("ERROR failed\n")
writer:write("INFO done\n")
writer:write("ERROR timeout\n")
writer:close()
local counts = { INFO = 0, WARN = 0, ERROR = 0 }
local total = 0
local reader = io.open("app.log", "r")
local line = reader:read("*l")
while line do
  local level = string.match(line, "^(%a+)")
  counts[level] = counts[level] + 1
  total = total + 1
  line = reader:read("*l")
end
reader:close()
print("INFO", counts.INFO)
print("WARN", counts.WARN)
print("ERROR", counts.ERROR)
print("TOTAL", total)

Exercise

A log file has three INFO lines, one WARN line and two ERROR lines. What does a correct summary print for ERROR?

1 2 3 6