🎓 A1Learner
🔍
0 of 46 lessons completed

Bash Awk Patterns and Counting

An awk pattern is a test that picks which lines an action runs on. Counting adds one to a stored total each time a pattern matches.

BEGIN, a pattern, and END

The BEGIN block prints a heading first. Only lines with ERROR run the middle action, and END prints a closing line last.

printf 'INFO start\nERROR disk full\nINFO retry\nERROR timeout\n' > log.txt
awk 'BEGIN{print "Problems:"} /ERROR/ {print $0} END{print "Scan done"}' log.txt

Two pattern-action pairs

Each pair tests the score field on its own. A score of 72 matches neither pair, so ben prints nothing.

printf 'ada 95\nben 72\ncy 45\n' > scores.txt
awk '$2 >= 90 {print $1, "top"} $2 < 60 {print $1, "review"}' scores.txt

Counting with an associative array

Each word adds one to its own stored count. The counts are piped to LC_ALL=C sort, so the keys print in byte order.

printf 'apple\nbanana\napple\napple\nbanana\ncherry\n' > items.txt
awk '{count[$1]++} END{for (k in count) print k, count[k]}' items.txt | LC_ALL=C sort

Example

printf 'GET 200\nPOST 404\nGET 200\nGET 500\nPOST 200\n' > hits.txt
echo "Status report"
awk '{count[$2]++} END{for (k in count) print k, count[k]}' hits.txt | LC_ALL=C sort
awk 'END{print "total", NR}' hits.txt

Key points

  • BEGIN: a block that runs once, before any input line is read.
  • END: a block that runs once, after the last input line is read.
  • Pattern-action pair: a test followed by the action for matching lines.
  • Associative array: a store indexed by a word, such as a status code, not by a number.
  • Sorted keys: pipe the counts to LC_ALL=C sort, because awk alone does not order its keys.
  • NR in END: the total number of lines read.

In this example, the shell first prints the report heading. The next awk block counts hits by status code and pipes the counts to LC_ALL=C sort, so the codes print in order. A final awk block reads the file again and prints NR as the total.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

When does an END block run?