🎓 A1Learner
🔍
0 of 45 lessons completed

Bash Loop Optimisation

Bash loop optimisation is the practice of shaping loops so they do less repeated work. The shape of the loop decides how many processes start and whether results survive it.

While-read keeps lines whole, for-cat splits them

The for loop prints four separate words from two lines. The while loop prints the same two lines whole.

printf 'hello world\nsecond line\n' > lines.txt
for word in $(cat lines.txt); do echo "for: $word"; done
while IFS= read -r line; do echo "while: $line"; done < lines.txt

A builtin length costs no call per line

The first loop calls an external counter for each line. The second loop prints the same lengths with a builtin, so it starts nothing.

: > calls.txt
counted() {
  echo hit >> calls.txt
  command wc -c
}
printf 'aa\nbbbb\n' > words.txt
while IFS= read -r line; do printf '%s' "$line" | counted > /dev/null; done < words.txt
echo "external calls: $(wc -l < calls.txt)"
while IFS= read -r line; do echo "${#line}"; done < words.txt

A piped loop loses its count, a redirected loop keeps it

Both loops count the same three lines. Only the redirected loop still holds its count after the loop ends.

printf 'a\nb\nc\n' > nums.txt
n=0
cat nums.txt | while IFS= read -r line; do n=$((n + 1)); done
echo "pipeline loop count: $n"
n=0
while IFS= read -r line; do n=$((n + 1)); done < nums.txt
echo "redirect loop count: $n"

Example

printf '10\n20\n30\n' > nums.txt
: > calls.txt
awk() {
  echo hit >> calls.txt
  command awk "$@"
}
total=0
while IFS= read -r n; do total=$((total + n)); done < nums.txt
echo "total: $total"
echo "external calls: $(wc -l < calls.txt)"

Key points

  • While-read keeps lines whole: while IFS= read -r line reads one full line at a time.
  • For-cat splits lines: for word in $(cat file) splits on whitespace, so multi-word lines break apart.
  • Redirect, do not pipe: while read done < file keeps variables set inside the loop.
  • Pipeline loops lose results: a loop after a pipe runs in a subshell, so its variables vanish.
  • Builtins inside loops: ${#line} and $(( )) cost no new process per iteration.
  • One call beats many: read the whole file once instead of calling an external command per line.

In this example, the script writes three numbers to a file. A redirected while-read loop adds each number with builtin arithmetic. The total is 60, and the awk counter stays at 0 because no external command ran.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

Why does a count set inside cat file | while read vanish after the loop?