I extensively use awk (along with cut and grep) for data processing of log files or for any text processing needs. AWK is an excellent text processing tool / filter and report generator.

Why is AWK so important?
It is an excellent filter and report writer. Many UNIX utilities generates rows and columns of information. AWK is an excellent tool for processing these rows and columns, and is easier to use AWK than most conventional programming languages. It can be considered to be a pseudo-C interpretor, as it understands the same arithmatic operators as C. AWK also has string manipulation functions, so it can search for particular strings and modify the output. AWK also has associative arrays, which are incredible useful, and is a feature most computing languages lack. Associative arrays can make a complex problem a trivial exercise.


While looking for an obscure awk command I came across this nice awk resource. Highly recommended.

For example I just used awk to find out how much data I was serving to image leechers (websites which directly link to your images) from my sites. I used the simple script below to get the data (in bytes) from my log files.

egrep 'gif|jpg|png' /var/log/httpd/access_log*|grep -v taragana|cut -f10 -d' '|awk '{if($0 != "-") sum = sum + $0} END {print sum}'