Worpress:~ TheZakMan$: awk 'END {print NR, "and"}' Day2.txt
Searching with awk
This simple awk cheat sheet that I did to learn how to use and improve my life in linux. I use only one-liner on this page although awk also supports writing huge scripts which can be called with awk -f SCRIPT.
Working material
For playing around with awk and reproducing the examples mentioned below you will need some material to work with. In some cases we will use your local /etc/passwd file, otherwise we will have a look at the text files below.
File called “info.txt”:
dn: cn=John Doe,ou=people,dc=example,dc=comobjectclass: inetOrgPersoncn: John Walker Doebirthday: 1975uid: jdoeuserpassword: topsecretcarlicense: AKAHH 123homephone: 123-458-362mail: j.doe@example.commail: john-doe@example.com
File called “gold.txt”:
01.gold 1 1986 USA American Eagle
02.gold 1 1908 Austria-Hungary Franz Josef 100 Korona
03.silver 10 1981 USA ingot
04.gold 1 1984 Switzerland ingot
05.gold 1 1979 RSA Krugerrand
06.gold 0.5 1981 RSA Krugerrand
07.gold 0.1 1986 PRC Panda
08.silver 1 1986 USA Liberty dollar
09.gold 0.25 1986 USA Liberty 5-dollar piece
10.silver 0.5 1986 USA Liberty 50-cent piece
11.silver 1 1987 USA Constitution dollar
12.gold 0.25 1987 USA Constitution 5-dollar piece
13.gold 1 1988 Canada Maple Leaf
- Search the input for a specific string:
awk '/uid/' info.txt
result: uid: jdoe
- Match on lines beginning with a specific word:
awk '/^dc=:/' info.txt
result: dc=example,dc=com
- Find lines beginning with a specific word and containing a predefined string:
awk '/^dc:.*example.*/' info.txt
result: dc=example
- Find a string under specific conditions (sorry for the screwed greater-than character):
awk '{if ($3 < 1985) print "Type: " $1 " Year: " $3}' gold.txt
result:
Type: gold Year: 1986
Type: gold Year: 1986
Type: silver Year: 1986
Type: gold Year: 1986
Type: silver Year: 1986
Type: silver Year: 1987
Type: gold Year: 1987
Type: gold Year: 1988
- Match only a specific field, e.g. show only the coins which were made in the USA:
awk '$4 ~ /USA/' gold.txt
result:
gold 1 1986 USA American Eagle
silver 10 1981 USA ingot
silver 1 1986 USA Liberty dollar
gold 0.25 1986 USA Liberty 5-dollar piece
silver 0.5 1986 USA Liberty 50-cent piece
silver 1 1987 USA Constitution dollar
gold 0.25 1987 USA Constitution 5-dollar piece
- Print only some fields of the input. This can also be done with grep or sed:
awk '/silver/ {print "Type:", $1, " Country:" , $4}' gold.txt
result:
Type: silver Country: USA
Type: silver Country: USA
Type: silver Country: USA
Type: silver Country: USA
Calucations with awk
- It is possible to count the lines of the text input:
awk 'END {print NR, "coins"}' gold.txt
result: 13 coins
- Simple additions of a specific field: First match the lines containing “gold”, then summarize their value of the second field.
awk '/gold/ {sum += $2} END {print sum}' gold.txt
result: 6.1
- Display the number of fields:
awk '/gold/ {sum += $2} END {print sum}' gold.txt
result: 78
- You receive a different result for counting the fields of a file when you define a field separator. First count the fields in your /etc/password file the “normal” way, after that use:
awk -F: '{total +=NF }; END {print total}' /etc/passwd
result: 143
- Another way to count the matches (here: count the amount of users which have /bin/bash as default shell):
awk -F: '$NF ~ /\/bin\/bash/ { n++ }; END { print n }' /etc/passwd
result: 4
- Print the length of the longest input line:
awk ' { if (length($0) > max) max = length($0) } END {print max}' info.txt
result: 42
- Print all lines which are longer than 40 characters:
awk 'length($0) > 40' info.txt
result: dn: cn=John Doe,ou=people,dc=example,dc=com
Variables in awk
- How to store one specific values of the text input in a variable and print each result only once:
awk '/^mail:/ { mail = $2; print mail };' info.txt
result:
j.doe@example.com
john-doe@example.com
- Some real life examples with awk:
Only print parts of info.txt
awk ‘/^dn:/ {print $0}; /^uid:/ {print $0}; /^mail:/ {print $0; print ” “};’ info.txt
result:
dn: cn=John Doe,ou=people,dc=example,dc=com
uid: jdoe
mail: j.doe@example.com
mail: johndoe@example.com
source: www.xenuser.org
Helpful sources
http://www.vectorsite.net/tsawk_1.html#m2
http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples
http://www.hcs.harvard.edu/~dholland/computers/awk.html
http://www.cyberciti.biz/faq/linux-unix-appleosx-bsd-bash-passing-variables-to-awk/
http://www.gnu.org/software/gawk/manual/gawk.html#Foreword
TheZakMan-Box:~ 