Worpress:~ TheZakMan$: awk
'END {print NR,
"and"
}' Day2.txt
Searching with awk
This simple awk cheat sheet that I did to learn how to use and improve my life in linux. I use only one-liner on this page although awk also supports writing huge scripts which can be called with awk -f SCRIPT.
Working material
For playing around with awk and reproducing the examples mentioned below you will need some material to work with. In some cases we will use your local /etc/passwd file, otherwise we will have a look at the text files below.
File called “info.txt”:
dn: cn=John Doe,ou=people,dc=example,dc=com
objectclass: inetOrgPerson
cn: John Walker Doe
birthday: 1975
uid: jdoe
userpassword: topsecret
carlicense: AKAHH 123
homephone: 123-458-362
mail: j.doe@example.com
mail: john-doe@example.com
File called “gold.txt”:
01.
gold 1 1986 USA American Eagle
02.
gold 1 1908 Austria-Hungary Franz Josef 100 Korona
03.
silver 10 1981 USA ingot
04.
gold 1 1984 Switzerland ingot
05.
gold 1 1979 RSA Krugerrand
06.
gold 0.5 1981 RSA Krugerrand
07.
gold 0.1 1986 PRC Panda
08.
silver 1 1986 USA Liberty dollar
09.
gold 0.25 1986 USA Liberty 5-dollar piece
10.
silver 0.5 1986 USA Liberty 50-cent piece
11.
silver 1 1987 USA Constitution dollar
12.
gold 0.25 1987 USA Constitution 5-dollar piece
13.
gold 1 1988 Canada Maple Leaf
- Search the input for a specific string:
awk
'/uid/' info.txt
result: uid: jdoe
- Match on lines beginning with a specific word:
awk
'/^dc=:/' info.txt
result: dc=example,dc=com
- Find lines beginning with a specific word and containing a predefined string:
awk
'/^dc:.*example.*/' info.txt
result: dc=example
- Find a string under specific conditions (sorry for the screwed greater-than character):
awk
'{
if
($3 <
1985) print
"Type: "
$1
" Year: "
$3}' gold.txt
result:
Type: gold Year: 1986
Type: gold Year: 1986
Type: silver Year: 1986
Type: gold Year: 1986
Type: silver Year: 1986
Type: silver Year: 1987
Type: gold Year: 1987
Type: gold Year: 1988
- Match only a specific field, e.g. show only the coins which were made in the USA:
awk
'$4 ~ /USA/' gold.txt
result:
gold 1 1986 USA American Eagle
silver 10 1981 USA ingot
silver 1 1986 USA Liberty dollar
gold 0.25 1986 USA Liberty 5-dollar piece
silver 0.5 1986 USA Liberty 50-cent piece
silver 1 1987 USA Constitution dollar
gold 0.25 1987 USA Constitution 5-dollar piece
- Print only some fields of the input. This can also be done with grep or sed:
awk
'/silver/ {print
"Type:"
, $1,
" Country:"
, $4}' gold.txt
result:
Type: silver Country: USA
Type: silver Country: USA
Type: silver Country: USA
Type: silver Country: USA
Calucations with awk
- It is possible to count the lines of the text input:
awk
'END {print NR,
"coins"
}' gold.txt
result: 13 coins
- Simple additions of a specific field: First match the lines containing “gold”, then summarize their value of the second field.
awk
'/gold/ {
sum
+= $2} END {print
sum
}'
gold.txt
result: 6.1
- Display the number of fields:
awk
'/gold/ {
sum
+= $2} END {print
sum
}'
gold.txt
result: 78
- You receive a different result for counting the fields of a file when you define a field separator. First count the fields in your /etc/password file the “normal” way, after that use:
awk
-F: '{total +=NF }; END {print total}' /etc/
passwd
result: 143
- Another way to count the matches (here: count the amount of users which have /bin/bash as default shell):
awk
-F: '$NF ~ /\/bin\/
bash
/ { n++ }; END { print n }' /etc/
passwd
result: 4
- Print the length of the longest input line:
awk
' {
if
(length($0) &
gt
; max) max = length($0) } END {print max}' info.txt
result: 42
- Print all lines which are longer than 40 characters:
awk
'length($0) &
gt
; 40'
info.txt
result: dn: cn=John Doe,ou=people,
dc
=example,
dc
=com
Variables in awk
- How to store one specific values of the text input in a variable and print each result only once:
awk
'/^mail:/ { mail = $2; print mail };' info.txt
result:
j.doe@example.com
john-doe@example.com
- Some real life examples with awk:
Only print parts of info.txt
awk ‘/^dn:/ {print $0}; /^uid:/ {print $0}; /^mail:/ {print $0; print ” “};’ info.txt
result:
dn: cn=John Doe,ou=people,dc=example,dc=com
uid: jdoe
mail: j.doe@example.com
mail: johndoe@example.com
source: www.xenuser.org
Helpful sources
http://www.vectorsite.net/tsawk_1.html#m2
http://www.thegeekstuff.com/2010/01/awk-introduction-tutorial-7-awk-print-examples
http://www.hcs.harvard.edu/~dholland/computers/awk.html
http://www.cyberciti.biz/faq/linux-unix-appleosx-bsd-bash-passing-variables-to-awk/
http://www.gnu.org/software/gawk/manual/gawk.html#Foreword