Monday, August 29, 2011

Unix Grep Commands for Developers


I was trying to find an entry in a txt file, but I didn't knew the exact file name and where it was located. This is where find & grep came together and helped me -

 find / -name "*.txt" | xargs grep "sitemap"


To find only unique occurrences of a pattern in a file


grep word myfile.txt | sort | uniq


sort - sorts the matches in natural order, and when sorted order piped with uniq -> gives only the unique matches


grep '400' me.txt | sort | uniq | grep -o http://[^[:space:]]*

The above command find only unique urls with 400 errors
-o returns only matches, and not the whole line.


Many people use grep just for finding words in a file and miss the real potential of grep. It has saved a lot of time when I have to analysis over some log files in production.

Based on my experience, here I am listing what I use the most -


###########################################################
Match a pattern (regular expression). Lists Filename with matched line.

grep matchPattern myfile.txt

To match COMPLETE line



grep smug files{search files for lines with 'smug'}
grep '^smug' files{'smug' at the start of a line}
grep 'smug$' files{'smug' at the end of a line}
grep '^smug$' files{lines containing only 'smug'}
grep '\^s' files{lines starting with '^s', "\" escapes the ^}
grep '[Ss]mug' files{search for 'Smug' or 'smug'}
grep 'B[oO][bB]' files{search for BOB, Bob, BOb or BoB }
grep '^$' files{search for blank lines}
grep '[0-9][0-9]' file{search for pairs of numeric digits}



To match a WORD in a line




grep '^From: ' /usr/mail/$USER{list your mail}
grep '[a-zA-Z]'{any line with at least one letter}
grep '[^a-zA-Z0-9]{anything not a letter or number}
grep '[0-9]\{3\}-[0-9]\{4\}'{999-9999, like phone numbers. Like in java we use [a-z]{3,}}
grep '^.$'{lines with exactly one character}
grep '"smug"'{'smug' within double quotes}
grep '"*smug"*'{'smug', with or without quotes}
grep '^\.'{any line that starts with a Period "."}
grep '^\.[a-z][a-z]'{line start with "." and 2 lc letters}




###########################################################
Match text. Lists Filename with matched line.

grep "this world is" myfile.txt


###########################################################
Match exact word

grep -w word myfile


###########################################################



Count
 of a particular word in file

grep -c "word" myfile


###########################################################




Case Insensitive

grep -i "Word" myfile


###########################################################

See line numbers of matching lines

grep -n word myfile


###########################################################



Show matches with as colored

grep word myfile --color


###########################################################
Finding relevant word and exclusion irrelevant word.

grep Exception logfile.txt | grep -v ERROR


###########################################################
Print also lines around matched word's line. E.g. 2 lines -

grep -C 2 "word" myfile


###########################################################
Match either this OR that. Using egrep -

egrep "Exception|Error" myfile


###########################################################

Match through gzipped *.gz files. Using zgrep -

zgrep -i "Word" myfile.gz


###########################################################
Only give the filenames of matched files

grep -l word myfile


###########################################################

1 comment:

  1. Excellent tip man indeed pretty useful I have also blogged my unix experience as grep command in unix with examples .

    Thanks
    Javin

    ReplyDelete