Counting IP Addresses in a Log File
I've been using
grep to search through files on linux / mac for years, but one flag I didn't use much until recently is the
-o flag. This tells grep to only output the matched pattern (instead of lines that mach the pattern).
This feature turns out to be pretty handy, lets say you want to find all the IP addresses in a file. You just need to come up with a regular expression to match an IP, I'll use this:
"[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" it's not perfect, but it will work.
grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" httpd.log
What if I want to see just unique IPs
We can use the
uniq command to remove duplicate ip addresses, but
uniq needs a sorted input. We can do that with the
sort command, like so:
grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" httpd.log | sort | uniq
Show me the number of times each IP shows up in the log
Now we can use the
-c flag for
uniq to display counts:
grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" httpd.log | sort | uniq -c
This will output something like:
7 10.0.0.30 1 10.0.0.80 3 10.0.0.70
The counts are not in order, so we can pass our results through sort again, this time with the
-n flag to use a numeric sort.
grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" httpd.log | sort | uniq -c | sort -n
The above will put them in order from least to greatest, you can pipe the result to
tail if you only want to see the top N IP addresses!
Pretty handy right?
Like this? Follow me ↯Tweet Follow @pfreitag
You might also like:
- Recursively Counting files by Extension on Mac or Linux - October 9, 2019
- The 15 Essential UNIX commands - July 29, 2005
- Passing Environment Variables to Sudo Command - September 23, 2019
- Difference between cd - vs cd ~- - September 19, 2019
- Linux and OS X Shell Commands - March 7, 2005
- Counting files - September 15, 2004
- Backtracking with bash - April 1, 2003