I have a 70G+ log file, and i'd like the most recent entries (apache log append new items at the end) that match a pattern. i can either:
run grep | tail or
run tail | grep Option 1 will take forever. Option 2 may return nothing, then I will have to increase the count for tail and keep running until I get something.
If I could grep from the last line up to the first, it would be ideal. But I could not find any option on grep's man page.
Is there any trick to do that? either on grep alone or with any other combination of linux tools?
22 Answers
I think the command that will best help you is tac:
As it states:
tac - concatenate and print files in reverse
So you could pipe it to grep and match nnn number of lines before stopping, or something along those lines.
That's a big file. You should rotate those logs more often.
If tac is too slow, you could pick a programming language with a seek command (perl, for instance), then:
- open the file
- seek to the end
- iteratively:
- seek backwards some amount (4K, or larger)
- read that amount of text
- split on newlines, and search for whatever.
