Zcat and zgrep

UPDATE 2016-04-26: thanks for article here http://unix.stackexchange.com/questions/77296/is-there-a-tool-that-combines-zcat-and-cat-transparently Forget my lcat function, just use the already provided zgrep and zcat commands.

FORMAT:

zcat -f -- *           # reads all files (normal and gzipped)
zgrep -- PATTERN *     # looks for pattern in all files (normal and gzipped)
zgrep <dashed options> -- PATTERN *  # any grep options before the --

Examples (all of these work):

zcat -f -- messages
zcat -f -- messages.0.gz
zcat -f -- messages*
zcat -f -- *

zgrep -- PATTERN messages
zgrep -- PATTERN messages.0.gz
zgrep -- PATTERN messages*
zgrep -- PATTERN *

Note: dont forget the — with zcat,  zgrep seemed to have worked without the —

Note: put all of your grep options before the double dashes –. After the — should follow pattern and filenames. Double dash tells bash that the single and double dashes arguments have been completed (so that filenames or PATTERN space doesnt expand to a dash)

---

Old Lcat Article

lcat – reads through all gzipped and regular logs in chronological order. I provide them as a script and also as function.

Common files that this will be useful for: syslog, messages, samba logs, anything that gets rotated.

NOTE: code for lcat is half way down the page

We have all seen logs that look like this (especially in Ubuntu). There is probably an app that does what this script/article is about to show, but either way here is my implementation. (if you know of such a method, please leave it in the comments)

cd /var/log/
ls -1

syslog
syslog.1
syslog.2.gz
syslog.3.gz
syslog.4.gz
syslog.5.gz
syslog.6.gz
syslog.7.gz

Now lets say I want to look through the logs but I dont just wanna look thru

cat syslog

or

cat syslog.1

or better yet:

cat syslog.1 syslog

Note: how syslog.1 comes first as its chronologically first

Similarily you can read the gzipped ones with zcat (which on the fly uncompresses gzips and cats the contents – good for single files that got gzipped)

zcat syslog.2.gz

or in order

zcat syslog.7.gz syslog.6.gz syslog.5.gz syslog.4.gz syslog.3.gz syslog.2.gz

Or if you want to read all of them:

zcat syslog.7.gz syslog.6.gz syslog.5.gz syslog.4.gz syslog.3.gz syslog.2.gz; cat syslog.1 syslog

Now thats annoying, how do I just do a single command.

lcat script below

#!/bin/bash
# lcat - log read (reads old gzip logs followed by newer ascii logs) - save the file as lcat in one of your PATHS (echo $PATH)
# for help: lcat
[[ $# -eq 0 ]] && echo -e "lcat - log cat\nreads old gzip logs followed by newer ascii logs\nusage: lcat <prefix of file>\nexample: you have the following logs: syslog.3.gz syslog.2.gz syslog.1 and syslog\nto read them: lcat syslog" && exit 1
(logprefix=$1; zcat `ls -1Str $logprefix*gz`; cat `ls -1Str $logprefix* | grep -v "gz$"`)

The above will do good as script that you can run like this

cd syslog
lcat syslog

For reverse sort

lcat syslog | tac

Or if your just wanting a one liner without a script:

(logprefix=syslog; zcat `ls -1Str $logprefix*gz`; cat `ls -1Str $logprefix* | grep -v "gz$"`)

Anyhow this will be a golden script for looking thru logs that look like this:

infotinks@dbz:/var/log$ ls
alternatives.log auth.log.4.gz fsck speech-dispatcher
alternatives.log.1 boot.log gpu-manager.log syslog
alternatives.log.2.gz bootstrap.log hp syslog.1
apache2 btmp installer syslog.2.gz
apport.log btmp.1 kern.log syslog.3.gz
apport.log.1 cups kern.log.1 syslog.4.gz
apport.log.2.gz dist-upgrade kern.log.2.gz syslog.5.gz
apport.log.3.gz dmesg kern.log.3.gz syslog.6.gz
apport.log.4.gz dmesg.0 kern.log.4.gz syslog.7.gz
apport.log.5.gz dmesg.1.gz lastlog teamviewer
apport.log.6.gz dmesg.2.gz lightdm udev
apt dmesg.3.gz pm-powersave.log unattended-upgrades
aptitude dmesg.4.gz pm-powersave.log.1 upstart
aptitude.1.gz dpkg.log pm-powersave.log.2.gz vbox-install.log
auth.log dpkg.log.1 pm-suspend.log wtmp
auth.log.1 dpkg.log.2.gz pm-suspend.log.1 wtmp.1
auth.log.2.gz faillog pm-suspend.log.2.gz Xorg.0.log
auth.log.3.gz fontconfig.log samba Xorg.0.log.old

To read the various logs here:

# examples from above files:
lcat alternatives
lcat auth
lcat kern | zcat
lcat syslog | grep "anything that you want to find"
lcat you-get-it

lcat2 script below – lcat2 differs from lcat by adding two features; 1. it tells you what it will read and in what order at the top of the output; 2. also puts a header before reading each file (useful for reading logged dmesg files – such as the dmesg listed above)

#!/bin/bash
[[ $# -eq 0 ]] && echo -e "lcat - log cat\nreads old gzips followed by newer ascii logs\nUsage: lcat <prefix of file>\nExample: you have the following logs: syslog.3.gz syslog.2.gz syslog.1 and syslog\nTo read them: lcat syslog" && exit 1
# list what reading
(logprefix=$1; echo -n ZCAT: `ls -1Str $logprefix*gz`; echo " - CAT:" `ls -1Str $logprefix* | grep -v "gz$"`)
# read the stuff
(logprefix=$1;
for i in `ls -1Str $logprefix*gz`; do echo -e "################################\nLCAT ZCAT: ${i}\n################################"; zcat $i; done;
for i in `ls -1Str $logprefix* | grep -v "gz$"`; do echo -e "################################\nLCAT CAT: ${i}\n################################"; cat $i; done;)

UPDATE 2016-04-18: provide functions of lcat

Lcat function

If you dont want to write a script. You can use bash functions. Just copy paste this into your .bashrc or directly into your running shell and you will have the lcat function installed (which is used just like a command). if your sick of it just run unset lcat and it will no longer be a function. Anytime you enter a new bash  you will not have lcat (unless its in your bashprofile or bashrc file) you will have to redo it.

unset lcat # gets rid of any old lcat
lcat () {
[[ $# -eq 0 ]] && echo -e "lcat - log cat\nreads old gzip logs followed by newer ascii logs\nusage: lcat <prefix of file>\nexample: you have the following logs: syslog.3.gz syslog.2.gz syslog.1 and syslog\nto read them: lcat syslog" && exit 1
ls -1 | grep -q "^${1}" || { echo 'Files with the '"$1"' prefix do not exist. Example: To read messages, messages.0, and messages.1, and messages.2.gz. Just run '"'lcat messages'"; exit 1; }
(logprefix=$1; zcat `ls -1Str ${logprefix}*gz`; cat `ls -1Str ${logprefix}* | grep -v "gz$"`)
}

To use just run “cd /var/log/; lcat messages;”

Here is the other version of lcat which has headers, lcat2:

unset lcat2 # gets rid of any old lcat2
lcat2 () {
[[ $# -eq 0 ]] && echo -e "lcat - log cat\nreads old gzips followed by newer ascii logs\nUsage: lcat <prefix of file>\nExample: you have the following logs: syslog.3.gz syslog.2.gz syslog.1 and syslog\nTo read them: lcat syslog" && exit 1
ls -1 | grep -q "^${1}" || { echo 'Files with the '"$1"' prefix do not exist. Example: To read messages, messages.0, and messages.1, and messages.2.gz. Just run '"'lcat messages'"; exit 1; }
(logprefix=$1; echo -n "ZCAT: " `ls -1Str ${logprefix}*gz`; echo " - CAT: " `ls -1Str ${logprefix}* | grep -v "gz$"`
for i in `ls -1Str ${logprefix}*gz`; do echo -e "################################\nLCAT ZCAT: ${i}\n################################"; zcat $i; done;
for i in `ls -1Str ${logprefix}* | grep -v "gz$"`; do echo -e "################################\nLCAT CAT: ${i}\n################################"; cat $i; done;)
}

This is used the same way “cd /var/log/; lcat messages;”