Sorting Grep Results When Log File Names Are Incremented Integers

While rotated log files can have a timestamp like YYYYMMDDHHmmss appended, a lot of log files are rotated with incremented integers (i.e. file.3 is removed, file.2 becomes file.3, file.1 becomes file.2, file becomes file.1, and file is a new file for current log ‘stuff’). We had some … challenges changing the log4j settings to use the timestamp format. When you grep¬†all of the log files, the results are organized by alpha-sorted file names. Problem is that alpha sorted file names are 1, 10, 11, 12, …, 19, 2, 20, 21 … which doesn’t produce results in ascending or descending date order. You get:

zwave.log.1:2018-07-01 22:51:21.450 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 58: Node not awake!
zwave.log.1:2018-07-01 22:51:21.450 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 55: Node not awake!
zwave.log.11:2018-07-02 00:47:05.375 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 70: Node not awake!
zwave.log.11:2018-07-02 00:47:05.376 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 46: Node not awake!
zwave.log.12:2018-07-02 01:01:12.850 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 82: Node not awake!
zwave.log.13:2018-07-02 01:11:22.465 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 48: Node not awake!
zwave.log.13:2018-07-02 01:11:22.478 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 82: Node not awake!
zwave.log.13:2018-07-02 01:11:22.478 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 49: Node not awake!
zwave.log.14:2018-07-02 01:25:22.632 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 112: Node not awake!
zwave.log.14:2018-07-02 01:25:22.632 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 212: Node not awake!
zwave.log.15:2018-07-02 01:40:29.053 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 58: Node not awake!
zwave.log.15:2018-07-02 01:40:29.053 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 70: Node not awake!
zwave.log.16:2018-07-02 01:55:01.702 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 50: Node not awake!
zwave.log.16:2018-07-02 01:55:01.702 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 122: Node not awake!
zwave.log.17:2018-07-02 02:08:48.818 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 67: Node not awake!
zwave.log.17:2018-07-02 02:08:48.818 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 212: Node not awake!
zwave.log.18:2018-07-02 02:23:53.281 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 55: Node not awake!
zwave.log.18:2018-07-02 02:23:53.281 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 46: Node not awake!
zwave.log.19:2018-07-02 02:44:23.567 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 48: Node not awake!
zwave.log.19:2018-07-02 02:44:23.567 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 86: Node not awake!
zwave.log.19:2018-07-02 02:44:23.567 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 50: Node not awake!
zwave.log.2:2018-07-01 23:03:36.704 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 57: Node not awake!
zwave.log.2:2018-07-01 23:03:36.704 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 64: Node not awake!
zwave.log.2:2018-07-01 23:03:36.704 [DEBUG] [ng.zwave.internal.protocol.ZWaveTransactionManager] – NODE 68: Node not awake!

So I put together a quick sequence of commands that will produce results sorted by date. This process strips off the file names (I could create a more complex command set to shuffle the file name off to the end, but I didn’t need the file name. I just needed to scroll through the times at which an event occurred).

grep “Node not awake” zwave.log* | sed -r ‘s/^zwave.log.*:2018/2018/’ | sort

The grep results are piped to sed to have the file name prefix removed. The subsequent results are then piped to sort to be, well, sorted.

Leave a Reply

Your email address will not be published. Required fields are marked *