logmerger

logmerger 是一个TUI，用于查看多个日志文件的合并显示，按时间戳合并。

给定这两个日志文件

# log1.txt
2023-07-14 08:00:01 WARN   Connection lost due to timeout
2023-07-14 08:00:04 ERROR  Request processed unsuccessfully
Something went wrong
Traceback (last line is latest):
    blah
    blah
ValueError("shouldn't have done that")
2023-07-14 08:00:06 INFO   User authentication succeeded
2023-07-14 08:00:08 DEBUG  Starting data synchronization
2023-07-14 08:00:11 INFO   Processing incoming request

# log2.txt
2023-07-14 08:00:01 INFO   Request processed successfully
2023-07-14 08:00:03 INFO   User authentication succeeded
2023-07-14 08:00:06 DEBUG  Starting data synchronization
2023-07-14 08:00:08 INFO   Processing incoming request
2023-07-14 08:00:11 DEBUG  Performing database backup
2023-07-14 08:00:14 WARN   Invalid input received: missing required field

此命令

logmerger log1.txt log2.txt

显示以下可浏览的合并显示（使用textual Python库启用）

使用 --output - 将合并的日志发送到标准输出

  Timestamp                 Files/Log1.Txt                        Files/Log2.Txt
 ────────────────────────────────────────────────────────────────────────────────────────────────────
  2023-07-14 08:00:01.000   WARN   Connection lost due to         INFO   Request processed
                            timeout                               successfully
  2023-07-14 08:00:03.000                                         INFO   User authentication
                                                                  succeeded
  2023-07-14 08:00:04.000   ERROR  Request processed
                            unsuccessfully
                             Something went wrong
                             Traceback (last line is latest):
                                 blah
                                 blah
                             ValueError("shouldn't have done
                            that")
  2023-07-14 08:00:06.000   INFO   User authentication            DEBUG  Starting data
                            succeeded                             synchronization
  2023-07-14 08:00:08.000   DEBUG  Starting data                  INFO   Processing incoming request
                            synchronization
  2023-07-14 08:00:11.000   INFO   Processing incoming request    DEBUG  Performing database backup
                            INFO   Processing incoming request
                            (a little more...)
  2023-07-14 08:00:14.000   DEBUG  Performing database backup     WARN   Invalid input received:
                                                                  missing required field

安装

从PyPI安装 logmerger（目前仍使用旧的项目名称 log_merger）

pip install log_merger

这将安装 logmerger 作为shell控制台命令，因此您可以直接运行它而无需调用 python。

命令行参数

logmerger -h 将显示以下帮助

usage: logmerger [-h] [--interactive] [--output OUTPUT] [--start START] [--end END] [--width WIDTH]
                 [--line_numbers] [--csv CSV] [--encoding ENCODING]
                 [--timestamp_format [TIMESTAMP_FORMATS ...]] [--demo]
                 [files ...]

positional arguments:
  files                 log files to be merged

options:
  -h, --help            show this help message and exit
  --interactive, -i     show merged output using interactive TUI browser (default)
  --output OUTPUT, -o OUTPUT
                        save merged output to file ('-' for stdout; files ending in '.md' are saved
                        using Markdown)
  --start START, -s START
                        start time to select time window for merging logs
  --end END, -e END     end time to select time window for merging logs
  --width WIDTH, -w WIDTH
                        total screen width to use for interactive mode (defaults to current screen
                        width)
  --line_numbers, -ln   add line number column
  --csv CSV, -csv CSV   save merged logs to CSV file
  --encoding ENCODING, -enc ENCODING
                        encoding to use when reading log files (defaults to the system default encoding)
  --timestamp_format [TIMESTAMP_FORMATS ...]
                        custom timestamp format
  --demo                Run interactive demo
  
Start and end timestamps to clip the given files to a particular time window can be
given in `YYYY-MM-DD HH:MM:SS.SSS` format, with trailing milliseconds and seconds
optional, and "," permissible for the decimal point. A "T" can be included between
the date and time to simplify entering the timestamp on a command line (otherwise
would require enclosing in quotes because of the intervening space). These command
line values do not need to match the timestamp formats in the log files.

These values may also be given as relative times, such as "15m" for "15 minutes ago".
Valid units are "s", "m", "h", and "d".

支持文件类型

日志数据通常从文本日志文件中提取，但也可以从其他相关日志文件中提取。

合并

日志文件通过交错每个日志行中的日志行进行合并，基于每行日志中的时间戳。 logmerger 尝试使用不同的时间戳格式，直到为每个输入文件找到匹配的格式。支持的格式有

未标记时间戳的日志行（如堆栈跟踪）将与前一条标记时间戳的行合并（参见上面的示例）。