TinyLogAnalyzer 0.4.0

简介

该项目向您的系统添加了一个新的实用命令： tinylogan。此实用程序仅与启用响应时间数据的Apache-like访问HTTP日志一起使用。

有关如何做到这一点，请参阅 这篇博客文章，或者简要地更改日志格式的配置，从类似以下内容

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

改为以下内容

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T/%D" combined

日志记录将变为类似以下内容

[31/Jan/2008:14:19:07 +0000] "GET / HTTP/1.1" 200 7918 ""
... "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20061201 Firefox/2.0.0.11 (Ubuntu-feisty)" 0/95491

如何使用

这里完整的帮助信息

Usage: tinylogan [options] logfile

Simple bash utility for analyze HTTP access log with enabled response time

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         verbose output during log analysis
  -s SIZE, --size=SIZE  choose the number of record to store in every log
  -q, --keep-query      keep query strings in URLs instead of cutting them.
                        Using this an URL with different query string is treat
                        like different URLs.
  -i INCLUDE_REGEX, --include=INCLUDE_REGEX
                        a regexp expression that an URLs must match of will be
                        discarded. Can be called multiple times, expanding the
                        set
  -e EXCLUDE_REGEX, --exclude=EXCLUDE_REGEX
                        a regexp expression that an URLs must not match of
                        will be discarded. Can be called multiple times,
                        reducing the set
  --skip-day=SKIP_DAY   A regexp that repr specific whole day or a set of
                        dates that must be ignored. Can be called multiple
                        times
  --min-time=MIN_TIME_MILLIS
                        ignore all entries that require less than this amount
                        of millisecs
  --max-time=MAX_TIME_MILLIS
                        ignore all entries that require more than this amount
                        of millisecs
  --min-times=MIN_TIMES
                        set a minimum number of times that a entry must be
                        found to be used in the "Top average time" statistic

  Date filters:
    For those kind of filters you need to specify a date. You are free to
    use a specific date in the format dd/mmm/aaaa, like "24/May/2011", but
    also some keywords for relative date like "today", "yesterday",
    "tomorrow", "week" and "month". Use of "week" and "month" mean
    referring to first day of the current week or month. You can also
    provide a numerical modifier using "+" or "-" followed by a day
    quantity (example: "week-5" for going back of 5 days from the start of
    the week).

    --start-date=START_DATE
                        date where to start analyze and record
    --end-date=END_DATE
                        date where to end analyze and record

  Time filters:
    When a time is needed, you must enter it in the format hh:mm:ss or
    simply hh:mm, like "09:21:30" or "09:21". Those filter are used for
    skip record that are registered "too late at night" or "too early in
    the morning".

    --skip-timeperiod-start=SKIP_TIME_START
                        do not analyse records before the given time
    --skip-timeperiod-end=SKIP_TIME_END
                        do not analyse records later the given time

  Default configuration profiles:
    You can read a set of default configuration options from a
    ".tinylogan" file placed in the user's home directory. If this file is
    found, parameters from the "DEFAULT" section are read, but you can
    also add other sections. You can always override those options from
    the command line.

    -c PROFILE          read a different profile section than DEFAULT
    -U                  Ignore the user default profile file (if exists)
    --example-profile   Print out an example profile file, then exit. You can
                        put this output in a ".tinylogan" file in your home,
                        then customize it

您也可以在您的用户主目录中的 .tinylogan 配置文件中配置默认值。有关详细信息，请参阅上面的帮助信息。

结果

让我们解释一下给出的结果

Starting from 15/Apr/2011:08:19:06
enough... stopped by user action
Ending at 28/Apr/2011:17:00:36
Elapsed time: 0:00:04.955008
Timedelta is 13 days, 8:41:30 (but only 7 days, 9:41:30 are counted due to time bounds)

Top total time
  0001 - /url1 46591.603 (4924 times, average 9.462, 7.28% of the total)
  0002 - /url2 12660.053 (1212 times, average 10.446, 1.98% of the total)
  ...

Top average time
  0001 - /url3 32.828 (15 times, 492 total)
  0002 - /url4 30.549 (7 times, 213 total)
  ...

从 ... 开始

在日志中找到的第一个有效条目

足够... 被用户操作停止

只有在您在日志分析期间按下CTRL+C的情况下。这将停止日志扫描并立即跳转到结果

到 ... 结束

分析的最后一条条目

耗时： ...

日志分析所需的时间

时间差是 ...

日志第一条和最后一条记录的天数，这对于给用户提供从记录开始的总时间的百分比非常重要。

如果您使用上述某些时间过滤器，用于统计的值是句子中给出的值 但只计算xxx，因为受到时间限制。

        Total number of seconds taken
                     |                    Average time per call
Entry position       |                             |
      |              |                             |
     0001 - /url1 46591.603 (4924 times, average 9.462, 7.28% of the total)
              |               |                          |
       URL of the entry       |                          |
                              |             Percentage of the total time
                         Times called

     Average number of seconds taken
                    |
Entry position      |         Total time in seconds
      |             |                  |
     0001 - /url3 32.828 (15 times, 492 total)
              |              |
       URL of the entry      |
                             |
                        Times called

待办事项

• 忽略匹配的多个出现中的最小和最大值的一种方法

• 目前，所有记录都存储在内存中…很明显，这不是解析可能长达数GB的文件的方法

• 识别默认视图（如：foo/other_foo 与 foo/other_foo/index.html 相同）的一种方法

• 目前，日志是从第一行读取的。因此，当使用 --start-date 时，达到远离第一条的记录可能非常慢

变更日志