跳转到主要内容

pytest插件,使测试复杂的长字符串输出变得容易编写和调试

项目描述

pytest-patterns 是一个pytest插件,提供针对测试优化的模式匹配引擎。

优点

  • 提供复杂模式在长字符串(1000+行)中的易读报告
  • 帮助推理哪些模式已匹配或未匹配 - 以及原因
  • 可以处理歧义、可选和重复匹配,以及非确定性并发过程的混合输出
  • 帮助编写易于阅读、维护和调整的模式的模式
  • 帮助使用pytest fixtures重用模式

长期目标

支持测试CLI输出,以及HTML和典型的文本/*类型,如JSON、YAML等。

示例

尝试使用源存储库中的示例进行尝试和实验。这些示例故意失败,因为失败报告是最重要和最有用的部分 - 除了使其更容易编写断言之外。

$ nix develop
$ hatch run test examples -vv

基本API

optional 匹配和省略号 ... 的变化

如果想要测试一个复杂的字符串,首先引入 patterns fixture,并创建一个接受包含单词“更好”的任意行数的命名模式

import this

zen = "".join([this.d.get(c, c) for c in this.s])


def test_zen(patterns):
    p = patterns.better_things
    p.optional("...better...")
    assert p == zen

这将不会给我们一个绿色的条形,因为模式确实匹配了一些行,但有一些行没有匹配,因此被视为 意外的

  🟢=EXPECTED | ⚪️=OPTIONAL | 🟡=UNEXPECTED | 🔴=REFUSED/UNMATCHED

  Here is the string that was tested:

  🟡                 | The Zen of Python, by Tim Peters
  🟡                 |
  ⚪️ better_things   | Beautiful is better than ugly.
  ⚪️ better_things   | Explicit is better than implicit.
  ⚪️ better_things   | Simple is better than complex.
  ⚪️ better_things   | Complex is better than complicated.
  ⚪️ better_things   | Flat is better than nested.
  ⚪️ better_things   | Sparse is better than dense.
  🟡                 | Readability counts.
  🟡                 | Special cases aren't special enough to break the rules.
  🟡                 | Although practicality beats purity.
  🟡                 | Errors should never pass silently.
  🟡                 | Unless explicitly silenced.
  🟡                 | In the face of ambiguity, refuse the temptation to guess.
  🟡                 | There should be one-- and preferably only one --obvious way to do it.
  🟡                 | Although that way may not be obvious at first unless you're Dutch.
  ⚪️ better_things   | Now is better than never.
  ⚪️ better_things   | Although never is often better than *right* now.
  🟡                 | If the implementation is hard to explain, it's a bad idea.
  🟡                 | If the implementation is easy to explain, it may be a good idea.
  🟡                 | Namespaces are one honking great idea -- let's do more of those!

报告突出显示了哪些行已匹配(以及导致匹配的模式)并且是好的,就像它们一样。可选匹配以白色圆圈显示(⚪️)。报告还突出显示了那些没有匹配并以黄色圆圈标记的行(🟡)。

我们已知禅宗方法是正确的,因此让我们使用更多API来继续完成模式。

continous 匹配

让我们使用需要行以特定顺序出现且不得被其他行中断的 continuous 匹配。我们创建一个新的命名模式并将其与之前的模式 merge

def test_zen(patterns):
    ...

    p = patterns.conclusio
    p.continous("""
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
""")
    full_pattern = patterns.full
    full_pattern.merge("better_things", "conclusio")

    assert full_pattern == zen

这让我们取得了一些进展

  🟢=EXPECTED | ⚪️=OPTIONAL | 🟡=UNEXPECTED | 🔴=REFUSED/UNMATCHED

  Here is the string that was tested:

  🟡                 | The Zen of Python, by Tim Peters
  🟡                 |
  ⚪️ better_things   | Beautiful is better than ugly.
  ⚪️ better_things   | Explicit is better than implicit.
  ⚪️ better_things   | Simple is better than complex.
  ⚪️ better_things   | Complex is better than complicated.
  ⚪️ better_things   | Flat is better than nested.
  ⚪️ better_things   | Sparse is better than dense.
  🟡                 | Readability counts.
  🟡                 | Special cases aren't special enough to break the rules.
  🟡                 | Although practicality beats purity.
  🟡                 | Errors should never pass silently.
  🟡                 | Unless explicitly silenced.
  🟡                 | In the face of ambiguity, refuse the temptation to guess.
  🟡                 | There should be one-- and preferably only one --obvious way to do it.
  🟡                 | Although that way may not be obvious at first unless you're Dutch.
  ⚪️ better_things   | Now is better than never.
  ⚪️ better_things   | Although never is often better than *right* now.
  🟢 conclusio       | If the implementation is hard to explain, it's a bad idea.
  🟢 conclusio       | If the implementation is easy to explain, it may be a good idea.
  🟢 conclusio       | Namespaces are one honking great idea -- let's do more of those!

注意,由 continuous 匹配的行以绿色突出显示,因为它们被认为是比 optional 的匹配更强的匹配。

in_order 匹配

仍然有一些东西缺失。让我们通过创建一个使用 in_order 的匹配来使测试结果为绿色,这期望行以给定的顺序出现,但可能与其他行混合。

def test_zen(patterns):
    ...
    p = patterns.top_and_middle
    p.in_order(
        """
The Zen of Python, by Tim Peters

Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
"""
    )
    full_pattern = patterns.full
    full_pattern.merge("top_and_middle", "better_things", "conclusio")

    assert full_pattern == zen

难道那应该给出一个绿色的条形吗?我仍然在那里看到一个黄色的圆圈!

  🟢=EXPECTED | ⚪️=OPTIONAL | 🟡=UNEXPECTED | 🔴=REFUSED/UNMATCHED

  Here is the string that was tested:

  🟢 top_and_middle  | The Zen of Python, by Tim Peters
  🟡                 |
  ⚪️ better_things   | Beautiful is better than ugly.
  ⚪️ better_things   | Explicit is better than implicit.
  ⚪️ better_things   | Simple is better than complex.
  ⚪️ better_things   | Complex is better than complicated.
  ⚪️ better_things   | Flat is better than nested.
  ⚪️ better_things   | Sparse is better than dense.
  🟢 top_and_middle  | Readability counts.
  🟢 top_and_middle  | Special cases aren't special enough to break the rules.
  🟢 top_and_middle  | Although practicality beats purity.
  🟢 top_and_middle  | Errors should never pass silently.
  🟢 top_and_middle  | Unless explicitly silenced.
  🟢 top_and_middle  | In the face of ambiguity, refuse the temptation to guess.
  🟢 top_and_middle  | There should be one-- and preferably only one --obvious way to do it.
  🟢 top_and_middle  | Although that way may not be obvious at first unless you're Dutch.
  ⚪️ better_things   | Now is better than never.
  ⚪️ better_things   | Although never is often better than *right* now.
  🟢 conclusio       | If the implementation is hard to explain, it's a bad idea.
  🟢 conclusio       | If the implementation is easy to explain, it may be a good idea.
  🟢 conclusio       | Namespaces are one honking great idea -- let's do more of those!

使用 <empty-line> 标记处理空行

之前的模式并不完全完美,因为 pytest-patterns 在模式内容和内容中都有处理换行符的特殊方式。

  1. 在测试的内容中,我们从不隐式接受任何未指定的行,包括空行。

  2. 然而,在模式中,空行并不重要,这允许你使用它们通过视觉分组来使你的模式更易于阅读。

我们通过在模式中使用特殊标记 <empty-line> 来解决这个问题,它将匹配包含 ``` 的行和空行

def test_zen(patterns):
    ...
    p = patterns.top_and_middle
    p.optional("<empty-line>")
    p.in_order(
        """
The Zen of Python, by Tim Peters

Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
"""
    )
    ...
    assert full_pattern == zen

我们最终得到了一个绿色的条形

examples/test_examples.py::test_zen_5_1 PASSED

refused 匹配

到目前为止,我们只创建了允许我们写下预期的内容的匹配。然而,我们也可以显式拒绝行——例如,任何包含“should”这个词的行

def test_zen(patterns):
    ...
    p = patterns.no_should
    p.refused("...should...")

    full_pattern = patterns.full
    full_pattern.merge(
        "no_should", "top_and_middle", "better_things", "conclusio"
    )

    assert full_pattern == zen

这是 pytest-patterns 真正大放异彩的地方。现在我们可以快速看到我们输出的哪些部分是正常的,哪些不是,以及为什么

  🟢=EXPECTED | ⚪️=OPTIONAL | 🟡=UNEXPECTED | 🔴=REFUSED/UNMATCHED

  Here is the string that was tested:

  🟢 top_and_middle  | The Zen of Python, by Tim Peters
  🟡                 |
  ⚪️ better_things   | Beautiful is better than ugly.
  ⚪️ better_things   | Explicit is better than implicit.
  ⚪️ better_things   | Simple is better than complex.
  ⚪️ better_things   | Complex is better than complicated.
  ⚪️ better_things   | Flat is better than nested.
  ⚪️ better_things   | Sparse is better than dense.
  🟢 top_and_middle  | Readability counts.
  🟢 top_and_middle  | Special cases aren't special enough to break the rules.
  🟢 top_and_middle  | Although practicality beats purity.
  🔴 no_should       | Errors should never pass silently.
  🟢 top_and_middle  | Unless explicitly silenced.
  🟢 top_and_middle  | In the face of ambiguity, refuse the temptation to guess.
  🔴 no_should       | There should be one-- and preferably only one --obvious way to do it.
  🟢 top_and_middle  | Although that way may not be obvious at first unless you're Dutch.
  ⚪️ better_things   | Now is better than never.
  ⚪️ better_things   | Although never is often better than *right* now.
  🟢 conclusio       | If the implementation is hard to explain, it's a bad idea.
  🟢 conclusio       | If the implementation is easy to explain, it may be a good idea.
  🟢 conclusio       | Namespaces are one honking great idea -- let's do more of those!`

  These are the matched refused lines:

  🔴 no_should       | ...should...
  🔴 no_should       | ...should...

使用 fixtures 重新使用模式

最后,我们希望在多个测试中重新使用模式。让我们将当前的模式重构为一个独立的 fixture,可以根据需要激活

import pytest
import this

zen = "".join([this.d.get(c, c) for c in this.s])


@pytest.fixture
def zen_patterns(patterns):
    p = patterns.better_things
    p.optional("...better...")

    p = patterns.conclusio
    p.continuous(
        """\
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
"""
    )

    p = patterns.top_and_middle
    p.in_order(
        """
The Zen of Python, by Tim Peters

Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
"""
    )
    full_pattern = patterns.full
    full_pattern.merge("top_and_middle", "better_things", "conclusio")


def test_zen_6(patterns, zen_patterns):
    full_pattern = patterns.full
    full_pattern.merge("better_things", "conclusio", "top_and_middle")

    assert full_pattern == zen

开发

$ pre-commit install
$ nix develop
$ hatch run test

TODO

  • 规范化功能

    -> json (+空白) -> python 对象引起序列化 -> json 对象引起反序列化和重新序列化(可读性顺序) -> 空白规范化

    -> html (+空白) -> 解析 html,然后以规范化的方式序列化 -> 规范化模式内容和测试内容中的空白

    -> 空白(模式和测试内容) -> 去除开头和结尾的空白 -> 将制表符替换为空格 -> 将多个空格折叠为单个空格

  • 适当的发布流程,包括标记、版本更新等。

  • 使覆盖率工作正常(https://cov.pytest.cn/en/latest/plugins.html 似乎不起作用 ...)

  • 将项目完全设置为对感兴趣的人和潜在贡献者有意义的。

  • 可选的不带颜色的报告

  • 针对多个 Python 版本的矩阵构建/在本地使用 tox 和在 GitHub Action 中?

  • 在报告未匹配的预期行时突出显示空白(例如)。这可能会在看到“空”行时造成混淆,因为你输入了错误,例如

    outmigrate.optional(
        """
simplevm             waiting                        interval=3 remaining=...
simplevm             check-staging-config           result='none'
simplevm             query-migrate                  arguments={} id=None subsystem='qemu/qmp'
simplevm             migration-status               mbps=... remaining='...' status='active'
simplevm             vm-destroy-kill-vm             attempt=... subsystem='qemu'
    """
    )

你看到它了吗?最后一行有四个空格,现在是一个预期行,有四个空格 ...

这也可以通过忽略仅包含空白的行(可选)来改进

DONE

  • 编码风格模板

  • 实际文档

  • 在状态报告中区分容忍和预期

  • 添加必须不出现的避免行

  • 允许模式预期/容忍/... 有名称,并使用这些名称标记报告为什么某些匹配?

    • (T) DEBUG | ....
    • (X) MIGRATION |
  • 如何处理HTML模板 -> 使用 optional("...")

  • 添加必须按顺序出现且不被中断的行

稍后

  • HTML规范化可能需要包含一个功能来抑制报告某些行(并在报告输出中仅添加 ...,例如如果某些内容失败则不报告owrap行

  • 添加行号

  • 在匹配避免处报告行号

  • structlog集成

  • 更全面的文档

措辞

与模式匹配的是形容词

  • 这些行必须出现,并且它们必须 连续
  • 这些行必须出现,并且它们必须 有序
  • 这些行是 可选的
  • 如果这些行出现,则它们是 拒绝的

对模式本身的修饰符是动词

  • 合并 这些模式中的规则到这个规则中。
  • 规范化 输入(以及规则)的方式。

项目详情


下载文件

下载您平台上的文件。如果您不确定选择哪个,请了解更多关于 安装包 的信息。

源代码分布

pytest_patterns-0.1.1.tar.gz (77.3 kB 查看哈希值)

上传时间 源代码

构建分布

pytest_patterns-0.1.1-py3-none-any.whl (8.3 kB 查看哈希值)

上传时间 Python 3

由以下机构支持

AWS AWS 云计算和安全赞助商 Datadog Datadog 监控 Fastly Fastly CDN Google Google 下载分析 Microsoft Microsoft PSF 赞助商 Pingdom Pingdom 监控 Sentry Sentry 错误记录 StatusPage StatusPage 状态页面