np-validator

快速概述

API

自动生成的步骤

默认情况下，run_validation可以在文件路径列表上运行。如果提供有效的项目名称，将自动生成一个ValidationStep列表。如果没有提供项目名称，则项目名称为“default”。

from np_validator import run_validation

filepaths = [
  "some/sort/of/path/prefix/uuid-maybe.mapping.pkl",
  "some/sort/of/path/prefix/uuid-maybe.behavior.pkl",
  "this/will/be/ignored/uuid-maybe.replay.pkl",
]

results = run_validation(filepaths, project="default")

手动生成验证步骤

from np_validator import Processor, Validator, ValidationStep, run_validation

# make a basic filesize validator
fs_validator = Validator(
  name="meets_filesize_threshold",
  args={
    "threshold": 10,
  },
)

# add validator to a validation step
validation_step_0 = ValidationStep(
  path_suffix=".mapping.pkl",
  validators=[fs_validator, ],
)

# make a validation step with a processor
# processors convert the data source from it's basic state, a filepath, to an easier to use object

# unpickle, unpickles an arbitrary filepath into a python object
unpickler = Processor(
  name="unpickle",
)

# has_dict_key, checks if a dict-like interface has a key at path
session_uuid_validator = Validator(
  name="has_dict_key",
  args={
    "path": ["session_uuid", ],
  }
)

# assembling it all together as a validation step

validation_step_1 = ValidationStep(
  path_suffix=".behavior.pkl",
  processor=unpickler,
  validators=[session_uuid_validator, ],
)

# running a validation
filepaths = [
  "some/sort/of/path/prefix/uuid-maybe.mapping.pkl",
  "some/sort/of/path/prefix/uuid-maybe.behavior.pkl",
  "this/will/be/ignored/uuid-maybe.replay.pkl",
]
results = run_validation(
  filepaths,
  [validation_step_0, validation_step_1, ],
)

更新自动生成的步骤

from np_validator import Processor, Validator, ValidationStep, autogenerate_validation_steps, update_project_validation_steps

# autogenerated steps are organized by projects
validation_step_0 = ValidationStep(
  path_suffix=".mapping.pkl",
  validators=[
    Validator(
      name="meets_filesize_threshold",
      args={
        "threshold": 10,
      },
    ),
  ],
)

validation_step_1 = ValidationStep(
  path_suffix=".behavior.pkl",
  processor=Processor(
    name="unpickle",
  ),
  validators=[
    Validator(
      name="has_dict_key",
      args={
        "path": ["session_uuid", ],
      }
    ),
  ],
)

# updating project validation steps overwrites the previous steps

update_project_validation_steps(
  "pretest",  # project name
  [validation_step_0, validation_step_1],
)

# to append, get the current steps with autogenerate_validation_steps and append to it
current = autogenerate_validation_steps("default")
updated = current + [validation_step_0, validation_step_1]
update_project_validation_steps(
  "default",  # project name
  updated, # append new steps
)

文档

有关使用此包的更多详细文档，请参阅文档。

贡献

处理器

要添加新的处理器，请将函数添加到np_validator/processors.py中。理想情况下，在tests中为其创建一个测试。运行tests*以帮助确保没有引入回归。每个处理器都应有一个字符串文件路径作为必需参数。

验证器

要添加新的验证器，请将函数添加到np_validator/validators.py中。理想情况下，在tests中为其创建一个测试。运行tests*以帮助确保没有引入回归。

*在pull_request上自动运行测试。

安装

pip install -U np-validator

或使用Poetry安装

poetry add np-validator

Makefile使用

Makefile包含许多用于快速开发的函数。

make poetry-download

make poetry-remove

make install

make pre-commit-install

make codestyle

# or use synonym
make formatting

make check-codestyle

make update-dev-deps

make check-safety

make check-safety

make mypy

make test

make lint

make test && make check-codestyle && make mypy && make check-safety

make docker-build

make docker-build VERSION=latest

make docker-remove

make pycache-remove

make build-remove

make dsstore-remove

make mypycache-remove

make cleanup

📈 发布

您可以在 GitHub 发布页面 上查看可用的发布列表。

我们遵循 语义版本 规范。

我们使用 Release Drafter。随着拉取请求的合并，草稿发布会保持最新，列出更改，准备好发布。使用类别选项，您可以使用标签对发布说明中的拉取请求进行分类。

标签列表及其对应标题

您可以在 release-drafter.yml 中更新它。

GitHub 为您创建了 bug、enhancement 和 documentation 标签。Dependabot 创建了 dependencies 标签。当您需要时，在 GitHub 仓库的“问题”选项卡上创建剩余的标签。

🛡 许可证

此项目根据 MIT 许可证的条款进行许可。有关更多详细信息，请参阅 LICENSE。

附加信息

cd np-validator && git init

make poetry-download

make install
make pre-commit-install

make codestyle

git add .
git commit -m ":tada: Initial commit"
git branch -M main
git remote add origin https://github.com/np_validator/np-validator.git
git push -u origin main