跳转到主要内容

生物图像模型库的Python功能

项目描述

License PyPI conda-version downloads conda-forge downloads code style

core-bioimage-io-python

bioimage.io资源的Python专用核心实用工具(特别是模型)。

开始使用

要开始,我们建议安装bioimageio.core与conda一起安装深度学习框架,例如pytorch,并运行几个bioimageio命令来查看bioimage.core能提供什么功能

  1. 使用conda安装(有关conda环境的更多详细信息,请查看conda文档

    conda install -c conda-forge bioimageio.core pytorch
    
  2. 测试模型

    $ bioimageio test powerful-chipmunk
    ...
    
    (点击展开输出)
      ✔️                 bioimageio validation passed
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
      source            https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/powerful-chipmunk/1/files/rdf.yaml
      format version    model 0.4.10
      bioimageio.spec   0.5.3post4
      bioimageio.core   0.6.8
    
    
    
      ❓   location                                     detail
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
      ✔️                                                 initialized ModelDescr to describe model 0.4.10
    
      ✔️                                                 bioimageio.spec format validation model 0.4.10
      🔍   context.perform_io_checks                    True
      🔍   context.root                                 https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/powerful-chipmunk/1/files
      🔍   context.known_files.weights.pt               3bd9c518c8473f1e35abb7624f82f3aa92f1015e66fb1f6a9d08444e1f2f5698
      🔍   context.known_files.weights-torchscript.pt   4e568fd81c0ffa06ce13061327c3f673e1bac808891135badd3b0fcdacee086b
      🔍   context.warning_level                        error
    
      ✔️                                                 Reproduce test outputs from test inputs
    
      ✔️                                                 Reproduce test outputs from test inputs
    

    $ bioimageio test impartial-shrimp
    ...
    
    (点击展开输出)
      ✔️                 bioimageio validation passed
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
      source            https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/impartial-shrimp/1.1/files/rdf.yaml
      format version    model 0.5.3
      bioimageio.spec   0.5.3.2
      bioimageio.core   0.6.9
    
    
      ❓   location                    detail
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
      ✔️                                initialized ModelDescr to describe model 0.5.3
    
    
      ✔️                                bioimageio.spec format validation model 0.5.3
    
      🔍   context.perform_io_checks   False
      🔍   context.warning_level       error
    
      ✔️                                Reproduce test outputs from test inputs (pytorch_state_dict)
    
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 1 and size parameter n:
    
                                      0
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 2 and size parameter n:
    
                                      0
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 1 and size parameter n:
    
                                      1
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 2 and size parameter n:
    
                                      1
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 1 and size parameter n:
    
                                      2
    
      ✔️                                Run pytorch_state_dict inference for inputs with batch_size: 2 and size parameter n:
    
                                      2
    
      ✔️                                Reproduce test outputs from test inputs (torchscript)
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 1 and size parameter n: 0
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 2 and size parameter n: 0
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 1 and size parameter n: 1
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 2 and size parameter n: 1
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 1 and size parameter n: 2
    
    
      ✔️                                Run torchscript inference for inputs with batch_size: 2 and size parameter n: 2
    
  3. 在您的数据上运行预测

  • 显示bioimageio-predict命令帮助以获取概述

    $ bioimageio predict --help
    ...
    
    (点击展开输出)
    usage: bioimageio predict [-h] [--inputs Sequence[Union[str,Annotated[Tuple[str,...],MinLenmin_length=1]]]]
                              [--outputs {str,Tuple[str,...]}] [--overwrite bool] [--blockwise bool] [--stats Path]
                              [--preview bool]
                              [--weight_format {typing.Literal['keras_hdf5','onnx','pytorch_state_dict','tensorflow_js','tensorflow_saved_model_bundle','torchscript'],any}]
                              [--example bool]
                              SOURCE
    
    bioimageio-predict - Run inference on your data with a bioimage.io model.
    
    positional arguments:
      SOURCE                Url/path to a bioimageio.yaml/rdf.yaml file
                            or a bioimage.io resource identifier, e.g. 'affable-shark'
    
    optional arguments:
      -h, --help            show this help message and exit
      --inputs Sequence[Union[str,Annotated[Tuple[str,...],MinLen(min_length=1)]]]
                            Model input sample paths (for each input tensor)
    
                            The input paths are expected to have shape...
                            - (n_samples,) or (n_samples,1) for models expecting a single input tensor
                            - (n_samples,) containing the substring '{input_id}', or
                            - (n_samples, n_model_inputs) to provide each input tensor path explicitly.
    
                            All substrings that are replaced by metadata from the model description:
                            - '{model_id}'
                            - '{input_id}'
    
                            Example inputs to process sample 'a' and 'b'
                            for a model expecting a 'raw' and a 'mask' input tensor:
                            --inputs="[["a_raw.tif","a_mask.tif"],["b_raw.tif","b_mask.tif"]]"
                            (Note that JSON double quotes need to be escaped.)
    
                            Alternatively a `bioimageio-cli.yaml` (or `bioimageio-cli.json`) file
                            may provide the arguments, e.g.:
                            ```yaml
                            inputs:
                            - [a_raw.tif, a_mask.tif]
                            - [b_raw.tif, b_mask.tif]
                            ```
    
                            `.npy` and any file extension supported by imageio are supported.
                            Aavailable formats are listed at
                            https://imageio.readthedocs.io/en/stable/formats/index.html#all-formats.
                            Some formats have additional dependencies.
    
                              (default: ('{input_id}/001.tif',))
      --outputs {str,Tuple[str,...]}
                            Model output path pattern (per output tensor)
    
                            All substrings that are replaced:
                            - '{model_id}' (from model description)
                            - '{output_id}' (from model description)
                            - '{sample_id}' (extracted from input paths)
    
                              (default: outputs_{model_id}/{output_id}/{sample_id}.tif)
      --overwrite bool      allow overwriting existing output files (default: False)
      --blockwise bool      process inputs blockwise (default: False)
      --stats Path          path to dataset statistics
                            (will be written if it does not exist,
                            but the model requires statistical dataset measures)
                              (default: dataset_statistics.json)
      --preview bool        preview which files would be processed
                            and what outputs would be generated. (default: False)
      --weight_format {typing.Literal['keras_hdf5','onnx','pytorch_state_dict','tensorflow_js','tensorflow_saved_model_bundle','torchscript'],any}
                            The weight format to use. (default: any)
      --example bool        generate and run an example
    
                            1. downloads example model inputs
                            2. creates a `{model_id}_example` folder
                            3. writes input arguments to `{model_id}_example/bioimageio-cli.yaml`
                            4. executes a preview dry-run
                            5. executes prediction with example input
    
                              (default: False)
    
  • 创建一个示例并本地运行预测!

    $ bioimageio predict impartial-shrimp --example=True
    ...
    
    (点击展开输出)
    🛈 bioimageio prediction preview structure:
    {'{sample_id}': {'inputs': {'{input_id}': '<input path>'},
                    'outputs': {'{output_id}': '<output path>'}}}
    🔎 bioimageio prediction preview output:
    {'1': {'inputs': {'input0': 'impartial-shrimp_example/input0/001.tif'},
          'outputs': {'output0': 'impartial-shrimp_example/outputs/output0/1.tif'}}}
    predict with impartial-shrimp: 100%|███████████████████████████████████████████████████| 1/1 [00:21<00:00, 21.76s/sample]
    🎉 Sucessfully ran example prediction!
    To predict the example input using the CLI example config file impartial-shrimp_example\bioimageio-cli.yaml, execute `bioimageio predict` from impartial-shrimp_example:
    $ cd impartial-shrimp_example
    $ bioimageio predict "impartial-shrimp"
    
    Alternatively run the following command in the current workind directory, not the example folder:
    $ bioimageio predict --preview=False --overwrite=True --stats="impartial-shrimp_example/dataset_statistics.json" --inputs="[[\"impartial-shrimp_example/input0/001.tif\"]]" --outputs="impartial-shrimp_example/outputs/{output_id}/{sample_id}.tif" "impartial-shrimp"
    (note that a local 'bioimageio-cli.json' or 'bioimageio-cli.yaml' may interfere with this)
    

安装

通过Mamba/Conda

您可以从conda-forge通过以下方式安装bioimageio.core

mamba install -c conda-forge bioimageio.core

如果您没有安装任何额外的深度学习库,您将只能使用通用便利功能,但不能使用模型预测功能。要安装额外的深度学习库,请使用

  • Pytorch/Torchscript

    CPU安装(如果您没有NVIDIA显卡)

    mamba install -c pytorch -c conda-forge bioimageio.core pytorch torchvision cpuonly
    

    GPU安装(对于cuda 11.6,请选择适合您系统的适当cuda版本)

    mamba install -c pytorch -c nvidia -c conda-forge bioimageio.core pytorch torchvision pytorch-cuda=11.8
    

    请注意,Pytorch安装说明可能在未来发生变化。有关最新说明,请参阅pytorch.org

  • Tensorflow

    目前仅支持CPU版本

    mamba install -c conda-forge bioimageio.core tensorflow
    
  • ONNXRuntime

    目前仅支持CPU版本

    mamba install -c conda-forge bioimageio.core onnxruntime
    

通过pip

该包还通过pip提供(例如,使用推荐的额外onnxpytorch

pip install "bioimageio.core[onnx,pytorch]"

设置开发环境

要设置开发conda环境,请运行以下命令

mamba env create -f dev/env.yaml
mamba activate core
pip install -e . --no-deps

有不同环境文件可供选择,这些文件只安装TensorFlow或PyTorch作为依赖项。

💻 使用命令行界面

bioimageio.core 安装了一个用于测试模型和其他功能的命令行界面(CLI)。您可以通过以下方式列出所有可用的命令:

bioimageio

CLI从文件输入

为了方便起见,命令行选项(不是参数)可以放在一个 bioimageio-cli.jsonbioimageio-cli.yaml 文件中,例如:

# bioimageio-cli.yaml
inputs: inputs/*_{tensor_id}.h5
outputs: outputs_{model_id}/{sample_id}_{tensor_id}.h5
overwrite: true
blockwise: true
stats: inputs/dataset_statistics.json

🐍 在Python中使用

bioimageio.core 是一个Python包,它实现了bioimageio模型的预测,包括标准的前处理和后处理操作。这些模型由bioimageio.spec包描述,并可以用它加载。

此外,bioimageio.core提供了转换模型权重格式的功能。

要了解此功能概述,请查看以下示例笔记本

以及开发者文档

日志级别

bioimageio.specbioimageio.core 使用 loguru 进行日志记录,因此可以使用 LOGURU_LEVEL 环境变量控制日志级别。

模型规范

模型规范及其验证工具可在https://github.com/bioimage-io/spec-bioimage-io找到。

变更日志

0.6.9

  • 改进bioimageio命令行界面(详细信息请见#157)
    • 添加predict命令
    • 包命令输入path现在为必填项

0.6.8

  • 测试模型推理将现在检查所有权重格式(以前只检查模型适配器创建成功后的第一个格式)
  • 修复了使用分块预测的问题(感谢@thodkatz)

0.6.7

  • predict()参数inputs可以是样本

0.6.6

  • 添加别名以更接近以前的API

0.6.5

  • 改进适配器错误消息

0.6.4

  • 添加bioimageio validate-format命令
  • 改进错误消息和命令结果的显示

0.6.3

  • 修复#386
  • (在模型推理测试中)停止假设模型输入可分块

0.6.2

0.6.1

0.6.0

  • 添加与新的bioimageio.spec 0.5(0.5.2post1)的兼容性
  • 改进接口

0.5.10

项目详情


下载文件

下载适合您平台的应用程序。如果您不确定要选择哪一个,请了解有关安装包的更多信息。

源代码分发

bioimageio.core-0.6.9.tar.gz (79.0 kB 查看哈希值)

上传时间 源代码

构建版本

bioimageio.core-0.6.9-py3-none-any.whl (81.8 kB 查看哈希值)

上传时间 Python 3

由以下支持

AWSAWS云计算和安全赞助商DatadogDatadog监控FastlyFastlyCDNGoogleGoogle下载分析MicrosoftMicrosoftPSF赞助商PingdomPingdom监控SentrySentry错误日志StatusPageStatusPage状态页面