conda-mirror-ng

将上游conda通道镜像到本地目录。

安装

conda-mirror-ng 可在PyPI和conda-forge上找到。

使用以下方式安装

pip安装conda-mirror-ng

或

conda install conda-mirror-ng -c conda-forge

兼容性

conda-mirror-ng 故意仅是一个 Py3.6+ 包

CLI

conda-mirror-ng 的CLI接口

usage: conda-mirror-ng [-h] [--upstream-channel UPSTREAM_CHANNEL]
                       [--target-directory TARGET_DIRECTORY]
                       [--temp-directory TEMP_DIRECTORY] [--platform PLATFORM]
                       [-v] [--config CONFIG] [--pdb] [--num-threads NUM_THREADS]
                       [--version] [--dry-run] [--no-validate-target]
                       [--minimum-free-space MINIMUM_FREE_SPACE]

CLI interface for conda-mirror-ng

optional arguments:
  -h, --help            show this help message and exit
  --upstream-channel UPSTREAM_CHANNEL
                        The target channel to mirror. Can be a channel on
                        anaconda.org like "conda-forge" or a full qualified
                        channel like "https://repo.continuum.io/pkgs/free/"
  --target-directory TARGET_DIRECTORY
                        The place where packages should be mirrored to
  --temp-directory TEMP_DIRECTORY
                        Temporary download location for the packages. Defaults
                        to a randomly selected temporary directory. Note that
                        you might need to specify a different location if your
                        default temp directory has less available space than
                        your mirroring target
  --platform PLATFORM   The OS platform(s) to mirror. one of: {'linux-64',
                        'linux-32','osx-64', 'win-32', 'win-64'}
  -v, --verbose         logging defaults to error/exception only. Takes up to
                        three '-v' flags. '-v': warning. '-vv': info. '-vvv':
                        debug.
  --config CONFIG       Path to the yaml config file
  --pdb                 Enable PDB debugging on exception
  --num-threads NUM_THREADS
                        Num of threads for validation. 1: Serial mode. 0: All
                        available.
  --version             Print version and quit
  --dry-run             Show what will be downloaded and what will be removed.
                        Will not validate existing packages
  --no-validate-target  Skip validation of files already present in target-
                        directory
  --minimum-free-space MINIMUM_FREE_SPACE
                        Threshold for free diskspace. Given in megabytes.

示例用法

警告：执行此命令将下载 ~10GB 并至少需要一个小时

conda-mirror-ng --upstream-channel conda-forge --target-directory local_mirror --platform linux-64

更多信息

黑名单/白名单配置

example-conf.yaml

blacklist:
    - license: "*agpl*"
    - license: None
    - license: ""

whitelist:
    - name: system

blacklist 从上游repodata中删除匹配条件中的软件包。

whitelist 重新包含黑名单中匹配白名单条件的任何软件包。

黑名单和白名单都采用字典列表。字典中的键必须是 repodata.json 元数据中的值。值是匹配的（Unix）glob。有关上游“defaults”通道的完整repodata，请访问： http://conda.anaconda.org/anaconda/linux-64/repodata.json

以下是repodata['packages']中的条目内容之一

{'botocore-1.4.10-py34_0.tar.bz2': {'arch': 'x86_64',
  'binstar': {'channel': 'main',
   'owner_id': '55fc8527d3234d09d4951c71',
   'package_id': '56b88ea1be1cc95a362b218e'},
  'build': 'py34_0',
  'build_number': 0,
  'date': '2016-04-11',
  'depends': ['docutils >=0.10',
   'jmespath >=0.7.1,<1.0.0',
   'python 3.4*',
   'python-dateutil >=2.1,<3.0.0'],
  'license': 'Apache',
  'md5': 'b35a5c1240ba672e0d9d1296141e383c',
  'name': 'botocore',
  'platform': 'linux',
  'requires': [],
  'size': 1831799,
  'version': '1.4.10'}}

有关更多信息，请参阅 conda_mirror:match 函数的实现细节。

常见用法模式

仅镜像特定软件包

如果您想要与上述配置精确匹配botocore软件包，则可以使用以下配置首先黑名单 所有 软件包，然后仅包括botocore软件包

blacklist:
    - name: "*"
whitelist:
    - name: botocore
      version: 1.4.10
      build: py34_0

除了agpl许可证外，镜像所有内容

blacklist:
    - license: "*agpl*"

仅镜像python 3软件包

blacklist:
    - name: "*"
whitelist:
    - build: "*py3*"

测试

安装测试需求

注意：将安装来自pip的软件包

$ pip install -r test-requirements.txt
Requirement already satisfied: pytest in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 1))
Requirement already satisfied: coverage in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 2))
Requirement already satisfied: pytest-ordering in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 3))
Requirement already satisfied: py>=1.4.29 in /home/edill/miniconda/lib/python3.5/site-packages (from pytest->-r test-requirements.txt (line 1))

运行测试，使用 coverage 工具调用。

$ coverage run run_tests.py
sys.argv=['run_tests.py']
========================================= test session starts ==========================================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.31, pluggy-0.4.0 -- /home/edill/miniconda/bin/python
cachedir: .cache
rootdir: /home/edill/dev/maxpoint/github/conda-mirror, inifile:
plugins: xonsh-0.5.2, ordering-0.4
collected 4 items

test/test_conda_mirror.py::test_match PASSED
test/test_conda_mirror.py::test_cli[https://repo.continuum.io/pkgs/free-linux-64] PASSED
test/test_conda_mirror.py::test_cli[conda-forge-linux-64] PASSED
test/test_conda_mirror.py::test_handling_bad_package PASSED

======================================= 4 passed in 4.41 seconds =======================================

发布

要发布，您需要以下三样东西

1. 对conda-mirror-ng的提交权限
2. 对PyPI上的 conda-mirror-ng 的访问权限
3. 对conda-forge上的 conda-mirror-ng-feedstock 的访问权限

在您拥有这三样东西之后，添加git标签，使用 twine 上传sdist，并在conda-forge上合并自动生成的PR。