pypi-to-sqlite

从PyPI将Python包的数据加载到SQLite中

安装

使用pip安装此工具

pip install pypi-to-sqlite

用法

要创建包含一个或多个包详情的SQLite数据库，运行

pypi-to-sqlite pypi.db datasette sqlite-utils

您还可以处理像这样先前保存到磁盘的JSON

curl -o datasette.json https://pypi.ac.cn/pypi/datasette/json
pypi-to-sqlite pypi.db -f datasette.json

此工具将创建三个表：packages、versions和releases。完整的表模式如下所示。

要使用前缀创建表，请使用--prefix prefix。例如

pypi-to-sqlite pypi.db datasette --prefix pypi_

这将创建名为pypi_packages、pypi_versions和pypi_releases的表。

演示

您可以在Datasette中看到使用此工具运行的表示例

• packages
• versions
• releases

数据库模式

CREATE TABLE [packages] (
   [name] TEXT PRIMARY KEY,
   [summary] TEXT,
   [classifiers] TEXT,
   [description] TEXT,
   [author] TEXT,
   [author_email] TEXT,
   [description_content_type] TEXT,
   [home_page] TEXT,
   [keywords] TEXT,
   [license] TEXT,
   [maintainer] TEXT,
   [maintainer_email] TEXT,
   [package_url] TEXT,
   [platform] TEXT,
   [project_url] TEXT,
   [project_urls] TEXT,
   [release_url] TEXT,
   [requires_dist] TEXT,
   [requires_python] TEXT,
   [version] TEXT,
   [yanked] INTEGER,
   [yanked_reason] TEXT
);
CREATE TABLE [versions] (
   [id] TEXT PRIMARY KEY,
   [package] TEXT REFERENCES [packages]([name]),
   [name] TEXT
);
CREATE TABLE [releases] (
   [md5_digest] TEXT PRIMARY KEY,
   [package] TEXT REFERENCES [packages]([name]),
   [version] TEXT REFERENCES [versions]([id]),
   [packagetype] TEXT,
   [filename] TEXT,
   [comment_text] TEXT,
   [digests] TEXT,
   [has_sig] INTEGER,
   [python_version] TEXT,
   [requires_python] TEXT,
   [size] INTEGER,
   [upload_time] TEXT,
   [upload_time_iso_8601] TEXT,
   [url] TEXT,
   [yanked] INTEGER,
   [yanked_reason] TEXT
);

pypi-to-sqlite --help

Usage: pypi-to-sqlite [OPTIONS] DB_PATH [PACKAGE]...

  Load data about Python packages from PyPI into SQLite

  Usage example:

      pypi-to-sqlite pypy.db datasette sqlite-utils

  Use -f to load data from a JSON file instead:

      pypi-to-sqlite pypy.db -f datasette.json

  Created tables will be packages, versions and releases

  To create tables called pypi_packages, pypi_versions, pypi_releases use
  --prefix pypi_:

      pypi-to-sqlite pypy.db datasette sqlite-utils --prefix pypi_

Options:
  --version            Show the version and exit.
  -f, --file FILENAME  Import JSON from this file
  -d, --delay FLOAT    Wait this many seconds between requests
  --prefix TEXT        Prefix to use for the created database tables
  --help               Show this message and exit.

开发

要为此工具做出贡献，首先检出代码。然后创建一个新的虚拟环境

cd pypi-to-sqlite
python -m venv venv
source venv/bin/activate

现在安装依赖项并测试依赖项

pip install -e '.[test]'

要运行测试

pytest