🏛️ socrata2sql

许多州和地方政府使用Socrata来运行他们的开放数据门户。此工具允许您从一个这样的门户中抓取数据集并将其复制到您选择的SQL数据库中。它使用Socrata API来理解数据集中的列，并尝试在SQL数据库中创建正确类型化的列以匹配，包括如果数据库和源数据集支持的话，PostGIS几何形状。

需求

• Python 3.x

安装

$ pipenv install socrata2sql

或

$ pip install socrata2sql

使用

Socrata to SQL database loader

Load a dataset from a Socrata-powered open data portal into a SQL database.
Uses the Socrata API to inspect the dataset, then sets up a table with matching
SQL types and loads all rows. The loader supports any database supported by
SQLalchemy.

Usage:
  socrata2sql insert <site> <dataset_id> [-d=<database_url>] [-a=<app_token>] [-t=<table_name>]
  socrata2sql ls <site> [-a=<app_token>]
  socrata2sql (-h | --help)
  socrata2sql (-v | --version)

Options:
  <site>             The domain for the open data site. Ex: www.dallasopendata.com
  <dataset_id>       The ID of the dataset on the open data site. This is usually
                     a few characters, separated by a hyphen, at the end of the
                     URL. Ex: 64pp-jeba
  -d=<database_url>  Database connection string for destination database as
                     dialect+driver://username:password@host:port/database.
                     Default: sqlite:///<dataset name>.sqlite
  -t=<table_name>    Destiation table in the database. Defaults to a sanitized
                     version of the dataset's name on Socrata.
  -a=<app_token>     App token for the site. Only necessary for high-volume
                     requests. Default: None
  -h --help          Show this screen.
  -v --version       Show version.

Examples:
  List all datasets on the Dallas open data portal:
  $ socrata2sql ls www.dallasopendata.com

  Load the Dallas check register into a local SQLite file (file name chosen
  from the dataset name):
  $ socrata2sql insert www.dallasopendata.com 64pp-jeba

  Load it into a PostgreSQL database called mydb:
  $ socrata2sql insert www.dallasopendata.com 64pp-jeba -d=postgresql:///mydb

本地开发

目前测试非常有限。可以使用以下命令运行：

$ python setup.py tests

版权

© 2019 The Dallas Morning News