nyt-prb-scraper 0.0.10

用法

PRB 对关塔那摩囚犯的文件进行三种不同形式的审查：初步审查、文件审查和全面审查。技术上，还有第四种类型，即后续全面审查，但目前尚未发布后续全面审查。

initial_review --csv > initial_review.csv
initial_review --json > initial_review.json
initial_review --tsv > initial_review.tsv

file_review --csv > file_review.csv
file_review --json > file_review.json
file_review --tsv > file_review.tsv

full_review --csv > full_review.csv
full_review --json > full_review.json
full_review --tsv > full_review.tsv

架构

为每个文档返回一行或一个对象。每个文档包含特定于文档的字段，如 type_name、type_id 和 url，以及特定于囚犯的字段，如 name 和 isn。从 isn-type_id-hearing_or_review_date 为每个文档构建唯一的id。

[
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"1",
        "url":"http:\/\/www.prs.mil\/Portals\/60\/Documents\/ISN037\/141105_U_ISN037_GOVERNMENT'S_UNCLASSIFIED_SUMMARY_PUBLIC.pdf",
        "type_name":"Government's Unclassified Summary",
        "id":"037-initial-review-1-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    },
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"2",
        "url":"http:\/\/www.prs.mil\/Portals\/60\/Documents\/ISN037\/141105_U_ISN037_PR_STATEMENT_PRB.pdf",
        "type_name":"Opening Statements of Detainee's Representatives",
        "id":"037-initial-review-2-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    },
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"3",
        "url":"http:\/\/www.prs.mil\/Portals\/60\/Documents\/ISN037\/141216_U_ISN037_DETAINEE_WRITTEN_SUBMISSION_PUBLIC.pdf",
        "type_name":"Detainee's Written Submission",
        "id":"037-initial-review-3-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    },
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"4",
        "url":"http:\/\/www.prs.mil\/LinkClick.aspx?fileticket=RFOMdQD69k4%3d&tabid=8447&portalid=60&mid=20067",
        "type_name":"Transcript of Public Session",
        "id":"037-initial-review-4-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    },
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"5",
        "url":"http:\/\/www.prs.mil\/Portals\/60\/Documents\/ISN037\/141105_U_ISN037_TRANSCRIPT_OF_DETAINEE_SESSION_PUBLIC.pdf",
        "type_name":"Transcript of Detainee Session",
        "id":"037-initial-review-5-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    },
    {
        "review_type": "full-review",
        "review_url": "http://www.prs.mil/Review-Information/Initial-Review/",
        "hearing_or_review_date":"2014-11-05",
        "denial":null,
        "name":"Abdel Malik Ahmed Abdel Wahab Al Rahabi",
        "type_id":"6",
        "url":"http:\/\/www.prs.mil\/LinkClick.aspx?fileticket=s0XT-7qYc94%3d&tabid=8447&portalid=60&mid=20067",
        "type_name":"Unclassified Summary of Final Determination",
        "id":"037-initial-review-6-2014-11-05",
        "isn":"037",
        "denied":false,
        "notification_date":"2014-08-26",
        "final_determination_date":"2014-12-05"
    }
]

输出

爬虫可以返回CSV、JSON或TSV格式。如果没有传递任何选项，默认为CSV格式。