twarc-csv

此模块为twarc添加了推文的CSV导出功能。

请确保已安装和配置twarc

pip3 install --upgrade twarc
twarc2 configure

安装此插件

pip3 install --upgrade twarc-csv

twarc中将出现一个新的csv命令。如果您已在文件tweets.jsonl中收集了一些推文，您现在可以将它们转换为CSV格式

twarc2 search --limit 500 "blacklivesmatter" tweets.jsonl # collect some tweets
twarc2 csv tweets.jsonl tweets.csv # convert to CSV

额外的命令行选项

运行

twarc2 csv --help

以获取选项列表。

Usage: twarc2 csv [OPTIONS] [INFILE] [OUTFILE]

  Convert tweets to CSV.

Options:
  --input-data-type [tweets|users|counts|compliance|lists]
                                  Input data type - you can turn "tweets",
                                  "users", "counts" or "compliance" or "lists"
                                  data into CSV.
  --inline-referenced-tweets / --no-inline-referenced-tweets
                                  Output referenced tweets inline as separate
                                  rows. Default: no.
  --merge-retweets / --no-merge-retweets
                                  Merge original tweet metadata into retweets.
                                  The Retweet Text, metrics and entities are
                                  merged from the original tweet. Default:
                                  Yes.
  --process-entities / --no-process-entities
                                  Preprocess entities like URLs, mentions and
                                  hashtags, providing expanded urls and lists
                                  only instead of full json objects. Default:
                                  Yes.
  --json-encode-all / --no-json-encode-all
                                  JSON encode / escape all fields. Default: no
  --json-encode-text / --no-json-encode-text
                                  Apply JSON encode / escape to text fields.
                                  Default: no
  --json-encode-lists / --no-json-encode-lists
                                  JSON encode / escape lists. Default: yes
  --allow-duplicates              List every tweets as is, including
                                  duplicates. Default: No, only unique tweets
                                  per row. Retweets are not duplicates.
  --extra-input-columns TEXT      Manually specify extra input columns. Comma
                                  separated string. Only modify this if you
                                  have processed the json yourself. Default
                                  output is all available object columns, no
                                  extra input columns.
  --output-columns TEXT           Specify what columns to output in the CSV.
                                  Default is all input columns.
  --batch-size INTEGER            How many lines to process per chunk. Default
                                  is 100. Reduce this if output is slow.
  --hide-stats                    Hide stats about the dataset on completion.
                                  Always hidden if you're using stdin / stdout
                                  pipes.
  --hide-progress                 Hide the Progress bar. Always hidden if
                                  you're using stdin / stdout pipes.
  --help                          Show this message and exit.

CSV中的Twitter数据问题

CSV不是存储Twitter数据的最佳选择。始终保留原始API响应，并在json对象上执行特征提取。

此导出脚本旨在方便使用，对于将数据样本导入其他工具，有许多将推文CSV格式化的方法，这只是其中一种。

贡献

欢迎和建议，鼓励提交pull请求。即使您只是对此插件感兴趣，也请将您的用例发布在Issues中。