elasticsearch-pii-redacter 1.11.0

客户端配置

该工具通过使用Python模块es_client连接。

您可以使用命令行选项或YAML配置文件来配置客户端连接。如果希望使用配置文件，则配置文件结构需要在根级别包含elasticsearch，如下所示

---
elasticsearch:
  client:
    hosts: https://10.11.12.13:9200
    cloud_id:
    request_timeout: 60
    verify_certs:
    ca_certs:
    client_cert:
    client_key:
  other_settings:
    username:
    password:
    api_key:
      id:
      api_key:
      token:

logging:
  loglevel: INFO
  logfile: /path/to/file.log
  logformat: default
  blacklist: []

REDACTIONS_FILE 配置

注意：如果在forcemerge下，only_expunge_deletes为True，则任何配置的max_num_segments值都将被忽略，并且仅清除标记为删除的文档。注意这个区别非常重要，因为默认行为是将每个分片合并为1个段。

---
redactions:
  - job_name_20240506_redact_hot:
      pattern: hot-*
      query: {'match': {'message': 'message1'}}
      fields: ['message']
      message: REDACTED
      expected_docs: 1
      restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
  - job_name_20240506_redact_cold:
      pattern: restored-cold-*
      query: {'match': {'nested.key': 'nested19'}}
      fields: ['nested.key']
      message: REDACTED
      expected_docs: 1
      restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
      forcemerge:
        max_num_segments: 1
  - job_name_20240506_redact_frozen:
      pattern: partial-frozen-*
      query: {'range': {'number': {'gte': 8, 'lte': 11}}}
      fields: ['deep.l1.l2.l3']
      message: REDACTED
      expected_docs: 4
      forcemerge:
        only_expunge_deletes: True