ChunkedFile

一个文件句柄，将子序列写入分组在一起，并定期将它们作为单独的文件写入。对于不支持文件追加的文件系统（如GCS FUSE）非常有用。可以将Pathlib修补以用分块文件替换以追加模式打开的文件，因此外部库无需更改。

用法

使用ChunkedFile

import time
import chunkedfile

# Write chunks up to every 10 minutes.
with chunkedfile.ChunkedFile('filename.txt', 600) as f:

  # The first chunk will be written on the first write after the sleep.
  f.write('Lorem\n')
  time.sleep(1000)
  f.write('Ipsum\n')

  # The last chunk is automatically written when the file is closed.
  f.write('Dolor\n')
  f.write('Sit\n')

# The directory now contains two files:
# - filename.txt-1-20210904T130400
# - filename.txt-2-20210904T132100

使用pathlib.Path

import time
import pathlib
import chunkedfile

# Path objects opened in append mode will be replaced by chunked files.
chunkedfile.patch_pathlib_append(600)

# Write chunks up to every 10 minutes.
with pathlib.Path('filename.txt').open('a') as f:

  # The first chunk will be written during the sleep command.
  f.write('Lorem\n')
  f.write('Ipsum\n')
  time.sleep(1000)

  # The last chunk is automatically written when the file is closed.
  f.write('Dolor\n')
  f.write('Sit\n')

# The directory now contains two files:
# - filename.txt-1-20210904T130400
# - filename.txt-2-20210904T132100