使用异步缓冲流式读取/写入Google Storage blob。
项目描述
gs-chunked-io: Google Storage流
gs-chunked-io为Google存储对象提供透明的分块io流。可写流作为多部分对象管理，在流关闭时组成。
IO操作默认为并发。可以通过使用threads参数调整并发线程数，或者通过threads=None完全禁用。
import gs_chunked_io as gscio
from google.cloud.storage import Client
client = Client()
bucket = client.bucket("my-bucket")
blob = bucket.get_blob("my-key")
# Readable stream:
with gscio.Reader(blob) as fh:
fh.read(size)
# Writable stream:
with gscio.Writer("my_new_key", bucket) as fh:
fh.write(data)
# Process blob in chunks:
for chunk in gscio.for_each_chunk(blob):
my_chunk_processor(chunk)
# Multipart copy with processing:
dst_bucket = client.bucket("my_dest_bucket")
with gscio.Writer("my_dest_key", dst_bucket) as writer:
for chunk in gscio.for_each_chunk(blob)
process_my_chunk(chunk)
writer(chunk)
# Extract .tar.gz on the fly:
import gzip
import tarfile
with gscio.Reader(blob) as fh:
gzip_reader = gzip.GzipFile(fileobj=fh)
tf = tarfile.TarFile(fileobj=gzip_reader)
for tarinfo in tf:
process_my_tarinfo(tarinfo)
安装
pip install gs-chunked-io
链接
错误
请将错误、问题、功能请求等报告到GitHub。
关闭
gs-chunked-io-0.5.2.tar.gz 的哈希值
| 算法 | 哈希摘要 | |
|---|---|---|
| SHA256 | 722e5ff718d9c91f4ac5e4d285ace599642e7c7c31188c025f3d07216c8e61e6 |
|
| MD5 | 910d44662bf4cbb5aae2c1329c92e1b2 |
|
| BLAKE2b-256 | aa765f86e2c4cc5d09c74e39f633a01bb75c3ad78d59e00b90105bb51e611288 |