fitsiochunked 0.2.1

功能

• 顺序读取在给定的固定内存限制内的大型fits文件

快速使用

以下示例显示了在一个内存限制为2048MB的fits hdu中读取的示例，假设光曲线存储在行中

import numpy as np
import fitsio
import fitsiochunked as fc

with fitsio.FITS(filename) as infile:
    hdu = infile['flux']
    napertures = hdu.get_info()['ndim'][0]
    mean_flux = np.zeros(napertures)

    for chunk in fc.chunks(hdu, memory_limit_mb=2048):

        # `chunk` is a namedtuple with `.data` and `.slice` properties
        chunk_data = chunk.data
        print('Data shape:', chunk_data.shape)
        print('Data dtype:', chunk_data.dtype)


        chunk_slice = chunk.slice
        print('Chunk starting from aperture:', chunk_slice.start)
        print('Chunk up to:', chunk_slice.stop)

        chunk_mean = np.average(chunk_data, axis=1)
        mean_flux[chunk_slice] = chunk_mean

该库可以处理任意数量的hdus

import numpy as np
import fitsio
import fitsiochunked as fc

with fitsio.FITS(filename) as infile:
    hjd_hdu = infile['hjd']
    flux_hdu = infile['flux']
    fluxerr_hdu = infile['fluxerr']

    napertures = flux_hdu.get_info()['ndim'][0]
    mean_flux = np.zeros(napertures)

    for chunks in fc.chunks(hjd_hdu, flux_hdu, fluxerr_hdu, memory_limit_mb=2048):
        # chunks is a tuple of chunks
        hjd_chunk, flux_chunk, fluxerr_chunk = chunks

        # `chunk` is a namedtuple with `.data` and `.slice` properties
        flux_chunk_data = flux_chunk.data
        print('Data shape:', flux_chunk_data.shape)
        print('Data dtype:', flux_chunk_data.dtype)

        # and so on

注意：如果提供了多个hdus，则chunks函数的memory_limit_mb和chunksize参数应用于每个 HDU。即三个HDU和一个2048MB的内存限制将导致3x2048 = 6144MB的内存使用。

安装

使用pip安装

pip install fitsiochunked
# or get the latest development version from github
pip install git+https://github.com/mindriot101/fitsiochunked

或下载并运行设置文件

git clone https://github.com/mindriot101/fitsiochunked
cd fitsiochunked
python setup.py install

详细信息

高级接口是chunks函数，它创建一个包装fitsio.ImageHDU对象的ChunkedAdapter对象。

ChunkedAdapter包装fitsio HDU对象。当构造时，它成为一个可调用的对象，它以块的形式产生该HDU中的图像数据。

块大小可以通过参数 chunksize 设置，每次简单地返回 chunksize 行，或者通过 memory_limit_mb 设置，该设置尝试（没有保证！）自动计算可以放入 memory_limit_mb 兆字节内存中的光曲线数量。