mqdm: 多进程进度条

使用rich库在子进程中实现漂亮的进度条。

安装

pip install mqdm

工作进程进度

import mqdm
import time

def my_work(n, sleep, pbar: mqdm.RemoteBar):
    for i in pbar(range(n), description=f'counting to {n}'):
        time.sleep(sleep)

# executes my task in a concurrent futures process pool
mqdm.pool(
    my_work,
    range(1, 10),
    sleep=1,
    n_workers=3,
)

更低的层次

基本上，机制是这样的

# use context manager to start background listener and message queue
with mqdm.Bars() as pbars:
    # create progress bars and send them to the remote processes
    pool.submit(my_work, 1, pbar=pbars.add())
    pool.submit(my_work, 2, pbar=pbars.add())
    pool.submit(my_work, 3, pbar=pbars.add())

# your worker function can look like this
def my_work(n, sleep, pbar):
    for i in pbar(range(n), description=f'counting to {n}'):
        time.sleep(sleep)

# or this
def my_work(n, pbar: mqdm.RemoteBar, sleep=0.2):
    import time
    with pbar(description=f'counting to {n}', total=n):
        for i in range(n):
            pbar.update(0.5, description=f'Im counting - {n}  ')
            time.sleep(sleep/2)
            pbar.update(0.5, description=f'Im counting - {n+0.5}')
            time.sleep(sleep/2)

您可以使用它在这个池中

import mqdm
from concurrent.futures import ProcessPoolExecutor, as_completed

items = range(1, 10)

with ProcessPoolExecutor(max_workers=n_workers) as pool, mqdm.Bars() as pbars:
    futures = [
        pool.submit(my_work, i, pbar=pbars.add())
        for i in items
    ]
    for f in as_completed(futures):
        print(f.result())

它是通过在多进程队列中启动一个后台线程来工作的。Bars实例监听来自子进程中的进度条代理的消息。