Skip to main content
2025 Python Packaging Survey is now live!  Take the survey now

Run code on a dask worker via a context manager

Project description

Afar

Python Version Version License Code style

One man's magic is another man's engineering
Robert A. Heinlein


afar explores new syntax around context managers. For example:

import afar
with afar.run() as results, locally:
    x = 1
    y = x + 1

>>> results.x
1
>>> results.y
2

Soon, we will be able to run code on a remote dask worker with syntax like:

with afar.run() as result, remotely:
    import dask_cudf
    df = dask_cudf.read_parquet("s3://...")
    result = df.sum().compute()

For motivation, see https://github.com/dask/distributed/issues/4003

This code is highly experimental and magical!

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page