Skip to main content
2025 Python Packaging Survey is now live!  Take the survey now

Data processing in distributed environments

Project description

PyPI release Conda version BSD license Build status Supported Python versions Code style: black Language grade: Python Total alerts

Workflows enables light-weight services to process tasks in a message-oriented environment.

It is comprised of a communications layer (workflows.transport) that provides a common interface to queues and topics over different transport providers, a service abstraction layer (workflows.frontend) which encapsulates a service in a separate process, tools for status reporting, logging and controlling services via a transport mechanism, and a service class skeleton alongside example services (workflows.services.common_service et al.). To achieve more complex workflows services can be interconnected using recipes whereby the output of one service can be directed onwards to other services.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page