tethys_dask_scheduler

一个支持与Tethys平台集成的自定义SchedulerPlugins的Dask调度器。

安装

pip install tethys_dask_scheduler

或

conda install -c conda-forge tethys_dask_scheduler

启动调度器

启动一个向Tethys门户报告状态更新的调度器。

$ dask-scheduler --preload tethys_dask_scheduler.plugin --tethys-host https://:8000

distributed.preloading - INFO - Import preload module: tethys_dask_scheduler.plugin
distributed.scheduler - INFO - -----------------------------------------------
distributed.preloading - INFO - Import preload module: tethys_dask_scheduler.plugin
distributed.http.proxy - INFO - To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at: tcp://192.168.10.100:8786
distributed.scheduler - INFO -   dashboard at:                     :8787
distributed.scheduler - INFO - Tethys Host at:     https://:8000
distributed.preloading - INFO - Run preload setup click command: tethys_dask_scheduler.plugin
distributed.scheduler - INFO - Register worker <WorkerState 'tcp://192.168.10.100:34297', name: tcp://192.168.10.100:34297, status: running, memory: 0, processing: 0>
distributed.scheduler - INFO - Starting worker compute stream, tcp://192.168.10.100:34297
distributed.core - INFO - Starting established connection

启动工作进程

通常启动Dask分布式工作进程

$ dask-worker tcp://192.168.10.100:8786

distributed.nanny - INFO -         Start Nanny at: 'tcp://192.168.10.100:34359'
distributed.worker - INFO -       Start worker at: tcp://192.168.10.100:39569
distributed.worker - INFO -          Listening to: tcp://192.168.10.100:39569
distributed.worker - INFO -          dashboard at:       192.168.10.100:34857
distributed.worker - INFO - Waiting to connect to:  tcp://192.168.10.100:8786
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          6
distributed.worker - INFO -                Memory:                   7.77 GiB
distributed.worker - INFO -       Local Directory: /home/tethys/tethysdev/tethys_dask_scheduler/dask-worker-space/worker-4j_bj5hv
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -         Registered to:  tcp://192.168.10.100:8786
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection

测试

测试使用tox执行

pip install .[tests]
tox