ArchNGV

用于在硅中构建神经元-胶质细胞-血管结构架构的模块。

文档

• 最新版

• 文件格式规范

安装

从PyPI安装

pip install archngv[all]

从源安装

# Clone this repository
git clone https://github.com/BlueBrain/ArchNGV.git

# Create a Python virtualenv in repository source directory
python -m venv /path/to/repo/.venv

# Bring the virtualenv in this shell environment
. /path/to/repo/.venv/bin/activate

# Install ArchNGV
cd ArchNGV/
pip install .[all]

示例

创建电路示例

# Create a directory for your circuit
circuit_dir=./circuits
mkdir -p $circuit_dir

# Create an exemplar
python ./exemplar/create_exemplar.py $circuit_dir

执行细胞定位

继续到创建的示例之一中的细胞定位

# Change directory to one of the created exemplar
cd $circuit_dir/exemplar_ID

# Execute the "cell-placement" snakemake target
./run.sh cell-placement
# -> creates file build/cell_data.h5

使用cell_data_sonata SnakeMake任务在细胞定位后执行输出转换为Sonata格式

./run.sh cell_data_sonata

# sonata file glia.h5.somata is created in the sonata.tmp directory
find build/sonata.tmp
# build/sonata.tmp
# build/sonata.tmp/nodes
# build/sonata.tmp/nodes/glia.h5.somata

星形胶质细胞合成

它使用Dask进行并行计算。

本地机器使用示例。请注意，未使用--parallel选项

ngv synthesis \
    --config /path/to/synthesis_config.yaml' \
    --tns-distributions /path/to/tns_distributions.json \
    --tns-parameters /path/to/tns_parameters.json \
    --tns-context /path/to/tns_context.json \
    --astrocytes /path/to/glia.h5 \
    --microdomains /path/to/microdomains.h5 \
    --gliovascular-connectivity /path/to/gliovascular.h5 \
    --neuroglial-connectivity /path/to/neuroglial.h5 \
    --endfeet-areas /path/to/endfeet_areas.h5 \
    --neuronal-connectivity /path/to/edges.h5 \
    --out-morph-dir /path/to/out_morphologies

BB5使用示例。使用--parallel选项与多个专用节点结合使用，否则不会从--parallel中获得任何好处

srun -Aproj<your_project> -N2 -t=24:00:00 --exclusive \
ngv synthesis \
    --config ... \
    // all above options as before
    --parallel

Snakemake使用示例。

snakemake \
    --snakefile snakemake/Snakefile \
    --cluster-config path/to/cluster.yaml \  # see an example below
    --config bioname path/to/bioname \
    synthesis

不要忘记在您的cluster.yaml中为synthesis添加条目

synthesis:
  jobname: ngv_synthesis
  account: '<your project>'
  nodes: <better have at least 2, 4 is recommended>
  partition: 'prod'
  constraint: 'cpu'
  time: '04:00:00' # feel free to increase time 
  cpus-per-task: 2
  exclusive: true
  mem: 0

此外，强烈建议提前导出以下Dask变量以获得更好的性能

export DASK_DISTRIBUTED__WORKER__USE_FILE_LOCKING=False
export DASK_DISTRIBUTED__WORKER__MEMORY__TARGET=False  # don't spill to disk
export DASK_DISTRIBUTED__WORKER__MEMORY__SPILL=False  # don't spill to disk
export DASK_DISTRIBUTED__WORKER__MEMORY__PAUSE=0.80  # pause execution at 80% memory use
export DASK_DISTRIBUTED__WORKER__MEMORY__TERMINATE=0.95  # restart the worker at 95% use
# Reduce dask profile memory usage/leak (see https://github.com/dask/distributed/issues/4091)
export DASK_DISTRIBUTED__WORKER__PROFILE__INTERVAL=10000ms  # Time between statistical profiling queries
export DASK_DISTRIBUTED__WORKER__PROFILE__CYCLE=1000000ms  # Time between starting new profile

最后的sbatch脚本示例

#!/bin/bash
# below SBATCH options are not related to options from cluster.yaml 
#SBATCH --partition prod
#SBATCH --account proj62
#SBATCH --nodes 1  # It is not recommended to set higher because synthesis task will be launched with SBATCH options from its entry in `cluster.yaml`  
#SBATCH --time 08:00:00
#SBATCH --job-name sNGV
#SBATCH --output out-%J.log
#SBATCH --error err-%J.log
#SBATCH --mem 200000
#SBATCH --exclusive
export DASK_DISTRIBUTED__WORKER__USE_FILE_LOCKING=False
export DASK_DISTRIBUTED__WORKER__MEMORY__TARGET=False  # don't spill to disk
export DASK_DISTRIBUTED__WORKER__MEMORY__SPILL=False  # don't spill to disk
export DASK_DISTRIBUTED__WORKER__MEMORY__PAUSE=0.80  # pause execution at 80% memory use
export DASK_DISTRIBUTED__WORKER__MEMORY__TERMINATE=0.95  # restart the worker at 95% use
# Reduce dask profile memory usage/leak (see https://github.com/dask/distributed/issues/4091)
export DASK_DISTRIBUTED__WORKER__PROFILE__INTERVAL=10000ms  # Time between statistical profiling queries
export DASK_DISTRIBUTED__WORKER__PROFILE__CYCLE=1000000ms  # Time between starting new profile

source <venv with ArchNGV installed>/bin/activate

snakemake --snakefile <path to Snakefile of this project> \
          --config bioname=<path to bioname> \
          --directory <path to save results> \
          --cluster-config <path to your cluster.yaml> \
          -f synthesis

引用

当您在研究中使用ArchNGV软件或方法时，我们要求您引用以下出版物（包括海报展示）

Zisis E, Keller D, Kanari L, Arnaudon A, Gevaert M, Delemontex T, Coste B, Foni A, Abdellah M, Calì C, Hess K, Magistretti PJ, Schürmann F, Markram H. 2021. Digital Reconstruction of the Neuro-Glia-Vascular Architecture. Cerebral Cortex. 31:5686–5703.

@article{10.1093/cercor/bhab254,
    author = {Zisis, Eleftherios and Keller, Daniel and Kanari, Lida and Arnaudon, Alexis and Gevaert, Michael and Delemontex, Thomas and Coste, Benoît and Foni, Alessandro and Abdellah, Marwan and Calì, Corrado and Hess, Kathryn and Magistretti, Pierre Julius and Schürmann, Felix and Markram, Henry},
    title = "{Digital Reconstruction of the Neuro-Glia-Vascular Architecture}",
    journal = {Cerebral Cortex},
    volume = {31},
    number = {12},
    pages = {5686-5703},
    year = {2021},
    month = {08},
    issn = {1047-3211},
    doi = {10.1093/cercor/bhab254},
    url = {https://doi.org/10.1093/cercor/bhab254},
    eprint = {https://academic.oup.com/cercor/article-pdf/31/12/5686/40814577/bhab254.pdf},
}

致谢

本出版物是基于由国王阿卜杜拉科技大学（KAUST）资助研究办公室（OSR）资助的工作，资助编号为OSR-2017-CRG6-3438。

本软件的开发得到了瑞士联邦理工学院（EPFL）的蓝脑项目资助，该项目是洛桑联邦理工学院的研究中心，由瑞士政府ETH董事会资助。

版权（c）2019-2024 蓝脑项目/EPFL