PyNIDM 4.1.0

1.6.1 BIDS MRI转换为NIDM

此程序将BIDS MRI数据集转换为NIDM-Experiment RDF文档。它将解析表型信息，简单存储变量/值并链接到相关的json数据字典文件。要使用此工具，请设置您的INTERLEX_API_KEY环境变量为您的唯一API密钥。要获取Interlex API密钥，请访问SciCrunch，注册账户，然后点击“我的账户”和“API密钥”为您的账户添加新的API密钥。

$ bidsmri2nidm -d [ROOT BIDS DIRECT] -bidsignore

usage: bidsmri2nidm [-h] -d DIRECTORY [-jsonld] [-bidsignore] [-no_concepts]
                 [-json_map JSON_MAP] [-log LOGFILE] [-o OUTPUTFILE]

This program will represent a BIDS MRI dataset as a NIDM RDF document and provide user with opportunity to annotate
the dataset (i.e. create sidecar files) and associate selected variables with broader concepts to make datasets more
FAIR.

Note, you must obtain an API key to Interlex by signing up for an account at scicrunch.org then going to My Account
and API Keys.  Then set the environment variable INTERLEX_API_KEY with your key.

optional arguments:
  -h, --help            show this help message and exit
  -d DIRECTORY          Full path to BIDS dataset directory
  -jsonld, --jsonld     If flag set, output is json-ld not TURTLE
  -bidsignore, --bidsignore
                     If flag set, tool will add NIDM-related files to .bidsignore file
  -no_concepts, --no_concepts
                     If flag set, tool will no do concept mapping
  -log LOGFILE, --log LOGFILE
                     Full path to directory to save log file. Log file name is bidsmri2nidm_[basename(args.directory)].log
  -o OUTPUTFILE         Outputs turtle file called nidm.ttl in BIDS directory by default..or whatever path/filename is set here

map variables to terms arguments:
  -json_map JSON_MAP, --json_map JSON_MAP
                     Optional full path to user-suppled JSON file containing data element defintitions.

1.6.2 CSV文件转换为NIDM

本程序将加载CSV文件，并对标题变量名进行遍历，对https://scicrunch.org/nidm-terms进行弹性搜索，以匹配变量名的NIDM-ReproNim标签术语。用户将交互式地选择一个术语与变量名关联。然后将结果注释的CSV数据写入NIDM数据文件。要使用此工具，请将您的INTERLEX_API_KEY环境变量设置为您唯一的API密钥。要获取Interlex API密钥，请访问SciCrunch，注册一个账户，然后点击“我的账户”和“API密钥”为您的账户添加新的API密钥。

usage: csv2nidm [-h] -csv CSV_FILE [-json_map JSON_MAP | -redcap REDCAP]
                [-nidm NIDM_FILE] [-no_concepts] [-log LOGFILE] -out
                OUTPUT_FILE

This program will load in a CSV file and iterate over the header variable
names performing an elastic search of https://scicrunch.org/ for NIDM-ReproNim
tagged terms that fuzzy match the variable names. The user will then
interactively pick a term to associate with the variable name. The resulting
annotated CSV data will then be written to a NIDM data file. Note, you must
obtain an API key to Interlex by signing up for an account at scicrunch.org
then going to My Account and API Keys. Then set the environment variable
INTERLEX_API_KEY with your key.  The tool supports import of RedCap data
dictionaries and will convert relevant information into a json-formatted
annotation file used to annotate the data elements in the resulting NIDM file.

optional arguments:
  -h, --help            show this help message and exit
  -csv CSV_FILE         Full path to CSV file to convert
  -json_map JSON_MAP    Full path to user-suppled JSON file containing
                        variable-term mappings.
  -redcap REDCAP        Full path to a user-supplied RedCap formatted data
                        dictionary for csv file.
  -nidm NIDM_FILE       Optional full path of NIDM file to add CSV->NIDM
                        converted graph to
  -no_concepts          If this flag is set then no concept associations will
                        beasked of the user. This is useful if you already
                        have a -json_map specified without concepts and want
                        tosimply run this program to get a NIDM file with user
                        interaction to associate concepts.
  -log LOGFILE, --log LOGFILE
                        full path to directory to save log file. Log file name
                        is csv2nidm_[arg.csv_file].log
  -out OUTPUT_FILE      Full path with filename to save NIDM file

1.6.3 convert

此功能将NIDM文件转换为各种RDF支持的格式，并按输入文件的名称和位置命名/存储。

Usage: pynidm convert [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT      A comma separated list of NIDM files with
                                full path  [required]
  -t, --type [turtle|jsonld|xml-rdf|n3|trig]
                                If parameter set then NIDM file will be
                                exported as JSONLD  [required]
  --help                          Show this message and exit.

1.6.4 concatenate

此功能将连接NIDM文件。警告，不会进行合并，如果您从同一研究的多次访问中连接NIDM文件，您可能会遇到具有相同主题ID的多个prov:agents。如果您想按主题ID合并NIDM文件，请参阅pynidm merge。

Usage: pynidm concat [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT  A comma separated list of NIDM files with full
                            path  [required]
  -o, --out_file TEXT         File to write concatenated NIDM files
                            [required]
  --help                      Show this message and exit.

1.6.5 visualize

此命令将为提供的NIDM文件生成可视化(pdf)，命名为与输入文件相同的名称，并存储在同一目录中。

Usage: pynidm visualize [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT  A comma separated list of NIDM files with full
                            path  [required]
  --help                      Show this message and exit.

1.6.6 merge

此功能将合并NIDM文件。有关支持的合并操作，请参阅命令行参数。

Usage: pynidm merge [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT  A comma separated list of NIDM files with full
                           path  [required]
  -s, --s                     If parameter set then files will be merged by
                           ndar:src_subjec_id of prov:agents
      -o, --out_file TEXT         File to write concatenated NIDM files
                           [required]
      --help                      Show this message and exit.

1.6.7 查询

此功能为NIDM图提供查询支持。

Usage: pynidm query [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT      A comma separated list of NIDM files with
                                  full path  [required]
  -nc, --cde_file_list TEXT       A comma separated list of NIDM CDE files
                                  with full path. Can also be set in the
                                  CDE_DIR environment variable
  -q, --query_file FILENAME       Text file containing a SPARQL query to
                                  execute
  -p, --get_participants          Parameter, if set, query will return
                                  participant IDs and prov:agent entity IDs
  -i, --get_instruments           Parameter, if set, query will return list of
                                  onli:assessment-instrument:
  -iv, --get_instrument_vars      Parameter, if set, query will return list of
                                  onli:assessment-instrument: variables
  -de, --get_dataelements         Parameter, if set, will return all
                                  DataElements in NIDM file
  -debv, --get_dataelements_brainvols
                                  Parameter, if set, will return all brain
                                  volume DataElements in NIDM file along with
                                  details
  -bv, --get_brainvols            Parameter, if set, will return all brain
                                  volume data elements and values along with
                                  participant IDs in NIDM file
  -o, --output_file TEXT          Optional output file (CSV) to store results
                                  of query
  -u, --uri TEXT                  A REST API URI query
  -j / -no_j                      Return result of a uri query as JSON
  -v, --verbosity TEXT            Verbosity level 0-5, 0 is default
  --help                          Show this message and exit.

有关REST API URI格式和使用方法的详细信息，可以在REST API使用页面上找到。

1.6.8 线性回归

此功能为NIDM图提供线性回归支持。

Usage: pynidm linear-regression [OPTIONS]

Options:
  -nl, --nidm_file_list TEXT      A comma-separated list of NIDM files with
                                  full path  [required]
  -r, --regularization TEXT       Parameter, if set, will return the results of
                                  the linear regression with L1 or L2 regularization
                                  depending on the type specified, and the weight
                                  with the maximum likelihood solution. This will
                                  prevent overfitting. (Ex: -r L1)
  -model, --ml TEXT                   An equation representing the linear
                                  regression. The dependent variable comes
                                  first, followed by "=" or "~", followed by
                                  the independent variables separated by "+"
                                  (Ex: -model "fs_003343 = age*sex + sex +
                                  age + group + age*group + bmi") [required]
  -contstant, --ctr TEXT              Parameter, if set, will return differences in
                                  variable relationships by group. One or
                                  multiple parameters can be used (multiple
                                  parameters should be separated by a comma-
                                  separated list) (Ex: -contrast group,age)
  -o, --output_file TEXT          Optional output file (TXT) to store results
                                  of query
  --help                          Show this message and exit.

要成功使用线性回归算法，结构、语法和查询非常重要。以下是最大化工具有用性的方法

首先，使用pynidm查询发现要使用的变量。PyNIDM允许使用数据元素（PIQ_tca9ck）、特定的URL（http://uri.interlex.org/ilx_0100400）或源变量（DX_GROUP）。

一个可能的查询示例是

pynidm query -nl /simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_a/nidm.ttl,/simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_b/nidm.ttl -u /projects?fields=fs_000008,DX_GROUP,PIQ_tca9ck,http://uri.interlex.org/ilx_0100400

您还可以这样做

pynidm query -nl /simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_a/nidm.ttl,/Users/Ashu/Downloads/simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_b/nidm.ttl -gf fs_000008,DX_GROUP,PIQ_tca9ck,http://uri.interlex.org/ilx_0100400

查询将在-nl参数指定的两个文件中查找指定的变量。在这种情况下，我们使用fs_000008和DX_GROUP（源变量）、一个URL（http://uri.interlex.org/ilx_0100400）和一个数据元素（PIQ_tca9ck）。根据您是否使用-gf或-u，文件输出会有所不同。使用-gf时，它将分别返回两个文件中的变量，而-u将它们合并。

现在我们已经选择了变量，我们可以执行线性回归。在这个例子中，我们将研究DX_GROUP、扫描时的年龄和PIQ对额叶脑容量的影响。

为此特定数据使用的命令是

pynidm linear-regression -nl /simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_a/nidm.ttl,/simple2_NIDM_examples/datasets.datalad.org/abide/RawDataBIDS/CMU_b/nidm.ttl -model "fs_000008 = DX_GROUP + PIQ_tca9ck + http://uri.interlex.org/ilx_0100400" -contrast "DX_GROUP" -r L1

-nl指定从其中提取数据的文件，而-model是在其上执行线性回归模型的模型。在这种情况下，变量是fs_000008（因变量，额叶脑容量）、DX_GROUP（诊断组）、PIQ_tca9ck（PIQ）和http://uri.interlex.org/ilx_0100400（扫描时的年龄）。-contrast参数表示使用DX_GROUP对比数据，然后进行L1正则化以防止过拟合。

有关REST API URI格式和使用方法的详细信息，请参阅下文。

2.3.1 操作

/projects

获取所有可用项目ID的列表。

支持的查询参数：无

/projects/{project_id}

查看项目的一些详细信息。这将包括项目中使用的受试者ID和数据元素列表

支持的查询参数：filter

/projects/{project_id}/subjects

获取项目中的受试者列表

支持的查询参数：filter

/projects/{project_id}/subjects/{subject_id}

获取特定受试者的详细信息。这包括与受试者相关的任何仪器或衍生物的结果，以及相关活动的列表。

支持的查询参数：无

/projects/{project_id}/subjects/{subject_id}/instruments/{instrument_id}

获取特定仪器的值

支持的查询参数：无

/projects/{project_id}/subjects/{subject_id}/derivatives/{derivative_id}

获取特定衍生物的值

支持的查询参数：无

/statistics/projects/{project_id}

查看项目统计信息。您还可以使用此操作通过使用字段查询选项获取特定仪器或衍生物条目的统计信息。

支持的查询参数：filter, field

/statistics/projects/{project_id}/subjects/{subject_id}

查看项目的一些详细信息。这将包括项目中使用的受试者ID和数据元素列表

支持的查询参数：无

2.3.2 查询参数

filter

当您只想接收符合某些标准的受试者数据时，使用filter查询参数。filter值的格式应为以下形式

identifier op value [ and identifier op value and ... ]

标识符应格式化为“instrument.ID”或“derivatives.ID”。您可以使用仪器或项目详细信息中的数据元素部分显示的任何仪器ID。对于衍生物ID，您可以使用衍生物字段URI的最后一部分（例如，对于URI http://purl.org/nidash/fsl#fsl_000007，ID为“fsl_000007”）或查看衍生物数据时显示的精确标签（例如，“Left-Caudate (mm^3)”）。

“op”可以是“eq”，“gt”，“lt”之一。

示例过滤器

?filter=instruments.AGE_AT_SCAN gt 30 ?filter=instrument.AGE_AT_SCAN eq 21 and derivative.fsl_000007 lt 3500

fields

fields查询参数用于指定在统计操作中应详细说明哪些字段。对于每个指定的字段，结果将显示该字段在所有受试者中匹配操作和filter的值的最大值、最小值、平均值、中位数和标准差。可以通过用逗号分隔每个字段来指定多个字段。

字段应格式化为与filter参数中指定的标识符相同的方式。

示例字段查询

http://localhost:5000/statistics/projects/abc123?field=instruments.AGE_AT_SCAN,derivatives.fsl_000020

2.4.1 示例

curl http://localhost:5000/projects

[
    "dc1bf9be-10a3-11ea-8779-003ee1ce9545"
]

curl http://localhost:5000/projects/dc1bf9be-10a3-11ea-8779-003ee1ce9545

{
 "http://www.w3.org/1999/02/22-rdf-syntax-ns#type": "http://purl.org/nidash/nidm#Project",
 "dctypes:title": "ABIDE CMU_a Site",
 "http://www.w3.org/ns/prov#Location": "/datasets.datalad.org/abide/RawDataBIDS/CMU_a",
 "sio:Identifier": "1.0.1",
 "nidm:NIDM_0000171": 14,
 "age_max": 33.0,
 "age_min": 21.0,
 "ndar:gender": [
     "1",
     "2"
 ],
 "obo:handedness": [
     "R",
     "L",
     "Ambi"
 ]
}

pynidm query -nl "cmu_a.nidm.ttl" -u http://localhost:5000/projects/dc1bf9be-10a3-11ea-8779-003ee1ce9545/subjects

deef8eb2-10a3-11ea-8779-003ee1ce9545
df533e6c-10a3-11ea-8779-003ee1ce9545
ddbfb454-10a3-11ea-8779-003ee1ce9545
df21cada-10a3-11ea-8779-003ee1ce9545
dcfa35b2-10a3-11ea-8779-003ee1ce9545
de89ce4c-10a3-11ea-8779-003ee1ce9545
dd2ce75a-10a3-11ea-8779-003ee1ce9545
ddf21020-10a3-11ea-8779-003ee1ce9545
debc0f74-10a3-11ea-8779-003ee1ce9545
de245134-10a3-11ea-8779-003ee1ce9545
dd5f2f30-10a3-11ea-8779-003ee1ce9545
dd8d4faa-10a3-11ea-8779-003ee1ce9545
df87cbaa-10a3-11ea-8779-003ee1ce9545
de55285e-10a3-11ea-8779-003ee1ce9545

pynidm query -nl ttl/cmu_a.nidm.ttl -u /statistics/projects/dc1bf9be-10a3-11ea-8779-003ee1ce9545?fields=instruments.AGE_AT_SCAN,derivatives.fsl_000001

-------------------------------------------------  ---------------------------------------------
"http://www.w3.org/1999/02/22-rdf-syntax-ns#type"  http://www.w3.org/ns/prov#Activity
"title"                                            ABIDE CMU_a Site
"Identifier"                                       1.0.1
"prov:Location"                                    /datasets.datalad.org/abide/RawDataBIDS/CMU_a
"NIDM_0000171"                                     14
"age_max"                                          33.0
"age_min"                                          21.0

  gender
--------
       1
       2

handedness
------------
R
L
Ambi

subjects
------------------------------------
de89ce4c-10a3-11ea-8779-003ee1ce9545
deef8eb2-10a3-11ea-8779-003ee1ce9545
dd8d4faa-10a3-11ea-8779-003ee1ce9545
ddbfb454-10a3-11ea-8779-003ee1ce9545
de245134-10a3-11ea-8779-003ee1ce9545
debc0f74-10a3-11ea-8779-003ee1ce9545
dd5f2f30-10a3-11ea-8779-003ee1ce9545
ddf21020-10a3-11ea-8779-003ee1ce9545
dcfa35b2-10a3-11ea-8779-003ee1ce9545
df21cada-10a3-11ea-8779-003ee1ce9545
df533e6c-10a3-11ea-8779-003ee1ce9545
de55285e-10a3-11ea-8779-003ee1ce9545
df87cbaa-10a3-11ea-8779-003ee1ce9545
dd2ce75a-10a3-11ea-8779-003ee1ce9545

-----------  ------------------  --------
AGE_AT_SCAN  max                 33
AGE_AT_SCAN  min                 21
AGE_AT_SCAN  median              26
AGE_AT_SCAN  mean                26.2857
AGE_AT_SCAN  standard_deviation   4.14778
-----------  ------------------  --------

----------  ------------------  -----------
fsl_000001  max                 1.14899e+07
fsl_000001  min                 5.5193e+06
fsl_000001  median              7.66115e+06
fsl_000001  mean                8.97177e+06
fsl_000001  standard_deviation  2.22465e+06
----------  ------------------  -----------

pynidm query -j -nl "cmu_a.nidm.ttl" -u http://localhost:5000/projects/dc1bf9be-10a3-11ea-8779-003ee1ce9545/subjects/df21cada-10a3-11ea-8779-003ee1ce9545

 {
"uuid": "df21cada-10a3-11ea-8779-003ee1ce9545",
"id": "0050665",
"activity": [
  "e28dc764-10a3-11ea-a7d3-003ee1ce9545",
  "df28e95a-10a3-11ea-8779-003ee1ce9545",
  "df21c76a-10a3-11ea-8779-003ee1ce9545"
],
"instruments": {
  "e28dd218-10a3-11ea-a7d3-003ee1ce9545": {
    "SRS_VERSION": "nan",
    "ADOS_MODULE": "nan",
    "WISC_IV_VCI": "nan",
    "WISC_IV_PSI": "nan",
    "ADOS_GOTHAM_SOCAFFECT": "nan",
    "VINELAND_PLAY_V_SCALED": "nan",
    "null": "http://www.w3.org/ns/prov#Entity",
    "VINELAND_EXPRESSIVE_V_SCALED": "nan",
    "SCQ_TOTAL": "nan",
    "SRS_MOTIVATION": "nan",
    "PIQ": "104.0",
    "FIQ": "109.0",
    "WISC_IV_PRI": "nan",
    "FILE_ID": "CMU_a_0050665",
    "VIQ": "111.0",
    "WISC_IV_VOCAB_SCALED": "nan",
    "VINELAND_DAILYLVNG_STANDARD": "nan",
    "WISC_IV_SIM_SCALED": "nan",
    "WISC_IV_DIGIT_SPAN_SCALED": "nan",
    "AGE_AT_SCAN": "33.0"
    }
 },
"derivatives": {
    "b9fe0398-16cc-11ea-8729-003ee1ce9545": {
       "URI": "http://iri.nidash.org/b9fe0398-16cc-11ea-8729-003ee1ce9545",
       "values": {
         "http://purl.org/nidash/fsl#fsl_000005": {
           "datumType": "ilx_0102597",
           "label": "Left-Amygdala (voxels)",
           "value": "1573",
           "units": "voxel"
         },
         "http://purl.org/nidash/fsl#fsl_000004": {
           "datumType": "ilx_0738276",
           "label": "Left-Accumbens-area (mm^3)",
           "value": "466.0",
           "units": "mm^3"
         },
         "http://purl.org/nidash/fsl#fsl_000003": {
           "datumType": "ilx_0102597",
           "label": "Left-Accumbens-area (voxels)",
           "value": "466",
           "units": "voxel"
         }
       },
       "StatCollectionType": "FSLStatsCollection"
    }
 }