DOMIAS：通过过拟合检测对合成数据进行成员推理攻击

安装

可以使用以下命令从PyPI安装库

$ pip install domias

或从源安装，使用

$ pip install .

API

主要的API调用是

from domias.evaluator import evaluate_performance

evaluate_performance期望输入一个实现了domias.models.generator.GeneratorInterface接口的生成器和一个评估数据集。

evaluate_performance支持的参数是

  generator: GeneratorInterface
      Generator with the `fit` and `generate` methods. The generator MUST not be fitted.
  dataset: int
      The evaluation dataset, used to derive the training and test datasets.
  mem_set_size: int
      The split for the training dataset out of `dataset`
  reference_set_size: int
      The split for the reference dataset out of `dataset`.
  training_epochs: int
      Training epochs
  synthetic_sizes: List[int]
      For how many synthetic samples to test the attacks.
  density_estimator: str, default = "prior"
      Which density to use. Available options:
          * prior
          * bnaf
          * kde
  seed: int
      Random seed
  device: PyTorch device
      CPU or CUDA
  shifted_column: Optional[int]
      Shift a column
  zero_quantile: float
      Threshold for shifting the column.
  reference_kept_p: float
      Held-out dataset parameter

输出是一个字典，其中每个键对应于synthetic_sizes值。

对于每个synthetic_sizes值，字典包含以下键

• MIA_performance：每个攻击的准确率和AUCROC
• MIA_scores：每个攻击的输出分数
• data：评估数据

对于MIA_performance和MIA_scores，以下攻击被评估

• "ablated_eq1" (KDE的式1)
• "ablated_eq2" (DOMIAS (KDE))
• "LOGAN_D1"
• "MC"
• "gan_leaks"
• "gan_leaks_cal"
• "LOGAN_0"
• "eq1" (BNAF的式1)
• "domias"

示例用法

evaluate_performance使用的示例

# third party
import pandas as pd
from sdv.tabular import TVAE

# domias absolute
from domias.evaluator import evaluate_performance
from domias.models.generator import GeneratorInterface


def get_generator(
    epochs: int = 1000,
    seed: int = 0,
) -> GeneratorInterface:
    class LocalGenerator(GeneratorInterface):
        def __init__(self) -> None:
            self.model = TVAE(epochs=epochs)

        def fit(self, data: pd.DataFrame) -> "LocalGenerator":
            self.model.fit(data)
            return self

        def generate(self, count: int) -> pd.DataFrame:
            return self.model.sample(count)

    return LocalGenerator()


dataset = ...  # Load your dataset as numpy array

mem_set_size = 1000
reference_set_size = 1000
training_epochs = 2000
synthetic_sizes = [1000]
density_estimator = "prior"  # prior, kde, bnaf

generator = get_generator(
    epochs=training_epochs,
)

perf = evaluate_performance(
    generator,
    dataset,
    mem_set_size,
    reference_set_size,
    training_epochs=training_epochs,
    synthetic_sizes=[100],
    density_estimator=density_estimator,
)

assert 100 in perf
results = perf[100]

assert "MIA_performance" in results
assert "MIA_scores" in results

print(results["MIA_performance"])

实验

1. 主要实验论文

为了重现DOMIAS、基线模型和删减模型的结果，请运行

cd experiments
python3 domias_main.py --seed 0 --gan_method TVAE --dataset housing --mem_set_size_list 30 50 100 300 500 1000 --reference_set_size_list 10000 --synthetic_sizes 10000 --training_epoch_list 2000

更改特定实验的参数 mem_set_size_list、reference_set_size_list、synthetic_sizes 和 training_epoch_list 的范围（实验 5.1 和 5.2，详细信息见附录 A）以及用于感兴趣生成模型的 gan_method。

或者等价地，运行

cd experiments && bash run_tabular.sh

2. 无先验知识实验（附录 D）

如果使用先验知识（即，没有参考数据集设置），则添加

--density_estimator prior

3. 图像实验（附录 B.3）

注意：CelebA 数据集必须在 experiments/data 文件夹中可用。

要运行带有 CelebA 数据集的实验，首先运行

cd experiments && python3 celeba_gen.py --seed 0 --mem_set_size 4000

然后

cd experiments && python3 celeba_eval.py --seed 0 --mem_set_size 4000

测试

使用以下命令安装测试依赖项

pip install .[testing]

可以使用以下方式执行测试

pytest -vsx

引用

待办事项