pytest-inmanta-lsm

一个pytest插件，用于测试使用lsm的inmanta模块，它建立在pytest-inmanta和pytest-inmanta-extensions之上

安装

pip install pytest-inmanta-lsm

上下文

此插件用于将代码推送到远程编排器并与其交互，通过LSM北向API。它需要一个启用了LSM的编排器，默认设置下未启用ssl或身份验证，并且需要ssh访问编排器机器，以及一个具有sudo权限的用户。

用法

第一种情况：使用远程编排器

此插件围绕remote_orchestrator固定值和RemoteServiceInstance类构建。

您可以轻松编写一个测试用例，将您的项目发送到远程编排器，导出其服务目录，然后部署一个服务。

def test_deploy_service(project: plugin.Project, remote_orchestrator: remote_orchestrator.RemoteOrchestrator) -> None:
    # get connection to remote_orchestrator
    client = remote_orchestrator.client

    # setup project
    project.compile("import quickstart")

    # sync project and export service entities
    remote_orchestrator.export_service_entities()

    # verify the service is in the catalog
    result = client.lsm_service_catalog_get_entity(remote_orchestrator.environment, SERVICE_NAME)
    assert result.code == 200

    # Test the synchronous service instance class
    instance = remote_service_instance.RemoteServiceInstance(
        remote_orchestrator=remote_orchestrator,
        service_entity_name=SERVICE_NAME,
    )

    # Create the service instance and stop waiting in a transient state, and get the service instance
    # as it is in the target state.
    created = instance.create(
        {
            "router_ip": "10.1.9.17",
            "interface_name": "eth1",
            "address": "10.0.0.254/24",
            "vlan_id": 14,
        },
        wait_for_state="creating",
        timeout=60,
    )

    # Wait for up state.  The method will check each version that the service goes through,
    # starting AFTER the start_version if it is provided, or the current version of the service
    # if start_version is not provided.  As soon as a version is a match, the helper will return the
    # service instance at this given state.
    # It is important to provide the correct start_version to avoid falling in any of these situations:
    # 1. We could miss the target state, if at the moment we start waiting for the target state, it is
    #    already reached and no start_version is provided (as we would start looking AFTER the current
    #    version).
    # 2. We could select the wrong target state if we start looking at too old versions (in the case of
    #    an update for example, we might mistake the source up state for the target one).
    # In this example, we start after the version of the creating state, as we know it is before our up
    # state.
    instance.wait_for_state(
        target_state="up",
        start_version=created.version,
        timeout=60,
    )

    # Delete the instance
    instance.delete(wait_for_state="terminated", timeout=60)

来源: test_quickstart.py::test_transient_state

对于更复杂的测试用例，您可能还想部署多个服务。您可以逐个测试它们，或者并行化它们，以便

1. 加快测试用例的速度。
2. 测试服务之间的干扰。

在这种情况下，建议创建一个async辅助程序，跟踪服务的进度，并在sync测试用例中使用它实例化多个服务。

async def service_full_cycle(
    remote_orchestrator: remote_orchestrator.RemoteOrchestrator,
    router_ip: str,
    interface_name: str,
    address: str,
    vlan_id: int,
    vlan_id_update: int,
) -> None:
    # Create an async service instance object
    instance = remote_service_instance_async.RemoteServiceInstance(
        remote_orchestrator=remote_orchestrator,
        service_entity_name=SERVICE_NAME,
    )

    # Create the service instance on the remote orchestrator
    await instance.create(
        {
            "router_ip": router_ip,
            "interface_name": interface_name,
            "address": address,
            "vlan_id": vlan_id,
        },
        wait_for_state="up",
        timeout=60,
    )

    # Update the vlan id
    await instance.update(
        [
            inmanta_lsm.model.PatchCallEdit(
                edit_id=str(uuid.uuid4()),
                operation=inmanta_lsm.model.EditOperation.replace,
                target="vlan_id",
                value=vlan_id_update,
            ),
        ],
        wait_for_state="up",
        timeout=60,
    )

    # Delete the instance
    await instance.delete(wait_for_state="terminated", timeout=60)


def test_full_cycle(project: plugin.Project, remote_orchestrator: remote_orchestrator.RemoteOrchestrator) -> None:
    # get connection to remote_orchestrator
    client = remote_orchestrator.client

    # setup project
    project.compile("import quickstart")

    # sync project and export service entities
    remote_orchestrator.export_service_entities()

    # verify the service is in the catalog
    result = client.lsm_service_catalog_get_entity(remote_orchestrator.environment, SERVICE_NAME)
    assert result.code == 200

    # Create a first service that should be deployed
    first_service = service_full_cycle(
        remote_orchestrator=remote_orchestrator,
        router_ip="10.1.9.17",
        interface_name="eth1",
        address="10.0.0.254/24",
        vlan_id=14,
        vlan_id_update=42,
    )

    # Create another valid service
    another_service = service_full_cycle(
        remote_orchestrator=remote_orchestrator,
        router_ip="10.1.9.18",
        interface_name="eth2",
        address="10.0.0.253/24",
        vlan_id=15,
        vlan_id_update=52,
    )

    # Run all the services
    util.sync_execute_scenarios(first_service, another_service)

来源: test_quickstart.py::test_full_cycle

第二种情况：模拟lsm API

此工具箱还包含一个固定值：lsm_project。此固定值允许您在本地使用lsm模型运行编译。它的优点是

• 您可以对编译过程中想要看到的内容有更细致的控制（选择服务属性、状态、版本等）。
• 如果您只关心测试一个特定案例，那么这比在远程编排器上完成整个生命周期要快得多。
• 您不需要运行远程编排器，因此不需要在任何地方同步整个项目。

简单用法如下。

def test_model(lsm_project: pytest_inmanta_lsm.lsm_project.LsmProject) -> None:
    # Export the service entities
    lsm_project.export_service_entities("import quickstart")

    # Create a service.  This will add it to our inventory, in its initial state
    # (as defined in the lifecycle), and fill in any default attributes we didn't
    # provide.
    service = lsm_project.create_service(
        service_entity_name="vlan-assignment",
        attributes={
            "router_ip": "10.1.9.17",
            "interface_name": "eth1",
            "address": "10.0.0.254/24",
            "vlan_id": 14,
        },
        # With auto_transfer=True, we follow the first auto transfers of the service's
        # lifecycle, triggering a compile (validating compile when appropriate) for
        # each state we meets.
        auto_transfer=True,
    )

    # Assert that the service has been created and is now in creating state
    assert service.state == "creating"

    # Assert that the default value has been added to our attributes
    assert "value_with_default" in service.active_attributes

    # Do a second compile, in the non-validating creating state
    lsm_project.compile(service_id=service.id)

    # Move to the up state
    service.state = "up"
    lsm_project.compile(service_id=service.id)

    # Trigger an update on our service from the up state.  Change the vlan id
    new_attributes = copy.deepcopy(service.active_attributes)
    new_attributes["vlan_id"] = 15
    lsm_project.update_service(
        service_id=service.id,
        attributes=new_attributes,
        auto_transfer=True,
    )

    # Assert that the service has been updated and is now in update_inprogress state
    assert service.state == "update_inprogress"

第三种情况：在活动环境中进行开发。

在某些情况下（例如 PoC），您可能想要更新当前部署在环境中的模块代码。您可以使用 pytest-inmanta-lsm 的 remote_orchestrator 修复程序启动一个新的测试案例，这将清除一切，让您从头开始。或者，您可以使用类似的 remote_orchestrator_access 修复程序，它给您相同的方便的 RemoteOrchestrator 对象，但不会清除任何现有服务或资源的环境。这允许您例如重新导出服务目录，或重新同步模块的源代码并保留所有现有服务。

要这样做，只需使用 remote_orchestrator_access 修复程序创建一个测试案例，并使用与正常 pytest-inmanta-lsm 测试案例相同的 cli/env 变量选项。

def test_update_existing_environment(
    project: plugin.Project,
    remote_orchestrator_access: remote_orchestrator.RemoteOrchestrator,
) -> None:
    """
    Make sure that it is possible to simply run a compile and export service entities,
    without initially cleaning up the environment.
    """

    # Setup the compiler config
    remote_orchestrator_access.setup_config()

    # Do a local compile of our model
    project.compile("import quickstart")

    # Export service entities (and update the project)
    remote_orchestrator_access.export_service_entities()

选项和环境变量

以下选项可用，每个选项都对应一个环境变量。

pytest-inmanta-lsm:
  --lsm-ca-cert
                        The path to the CA certificate file used to authenticate
                        the remote orchestrator. (overrides INMANTA_LSM_CA_CERT)
  --lsm-container-env   If set to true, expect the orchestrator to be running in
                        a container without systemd.  It then assumes that all
                        environment variables required to install the modules
                        are loaded into each ssh session automatically.
                        (overrides INMANTA_LSM_CONTAINER_ENV, defaults to False)
  --lsm-ctr             If set, the fixtures will deploy and orchestrator on the
                        host, using docker (overrides INMANTA_LSM_CONTAINER,
                        defaults to False)
  --lsm-ctr-cfg-file
                        A path to a config file that should be loaded inside the
                        container a server conf. (overrides
                        INMANTA_LSM_CONTAINER_CONFIG_FILE, defaults to
                        src/pytest_inmanta_lsm/resources/my-server-
                        conf.cfg)
  --lsm-ctr-compose-file
                        The path to a docker-compose file, that should be used
                        to setup an orchestrator (overrides
                        INMANTA_LSM_CONTAINER_COMPOSE_FILE, defaults to
                        src/pytest_inmanta_lsm/resources/docker-
                        compose.yml)
  --lsm-ctr-db-version
                        The version of postgresql to use for the db of the
                        orchestrator (overrides
                        INMANTA_LSM_CONTAINER_DB_VERSION, defaults to 10)
  --lsm-ctr-env-file
                        A path to an env file that should be loaded in the
                        container. (overrides INMANTA_LSM_CONTAINER_ENV_FILE,
                        defaults to
                        src/pytest_inmanta_lsm/resources/my-env-file)
  --lsm-ctr-image
                        The container image to use for the orchestrator
                        (overrides INMANTA_LSM_CONTAINER_IMAGE, defaults to
                        containers.inmanta.com/containers/service-
                        orchestrator:4)
  --lsm-ctr-jwe-file
                        A path to an entitlement file, required by the
                        orchestrator (overrides INMANTA_LSM_CONTAINER_JWE_FILE,
                        defaults to /etc/inmanta/license/com.inmanta.jwe)
  --lsm-ctr-license-file
                        A path to a license file, required by the orchestrator
                        (overrides INMANTA_LSM_CONTAINER_LICENSE_FILE, defaults
                        to /etc/inmanta/license/com.inmanta.license)
  --lsm-ctr-pub-key-file
                        A path to a public key that should be set in the
                        container (overrides INMANTA_LSM_CONTAINER_PUB_KEY_FILE,
                        defaults to $HOME/.ssh/id_rsa.pub)
  --lsm-environment
                        The environment to use on the remote server (is created
                        if it doesn't exist) (overrides INMANTA_LSM_ENVIRONMENT,
                        defaults to 719c7ad5-6657-444b-b536-a27174cb7498)
  --lsm-host=LSM_HOST   IP address or domain name of the remote orchestrator api we
                        wish to use in our test. It will be picked up and used by the
                        remote_orchestrator fixture.  This is also the default remote
                        hostname, if it is not specified in the --lsm-rh option.
                        (overrides INMANTA_LSM_HOST, defaults to 127.0.0.1)
  --lsm-no-clean        Don't cleanup the orchestrator after tests (for
                        debugging purposes) (overrides INMANTA_LSM_NO_CLEAN,
                        defaults to False)
  --lsm-srv-port
                        Port the orchestrator api is listening to (overrides
                        INMANTA_LSM_SRV_PORT, defaults to 8888)
  --lsm-rsh=LSM_RSH     A command which allows us to start a shell on the remote
                        orchestrator or send file to it.  When sending files, this value
                        will be passed to the `-e` argument of rsync.  When running a
                        command, we will append the host name and `sh` to this value,
                        and pass the command to execute as input to the open remote
                        shell. (overrides INMANTA_LSM_REMOTE_SHELL)
  --lsm-rh=LSM_RH       The name of the host that we should try to open the remote
                        shell on, as recognized by the remote shell command.  This
                        doesn't have to strictly be a hostname, as long as it is a
                        valid host identifier to the chosen rsh protocol. (overrides
                        INMANTA_LSM_REMOTE_HOST)
  --lsm-ssh-port
                        Port to use to ssh to the remote orchestrator (overrides
                        INMANTA_LSM_SSH_PORT, defaults to 22)
  --lsm-ssh-user
                        Username to use to ssh to the remote orchestrator
                        (overrides INMANTA_LSM_SSH_USER, defaults to centos)
  --lsm-ssl             [True | False] Choose whether to use SSL/TLS or not when
                        connecting to the remote orchestrator. (overrides
                        INMANTA_LSM_SSL, defaults to False)
  --lsm-token
                        The token used to authenticate to the remote
                        orchestrator when authentication is enabled. (overrides
                        INMANTA_LSM_TOKEN)
  --lsm-dump-on-failure
                        Whether to create and save a support archive when a test fails.
                        The support archive will be saved in the /tmp directory of the
                        host running the test and will not be cleaned up. The value of
                        this option can be overwritten for each test case individually
                        by overwriting the value of the remote_orchestrator_dump_on_failure
                        fixture. (overrides INMANTA_LSM_DUMP_ON_FAILURE, defaults to False)

运行测试

测试套件的构建方式

测试套件由两部分组成

• tests/test_containerized_orchestrator.py 文件中定义的测试始终针对由测试套件自己启动的容器运行。
• 所有其他测试都针对通过 pytest 命令传递的选项指定的编排器运行。

先决条件

测试（和使用）pytest-inmanta-lsm 需要

• 一个可用的编排器进行测试
• 访问此编排器的 ssh 权限

步骤

1. 安装依赖项

 pip install -r  requirements.dev.txt  -r  requirements.txt

2. 通过环境变量传递 pytest-inmanta-lsm 的配置。例如：

export INMANTA_LSM_HOST=<the orchestrator>
export INMANTA_LSM_USER=<user>

3. 设置从 LSM 拉取 inmanta 的存储库

export INMANTA_MODULE_REPO=https://USER:LICENSE_TOKEN@modules.inmanta.com/git/inmanta-service-orchestrator/5/{}.git

有关如何配置 Python 软件包存储库以用于 V2 模块（而不是 Git URL）的信息，请参阅此处。

4. 运行测试

   pytest tests

部署本地编排器

您可以在本地部署编排器并对它运行测试。编排器将以容器形式部署，使用 docker。以下是使其工作的先决条件：

1. 在您的机器上安装 docker。

   $ docker version
   

2. 访问编排器镜像（例如 containers.inmanta.com/containers/service-orchestrator:4）。

   $ export INMANTA_LSM_CONTAINER_IMAGE=containers.inmanta.com/containers/service-orchestrator:4
   $ docker pull $INMANTA_LSM_CONTAINER_IMAGE
   

3. 拥有编排器的许可证和权限文件。

   $ ls /etc/inmanta/license/com.inmanta.*
   /etc/inmanta/license/com.inmanta.jwe  /etc/inmanta/license/com.inmanta.license
   $ export INMANTA_LSM_CONTAINER_LICENSE_FILE=/etc/inmanta/license/com.inmanta.license
   $ export INMANTA_LSM_CONTAINER_JWE_FILE=/etc/inmanta/license/com.inmanta.jwe
   

4. 拥有访问编排器的私钥/公钥。

   $ export PRIVATE_KEY=$HOME/.ssh/id_rsa
   $ if [ -f $PRIVATE_KEY ]; then echo "Private key already exists"; else ssh-keygen -t rsa -b 4096 -f $PRIVATE_KEY -N ''; fi
   $ export INMANTA_LSM_CONTAINER_PUB_KEY_FILE="${PRIVATE_KEY}.pub"
   $ if [ -f $INMANTA_LSM_CONTAINER_PUB_KEY_FILE ]; then echo "Public key already exists"; else ssh-keygen -y -f $PRIVATE_KEY > $INMANTA_LSM_CONTAINER_PUB_KEY_FILE; fi
   

如果设置正确，则需要设置此选项

  --lsm-ctr        If set, the fixtures will deploy and orchestrator on the host, using docker (overrides INMANTA_LSM_CONTAINER, defaults to False)

然后，设置任何以 lsm-ctr 前缀开始的选项来正确配置 pytest-inmanta-lsm。您可以指定：

• 许可证和权限文件的路径
• 要使用的容器镜像
• 要使用的 postgres 版本
• 要添加到编排器的公钥
• 应由编排器加载的任何环境文件
• 要覆盖 pytest-inmanta-lsm 使用的 Docker Compose 文件的新 Docker Compose 文件
• 新的服务器配置文件

:警告: 当设置 --lsm-ctr 时，某些选项没有效果。这是以下情况：

• --lsm-host 主机将被覆盖为容器的 ip 地址
• --lsm-srv-port 端口将被覆盖为容器中服务器监听的端口
• --lsm-ssh-port 端口为 22
• --lsm-ssh-user 用户为 inmanta
• --lsm-container-env 自动设置为 true

:灯泡: 当设置 --lsm-ctr 时，某些选项会改变其行为。这是以下情况：

• --lsm-no-clean 当设置时，Docker编排器在测试完成后不会清理。您必须手动进行。