docker-charon

高效地将您的Docker镜像转移到断网系统。

(断网系统是指未连接到互联网的系统)

来自维基百科

在希腊神话中，卡戎（/ˈkɛərɒn, -ən/；古希腊语：Χάρων）是一位引渡者，是冥王哈迪斯的渡船夫，负责将新近去世并接受了葬礼仪式的灵魂，渡过阿刻戎河（或在某些后来的叙述中，渡过斯提克斯河），将生者的世界与死者的世界分开。

安装

pip install docker-charon

您不需要在本地安装Docker来运行此工具。

示例

您可以直接从命令行运行这些示例。这里我们使用docker，但这只是为了演示目的。在真实场景中，您根本不需要Docker。只需要两个注册表。

作为命令行使用

pip install docker-charon

# we setup a local registry and will pretend it's air-gapped
# we'll transfer docker images from dockerhub to our local registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2


docker-charon make-payload -f ./payload.zip python:3.9.2-alpine,elasticsearch:7.14.1
docker-charon push-payload -f ./payload.zip --insecure --registry=localhost:5000

# our images are now in the air-gapped registry
# you can verify it with 
docker pull localhost:5000/python:3.9.2-alpine

# now let's upgrade images in our local registry without taking the layers that are already there
# we take higher versions, python:3.9.2 -> python:3.9.3 for example

docker-charon make-payload -f ./payload2.zip --already-transferred=python:3.9.2-alpine,elasticsearch:7.14.1 python:3.9.3-alpine,elasticsearch:7.14.2
# you'll see that some layers are skipped because they are already in the registry
# the outputs will be something like this for those layers:
# Skipping elasticsearch/sha256:7a0437... because it's already in the destination registry in the repository elasticsearch
# the argument --already-transferred is the one that does the magic

docker-charon push-payload -f ./payload2.zip --insecure --registry=localhost:5000 
# you can verify it with 
docker pull localhost:5000/python:3.9.3-alpine

作为Python库使用

pip install docker-charon

# we setup a local registry and will pretend it's air-gapped
# we'll transfer docker images from dockerhub to our local registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2

from docker_charon import make_payload, push_payload

make_payload("./payload.zip", ["python:3.9.2-alpine", "elasticsearch:7.14.1"])
push_payload("./payload.zip", secure=False, registry="localhost:5000")

# our images are now in the air-gapped registry
# you can verify it with 
# docker pull localhost:5000/python:3.9.2-alpine

# now let's upgrade images in our local registry without taking the layers that are already there
# we take higher versions, python:3.9.2 -> python:3.9.3 for example
make_payload(
    "./payload2.zip", 
    ["python:3.9.3-alpine", "elasticsearch:7.14.2"], 
    docker_images_already_transferred=["python:3.9.2-alpine", "elasticsearch:7.14.1"]
)
# you'll see that some layers are skipped because they are already in the registry
# the outputs will be something like this for those layers:
# Skipping elasticsearch/sha256:7a0437... because it's already in the destination registry in the repository elasticsearch
# the argument docker_images_already_transferred is the one that does the magic
push_payload("./payload2.zip", secure=False, registry="localhost:5000")

# you can verify it with 
# docker pull localhost:5000/python:3.9.3-alpine

作为Docker镜像使用

它与命令行版本具有相同的命令行界面，但我们将使用stdin/stdout而不是-f以避免使用Docker卷。

# we setup a local registry and will pretend it's air-gapped
# we'll transfer docker images from dockerhub to our local registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2

docker run gabrieldemarmiesse/docker-charon make-payload python:3.9.2-alpine,elasticsearch:7.14.1 > ./payload.zip
# here we use the -i argument of docker run to read from stdin (by default stdin is not available in docker)
# we also use --net=host to be able to communicate with localhost:5000
docker run -i --net=host gabrieldemarmiesse/docker-charon push-payload --insecure --registry=localhost:5000 < ./payload.zip

# our images are now in the air-gapped registry
# you can verify it with 
docker pull localhost:5000/python:3.9.2-alpine

# now let's upgrade images in our local registry without taking the layers that are already there
# we take higher versions, python:3.9.2 -> python:3.9.3 for example

docker run gabrieldemarmiesse/docker-charon make-payload --already-transferred=python:3.9.2-alpine,elasticsearch:7.14.1 python:3.9.3-alpine,elasticsearch:7.14.2 > ./payload2.zip
# you'll see that some layers are skipped because they are already in the registry
# the outputs will be something like this for those layers:
# Skipping elasticsearch/sha256:7a0437... because it's already in the destination registry in the repository elasticsearch
# the argument --already-transferred is the one that does the magic

docker run  -i --net=host gabrieldemarmiesse/docker-charon push-payload --insecure --registry=localhost:5000 < ./payload2.zip
# you can verify it with 
docker pull localhost:5000/python:3.9.3-alpine

参数

命令行和Docker镜像

docker-charon make-payload

$ docker-charon make-payload --help
Usage: docker-charon make-payload [OPTIONS] DOCKER_IMAGES_TO_TRANSFER

  Create a payload (.zip file) with docker images inside. This zip file can
  then be unpacked into a registry in another system.

  By providing images that were already transferred to the new registry, you
  can reduce the size and creation time of the payload. This is because
  docker-charon only takes the layers that were not already transferred.

  The payload is written to stdout by default. You can provide a file path
  to write the payload to by using the --file (or -f) option.

Arguments:
  DOCKER_IMAGES_TO_TRANSFER  docker images to transfer, a commas delimited
                             list of docker image names. Do not include the
                             registry name.  [required]


Options:
  -a, --already-transferred TEXT  docker images already present in the remote
                                  registry, a commas delimited list of docker
                                  image names. Do not include the registry
                                  name.

  -f, --file TEXT                 Where to write the payload zip file. If this
                                  is not provided, the payload will be written
                                  to stdout.

  -r, --registry TEXT             The registry to push the payload to. It
                                  defaults to dockerhub (registry-1.docker.io)

  -i, --insecure                  Use --insecure if the registry uses http
                                  instead of https

  -u, --username TEXT             The username to use to connect to the
                                  registry. If you want more security and
                                  don't want your username to appear in your
                                  shell history, you can also use the
                                  environment variable DOCKER_CHARON_USERNAME

  -p, --password TEXT             The password to use to connect to the
                                  registry. If you want more security and
                                  don't want your password to appear in your
                                  shell history, you can also use the
                                  environment variable DOCKER_CHARON_PASSWORD

docker-charon push-payload

$ docker-charon push-payload --help
Usage: docker-charon push-payload [OPTIONS]

  Unpack the payload (.zip file) into a docker registry.

  The zip file must have been created by 'docker-charon make-payload ...'

  This command will output to stdout the list of images that were
  transferred. One image per line.

  By default, the payload is read from stdin. You can provide a file path to
  read the payload from by using the --file (or -f) option.

Options:
  -f, --file TEXT      The payload zip file. If this is not provided, the
                       payload will be read from stdin.

  -s, --strict         Fails if there is a mismatch between what was given
                       with --already-transferred and what is in the registry.
                       [default: False]

  -r, --registry TEXT  The registry to push the payload to. It defaults to
                       dockerhub (registry-1.docker.io)  [default:
                       registry-1.docker.io]

  -i, --insecure       Use --insecure if the registry uses http instead of
                       https

  -u, --username TEXT  The username to use to connect to the registry. If you
                       want more security and don't want your username to
                       appear in your shell history, you can also use the
                       environment variable DOCKER_CHARON_USERNAME

  -p, --password TEXT  The password to use to connect to the registry. If you
                       want more security and don't want your password to
                       appear in your shell history, you can also use the
                       environment variable DOCKER_CHARON_PASSWORD

Python库

make_payload

从Docker镜像列表创建有效载荷

所有Docker镜像必须在同一注册表中。这是docker-charon包的一个限制。

如果您对多注册表感兴趣，请提交一个issue。

参数

• zip_file：要创建的zip文件的路径。可以是pathlib.Path或str。也可以传递一个文件对象。包含所有Docker镜像的有效载荷是一个单一的zip文件。
• docker_images_to_transfer：要转移的Docker镜像列表。不要在镜像名称中包含注册表名称。
• docker_images_already_transferred：已传输到空气隔离注册表的Docker镜像列表。不要在镜像名称中包含注册表名称。这是可选的，但如果使用它，可以使有效负载更小。
• registry：从中提取镜像的注册表。注册表的名称不能包含在docker_images_to_transfer和docker_images_already_transferred中。默认为dockerhub（registry-1.docker.io）。
• secure：如果注册表不支持HTTPS（TLS），则设置为False。默认为True。
• username：用于注册表身份验证的用户名。如果注册表不需要身份验证，则可选。
• password：用于注册表身份验证的密码。如果注册表不需要身份验证，则可选。

push_payload

将有效负载推送到注册表。

它将遍历Docker镜像，并推送blob和manifest。

参数

• zip_file：包含有效负载的zip文件。可以是pathlib.Path、str或类似文件的对象。
• strict：False为默认值。如果为True，则在某些blob/镜像缺失时将引发错误。这可能发生在用户在docker_images_already_transferred中设置了一个不在注册表中的镜像。
• registry：推送镜像的注册表。默认为dockerhub（registry-1.docker.io）。
• secure：是否使用TLS（HTTPS）连接到注册表，默认为True。
• username：用于连接注册表的用户名。如果注册表不需要身份验证，则可选。
• password：用于连接注册表的密码。如果注册表不需要身份验证，则可选。

返回

注册表中加载的Docker镜像列表。

它还包括已经存在于注册表中且未包含在有效负载中以优化大小的Docker镜像列表。换句话说，这是您传递给函数docker_charon.make_payload(...)的docker_images_to_transfer参数。

为什么需要这样的包？

常规方法：docker save和load

这是一个已经有简单解决方案的问题。您可以使用docker save和docker load命令，通过tar将镜像传输到空气隔离系统。这实际上是简单用例的推荐方法。

以下是docker pull -> docker save -> docker load -> docker push方法的总结

但假设您想要扩展交付，进行定期更新，您很快就会注意到docker save和docker load的问题

• 创建有效负载时的速度：必须从注册表提取镜像。由Docker引擎解包，然后再次打包到tar文件中。这涉及大量的磁盘访问。
• 加载有效负载时的速度：tar文件由Docker引擎解包。然后您可以推送镜像到注册表。这涉及到大量层的解包和重新打包。
• 有效负载的大小。docker save将所有声明的层和镜像都取走。即使某些镜像和层已经在空气隔离系统中存在。

Docker-charon方法

Docker-charon是一个解决这些问题的包。

它直接读取注册表来创建有效负载。您甚至不需要在制作有效负载和加载有效负载的机器上安装Docker。

如果您提供了空气隔离系统注册表中已存在的镜像，它还可以以智能方式计算差异。这意味着有效负载会更小，因为某些层不需要再次传输。

以下是Docker-charon方法的总结

它如何工作？

Docker-charon将查询注册表，获取Docker镜像的manifest，然后只下载尚未传输到空气隔离注册表的blob。

所有内容都被放入了一个单独的压缩文件中。选择压缩文件是因为可以在其中随机访问文件，然后按所需顺序解压缩。

在断网系统中，push_payload 函数会读取压缩文件索引，并实时将数据块和清单推送到注册表。

然后，Docker 镜像就准备在您的断网集群中使用了！