mls.fabfile 0.4

需求

mls.fabfile当前使用knife与Rackspace服务器通信。请确保在您的系统上成功安装并配置了knife。

安装

您可以使用PIP安装mls.fabfile

pip install mls.fabfile

所有必需的依赖项将自动安装。

用法

首先，我们需要一个可工作的knife.rb文件来与我们的Chef服务器和Rackspace云生态系统交互。以下是一个示例knife.rb文件，它从环境变量中获取所有所需信息。这样，您可以将此knife.rb文件添加到项目的.chef目录中，并将其安全地置于版本控制之下。

# Logging.
log_level                         :info
log_location                      STDOUT

# Chef server configuration.
chef_server_url                   "#{ENV['KNIFE_CHEF_SERVER']}"
client_key                        "#{ENV['KNIFE_CLIENT_KEY']}"
node_name                         "#{ENV['KNIFE_NODE_NAME']}"
validation_client_name            "#{ENV['KNIFE_VALIDATION_CLIENT_NAME']}"
validation_key                    "#{ENV['KNIFE_VALIDATION_CLIENT_KEY']}"
encrypted_data_bag_secret         "#{ENV['ENCRYPTED_DATA_BAG_SECRET_FILE']}"

# Rackspace API configuration.
knife[:rackspace_api_key]       = "#{ENV['RACKSPACE_API_KEY']}"
knife[:rackspace_api_username]  = "#{ENV['RACKSPACE_USERNAME']}"
knife[:rackspace_endpoint]      = "#{ENV['RACKSPACE_ENDPOINT']}"
knife[:rackspace_version]       = "#{ENV['RACKSPACE_VERSION']}"

接下来，我们需要一个 fabfile.py 文件。我们只需导入 mls.fabfile 使 fabric 命令可用，并从 propertyshelf.fabfile.common 中获取我们可以工作的可用环境。

# -*- coding: utf-8 -*-
"""Sample MLS deployment script."""

from fabric import api

from mls.fabfile import *
from propertyshelf.fabfile.common.environments import *


# Definition of role names to be used.
api.env.role_database = 'mls_db'
api.env.role_frontend = 'mls_fe'
api.env.role_staging = 'mls_staging'
api.env.role_worker = 'mls_app'

# Definition of used Rackspace flavors (server sized) for our servers.
api.env.flavor_database = '5'
api.env.flavor_frontend = '2'
api.env.flavor_staging = '2'
api.env.flavor_worker = '3'

# Definition of node names to be used.
api.env.nodename_database = 'mls-db'
api.env.nodename_frontend = 'mls-fe'
api.env.nodename_staging = 'mls-staging'
api.env.nodename_worker = 'mls-app'

# The Rackspace server image we use. This is a Debian 6.0.6.
api.env.os_image = '92a28e50-181d-4fc7-a071-567d26fc95f6'

# MLS specific configuration.
api.env.domain = 'mls-example.com'
api.env.mls_customizations = ['mlsext.realtorcom', ]
api.env.mls_policy_enabled = True
api.env.mls_policy_package = 'mlspolicy.example'
api.env.mls_policy_package_url = 'git https://github.com/yourname/mlspolicy.example'

现在您可以使用 fabric 来管理您的 MLS。

$ fab -l
Sample MLS deployment script.

Available commands:

    development                 Work locally with vagrant.
    production                  Work with the production environment.
    staging                     Work with the staging environment.
    bootstrap.bundle_db_fe_app  Bootstrap a new MLS bundle: Database, Frontend, App Worker.
    bootstrap.database          Bootstrap a new standalone database server.
    bootstrap.frontend          Bootstrap a new standalone frontend server.
    bootstrap.staging           Bootstrap a staging system.
    bootstrap.worker            Bootstrap a new standalone application worker.
    client.remove               Remove an existing MLS application client.
    client.restart              Restart the application client component.
    client.update               Update the client packages.
    database.backup             Perform a backup of Zope's data on the server.
    database.download_blobs     Download blob part of Zope's data from the server.
    database.download_data      Download the database files from the server.
    database.download_zodb      Download ZODB part of Zope's data from the server.
    database.restart            Restart the database component.
    database.restore            Restore an existing backup of Zope's data on the server.
    database.upload_blob        Upload blob part of Zope's data to the server.
    database.upload_data        Upload the database files to the server.
    database.upload_zodb        Upload ZODB part of Zope's data to the server.
    frontend.restart            Restart the frontend components.
    frontend.restart_haproxy    Restart the HA-Proxy load balancer component.
    frontend.restart_nginx      Restart the NginX web server component.
    frontend.restart_varnish    Restart the Varnish caching proxy component.
    roles.check                 Check if the required roles are available.
    roles.create_missing        Create missing roles on the chef server.

在我们开始之前，检查我们定义的所有角色是否在 chef 服务器上可用是个好主意。

$ fab roles.check
Role mls_fe NOT available.
Role mls_db NOT available.
Role mls_staging NOT available.
Role mls_app NOT available.

Done.

要创建基于我们配置的缺失角色，我们只需做以下操作：

$ fab roles.create_missing
Created role mls_db
Created role mls_fe
Created role mls_app
Created role mls_staging

Done.

现在我们可以创建我们的预发布系统。

$ fab bootstrap.staging
[localhost] local: knife rackspace server create -S mls-staging -N mls-staging -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_staging] -E staging

...

Done.

请注意，只能有一个预发布系统。如果您尝试添加另一个具有相同名称的系统，您将收到错误消息。

$ fab bootstrap.staging

Fatal error: Server "mls-staging" already exists in environment "staging".

Aborting.

如果您需要第二个，可以手动调整节点名称。

$ fab bootstrap.staging:nodename=mls-staging2
[localhost] local: knife rackspace server create -S mls-staging2 -N mls-staging2 -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_ni_staging] -E staging

...

Done.

现在您可以管理单个组件。

$ fab staging frontend.restart
[x.x.x.x] Executing task 'frontend.restart'
[x.x.x.x] sudo: /etc/init.d/haproxy restart
[x.x.x.x] out: sudo password:

[x.x.x.x] out: Restarting haproxy: haproxy.
[x.x.x.x] out:

[x.x.x.x] sudo: /etc/init.d/varnish restart
[x.x.x.x] out: sudo password:
[x.x.x.x] out: Stopping HTTP accelerator: varnishd.
[x.x.x.x] out: Starting HTTP accelerator: varnishd.
[x.x.x.x] out:

[x.x.x.x] sudo: /etc/init.d/nginx restart
[x.x.x.x] out: sudo password:
[x.x.x.x] out: Restarting nginx: nginx.
[x.x.x.x] out:


Done.
Disconnecting from x.x.x.x... done.

我们也支持下载数据库文件进行本地测试。

$ fab production database.download_data
[x.x.x.x] Executing task 'database.download_data'
This will overwrite your local Data.fs. Are you sure you want to continue? [Y/n]
[localhost] local: mkdir -p var/filestorage
[localhost] local: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak
[x.x.x.x] out: sudo password:
[x.x.x.x] sudo: rsync -a var/filestorage/Data.fs /tmp/Data.fs
[x.x.x.x] out: sudo password:
[x.x.x.x] out:
[x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/filestorage/Data.fs <- /tmp/Data.fs
This will overwrite your local blob files. Are you sure you want to continue? [Y/n]
[localhost] local: rm -rf var/blobstorage_bak
[localhost] local: mv var/blobstorage var/blobstorage_bak
[x.x.x.x] sudo: rsync -a ./var/blobstorage /tmp/
[x.x.x.x] out: sudo password:
[x.x.x.x] out:
[x.x.x.x] sudo: tar czf blobstorage.tgz blobstorage
[x.x.x.x] out: sudo password:
[x.x.x.x] out:
[x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz <- /tmp/blobstorage.tgz

Warning: Local file /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz already exists and is being overwritten.

[localhost] local: tar xzf blobstorage.tgz

Done.
Disconnecting from x.x.x.x... done.

一旦我们有了本地数据文件，我们就可以将它们上传到我们的开发环境（vagrant 虚拟机）。

$ fab development database.upload_data client.restart
[localhost] local: vagrant ssh-config | grep IdentityFile
[127.0.0.1:2222] Executing task 'database.upload_data'
This will overwrite your remote Data.fs. Are you sure you want to continue? [y/N] y
[127.0.0.1:2222] sudo: mkdir -p /tmp/upload
[127.0.0.1:2222] put: var/filestorage/Data.fs -> /tmp/upload/Data.fs
[127.0.0.1:2222] sudo: chown mls /tmp/upload/Data.fs
[127.0.0.1:2222] sudo: supervisorctl stop zeoserver
[127.0.0.1:2222] out: zeoserver: stopped
[127.0.0.1:2222] out:

[127.0.0.1:2222] sudo: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak
[127.0.0.1:2222] sudo: mv /tmp/upload/Data.fs var/filestorage/Data.fs
This will overwrite your remote blob files. Are you sure you want to continue? [y/N] y
[127.0.0.1:2222] sudo: mkdir -p /tmp/upload
[localhost] local: tar czf blobstorage_upload.tgz blobstorage
[127.0.0.1:2222] put: var/blobstorage_upload.tgz -> /tmp/upload/blobstorage.tgz
[127.0.0.1:2222] sudo: chown mls /tmp/upload/blobstorage.tgz
[127.0.0.1:2222] sudo: tar xzf blobstorage.tgz
[127.0.0.1:2222] sudo: supervisorctl stop zeoserver
[127.0.0.1:2222] out: zeoserver: ERROR (not running)
[127.0.0.1:2222] out:

[127.0.0.1:2222] sudo: mv var/blobstorage var/blobstorage_bak
[127.0.0.1:2222] sudo: mv /tmp/upload/blobstorage var
[127.0.0.1:2222] sudo: supervisorctl start zeoserver
[127.0.0.1:2222] out: zeoserver: started
[127.0.0.1:2222] out:

[127.0.0.1:2222] Executing task 'client.restart'
[127.0.0.1:2222] sudo: supervisorctl restart application
[127.0.0.1:2222] out: application: stopped
[127.0.0.1:2222] out: application: started
[127.0.0.1:2222] out:


Done.
Disconnecting from 127.0.0.1:2222... done.

我们还可以获取已定义角色的节点列表。

$ fab roles.list_nodes
Role: mls_fe
- mls-fe: x.x.x.x

Role: mls_db
- mls-db: x.x.x.x

Role: mls_staging
- mls-staging: x.x.x.x
- vagrant-mls-staging: 10.0.2.15

Role: mls_app
- mls-app-01: x.x.x.x


Done.

如果您只想为特定的节点执行任务，这可能很有用。

$ fab frontend.restart_nginx:hosts=x.x.x.x
[x.x.x.x] Executing task 'frontend.restart_nginx'
[x.x.x.x] sudo: /etc/init.d/nginx restart
[x.x.x.x] out: sudo password:
[x.x.x.x] out: Restarting nginx: nginx.
[x.x.x.x] out:


Done.
Disconnecting from x.x.x.x... done.

0.4 (2014-04-10)

• 为数据库角色添加了 snapshotbackup 选项。

• 现在可以更新维护客户端。

0.3 (2013-12-21)

• 将公共功能移动到 propertyshelf.fabfile.common。

• 新任务：重建客户端，列出角色的节点。

0.2 (2013-11-19)

• 为开发环境设置自定义主机名，以便 chef 搜索正常工作。

• 启用了数据上传（Data.fs 和 blobs）。

0.1.1 (2013-11-19)

• 添加了缺失的文件。

0.1 (2013-11-19)

• 与开发、预发布和生产环境一起工作。

• 检查 chef 角色，并从配置中创建缺失的角色。

• 引导 MLS 系统。

• 如果不再需要，则删除 MLS 客户端。

• 管理数据库、前端和工作组件。

• 更新客户端构建包。

• 下载数据库文件进行本地测试。