Ansible 扩展 | 珊瑚贝

Ansible 简介

Ansible 是由 Python 开发的一个运维工具,因为工作需要接触到 Ansible,经常会集成一些东西到 Ansible,所以对 Ansible 的了解越来越多。 那 Ansible 到底是什么呢?在我的理解中,原来需要登录到服务器上,然后执行一堆命令才能完成一些操作。而 Ansible 就是来代替我们去执行那些命令。并且可以通过 Ansible 控制多台机器,在机器上进行任务的编排和执行,在 Ansible 中称为 playbook。 那 Ansible 是如何做到的呢?简单点说,就是 Ansible 将我们要执行的命令生成一个脚本,然后通过 sftp 将脚本上传到要执行命令的服务器上,然后在通过 ssh 协议,执行这个脚本并将执行结果返回。 那 Ansible 具体是怎么做到的呢?下面从模块和插件来看一下 Ansible 是如何完成一个模块的执行 PS:下面的分析都是在对 Ansible 有一些具体使用经验之后,通过阅读源代码进一步得出的执行结论,所以希望在看本文时,是建立在对 Ansible 有一定了解的基础上,最起码对于 Ansible 的一些概念有了解,例如 inventory,module,playbooks 等

Ansible 模块

模块是 Ansible 执行的最小单位,可以是由 Python 编写,也可以是 Shell 编写,也可以是由其他语言编写。模块中定义了具体的操作步骤以及实际使用过程中所需要的参数 执行的脚本就是根据模块生成一个可执行的脚本。 那 Ansible 是怎么样将这个脚本上传到服务器上,然后执行获取结果的呢?

Ansible 插件

connection 插件

连接插件,根据指定的 ssh 参数连接指定的服务器,并切提供实际执行命令的接口

shell 插件

命令插件,根据 sh 类型,来生成用于 connection 时要执行的命令

strategy 插件

执行策略插件,默认情况下是线性插件,就是一个任务接着一个任务的向下执行,此插件将任务丢到执行器去执行。

action 插件

动作插件,实质就是任务模块的所有动作,如果 ansible 的模块没有特别编写的 action 插件,默认情况下是 normal 或者 async(这两个根据模块是否 async 来选择),normal 和 async 中定义的就是模块的执行步骤。例如,本地创建临时文件,上传临时文件,执行脚本,删除脚本等等,如果想在所有的模块中增加一些特殊步骤,可以通过增加 action 插件的方式来扩展。

Ansible 执行模块流程

  1. ansible 命令实质是通过 ansible/cli/adhoc.py 来运行,同时会收集参数信息
    1. 设置 Play 信息,然后通过 TaskQueueManager 进行 run,
    2. TaskQueueManager 需要 Inventory (节点仓库),variable_manager (收集变量),options (命令行中指定的参数),stdout_callback (回调函数)
  2. 在 task_queue_manager.py 中找到 run 中
    1. 初始化时会设置队列
    2. 会根据 options,,variable_manager,passwords 等信息设置成一个 PlayContext 信息 (playbooks/playcontext.py)
    3. 设置插件 (plugins) 信息 callback_loader (回调), strategy_loader (执行策略), module_loader (任务模块)
    4. 通过 strategy_loader(strategy 插件)的 run(默认的 strategy 类型是 linear,线性执行),去按照顺序执行所有的任务(执行一个模块,可能会执行多个任务)
    5. 在 strategy_loader 插件 run 之后,会判断 action 类型。如果是 meta 类型的话会单独执行 (不是具体的 ansible 模块时),而其他模块时,会加载到队列_queue_task
    6. 在队列中会调用 WorkerProcess 去处理,在 workerproces 实际的 run 之后,会使用 TaskExecutor 进行执行
    7. 在 TaskExecutor 中会设置 connection 插件,并且根据 task 的类型(模块。或是 include 等)获取 action 插件,就是对应的模块,如果模块有自定义的执行,则会执行自定义的 action,如果没有的会使用 normal 或者 async,这个是根据是否是任务的 async 属性来决定
    8. 在 Action 插件中定义着执行的顺序,及具体操作,例如生成临时目录,生成临时脚本,所以要在统一的模式下,集成一些额外的处理时,可以重写 Action 的方法
    9. 通过 Connection 插件来执行 Action 的各个操作步骤

扩展 Ansible 实例

执行节点 Python 环境扩展

实际需求中,我们扩展的一些 Ansible 模块需要使用三方库,但每个节点中安装这些库有些不易于管理。ansible 执行模块的实质就是在节点的 python 环境下执行生成的脚本,所以我们采取的方案是,指定节点上的 Python 环境,将局域网内一个 python 环境作为 nfs 共享。通过扩展 Action 插件,增加节点上挂载 nfs,待执行结束后再将节点上的 nfs 卸载。具体实施步骤如下: 扩展代码:

重写 ActionBase 的 execute_module 方法

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
# execute_module

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

import json
import pipes

from ansible.compat.six import text_type, iteritems

from ansible import constants as C
from ansible.errors import AnsibleError
from ansible.release import __version__

try:
from __main__ import display
except ImportError:
from ansible.utils.display import Display
display = Display()


class MagicStackBase(object):

def _mount_nfs(self, ansible_nfs_src, ansible_nfs_dest):
cmd = ['mount',ansible_nfs_src, ansible_nfs_dest]
cmd = [pipes.quote(c) for c in cmd]
cmd = ' '.join(cmd)
result = self._low_level_execute_command(cmd=cmd, sudoable=True)
return result

def _umount_nfs(self, ansible_nfs_dest):
cmd = ['umount', ansible_nfs_dest]
cmd = [pipes.quote(c) for c in cmd]
cmd = ' '.join(cmd)
result = self._low_level_execute_command(cmd=cmd, sudoable=True)
return result

def _execute_module(self, module_name=None, module_args=None, tmp=None, task_vars=None, persist_files=False, delete_remote_tmp=True):
'''
Transfer and run a module along with its arguments.
'''

# display.v(task_vars)

if task_vars is None:
task_vars = dict()

# if a module name was not specified for this execution, use
# the action from the task
if module_name is None:
module_name = self._task.action
if module_args is None:
module_args = self._task.args

# set check mode in the module arguments, if required
if self._play_context.check_mode:
if not self._supports_check_mode:
raise AnsibleError("check mode is not supported for this operation")
module_args['_ansible_check_mode'] = True
else:
module_args['_ansible_check_mode'] = False

# Get the connection user for permission checks
remote_user = task_vars.get('ansible_ssh_user') or self._play_context.remote_user

# set no log in the module arguments, if required
module_args['_ansible_no_log'] = self._play_context.no_log or C.DEFAULT_NO_TARGET_SYSLOG

# set debug in the module arguments, if required
module_args['_ansible_debug'] = C.DEFAULT_DEBUG

# let module know we are in diff mode
module_args['_ansible_diff'] = self._play_context.diff

# let module know our verbosity
module_args['_ansible_verbosity'] = display.verbosity

# give the module information about the ansible version
module_args['_ansible_version'] = __version__

# set the syslog facility to be used in the module
module_args['_ansible_syslog_facility'] = task_vars.get('ansible_syslog_facility', C.DEFAULT_SYSLOG_FACILITY)

# let module know about filesystems that selinux treats specially
module_args['_ansible_selinux_special_fs'] = C.DEFAULT_SELINUX_SPECIAL_FS

(module_style, shebang, module_data) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)
if not shebang:
raise AnsibleError("module (%s) is missing interpreter line" % module_name)

# get nfs info for mount python packages
ansible_nfs_src = task_vars.get("ansible_nfs_src", None)
ansible_nfs_dest = task_vars.get("ansible_nfs_dest", None)

# a remote tmp path may be necessary and not already created
remote_module_path = None
args_file_path = None
if not tmp and self._late_needs_tmp_path(tmp, module_style):
tmp = self._make_tmp_path(remote_user)

if tmp:
remote_module_filename = self._connection._shell.get_remote_filename(module_name)
remote_module_path = self._connection._shell.join_path(tmp, remote_module_filename)
if module_style in ['old', 'non_native_want_json']:
# we'll also need a temp file to hold our module arguments
args_file_path = self._connection._shell.join_path(tmp, 'args')

if remote_module_path or module_style != 'new':
display.debug("transferring module to remote")
self._transfer_data(remote_module_path, module_data)
if module_style == 'old':
# we need to dump the module args to a k=v string in a file on
# the remote system, which can be read and parsed by the module
args_data = ""
for k,v in iteritems(module_args):
args_data += '%s=%s ' % (k, pipes.quote(text_type(v)))
self._transfer_data(args_file_path, args_data)
elif module_style == 'non_native_want_json':
self._transfer_data(args_file_path, json.dumps(module_args))
display.debug("done transferring module to remote")

environment_string = self._compute_environment_string()

remote_files = None

if args_file_path:
remote_files = tmp, remote_module_path, args_file_path
elif remote_module_path:
remote_files = tmp, remote_module_path

# Fix permissions of the tmp path and tmp files. This should be
# called after all files have been transferred.
if remote_files:
self._fixup_perms2(remote_files, remote_user)


# mount nfs
if ansible_nfs_src and ansible_nfs_dest:
result = self._mount_nfs(ansible_nfs_src, ansible_nfs_dest)
if result['rc'] != 0:
raise AnsibleError("mount nfs failed!!! {0}".format(result['stderr']))

cmd = ""
in_data = None

if self._connection.has_pipelining and self._play_context.pipelining and not C.DEFAULT_KEEP_REMOTE_FILES and module_style == 'new':
in_data = module_data
else:
if remote_module_path:
cmd = remote_module_path

rm_tmp = None
if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp:
if not self._play_context.become or self._play_context.become_user == 'root':
# not sudoing or sudoing to root, so can cleanup files in the same step
rm_tmp = tmp

cmd = self._connection._shell.build_module_command(environment_string, shebang, cmd, arg_path=args_file_path, rm_tmp=rm_tmp)
cmd = cmd.strip()
sudoable = True
if module_name == "accelerate":
# always run the accelerate module as the user
# specified in the play, not the sudo_user
sudoable = False


res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)

# umount nfs
if ansible_nfs_src and ansible_nfs_dest:
result = self._umount_nfs(ansible_nfs_dest)
if result['rc'] != 0:
raise AnsibleError("umount nfs failed!!! {0}".format(result['stderr']))

if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp:
if self._play_context.become and self._play_context.become_user != 'root':
# not sudoing to root, so maybe can't delete files as that other user
# have to clean up temp files as original user in a second step
tmp_rm_cmd = self._connection._shell.remove(tmp, recurse=True)
tmp_rm_res = self._low_level_execute_command(tmp_rm_cmd, sudoable=False)
tmp_rm_data = self._parse_returned_data(tmp_rm_res)
if tmp_rm_data.get('rc', 0) != 0:
display.warning('Error deleting remote temporary files (rc: {0}, stderr: {1})'.format(tmp_rm_res.get('rc'), tmp_rm_res.get('stderr', 'No error string available.')))

# parse the main result
data = self._parse_returned_data(res)

# pre-split stdout into lines, if stdout is in the data and there
# isn't already a stdout_lines value there
if 'stdout' in data and 'stdout_lines' not in data:
data['stdout_lines'] = data.get('stdout', u'').splitlines()

display.debug("done with _execute_module (%s, %s)" % (module_name, module_args))
return data

集成到 normal.py 和 async.py 中,记住要将这两个插件在 ansible.cfg 中进行配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

from ansible.plugins.action import ActionBase
from ansible.utils.vars import merge_hash

from common.ansible_plugins import MagicStackBase


class ActionModule(MagicStackBase, ActionBase):

def run(self, tmp=None, task_vars=None):
if task_vars is None:
task_vars = dict()

results = super(ActionModule, self).run(tmp, task_vars)
# remove as modules might hide due to nolog
del results['invocation']['module_args']
results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
# Remove special fields from the result, which can only be set
# internally by the executor engine. We do this only here in
# the 'normal' action, as other action plugins may set this.
#
# We don't want modules to determine that running the module fires
# notify handlers. That's for the playbook to decide.
for field in ('_ansible_notify',):
if field in results:
results.pop(field)

return results
  • 配置 ansible.cfg,将扩展的插件指定为 ansible 需要的 action 插件
  • 重写插件方法,重点是 execute_module
  • 执行命令中需要指定 Python 环境,将需要的参数添加进去 nfs 挂载和卸载的参数
1
ansible 51 -m mysql_db -a "state=dump name=all target=/tmp/test.sql" -i hosts -u root -v -e "ansible_nfs_src=172.16.30.170:/web/proxy_env/lib64/python2.7/site-packages ansible_nfs_dest=/root/.pyenv/versions/2.7.10/lib/python2.7/site-packages ansible_python_interpreter=/root/.pyenv/versions/2.7.10/bin/python"

来源:https://cuiqingcai.com/4989.html

微信公众号
手机浏览(小程序)

Warning: get_headers(): SSL operation failed with code 1. OpenSSL Error messages: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed in /mydata/web/wwwshanhubei/web/wp-content/themes/shanhuke/single.php on line 57

Warning: get_headers(): Failed to enable crypto in /mydata/web/wwwshanhubei/web/wp-content/themes/shanhuke/single.php on line 57

Warning: get_headers(https://static.shanhubei.com/qrcode/qrcode_viewid_11334.jpg): failed to open stream: operation failed in /mydata/web/wwwshanhubei/web/wp-content/themes/shanhuke/single.php on line 57
0
分享到:
没有账号? 忘记密码?