openstack之novaclient详解

int32位 posted @ Oct 08, 2014 03:47:28 PM in openstack , 4680 阅读
转载请注明:http://krystism.is-programmer.com/若有错误,请多多指正,谢谢!

要想知道nova的工作过程,首先就要掌握它的入口,即novaclient!命令nova和horizon都调用了novaclient。

github地址:https://github.com/openstack/python-novaclient

novaclient的功能很简单,即解析参数,构造url并发送请求,处理结果。比如运行nova --debug list,首先需要解析出选项参数--debug,另外还要获取环境变量参数和默认参数,然后解析子命令list,通过子命令获取相对应的回调函数,list对应为novaclient.v1_1.shell.do_list。

下面详细看看它的工作原理,首先看看命令nova到底是什么?

which nova | xargs -I{} file {}
# 返回/usr/bin/nova: a /usr/bin/python script, ASCII text executable

可见命令nova只是一个python程序,让我们打开它

#!/usr/bin/python
# PBR Generated from 'console_scripts'

import sys

from novaclient.shell import main


if __name__ == "__main__":
    sys.exit(main())

命令nova调用了novaclient.shell的main函数,从这里开始进入了novaclient,现在让我们开始novaclient吧!

首先看看novaclient.shell的main函数:

def main():
    """入口函数"""
    try:
        OpenStackComputeShell().main(map(strutils.safe_decode, sys.argv[1:]))
    except Exception as e:
        logger.debug(e, exc_info=1)
        print("ERROR: %s" % strutils.safe_encode(six.text_type(e)),
              file=sys.stderr)
        sys.exit(1)

发现它又调用了OpenstackComputeShell()的main函数。这个main函数才是真正的入口函数,以下是前半部分代码:

 def main(self, argv):

        # Parse args once to find version and debug settings
        parser = self.get_base_parser() # 添加选项,比如--user, --password等
        (options, args) = parser.parse_known_args(argv)
        self.setup_debugging(options.debug) # 如果options中有--debug,则设置logger的level为DEBUG,并输出到标准输出流

        # Discover available auth plugins
        novaclient.auth_plugin.discover_auth_systems()

        # build available subcommands based on version
        self.extensions = self._discover_extensions(
                options.os_compute_api_version)
        self._run_extension_hooks('__pre_parse_args__')

        # NOTE(dtroyer): Hackery to handle --endpoint_type due to argparse
        #                thinking usage-list --end is ambiguous; but it
        #                works fine with only --endpoint-type present
        #                Go figure.
        if '--endpoint_type' in argv:
            spot = argv.index('--endpoint_type')
            argv[spot] = '--endpoint-type'

	# 根据版本解析子命令
        subcommand_parser = self.get_subcommand_parser(
                options.os_compute_api_version)
        self.parser = subcommand_parser

	# 如果--help,则打印help信息,并退出
        if options.help or not argv:
            subcommand_parser.print_help()
            return 0

        args = subcommand_parser.parse_args(argv) #解析命令行参数 argv=['list']
	#print("args = %s" % args)
        self._run_extension_hooks('__post_parse_args__', args)

        # Short-circuit and deal with help right away.
	# nova help xxxx 命令
        if args.func == self.do_help:
            self.do_help(args)
            return 0
    	# nova bash-completion
        elif args.func == self.do_bash_completion:
            self.do_bash_completion(args)
            return 0

parser是 NovaClientArgumentParser类型,该类型继承自argparse.ArgumentParser,argparse是python中的参数解析库。

get_base_parser方法即添加选项参数,诸如--debug, --timing,--os-username 等等,并会读取环境变量和设置默认值,下面是部分代码:

 # Global arguments
        parser.add_argument('-h', '--help',
            action='store_true',
            help=argparse.SUPPRESS,
        )

        parser.add_argument('--version',
                            action='version',
                            version=novaclient.__version__)

        parser.add_argument('--debug',
            default=False,
            action='store_true',
            help="Print debugging output")

        parser.add_argument('--no-cache',
            default=not utils.bool_from_str(
                    utils.env('OS_NO_CACHE', default='true')),
            action='store_false',
            dest='os_cache',
            help=argparse.SUPPRESS)
        parser.add_argument('--no_cache',
            action='store_false',
            dest='os_cache',
            help=argparse.SUPPRESS)

        parser.add_argument('--os-cache',
            default=utils.env('OS_CACHE', default=False),
            action='store_true',
            help="Use the auth token cache.")

        parser.add_argument('--timings',
            default=False,
            action='store_true',
            help="Print call timing info")

        parser.add_argument('--timeout',
            default=600,
            metavar='<seconds>',
            type=positive_non_zero_float,
            help="Set HTTP call timeout (in seconds)")

用过argparse库的一定不会陌生了。

回到main函数,接下来会设置debug,即如果有--debug选项,则设置logger的level为DEBUG并传入到标准输出流。

(options, args) = parser.parse_known_args(argv)返回解析结果,即options保存所有的选项参数,args保存位置参数,比如nova --debug list, options.debug等于True,args为['list']。

下一个函数get_subcommand_parser是一个核心方法,用于处理子命令比如list, flavor-list, boot等,以下是代码:

 def get_subcommand_parser(self, version):
        parser = self.get_base_parser()

        self.subcommands = {}
        subparsers = parser.add_subparsers(metavar='<subcommand>')

        try:
            actions_module = {
                '1.1': shell_v1_1,
                '2': shell_v1_1,
                '3': shell_v3,
            }[version]
        except KeyError:
            actions_module = shell_v1_1 #默认是1.1版本

        self._find_actions(subparsers, actions_module)
        self._find_actions(subparsers, self)

        for extension in self.extensions:
            self._find_actions(subparsers, extension.module)

        self._add_bash_completion_subparser(subparsers)

        return parser

这个方法是根据版本(默认是1.1)寻找可用的方法,我们假设使用shell_v1_1模块,它导入自from novaclient.v1_1 import shell as shell_v1_1,然后调用_find_actions方法。注意:这个方法传入的是一个模块,python中所有东西都是对象,模块也不例外,不过这里我们姑且认为它传入了一个类,类似与java的XXXClass.class类型,以下是代码:

def _find_actions(self, subparsers, actions_module):
	# actions_module = shell_v1.1
        for attr in (a for a in dir(actions_module) if a.startswith('do_')): # attr = do_flavor_list
            # I prefer to be hypen-separated instead of underscores.
            command = attr[3:].replace('_', '-') # do_flavor_list -> flavor-list
            callback = getattr(actions_module, attr)
            desc = callback.__doc__ or ''
            action_help = desc.strip()
            arguments = getattr(callback, 'arguments', [])

            subparser = subparsers.add_parser(command,
                help=action_help,
                description=desc,
                add_help=False,
                formatter_class=OpenStackHelpFormatter
            )
            subparser.add_argument('-h', '--help',
                action='help',
                help=argparse.SUPPRESS,
            )
            self.subcommands[command] = subparser
            for (args, kwargs) in arguments:
                subparser.add_argument(*args, **kwargs)
            subparser.set_defaults(func=callback)

可见这个方法是利用反射机制获取所有以do_开头的方法,这个do_XXX_XXX,XXX-XXX就是命令名,而do_XXX_XXX就是回调函数,把函数作为变量赋值给callback,是函数式编程的经典用法。最后把callback传入set_defaults方法。

至此我们知道nova list其实调用了novaclient.v1_1.shell.do_list()方法,而nova flavor-list调用了novaclient.v1_1.shell.do_flavor_list()方法,下面以nova --debug flavor-list为例继续深入。

我们看novaclient.v1_1.shell源码,发现好多do_XXX方法,但它本身并不做什么工作,而是调用cs去做,cs是什么现在不管。下面是do_flavor_list方法:

def do_flavor_list(cs, args):
    """Print a list of available 'flavors' (sizes of servers)."""
    if args.all:
        flavors = cs.flavors.list(is_public=None)
    else:
        flavors = cs.flavors.list()
    _print_flavor_list(flavors, args.extra_specs)

现在我们不知道cs是什么东西,那我们继续回到main函数,main函数中间其余代码均是在各种参数检查,我们忽略不管,直接跳到main函数结尾

 def main(self, argv):

        # Parse args once to find version and debug settings
        parser = self.get_base_parser() # 添加选项,比如--user, --password等
        (options, args) = parser.parse_known_args(argv)
        self.setup_debugging(options.debug) # 如果options中有--debug,则设置logger的level为DEBUG,并输出到标准输出流

        # Discover available auth plugins
        novaclient.auth_plugin.discover_auth_systems()

        # build available subcommands based on version
        self.extensions = self._discover_extensions(
                options.os_compute_api_version)
        self._run_extension_hooks('__pre_parse_args__')

        # NOTE(dtroyer): Hackery to handle --endpoint_type due to argparse
        #                thinking usage-list --end is ambiguous; but it
        #                works fine with only --endpoint-type present
        #                Go figure.
        if '--endpoint_type' in argv:
            spot = argv.index('--endpoint_type')
            argv[spot] = '--endpoint-type'

	# 根据版本解析子命令
        subcommand_parser = self.get_subcommand_parser(
                options.os_compute_api_version)
        self.parser = subcommand_parser

	# 如果--help,则打印help信息,并退出
        if options.help or not argv:
            subcommand_parser.print_help()
            return 0

        args = subcommand_parser.parse_args(argv) #解析命令行参数 argv=['list']
	#print("args = %s" % args)
        self._run_extension_hooks('__post_parse_args__', args)

        # Short-circuit and deal with help right away.
	# nova help xxxx 命令
        if args.func == self.do_help:
            self.do_help(args)
            return 0
    	# nova bash-completion
        elif args.func == self.do_bash_completion:
            self.do_bash_completion(args)
            return 0
        # 这里省略大量代码
        self.cs = client.Client(options.os_compute_api_version, os_username,
                os_password, os_tenant_name, tenant_id=os_tenant_id,
                auth_url=os_auth_url, insecure=insecure,
                region_name=os_region_name, endpoint_type=endpoint_type,
                extensions=self.extensions, service_type=service_type,
                service_name=service_name, auth_system=os_auth_system,
                auth_plugin=auth_plugin,
                volume_service_name=volume_service_name,
                timings=args.timings, bypass_url=bypass_url,
                os_cache=os_cache, http_log_debug=options.debug,
                cacert=cacert, timeout=timeout)

        # 这里省略大量代码
        args.func(self.cs, args) # 此时func等于do_flavor_list

        if args.timings: #如果有--timing选项,则打印请求时间
            self._dump_timings(self.cs.get_timings())

可见cs是调用client.Client方法返回的,我们查看其代码client.py:

def get_client_class(version):
    version_map = {
        '1.1': 'novaclient.v1_1.client.Client',
        '2': 'novaclient.v1_1.client.Client',
        '3': 'novaclient.v3.client.Client',
    }
    try:
        client_path = version_map[str(version)]
    except (KeyError, ValueError):
        msg = "Invalid client version '%s'. must be one of: %s" % (
              (version, ', '.join(version_map.keys())))
        raise exceptions.UnsupportedVersion(msg)

    return utils.import_class(client_path)


def Client(version, *args, **kwargs):
    client_class = get_client_class(version)
    return client_class(*args, **kwargs)

不难看出cs即根据版本选择的Client类型,这里我们用的是novaclient.v1_1.client.Client。这个模块可以认为是功能模块的注册类,比如flavors操作模块为flavors.py,为了让他生效,必须注册,即在Client中设置self.flavors=flavors.FlavorManager(self):

 self.projectid = project_id
        self.tenant_id = tenant_id
        self.flavors = flavors.FlavorManager(self)
        self.flavor_access = flavor_access.FlavorAccessManager(self)
        self.images = images.ImageManager(self)
        self.limits = limits.LimitsManager(self)
        self.servers = servers.ServerManager(self)

从do_flavor_list方法中cs.flavor.list()即调用了flavors.FlavorManager().list方法。从这里我们可以看出openstack的设计原则,即支持自由灵活的可扩展性,如果需要添加新功能,几乎不需要修改太多代码,只要修改Client注册即可。

我们查看flavors.py中的list方法:

 def list(self, detailed=True, is_public=True):
        """
        Get a list of all flavors.

        :rtype: list of :class:`Flavor`.
        """
        qparams = {}
        # is_public is ternary - None means give all flavors.
        # By default Nova assumes True and gives admins public flavors
        # and flavors from their own projects only.
        if not is_public:
            qparams['is_public'] = is_public
        query_string = "?%s" % urlutils.urlencode(qparams) if qparams else ""

        detail = ""
        if detailed:
            detail = "/detail"

        return self._list("/flavors%s%s" % (detail, query_string), "flavors")

很明显这里是把list命令组装url请求,然后调用_list方法,由于FlavorManager继承自base.ManagerWithFind,而base.ManagerWithFind继承自Manager,_list方法在Manager中定义。

def _list(self, url, response_key, obj_class=None, body=None):
        if body:
            _resp, body = self.api.client.post(url, body=body)
        else:
            _resp, body = self.api.client.get(url)

        if obj_class is None:
            obj_class = self.resource_class

        data = body[response_key]
        # NOTE(ja): keystone returns values as list as {'values': [ ... ]}
        #           unlike other services which just return the list...
        if isinstance(data, dict):
            try:
                data = data['values']
            except KeyError:
                pass

        with self.completion_cache('human_id', obj_class, mode="w"):
            with self.completion_cache('uuid', obj_class, mode="w"):
                return [obj_class(self, res, loaded=True)
                        for res in data if res]

有源码中看出主要发送url请求即self.api.client.post(url, body=dody)或者self.api.client.get(url),具体根据是否有body,即是否数据选择GET或者POST请求。然后处理返回的数据。self.api在这里其实就是novaclient.v1_1.client.Client,只是前面用cs,这里用api。

我们回到novaclient.v1_1.client.Client的方法中,我们发现除了注册一系列功能外,还有一个比较特殊的,

 self.client = client.HTTPClient(username,
                                    password,
                                    projectid=project_id,
                                    tenant_id=tenant_id,
                                    auth_url=auth_url,
                                    insecure=insecure,
                                    timeout=timeout,
                                    auth_system=auth_system,
                                    auth_plugin=auth_plugin,
                                    proxy_token=proxy_token,
                                    proxy_tenant_id=proxy_tenant_id,
                                    region_name=region_name,
                                    endpoint_type=endpoint_type,
                                    service_type=service_type,
                                    service_name=service_name,
                                    volume_service_name=volume_service_name,
                                    timings=timings,
                                    bypass_url=bypass_url,
                                    os_cache=self.os_cache,
                                    http_log_debug=http_log_debug,
                                    cacert=cacert)

这个self.client是client.HTTPClient类型,真正负责发送url请求的类,部分代码为:

def _cs_request(self, url, method, **kwargs):
        if not self.management_url:
            self.authenticate()

        # Perform the request once. If we get a 401 back then it
        # might be because the auth token expired, so try to
        # re-authenticate and try again. If it still fails, bail.
        try:
            kwargs.setdefault('headers', {})['X-Auth-Token'] = self.auth_token
            if self.projectid:
                kwargs['headers']['X-Auth-Project-Id'] = self.projectid

            resp, body = self._time_request(self.management_url + url, method,
                                            **kwargs)
            return resp, body
        except exceptions.Unauthorized as e:
            try:
                # frist discard auth token, to avoid the possibly expired
                # token being re-used in the re-authentication attempt
                self.unauthenticate()
                self.authenticate()
                kwargs['headers']['X-Auth-Token'] = self.auth_token
                resp, body = self._time_request(self.management_url + url,
                                                method, **kwargs)
                return resp, body
            except exceptions.Unauthorized:
                raise e

    def get(self, url, **kwargs):
        return self._cs_request(url, 'GET', **kwargs)

最后会调用http.request方法发送请求,这里使用了python库Requests: HTTP for Humans,这个库比httplib2更好,查看地址:http://docs.python-requests.org/en/latest/。接收请求的工作就由nova-api负责了,这里不再深入。

接下来我们简单增加一个没用的功能test,首先在novaclient/v1_1下touch test.py,使用vim增加以下代码:

"""
Test interface.
"""

from novaclient import base

class Test(base.Resource):
	def test(self):
		print("This is a test")
class TestManager(base.Manager):
	def test(self):
		print("This is a test")

然后我们需要在client中注册,编辑novaclient/v1_1/client.py文件,增加self.test = test.TestManager(self)

然后在shell下增加入口函数,注册新功能,

def do_test(cs, _args):
	""" do test. """
	cs.test.test()

运行nova test, nova help test查看效果。

转载请注明:http://krystism.is-programmer.com/若有错误,请多多指正,谢谢!
  • 无匹配
  • 无匹配
Digital Ali 说:
2021年9月07日 16:51 Wow, excellent post. I'd like to draft like this too - taking time and real hard work to make a great article. This post has encouraged me to write some posts that I am going to write soon. 123movies official
Emma 说:
2023年1月24日 20:10

If you want to get a better understanding of how Nova works, you must become familiar with its entrance, novaclient! Both the commands nova and horizon use novaclient for their functions. The Python-Novaclient Github page has detailed information about real estate services Central East Austin how novaclient works. It has been programmed to parse out the command parameter, construct the URL, and send a request. For example, when running the command nova --debug list, the novaclient parses the option parameter --debug and gets the environment variable parameters and default parameters. Additionally, it parses the subcommand list and obtains the corresponding callback function through the subcommand.


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter