要想知道nova的工作过程,首先就要掌握它的入口,即novaclient!命令nova和horizon都调用了novaclient。
github地址:https://github.com/openstack/python-novaclient
novaclient的功能很简单,即解析参数,构造url并发送请求,处理结果。比如运行nova --debug list,首先需要解析出选项参数--debug,另外还要获取环境变量参数和默认参数,然后解析子命令list,通过子命令获取相对应的回调函数,list对应为novaclient.v1_1.shell.do_list。
下面详细看看它的工作原理,首先看看命令nova到底是什么?
which nova | xargs -I{} file {} # 返回/usr/bin/nova: a /usr/bin/python script, ASCII text executable
可见命令nova只是一个python程序,让我们打开它
#!/usr/bin/python # PBR Generated from 'console_scripts' import sys from novaclient.shell import main if __name__ == "__main__": sys.exit(main())
命令nova调用了novaclient.shell的main函数,从这里开始进入了novaclient,现在让我们开始novaclient吧!
首先看看novaclient.shell的main函数:
def main(): """入口函数""" try: OpenStackComputeShell().main(map(strutils.safe_decode, sys.argv[1:])) except Exception as e: logger.debug(e, exc_info=1) print("ERROR: %s" % strutils.safe_encode(six.text_type(e)), file=sys.stderr) sys.exit(1)
发现它又调用了OpenstackComputeShell()的main函数。这个main函数才是真正的入口函数,以下是前半部分代码:
def main(self, argv): # Parse args once to find version and debug settings parser = self.get_base_parser() # 添加选项,比如--user, --password等 (options, args) = parser.parse_known_args(argv) self.setup_debugging(options.debug) # 如果options中有--debug,则设置logger的level为DEBUG,并输出到标准输出流 # Discover available auth plugins novaclient.auth_plugin.discover_auth_systems() # build available subcommands based on version self.extensions = self._discover_extensions( options.os_compute_api_version) self._run_extension_hooks('__pre_parse_args__') # NOTE(dtroyer): Hackery to handle --endpoint_type due to argparse # thinking usage-list --end is ambiguous; but it # works fine with only --endpoint-type present # Go figure. if '--endpoint_type' in argv: spot = argv.index('--endpoint_type') argv[spot] = '--endpoint-type' # 根据版本解析子命令 subcommand_parser = self.get_subcommand_parser( options.os_compute_api_version) self.parser = subcommand_parser # 如果--help,则打印help信息,并退出 if options.help or not argv: subcommand_parser.print_help() return 0 args = subcommand_parser.parse_args(argv) #解析命令行参数 argv=['list'] #print("args = %s" % args) self._run_extension_hooks('__post_parse_args__', args) # Short-circuit and deal with help right away. # nova help xxxx 命令 if args.func == self.do_help: self.do_help(args) return 0 # nova bash-completion elif args.func == self.do_bash_completion: self.do_bash_completion(args) return 0
parser是 NovaClientArgumentParser类型,该类型继承自argparse.ArgumentParser,argparse是python中的参数解析库。
get_base_parser方法即添加选项参数,诸如--debug, --timing,--os-username 等等,并会读取环境变量和设置默认值,下面是部分代码:
# Global arguments parser.add_argument('-h', '--help', action='store_true', help=argparse.SUPPRESS, ) parser.add_argument('--version', action='version', version=novaclient.__version__) parser.add_argument('--debug', default=False, action='store_true', help="Print debugging output") parser.add_argument('--no-cache', default=not utils.bool_from_str( utils.env('OS_NO_CACHE', default='true')), action='store_false', dest='os_cache', help=argparse.SUPPRESS) parser.add_argument('--no_cache', action='store_false', dest='os_cache', help=argparse.SUPPRESS) parser.add_argument('--os-cache', default=utils.env('OS_CACHE', default=False), action='store_true', help="Use the auth token cache.") parser.add_argument('--timings', default=False, action='store_true', help="Print call timing info") parser.add_argument('--timeout', default=600, metavar='<seconds>', type=positive_non_zero_float, help="Set HTTP call timeout (in seconds)")
用过argparse库的一定不会陌生了。
回到main函数,接下来会设置debug,即如果有--debug选项,则设置logger的level为DEBUG并传入到标准输出流。
(options, args) = parser.parse_known_args(argv)返回解析结果,即options保存所有的选项参数,args保存位置参数,比如nova --debug list, options.debug等于True,args为['list']。
下一个函数get_subcommand_parser是一个核心方法,用于处理子命令比如list, flavor-list, boot等,以下是代码:
def get_subcommand_parser(self, version): parser = self.get_base_parser() self.subcommands = {} subparsers = parser.add_subparsers(metavar='<subcommand>') try: actions_module = { '1.1': shell_v1_1, '2': shell_v1_1, '3': shell_v3, }[version] except KeyError: actions_module = shell_v1_1 #默认是1.1版本 self._find_actions(subparsers, actions_module) self._find_actions(subparsers, self) for extension in self.extensions: self._find_actions(subparsers, extension.module) self._add_bash_completion_subparser(subparsers) return parser
这个方法是根据版本(默认是1.1)寻找可用的方法,我们假设使用shell_v1_1模块,它导入自from novaclient.v1_1 import shell as shell_v1_1,然后调用_find_actions方法。注意:这个方法传入的是一个模块,python中所有东西都是对象,模块也不例外,不过这里我们姑且认为它传入了一个类,类似与java的XXXClass.class类型,以下是代码:
def _find_actions(self, subparsers, actions_module): # actions_module = shell_v1.1 for attr in (a for a in dir(actions_module) if a.startswith('do_')): # attr = do_flavor_list # I prefer to be hypen-separated instead of underscores. command = attr[3:].replace('_', '-') # do_flavor_list -> flavor-list callback = getattr(actions_module, attr) desc = callback.__doc__ or '' action_help = desc.strip() arguments = getattr(callback, 'arguments', []) subparser = subparsers.add_parser(command, help=action_help, description=desc, add_help=False, formatter_class=OpenStackHelpFormatter ) subparser.add_argument('-h', '--help', action='help', help=argparse.SUPPRESS, ) self.subcommands[command] = subparser for (args, kwargs) in arguments: subparser.add_argument(*args, **kwargs) subparser.set_defaults(func=callback)
可见这个方法是利用反射机制获取所有以do_开头的方法,这个do_XXX_XXX,XXX-XXX就是命令名,而do_XXX_XXX就是回调函数,把函数作为变量赋值给callback,是函数式编程的经典用法。最后把callback传入set_defaults方法。
至此我们知道nova list其实调用了novaclient.v1_1.shell.do_list()方法,而nova flavor-list调用了novaclient.v1_1.shell.do_flavor_list()方法,下面以nova --debug flavor-list为例继续深入。
我们看novaclient.v1_1.shell源码,发现好多do_XXX方法,但它本身并不做什么工作,而是调用cs去做,cs是什么现在不管。下面是do_flavor_list方法:
def do_flavor_list(cs, args): """Print a list of available 'flavors' (sizes of servers).""" if args.all: flavors = cs.flavors.list(is_public=None) else: flavors = cs.flavors.list() _print_flavor_list(flavors, args.extra_specs)
现在我们不知道cs是什么东西,那我们继续回到main函数,main函数中间其余代码均是在各种参数检查,我们忽略不管,直接跳到main函数结尾
def main(self, argv): # Parse args once to find version and debug settings parser = self.get_base_parser() # 添加选项,比如--user, --password等 (options, args) = parser.parse_known_args(argv) self.setup_debugging(options.debug) # 如果options中有--debug,则设置logger的level为DEBUG,并输出到标准输出流 # Discover available auth plugins novaclient.auth_plugin.discover_auth_systems() # build available subcommands based on version self.extensions = self._discover_extensions( options.os_compute_api_version) self._run_extension_hooks('__pre_parse_args__') # NOTE(dtroyer): Hackery to handle --endpoint_type due to argparse # thinking usage-list --end is ambiguous; but it # works fine with only --endpoint-type present # Go figure. if '--endpoint_type' in argv: spot = argv.index('--endpoint_type') argv[spot] = '--endpoint-type' # 根据版本解析子命令 subcommand_parser = self.get_subcommand_parser( options.os_compute_api_version) self.parser = subcommand_parser # 如果--help,则打印help信息,并退出 if options.help or not argv: subcommand_parser.print_help() return 0 args = subcommand_parser.parse_args(argv) #解析命令行参数 argv=['list'] #print("args = %s" % args) self._run_extension_hooks('__post_parse_args__', args) # Short-circuit and deal with help right away. # nova help xxxx 命令 if args.func == self.do_help: self.do_help(args) return 0 # nova bash-completion elif args.func == self.do_bash_completion: self.do_bash_completion(args) return 0 # 这里省略大量代码 self.cs = client.Client(options.os_compute_api_version, os_username, os_password, os_tenant_name, tenant_id=os_tenant_id, auth_url=os_auth_url, insecure=insecure, region_name=os_region_name, endpoint_type=endpoint_type, extensions=self.extensions, service_type=service_type, service_name=service_name, auth_system=os_auth_system, auth_plugin=auth_plugin, volume_service_name=volume_service_name, timings=args.timings, bypass_url=bypass_url, os_cache=os_cache, http_log_debug=options.debug, cacert=cacert, timeout=timeout) # 这里省略大量代码 args.func(self.cs, args) # 此时func等于do_flavor_list if args.timings: #如果有--timing选项,则打印请求时间 self._dump_timings(self.cs.get_timings())
可见cs是调用client.Client方法返回的,我们查看其代码client.py:
def get_client_class(version): version_map = { '1.1': 'novaclient.v1_1.client.Client', '2': 'novaclient.v1_1.client.Client', '3': 'novaclient.v3.client.Client', } try: client_path = version_map[str(version)] except (KeyError, ValueError): msg = "Invalid client version '%s'. must be one of: %s" % ( (version, ', '.join(version_map.keys()))) raise exceptions.UnsupportedVersion(msg) return utils.import_class(client_path) def Client(version, *args, **kwargs): client_class = get_client_class(version) return client_class(*args, **kwargs)
不难看出cs即根据版本选择的Client类型,这里我们用的是novaclient.v1_1.client.Client。这个模块可以认为是功能模块的注册类,比如flavors操作模块为flavors.py,为了让他生效,必须注册,即在Client中设置self.flavors=flavors.FlavorManager(self):
self.projectid = project_id self.tenant_id = tenant_id self.flavors = flavors.FlavorManager(self) self.flavor_access = flavor_access.FlavorAccessManager(self) self.images = images.ImageManager(self) self.limits = limits.LimitsManager(self) self.servers = servers.ServerManager(self)
从do_flavor_list方法中cs.flavor.list()即调用了flavors.FlavorManager().list方法。从这里我们可以看出openstack的设计原则,即支持自由灵活的可扩展性,如果需要添加新功能,几乎不需要修改太多代码,只要修改Client注册即可。
我们查看flavors.py中的list方法:
def list(self, detailed=True, is_public=True): """ Get a list of all flavors. :rtype: list of :class:`Flavor`. """ qparams = {} # is_public is ternary - None means give all flavors. # By default Nova assumes True and gives admins public flavors # and flavors from their own projects only. if not is_public: qparams['is_public'] = is_public query_string = "?%s" % urlutils.urlencode(qparams) if qparams else "" detail = "" if detailed: detail = "/detail" return self._list("/flavors%s%s" % (detail, query_string), "flavors")
很明显这里是把list命令组装url请求,然后调用_list方法,由于FlavorManager继承自base.ManagerWithFind,而base.ManagerWithFind继承自Manager,_list方法在Manager中定义。
def _list(self, url, response_key, obj_class=None, body=None): if body: _resp, body = self.api.client.post(url, body=body) else: _resp, body = self.api.client.get(url) if obj_class is None: obj_class = self.resource_class data = body[response_key] # NOTE(ja): keystone returns values as list as {'values': [ ... ]} # unlike other services which just return the list... if isinstance(data, dict): try: data = data['values'] except KeyError: pass with self.completion_cache('human_id', obj_class, mode="w"): with self.completion_cache('uuid', obj_class, mode="w"): return [obj_class(self, res, loaded=True) for res in data if res]
有源码中看出主要发送url请求即self.api.client.post(url, body=dody)或者self.api.client.get(url),具体根据是否有body,即是否数据选择GET或者POST请求。然后处理返回的数据。self.api在这里其实就是novaclient.v1_1.client.Client,只是前面用cs,这里用api。
我们回到novaclient.v1_1.client.Client的方法中,我们发现除了注册一系列功能外,还有一个比较特殊的,
self.client = client.HTTPClient(username, password, projectid=project_id, tenant_id=tenant_id, auth_url=auth_url, insecure=insecure, timeout=timeout, auth_system=auth_system, auth_plugin=auth_plugin, proxy_token=proxy_token, proxy_tenant_id=proxy_tenant_id, region_name=region_name, endpoint_type=endpoint_type, service_type=service_type, service_name=service_name, volume_service_name=volume_service_name, timings=timings, bypass_url=bypass_url, os_cache=self.os_cache, http_log_debug=http_log_debug, cacert=cacert)
这个self.client是client.HTTPClient类型,真正负责发送url请求的类,部分代码为:
def _cs_request(self, url, method, **kwargs): if not self.management_url: self.authenticate() # Perform the request once. If we get a 401 back then it # might be because the auth token expired, so try to # re-authenticate and try again. If it still fails, bail. try: kwargs.setdefault('headers', {})['X-Auth-Token'] = self.auth_token if self.projectid: kwargs['headers']['X-Auth-Project-Id'] = self.projectid resp, body = self._time_request(self.management_url + url, method, **kwargs) return resp, body except exceptions.Unauthorized as e: try: # frist discard auth token, to avoid the possibly expired # token being re-used in the re-authentication attempt self.unauthenticate() self.authenticate() kwargs['headers']['X-Auth-Token'] = self.auth_token resp, body = self._time_request(self.management_url + url, method, **kwargs) return resp, body except exceptions.Unauthorized: raise e def get(self, url, **kwargs): return self._cs_request(url, 'GET', **kwargs)
最后会调用http.request方法发送请求,这里使用了python库Requests: HTTP for Humans,这个库比httplib2更好,查看地址:http://docs.python-requests.org/en/latest/。接收请求的工作就由nova-api负责了,这里不再深入。
接下来我们简单增加一个没用的功能test,首先在novaclient/v1_1下touch test.py,使用vim增加以下代码:
""" Test interface. """ from novaclient import base class Test(base.Resource): def test(self): print("This is a test") class TestManager(base.Manager): def test(self): print("This is a test")
然后我们需要在client中注册,编辑novaclient/v1_1/client.py文件,增加self.test = test.TestManager(self)
然后在shell下增加入口函数,注册新功能,
def do_test(cs, _args): """ do test. """ cs.test.test()
运行nova test, nova help test查看效果。
2021年9月07日 16:51 Wow, excellent post. I'd like to draft like this too - taking time and real hard work to make a great article. This post has encouraged me to write some posts that I am going to write soon. 123movies official