Extending the parser¶
Clize allows the parser to be extended through, most notably, new parameters. They allow you to introduce new argument parsing behaviour within the parser’s contraints:
The finality of argument parsing in Clize is to determine
func
,args
andkwargs
in order to dofunc(*args, **kwargs)
.Arguments that start with
-
are looked up in the named parameters table by their first letter when there is only one dash, or until the end of the argument or=
when there are two dashes.Other arguments are processed by the positional parameters in the order they are stored.
You are free to have your custom parameters edit the CliBoundArguments
object, including changing the available parameters mid-parsing through
ba.posparam
or ba.namedparam
<CliBoundArguments.namedparam>`
.
This document explains each step of the CLI inference and argument parsing process. There is a walkthrough of the argument process and an example that shows how you can add a custom parameter.
Execution overview¶
clize.run
is called with either a function, or a sequence or mapping of functions.It forwards that to
Clize.get_cli
in order to get a cli object to run.If there is more than one function,
Clize.get_cli
usesSubcommandDispatcher
to wrap it, otherwise it creates a newClize
instance with it and returns it.run
calls that cli object withsys.argv
unpacked (ie.obj(*sys.argv)
).Assuming that cli object is a
Clize
instance, it will look at the function that was passed and determine aCliSignature
object from it.Clize
calls the cli signature’sread_arguments
method with the command-line arguments, which returns aCliBoundArguments
instance.That
CliBoundArguments
instance carries information such as the arguments that the function will be called with or the instruction to replace that function with another entirely.Clize
then runs the chosen function andclize.run
prints its return value.
Parameter conversion¶
In step 5 above, CliSignature.from_signature
converts each parameter. Here
is the process followed for each parameter:
The annotation for each parameter is read as a sequence. If it isn’t one then it is wrapped in one.
The annotations are searched for
clize.parser.Parameter.IGNORE
. If found, the parameter is dropped with no further processing.The annotations are searched for a parameter converter function. If none is found,
default_converter
is used.The parameter converter is called with the
inspect.Parameter
object representing the parameter and the sequence of annotations without the parameter converter. Its return value, expected to be aclize.parser.Parameter
instance, is added to the list of parameters for the resultingCliSignature
instance.
The default parameter converter¶
The default parameter converter works as follows:
It looks at the parameter’s type and checks whether it is a named or positional parameter. This is used to check if it is legal to assign aliases to it and to determine what cli parameter class will be used to represent it.
It looks at the parameter’s default value and extracts its type, expecting that it is a valid value converter. If there isn’t one the parameter is marked as required.
The annotations sequence is iterated on:
If the annotation is a
Parameter
instance, it is returned immediately with no processing.If the annotation is a value converter it is used instead of the default value’s type. Specifying a value converter is required when the default value’s type isn’t a valid one itself.
If it is a string, it is used as an alias unless the parameter is positional.
Finally, depending on the above, a parameter class is picked, instantiated and returned:
PositionalParameter
if the parameter was positional,ExtraPosArgsParameter
for a*args
parameter,OptionParameter
for a named parameter that takes an argument,IntOptionParameter
if that argument is of typeint
FlagParameter
for a named parameter withFalse
as default andbool
as type,An error is raised for
**kwargs
parameters, as their expected equivalent in a CLI is largely subjective. If you want to forward arguments to another function, consider using function compositing instead of having a CLI parameter handle it.
The argument parser¶
The argument parsing is orchestrated by CliBoundArguments
during its
initialization. For each argument of its input, it selects the appropriate
Parameter
instance to handle it. If the argument on the input starts with
-
it looks in the CliSignature.named
dictionary. If not, it picks the
next positional parameter from CliSignature.positional
. The parameter’s
read_argument
and apply_generic_flags
methods are
called.
- Parameter.read_argument(ba, i)[source]¶
Reads one or more arguments from
ba.in_args
from positioni
.- Parameters
ba (clize.parser.CliBoundArguments) – The bound arguments object this call is expected to mutate.
i (int) – The current position in
ba.args
.
This method is expected to mutate ba
, an instance of CliBoundArguments
.
In particular, it should add any relevant arguments to ba
’s
args
and kwargs
attributes which
are used when calling the wrapped callable as in func(*args, **kwargs)
. It
can also set the func
attribute which overrides the
Clize
object’s wrapped callable.
Part of the parameter’s behavior is split from read_argument
into
apply_generic_flags
in order to facilitate subclassing:
- Parameter.apply_generic_flags(ba)[source]¶
Called after
read_argument
in order to set attributes onba
independently of the arguments.- Parameters
ba (clize.parser.CliBoundArguments) – The bound arguments object this call is expected to mutate.
The base implementation of this method applies the
last_option
setting if applicable and discards itself fromCliBoundArguments.unsatisfied
The both of these methods are expected to discard the parameter from
unsatisfied
, the list of still-unsatisfied required
parameters, when applicable. The sticky
,
posarg_only
and skip
can also be
modified to change the ongoing argument reading process.
Walkthrough: The parsing process¶
Let’s examine the steps Clize takes when executing this function:
from clize import run, parameters, converters
def main(pos1, pos2:converters.file(), *, opt, mul: (parameters.multi(), 'm')):
...
run(main)
As an example, we’ll use this command. It should fail because --opt
is
given twice. However, --help
will still display the help.
$ python3 ./example.py one two --opt o1 -mm1 --mul m2 --opt o2 --help -mm3
First of all Clize will convert the function parameters to a CLI object it can use. For this it looks at every parameter:
pos1
has no annotations. The default parameter converter transforms it into aPositionalParameter
instance to match its behavior as a Python positional parameter.pos2
has an annotation. It’s a value converter. Again, the default parameter converter usesPositionalParameter
to represent it.opt
is a keyword-only parameter. The default converter uses anOptionParameter
instance to represent it.mul
has a tuple with two items as annotation. The first one,clize.parameters.multi
is a parameter converter. Clize uses it instead of the default converter. This parameter converter returns a custom parameter for this parameter.Clize
adds a parameter to trigger the help, an instance ofFallbackCommandParameter
.
These parameters are used to create an instance of CliSignature
.
CliSignature.read_arguments()
is called by Clize.read_commandline
with the arguments sys.argv[1:]
and the program name sys.argv[0]
.
This creates an instance of CliBoundArguments
with this data and starts the
parsing process with its process_arguments
method.
This method enumerates every input argument (ba.in_args
, sys.argv[1:]
):
one
doesn’t start with a-
, so it tries to get the next positional parameter fromba.posparam
.This is the
PositionalParameter
instance that was created forpos1
.CliBoundArguments
callsread_argument(ba, i)
on this instance.ba
is theCliBoundArguments
instance andi
is the position inba.in_args
No converter or default was specified so it just takes
ba.in_args[i]
and stores it inba.args
.two
also doesn’t start with a-
, so it is matched with the next positional parameter, thePositionalParameter
forpos2
.This one has a value converter, which gets called with the text value. The converter returns an object which the parameter just stores in
ba.args
.--opt
starts with-
, soCliBoundArguments
looks it up inba.namedparam
.OptionParameter.read_argument
readsba.in_args[i+1]
and saves it asba.kwargs[opt]
. It setsba.skip
to 1 so that the parser skips overo1
instead of processing it as a positional argument.-mm1
starts with-
so it is recognized as an option.Because there is only one
-
, Clize looks up the parameter with just the first letter. It finds that-m
maps to the parameter created byclize.parameters.multi
.That parameter instance sees that it’s being invoked as a short-form parameter and that a value is attached to the parameter name. It extracts this value and adds it to a list assigned to
ba.kwargs['mul']
.--mul
is also matched to the same parameter instance. It adds it toba.kwargs['mul']
and setsba.skip = 1
--opt
is seen agsin.OptionParameter.read_argument
sees thatba.kwargs['opt']
already exists and raises an exception.Before the exception propagates past
CliBoundArguments
it scans the remaining arguments for any argument matching one inba.sig.alternate
. It finds--help
and calls the itsread_argument
method.That method clears the existing values in
ba.args
andba.kwargs
and setsba.func
.
Execution leaves the parser loop at this point before it has a chance to
process -mm3
. The CliBoundArguments
object is complete. At this stage
it has:
ba.func
set to a function that will display the help. (It is a CLI itself.)ba.args
is set to a list with the program name suffixed with--help
. This is so that the--help
CLI can receive an accurate program name as first argument.
Going back up the call chain, Clize.read_commandline
calls
ba.func(^ba.args)
and thus launching the help.
Example: Implementing one_of
¶
clize.parameters.one_of
creates a parameter annotation that modifies the
parameter to only allow values from a given list:
from clize import run, parameters
def func(breakfast:parameters.one_of('ham', 'spam')):
"""Serves breakfast
:parser breakfast: what food to serve
"""
print("{0} is served!".format(breakfast))
run(func)
The breakfast
parameter now only allows ham
and spam
:
$ python breakfast.py ham
ham is served!
$ python breakfast.py spam
spam is served!
$ python breakfast.py eggs
breakfast.py: Bad value for breakfast: eggs
Usage: breakfast.py breakfast
A list is produced when list
is supplied:
$ python breakfast.py list
breakfast.py: Possible values for breakfast:
ham
spam
Also, it hints at the list
keyword on the help page:
$ python breakfast.py --help
Usage: breakfast.py breakfast
Serves breakfast
Arguments:
breakfast what food to serve (use "list" for options)
Other actions:
-h, --help Show the help
one_of
is implemented in Clize as a wrapper around
mapped
which offers several more features. In this example
we will only reimplement the features described above.
Creating a parameter class for us to edit¶
from clize import run, parser
class OneOfParameter(object):
def __init__(self, values, **kwargs):
super().__init__(**kwargs)
self.values = values
def one_of(*values):
return parser.use_mixin(OneOfParameter, kwargs={'values': values})
def func(breakfast:one_of('ham', 'spam')):
"""Serves breakfast
:parser breakfast: what food to serve
"""
print("{0} is served!".format(breakfast))
run(func)
Here we used parser.use_mixin
to implement the parameter annotation. It will
create a parameter instance that inherits from both OneOfParameter
and the
appropriate class for the parameter being annotated:
PositionalParameter
, OptionParameter
or
ExtraPosArgsParameter
. This means our class will be able to override
some of those classes’ methods.
For now, it works just like a regular parameter:
$ python breakfast.py abcdef
abcdef is served!
Changing coerce_value
to validate the value¶
PositionalParameter
, OptionParameter
and
ExtraPosArgsParameter
all use ParameterWithValue.coerce_value
. We
override it to only accept the values we recorded:
from clize import errors
class OneOfParameter(parser.ParameterWithValue):
def __init__(self, values, **kwargs):
super().__init__(**kwargs)
self.values = set(values)
def coerce_value(self, arg, ba):
if arg in self.values:
return arg
else:
raise errors.BadArgumentFormat(arg)
It now only accepts the provided values:
$ python breakfast.py ham
ham is served!
$ python breakfast.py spam
spam is served!
$ python breakfast.py eggs
breakfast.py: Bad value for breakfast: eggs
Usage: breakfast.py breakfast
Displaying the list of choices¶
We can check if the passed value is list
within coerce_value
. When that
is the case, we change func
and swallow the
following arguments. However, to ensure that the
read_argument
method doesn’t alter this, we need to skip
its execution. In order to do this we will raise an exception from
coerce_value
and catch it in read_argument
:
class _ShowList(Exception):
pass
class OneOfParameter(parser.ParameterWithValue):
def __init__(self, values, **kwargs):
super().__init__(**kwargs)
self.values = values
def coerce_value(self, arg, ba):
if arg == 'list':
raise _ShowList
elif arg in self.values:
return arg
else:
raise errors.BadArgumentFormat(arg)
def read_argument(self, ba, i):
try:
super(OneOfParameter, self).read_argument(ba, i)
except _ShowList:
ba.func = self.show_list
ba.args[:] = []
ba.kwargs.clear()
ba.sticky = parser.IgnoreAllArguments()
ba.posarg_only = True
def show_list(self):
for val in self.values:
print(val)
On ba
, setting func
overrides the function to be run
(normally the function passed to run
). args
and
kwargs
are the positional and keyword argument that will
be passed to that function. Setting sticky
to an
IgnoreAllArguments
instance will swallow all positional arguments instead of
adding them to args
, and posarg_only
makes keyword arguments be processed as if they were positional arguments so
they get ignored too.
$ python breakfast.py list
ham
spam
$ python breakfast.py list --ERROR
ham
spam
The list is printed, even if erroneous arguments follow.
Adding a hint to the help page¶
Clize uses Parameter.show_help
to produce the text used to describe
parameters. It uses Parameter.help_parens
to provide the content inside the
parenthesis after the parameter description.
class OneOfParameter(parser.ParameterWithValue):
...
def help_parens(self):
for s in super(OneOfParameter, self).help_parens():
yield s
yield 'use "list" for options'
The help page now shows the hint:
$ python breakfast.py --help
Usage: breakfast.py breakfast
Serves breakfast
Arguments:
breakfast what food to serve (use "list" for options)
Other actions:
-h, --help Show the help
The full example is available in examples/bfparam.py
.