[python-libvirt] RFC:PEP 484 type annotation - how to continue

[Philipp Hahn]

Hello,

currently my Merge Quests
<https://gitlab.com/libvirt/libvirt-python/-/merge_requests/9> is stuck
because there are some open technical discussions:

1. The Python binding consists of two layers:
   - libvirtmod* is the low-level C-like API, which mostly is just a
thin wrapper around the C-library. It is generated from the API.xml files
  - The high-level libvirt* modules, providing an object oriented API,
which is also generated from the API.xml files.
  In both cases there are many exceptions defines via
libvirt*-override-api.xml or coded in generator.py:
  - skip_impl: Generate Python code, but skip C implementation
  - custom_function: Generate neither C nor Python code - hand crafted code
  - skip_function: Generate neither C nor Python code - not used at all
  - function_skip_python_impl: Generate C code, but skip python impl

Previously the functions listed in "skip_impl" were also duplicated in
the "libvirt-*override-api.xml" files, which is somehow hard to
maintain. With d19699e48ce9c17481f94b4dc0715e18e05d5adb I removed all
those names from "skip_impl", which are also listed in the overrides and
added code to add them back when "generator.py" is invoked.
For this to work I had to change several "files" attributes to start
with "python", which looks okay to me, but still would like to get verified.

2. The low-level library works mostly with bytes, but the high-level
Python API uses nested Lists, Tuples and Dicts. To get most out of the
Python static type annotations the input and output parameters should be
modled in as much detail as possible.
This requires to store those types somewhere. My current
46592e28124858ed863bceba65c3dafa2bbb02cd adds some often used types to
generator.py and then references them from the overrides.xml:

> +    'PyListSecret': ('', 'List["virSecret"]', '', ''),
> +    'PyDictAny': ('', 'Dict[str, Any]', '', ''),

> +      <return type='PyListSecret' info='the list of secrets or None in case of error'/>

This still is very painful as there are many "used-once" types, which
still must be declared and used in two places.
So far the "return" type was used for the "C" type and currently I
mis-use it for referencing it for my Python types. This doesn't look
right, so I would prefer adding a new attribute which can be used to
override the Python type.
This also would probably nicely fit with functions like
"virDomainOpenConsole", when "dev_name" is optional and can be "None",
which the current type hint requires a "string".

Is extending the libvirt*-override-api.xml files okay or must this be
coordinated with the other parts of the project?


Thank you for your time
Philipp Hahn
-- 
Philipp Hahn
Open Source Software Engineer

Univention GmbH
be open.
Mary-Somerville-Str. 1
D-28359 Bremen

📞 +49-421-22232-57
🖶 +49-421-22232-99

✉️ hahn at univention.de
🌐 https://www.univention.de/

Geschäftsführer: Peter H. Ganten
HRB 20755 Amtsgericht Bremen
Steuer-Nr.: 71-597-02876