[libvirt] [PATCH v5 00/23] scripts: convert most perl scripts to python
Daniel P. Berrangé
berrange at redhat.com
Mon Nov 11 14:38:03 UTC 2019
This series is an effort to reduce the number of different
languages we use by eliminating most use of perl in favour
of python.
This aligns with fact that the likely future build system
we'll use (meson) is written in python, and that python
is much more commonly used/understood by developers these
days than perl.
With this applied we use perl in a handful of places only:
- src/rpc/gendispatch.pl - this is a horrendously large
script and very hard to understand/follow. A straight
syntax conversion to Python would still leave a hgue
and hard to understand/follow script. It really needs
a clean room rewrite from scratch, with better structure.
- src/rpc/genprotocol.pl - fairly easy to convert, but
might be obsolete depending on approach for rewriting
gendispatch.pl, so ignored for now
- tests/oomtrace.pl - will be purge by the patches that
drop OOM handling anyway
- tools/wireshark/util/genxdrstub.pl - a very large
script, which I haven't got the courage to tackle
yet.
- cfg.mk/maint.mk - many syntax rules involve regexes
which are fed to perl. Decision on what to do
with syntax-check rules punted to another time.
- build-aux/gitlog-to-changelog
- build-aux/useless-if-before-free - Both pulled in
from gnulib. Could be rewritten quite easily if
desired, but given that we aren't maintainers of
them right now, they're ignored as they don't
really impact our developers.
Note that the check-spacing.py script is significantly
slower in Python than in Perl. After researching this
it appears there is nothing that can be done. The Perl
regex engine is simply much better optimized than the
Python one. As previously discussed we need to loook
at uncrustify or clang-format or some other tool to
validate whitespace formatting. This is ongoing. We
can either accept the slow down in the short term or
keep the Perl version in the short term.
In v5:
- Rebased to cope with changes to require VPATH build
- Merged the already-ACKd scripts
In v4:
- Moved all scripts into the scripts/ directory instead
of having them scattered around source tree
- Use re.search instead of re.match
- Don't bother re-compiling regexes
In v3:
- All scripts comply with all flake8 style rules with
exception of
E129 visually indented line with same indent as next logical line
In v2:
- Pulled in patch to hacking file
- Converted many more scripts
- Forced UTF-8 character set to avoid ascii codec
on py3 < 3.7
Daniel P. Berrangé (23):
build-aux: rewrite duplicate header checker in Python
build-aux: rewrite whitespace checker in Python
build-aux: rewrite mock inline checker in Python
build-aux: rewrite header ifdef checker in Python
src: rewrite ACL permissions checker in Python
src: rewrite symfile sorting checker in Python
src: rewrite symfile library checker in Python
src: rewrite systemtap probe generator in Python
src: rewrite systemtap function generator in Python
src: rewrite driver name checker in Python
src: rewrite driver impl checker in Python
src: rewrite ACL rule checker in Python
src: rewrite polkit ACL generator in Python
src: rewrite remote protocol checker in Python
tests: rewrite test argv line wrapper in Python
tests: rewrite qemu capability grouper in Python
tests: rewrite file access checker in Python
docs: rewrite hvsupport.html page generator in python
docs: rewrite polkit docs generator in Python
docs: move apibuild.py to the scripts/ directory
docs: move reformat-news.py to the scripts/ directory
docs: move esx_vi_generator.py to the scripts/ directory
docs: move hyperv_wmi_generator.py to the scripts/ directory
Makefile.am | 33 +-
build-aux/check-spacing.pl | 198 -------
build-aux/header-ifdef.pl | 182 -------
build-aux/mock-noinline.pl | 75 ---
build-aux/prohibit-duplicate-header.pl | 26 -
build-aux/syntax-check.mk | 32 +-
docs/Makefile.am | 16 +-
docs/genaclperms.pl | 125 -----
docs/hvsupport.pl | 459 ----------------
{docs => scripts}/apibuild.py | 0
scripts/check-aclperms.py | 75 +++
scripts/check-aclrules.py | 263 +++++++++
scripts/check-driverimpls.py | 102 ++++
scripts/check-drivername.py | 114 ++++
scripts/check-file-access.py | 123 +++++
scripts/check-remote-protocol.py | 136 +++++
scripts/check-spacing.py | 229 ++++++++
scripts/check-symfile.py | 80 +++
scripts/check-symsorting.py | 117 ++++
scripts/dtrace2systemtap.py | 143 +++++
{src/esx => scripts}/esx_vi_generator.py | 0
scripts/genaclperms.py | 123 +++++
scripts/genpolkit.py | 122 +++++
scripts/gensystemtap.py | 184 +++++++
scripts/group-qemu-caps.py | 123 +++++
scripts/header-ifdef.py | 231 ++++++++
scripts/hvsupport.py | 504 ++++++++++++++++++
.../hyperv_wmi_generator.py | 0
scripts/mock-noinline.py | 85 +++
scripts/prohibit-duplicate-header.py | 56 ++
{docs => scripts}/reformat-news.py | 0
scripts/test-wrap-argv.py | 170 ++++++
src/Makefile.am | 146 ++---
src/access/Makefile.inc.am | 6 +-
src/access/genpolkit.pl | 119 -----
src/admin/Makefile.inc.am | 6 +-
src/check-aclperms.pl | 73 ---
src/check-aclrules.pl | 252 ---------
src/check-driverimpls.pl | 80 ---
src/check-drivername.pl | 83 ---
src/check-symfile.pl | 70 ---
src/check-symsorting.pl | 106 ----
src/dtrace2systemtap.pl | 130 -----
src/esx/Makefile.inc.am | 6 +-
src/hyperv/Makefile.inc.am | 5 +-
src/rpc/Makefile.inc.am | 1 -
src/rpc/gensystemtap.pl | 193 -------
tests/Makefile.am | 3 +-
tests/check-file-access.pl | 126 -----
tests/file_access_whitelist.txt | 2 +-
tests/group-qemu-caps.pl | 124 -----
tests/test-wrap-argv.pl | 174 ------
tests/testutils.c | 16 +-
53 files changed, 3092 insertions(+), 2755 deletions(-)
delete mode 100755 build-aux/check-spacing.pl
delete mode 100644 build-aux/header-ifdef.pl
delete mode 100644 build-aux/mock-noinline.pl
delete mode 100644 build-aux/prohibit-duplicate-header.pl
delete mode 100755 docs/genaclperms.pl
delete mode 100755 docs/hvsupport.pl
rename {docs => scripts}/apibuild.py (100%)
create mode 100755 scripts/check-aclperms.py
create mode 100755 scripts/check-aclrules.py
create mode 100755 scripts/check-driverimpls.py
create mode 100644 scripts/check-drivername.py
create mode 100755 scripts/check-file-access.py
create mode 100644 scripts/check-remote-protocol.py
create mode 100755 scripts/check-spacing.py
create mode 100755 scripts/check-symfile.py
create mode 100755 scripts/check-symsorting.py
create mode 100755 scripts/dtrace2systemtap.py
rename {src/esx => scripts}/esx_vi_generator.py (100%)
create mode 100755 scripts/genaclperms.py
create mode 100755 scripts/genpolkit.py
create mode 100755 scripts/gensystemtap.py
create mode 100755 scripts/group-qemu-caps.py
create mode 100644 scripts/header-ifdef.py
create mode 100755 scripts/hvsupport.py
rename {src/hyperv => scripts}/hyperv_wmi_generator.py (100%)
create mode 100644 scripts/mock-noinline.py
create mode 100644 scripts/prohibit-duplicate-header.py
rename {docs => scripts}/reformat-news.py (100%)
create mode 100755 scripts/test-wrap-argv.py
delete mode 100755 src/access/genpolkit.pl
delete mode 100755 src/check-aclperms.pl
delete mode 100755 src/check-aclrules.pl
delete mode 100755 src/check-driverimpls.pl
delete mode 100755 src/check-drivername.pl
delete mode 100755 src/check-symfile.pl
delete mode 100755 src/check-symsorting.pl
delete mode 100755 src/dtrace2systemtap.pl
delete mode 100755 src/rpc/gensystemtap.pl
delete mode 100755 tests/check-file-access.pl
delete mode 100755 tests/group-qemu-caps.pl
delete mode 100755 tests/test-wrap-argv.pl
--
2.21.0
More information about the libvir-list
mailing list