[libvirt] [PATCH v5 02/23] build-aux: rewrite whitespace checker in Python

Cole Robinson crobinso at redhat.com
Fri Nov 15 20:30:15 UTC 2019


On 11/11/19 9:38 AM, Daniel P. Berrangé wrote:
> As part of an goal to eliminate Perl from libvirt build tools,
> rewrite the check-spacing.pl tool in Python.
> 
> This was a straight conversion, manually going line-by-line to
> change the syntax from Perl to Python. Thus the overall structure
> of the file and approach is the same.
> 

I checked the speed difference mentioned in the cover letter.

Old:
time ./build-aux/check-spacing.pl `cat ~/src/libvirt/cfiles`
real	0m1.423s

New:
time ./scripts/check-spacing.py `cat cfiles`
real	0m5.607s

About around there. The script can easily be parallelized though:

diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py
index 6b9f3ec1ba..f2ce376e80 100755
--- a/scripts/check-spacing.py
+++ b/scripts/check-spacing.py
@@ -222,8 +222,12 @@ def check_whitespace(filename):


 ret = 0
-for filename in sys.argv[1:]:
-    if check_whitespace(filename):
-        ret = 1
+filenames = sys.argv[1:]
+import multiprocessing
+pool = multiprocessing.Pool()
+results = pool.map(check_whitespace, filenames)
+pool.close()
+pool.join()
+ret = int(any(results))

 sys.exit(ret)


After that:

[:~/src/libvirt] (python3 *) $ time ./scripts/check-spacing.py `cat cfiles`

real	0m1.674s
user	0m10.506s
sys	0m0.041s


Which is pretty close to the perl version, but at the cost of more
cycles. Not sure how that will play with 'make' job control. I believe
the Pool defaults to number of logical host CPUs, which is 8 on my
T480s. Going above 4 doesn't seem to make much of a difference in my
testing for this case

- Cole




More information about the libvir-list mailing list