[libvirt] [PATCH v5 02/23] build-aux: rewrite whitespace checker in Python
Cole Robinson
crobinso at redhat.com
Fri Nov 15 20:30:15 UTC 2019
On 11/11/19 9:38 AM, Daniel P. Berrangé wrote:
> As part of an goal to eliminate Perl from libvirt build tools,
> rewrite the check-spacing.pl tool in Python.
>
> This was a straight conversion, manually going line-by-line to
> change the syntax from Perl to Python. Thus the overall structure
> of the file and approach is the same.
>
I checked the speed difference mentioned in the cover letter.
Old:
time ./build-aux/check-spacing.pl `cat ~/src/libvirt/cfiles`
real 0m1.423s
New:
time ./scripts/check-spacing.py `cat cfiles`
real 0m5.607s
About around there. The script can easily be parallelized though:
diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py
index 6b9f3ec1ba..f2ce376e80 100755
--- a/scripts/check-spacing.py
+++ b/scripts/check-spacing.py
@@ -222,8 +222,12 @@ def check_whitespace(filename):
ret = 0
-for filename in sys.argv[1:]:
- if check_whitespace(filename):
- ret = 1
+filenames = sys.argv[1:]
+import multiprocessing
+pool = multiprocessing.Pool()
+results = pool.map(check_whitespace, filenames)
+pool.close()
+pool.join()
+ret = int(any(results))
sys.exit(ret)
After that:
[:~/src/libvirt] (python3 *) $ time ./scripts/check-spacing.py `cat cfiles`
real 0m1.674s
user 0m10.506s
sys 0m0.041s
Which is pretty close to the perl version, but at the cost of more
cycles. Not sure how that will play with 'make' job control. I believe
the Pool defaults to number of logical host CPUs, which is 8 on my
T480s. Going above 4 doesn't seem to make much of a difference in my
testing for this case
- Cole
More information about the libvir-list
mailing list