[libvirt] [PATCH v4 00/26] scripts: convert most perl scripts to python

Daniel P. Berrangé berrange at redhat.com
Thu Oct 10 13:43:49 UTC 2019


On Thu, Oct 10, 2019 at 03:28:39PM +0200, Ján Tomko wrote:
> On Wed, Oct 09, 2019 at 12:37:15PM +0100, Daniel P. Berrangé wrote:
> > Note that the check-spacing.py script is significantly
> > slower in Python than in Perl. After researching this
> > it appears there is nothing that can be done. The Perl
> > regex engine is simply much better optimized than the
> > Python one.
> 
> I'm afraid immutability of strings does not help either.
> I tried speeding it up a bit by running the regexes conditionally, e.g.:
> 
> +            if '"' in data:
> +                data = quotedstringprog.sub('"XXX"', data)
> 
> That got it down from 0m6.368s to 0m3.802s (compared to perl's 1.5s),
> but even before that change, I don't consider the Python version of this
> one script to be more readable.
> 
> Note that perl's optimizations aren't perfect either - merely
> moving all the checks into a subroutine resulted in a significant
> slowdown.
> 
> > As previously discussed we need to loook
> > at uncrustify or clang-format or some other tool to
> > validate whitespace formatting. This is ongoing. We
> > can either accept the slow down in the short term or
> > keep the Perl version in the short term.
> > 
> 
> Just to set the expectations, running clang-format on all the
> C files took me ~40 s. Not sure how much that can be efficiently
> parallelized or if meson could be convinced to start clang-format
> as the first thing for the 'check' target, even before starting
> to compile anything.

Not entirely surprising since it is actually doing proper parsing
of the C syntax, which is quite a bit of work. 

FWIW, when I'm writing Go code I have used emacs 'go-mode' and
this runs 'go fmt' to automatically fix your style problems
every time you save the file. So your code is basically always
correct, and you don't need a separate job to check it later.

For this approach to be viable though, you need to have an
automated CI check that is run at time of pushing to git master
to block pushes of code which is non-compliant. It is the kind 
of thing projects do when using pull requests & gating CI in 
gitlab/github though. So this is itself another can of worms.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|




More information about the libvir-list mailing list