[Open-scap] Latest OpenSCAP changes to speed up SSG builds

[Jan Lieskovsky]

----- Original Message -----
> From: "Gabe Alford" <redhatrises at gmail.com>
> To: "SCAP Security Guide" <scap-security-guide at lists.fedorahosted.org>
> Cc: "open-scap-list" <open-scap-list at redhat.com>
> Sent: Wednesday, July 27, 2016 3:00:34 PM
> Subject: Re: Latest OpenSCAP changes to speed up SSG builds
> 
> 
> On Tue, Jul 26, 2016 at 3:24 PM, Martin Preisler < mpreisle at redhat.com >
> wrote:
> 
> 
> I found pretty bad inefficiencies in some of our XSLTs.
> 
> Check out
> https://github.com/OpenSCAP/openscap/commit/a65bf27dec4a93e2b87cec8cbcd80bec4fd2328a
> or
> https://github.com/OpenSCAP/openscap/commit/1dd5573b2c964b00af57215cadb7f13b1938bac6
> 
> Long story short I was able to cut SSG build time on my machine from
> 2m21.061s to 1m08.771s. In other words my machine now needs roughly half
> the time to build SSG. I was timing `make -j 4`.
> 
> It will take a while for these changes to make it into releases
> and into distributions but I am writing this email because the
> savings are significant and perhaps you want to enjoy them sooner.
> SCAP Security Guide contributors are doing a lot of content builds.
> 
> This will only work if you !!! have OPENSCAP 1.2.x !!!
> 
> If you want to speed up the build, replace
> 
> /usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl with
> https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf_1.1_to_1.2.xsl
> 
> /usr/share/openscap/xsl/xccdf-share.xsl with
> https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf-share.xsl
> 
> and
> 
> /usr/share/openscap/xsl/xccdf-guide-impl.xsl with
> https://github.com/OpenSCAP/openscap/blob/maint-1.2/xsl/xccdf-guide-impl.xsl
> 
> 
> Perhaps it would be worth it to deploy this on our Jenkins slaves
> even before it hits releases? What do you think?
> 
> +1

Not like I would want to be the blocker for speedups / SSG Jenkins
optimizations, but how would you like to deploy these changes?

Two scenarios come to mind:
* just replace the affected XSLT transformations in recent RHEL openscap
  builds,
* make a separate openscap RPM (applied on Jenkins slaves outside of RHEL
  release cycle process)

While the advantage might be more quicker completion of Jenkins SSG build
jobs, there are two disadvantages / concerns related with this:
* if modified oscap returns different result than latest oscap available
  in RHEL - which report should be considered the right one? Which one is valid?
* the concern with maybe even more importance is we wouldn't test on default
  RHEL / Fedora systems like we do now. The environment would be modified a bit
  (and it might or might not return same results). So to ensure we "will
  continue working" on default RHEL / Fedora - should we create another Jenkins
  CI jobs for default RHEL / Fedora?

Maybe we could use "speed optimized" openscap on 'scap-security-guide-pull-requests'
Jenkins CI job, and 'default one' on 'scap-security-guide' Jenkins CI job (the latter
one being which actually tells if we don't regress across PRs).

But even this approach won't solve the issue when the result of 'speed optimized'
openscap would differ from 'default' openscap (but maybe the result on the default
one would be more authoritative / determinant in this case).

Just my 2cs.

Jan
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

> 
> 
> --
> SCAP Security Guide mailing list
> scap-security-guide at lists.fedorahosted.org
> https://lists.fedorahosted.org/admin/lists/scap-security-guide@lists.fedorahosted.org
> https://github.com/OpenSCAP/scap-security-guide/
>