<div dir="ltr">+1<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 4, 2020 at 10:16 PM Daniel Alley <<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">+1<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 4, 2020 at 4:05 PM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Looking at Travis insights[0], it seems our average build queue times before February 17 were 10-20+ min and now it looks like they are down to 4-5 min.<div><br></div><div>As a next step, I'd like to propose that for the next sprint we test out a plugin against Github Actions. I was thinking we could merge the following pulp_npm PR and do a few alpha releases to ensure the CD code works.<div><br></div><div><a href="https://github.com/pulp/pulp_npm/pull/2" target="_blank">https://github.com/pulp/pulp_npm/pull/2</a></div><div><br></div><div>Thoughts?</div><div><br></div><div>[0] <a href="https://travis-ci.com/pulp?tab=insights" target="_blank">https://travis-ci.com/pulp?tab=insights</a><br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div>David</div></div></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 17, 2020 at 3:24 PM Fabricio Aguiar <<a href="mailto:fabricio.aguiar@redhat.com" target="_blank">fabricio.aguiar@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">ansible-pulp and pulp_rpm_prerequisites were moved to Github Actions:<div><a href="https://github.com/pulp/ansible-pulp/actions" target="_blank">https://github.com/pulp/ansible-pulp/actions</a></div><div><a href="https://github.com/pulp/pulp_rpm_prerequisites/actions" target="_blank">https://github.com/pulp/pulp_rpm_prerequisites/actions</a><br clear="all"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><br>Best regards,</div><div dir="ltr"><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize">Fabricio</span><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize"> </span><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize">Aguiar</span><div>Software Engineer, Pulp Project</div><div><a href="https://www.redhat.com/" style="color:rgb(0,136,206);font-family:RedHatText,sans-serif;font-size:12px;margin:0px" target="_blank">Red Hat Brazil - Latam</a><br></div><div>+55 11 999652368</div><div><img src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png" width="96" height="22"></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 13, 2020 at 2:50 PM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">We talked at the CI/CD meeting about Fedora Zuul and also I talked to some of their developers. We're concerned about some of the extra costs that we'd incur by using it instead of Github Actions. For one, we'd have to set up and maintain our own compute resource..<div><div><br></div><div>I went ahead and updated the Github Actions epic in redmine:</div><div><br></div><div><a href="https://pulp.plan.io/issues/6065" target="_blank">https://pulp.plan.io/issues/6065</a></div><div><br></div><div>If there's no objections, I'd like to merge the Github Actions PRs for ansible-pulp and pulp_rpm_prerequisites PRs on February 18th to start testing out Github Actions.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div>David<br></div></div></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 12, 2020 at 11:59 AM Fabricio Aguiar <<a href="mailto:fabricio.aguiar@redhat.com" target="_blank">fabricio.aguiar@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">bringing in some data about CI,<div><br></div><div>Last month, we had a considerable increase in total builds and in queue time:</div><div><div><div><img src="cid:ii_k6jjzt8i2" alt="image.png" style="margin-right: 0px;"><br></div></div></div><div> <a href="https://travis-ci.org/pulp?tab=insights" target="_blank">https://travis-ci.org/pulp?tab=insights</a><br></div><div><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><div><img src="cid:ii_k6jjyqag1" alt="image.png" style="margin-right: 0px;"><br></div><div> <a href="https://travis-ci.com/pulp?tab=insights" target="_blank">https://travis-ci.com/pulp?tab=insights</a><br></div></div><div dir="ltr"><br>Best regards,</div><div dir="ltr"><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize">Fabricio</span><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize"> </span><span style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:14px;font-weight:700;text-transform:capitalize">Aguiar</span><div>Software Engineer, Pulp Project</div><div><a href="https://www.redhat.com/" style="color:rgb(0,136,206);font-family:RedHatText,sans-serif;font-size:12px;margin:0px" target="_blank">Red Hat Brazil - Latam</a><br></div><div>+55 11 999652368</div><div><img src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png" width="96" height="22"></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 11, 2020 at 3:35 PM Mike DePaulo <<a href="mailto:mikedep333@redhat.com" target="_blank">mikedep333@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Here are my set of thoughts on many things mentioned.</div><div><br></div><div>TL;DR: We still need to run CI on CentOS/Fedora, but using cloud instances of CentOS/Fedora (interacted with via SSH/Ansible from the GHA Ubuntu client VM) might be preferable to using Fedora CI for certain tests.<br></div><div><br></div><div>1. "We should test GHA via the ansible-pulp related repos now, and then come up with a thorough & quick schedule to migrate from Travis to GHA entirely, resources permitting."</div><div>I totally agree with this.<br></div><div><br></div><div>2. "We must use Fedora/Centos CI for SELinux policy testing at all, because Travis & GHA use Ubuntu, whose kernel doesn't support SELinux."</div><div>I do not think this is correct. I've researched this, but haven't test it.<br></div><div>SELinux upstream seems to run their CI on Ubuntu:</div><div><a href="https://github.com/SELinuxProject/selinux/blob/master/.travis.yml" target="_blank">https://github.com/SELinuxProject/selinux/blob/master/.travis.yml</a> <br></div><div><a href="https://github.com/SELinuxProject/selinux-testsuite/blob/master/.travis.yml#L53" target="_blank">https://github.com/SELinuxProject/selinux-testsuite/blob/master/.travis.yml#L53</a></div><div>How did they make this work?</div><div>My 1 theory is that Ubuntu's kernel has support for both SELinux and AppArmor, but Travis slims down the image so that AppArmor does not get enabled like on a default Ubuntu install. So SELinux can be enabled at runtime.</div><div>My 2nd theory is that enough of a shim (the 2nd link on fake-selinuxfs in particular) is sufficient to avoid reboot.</div><div>However, #4 still negates this option.<br></div><div><br></div><div>3. "We must use Fedora/CentOS CI because pulp-certguard's dependencies are an issue on Ubuntu"</div><div>The plugin-template uses our Fedora containers w/ pulp-operator & k3s. Isn't this sufficient?</div><div><br></div><div><div>4. "We must use Fedora/Centos CI for SELinux policy testing with pulp-rpm / pulp-certguard. Containers will provide the pulp-rpm / pulp-container deps on ubuntu, but break SELinux testing because SELinux wraps around the entire container."<br></div><div>This is a bigger concern. My research confirms this, even lxc does not support SELinux policies *within* the container. Our SELinux policies currently support the plugins Katello is integrating: pulp-container, pulp-file, pulp-rpm, and pulp-certguard. We could still do CI testing of the 1st 2 on Ubuntu though.<br></div></div><div><br></div><div>5. "Our CI must run be either capable of running entirely on CentOS/Fedora CI, or have only certain tests run on them."</div><div>I am more in favor of the latter, but there is another possible solution for SELinux testing. We could have an Amazon EC2 account, openstack account, etc that GHA or Travis calls out to. Ansilbe molecule has drivers for many cloud compute types:</div><div><a href="https://molecule.readthedocs.io/en/2.22/configuration.html#driver" target="_blank">https://molecule.readthedocs.io/en/2.22/configuration.html#driver</a></div><div> It would create a Fedora / CentOS instance specifically for testing via cloud APIs, and then run our ansible installer (SSH) against it from the GHA / Travis instance, then delete it at the end. It would mean that the Pulp Project would still have no persistent infrastructure.<br></div><div><br></div><div>6. On using Fedora with their Zuul CI instance:</div><div>This looks promising, but having read their PDF, I am concerned that Fedora's instance would be specifically configured for their use case in ways that we can reconfigure. Hopefully we can. Some of their use case is integration with pagure rather than GitHub, Koji artifact storage integration, etc:</div><div><a href="https://fedoraproject.org/w/uploads/1/1e/CI_CD_for_Fedora_packaging_with_Zuul_-_final_-_with_notes.pdf" target="_blank">https://fedoraproject.org/w/uploads/1/1e/CI_CD_for_Fedora_packaging_with_Zuul_-_final_-_with_notes.pdf</a></div><div><br></div><div>-Mike<br></div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 10, 2020 at 8:29 AM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Is this the Software Factory instance of Zuul[0]? I can reach out to them and see if it would make sense as an option for Pulp.<div><br></div><div>[0] <a href="https://softwarefactory-project.io/zuul/t/local/status" target="_blank">https://softwarefactory-project.io/zuul/t/local/status</a><br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div>David<br></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Feb 9, 2020 at 11:51 AM Neal Gompa <<a href="mailto:ngompa13@gmail.com" target="_blank">ngompa13@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:"times new roman",serif;font-size:small"><span style="font-family:Arial,Helvetica,sans-serif">On Sun, Feb 9, 2020 at 9:46 AM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:</span><br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks Brian and Daniel. I agree on the points you both raised. <div><br></div><div>Brian, to you specific questions/points:</div><div><br></div><div>## We need details on each piece of the Travis workflow, where it will be ported to, and a rough estimate of how long each piece would take. I think these things would make a great EPIC.</div><br><div>I have a Github Actions epic. I plan to update it this week based on our conversation and will add more specific details, estimates, etc. I'll respond when it's ready for review.<br></div><div><div><br></div><div><a href="https://pulp.plan.io/issues/6065" target="_blank">https://pulp.plan.io/issues/6065</a></div><div><br></div><div><br></div><div>## Who will work on it? It needs I think 2 fully dedicated people who already completely understand the Travis stuff in detail. It's too hard for one person and would take too long...</div><div><br></div><div>I definitely agree we need at least 2 people to work on this. We need as many people as possible to understand Github Actions.</div><div><br></div><div>I don't know who has time for this right now. I imagine it'll probably have to wait until next sprint (Sprint 67). Or at least I personally won't have time until next week at the earliest. That'll give us time to plan though. </div><div><br></div><div>In the meantime, I'd consider letting the installer team merge Fabricio's ansible-pulp PR[0]. This will also alleviate much of the immediate need and let us begin collecting real world data/experience as well.</div><div><br></div></div></div></blockquote><div><br></div><div><div class="gmail_default" style="font-family:"times new roman",serif;font-size:small">Has anyone reached out to the Fedora CI team about using their Zuul instance? Perhaps they've got an easy automated process for using that on projects. Zuul can spin up either Fedora or CentOS environments, which should satisfy the need for being able to test esoteric things like FIPS mode while also being able to get fresh environments and dependencies through Fedora environments.</div><div class="gmail_default" style="font-family:"times new roman",serif;font-size:small"><br></div><div class="gmail_default" style="font-family:"times new roman",serif;font-size:small">It may be better to consider using Fedora CI over CentOS CI due to the better system overall, too...</div><br></div><div> </div></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr">真実はいつも一つ！/ Always, there's only one truth!<br></div></div></div> </blockquote></div> _______________________________________________<br> Pulp-dev mailing list<br> <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br> </blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:capitalize"><span>Mike</span> <span>DePaulo</span><span style="text-transform:uppercase;color:rgb(170,170,170);margin:0px"></span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px 0px 4px;text-transform:capitalize">He / Him / His</p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px;text-transform:capitalize"><span>Service Reliability Engineer, Pulp</span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px 0px 4px;font-size:12px"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat<span></span></a></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-bottom:4px"></div><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px;font-size:12px"><span>IM: <span>mikedep333</span></span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px"><span>GPG: 51745404</span></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-top:12px"><table border="0"><tbody><tr><td width="100px"><a href="https://www.redhat.com/" target="_blank"><img src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png" width="90" height="auto"></a></td></tr></tbody></table></div></div></div></div> </blockquote></div> </blockquote></div> </blockquote></div> </blockquote></div> _______________________________________________<br> Pulp-dev mailing list<br> <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br> </blockquote></div> _______________________________________________<br> Pulp-dev mailing list<br> <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br> </blockquote></div>