<html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> Hello, There was a strange issue this morning with sf-rpm-build job, the job failed with the following errors: <pre><a name="_2018-05-10_15_09_19_908300" class="date" href="https://softwarefactory-project.io/logs/42/12142/18/check/sf-rpm-build/3ad66b9/job-output.txt.gz#_2018-05-10_15_09_19_908300">2018-05-10 15:09:19.908300</a> | mock-host | DEBUG util.py:417: <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/updates/x86_64/Packages/ncurses-5.9-14.20130511.el7_4.x86_64.rpm">http://mirror.centos.org/centos/7/updates/x86_64/Packages/ncurses-5.9-14.20130511.el7_4.x86_64.rpm</a>: [Errno 14] HTTP Error 404 - Not Found <a name="_2018-05-10_15_09_19_908644" class="date" href="https://softwarefactory-project.io/logs/42/12142/18/check/sf-rpm-build/3ad66b9/job-output.txt.gz#_2018-05-10_15_09_19_908644">2018-05-10 15:09:19.908644</a> | mock-host | DEBUG util.py:417: <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/updates/x86_64/Packages/ncurses-base-5.9-14.20130511.el7_4.noarch.rpm">http://mirror.centos.org/centos/7/updates/x86_64/Packages/ncurses-base-5.9-14.20130511.el7_4.noarch.rpm</a>: [Errno 14] HTTP Error 404 - Not Found ... </pre> The packages are not in the updates target but in os. I found these packages came from built-release, and built release point to our koji instance. I did not find any documentation about how we manage our mirrors when centos release a new version, but I found a command to regen-repo in bash_history and read the koji doc about this option [1]: "koji doesn’t monitor external repositories for changes. new repositories will be generated when packages you build land in a tag that populates the buildroot or you manually regenerate the repository. you should be sure to regularly regenerate the repositories manually to pick up updates." I executed this command for sf-master-el7-build and sf-master-centos7-build, but it failed for sf-master-centos7-build: [kojiadmin@koji ~]$ koji regen-repo sf-master-centos7-build Regenerating repo for tag: sf-master-centos7-build Created task: 14277 Task info: <a class="moz-txt-link-freetext" href="https://koji.softwarefactory-project.io/koji/taskinfo?taskID=14277">https://koji.softwarefactory-project.io/koji/taskinfo?taskID=14277</a> Watching tasks (this may be safely interrupted)... 14277 newRepo (sf-master-centos7-build): free 14277 newRepo (sf-master-centos7-build): free -> open (koji.softwarefactory-project.io) 14278 createrepo (x86_64): open (koji.softwarefactory-project.io) 14277 newRepo (sf-master-centos7-build): open (koji.softwarefactory-project.io) -> FAILED: GenericError: failed to merge repos: /usr /libexec/kojid/mergerepos --tempdir /tmp/koji/tasks/4278/14278 -a x86_64 -b /mnt/koji/repos/sf-master-centos7-build/1968/x86_64/bloc klist -o /tmp/koji/tasks/4278/14278/repo -g /mnt/koji/repos/sf-master-centos7-build/1968/groups/comps.xml -r <a class="moz-txt-link-freetext" href="file:///tmp/koji/tasks/">file:///tmp/koji/tasks/</a> 4278/14278/repo_1968_premerge/ -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/">http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/</a> -r <a class="moz-txt-link-freetext" href="http://download.fedoraproject">http://download.fedoraproject</a>. org/pub/epel/7/x86_64/ -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/extras/x86_64/">http://mirror.centos.org/centos/7/extras/x86_64/</a> -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/updates/x86_64/">http://mirror.centos.org/centos/7/updates/x86_64/</a> -r h ttp://mirror.centos.org/centos/7/os/x86_64/ exited with status 1 0 free 1 open 0 done 1 failed 14278 createrepo (x86_64): open (koji.softwarefactory-project.io) -> FAILED: GenericError: failed to merge repos: /usr/libexec/koj id/mergerepos --tempdir /tmp/koji/tasks/4278/14278 -a x86_64 -b /mnt/koji/repos/sf-master-centos7-build/1968/x86_64/blocklist -o /tm p/koji/tasks/4278/14278/repo -g /mnt/koji/repos/sf-master-centos7-build/1968/groups/comps.xml -r <a class="moz-txt-link-freetext" href="file:///tmp/koji/tasks/4278/14278/r">file:///tmp/koji/tasks/4278/14278/r</a> epo_1968_premerge/ -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/">http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/</a> -r <a class="moz-txt-link-freetext" href="http://download.fedoraproject.org/pub/epel">http://download.fedoraproject.org/pub/epel</a> /7/x86_64/ -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/extras/x86_64/">http://mirror.centos.org/centos/7/extras/x86_64/</a> -r <a class="moz-txt-link-freetext" href="http://mirror.centos.org/centos/7/updates/x86_64/">http://mirror.centos.org/centos/7/updates/x86_64/</a> -r <a class="moz-txt-link-freetext" href="http://mirror">http://mirror</a> .centos.org/centos/7/os/x86_64/ exited with status 1 0 free 0 open 0 done 2 failed 14277 newRepo (sf-master-centos7-build) failed I will try to regen sf-master-centos7-build when centos repo sync will be finished (centos was not officially released yet). What do you think about add a crontab to execute this command every day ? Also, we need to update our koji instance, but I will wait monday to do the update. Have a nice day. Nico [1] <pre></pre> </body> </html>