- Thanks and good luck From: Lachlan Musicman <datakid@gmail.com> To: spacewalk-list@redhat.com Date: 03/31/2016 07:31 PM Subject: [Spacewalk-list] Kickstart, Config management and debugging Sent by: spacewalk-list-bounces@redhat.com <hr noshade> This email originated from outside of the company. Please use discretion if opening attachments or clicking on links.<hr> The other thread got confused, so I thought I'd move to a new one since I now have better questions (I think). Problem remains the same - I have a kickstart profile linked with a configuration channel that works after first boot, but the files are not being uploaded during the installation and must be loaded via a hack (thanks William) in a post install script. From my most recent installation (5 mins ago): Before the post install script hack, I run this command: date > /root/verify_configs.out rhncfg-client verify >> /root/verify_configs.out and get this: Fri Apr 1 10:43:57 AEDT 2016 Using server name emts-res-utils1 modified /etc/cntlm.conf missing /etc/cron.d/15min missing /etc/cron.quarter_hour/check_nfs_mounts.sh missing /etc/munge/munge.key modified /etc/netconfig missing /etc/profile.d/proxy.sh missing /etc/samba/user missing /etc/slurm/slurm.conf missing /etc/slurm/slurmdbd.conf modified /etc/sysctl.conf >From this I can see that 7 files are missing. What do the three "modified" files represent? Does modified represent that : - "the default installation files are in place, they are different from what's in the Configuration Channel in question, so we represent this as modified" OR - "we have uploaded these files in particular, but not the others"? Modified means that the file exists but it does not match what exists in the configuration channel. These files, should be part of the rpms that are installed during initial kickstart. Find the rpms: "rpm -qf {path/to/configfile} or yum whatprovides {path/to/configfile}". I noticed in the ks-rhn-post.log that there were a bunch of attempts to get to the centos mirror list (I have no idea why that is happening and would rather it didn't tbh) that were failing because of IPv6 problems. IPv6 isn't supported in house (out of my control) and part of my config change is that IPv6 is disabled. We faced the same issue. Even if you "disable"or remove all of the files in /etc/yum.repos.d/, they could still be enabled or added back via the centos-release rpm. It would be nice, if registered to a spacewalk, that these would just be disabled or removed. Also, if a server would have access to the public CentOS repos, anything newer that would be available in the public yum CentOS repo could override the spacewalk channel as it would be newer. In addition this work around provides some security/administration benefits (In our opinion). Any third party application defaults to /etc/yum.repos.d/, thus they will not be able to install anything without an admin blessing the conf file. Besides the notes below, we found that by doing this reduced the amount of time to build a server because it doesn't attempt and wait for failures. The work around that can be put in place is a post nochroot script prior to registration and server actions. %post -nochroot <tt># Whenever centos-release rpm is updated, will lay down new files in /etc/yum.repos.d/ # This has caused issues with updates and with rhn_check in the registration. # Creating a new directory and updating yum.conf </tt> mkdir -p <tt>/mnt/sysimage/</tt>etc/yum.repos.d/{mynewdir} echo "reposdir=/etc/yum.repos.d/{mynewdir}" >> <tt>/mnt/sysimage</tt>/etc/yum.conf Obv, with those files being uploaded in a post script, it's not happening in time for the cobbler.ks script. So I put --noipv6 in the network line of my Advanced Options, but that has obviously failed to do what I expected it to do - as I can see because the ks-rhn-post.log still makes an attempt to connect via ipv6 to the centos mirrors. (by the by, we have a proxy, so the curl would most likely fail anyway - the cntlm file I'm uploading would solve that). If --noipv6 is not working (did not work for me either), however we still pass it just in case. %post script. The following normally works, except for Cent7. sed -i -e 's/NETWORKING_IPV6=yes/NETWORKING_IPV6=no/' /etc/sysconfig/network #However based off of Red hat documentation(solution 8709) and the google on how to disable ipv6 This is what comes out of it. #Note: <tt>ie: ntpd, ssh, X11 forwarding, and bonding are some of the items that could break per the google.</tt> <tt> #This is not exact code. Make sure you backup files.</tt> Linux7: sed -i "s/GRUB_CMDLINE_LINUX=\"$.*$\"/GRUB_CMDLINE_LINUX=\"\1 ipv6.disable=1\"/" /etc/default/grub grub2-mkconfig -o /boot/grub2/grub.cfg Linux6: echo "options ipv6 disable=1" >> /etc/modprobe.d/ipv6.conf chkconfig ip6tables off sed -i "/NETWORKING_IPV6/d" /etc/sysconfig/network echo "NETWORKING_IPV6=no" >> /etc/sysconfig/network Linux5: chkconfig ip6tables off sed -i "/alias net-pf-10 ipv6/d" /etc/modprobe.conf echo "alias net-pf-10 off" >> /etc/modprobe.conf echo "options ipv6 disable=1" >> /etc/modprobe.conf sed -i "/NETWORKING_IPV6/d" /etc/sysconfig/network echo "NETWORKING_IPV6=no" >> /etc/sysconfig/network Linux4: sed -i "/alias net-pf-10 ipv6/d" /etc/modprobe.conf echo "alias net-pf-10 off" >> /etc/modprobe.conf If Not Linux4 #This is to address X11 forwarding. Disabling ipv6 breaks it. sed -i 's/^[[:space:]]*::/#::/' /etc/hosts sed -i "s/#AddressFamily any/AddressFamily inet/g" /etc/ssh/sshd_config When I look in the Kickstart File tab of the profile in question, I see a script getting run that I haven't put in, which I presume is the one on my Scripts tab called "Registration and server actions". In that section, I noticed that there is a small stanza: # begin Red Hat management server registration mkdir -p /usr/share/rhn/ wget <a href="http://emts-res-utils1/pub/RHN-ORG-TRUSTED-SSL-CERT">http://emts-res-utils1/pub/RHN-ORG-TRUSTED-SSL-CERT</a>\ -O /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT perl -Xnpe 's/RHNS-CA-CERT/RHN-ORG-TRUSTED-SSL-CERT/g' -i /etc/sysconfig/rhn/* if [ -f /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release ]; then rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release fi key=1-xxx,1-xxy,1-xyy,1-yyy if [ -f /tmp/key ]; then key=`cat /tmp/key`,$key fi rhnreg_ks --serverUrl=<a href="https://emts-res-utils1/XMLRPC">https://emts-res-utils1/XMLRPC</a>\ --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=$key # end Red Hat management server registration I presume it is in here that the error is occurring. I also note that another error in this part of the script is that if [ -f /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release ]; then will always fail because all our keys are of the format /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Can I just edit the Kickstart File in that profile tab, or is there another way I might get some output into a log file to see where the problem is? An if this is a cobbler snippet, can I just edit the appropriate file in /var/lib/cobbler/snippets or is that not recommended? Do not modify those files. They will be overwritten when you update Spacewalk (It is painful, because you upgrade then it breaks and you have to remember to restore that file). You can validate that the GPG key is installed. gpg --quiet --with-fingerprint /etc/pki/rpm-gpg/RP-GPG-KEY-CentOS-7 Output: pub...... /####x##x# Description key fingerprint = sub..... The following will print out the key if it is installed on the new system. rpm -q gpg-pubkey --qf '%{version}\n' |grep -i ####x##x# If you want to ensure that it is installed: Upload the RPM-GPG-KEY-CentOS-7 to the spacewalk under System->Kickstarts->GPG and SSL Keys. Then associate that GPG key to each associated OS kickstart profile under {ksprofile}-> System Details -> GPG & SSL. That will guarantee that the key is imported. You will be able to see it in the kickstart. cheers L. ------ The most dangerous phrase in the language is, "We've always done it this way." - Grace Hopper<tt>This email originated from outside of the company. Please use discretion if opening attachments or clicking on links. _______________________________________________ Spacewalk-list mailing list Spacewalk-list@redhat.com </tt><a href="https://www.redhat.com/mailman/listinfo/spacewalk-list"><tt>https://www.redhat.com/mailman/listinfo/spacewalk-list</tt></a> ** This email and any attachments may contain information that is confidential and/or privileged for the sole use of the intended recipient. Any use, review, disclosure, copying, distribution or reliance by others, and any forwarding of this email or its contents, without the express permission of the sender is strictly prohibited by law. If you are not the intended recipient, please contact the sender immediately, delete the e-mail and destroy all copies. **