[Spacewalk-list] database usage - size - io performance - repo sync - repo re-sync update times

[Kevin Abbey]

Hi,

I've installed spacewalk 2.2 and used the util script to create 
channels.  I completed one repo sync for fedoa 20 updates which took 1 
day and 9 hours.  I noticed that the iowait was ~4 - 6% the entire 
time.  When complete the db files or postgres were ~3GB.


*Questions*

1.  Will the sync take this long each time or only the 1st initial 
sync?  It is not clear to me how much io will be repeated during an 
update sync.


2.  The database and memory usage is ok.  The server is centos 6.5 as a 
guest kvm vm on a centos host.  The underlying file system is a raid 10 
with 4  10k sas drives 600GB each, ~1.1TB, 18% used .  The file system 
is xfs on the host, with a qcow2 disk for the vm, and ext4 within the 
guest.  The driver in the guest is virtio.  Is there a better fs/driver 
combination configuration?
I also notices that the iowait in the host is ~50% less then iowait in 
the guest vm.  The vm disk was created to be 124GB and 15GB were 
pre-allocated.

------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
   Controller type                         : SAS2008
   BIOS version                            : 7.11.10.00
   Firmware version                        : 7.15.08.00
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
IR volume 1
   Volume ID                               : 78
   Volume Name                             : VD_1
   Status of volume                        : Okay (OKY)
   RAID level                              : RAID10
   Size (in MB)                            : 1143552
------------------------------------------------------------------------
Model Number for each disk                : WD6001BKHG
------------------------------------------------------------------------



3.  I am tempted to add an ssd pcie card or an external ssd based disk 
for the postgres database and give it to the vm as a passthrough 
device.  Is there any recommendation for sizing, configuration to handle 
the io required?  The channels are listed below.


4. Is it possible for fedora to provide a database dump for the 
repo-sync that could just be imported?  Would this be more efficient?  I 
assume this is not helpful since it has not been done and I should just 
add a file system and hardware to meet the io requirement.


5. Could anyone offer a recommendation for capacity and io for the 
database specifically?  The rpms reside on an nfs share external to the vm.


6. Is it likely to take about a month to sync the repos listed below?  
And the sync - updated processes?



The spacewalk requirements documented seem to need more elaboration on 
these details, especially the database io (unless my system is not 
normal and sub-par standard.).  This scenario is for testing and I'd 
like to scale it properly.  Advice is requested.

Thank you,
Kevin



=======================================================================
spacewalk-repo-sync --channel


fedora20-x86_64
fedora20-x86_64-updates

centos6-x86_64
centos6-x86_64-updates
epel6-centos6-x86_64

centos7-x86_64
centos7-x86_64-updates
epel7-centos7-x86_64

centos6-x86_64-addons
centos6-x86_64-centosplus
centos6-x86_64-contrib
centos6-x86_64-extras
centos6-x86_64-fasttrack

centos7-x86_64-addons
centos7-x86_64-centosplus
centos7-x86_64-contrib
centos7-x86_64-extras
centos7-x86_64-fasttrack

spacewalk-nightly-client-fedora20-x86_64
spacewalk-nightly-server-fedora20-x86_64
spacewalk-nightly-client-centos6-x86_64
spacewalk-nightly-server-centos6-x86_64

=======================================================================




spacewalk-repo-sync --channel fedora20-x86_64
#### Channel label: fedora20-x86_64 ####
Repo URL: 
https://mirrors.fedoraproject.org/metalink?repo=fedora-20&arch=x86_64
Packages in repo:             38597
Packages already synced:          0
Packages to sync:             38597
1/38597 : vala-devel-0.22.1-1.fc20-0.i686
.........
..............



spacewalk-repo-sync --channel fedora20-x86_64-updates
=======================================================================

#### Channel label: fedora20-x86_64-updates ####
Repo URL: 
https://mirrors.fedoraproject.org/metalink?repo=updates-released-f20&arch=x86_64
Packages in repo:             18004
Packages already synced:          0
Packages to sync:             18004
1/18004 : autocorr-is-4.2.5.2-6.fc20-1.noarch
2/18004 : texlive-everypage-doc-svn15878.1.1-5.fc20-3.noarch
........
................
........


18002/18004 : bigloo-libs-4.1a-4.2.fc20-0.i686
18003/18004 : libinfinity-gtk-devel-0.5.5-1.fc20-0.x86_64
18004/18004 : kwin-gles-libs-4.11.10-2.fc20-0.x86_64
Linking packages to channel.
Repo 
https://mirrors.fedoraproject.org/metalink?repo=updates-released-f20&arch=x86_64 
has comps file 
b3eecc6977ab971945f5fcbf548e472869ba1af75e354160e03c79f70e965493-comps-f20.xml.gz.
ERROR: failure: 
repodata/a558b061a3004e402bf0b2396b3a797d9182dbf2dcb3abd1be90fd3c11698974-updateinfo.xml.gz 
from fedora20-x86_64-updates: [Errno 256] No more mirrors to try.
Sync completed.
Total time: 1 day, 9:42:52
[root at spacewalk ~]#




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20140725/564cac0c/attachment.htm>