[Spacewalk-list] database usage - size - io performance - repo sync - repo re-sync update times

[Michael Mraka]

Kevin Abbey wrote:
% Hi,
% 
% I've installed spacewalk 2.2 and used the util script to create
% channels.  I completed one repo sync for fedoa 20 updates which took
% 1 day and 9 hours.  I noticed that the iowait was ~4 - 6% the entire
% time.  When complete the db files or postgres were ~3GB.
% 
% 
% *Questions*
% 
% 1.  Will the sync take this long each time or only the 1st initial
% sync?  It is not clear to me how much io will be repeated during an
% update sync.

Hi Kevin,

It will download only newly discovered packages next time so it will be
much faster.

% 2.  The database and memory usage is ok.  The server is centos 6.5
% as a guest kvm vm on a centos host.  The underlying file system is a
% raid 10 with 4  10k sas drives 600GB each, ~1.1TB, 18% used .  The
% file system is xfs on the host, with a qcow2 disk for the vm, and
% ext4 within the guest.  The driver in the guest is virtio.  Is there
% a better fs/driver combination configuration?

Raw partition (either physical or lvm logical volume) on host would be
better than any image on the filesystem (one filesystem layer
overhead less).
Also raw disk instead of qcow2 is a bit better choice.

% I also notices that the iowait in the host is ~50% less then iowait
% in the guest vm.  The vm disk was created to be 124GB and 15GB were
% pre-allocated.
% 
% ------------------------------------------------------------------------
% Controller information
% ------------------------------------------------------------------------
%   Controller type                         : SAS2008
%   BIOS version                            : 7.11.10.00
%   Firmware version                        : 7.15.08.00
% ------------------------------------------------------------------------
% IR Volume information
% ------------------------------------------------------------------------
% IR volume 1
%   Volume ID                               : 78
%   Volume Name                             : VD_1
%   Status of volume                        : Okay (OKY)
%   RAID level                              : RAID10
%   Size (in MB)                            : 1143552
% ------------------------------------------------------------------------
% Model Number for each disk                : WD6001BKHG
% ------------------------------------------------------------------------
% 
% 
% 
% 3.  I am tempted to add an ssd pcie card or an external ssd based
% disk for the postgres database and give it to the vm as a
% passthrough device.  Is there any recommendation for sizing,
% configuration to handle the io required?  The channels are listed
% below.
% 
% 
% 4. Is it possible for fedora to provide a database dump for the
% repo-sync that could just be imported?  Would this be more
% efficient?  I assume this is not helpful since it has not been done
% and I should just add a file system and hardware to meet the io
% requirement.
% 
% 
% 5. Could anyone offer a recommendation for capacity and io for the
% database specifically?  The rpms reside on an nfs share external to
% the vm.

AFAIK in most cases download part (i.e. network bandwidth) is the
bottleneck not the database performance.

% 
% 6. Is it likely to take about a month to sync the repos listed
% below?  And the sync - updated processes?
% 
% 
% The spacewalk requirements documented seem to need more elaboration
% on these details, especially the database io (unless my system is
% not normal and sub-par standard.).  This scenario is for testing and
% I'd like to scale it properly.  Advice is requested.
% 
% Thank you,
% Kevin

Regards,


--
Michael Mráka
Satellite Engineering, Red Hat