[Ovirt-devel] [PATCH] Test System Updates
Alan Pevec
apevec at redhat.com
Wed Jul 23 23:55:12 UTC 2008
From: Mohammed Morsi <mmorsi at redhat.com>
updates to oVirt for new testing stuff so far. Included are
1. autobuild.sh script, incorporating suggestions made by Hugh, Jim, and others.
2. common-pkgs.ks, I had to add the autobuild packages here since build-all.sh uses pungi and this kickstart to generate the repo from which the appliance installs packages
3. wui-devel.ks, changes to the wui developer appliance kickstart to get the 2nd stage autobuild working
4. ovirt-autobuild, place in /etc/sysconfig on the host and set values accordingly to what you need set in the second stage auto-build.conf
5. application.rb, since rake test doesn't go through httpd to run the unit / functional tests, we need to set a fixed user that doesn't depend on kerberos to run all the tests under
6. StatsData.rb, change to Mark's Stats utility that sets the data source to a demo/test directory when the rails environment is in test mode
7. demo-rrd-data.rb, a small ruby script I wrote to generate a kickstart file containing demo/test rrd data collected from the local /var/lib/collectd/rrd dir. I have a data kickstart file generated from my local data, ready to be added to the repo to automatically add some test data to the wui appliance when its created, but its pretty large (eg ~105MB) and I didn't want to clutter up anyone's email (or make the initial pull from the ovirt repo take forever). Perhaps at some point we can truncate this to give us a little less data.
-Mo
---
autobuild.sh | 78 ++++++++++-
wui-appliance/common-pkgs.ks | 3 +
wui-appliance/wui-devel.ks | 237 ++++++++++++++++++++++++++++++++
wui/conf/ovirt-autobuild | 6 +
wui/src/app/controllers/application.rb | 18 ++-
wui/src/app/util/stats/Stats.rb | 10 +-
wui/src/script/demo-rrd-data.rb | 43 ++++++
7 files changed, 381 insertions(+), 14 deletions(-)
create mode 100644 wui/conf/ovirt-autobuild
create mode 100644 wui/src/script/demo-rrd-data.rb
diff --git a/autobuild.sh b/autobuild.sh
index ccae050..1148df6 100755
--- a/autobuild.sh
+++ b/autobuild.sh
@@ -1,6 +1,74 @@
-#oVirt wui autobuild script
+#oVirt autobuild script
+#
+# Copyright (C) 2008 Red Hat, Inc.
+# Written by Mohammed Morsi <mmorsi at redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+# MA 02110-1301, USA. A copy of the GNU General Public License is
+# also available at http://www.gnu.org/copyleft/gpl.html.
+#
+# NOTE StrictHostKeyChecking must be set to "no" in /etc/ssh/sshd_config for this to work
-#run tests
-cd wui/src/
-rake db:migrate
-rake test
+echo "Running oVirt Autobuild"
+
+# determine if we're on the wui appliance or not
+hn=`hostname`
+if [ ! "$hn" == "management.priv.ovirt.org" ]; then
+
+ # create appliance
+ ./build-all.sh -ac
+
+ # start appliance
+ virsh start ovirt-appliance
+
+ # wait until started
+ sleep 180
+
+ # the appliance's ssh key changed
+ rm -f /root/.ssh/known_hosts
+
+ # substitue values on the appliance autobuild w/
+ # that stored in a local config
+ [ -r /etc/sysconfig/ovirt-autobuild ] && . /etc/sysconfig/ovirt-autobuild
+ OAB_ADMIN_EMAIL="${ADMIN_EMAIL:-root at localhost.localdomain}"
+ OAB_ADMIN_NAME="${ADMIN_NAME:-root}"
+ OAB_GROUP_EMAIL="${GROUP_EMAIL:-root at localhost.localdomain}"
+ OAB_GROUP_NAME="${GROUP_NAME:-root}"
+ OAB_HOSTNAME="${HOSTNAME:-localhost.localdomain}"
+
+
+ echo 'Starting oVirt Wui Autobuild'
+ autobuild_conf=/etc/auto-build.d/auto-build.conf
+ ssh root at 192.168.50.2 "sed -i \
+ -e \"s/ADMIN_EMAIL/$OAB_ADMIN_EMAIL/g\" \
+ -e \"s/ADMIN_NAME/$OAB_ADMIN_NAME/g\" \
+ -e \"s/GROUP_EMAIL/$OAB_GROUP_EMAIL/g\" \
+ -e \"s/GROUP_NAME/$OAB_GROUP_NAME/g\" \
+ -e \"s/HOSTNAME/$OAB_HOSTNAME/g\" $autobuild_conf && \
+ auto-build-make-root && \
+ auto-build --config $autobuild_conf"
+
+ # mount the results in an httpd accessible location
+ mkdir -p /var/www/html/builder-wui/
+ sshfs -o allow_other 192.168.50.2:/var/lib/builder/public_html/ /var/www/html/builder-wui/
+
+else
+
+ # verify a successful response when trying to access the ovirt site
+ curl -i --negotiate -u : management.priv.ovirt.org/ovirt/ | grep "HTTP/1.1 200 OK"
+
+ # enter wui dir and run tests
+ cd wui/src/
+ rake test
+fi
diff --git a/wui-appliance/common-pkgs.ks b/wui-appliance/common-pkgs.ks
index 7a8866c..27db288 100644
--- a/wui-appliance/common-pkgs.ks
+++ b/wui-appliance/common-pkgs.ks
@@ -25,3 +25,6 @@ cobbler
bind-utils
augeas
/usr/sbin/lokkit
+perl-Test-AutoBuild
+perl-Test-AutoBuild-git
+perl-Class-MethodMaker
diff --git a/wui-appliance/wui-devel.ks b/wui-appliance/wui-devel.ks
index e36f3a7..da9b5d3 100644
--- a/wui-appliance/wui-devel.ks
+++ b/wui-appliance/wui-devel.ks
@@ -282,4 +282,241 @@ EOF
chmod +x /etc/init.d/ovirt-wui-dev
chkconfig ovirt-wui-dev on
+mkdir -p /root/.ssh/
+cat > /root/.ssh/authorized_keys << \EOF
+ssh-dss AAAAB3NzaC1kc3MAAACBAMPlOUpZU6DV8JhlFnglAF5FUtbCoKqY1pwdQkwUQXk67lXojWdEnWSkTKmRA/nm9SArEb6d7luJxi4CSiaSEmr3+L4H40mI2fKqilkPjA46rU/v/xm+DlQnGpH9M9HuBwglQD5iCjMKOh3qbK3JF6NTzREfIghIovbf11fiad0RAAAAFQC6FLsgY9PPHXVrCiOkRb24TLLsAwAAAIB3nqhR5TQJDLQgfjH46SEv6oBg85PKEVDfwhhGkQwxyL9jaMN/SGMV1dn7FuWk0In1r0vKMTHLW5AtARgT32OD+tReSTG0hWCcYWCm4LkeoOHijRZKTCbQo7smV2o65IeVaZ7Oetqpjc7MZlZETutrnVHjC/E8p7Igf417KBDuNQAAAIB1M8YkiYoyqHORXUNVA+72TXtS/NJbYwPa0zRGCKNaKERvRIMasCq4ZuGLIb4yMZzV+3j+iOhYEOw6q7y2kuDG5lo8cMECz1rOqyhxLxBlqtNrJjFJeiYL585b5z+mHNvMryjDQKJWk1isURjlyt7LoEQHBDV4lIbdEcyF0D83tg== root at management.priv.ovirt.org
+EOF
+chmod 640 /root/.ssh/authorized_keys
+
+mkdir -p /etc/auto-build.d/
+cat > /etc/auto-build.d/auto-build.conf << \EOF
+#
+# Test-AutoBuild main configuration file.
+#
+# Short documentation can be found inline, and for further
+# information consult the manual page 'man 5 auto-build.conf'
+#
+
+# Path to the build engine definition. This determines the
+# sequence of stages executed by the build engine, package
+# types to publish, and other miscellaneous aspects of the
+# build engine's runtime environment.
+# It will typically suffice to leave this on the default
+# setting
+engine = /etc/auto-build.d/engine/host-build.conf
+# If you want to use SELinux MAC, then uncomment this
+# and run with '/usr/bin/auto-build-secure' instead
+#engine = /etc/auto-build.d/engine/host-build-secure.conf
+
+# The location of the directory containing the template files
+# for the HTML status pages, and email alert messages
+templateDir = /etc/auto-build.d/templates
+
+# The directory under which the build engine will run. This
+# should match whatever path was provided when running the
+# 'auto-build-make-root' script. /var/lib/builder is the default
+# directory created by the RPM perl-Test-AutoBuild-account
+buildRoot = /var/lib/builder
+
+# A name for the build instance. This is used in the HTML
+# status pages, email alert subject, and RSS feeds
+label = Continous Automatic Builder
+
+# Name and email address of the build administrator
+adminEmail = ADMIN_EMAIL
+adminName = ADMIN_NAME
+
+# Name and email address of the development team writing the
+# software being built. Typically point it to the main developer
+# mailing list
+groupEmail = GROUP_EMAIL
+groupName = GROUP_NAME
+
+# If the build host has multiple names, then specify which
+# one is used for the virtual host publishing the HTML status
+# pages. This will be used to construct links in the RSS feed
+# and email alert messages. If not set, defaults to the primary
+# hostname of the machine
+#hostname = example.com
+hostname = HOSTNAME
+
+# The prefix under which the HTML status pages are located.
+# If using /etc/auto-build.d/httpd/aliased.conf, then
+#httpPrefix = /builder
+# Else with /etc/auto-build.d/httpd/user.conf, then
+#httpPrefix = /~builder
+# Finally with /etc/auto-build.d/httpd/vhost.conf, then
+httpPrefix =
+
+# Where to save a log of the build engine progress (this
+# data is also sent to STDERR, but when run from cron this
+# typically ends up in /dev/null)
+engineLogFile = /var/lib/builder/autobuild.log
+
+# The method used for acquiring a lock file to prevent multiple
+# build instances running at the same time against the same
+# buildRoot directory.
+# * fcntl - Best option, but only implemented for Linux, SunOS
+# and FreeBSD
+# * flock - Portable to any UNIX, but does not completely protect
+# if buildRoot is on an NFS volume
+# * file - Simple file creation/deletion test. If the build engine
+# or host machine crashes, it will be neccessary to manually
+# delete the lock file ($buildRoot/.build.mutex)
+lockMethod = fcntl
+
+# Define what high level features you want enabled
+features = {
+ # If the cache is enabled, then modules will only be built if their
+ # sources have changed since the previous build cycle. This provides
+ # much faster cycle times if there are many modules in the build
+ # config, only a few of which ever change
+ cache = 1
+
+ # Whether to try and checkout the latest sources from version control
+ # If checkout is disabled, the build will run with whatever sources
+ # were checked out on the previous cycle.
+ checkout = 1
+
+ # Whether to run the 'createrepo' tool against the RPMs on the HTTP
+ # distribution site
+ createrepo_index = 1
+
+ # Whether to run the 'yum-arch' tool against the RPMs on the HTTP
+ # distribution site
+ yum_index = 0
+
+ # Whether to generate an APT index of RPMs / Debian packages on the
+ # HTTP distribution site
+ apt_index = 0
+
+ # Whether to send email alerts at the end of cycles
+ email_alerts = 0
+}
+
+# Describe 'interesting' metadata about the build platform
+platform = {
+}
+
+# Criteria for sending email alerts
+alert = {
+ # If the scope is set to
+ # * builder - one email is sent with info on all modules
+ # * module - one email is sent for each module
+ scope = builder
+
+ # Conditions for sending alerts
+ # * always - send regardless of status
+ # * fail - send whenever the status is 'failed'
+ # * first-fail - send on the first failure only
+ trigger = first-fail
+
+ # If scope is set to 'builder' then
+ #
+ # * admin - use the global address defined for $adminEmail
+ # * group - use the global address defined for $groupEmail
+ #
+ # If scope is set to 'module' then
+ #
+ # * admin - use the module specific address defined for $adminEmail
+ # * group - use the module specific address defined for $groupEmail
+ #
+ # Or just set an explicit email address (separate multiple address with ,)
+ to = group
+
+ # What to set the 'From' address to
+ #from = builder at example.com
+
+ # The SMTP server to relay the mail via, defaults to localhost
+ #smtpServer = mail.example.com
+}
+
+# Set the maximum disk space allowed for use by the cache
+maxCacheSize = 100M
+
+# Set the maximum number of days to keep a cache around before
+# considering it expired
+maxCacheAge = 7d
+
+
+# Module groups (for web status)
+groups = {
+ virt = {
+ label = Virtualization Modules
+ }
+}
+
+# Global environment variables which will be set whenever
+# any command is run by the build engine
+env = {
+ USER = builder
+ PKG_CONFIG_PATH = /var/lib/builder/install-root/lib/pkgconfig/
+ PYTHONPATH = /var/lib/builder/install-root/lib/python2.5/site-packages
+ LD_LIBRARY_PATH = /var/lib/builder/install-root/lib
+}
+
+# Define the source control repositories from which modules
+# will be checked out. There are 7 supported repository types
+# at this time. Depending on the 'type' parameter, various
+# 'env' or 'option' parameters may be required
+repositories = {
+ ovirt = {
+ label = oVirt
+ type = git
+ options = {
+ base-url = git://git.et.redhat.com/ovirt.git
+ }
+ }
+}
+
+# Now, the list of modules to checkout from the repositories
+# defined above.
+modules = {
+ ovirt--devel = {
+ label = oVirt (Development Branch)
+
+ source = {
+ repository = ovirt
+ path = :next
+ }
+
+ groups = (
+ virt
+ )
+
+ links = (
+ {
+ href = http://git.et.redhat.com/?p=ovirt.git;a=summary
+ label = Browse Source Repository
+ }
+ {
+ href = http://HOSTNAME/builder
+ label = oVirt Host Test Results (first stage of oVirt autobuild)
+ }
+ )
+
+ artifacts = (
+ {
+ src = wui/src/*
+ dst = wui/
+ label = oVirt Wui
+ }
+ {
+ src = coverage/*
+ dst = coverage
+ label = Code coverage metrics
+ }
+ )
+ }
+}
+
+# Optionally, the packages (RPMs, etc) generated during the
+# build can be built into an ISO image for distribution
+isos = {
+}
+
+# That's all folks!
+EOF
+
%end
+
diff --git a/wui/conf/ovirt-autobuild b/wui/conf/ovirt-autobuild
new file mode 100644
index 0000000..5cf210e
--- /dev/null
+++ b/wui/conf/ovirt-autobuild
@@ -0,0 +1,6 @@
+# these parameters are set in the 2nd stage autobuild configuration file
+ADMIN_EMAIL="root at localhost.localdomain"
+ADMIN_NAME="root"
+GROUP_EMAIL="root at localhost.localdomain"
+GROUP_NAME="root"
+HOSTNAME="localhost.localdomain"
diff --git a/wui/src/app/controllers/application.rb b/wui/src/app/controllers/application.rb
index aa8fd7c..ec151ae 100644
--- a/wui/src/app/controllers/application.rb
+++ b/wui/src/app/controllers/application.rb
@@ -1,4 +1,4 @@
-#
+#
# Copyright (C) 2008 Red Hat, Inc.
# Written by Scott Seago <sseago at redhat.com>
#
@@ -29,14 +29,18 @@ class ApplicationController < ActionController::Base
before_filter :pre_new, :only => [:new]
before_filter :pre_create, :only => [:create]
before_filter :pre_edit, :only => [:edit, :update, :destroy]
- before_filter :pre_show, :only => [:show, :show_vms, :show_users,
+ before_filter :pre_show, :only => [:show, :show_vms, :show_users,
:show_hosts, :show_storage]
before_filter :authorize_admin, :only => [:new, :create, :edit, :update, :destroy]
def get_login_user
- user_from_principal(request.env["HTTP_X_FORWARDED_USER"])
+ if ENV["RAILS_ENV"] != 'test'
+ user_from_principal(request.env["HTTP_X_FORWARDED_USER"])
+ else
+ 'ovirtadmin'
+ end
end
-
+
def user_from_principal(principal)
principal.split('@')[0]
end
@@ -91,8 +95,8 @@ class ApplicationController < ActionController::Base
# don't define find_opts for array inputs
def json_list(full_items, attributes, arg_list=[], find_opts={})
page = params[:page].to_i
- paginate_opts = {:page => page,
- :order => "#{params[:sortname]} #{params[:sortorder]}",
+ paginate_opts = {:page => page,
+ :order => "#{params[:sortname]} #{params[:sortorder]}",
:per_page => params[:rp]}
arg_list << find_opts.merge(paginate_opts)
item_list = full_items.paginate(*arg_list)
@@ -102,7 +106,7 @@ class ApplicationController < ActionController::Base
json_hash[:rows] = item_list.collect do |item|
item_hash = {}
item_hash[:id] = item.id
- item_hash[:cell] = attributes.collect do |attr|
+ item_hash[:cell] = attributes.collect do |attr|
if attr.is_a? Array
value = item
attr.each { |attr_item| value = value.send(attr_item)}
diff --git a/wui/src/app/util/stats/Stats.rb b/wui/src/app/util/stats/Stats.rb
index f6ced4b..83d5d7b 100644
--- a/wui/src/app/util/stats/Stats.rb
+++ b/wui/src/app/util/stats/Stats.rb
@@ -24,6 +24,13 @@ require 'util/stats/StatsData'
require 'util/stats/StatsDataList'
require 'util/stats/StatsRequest'
+# set path to rrd stats to use
+if ENV["RAILS_ENV"] != 'test'
+ $rrdBase="/var/lib/collectd/rrd/"
+else
+ $rrdBase="/var/lib/collectd/rrd-test/"
+end
+
# This fetches a rolling average, basically average points before and after.
@@ -210,8 +217,7 @@ def fetchData?(node, devClass, instance, counter, startTime, duration, interval,
start = (sTime / interval).to_i * interval
endTime = (eTime / interval).to_i * interval
- rrdBase="/var/lib/collectd/rrd/"
- rrdNode=rrdBase + node + "/"
+ rrdNode=$rrdBase + node + "/"
# Now we need to mess a bit to get the right combos
case devClass
diff --git a/wui/src/script/demo-rrd-data.rb b/wui/src/script/demo-rrd-data.rb
new file mode 100644
index 0000000..636bd5b
--- /dev/null
+++ b/wui/src/script/demo-rrd-data.rb
@@ -0,0 +1,43 @@
+# generate kickstart file containing demo rrd data
+#
+# Copyright (C) 2008 Red Hat, Inc.
+# Written by Mohammed Morsi <mmorsi at redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+# MA 02110-1301, USA. A copy of the GNU General Public License is
+# also available at http://www.gnu.org/copyleft/gpl.html.
+#
+
+require 'find'
+
+rrddir = '/var/lib/collectd/rrd'
+filename = ARGV.size > 0 ? ARGV[0] : '/tmp/demo-rrd-data'
+
+dfile = File.new(filename, "w")
+
+Find.find(rrddir) do |path|
+ if ! FileTest.directory?(path)
+ sfile = File.new(path, "r")
+ dfile.puts 'cat > ' + path + '<< \EOF'
+ while (str = sfile.gets) != nil
+ dfile.puts str
+ end
+ dfile.puts 'EOF'
+ dfile.puts
+ else
+ dfile.puts 'mkdir -p ' + path
+ end
+end
+
+# TODO change /var/lib/collectd/rrd to /var/lib/collectd/rrd-test when writing
--
1.5.5.1
More information about the ovirt-devel
mailing list