[fedora-virt] suggestion: a service for tuning ksm according to memory load

[Dan Kenigsberg]

If you are running a number of qemu-kvm's with similar guests, you can
gain a lot of memory by using ksm properly.

An unattended host running a variable number of qemu-kvm's needs to tune
ksm automatically, since when memory is tight, it's better to spend more
cpu on merging pages. In more relaxed cases, it's just a waste of time.

The attached service tries to do just that.

It monitors how much memory is used by qemu-kvm processes, and starts
ksm when a threshold is passed. Ksm usually manages to free up some
memory.

As long as memory used by qemu is above the defined threshold, ksm tries
harder and harder to share memory pages (up to a limit). This may happen
if a guest starts working and consumes new memory. If there's enough
free memory, ksm cools down.

Ksmd service has the usual start/status/stop verbs, and an additional
one: signal. One should use that verb just after one starts a new
qemu-kvm process or just after such process dies, to let ksm adjust
immediately.

Comments and suggestion are welcome.

Thanks,

Dan.
-------------- next part --------------
#!/bin/bash
#
# Copyright 2009 Red Hat, Inc. and/or its affiliates.
# Released under the GPL
#
# ksmd         Kernel Samepage Merging Daemon
#
# chkconfig: - 85 15
# description: The Kernel Samepage Merging control Daemon is a simple script  \
#	       that controls whether (and with what vigor) should ksm search  \
#              duplicated pages.
# processname: ksmd
# config: /etc/ksmd.conf
# pidfile: /var/run/ksmd.pid
#
### BEGIN INIT INFO
# Provides: ksmd
# Required-Start:
# Required-Stop:
# Should-Start:
# Short-Description: tune the speed of ksm
# Description: The Kernel Samepage Merging control Daemon is a simple script
#   that controls whether (and with what vigor) should ksm search duplicated
#   memory pages.
#   needs testing and ironing. contact danken at redhat.com if something breaks.
### END INIT INFO

###########################
if [ -f /etc/ksmd.conf ]; then
    . /etc/ksmd.conf
fi

KSM_MONITOR_INTERVAL=${KSM_MONITOR_INTERVAL:-60}
KSM_NPAGES_BOOST=${KSM_NPAGES_BOOST:-300}
KSM_NPAGES_DECAY=${KSM_NPAGES_DECAY:--50}

KSM_NPAGES_MIN=${KSM_NPAGES_MIN:-64}
KSM_NPAGES_MAX=${KSM_NPAGES_MAX:-1250}
# microsecond sleep between ksm scans for 16Gb server. Smaller servers sleep
# more, bigger sleep less.
KSM_SLEEP=${KSM_SLEEP:-10000}

KSM_THRES_COEF=${KSM_THRES_COEF:-20}
KSM_THRES_CONST=${KSM_THRES_CONST:-2048}

total=`awk '/^MemTotal:/ {print $2}' /proc/meminfo`
[ -n "$DEBUG" ] && echo total $total

npages=0
sleep=$[KSM_SLEEP * 16 * 1024 * 1024 / total]
[ -n "$DEBUG" ] && echo sleep $sleep
thres=$[total * KSM_THRES_COEF / 100]
if [ $KSM_THRES_CONST -gt $thres ]; then
    thres=$KSM_THRES_CONST
fi
[ -n "$DEBUG" ] && echo thres $thres

KSMCTL () {
    if [ -x /usr/bin/ksmctl ]; then
        /usr/bin/ksmctl $*
    else
        case x$1 in
            xstop)
                echo 0 > /sys/kernel/mm/ksm/run
                ;;
            xstart)
                echo $2 > /sys/kernel/mm/ksm/pages_to_scan
                echo $3 > /sys/kernel/mm/ksm/sleep
                echo 1 > /sys/kernel/mm/ksm/run
                ;;
        esac
    fi
}

committed_memory () {
    # calculate how much memory is committed to running qemu processes
    local progname
    progname=${1:-qemu}
    ps -o vsz `pgrep $progname` | awk '{ sum += $1 }; END { print sum }'
}

increase_napges() {
    local delta
    delta=${1:-0}
    npages=$[npages + delta]
    if [ $npages -lt $KSM_NPAGES_MIN ]; then
        npages=$KSM_NPAGES_MIN
    elif [ $npages -gt $KSM_NPAGES_MAX ]; then
        npages=$KSM_NPAGES_MAX
    fi
    echo $npages
}

adjust () {
    local free committed
    free=`awk '/^MemFree:/ { free += $2}; /^Buffers:/ {free += $2}; /^MemCached:/ {free += $2}; END {print free}' /proc/meminfo`
    committed=`committed_memory`
    [ -n "$DEBUG" ] && echo committed $committed free $free
    if [ $[committed + thres] -lt $total -a $free -gt $thres ]; then
        KSMCTL stop
        [ -n "$DEBUG" ] && echo "$[committed + thres] < $total and free > $thres, stop ksm"
        return 1
    fi
    [ -n "$DEBUG" ] && echo "$[committed + thres] > $total, start ksm"
    if [ $free -lt $thres ]; then
        npages=`increase_napges $KSM_NPAGES_BOOST`
        [ -n "$DEBUG" ] && echo "$free < $thres, boost"
    else
        npages=`increase_napges $KSM_NPAGES_DECAY`
        [ -n "$DEBUG" ] && echo "$free > $thres, decay"
    fi
    KSMCTL start $npages $sleep
    [ -n "$DEBUG" ] && echo "KSMCTL start $npages $sleep"
    return 0
}

loop () {
    while true
    do
        sleep $KSM_MONITOR_INTERVAL
        adjust
    done
}

###########################
. /etc/rc.d/init.d/functions

prog=ksmd
pidfile=${PIDFILE-/var/run/ksmd.pid}
RETVAL=0

start() {
    echo -n $"Starting $prog: "
    daemon --pidfile=${pidfile} $0 loop
    RETVAL=$?
    echo
    return $RETVAL
}

stop() {
    echo -n $"Stopping $prog: "
    killproc -p ${pidfile}
    RETVAL=$?
    echo
}

signal () {
    pkill -P `cat ${pidfile}` sleep
}

case "$1" in
  start)
	start
	;;
  stop)
	stop
	;;
  status)
        status -p ${pidfile} $prog
	RETVAL=$?
	;;
  restart)
	stop
	start
	;;
  signal)
	signal
	;;
  loop)
        RETVAL=1
	if [ -w `dirname ${pidfile}` ]; then
            loop &
            echo $! > ${pidfile}
            RETVAL=$?
        fi
	;;
  *)
	echo $"Usage: $prog {start|stop|status|signal|help}"
	RETVAL=3
esac

exit $RETVAL