<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Besides, If it didn't work as root or qemu, then you simply didn't
get the configuration setup correctly.<br>
<br>
I advise you to get it working correctly first (via opening another
shell and verifying that the limits are set by default)<br>
before embarking on a change to libvirt.<br>
<pre class="moz-signature" cols="72">/*
* Michael R. Hines
* Platform Engineer, DigitalOcean.
*/</pre>
<div class="moz-cite-prefix">On 02/19/2016 04:37 PM, Roy Shterman
wrote:<br>
</div>
<blockquote
cite="mid:CAOBqJ56NSUtPNQVFs6H4R_gDFghjC4mBwqNgKgpK7SAAqw4RkQ@mail.gmail.com"
type="cite">
<div dir="ltr">Yes,
<div><br>
</div>
<div>I tried also running it as root user and it also didn't
worked.</div>
<div><br>
</div>
<div>Do you know where libvirt (or QEMU) gets the value for
process MEMLOCK? maybe i can change this value in libvirt
code?</div>
<div><br>
</div>
<div>Regards,</div>
<div>Roy</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Feb 19, 2016 at 11:15 PM,
Michael R. Hines <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mhines@digitalocean.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mhines@digitalocean.com">mhines@digitalocean.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Is the QEMU process
(after startup) actually running as the QEMU userid ?<br>
<pre cols="72">/*
* Michael R. Hines
* Platform Engineer, DigitalOcean.
*/</pre>
<div>
<div class="h5">
<div>On 02/19/2016 02:43 PM, Roy Shterman wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5">
<div dir="ltr">First off all thank you for your
answer,
<div><br>
</div>
<div>I couldn't figured how to start virtual
machine with increased MEMLOCK,</div>
<div><br>
</div>
<div>tried to add into /etc/security/limits.d </div>
<div><br>
</div>
<div>
<div>qemu soft memlock 3221225</div>
<div>qemu hard memlock 3221225</div>
</div>
<div><br>
</div>
<div>so max locked-in-memory will be 3G, but it
didn't worked.</div>
<div><br>
</div>
<div>still has MEMLOCK of 60kb per each VM.</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">Maybe you can spot what
I'm doing wrong?</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Feb 9, 2016 at
5:16 PM, Michael R. Hines <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:michael@hinespot.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:michael@hinespot.com">michael@hinespot.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">Hi Roy,<span><br>
<br>
On 02/09/2016 03:57 AM, Roy Shterman
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex"> Hi,<br>
<br>
I tried to understand the rdma-migration
in qemu code and i have two questions
about it:<br>
<br>
1. I'm working with qemu-kvm using
libvirt and i'm getting<br>
<br>
MEMLOCK max locked-in-memory address
space 65536 65536 bytes<br>
<br>
in qemu process so I don't understand
how can you use rdma-pin-all with such
low MEMLOCK.<br>
<br>
I found a solution in libvirt to lock
all vm memory in advance and to enlarge
MEMLOCK.<br>
It uses memoryBacking locking and memory
tuning hard_limit of vm memory but I
couldn't find a usage of this in
rdma-migration code.<br>
<br>
</blockquote>
<br>
</span> You're absolutey right, the RDMA
migration code itself doesn't set this lock
limit explicitly because there are
system-wide restrictions in both appArmour,<br>
/etc/security, as well as SELINUX that
restrict applications from arbitrarily
setting their maximum memory lock limits.<br>
<br>
The other problem is CGROUPS: If someone
sets a cgroup control for maximum memory and
forgets about that mlock() limits, then<br>
there will be a conflict.<br>
<br>
So, libvirt must have a policy to deal with
all of these possibilities, not just handle
a special case for RDMA migration.<br>
<br>
The only way "simple" way (without patching
the problems above) to apply a higher lock
limit to QEMU is to set the ulimit for
libvirt<br>
(or for QEMU if starting QEMU manually) in
your environment or the command line with $
ulimit # before attempting the migration,<br>
then the RDMA subsystem will be able to lock
the memory successfully.<br>
<br>
The other option is to use
/etc/security/limits.conf and set the option
for a specific libvirt process user and make
sure your libvirt/qemu<br>
are not running as root.<br>
<br>
QEMU itself also has a "mlock" option built
into the command line, but it also suffers
from the same problem --- you have to find<br>
a way (currently) to increase the limit
before using the option.<span><br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex"> 2. Do you
have any comparison of IOPS and
bandwidth between TCP migration and rdma
migration?<br>
<br>
</blockquote>
</span> Yes, lots of comparisons.<br>
<br>
<a moz-do-not-send="true"
href="http://wiki.qemu.org/Features/RDMALiveMigration"
rel="noreferrer" target="_blank">http://wiki.qemu.org/Features/RDMALiveMigration</a><br>
<a moz-do-not-send="true"
href="http://www.canturkisci.com/ETC/papers/IBMJRD2011/preprint.pdf"
rel="noreferrer" target="_blank">http://www.canturkisci.com/ETC/papers/IBMJRD2011/preprint.pdf</a><br>
<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex"> Regards,<br>
Roy<br>
<br>
<br>
</blockquote>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<span class="HOEnZb"><font color="#888888">
<pre>--
libvir-list mailing list
<a moz-do-not-send="true" href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a>
<a moz-do-not-send="true" href="https://www.redhat.com/mailman/listinfo/libvir-list" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a></pre>
</font></span></blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>