[Libguestfs] [PATCH] Improve memory management of nbdkit python plugin example

Carl-Daniel Hailfinger c-d.hailfinger.devel.2006 at gmx.net
Mon Sep 26 15:07:41 UTC 2016


Hi,

the nbdkit python plugin example has suboptimal memory management:
- it creates the disk image as a string on init
- it casts the string to bytearray on every read
- it copies the string before and the string after the written region,
then reassembles those pieces together with the written region to a new
disk image string

This is not a problem as long as the image is small, but in my tests
with a 5 GB sized image nbdkit already used 15 GB RAM directly after
startup, and even more (20-25 GB) on the first write.

This changes the code to use bytearray everywhere and use the proper
methods to change bytearray objects directly. With the patch applied,
nbdkit with a 5 GB image will still only use 5 GB RAM even during heavy
read/write activity.

Regards,
Carl-Daniel

diff -r 521366d1854b -r d7d5078d08c7 plugins/python/example.py
--- a/plugins/python/example.py	Sun Jul 10 17:10:30 2016 +0100
+++ b/plugins/python/example.py	Sun Sep 25 05:04:02 2016 +0200
@@ -29,7 +29,7 @@
 # reconnect to the same server you should see the same disk.  You
 # could also put this into the handle, so there would be a fresh disk
 # per handle.
-disk = "\0" * (1024*1024);
+disk = bytearray(1024 * 1024)
 
 # This just prints the extra command line parameters, but real plugins
 # should parse them and reject any unknown parameters.
@@ -50,9 +50,9 @@
 
 def pread(h, count, offset):
     global disk
-    return bytearray (disk[offset:offset+count])
+    return disk[offset:offset+count]
 
 def pwrite(h, buf, offset):
     global disk
     end = offset + len (buf)
-    disk = disk[:offset] + buf + disk[end:]
+    disk[offset:end] = buf




More information about the Libguestfs mailing list