[Libguestfs] *** buffer overflow detected *** accessing invalid FD in libguestfs

[Richard W.M. Jones]

On Mon, Feb 24, 2020 at 01:49:38PM +0200, Veselin Kozhuharski wrote:
> We have extended collectd virt plugin to extract info about disk usage from
> a libvirt domain using libguestfs. In addition to my previous mail I am
> attaching some more infomration about the problem.
> Currently the collectd plugin works fine and retrieves the required
> statistics. The problem that I face happens after certain number of cycles
> (getting disk usage statistics). Collectd is terminated with the following
> error:
> 
> Feb 20 15:09:36 tve50 collectd[17720]: *** buffer overflow detected ***:
> /usr/sbin/collectd terminated
> 
> This happens after about 490-500 calls of my statistics read function.
> Collectd is terminated with signal SIGABRT with the following backtrace:
> 
> (gdb) bt
> #0  0x00007ffff71f2e97 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007ffff71f4801 in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x00007ffff723d897 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #3  0x00007ffff72e8cff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #4  0x00007ffff72e8d21 in __fortify_fail () from
> /lib/x86_64-linux-gnu/libc.so.6
> #5  0x00007ffff72e6a10 in __chk_fail () from /lib/x86_64-linux-gnu/libc.so.6
> #6  0x00007ffff72e8c0a in __fdelt_warn () from
> /lib/x86_64-linux-gnu/libc.so.6
> #7  0x00007ffff47ed8ba in loop (cmd=cmd at entry=0x7fffbc02abe0) at
> ../../../lib/command.c:662
> #8  0x00007ffff47ee2f5 in guestfs_int_cmd_run (cmd=0x7fffbc02abe0) at
> ../../../lib/command.c:767
> #9  0x00007ffff47efefc in disk_create_qcow2 (optargs=0x7fffbc008d60,
> backingfile=0x7fffbc09ef30
> "/var/lib/nova/instances/5ca86029-d296-4261-9a67-908bdd6c4eab/disk",
> size=-1,
>     orig_filename=0x7fffbc0301f0 "/tmp/libguestfszZtmRI/overlay1.qcow2",
> g=0x7fffbc008d60) at ../../../lib/create.c:348
> #10 guestfs_impl_disk_create (g=g at entry=0x7fffbc008d60,
> filename=filename at entry=0x7fffbc0301f0
> "/tmp/libguestfszZtmRI/overlay1.qcow2", format=<optimized out>,
>     format at entry=0x7ffff4843998 "qcow2", size=size at entry=-1,
> optargs=optargs at entry=0x7fffca7fb1a0) at ../../../lib/create.c:88
> #11 0x00007ffff4794ca5 in guestfs_disk_create_argv (g=g at entry=0x7fffbc008d60,
> filename=filename at entry=0x7fffbc0301f0
> "/tmp/libguestfszZtmRI/overlay1.qcow2",
>     format=format at entry=0x7ffff4843998 "qcow2", size=size at entry=-1,
> optargs=optargs at entry=0x7fffca7fb1a0) at ../../../lib/actions-3.c:224
> #12 0x00007ffff4807b18 in create_cow_overlay_direct (g=0x7fffbc008d60,
> datav=<optimized out>, drv=0x7fffbc03a890) at
> ../../../lib/launch-direct.c:89
> #13 0x00007ffff47f0b44 in create_overlay (g=0x7fffbc008d60,
> drv=0x7fffbc03a890) at ../../../lib/drives.c:87
> #14 0x00007ffff47f0d7b in create_drive_file (g=g at entry=0x7fffbc008d60,
> data=data at entry=0x7fffca7fb2a0) at ../../../lib/drives.c:119
> #15 0x00007ffff47f1c55 in guestfs_impl_add_drive_opts
> (g=g at entry=0x7fffbc008d60,
> filename=<optimized out>,
>     filename at entry=0x7fffbc10d540
> "/var/lib/nova/instances/5ca86029-d296-4261-9a67-908bdd6c4eab/disk",
> optargs=optargs at entry=0x7fffca7fb420) at ../../../lib/drives.c:826
> #16 0x00007ffff4784927 in guestfs_add_drive_opts_argv
> (g=g at entry=0x7fffbc008d60,
> 
>     filename=filename at entry=0x7fffbc10d540
> "/var/lib/nova/instances/5ca86029-d296-4261-9a67-908bdd6c4eab/disk",
> optargs=optargs at entry=0x7fffca7fb420) at ../../../lib/actions-2.c:180
> #17 0x00007ffff48128e0 in add_disk (g=g at entry=0x7fffbc008d60,
> filename=0x7fffbc10d540
> "/var/lib/nova/instances/5ca86029-d296-4261-9a67-908bdd6c4eab/disk",
> format=<optimized out>,
>     readonly_in_xml=<optimized out>, protocol=<optimized out>,
> server=<optimized out>, username=0x0, secret=0x0, datavp=0x7fffca7fb610) at
> ../../../lib/libvirt-domain.c:396
> #18 0x00007ffff4813cd6 in for_each_disk (f=0x7ffff48127e0 <add_disk>,
> data=0x7fffca7fb610, doc=0x7fffbc10cf80, conn=0x7fffbc01fa10,
> g=0x7fffbc008d60)
>     at ../../../lib/libvirt-domain.c:782
> #19 guestfs_impl_add_libvirt_dom (g=g at entry=0x7fffbc008d60,
> domvp=domvp at entry=0x7fffbc064000, optargs=optargs at entry=0x7fffca7fb790) at
> ../../../lib/libvirt-domain.c:323
> #20 0x00007ffff47ab2c3 in guestfs_add_libvirt_dom_argv
> (g=g at entry=0x7fffbc008d60,
> dom=dom at entry=0x7fffbc064000, optargs=optargs at entry=0x7fffca7fb790) at
> ../../../lib/actions-4.c:174
> #21 0x00007ffff4812cf6 in guestfs_impl_add_domain (g=g at entry=0x7fffbc008d60,
> domain_name=domain_name at entry=0x7fffbc0338e0 "tve50:00000013",
> optargs=optargs at entry=0x7fffca7fb8c0)
>     at ../../../lib/libvirt-domain.c:163
> #22 0x00007ffff4760368 in guestfs_add_domain_argv (g=0x7fffbc008d60,
> dom=0x7fffbc0338e0 "tve50:00000013", optargs=optargs at entry=0x7fffca7fb8c0)
> at ../../../lib/actions-0.c:139
> #23 0x00007ffff47dfc38 in guestfs_add_domain_va (g=<optimized out>,
> dom=<optimized out>, args=args at entry=0x7fffca7fb920) at
> ../../../lib/actions-variants.c:107
> #24 0x00007ffff47dfee4 in guestfs_add_domain (g=g at entry=0x7fffbc008d60,
> dom=dom at entry=0x7fffbc0338e0 "tve50:00000013") at
> ../../../lib/actions-variants.c:45
> #25 0x00007ffff4a78bec in refresh_lists (inst=inst at entry=0x7ffff4c7f940
> <lv_read_user_data>) at src/virt.c:2049
> #26 0x00007ffff4a7a327 in lv_read (ud=<optimized out>) at src/virt.c:1656
> #27 0x0000555555564a1c in plugin_read_thread (args=<optimized out>) at
> src/daemon/plugin.c:540
> #28 0x00007ffff79b66db in start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #29 0x00007ffff72d588f in clone () from /lib/x86_64-linux-gnu/libc.so.6
> (gdb) quit
> 
> Deeper analysis shows the error is related to file descriptors handling
> using FD_* macros in loop() function, most probably trying to access fd
> which is outside the valid range.

It's hard to say from the information so far.  What is the
file descriptor number when it crashes?

It's unfortunate that this loop is using select instead of
poll, so perhaps rewriting it to use poll would be useful.

Rich.

> Any hints how to avoid or handle this situation would be highly appreciated.
> 
> -- 
> 
> 
> *Veselin Kozhuharski** |* Software Engineer
> 
> Direct: +359 2 439 2590 ext. 3912 *|* Mobile: +359 887 412116 |
> veselin_k*@telco.com
> <mzabaruk at telco.com>*
> 
> *Telco Systems | **www.telco.com <http://www.telco.com/>*
> 
> Follow us: *LinkedIn <http://www.linkedin.com/company/telco-systems>*
> | *Twitter
> <http://twitter.com/TelcoSystems>* | *Facebook
> <https://www.facebook.com/TelcoSystems>* | *YouTube
> <http://www.youtube.com/TelcoSystems>* | *Blog <http://www.telco.com/blog>*
> |

> _______________________________________________
> Libguestfs mailing list
> Libguestfs at redhat.com
> https://www.redhat.com/mailman/listinfo/libguestfs


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html