[Libguestfs] Issue with downloading files whose path contains multi-byte utf-8 characters

Tue Feb 14 19:19:31 UTC 2023

Thanks Richard!
The patches are similar to what we did locally, and applying them fixed the
issue with the actual use-case (not just the sanitized repro).

On Tue, Feb 14, 2023 at 8:10 PM Richard W.M. Jones <rjones at redhat.com>
wrote:

> On Tue, Feb 14, 2023 at 08:02:59PM +0200, Yonatan Shtarkman wrote:
> > The attached code snippet reproduces the issue when running:
> > python3 libugestfs_segfault_repro.py
> >
> > With:
> > guestfs version: 1.48.6
> > guestfs-python version: 1.48.6
>
> Can confirm with {python3-,}libguestfs-1.51.1-1.fc39.x86_64
>
> The stack trace is roughly the same as yours.
>
> Rich.
>
> > Stacktrace:
> > #0  0x00007f6dd17b4701 in _Py_INCREF (op=<optimized out>) at
> /usr/include/
> > python3.9/object.h:408
> > 408    op->ob_refcnt++;
> > (gdb) bt
> > #0  0x00007f6dd17b4701 in _Py_INCREF (op=<optimized out>) at
> /usr/include/
> > python3.9/object.h:408
> > #1  guestfs_int_py_event_callback_wrapper
> >     (g=<optimized out>, flags=<optimized out>, array_len=0, array=0x0,
> buf_len=
> > 43, buf=0xd1dc90 "commandrvf: udevadm --debug settle -E /\347\232\204\303
> > available\r\ned7
> > #2  guestfs_int_py_event_callback_wrapper
> >     (g=<optimized out>, callback=0x7f6dd19ec040, event=16,
> event_handle=0,
> > flags=<optimized out>, buf=0xd1dc90 "commandrvf: udevadm --debug settle
> -E /\
> > 347\232\204\303 4
> > #3  0x00007f6dd166f9aa in guestfs_int_call_callbacks_message
> >     (g=0xae06f0, event=16, buf=0xd1dc90 "commandrvf: udevadm --debug
> settle -E
> > /\347\232\204\303 available\r\ned to rvf:e run  in, ignor data av",
> buf_len=43)
> >     at events.c:117
> > #4  0x00007f6dd168560e in guestfs_int_log_message_callback
> >     (g=g at entry=0xae06f0, buf=0xd1dc90 "commandrvf: udevadm --debug
> settle -E /\
> > 347\232\204\303 available\r\ned to rvf:e run  in, ignor data av",
> len=len at entry
> > =43)
> >     at proto.c:145
> > #5  0x00007f6dd166ac29 in handle_log_message (g=g at entry=0xae06f0, conn=
> > conn at entry=0xd19570) at conn-socket.c:395
> > #6  0x00007f6dd166b233 in read_data (len=4, bufv=<optimized out>, connv=
> > <optimized out>, g=<optimized out>) at conn-socket.c:179
> > #7  read_data (g=0xae06f0, connv=0xd19570, bufv=<optimized out>, len=4)
> at
> > conn-socket.c:142
> > #8  0x00007f6dd1685c2a in recv_from_daemon (buf_rtn=0x7fffa95cc548,
> size_rtn=
> > 0x7fffa95cc544, g=0xae06f0) at proto.c:545
> > #9  guestfs_int_recv_from_daemon (g=g at entry=0xae06f0,
> size_rtn=size_rtn at entry=
> > 0x7fffa95cc544, buf_rtn=buf_rtn at entry=0x7fffa95cc548) at proto.c:623
> > #10 0x00007f6dd168603a in guestfs_int_recv
> >     (g=g at entry=0xae06f0, fn=fn at entry=0x7f6dd16aa308 "download",
> hdr=hdr at entry=
> > 0x7fffa95cc610, err=err at entry=0x7fffa95cc5e0, xdrp=xdrp at entry=0x0, ret=
> > ret at entry=0x0)
> >     at proto.c:668
> > #11 0x00007f6dd163825e in guestfs_download (g=g at entry=0xae06f0,
> remotefilename=
> > <optimized out>, filename=0x7f6dd17edea0 "/tmp/1") at actions-5.c:2984
> > #12 0x00007f6dd17ab01a in guestfs_int_py_download (self=<optimized out>,
> args=
> > <optimized out>) at actions-5.c:794
> >
> >
> >
> > On Mon, Feb 13, 2023 at 10:16 PM Richard W.M. Jones <rjones at redhat.com>
> wrote:
> >
> >     On Mon, Feb 13, 2023 at 09:38:50PM +0200, Yonatan Shtarkman wrote:
> >     > Also, as a workaround, I avoided calling the event callback if
> null is
> >     returned
> >     > by Py_BuildValue (still print the error and release the thread).
> >     > This seems to work for our usage (we only use the event callbacks
> for
> >     logging),
> >     > any potential issues I'm missing?
> >
> >     Oh I see so the issue is actually when logging and not in the
> download
> >     call?  That's strange that it should be happening intermittently.
> I'd
> >     expect a UTF-8 encoding error would happen every time not sometimes.
> >
> >     Rich.
> >
> >     --
> >     Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/
> >     ~rjones
> >     Read my programming and virtualization blog:
> http://rwmj.wordpress.com
> >     virt-top is 'top' for virtual machines.  Tiny program with many
> >     powerful monitoring features, net stats, disk stats, logging, etc.
> >     http://people.redhat.com/~rjones/virt-top
> >
> >
>
> > import guestfs
> > from guestfs import GuestFS
> >
> > g = GuestFS(python_return_dict=True)
> > def callback(event, _event_handle, buf: str, array):
> >       pass
> >
> > g.set_event_callback(callback, guestfs.EVENT_ALL)
> > g.set_verbose(True)
> > path = '/tmp/tmp_disk'
> > g.disk_create(path, format='raw', size=32*1024*1024)
> > g.add_drive(path)
> > g.launch()
> > g.mkfs('ext4', '/dev/sda')
> > g.mount('/dev/sda', '/')
> > for _ in range(20000):
> >     try:
> >        g.download ('/的ó', '/tmp/1')
> >     except:
> >        pass
> >
>
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-builder quickly builds VMs from scratch
> http://libguestfs.org/virt-builder.1.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20230214/d84c85ed/attachment.htm>