[Libguestfs] nbdkit error: "write reply: NBD_CMD_WRITE: Broken pipe"

[Laszlo Ersek]

On 05/02/22 16:36, Nikolaus Rath wrote:
> On May 02 2022, Laszlo Ersek <lersek at redhat.com> wrote:
>> On 05/01/22 18:35, Nikolaus Rath wrote:
>>> Hi,
>>>
>>> I am developing a new nbdkit plugin, and occasionally I am getting
>>> errors like this:
>>>
>>> |nbdkit: s3backer.8: error: write reply: NBD_CMD_WRITE: Broken pipe
>>> nbdkit: s3backer.15: error: write reply: NBD_CMD_WRITE: Broken pipe|
>>>
>>>
>>> (where "s3backer" is the plugin name).
>>>
>>> I am not sure what to make of these. Can someone advise?
>>>
>>> Looking at the nbdkit source, it looks to me like these are generated
>>> when there is a problem sending a reply to the nbd client. On the other
>>> hand, I am using the standard 'nbd-client' program through a Unix
>>> socket, so I'd think this should not result in errors...?
>>
>> If your plugin managed to crash nbd-client remotely, that would be
>> consistent with this symptom.
> 
> So I tried to reproduce this, and noticed something odd. It seems I can
> disconnect the nbd device (nbd-client -d) while there are still requests
> in flight:
> 
> May 02 15:20:50 vostro.rath.org kernel: nbd1: detected capacity change from 0 to 52428800
> May 02 15:20:50 vostro.rath.org kernel: block nbd1: NBD_DISCONNECT
> May 02 15:20:50 vostro.rath.org kernel: block nbd1: Disconnected due to user request.
> May 02 15:20:50 vostro.rath.org kernel: block nbd1: shutting down sockets
> May 02 15:20:50 vostro.rath.org kernel: I/O error, dev nbd1, sector 776 op 0x0:(READ) flags 0x80700 phys_seg 29 prio class 0
> May 02 15:20:50 vostro.rath.org kernel: I/O error, dev nbd1, sector 776 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> May 02 15:20:50 vostro.rath.org kernel: Buffer I/O error on dev nbd1, logical block 97, async page read
> May 02 15:20:50 vostro.rath.org kernel: block nbd1: Attempted send on invalid socket
> May 02 15:20:50 vostro.rath.org kernel: I/O error, dev nbd1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
> May 02 15:20:50 vostro.rath.org kernel: block nbd1: Attempted send on invalid socket
> May 02 15:20:50 vostro.rath.org kernel: I/O error, dev nbd1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
> 
> This was generated by running:
> 
> $ nbd-client localhost /dev/nbd1 && mkfs.ext4 /dev/nbd1 && nbd-client -d
> /dev/nbd1
> 
> Is that expected behavior?

This is better answered by Eric and Rich. :)

Laszlo

> 
> 
> I would have thought that nb-client will block until any dirty data has
> been written.
> 
> Curiously enough, in this case I did *not* get the above warnings from
> nbdkit itself.
> 
> 
> Best,
> -Nikolaus
>