<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>Hi,<br></div><div><br></div><div>I am observing some strange errors when using the Kernel's NBD driver with NBDkit.<br></div><div><br></div><div>On the kernel side, I see:<br></div><div><br></div><div>May 15 16:16:11 vostro.rath.org kernel: nbd0: detected capacity change from 0 to 104857600<br></div><div>May 15 16:16:11 vostro.rath.org kernel: nbd1: detected capacity change from 0 to 104857600<br></div><div>May 15 16:18:23 vostro.rath.org kernel: block nbd0: Possible stuck request 00000000ae5feee7: control (write@4836316160,32768B). Runtime 30 seconds<br></div><div>May 15 16:18:25 vostro.rath.org kernel: block nbd0: Possible stuck request 000000007094eddc: control (write@5372947456,10240B). Runtime 30 seconds<br></div><div>May 15 16:18:27 vostro.rath.org kernel: block nbd0: Suspicious reply 89 (status 0 flags 0)<br></div><div>May 15 16:18:31 vostro.rath.org kernel: block nbd0: Possible stuck request 0000000075f8b9bc: control (write@8057764864,32768B). Runtime 30 seconds<br></div><div>May 15 16:18:41 vostro.rath.org kernel: block nbd0: Possible stuck request 000000002d1b3e8b: control (write@14499979264,32768B). Runtime 30 seconds<br></div><div>[...]<br></div><div><br></div><div>And userspace ('zfs snapshot" in this instance) is stuck afterwards.<br></div><div><br></div><div>On the NBDkit side, there seemingly are write errors when replying back to the kernel:<br></div><div><br></div><div>$ nbdkit --unix /tmp/tmpi5o59_y_/nbd_socket_sb --foreground --filter=exitlast --filter=stats --threads 16 S3 size=50G bucket=nikratio-backup key=sb statsfile=/tmp/tmpi5o59_y_/stats_sb.txt object-size=32K &<br></div><div>$ nbd-client -unix /tmp/tmpi5o59_y_/nbd_socket_sb /dev/nbd2<br></div><div>Warning: the oldstyle protocol is no longer supported.<br></div><div>This method now uses the newstyle protocol with a default export<br></div><div>Negotiation: ..size = 51200MB<br></div><div>Connected /dev/nbd0<br></div><div>[....]<br></div><div>nbdkit: python.10: error: write reply: NBD_CMD_WRITE: Broken pipe<br></div><div></div><div><br></div><div><br></div><div>What's the best way to narrow down who's the culprit here (kernel vs NBD server)?<br></div><div><br></div><div>Best,<br></div><div>-Nikolaus<br></div><div><br></div><div id="sig33580657"><div class="signature">--<br></div><div class="signature">GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F<br></div><div class="signature"><br></div><div class="signature"> »Time flies like an arrow, fruit flies like a Banana.«<br></div><div class="signature"><br></div></div><div><br></div></body></html>