[Virtio-fs] (no subject)
Hanna Czenczek
hreitz at redhat.com
Mon Oct 9 08:07:12 UTC 2023
On 06.10.23 22:49, Alex Bennée wrote:
> Hanna Czenczek <hreitz at redhat.com> writes:
>
>> On 06.10.23 17:17, Alex Bennée wrote:
>>> Hanna Czenczek <hreitz at redhat.com> writes:
>>>
>>>> On 06.10.23 12:34, Michael S. Tsirkin wrote:
>>>>> On Fri, Oct 06, 2023 at 11:47:55AM +0200, Hanna Czenczek wrote:
>>>>>> On 06.10.23 11:26, Michael S. Tsirkin wrote:
>>>>>>> On Fri, Oct 06, 2023 at 11:15:55AM +0200, Hanna Czenczek wrote:
>>>>>>>> On 06.10.23 10:45, Michael S. Tsirkin wrote:
>>>>>>>>> On Fri, Oct 06, 2023 at 09:48:14AM +0200, Hanna Czenczek wrote:
>>>>>>>>>> On 05.10.23 19:15, Michael S. Tsirkin wrote:
>>>>>>>>>>> On Thu, Oct 05, 2023 at 01:08:52PM -0400, Stefan Hajnoczi wrote:
>>>>>>>>>>>> On Wed, Oct 04, 2023 at 02:58:57PM +0200, Hanna Czenczek wrote:
>>> <snip>
>>>>>> What I’m saying is, 923b8921d21 introduced SET_STATUS calls that broke all
>>>>>> devices that would implement them as per virtio spec, and even today it’s
>>>>>> broken for stateful devices. The mentioned performance issue is likely
>>>>>> real, but we can’t address it by making up SET_STATUS calls that are wrong.
>>>>>>
>>>>>> I concede that I didn’t think about DRIVER_OK. Personally, I would do all
>>>>>> final configuration that would happen upon a DRIVER_OK once the first vring
>>>>>> is started (i.e. receives a kick). That has the added benefit of being
>>>>>> asynchronous because it doesn’t block any vhost-user messages (which are
>>>>>> synchronous, and thus block downtime).
>>>>>>
>>>>>> Hanna
>>>>> For better or worse kick is per ring. It's out of spec to start rings
>>>>> that were not kicked but I guess you could do configuration ...
>>>>> Seems somewhat asymmetrical though.
>>>> I meant to take the first ring being started as the signal to do the
>>>> global configuration, i.e. not do this once per vring, but once
>>>> globally.
>>>>
>>>>> Let's wait until next week, hopefully Yajun Wu will answer.
>>>> I mean, personally I don’t really care about the whole SET_STATUS
>>>> thing. It’s clear that it’s broken for stateful devices. The fact
>>>> that it took until 6f8be29ec17d to fix it for just any device that
>>>> would implement it according to spec to me is a strong indication that
>>>> nobody does implement it according to spec, and is currently only used
>>>> to signal to some specific back-end that all rings have been set up
>>>> and should be configured in a single block.
>>> I'm certainly using [GS]ET_STATUS for the proposed F_TRANSPORT
>>> extensions where everything is off-loaded to the vhost-user backend.
>> How do these back-ends work with the fact that qemu uses SET_STATUS
>> incorrectly when not offloading? Do you plan on fixing that?
> Mainly having a common base implementation which does it right and
> having very lightweight derivations for legacy stubs using it. The
> aim is to eliminate the need for QEMU stubs entirely by fully specifying
> the device from the vhost-user API.
If the current SET_STATUS use is overhauled, too, that would be good. I
wonder why you need the status byte, though.
>> (I.e. that we send SET_STATUS 0 when the VM is paused, potentially
>> resetting state that is not recoverable, and that we set DRIVER and
>> DRIVER_OK simultaneously.)
> This is QEMU simulating a SET_STATUS rather than the guest triggering
> it?
Yes, and the fact that we simulate it when the guest will not have
triggered it, i.e. we reset the device (SET_STATUS 0) when the VM is
paused. Effectively, qemu injects virtio commands that the guest has
never requested, which generally feels like a bad idea, because qemu
will need to get the device back to its previous state before the guest
is resumed, which may or may not work. Specifically, it won’t work for
devices that have internal state.
Furthermore, we use SET_STATUS to set ACKNOWLEDGE | DRIVER | DRIVER_OK
simultaneously, which is wrong. ACKNOWLEDGE | DRIVER may perhaps be set
simultaneously, but then comes feature negotiation (setting and checking
FEATURES_OK), and then DRIVER_OK.
Finally, how the status byte is to be used is not noted in the
vhost-user specification, which instead points to the virtio
specification. I think if we keep SET_STATUS, it must be documented how
it interacts with other vhost-user commands. For example, how the
FEATURES_OK protocol described in the virtio specification interacts
with GET_FEATURES/SET_FEATURES, or whether SET_STATUS 0 and RESET_DEVICE
are equivalent. Currently, the only implementation of SET_STATUS I know
(DPDK) ignores SET_STATUS 0, i.e. doesn’t do a reset. To me that
indicates that the spec must be clear on what these status values mean
with regards to the vhost-user protocol as a whole.
So every software implementation with STATUS support that I know
implements SET_STATUS wrongly right now, and that’s a problem, because
it prevents implementations like virtiofsd from doing so correctly.
Hanna
More information about the Virtio-fs
mailing list