[Question] sg_persist failed when using qemu-pr-helper

[Michal Privoznik]

On 11/17/20 2:42 AM, Zihao Chang wrote:
> 
> 
> On 2020/11/16 23:10, Michal Privoznik wrote:
>> On 11/16/20 3:22 PM, Zihao Chang wrote:
>>> Hi all,
>>>
>>> When using qemu-pr-helper，"sg_persist" in guest return Aborted Command after iscsiadm logout & login.
>>>
>>> My confusion:
>>> Is this problem caused by bugs of libvirt/qemu-pr-helper?
>>> Or iscsiadm logout & login should be forbidden when using qemu-pr-helper?
>>> Or There is something wrong with my config.
>>>
>>> Steps to Reproduce:
>>> 1. setup multipath env
>>>     1) set /etc/iscsi/initiatorname.iscsi
>>>     2) iscsiadm discovery & login
>>>     3) start multipathd
>>>
>>> 2. add the following xml to guest
>>>       <disk type='block' device='lun'>
>>>               <driver name='qemu' type='raw' cache='none' io='native'/>
>>>               <source dev='/dev/mapper/XXXXXXXXXXXXXXXXX'>
>>>                   **<reservations managed='yes'/>**
>>>               </source>
>>>               <target dev='sdc' bus='scsi'/>
>>>       </disk>
>>>
>>> 3. start the guest
>>>      1) "sg_persist -v -i -n -k -d  /dev/sdc" works well in the guest
>>>      2) here /dev/sdc(guest)  -> /dev/sdht(host)
>>>
>>> 4. iscsiadm logout & login on host
>>>      1) iscsiadm  -m node -u  (host)
>>>      2) iscsiadm -m node -l   (host)
>>>      3) here /dev/sdc(guest)  -(change)-> /dev/sdfo(host)
>>>
>>> 5. exec "sg_persist -v -i -n -k -d  /dev/sdc" in the guest
>>>      1) command return:
>>>            Persistent Reservation In cmd: 5e 00 00 00 00 00 00 20 00 00
>>>            persistent reservation in:
>>>            Fixed format, current; Sense key: Aborted Command
>>>            Additional sense: Logical unit communication failure
>>>            PR in (Read keys): Aborted command sense key, other than protection related (asc=0x10)
>>>      2) strace qemu-pr-helper show:
>>>            openat(AT_FDCWD, "/dev/sdht", O_RDONLY) = -1 ENOENT (No such file or directory)
>>>
>>> Additional info:
>>> 1. strace qemu-pr-helper show:
>>>          openat(AT_FDCWD, "/dev/sdht", O_RDONLY) = -1 ENOENT (No such file or directory)
>>> 2. This problem may be related to mnt(namespace). I try to set mnt of qemu-pr-helper as pid 1,  sg_persist works well after logout & login.
>>>
>>>
>>
>> Yeah, this smells like a libvirt bug. But I'm not sure how to solve it. Because from security POV we want to run qemu-pr-helper inside the same namespace as qemu is running (so that it doesn't see more devices than needed). But at the same time, when a /dev node changes from outside libvirt does not reflect that in the namespace.
>>
>> BTW: is /dev/mapper/XXX updated on re-login? Becuase if not, then you'd get the same error in qemu, if it wasn't for pr-helper hitting the issue first.
>>
>> A possible work around might be to use unmanaged pr-helper (reservations managed='no') and just start it by hand.
>>
>> Michal
>>
>> .
> 
> Actually, if use unmanaged pr-helper (reservations managed='no'), "sg_persist" work well without starting pr-helper by
> hand in this case. After logout & login qemu does not meet openat() ENOENT (No such file or directory), and qemu even
> does not use openat() according to the result of "strace" command. But the source code of libvirt shows that it start
> pr-helper inside the same namespace as qemu is running. It really confused me.

Can it be that qemu is running privileged and thus can do the 
reservation itself? Otherwise I don't understand what's going on.

Michal