[virt-tools-list] [libvirt] question about syntax of storage volume <target> element

Fri Oct 5 21:41:02 UTC 2018

On 10/5/18 3:27 PM, Cole Robinson wrote:
> On 10/02/2018 01:09 PM, Jim Fehlig wrote:
>> On 10/2/18 9:27 AM, Cole Robinson wrote:
>>> On 10/02/2018 10:53 AM, Jim Fehlig wrote:
>>>> On 10/2/18 5:13 AM, Pavel Hrdina wrote:
>>>>> On Mon, Oct 01, 2018 at 02:28:09PM -0400, Cole Robinson wrote:
>>>>>> On 09/28/2018 12:54 AM, Jim Fehlig wrote:
>>>>>>> I've attempted to use virt-manager to create a new VM that uses a volume
>>>>>>> from an rbd-based network pool, but have not been able to progress past
>>>>>>> step 4/5 where VM storage is selected. It appears virt-manager has
>>>>>>> problems properly detecting the volume as network-based storage, but
>>>>>>> before investigating those further I have a question about the syntax of
>>>>>>> the <target> element of a storage volume.
>>>>>>
>>>>>> Yeah virt-manager is known to be lacking WRT rbd. I did some work a few
>>>>>> years back but didn't finish it. At least it doesn't know how to correctly
>>>>>> use a volume with any auth data in the XML. I need to get another rbd setup
>>>>>> to test with and fix it all
>>>>>
>>>>> I was investigating the RBD support as well and in order to add proper
>>>>> support for it into virt-manager we need to add support for secrets.
>>>>
>>>> Right. But I was starting with the assumption that a "storage admin" setup 
>>>> an rbd-based pool, necessary secrets, etc. Then user creating a new VM could 
>>>> select existing volumes in the pool or create new ones at step 4/5 of the VM 
>>>> creation wizard. Currently a user can select an existing volume or create a 
>>>> new one, but can't progress beyond that point. I have a hack (attached, 
>>>> based against 1.5.1) to workaround the problem in virt-manager. Commit 
>>>> 582c1d3d fixed virt-install to copy auth data from the pool to the device 
>>>> config in domXML and as a first step I'm trying to do the same with 
>>>> virt-manager.
>>>>
>>>
>>> dropping libvir-list and adding virt-tools-list
>>>
>>> Cool, thanks for working on this. Can you provide:
>>>
>>> - pool XML you are using
>>
>> <pool type="rbd">
>>    <name>rbdpool</name>
>>    <source>
>>      <name>libvirt-pool</name>
>>      <host name='foo.bar.com' port='6789'/>
>>      <auth username='libvirt' type='ceph'>
>>        <secret uuid='c451d49d-a48b-43d8-bd78-97ccebd7d67a'/>
>>      </auth>
>>    </source>
>> </pool>
>>
>>> - vol XML
>>
>> <volume type='network'>
>>    <name>test-image</name>
>>    <key>libvirt-pool/test-image</key>
>>    <source>
>>    </source>
>>    <capacity unit='bytes'>21474836480</capacity>
>>    <allocation unit='bytes'>5481955328</allocation>
>>    <target>
>>      <path>libvirt-pool/test-image</path>
>>      <format type='raw'/>
>>    </target>
>> </volume>
>>
>>> - the backtrace you are hitting
>>
>> The first hunk in diskbackend.py of my hack fixes this backtrace (including a 
>> bit of preceding context)
>>
>> [Tue, 02 Oct 2018 10:53:41 virt-manager 20533] DEBUG (storagebrowse:53) 
>> Showing storage browser
>> [Tue, 02 Oct 2018 10:53:57 virt-manager 20533] DEBUG (storagebrowse:138) 
>> Chosen volume XML:
>> <volume type="network">
>>    <name>test-image</name>
>>    <key>libvirt-pool/test-image</key>
>>    <source>
>>    </source>
>>    <capacity unit="bytes">21474836480</capacity>
>>    <allocation unit="bytes">5481955328</allocation>
>>    <target>
>>      <path>libvirt-pool/test-image</path>
>>      <format type="raw"/>
>>    </target>
>> </volume>
>> [Tue, 02 Oct 2018 10:53:57 virt-manager 20533] DEBUG (storagebrowse:66) 
>> Closing storage browser
>> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] DEBUG (diskbackend:170) 
>> Attempting to build pool=libvirt-pool 
>> target=/usr/share/virt-manager/virtinst/libvirt-pool
>> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] DEBUG (storage:524) Creating 
>> storage pool 'libvirt-pool' with xml:
>> <pool type="dir">
>>    <name>libvirt-pool</name>
>>    <target>
>>      <path>/usr/share/virt-manager/virtinst/libvirt-pool</path>
>>    </target>
>> </pool>
>> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] ERROR (error:143) Validation 
>> Error: Storage parameter error. Could not start storage pool: cannot open 
>> directory '/usr/share/virt-manager/virtinst/libvirt-pool': No such file or 
>> directory
>> Traceback (most recent call last):
>>    File "/usr/share/virt-manager/virtManager/create.py", line 2305, in 
>> _validate_storage_page
>>      path=path)
>>    File "/usr/share/virt-manager/virtManager/addstorage.py", line 262, in 
>> validate_storage
>>      disk.path = path or None
>>    File "/usr/share/virt-manager/virtinst/devicedisk.py", line 519, in _set_path
>>      (vol_object, parent_pool) = diskbackend.manage_path(self.conn, newpath)
>>    File "/usr/share/virt-manager/virtinst/diskbackend.py", line 176, in 
>> manage_path
>>      pool = poolxml.install(build=False, create=True, autostart=True)
>>    File "/usr/share/virt-manager/virtinst/storage.py", line 559, in install
>>      raise RuntimeError(errmsg)
>> RuntimeError: Could not start storage pool: cannot open directory 
>> '/usr/share/virt-manager/virtinst/libvirt-pool': No such file or directory
>>
>> After fixing the above I see errors trying to change the permissions on the 
>> volume
>>
>> [Tue, 02 Oct 2018 10:58:35 virt-manager 20822] DEBUG (addstorage:143) No 
>> search access for dirs: ['libvirt-pool', '']
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (addstorage:156) 
>> Attempting to correct permission issues.
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:290) Ran 
>> command '['setfacl', '--modify', 'user:qemu:x', '']'
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:292) out=b''
>> err=b'setfacl: : No such file or directory\n'
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:325) setfacl 
>> failed, trying old fashioned way
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:297) Setting 
>> +x on
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:297) Setting 
>> +x on libvirt-pool
>> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (addstorage:171) 
>> Permission errors:
>>   : [Errno 2] No such file or directory: ''
>> libvirt-pool : [Errno 2] No such file or directory: 'libvirt-pool'
>> It is very likely the VM will fail to start up.
>>
>> The hunk in devicedisk.py fixes this by skipping the permissions check for 
>> network-based volumes.
>>
>>> Maybe I can distill that stuff into something that's easy to reproduce with 
>>> the testdriver
>>
> 
> Ahh now that I look at the diffs, they are both in sections of the code that 
> only run or fail in real scenarios: manage_storage will always succeed with the 
> testdriver, and generally the path perms checking isn't run in the test suite. I 
> can probably hack up some unit tests to demonstrate the fixes though.
> 
> But yeah as you've seen this boils down to virtinst having the wrong assumption 
> that a network volume <path> is easily identifiable as a URL, which while it 
> works for gluster doesn't work for rbd/sheepdog.
> 
> I'll help with unittests when the patches are ready for posting

I'm not really sure what else to do with the patch, other than remove the lame 
debugging lines. It's a small improvement that fixes use case of selecting an 
existing volume (or creating a new one) from an existing network pool when 
creating a new VM.

Were you looking for a more complete rbd solution where virt-manager supports 
creating rbd pools, secrets, etc. as Pavel suggested?

Regards,
Jim