[libvirt] problems with remote authentication with policykit

[Jim Paris]

Daniel P. Berrange wrote:
> > But when accessing remotely, I get no useful error, and a hang:
> > 
> > $ virsh -c qemu+ssh://jim@server/system
> > libvir: Remote error : authentication failed
> > <process hangs here>
> > 
> > $ virsh --readonly -c qemu+ssh://jim@server/system
> > libvir: Remote error : authentication failed
> > <process hangs here>
> > 
> > Furthermore, on the server, this leaves "nc" processes running,
> > and eventually there are enough that libvirtd stops accepting new
> > connections.
> 
> The hang is really odd. That suggests something is not closing the 
> socket connection properly. If you had been yusing 0.6.1/.2/.3 I
> would have said it was one of the libvirtd bugs, but 0.6.4 has all
> event handling bugs fixed.  Perhaps the libvirtd client is not
> killing the SSH session / process when it closes the connection
> after auth failure.

I was using 0.4.6 on the client side.  I upgraded that to 0.6.4, but I
still get the hang.  Virsh prints nothing; the LIBVIRT_DEBUG output
is:

17:34:58.524: debug : doRemoteOpen:505 : proceeding with name = qemu:///system
17:34:58.525: debug : virExecWithHook:573 : ssh server nc -U /var/run/libvirt/libvirt-sock
17:34:58.526: debug : call:6947 : Doing call 66 (nil)
17:34:58.527: debug : call:7017 : We have the buck 66 0x7fba56729010 0x7fba56729010
17:34:59.359: debug : processCallRecvLen:6605 : Got length, now need 36 total (32 more)
17:34:59.360: debug : processCalls:6873 : Giving up the buck 66 0x7fba56729010 (nil)
17:34:59.360: debug : call:7048 : All done with our call 66 (nil) 0x7fba56729010
17:34:59.360: debug : remoteAuthPolkit:6114 : Client initialize PolicyKit authentication
17:34:59.360: debug : call:6947 : Doing call 70 (nil)
17:34:59.360: debug : call:7017 : We have the buck 70 0xbccef0 0xbccef0
17:34:59.433: debug : processCallRecvLen:6605 : Got length, now need 128 total (124 more)
17:34:59.434: debug : processCalls:6873 : Giving up the buck 70 0xbccef0 (nil)
17:34:59.434: debug : call:7048 : All done with our call 70 (nil) 0xbccef0
17:34:59.434: error : server_error:7231 : authentication failed
17:35:13.585: debug : do_open:999 : driver 4 remote returned ERROR
17:35:13.585: debug : virUnrefConnect:232 : unref connection 0xbc6a60 1
17:35:13.585: debug : virReleaseConnect:191 : release connection 0xbc6a60

If I kill the libvirtd process on the server, the client then finally prints:

error: authentication failed
error: failed to connect to the hypervisor

and the client then exits.


On the server side, the libvirtd output is

17:34:59.378: debug : remoteDispatchAuthPolkit:3385 : Start PolicyKit auth 25
17:34:59.378: info : remoteDispatchAuthPolkit:3396 : Checking PID 7551 running as 1000
17:34:59.379: debug : virEventRunOnce:567 : Poll got 1 event
17:34:59.379: debug : virEventDispatchHandles:450 : Dispatch n=2 f=9 w=3 e=1 0x1a72790
17:34:59.379: debug : nodeDeviceLock:52 : LOCK node 0x1a748e0
17:34:59.379: debug : nodeDeviceUnlock:57 : UNLOCK node 0x1a748e0
17:34:59.426: error : remoteDispatchAuthPolkit:3451 : Policy kit denied action org.libvirt.unix.manage from pid 7551, uid 1000, result: auth_admin_keep_session

The hang aside, it seems libvirtd should be using
org.libvirt.unix.monitor for the readonly connection?


> > Is policykit authentication supposed to work over qemu+ssh?
> 
> Yes, but only if you ssh as root such that policykit is a no-op.
> 
> The problem you are seeing is becaue you SSH as non-root. PolicyKit
> relies on ConsoleKit to determine who is authorized, and SSH does not
> register ConsoleKit Sessions. 

As I mentioned, I've modified the PolicyKit libvirtd configuration to
not require a session:

 <match action="org.libvirt.unix.manage">
   <return result="auth_admin_keep_session"/>
 </match>

so I was hoping that wouldn't be a problem.  With this configuration,
I think even using libpam-ck-connector wouldn't change things?


> > I was hoping it would at least not break the --readonly case.
> 
> That all said --readonly is intended to work at all times. Our default
> policy file includes a rule <allow_any>yes</allow_any>  which is telling
> policykit to allow access even if the client is not associatied with
> any ConsoleKit session.  So this should have allowed it to work for
> you with --readonly.

Right, it seems libvirtd is missing readonly somehow?

-jim