[libvirt] [PATCH] remote: Fix TLS transport on Windows

[Matthias Bolte]

2010/11/22 Matthias Bolte <matthias.bolte at googlemail.com>:
> 2010/11/22 Daniel P. Berrange <berrange at redhat.com>:
>> On Sat, Nov 20, 2010 at 06:10:21PM +0100, Matthias Bolte wrote:
>>> gnulib wraps Windows' SOCKET handle based send() and recv() functions
>>> into file descriptor based ones that are use in libvirt. By default
>>> GnuTLS uses the SOCKET handle based send() and recv() on Windows. This
>>> makes gnutls_handshake() fail internally with a WSAENOTSOCK error because
>>> libvirt passes a file descriptor; GnuTLS needs the SOCKET handle.
>>
>> This doesn't entirely make any sense to me. GNUTLS also uses GNULIB,
>> including all its socket wrappers for send/recv. If the push/pull
>> function is NULL, gnulib does this
>>
>>      if (session->internals._gnutls_push_func == NULL)
>>        {
>>          i = send (GNUTLS_POINTER_TO_INT (fd), &ptr[n - left], left, 0);
>>
>> And this 'send' impl maps to GNULIBs replacement in GNUTLS tree, which
>> is identical to libvirt's 'send' impl
>>
>> So AFAICT, the syscall behaviour will be identical both with & without
>> your proposed patch.
>>
>> The only potential difference I can see is that when push/pull are
>> NULL, GNUTLS calls into  WSAGetLastError (); to set the errno
>> which is somewhat redundant as GNULIB has already called that and
>> used it to set errno.
>>
>> Daniel
>>
>
> You're right GNUTLS uses GNULIB. I missed that fact. But why does my
> patch make a difference then? Without it the TLS transport doesn't
> work and with the patch it works. I'll have to investigate.
>
> Matthias
>

Okay, yes GnuTLS uses gnulib, but they explicitly don't use gnulib's
replacements for send() and recv() on Windows. See
lib/gnutls_buffers.c:

/* We need to disable gnulib's replacement wrappers to get native
   Windows interfaces. */
#undef recv
#undef send

GnuTLS decided to use the native Windows versions of send() and
recv(). This cannot be changed, as that would break existing
applications using GnuTLS on Windows relying on GnuTLS using the
native Windows versions of send() and recv(). Therefore, I think my
patch is necessary, as libvirt requires GnuTLS to use gnulib's
replacement functions.

Matthias