[Freeipa-devel] [PATCH 537] ipapython: port p11helper C code to Python

Jan Cholasta jcholast at redhat.com
Wed Jan 20 14:36:25 UTC 2016 

On 20.1.2016 13:51, Martin Basti wrote:
>
>
> On 19.01.2016 11:43, Jan Cholasta wrote:
>> On 12.1.2016 16:06, Martin Basti wrote:
>>>
>>>
>>> On 12.01.2016 14:44, Jan Cholasta wrote:
>>>> On 12.1.2016 13:32, Martin Basti wrote:
>>>>>
>>>>>
>>>>> On 12.01.2016 12:24, Jan Cholasta wrote:
>>>>>> On 12.1.2016 12:17, Martin Basti wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 12.01.2016 10:19, Jan Cholasta wrote:
>>>>>>>> On 12.1.2016 09:32, Martin Basti wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07.01.2016 14:13, Jan Cholasta wrote:
>>>>>>>>>> On 7.1.2016 09:50, Jan Cholasta wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> the attached patch ports the _ipap11helper module to
>>>>>>>>>>> python-cffi.
>>>>>>>>>>>
>>>>>>>>>>> Combined with my patch 536 [1], this makes ipapython
>>>>>>>>>>> architecture
>>>>>>>>>>> independent.
>>>>>>>>>>
>>>>>>>>>> Updated patch attached.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I tried to run DNSSEC tests and it failed unexpectedly:
>>>>>>>>>
>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>> Connected
>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>> replica pub keys in LDAP:
>>>>>>>>> set(['0x51df7c70b9869a7dd2bbd27335dba3f8',
>>>>>>>>> '0xd8538e634797420ca86cda420234443c'])
>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>> replica pub keys in SoftHSM:
>>>>>>>>> set(['0x51df7c70b9869a7dd2bbd27335dba3f8',
>>>>>>>>> '0x1f7241a64d69ced6c0a14f6999410c59'])
>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>> new replica keys in LDAP:
>>>>>>>>> set(['0xd8538e634797420ca86cda420234443c'])
>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>> label=dnssec-replica:replica1.ipa.test.,
>>>>>>>>> id=d8538e634797420ca86cda420234443c,
>>>>>>>>> data=30820122300d06092a864886f70d01010105
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: Traceback
>>>>>>>>> (most
>>>>>>>>> recent call last):
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 664, in <module>
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]:
>>>>>>>>> ldap2master_replica_keys_sync(log, ldapkeydb, localhsm)
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 313, in
>>>>>>>>> ldap2master_replica_keys_sync
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]:
>>>>>>>>> localhsm.import_public_key(new_key_ldap,
>>>>>>>>> new_key_ldap['ipapublickey'])
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/localhsm.py",
>>>>>>>>> line
>>>>>>>>> 173, in import_public_key
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: h =
>>>>>>>>> self.p11.import_public_key(**params)
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/p11helper.py", line
>>>>>>>>> 1498, in
>>>>>>>>> import_public_key
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: pkey =
>>>>>>>>> d2i_PUBKEY(NULL, data_ptr, data_length)
>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: TypeError:
>>>>>>>>> 'int(*)(EVP_PKEY *, unsigned char * *)' expects 2 arguments, got 3
>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>> Main process exited, code=exited, status=1/FAILURE
>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>> Unit entered failed state.
>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>> Failed with result 'exit-code'.
>>>>>>>>>
>>>>>>>>> I haven't seen any other errors
>>>>>>>>
>>>>>>>> Updated patch attached. Added a patch which replaces calls to
>>>>>>>> libcrypto with calls to python-cryptography.
>>>>>>>>
>>>>>>>
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] Done
>>>>>>> configuring
>>>>>>> DNS (named).
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>> Configuring DNS
>>>>>>> key synchronization service (ipa-dnskeysyncd)
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [1/7]:
>>>>>>> checking
>>>>>>> status
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [2/7]:
>>>>>>> setting
>>>>>>> up bind-dyndb-ldap working directory
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [3/7]:
>>>>>>> setting
>>>>>>> up kerberos principal
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [4/7]:
>>>>>>> setting
>>>>>>> up SoftHSM
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [5/7]: adding
>>>>>>> DNSSEC containers
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [6/7]:
>>>>>>> creating
>>>>>>> replica keys
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [error]
>>>>>>> Error:
>>>>>>> export_RSA_public_key: internal error: EVP_PKEY_set1_RSA failed
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR
>>>>>>> export_RSA_public_key: internal error: EVP_PKEY_set1_RSA failed
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The
>>>>>>> ipa-server-install command failed. See
>>>>>>> /var/log/ipaserver-install.log
>>>>>>> for more information
>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] Exit code: 1
>>>>>>>
>>>>>>> ipa-server-install.log
>>>>>>> ....
>>>>>>>    File
>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
>>>>>>> line 436, in run_step
>>>>>>>      method()
>>>>>>>    File
>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/dnskeysyncinstance.py",
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> line 342, in __setup_replica_keys
>>>>>>>      public_key_blob = p11.export_public_key(public_key_handle)
>>>>>>>    File "/usr/lib/python2.7/site-packages/ipapython/p11helper.py",
>>>>>>> line
>>>>>>> 1275, in export_public_key
>>>>>>>      return self._export_RSA_public_key(object)
>>>>>>>    File "/usr/lib/python2.7/site-packages/ipapython/p11helper.py",
>>>>>>> line
>>>>>>> 1240, in _export_RSA_public_key
>>>>>>>      raise Error("export_RSA_public_key: internal error: "
>>>>>>>
>>>>>>> 2016-01-12T11:00:29Z DEBUG The ipa-server-install command failed,
>>>>>>> exception: Error: export_RSA_public_key: internal error:
>>>>>>> EVP_PKEY_set1_RSA failed
>>>>>>> 2016-01-12T11:00:29Z ERROR export_RSA_public_key: internal error:
>>>>>>> EVP_PKEY_set1_RSA failed
>>>>>>
>>>>>> Updated patch 538 attached.
>>>>>>
>>>>> Jan 12 12:31:43 master.ipa.test
>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: Connected
>>>>> Jan 12 12:31:44 master.ipa.test
>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: replica pub keys in LDAP:
>>>>> set(['0xf5edad67436d0ed36b75c3a70216fa43',
>>>>> '0x7164a931484d505f1e249e3dcbc313e2'])
>>>>> Jan 12 12:31:44 master.ipa.test
>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: replica pub keys in SoftHSM:
>>>>> set(['0xf5edad67436d0ed36b75c3a70216fa43',
>>>>> '0x7164a931484d505f1e249e3dcbc313e2', '0x28e302ae6b6ee7e9284cd5f6
>>>>> Jan 12 12:31:44 master.ipa.test
>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: new replica keys in LDAP:
>>>>> set([])
>>>>> Jan 12 12:31:44 master.ipa.test
>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: obsolete replica keys in
>>>>> local
>>>>> HSM: set(['0x28e302ae6b6ee7e9284cd5f61aadbbe7'])
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: Traceback
>>>>> (most
>>>>> recent call last):
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 664, in <module>
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>> ldap2master_replica_keys_sync(log, ldapkeydb, localhsm)
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 321, in
>>>>> ldap2master_replica_keys_sync
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>> localhsm.replica_pubkeys_wrap[key_id]['ipk11wrap'] = False
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/localhsm.py", line
>>>>> 65, in __setitem__
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: return
>>>>> self.p11.set_attribute(self.handle, attrs_name2id[key], value)
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>> "/usr/lib/python2.7/site-packages/ipapython/p11helper.py", line
>>>>> 1661, in
>>>>> set_attribute
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>> sizeof(CK_ATTRIBUTE)))
>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: TypeError: an
>>>>> integer is required
>>>>> Jan 12 12:31:44 master.ipa.test systemd[1]: ipa-ods-exporter.service:
>>>>> Main process exited, code=exited, status=1/FAILURE
>>>>>
>>>>
>>>> Updated patch 537 attached.
>>>>
>>> Jan 12 15:04:10 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: Connected
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: replica pub keys in LDAP:
>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: replica pub keys in SoftHSM:
>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: new replica keys in LDAP:
>>> set([])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: obsolete replica keys in local
>>> HSM: set([])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: keys in local HSM & LDAP:
>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: Updating attribute
>>> ipk11verifyrecover from "1" to "False"
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: master keys in local HSM:
>>> set([])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: master keys in LDAP HSM:
>>> set([])
>>> Jan 12 15:04:11 master.ipa.test
>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: new master keys in local HSM:
>>> set([])
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: Traceback (most
>>> recent call last):
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/libexec/ipa/ipa-ods-exporter", line 665, in <module>
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>> master2ldap_master_keys_sync(log, ldapkeydb, localhsm)
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/libexec/ipa/ipa-ods-exporter", line 340, in
>>> master2ldap_master_keys_sync
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>> ldapkeydb.flush()
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 311, in flush
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>> self._update_keys()
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 307, in _update_keys
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>> key._update_key()
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 179, in _update_key
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>> self._cleanup_key()
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 170, in _cleanup_key
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: if
>>> self.get(attr, empty) == default_attrs[attr]:
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib64/python2.7/_abcoll.py", line 382, in get
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: return
>>> self[key]
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 132, in __getitem__
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: val =
>>> ldap_bool(val)
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>> 39, in ldap_bool
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: raise
>>> AssertionError('invalid LDAP boolean "%s"' % val)
>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: AssertionError:
>>> invalid LDAP boolean "1"
>>> Jan 12 15:04:11 master.ipa.test systemd[1]: ipa-ods-exporter.service:
>>> Main process exited, code=exited, status=1/FAILURE
>>>
>>>
>>> You can run the dnssec test, it has been fixed.
>>
>> Updated patches attached. The test now passes.
>>
> Hello,
>
> pkcs11helper tests passed
> DNSSEC tests passed
>
> 1)
> Slot is unused argument here:
>
>     def __init__(self, slot, user_pin, library_path):
>          self.p11_ptr = new_ptr(CK_FUNCTION_LIST_PTR)
>          self.session_ptr = new_ptr(CK_SESSION_HANDLE)
>
>          self.slot = 0

Fixed.

>
> 2)
> should't string_to_pybytes_or_none raise exception instead of returning
> None? In C extension returning NULL means error, and exception was
> raised by python itself when function ends with returning NULL.
>
> in export_wrapped_key method
>
>          result = string_to_pybytes_or_none(wrapped_key,
> wrapped_key_len_ptr[0])
>          return result
>
> In this case method returns None instead of raising exception.
>
> Also I think that in _export_RSA_public_key method,
> string_to_pybytes_or_none should raise exception when it get NULL as
> string too

This is exactly how it behaves in the original C code, so I'm not 
changing it.

I noticed I don't return None in _export_RSA_public_key in case of 
encoding failure. Fixed.

>
> 3)
> Is possible to remove build dependencies added in commit c909690c ?

Removed, except for openssl-devel, which is used elsewhere.

Updated patches attached.

-- 
Jan Cholasta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-537.6-ipapython-port-p11helper-C-code-to-Python.patch
Type: text/x-patch
Size: 163635 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20160120/4e3783ec/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-538.3-ipapython-use-python-cryptography-instead-of-libcryp.patch
Type: text/x-patch
Size: 15136 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20160120/4e3783ec/attachment-0001.bin>

More information about the Freeipa-devel mailing list