[Freeipa-devel] [PATCH 537] ipapython: port p11helper C code to Python

Martin Basti mbasti at redhat.com
Thu Jan 21 09:42:56 UTC 2016 


On 20.01.2016 15:36, Jan Cholasta wrote:
> On 20.1.2016 13:51, Martin Basti wrote:
>>
>>
>> On 19.01.2016 11:43, Jan Cholasta wrote:
>>> On 12.1.2016 16:06, Martin Basti wrote:
>>>>
>>>>
>>>> On 12.01.2016 14:44, Jan Cholasta wrote:
>>>>> On 12.1.2016 13:32, Martin Basti wrote:
>>>>>>
>>>>>>
>>>>>> On 12.01.2016 12:24, Jan Cholasta wrote:
>>>>>>> On 12.1.2016 12:17, Martin Basti wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 12.01.2016 10:19, Jan Cholasta wrote:
>>>>>>>>> On 12.1.2016 09:32, Martin Basti wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 07.01.2016 14:13, Jan Cholasta wrote:
>>>>>>>>>>> On 7.1.2016 09:50, Jan Cholasta wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> the attached patch ports the _ipap11helper module to
>>>>>>>>>>>> python-cffi.
>>>>>>>>>>>>
>>>>>>>>>>>> Combined with my patch 536 [1], this makes ipapython
>>>>>>>>>>>> architecture
>>>>>>>>>>>> independent.
>>>>>>>>>>>
>>>>>>>>>>> Updated patch attached.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> I tried to run DNSSEC tests and it failed unexpectedly:
>>>>>>>>>>
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>>> Connected
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>>> replica pub keys in LDAP:
>>>>>>>>>> set(['0x51df7c70b9869a7dd2bbd27335dba3f8',
>>>>>>>>>> '0xd8538e634797420ca86cda420234443c'])
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>>> replica pub keys in SoftHSM:
>>>>>>>>>> set(['0x51df7c70b9869a7dd2bbd27335dba3f8',
>>>>>>>>>> '0x1f7241a64d69ced6c0a14f6999410c59'])
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>>> new replica keys in LDAP:
>>>>>>>>>> set(['0xd8538e634797420ca86cda420234443c'])
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test
>>>>>>>>>> /usr/libexec/ipa/ipa-ods-exporter[8667]:
>>>>>>>>>> label=dnssec-replica:replica1.ipa.test.,
>>>>>>>>>> id=d8538e634797420ca86cda420234443c,
>>>>>>>>>> data=30820122300d06092a864886f70d01010105
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: 
>>>>>>>>>> Traceback
>>>>>>>>>> (most
>>>>>>>>>> recent call last):
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 664, in <module>
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]:
>>>>>>>>>> ldap2master_replica_keys_sync(log, ldapkeydb, localhsm)
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 313, in
>>>>>>>>>> ldap2master_replica_keys_sync
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]:
>>>>>>>>>> localhsm.import_public_key(new_key_ldap,
>>>>>>>>>> new_key_ldap['ipapublickey'])
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/localhsm.py",
>>>>>>>>>> line
>>>>>>>>>> 173, in import_public_key
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: h =
>>>>>>>>>> self.p11.import_public_key(**params)
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: File
>>>>>>>>>> "/usr/lib/python2.7/site-packages/ipapython/p11helper.py", line
>>>>>>>>>> 1498, in
>>>>>>>>>> import_public_key
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: pkey =
>>>>>>>>>> d2i_PUBKEY(NULL, data_ptr, data_length)
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test ipa-ods-exporter[8667]: 
>>>>>>>>>> TypeError:
>>>>>>>>>> 'int(*)(EVP_PKEY *, unsigned char * *)' expects 2 arguments, 
>>>>>>>>>> got 3
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>>> Main process exited, code=exited, status=1/FAILURE
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>>> Unit entered failed state.
>>>>>>>>>> Jan 12 08:28:06 master.ipa.test systemd[1]:
>>>>>>>>>> ipa-ods-exporter.service:
>>>>>>>>>> Failed with result 'exit-code'.
>>>>>>>>>>
>>>>>>>>>> I haven't seen any other errors
>>>>>>>>>
>>>>>>>>> Updated patch attached. Added a patch which replaces calls to
>>>>>>>>> libcrypto with calls to python-cryptography.
>>>>>>>>>
>>>>>>>>
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] Done
>>>>>>>> configuring
>>>>>>>> DNS (named).
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>>> Configuring DNS
>>>>>>>> key synchronization service (ipa-dnskeysyncd)
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [1/7]:
>>>>>>>> checking
>>>>>>>> status
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [2/7]:
>>>>>>>> setting
>>>>>>>> up bind-dyndb-ldap working directory
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [3/7]:
>>>>>>>> setting
>>>>>>>> up kerberos principal
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [4/7]:
>>>>>>>> setting
>>>>>>>> up SoftHSM
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [5/7]: 
>>>>>>>> adding
>>>>>>>> DNSSEC containers
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [6/7]:
>>>>>>>> creating
>>>>>>>> replica keys
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] [error]
>>>>>>>> Error:
>>>>>>>> export_RSA_public_key: internal error: EVP_PKEY_set1_RSA failed
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR
>>>>>>>> export_RSA_public_key: internal error: EVP_PKEY_set1_RSA failed
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Server): ERROR The
>>>>>>>> ipa-server-install command failed. See
>>>>>>>> /var/log/ipaserver-install.log
>>>>>>>> for more information
>>>>>>>> [ipa.ipatests.test_integration.host.Host.master.cmd10] Exit 
>>>>>>>> code: 1
>>>>>>>>
>>>>>>>> ipa-server-install.log
>>>>>>>> ....
>>>>>>>>    File
>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
>>>>>>>> line 436, in run_step
>>>>>>>>      method()
>>>>>>>>    File
>>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/install/dnskeysyncinstance.py", 
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> line 342, in __setup_replica_keys
>>>>>>>>      public_key_blob = p11.export_public_key(public_key_handle)
>>>>>>>>    File "/usr/lib/python2.7/site-packages/ipapython/p11helper.py",
>>>>>>>> line
>>>>>>>> 1275, in export_public_key
>>>>>>>>      return self._export_RSA_public_key(object)
>>>>>>>>    File "/usr/lib/python2.7/site-packages/ipapython/p11helper.py",
>>>>>>>> line
>>>>>>>> 1240, in _export_RSA_public_key
>>>>>>>>      raise Error("export_RSA_public_key: internal error: "
>>>>>>>>
>>>>>>>> 2016-01-12T11:00:29Z DEBUG The ipa-server-install command failed,
>>>>>>>> exception: Error: export_RSA_public_key: internal error:
>>>>>>>> EVP_PKEY_set1_RSA failed
>>>>>>>> 2016-01-12T11:00:29Z ERROR export_RSA_public_key: internal error:
>>>>>>>> EVP_PKEY_set1_RSA failed
>>>>>>>
>>>>>>> Updated patch 538 attached.
>>>>>>>
>>>>>> Jan 12 12:31:43 master.ipa.test
>>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: Connected
>>>>>> Jan 12 12:31:44 master.ipa.test
>>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: replica pub keys in LDAP:
>>>>>> set(['0xf5edad67436d0ed36b75c3a70216fa43',
>>>>>> '0x7164a931484d505f1e249e3dcbc313e2'])
>>>>>> Jan 12 12:31:44 master.ipa.test
>>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: replica pub keys in 
>>>>>> SoftHSM:
>>>>>> set(['0xf5edad67436d0ed36b75c3a70216fa43',
>>>>>> '0x7164a931484d505f1e249e3dcbc313e2', '0x28e302ae6b6ee7e9284cd5f6
>>>>>> Jan 12 12:31:44 master.ipa.test
>>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: new replica keys in LDAP:
>>>>>> set([])
>>>>>> Jan 12 12:31:44 master.ipa.test
>>>>>> /usr/libexec/ipa/ipa-ods-exporter[31178]: obsolete replica keys in
>>>>>> local
>>>>>> HSM: set(['0x28e302ae6b6ee7e9284cd5f61aadbbe7'])
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: Traceback
>>>>>> (most
>>>>>> recent call last):
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 664, in <module>
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>>> ldap2master_replica_keys_sync(log, ldapkeydb, localhsm)
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 321, in
>>>>>> ldap2master_replica_keys_sync
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>>> localhsm.replica_pubkeys_wrap[key_id]['ipk11wrap'] = False
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/localhsm.py", 
>>>>>> line
>>>>>> 65, in __setitem__
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: return
>>>>>> self.p11.set_attribute(self.handle, attrs_name2id[key], value)
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: File
>>>>>> "/usr/lib/python2.7/site-packages/ipapython/p11helper.py", line
>>>>>> 1661, in
>>>>>> set_attribute
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]:
>>>>>> sizeof(CK_ATTRIBUTE)))
>>>>>> Jan 12 12:31:44 master.ipa.test ipa-ods-exporter[31178]: 
>>>>>> TypeError: an
>>>>>> integer is required
>>>>>> Jan 12 12:31:44 master.ipa.test systemd[1]: 
>>>>>> ipa-ods-exporter.service:
>>>>>> Main process exited, code=exited, status=1/FAILURE
>>>>>>
>>>>>
>>>>> Updated patch 537 attached.
>>>>>
>>>> Jan 12 15:04:10 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: Connected
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: replica pub keys in LDAP:
>>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: replica pub keys in SoftHSM:
>>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: new replica keys in LDAP:
>>>> set([])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: obsolete replica keys in 
>>>> local
>>>> HSM: set([])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: keys in local HSM & LDAP:
>>>> set(['0x0e3dfd7343999d2ea7d17ac4ce15e4ca',
>>>> '0x9fc77beeb4b8ef33402e4fbb67d9b5e1'])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: Updating attribute
>>>> ipk11verifyrecover from "1" to "False"
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: master keys in local HSM:
>>>> set([])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: master keys in LDAP HSM:
>>>> set([])
>>>> Jan 12 15:04:11 master.ipa.test
>>>> /usr/libexec/ipa/ipa-ods-exporter[20652]: new master keys in local 
>>>> HSM:
>>>> set([])
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: Traceback 
>>>> (most
>>>> recent call last):
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 665, in <module>
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>>> master2ldap_master_keys_sync(log, ldapkeydb, localhsm)
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/libexec/ipa/ipa-ods-exporter", line 340, in
>>>> master2ldap_master_keys_sync
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>>> ldapkeydb.flush()
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 311, in flush
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>>> self._update_keys()
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 307, in _update_keys
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>>> key._update_key()
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 179, in _update_key
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]:
>>>> self._cleanup_key()
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 170, in _cleanup_key
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: if
>>>> self.get(attr, empty) == default_attrs[attr]:
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib64/python2.7/_abcoll.py", line 382, in get
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: return
>>>> self[key]
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 132, in __getitem__
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: val =
>>>> ldap_bool(val)
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: File
>>>> "/usr/lib/python2.7/site-packages/ipapython/dnssec/ldapkeydb.py", line
>>>> 39, in ldap_bool
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: raise
>>>> AssertionError('invalid LDAP boolean "%s"' % val)
>>>> Jan 12 15:04:11 master.ipa.test ipa-ods-exporter[20652]: 
>>>> AssertionError:
>>>> invalid LDAP boolean "1"
>>>> Jan 12 15:04:11 master.ipa.test systemd[1]: ipa-ods-exporter.service:
>>>> Main process exited, code=exited, status=1/FAILURE
>>>>
>>>>
>>>> You can run the dnssec test, it has been fixed.
>>>
>>> Updated patches attached. The test now passes.
>>>
>> Hello,
>>
>> pkcs11helper tests passed
>> DNSSEC tests passed
>>
>> 1)
>> Slot is unused argument here:
>>
>>     def __init__(self, slot, user_pin, library_path):
>>          self.p11_ptr = new_ptr(CK_FUNCTION_LIST_PTR)
>>          self.session_ptr = new_ptr(CK_SESSION_HANDLE)
>>
>>          self.slot = 0
>
> Fixed.
>
>>
>> 2)
>> should't string_to_pybytes_or_none raise exception instead of returning
>> None? In C extension returning NULL means error, and exception was
>> raised by python itself when function ends with returning NULL.
>>
>> in export_wrapped_key method
>>
>>          result = string_to_pybytes_or_none(wrapped_key,
>> wrapped_key_len_ptr[0])
>>          return result
>>
>> In this case method returns None instead of raising exception.
>>
>> Also I think that in _export_RSA_public_key method,
>> string_to_pybytes_or_none should raise exception when it get NULL as
>> string too
>
> This is exactly how it behaves in the original C code, so I'm not 
> changing it.
>
> I noticed I don't return None in _export_RSA_public_key in case of 
> encoding failure. Fixed.
>
OK
>>
>> 3)
>> Is possible to remove build dependencies added in commit c909690c ?
>
> Removed, except for openssl-devel, which is used elsewhere.
>
> Updated patches attached.
>

ACK

Pushed to master: b808376e2f516297dfb8311e514e31f2933bce01
Pushed to ipa-4-3: 5cfb95355353f040b38dabe957a1f453e4318900

More information about the Freeipa-devel mailing list