[Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection

[Petr Spacek]

On 08/27/2012 02:40 PM, Dmitri Pal wrote:
> On 08/15/2012 05:18 AM, Simo Sorce wrote:
>> ----- Original Message -----
>>> On 08/14/2012 08:25 PM, Simo Sorce wrote:
>>>> See man ldap_result, the entries return with type
>>>> LDAP_RES_SEARCH_ENTRY, the last message is instead
>>>> LDAP_RES_SEARCH_RESULT which tells you the searc is complete.
>>>>
>>>> This last message is never sent for a persistent search of course
>>>> (IIRC :-).
>>> Right, it is what I tried to say with "there is no SearchResultDone
>>> LDAP message".
>> I see.
>>
>>> http://tools.ietf.org/html/draft-smith-psearch-ldap-01#page-3
>>> section 4 paragraph b).
>>>
>>> This patch deals with persistent search only. Non-persistent search
>>> scenarios
>>> have cache records with limited TTL value.
>>>
>>>> But in case of a persistent search you should never need to flush
>>>> as entries are valid until you see a change.
>>> Unfortunately, cache flush is necessary after persistent search
>>> reconnection.
>>> Records can change in meanwhile...
>>>
>>> Example:
>>> 1. BIND started, cache was populated.
>>> 2. Persistent search connection was lost
>>> 3. Record was deleted (but Entry Change Notification wasn't
>>> delivered)
>>> 4. Persistent search connection was re-established - there is no
>>> information
>>> about deleted record/zone, BIND still sees old data in the cache.
>>>
>>> For this reason https://fedorahosted.org/bind-dyndb-ldap/ticket/44
>>> was filled.
>>>
>>> What I missed?
>> You missed nothing, I did.
>>
>> Howeve I have an idea to handle this situation smartly.
>>
>> We can issue 2 searches on 2 separate connections.
>>
>> The first search will be a persistent search, however it will be started with a filter that has an additional element.
>> Before the persistent search we do a rootdse search and find either the higher modifyTimestamp or the higher entryUSN or equivalent, depending on what is available in the server. Worst case we do a ase search of the DNS tree and pick that modifyTimestamp.
>> Then we start the persistent search with (&(entryUSN>%d)<normalfilter>) or (&(modifyTimestamp>=%d<normalfilter>) (note modifyTimestamp requires >= due to low resolution).
>>
>> On the other connection we start instead a nornmal search. This normal search will terminate with a 'done' result, so you know that once that search is finished you can flush any cache that has not been re-validated. While you get any changes that are occurring live on the persistent search connection.
>>
>> This way we should be able to not loose any update and still know when we got all results and drop unvalidated caches.
>>
>
> What is the status of this proposal?

Oh, this mail got lost in my inbox, sorry.

I created ticket:
https://fedorahosted.org/bind-dyndb-ldap/ticket/86
"Improve psearch cache flush reliability"

I will return back to this proposal after resolving more actual issues.

Petr^2 Spacek