[Freeipa-devel] [PATCH 0042] Flush zones and RRs cache when handling persistent search reconnection

[Simo Sorce]

----- Original Message -----
> On 08/14/2012 08:25 PM, Simo Sorce wrote:
> > See man ldap_result, the entries return with type
> > LDAP_RES_SEARCH_ENTRY, the last message is instead
> > LDAP_RES_SEARCH_RESULT which tells you the searc is complete.
> >
> > This last message is never sent for a persistent search of course
> > (IIRC :-).
> 
> Right, it is what I tried to say with "there is no SearchResultDone
> LDAP message".

I see.

> http://tools.ietf.org/html/draft-smith-psearch-ldap-01#page-3
> section 4 paragraph b).
> 
> This patch deals with persistent search only. Non-persistent search
> scenarios
> have cache records with limited TTL value.
> 
> >
> > But in case of a persistent search you should never need to flush
> > as entries are valid until you see a change.
> Unfortunately, cache flush is necessary after persistent search
> reconnection.
> Records can change in meanwhile...
> 
> Example:
> 1. BIND started, cache was populated.
> 2. Persistent search connection was lost
> 3. Record was deleted (but Entry Change Notification wasn't
> delivered)
> 4. Persistent search connection was re-established - there is no
> information
> about deleted record/zone, BIND still sees old data in the cache.
> 
> For this reason https://fedorahosted.org/bind-dyndb-ldap/ticket/44
> was filled.
> 
> What I missed?

You missed nothing, I did.

Howeve I have an idea to handle this situation smartly.

We can issue 2 searches on 2 separate connections.

The first search will be a persistent search, however it will be started with a filter that has an additional element.
Before the persistent search we do a rootdse search and find either the higher modifyTimestamp or the higher entryUSN or equivalent, depending on what is available in the server. Worst case we do a ase search of the DNS tree and pick that modifyTimestamp.
Then we start the persistent search with (&(entryUSN>%d)<normalfilter>) or (&(modifyTimestamp>=%d<normalfilter>) (note modifyTimestamp requires >= due to low resolution).

On the other connection we start instead a nornmal search. This normal search will terminate with a 'done' result, so you know that once that search is finished you can flush any cache that has not been re-validated. While you get any changes that are occurring live on the persistent search connection.

This way we should be able to not loose any update and still know when we got all results and drop unvalidated caches.

HTH,
Simo.