[katello-devel] search over rest api - interface design

[Amos Benari]

----- Original Message -----
> From: "Lukas Zapletal" <lzap at redhat.com>
> To: katello-devel at redhat.com
> Sent: Thursday, July 14, 2011 4:07:24 PM
> Subject: Re: [katello-devel] search over rest api - interface design
> On 07/14/2011 12:01 PM, Amos Benari wrote:
 ... 
> Well I am surprised that scoped_search is not a fulltext. As a
> long-time Apache Lucene user I have to recommend this project. 

Lucene vs. scoped search
-------------------------
I am not really going to compere them because it's a bit like comparing oranges to apples.
I am just going to write a short description of scoped search and try to point what use case will each fit.

scoped_search is not an indexer, it's also not a data store. It's a light weight query parser, auto-completer and SQL query builder.
It enables feature rich search box in the GUI and a powerful API, on top of any RDBMS, no schema change is needed. 
scoped search on top of RDBMS can easily handle structured data, it returns updated data as soon as it's stored in the database. 
No scoring, no ranking, just a simple "order by" clause in the SQL. It will scale as much as the underline database does.

Lucene will index your data, store it in a document format optimized for searching.
It will scale-out well as long as you can live with eventual consistency.

So does Lucene it fit our needs?
In Kalpana, Foreman and Candlepin the data set is not large enough to make a scale issue.
In Pulp we already have a no-sql data store (mongoDB) with it's own search interface.

Adding an external index store for searching can be an interesting idea, but it comes in the cost of:
modeling the searchable data into document format.
Updating the index on write, otherwise when a user update an item, he might not see the changes he made reflected in the GUI.
RBAC model needs to be re-implemented in the index data store to prevent reading unauthorized data.

Seems to me a bit of a complex solution.
Amos.