[katello-devel] search over rest api - interface design

[Amos Benari]

----- Original Message -----
> From: "Justin Sherrill" <jsherril at redhat.com>
> To: katello-devel at redhat.com
> Sent: Monday, July 18, 2011 4:28:17 AM
> Subject: Re: [katello-devel] search over rest api - interface design
> On 07/17/2011 11:32 AM, Amos Benari wrote:
> > ----- Original Message -----
> >> From: "Lukas Zapletal"<lzap at redhat.com>
> >> To: katello-devel at redhat.com
> >> Sent: Thursday, July 14, 2011 4:07:24 PM
> >> Subject: Re: [katello-devel] search over rest api - interface
> >> design
> >> On 07/14/2011 12:01 PM, Amos Benari wrote:
> >   ...
> >> Well I am surprised that scoped_search is not a fulltext. As a
> >> long-time Apache Lucene user I have to recommend this project.
> > Lucene vs. scoped search
> > -------------------------
> > I am not really going to compere them because it's a bit like
> > comparing oranges to apples.
> > I am just going to write a short description of scoped search and
> > try to point what use case will each fit.
> >
> > scoped_search is not an indexer, it's also not a data store. It's a
> > light weight query parser, auto-completer and SQL query builder.
> > It enables feature rich search box in the GUI and a powerful API, on
> > top of any RDBMS, no schema change is needed.
> > scoped search on top of RDBMS can easily handle structured data, it
> > returns updated data as soon as it's stored in the database.
> > No scoring, no ranking, just a simple "order by" clause in the SQL.
> > It will scale as much as the underline database does.
> >
> > Lucene will index your data, store it in a document format optimized
> > for searching.
> > It will scale-out well as long as you can live with eventual
> > consistency.
> >
> > So does Lucene it fit our needs?
> 
> > In Kalpana, Foreman and Candlepin the data set is not large enough
> > to make a scale issue.
> 
> Is this true? From earlier discussions I seem to remember that Katello
> should be able to support ~100,000 systems eventually. I would think
> this could indicate scale may be an issue?

Well, I guess that if the rest of the application is going to behave nice with
 ~100K systems then a search over the database should be fine.
I think that the main area of concern here is if we are going to have a very high
rate of updates to the systems table, it might slow read operation significantly.
Otherwise, indexes and cache of the database engine should be able handle that scale.
Amos.
 
> 
> > In Pulp we already have a no-sql data store (mongoDB) with it's own
> > search interface.
> >
> > Adding an external index store for searching can be an interesting
> > idea, but it comes in the cost of:
> > modeling the searchable data into document format.
> > Updating the index on write, otherwise when a user update an item,
> > he might not see the changes he made reflected in the GUI.
> > RBAC model needs to be re-implemented in the index data store to
> > prevent reading unauthorized data.
> >
> > Seems to me a bit of a complex solution.
> > Amos.
> >
> > _______________________________________________
> > katello-devel mailing list
> > katello-devel at redhat.com
> > https://www.redhat.com/mailman/listinfo/katello-devel
> 
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel