[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] pagination design

On 03/11/2011 08:32 AM, Jay Dobies wrote:
Hash: SHA1

On 03/11/2011 08:14 AM, Todd B Sanders wrote:
On 03/11/2011 03:58 AM, Jason L Connor wrote:
Hi All,

A quick write up on pagination support for pulp. Comments welcome.

Pulp-list mailing list
Pulp-list redhat com
Thanks Jason.  Question.  Suppose I have 200 packages in a Repo, and I
want to present 25 packages per page to the end user.  Obviously this
would give me 8 pages.  What would happen if packages are added/removed
from the Repo while I am paging through the data?  Since we are hitting
mongo for each page set, won't this skew the paging?

Interesting question. Thinking out loud, I think this is an inevitability.

- - We can't preload them into memory, that would get us into trouble very
- - We can't lock the database from changes while something is being
paginated for a bunch of reasons.
- - I worry using some sort of view-like concept in the database would get
us into trouble as well. By view-like I mean we copy the data they are
paginating over to some temporary area and they iterate over that. Then
we'd have to get into expiration of that data, invalidation if they
re-execute the same query, and reapers to clean it all up since they may
not tell us they are done with the data set. It'd be kinda brutal

That's just first thoughts on your question though, so don't take that
as my final answer. More just thinking out loud.

If you look at many web sites (facebook, digg, reddit) they all suffer from this. I would suggest it is ok. As is not knowing the complete size of data set (see google)

-- bk

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]