[Pulp-list] pagination design

[Todd B Sanders]

On 03/11/2011 08:37 AM, Bryan Kearney wrote:
> On 03/11/2011 08:32 AM, Jay Dobies wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 03/11/2011 08:14 AM, Todd B Sanders wrote:
>>> On 03/11/2011 03:58 AM, Jason L Connor wrote:
>>>> Hi All,
>>>>
>>>> A quick write up on pagination support for pulp. Comments welcome.
>>>> https://fedorahosted.org/pulp/wiki/Pagination
>>>>
>>>>
>>>> _______________________________________________
>>>> Pulp-list mailing list
>>>> Pulp-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>> Thanks Jason.  Question.  Suppose I have 200 packages in a Repo, and I
>>> want to present 25 packages per page to the end user.  Obviously this
>>> would give me 8 pages.  What would happen if packages are added/removed
>>> from the Repo while I am paging through the data?  Since we are hitting
>>> mongo for each page set, won't this skew the paging?
>>
>> Interesting question. Thinking out loud, I think this is an 
>> inevitability.
>>
>> - - We can't preload them into memory, that would get us into trouble 
>> very
>> quickly.
>> - - We can't lock the database from changes while something is being
>> paginated for a bunch of reasons.
>> - - I worry using some sort of view-like concept in the database 
>> would get
>> us into trouble as well. By view-like I mean we copy the data they are
>> paginating over to some temporary area and they iterate over that. Then
>> we'd have to get into expiration of that data, invalidation if they
>> re-execute the same query, and reapers to clean it all up since they may
>> not tell us they are done with the data set. It'd be kinda brutal
>>
>> That's just first thoughts on your question though, so don't take that
>> as my final answer. More just thinking out loud.
>>
>>
>
> If you look at many web sites (facebook, digg, reddit) they all suffer 
> from this. I would suggest it is ok. As is not knowing the complete 
> size of data set (see google)
>
> -- bk

Fair enough.  Just figured I'd ask.  Does the current Satellite UI 
suffer from this side-effect?

-Todd
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list