[Pulp-list] pagination design

Jay Dobies jason.dobies at redhat.com
Fri Mar 11 13:32:29 UTC 2011


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2011 08:14 AM, Todd B Sanders wrote:
> On 03/11/2011 03:58 AM, Jason L Connor wrote:
>> Hi All,
>>
>> A quick write up on pagination support for pulp. Comments welcome.
>> https://fedorahosted.org/pulp/wiki/Pagination
>>
>>
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
> Thanks Jason.  Question.  Suppose I have 200 packages in a Repo, and I
> want to present 25 packages per page to the end user.  Obviously this
> would give me 8 pages.  What would happen if packages are added/removed
> from the Repo while I am paging through the data?  Since we are hitting
> mongo for each page set, won't this skew the paging?

Interesting question. Thinking out loud, I think this is an inevitability.

- - We can't preload them into memory, that would get us into trouble very
quickly.
- - We can't lock the database from changes while something is being
paginated for a bunch of reasons.
- - I worry using some sort of view-like concept in the database would get
us into trouble as well. By view-like I mean we copy the data they are
paginating over to some temporary area and they iterate over that. Then
we'd have to get into expiration of that data, invalidation if they
re-execute the same query, and reapers to clean it all up since they may
not tell us they are done with the data set. It'd be kinda brutal

That's just first thoughts on your question though, so don't take that
as my final answer. More just thinking out loud.


- -- 
Jay Dobies
RHCE# 805008743336126
Freenode: jdob
http://pulpproject.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJNeiRtAAoJEOMmcTqOSQHCMrgH/3vjgigJwk2tNbJ+zGFGzVOz
VJX1hrQA30yqZUIJhvfQYy+VH4DBOXLCoOrYQTWeKIrzufU1zLJNxIOvuu5/EL5J
wOVd6Yl/5WF52k3B52kR94fJDjp7mCL9ttPZbHVyoFpxjY6fq5a8KPS8WfYzxWXu
PjB7odK+N9iyti+bwkNuPALa189IztbmgXIqczf0foqdJpuSTfe9xeU6vMP7EDVh
Mp4TXx35Kr1nzEF0s1/xGlCG78UD1RTSS6TKN37gM06ibWfMej3JkM/n3tIWELhT
5ZZ1/Zva2p/LLBqBt2JsCN+zppiZaClMStjreqHOvXL7QG4kvP1InI73FY8Mcp0=
=9y7G
-----END PGP SIGNATURE-----




More information about the Pulp-list mailing list