[Pulp-dev] Pagination in Pulp

[David Davis]

I was looking at the pagination code this morning and there were two things
I wanted to discuss.

First, there’s no way to override the number of results per request.
Instead, page size has to be configured for the whole app. Allowing users
to override page size is trivial[0] so I wonder if we should enable it.

The second topic is a bit more complex. We currently use cursor based
pagination where pages must be fetched sequentially as opposed to the
default DRF pagination method of using page numbers. Cursors work great for
large data sets as you don’t have to figure out things like the number of
pages.

The first problem is that in Pulp we parallelize web requests for things
like fetching metadata. See our Ansible plugin as an example[1]. If we want
to support things like syncing content from one Pulp server to another, we
probably have to use offset/page-based pagination for certain endpoints.

Another consideration is Katello. In Katello’s UI they show the number of
pages and allow users to jump to arbitrary pages or the last page. If we
want Katello to stop indexing Pulp data and instead query Pulp directly,
we’ll need to allow them to use page numbers somehow.

Thoughts?

[0] https://gist.github.com/daviddavis/56a0b86629cd675d57aac61583c01944
[1]
https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/tasks/synchronizing.py#L147-L171

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180626/a3501b1c/attachment.htm>