[Pulp-dev] Pagination in Pulp

[David Davis]

I’ve opened the following issues:

https://pulp.plan.io/issues/3790
https://pulp.plan.io/issues/3801 - requires breaking changes

Feel free to leave feedback on the issues.

David


On Tue, Jun 26, 2018 at 3:58 PM Jeff Ortel <jortel at redhat.com> wrote:

>
>
> On 06/26/2018 08:48 AM, Dennis Kliban wrote:
>
> The user should be able to specify a page size at request time. The user
> should also be able to specify which page they are requesting.
>
>
> + 1
>
>
> On Tue, Jun 26, 2018 at 9:31 AM, David Davis <daviddavis at redhat.com>
> wrote:
>
>> I was looking at the pagination code this morning and there were two
>> things I wanted to discuss.
>>
>> First, there’s no way to override the number of results per request.
>> Instead, page size has to be configured for the whole app. Allowing users
>> to override page size is trivial[0] so I wonder if we should enable it.
>>
>> The second topic is a bit more complex. We currently use cursor based
>> pagination where pages must be fetched sequentially as opposed to the
>> default DRF pagination method of using page numbers. Cursors work great for
>> large data sets as you don’t have to figure out things like the number of
>> pages.
>>
>> The first problem is that in Pulp we parallelize web requests for things
>> like fetching metadata. See our Ansible plugin as an example[1]. If we want
>> to support things like syncing content from one Pulp server to another, we
>> probably have to use offset/page-based pagination for certain endpoints.
>>
>> Another consideration is Katello. In Katello’s UI they show the number of
>> pages and allow users to jump to arbitrary pages or the last page. If we
>> want Katello to stop indexing Pulp data and instead query Pulp directly,
>> we’ll need to allow them to use page numbers somehow.
>>
>> Thoughts?
>>
>> [0] https://gist.github.com/daviddavis/56a0b86629cd675d57aac61583c01944
>> [1]
>> https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/tasks/synchronizing.py#L147-L171
>>
>> David
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
>
> _______________________________________________
> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180702/b3701dcf/attachment.htm>