[Pulp-dev] Pagination Requirements and Defaults?

[Rohan McGovern]

Brian Bouterse <bmbouter at redhat.com> writes:

> Recently with pulp_ansible, users were interested in using pagination with
> LimitOffsetPagination [0]. Pulp currently defaults to PageNumberPagination.
> I looked at our current DRF defaults, and I noticed two things.
>
> 1. We default to the not-as-common PageNumberPagination based on examples
> in the drf docs.
> 2. We customize it here [1] in various ways.
>
> Can someone help me remember why these pagination style choices were made
> or where the requirements came from?
> Would our bindings work with a LimitOffsetPagination style?
> What use cases drove the use and customization in this area?
>
> Also, @katello how would a pagination style change (like switching to
> LimitOffsetPagination) affect you?
>
> Thanks for any info you can provide. Maybe what we have right now is just
> what we need, but I'm not sure.
>

With respect to page number/size vs limit/offset, I'm neutral and don't
see any reason our clients couldn't work equally well with either.

If you're looking into changes in this area, I only suggest that you
would keep it so that there is always a page size/limit applied by
default, whether or not the client requested it.  Ideally, a small-ish
limit to discourage that developers could forget about pagination until
it's too late.

This is to avoid a problem we've seen several times with Pulp 2.x now:

- search API by default doesn't paginate at all

- client writes naive search code which ignores pagination, requests all
  available data at once

- client tests their code against a small Pulp test server or repo
  and it appears to work fine

- later, when code is used against a real Pulp server/repo (=> much
  larger amount of data) or as the amount of data in Pulp grows, search
  becomes slow and prone to OOM crashes

- client blames Pulp for being slow, or service owners for not putting
  enough RAM on the Pulp servers

>
> [0]:
> https://www.django-rest-framework.org/api-guide/pagination/#setting-the-pagination-style
> [1]: https://github.com/pulp/pulpcore/blob/master/pulpcore/app/pagination.py