[Pulp-list] [devel] Pulp Performance Metrics

[Michael Hrivnak]

+1

An obvious challenge is stabilizing external variables, but they're all
solvable problems. We especially need hardware performance and network
performance to be consistent across test runs. That likely means dedicated
hardware, and a local copy of any content being sync'd. We should consider
imposing a limit on bandwidth to the local content.

Some low-priority nice-to-haves for sanity-checking:
- start each run with a quick pure disk benchmark to ensure disk
performance hasn't changed
- also consider a pure database performance benchmark as a quick way to
identify if its performance has changed

An alternative to controlling these variable over a long period of time;
any time a new version is ready for testing, re-run benchmarks of all
versions you're interested in on whatever hardware is available at the time.

Is one of those approaches a better fit than the other for lucene?

Michael


On Wed, Mar 9, 2016 at 12:11 PM, Brian Bouterse <bbouters at redhat.com> wrote:

> You know what's great? Knowing how code changes affect performance of a
> software project over time. To use an example from a completely separate
> project, here are performance benchmarks published by the Lucene
> project[0]. Click on the links in the "Results" section of that page to
> see the graphs. We could do the same thing but for Pulp.
>
> We need to plan this in more detail, but as a starting point, we could
> do the following for rpm content in particular.
>
> -Fresh metadata runtime of a large repo sync
> -Re-sync metadata runtime of a large repo sync with no packages modified
>
> -Fresh package download and associate runtime of a large repo sync
> -Re-sync package download and associate runtime of a large repo sync
>
> -Copy of 50k rpms from one repo to another without depsolve
> -Copy of 50k rpms from one repo to another with depsolve
>
> -Search performance using a simple Criteria filter
>
> -Fresh publish runtime of a large repo
> -Incremental (second) publish runtime of a large repo
>
> If you like this idea please +1 it via the mailing list and or give
> suggestions. Once we've got some support and/or ideas we can put it in
> pulp.plan.io.
>
> [0]: https://people.apache.org/~mikemccand/lucenebench/
>
> -Brian
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20160309/d2d1bd4e/attachment.htm>