[Pulp-dev] Transition from Mongo to Postgre
fnguyen at redhat.com
Wed Sep 14 11:16:02 UTC 2016
All that makes sense. Do you have any concerns/benchmarks regarding
performance after migration? Have you tried migrating your data in some
POC and measure the difference in some subset of functionality?
Just to give more context. The reason I am asking is that in Candlepin,
we always have had Postgres as a datastore. Some performance challenges
that we have been facing:
* The sheer amount of data on one node makes and the amount of time
necessary to migrate when changing schema
* Transaction lock waits - which, at occasion, can stop client
requests and bring down the service
* Transaction deadlocks
With every new functionality I sometimes think about a possibility to
use some NoSQL to make it horizontally scalable. So it would be
interesting to hear that Mongo didn't deliver the performance in Pulp's
On 09/13/2016 03:11 PM, Michael Hrivnak wrote:
> We have a thread here about a lot of the 3.0 stack choices, although
> it seems to skip past the assumption that we're moving to postgres:
> I can't quickly find another summary of why, so I'll describe the
> highlights here:
> - Pulp has highly relational data. The core use case is managing the
> relationships between content and repositories. Using a relational DB
> makes that a lot easier.
> - A schemaless DB makes it easy to do writes, but you have to be very
> careful when doing reads that the your software is prepared for
> whatever data structure comes out. If you want to enforce a schema, it
> has to be done in software. It's doable, but requires great care.
> - Transactions!
> - The HA story with mongodb is more complex than most people realize
> (certainly more complex than we expected). To get real HA with data
> safety, you have to do a lot of the work in your own software.
> MongoDB is great at what it does and a good fit for some use cases,
> but we learned that it's not the best fit for Pulp.
> On Tue, Sep 13, 2016 at 3:21 AM, Filip Nguyen <fnguyen at redhat.com
> <mailto:fnguyen at redhat.com>> wrote:
> I heard that Pulp is switching from Mongo to Postgre. Just out of
> curiosity, I would like to learn more about the reasons why you
> decided to go this direction. Is there any document/email thread
> about it?
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pulp-dev