[Pulp-dev] SQlite Issues w/ Pulp3. Switch back to PostgreSQL

Brian Bouterse bbouters at redhat.com
Wed Apr 25 21:18:49 UTC 2018

On Wed, Apr 25, 2018 at 3:16 PM, Brian Bouterse <bbouters at redhat.com> wrote:

> tldr: A bug in sqlite is preventing Pulp from running pulp-smash tests
> without error making sqlite a non-viable default for Pulp. We should switch
> back to Postgres as the default for Pulp and its docs, the dev environment,
> and remove sqlite from the Travis matrix.
> Q: Why isn't the timeout [0] being applied?
> A: because it's broken in sqlite's C code (the evidence suggests)
> Q: What units is the timeout in exactly?
> A: it's in seconds and our configuration of it was correct. In cPython's
> code it gets converted to milliseconds which is the right unit for SQLite's
> interface.
> Q: If this gets fixed in the future will we switch back?
> A: Once we enter RC or GA for core we cannot switch the default. So if
> this gets resolved "real soon" by the sqlite folks then yes, otherwise no.
> Q: How do you know it's broken?
> A: I set the timeout to 300 seconds, but the database locked error is
> still raised after the default of 5 seconds. I use the `PRAGMA
> busy_timeout;` to confirm that the sqlite knows the 300 second timeout has
> been set. I've analyzed the django [1] and Python C [2] code also to
> confirm that our setting is being handled properly all the way down to
> sqlite's C interface. I've raised the issue with the sqlite community and
> I'm waiting to hear back.
> Q: What are the next steps?
> A: We need issues to switch Pulp's settings and docs to Postgresql.
> Similarly for switching the devel environment. Also one to remove sqlite
> from Travis. We need these like real soon.

I think these two issues should be groomed and added to the sprint. There
is outstanding that can't merge because of this issue causes Travis tests
to fail which blocks merging.


> Q: What about the community's concerns?
> A: They can help contribute to the sqlite bug that is prevent Pulp from
> being meaningfully used on top of sqlite. They can still configure sqlite
> and use it just like any django supported db, we just don't test on top of
> it on Travis anymore. It is erroring almost all of our Travis jobs.
> In terms of the how we document PostgreSQL, I want to advocate avoiding
> documenting command by command installs of Postgres. I used to feel
> differently about this, but I think if users start on our docs and they
> don't work, we're probably not going to help them get their Postgresql
> going on some random distro. Soon the Ansible installer will solve this for
> our users anyway so I don't think it's a big problem.
> Ideas, concerns, help writing issues, and questions are all welcome!
> [0]: https://docs.djangoproject.com/en/1.11/ref/databases/#databa
> se-is-locked-errors
> [1]: https://github.com/django/django/blob/1.11.12/django/db/back
> ends/sqlite3/base.py#L177
> [2]: https://github.com/python/cpython/blob/master/Modules/_sqlit
> e/connection.c#L181
> -Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180425/0f9f541d/attachment.htm>

More information about the Pulp-dev mailing list