[Pulp-dev] SQlite Issues w/ Pulp3. Switch back to PostgreSQL

Brian Bouterse bbouters at redhat.com
Wed Apr 25 19:16:58 UTC 2018

tldr: A bug in sqlite is preventing Pulp from running pulp-smash tests
without error making sqlite a non-viable default for Pulp. We should switch
back to Postgres as the default for Pulp and its docs, the dev environment,
and remove sqlite from the Travis matrix.

Q: Why isn't the timeout [0] being applied?
A: because it's broken in sqlite's C code (the evidence suggests)

Q: What units is the timeout in exactly?
A: it's in seconds and our configuration of it was correct. In cPython's
code it gets converted to milliseconds which is the right unit for SQLite's

Q: If this gets fixed in the future will we switch back?
A: Once we enter RC or GA for core we cannot switch the default. So if this
gets resolved "real soon" by the sqlite folks then yes, otherwise no.

Q: How do you know it's broken?
A: I set the timeout to 300 seconds, but the database locked error is still
raised after the default of 5 seconds. I use the `PRAGMA busy_timeout;` to
confirm that the sqlite knows the 300 second timeout has been set. I've
analyzed the django [1] and Python C [2] code also to confirm that our
setting is being handled properly all the way down to sqlite's C interface.
I've raised the issue with the sqlite community and I'm waiting to hear

Q: What are the next steps?
A: We need issues to switch Pulp's settings and docs to Postgresql.
Similarly for switching the devel environment. Also one to remove sqlite
from Travis. We need these like real soon.

Q: What about the community's concerns?
A: They can help contribute to the sqlite bug that is prevent Pulp from
being meaningfully used on top of sqlite. They can still configure sqlite
and use it just like any django supported db, we just don't test on top of
it on Travis anymore. It is erroring almost all of our Travis jobs.

In terms of the how we document PostgreSQL, I want to advocate avoiding
documenting command by command installs of Postgres. I used to feel
differently about this, but I think if users start on our docs and they
don't work, we're probably not going to help them get their Postgresql
going on some random distro. Soon the Ansible installer will solve this for
our users anyway so I don't think it's a big problem.

Ideas, concerns, help writing issues, and questions are all welcome!

[0]: https://docs.djangoproject.com/en/1.11/ref/databases/#
[1]: https://github.com/django/django/blob/1.11.12/django/db/
[2]: https://github.com/python/cpython/blob/master/Modules/_

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180425/b827b70b/attachment.htm>

More information about the Pulp-dev mailing list