[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Pulp-dev] SQlite Issues w/ Pulp3. Switch back to PostgreSQL



tldr: A bug in sqlite is preventing Pulp from running pulp-smash tests without error making sqlite a non-viable default for Pulp. We should switch back to Postgres as the default for Pulp and its docs, the dev environment, and remove sqlite from the Travis matrix.

Q: Why isn't the timeout [0] being applied?
A: because it's broken in sqlite's C code (the evidence suggests)

Q: What units is the timeout in exactly?
A: it's in seconds and our configuration of it was correct. In cPython's code it gets converted to milliseconds which is the right unit for SQLite's interface.

Q: If this gets fixed in the future will we switch back?
A: Once we enter RC or GA for core we cannot switch the default. So if this gets resolved "real soon" by the sqlite folks then yes, otherwise no.

Q: How do you know it's broken?
A: I set the timeout to 300 seconds, but the database locked error is still raised after the default of 5 seconds. I use the `PRAGMA busy_timeout;` to confirm that the sqlite knows the 300 second timeout has been set. I've analyzed the django [1] and Python C [2] code also to confirm that our setting is being handled properly all the way down to sqlite's C interface. I've raised the issue with the sqlite community and I'm waiting to hear back.

Q: What are the next steps?
A: We need issues to switch Pulp's settings and docs to Postgresql. Similarly for switching the devel environment. Also one to remove sqlite from Travis. We need these like real soon.

Q: What about the community's concerns?
A: They can help contribute to the sqlite bug that is prevent Pulp from being meaningfully used on top of sqlite. They can still configure sqlite and use it just like any django supported db, we just don't test on top of it on Travis anymore. It is erroring almost all of our Travis jobs.

In terms of the how we document PostgreSQL, I want to advocate avoiding documenting command by command installs of Postgres. I used to feel differently about this, but I think if users start on our docs and they don't work, we're probably not going to help them get their Postgresql going on some random distro. Soon the Ansible installer will solve this for our users anyway so I don't think it's a big problem.

Ideas, concerns, help writing issues, and questions are all welcome!

[0]: https://docs.djangoproject.com/en/1.11/ref/databases/#database-is-locked-errors
[1]: https://github.com/django/django/blob/1.11.12/django/db/backends/sqlite3/base.py#L177
[2]: https://github.com/python/cpython/blob/master/Modules/_sqlite/connection.c#L181

-Brian



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]