[Pulp-dev] Database support in Pulp 3

Mon Jul 15 18:32:38 UTC 2019

I filed an issue[0] to track the effort to remove MariaDB support in Pulp 3.

[0] https://pulp.plan.io/issues/5129

On Mon, Jul 15, 2019 at 1:01 PM Robin Chan <rchan at redhat.com> wrote:

> +1
> In May, the Foreman community warned us against trying to support many
> databases - saying the cost was too high.  Without any compelling reason to
> abandon earlier efforts to stay compatible, no action was taken at that
> time since the cost was nominal until now. With the new information, I
> agree the list of reasons in this thread justify the decision to drop
> MariaDB/MySQL.
>
> Mixed plugin support is the #1 reason for me.
>
> Robin Chan
> Satellite Software Engineering Manager - Pulp
>
>
>
> On Mon, Jul 15, 2019 at 11:59 AM Mike DePaulo <mikedep333 at redhat.com>
> wrote:
>
>> +1 to drop
>>
>> My rationale:
>> At a previous job, MariaDB support would have helped us; we could use our
>> cluster.
>> But a non-clustered postgresql server + Pulp having a lot more features
>> (due to the limitations of MariaDB, and the saved developer time) would
>> have been overall more valuable.
>>
>> -Mike
>>
>> On Mon, Jul 15, 2019 at 11:06 AM Dana Walker <dawalker at redhat.com> wrote:
>>
>>> +1 to drop MariaDB testing/support
>>>
>>> Dana Walker
>>>
>>> She / Her / Hers
>>>
>>> Software Engineer, Pulp Project
>>>
>>> Red Hat <https://www.redhat.com>
>>>
>>> dawalker at redhat.com
>>> <https://www.redhat.com>
>>>
>>>
>>>
>>> On Mon, Jul 15, 2019 at 10:41 AM Ina Panova <ipanova at redhat.com> wrote:
>>>
>>>> +1 to drop MariaDB support.
>>>>
>>>>
>>>> --------
>>>> Regards,
>>>>
>>>> Ina Panova
>>>> Senior Software Engineer| Pulp| Red Hat Inc.
>>>>
>>>> "Do not go where the path may lead,
>>>>  go instead where there is no path and leave a trail."
>>>>
>>>>
>>>> On Sun, Jul 14, 2019 at 10:10 PM Brian Bouterse <bbouters at redhat.com>
>>>> wrote:
>>>>
>>>>> I believe we have reached a point where Pulp (core and its plugins)
>>>>> can no longer support MariaDB due to technical problems. I've been an
>>>>> advocate for Pulp to support MariaDB because it's what our users want. The
>>>>> community survey has 16 respondents (IIRC) and 30% of them said they wanted
>>>>> to use MariaDB. Also, anecdotally at conference booths, users want choice
>>>>> in their database. To not give them that, we need good, technical reasons.
>>>>> I didn't feel we had enough of them before, but now I feel we're at a point
>>>>> where we aren't able to solve these issues; it's becoming a choice of
>>>>> giving users features they want versus db portability.
>>>>>
>>>>> Here are the main reasons I think about (mostly what others have also
>>>>> stated):
>>>>>
>>>>> * utf8 issues - 3-byte utf-8 issues   <---- is this fixable with just
>>>>> a schema change?
>>>>> * full text search - pulp_ansible recently implemented full text
>>>>> search. To my knowledge it's not possible with MySQL. full text search is
>>>>> amazing, and it's driving pulp_ansible to drop postgreSQL. I bet all
>>>>> plugins will want this. With MariaDB, there is no search.
>>>>> * mixed plugin support concerns - With some plugins dropping support
>>>>> for MariaDB, the pulp community will fracture based on what DB you choose
>>>>> to use because not all plugins can run in all places.
>>>>> * specific field support - plugin writers have expressed the desire to
>>>>> store json data when their plugins natively contain json data and perhaps
>>>>> you want to filter on it for example. Having mariaDB support prevents us
>>>>> from using the JSONField.
>>>>> * performance concerns - The performance testing showed that Pulp does
>>>>> run significantly slower
>>>>>
>>>>> If we drop MariaDB we should publish a blog post and drop it with RC4.
>>>>> To remove MariaDB testing from Travis I propose we remove it from the
>>>>> plugin_template and use the Travis CI tool from @dkliban to push that
>>>>> config out to all repositories.
>>>>>
>>>>> I'll be offline this week. I wanted to get this reply out there in the
>>>>> hope that you all can make and enact the final decision.
>>>>>
>>>>> Thanks,
>>>>> Brian
>>>>>
>>>>>
>>>>> On Thu, Jul 11, 2019 at 4:02 PM Daniel Alley <dalley at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> One more note:  Not all MySQL / MariaDB installations support
>>>>>> transactions, which we use heavily (and rely on?)
>>>>>>
>>>>>>
>>>>>> https://docs.djangoproject.com/en/2.2/topics/db/transactions/#transactions-in-mysql
>>>>>>
>>>>>> On Thu, Jul 11, 2019 at 3:55 PM David Davis <daviddavis at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Two plugins are currently running into issues trying to support
>>>>>>> mariadb/mysql. The pulp_ansible plugin is interested in adding full text
>>>>>>> search and JSONFields. Meanwhile, the pulp_python plugin is trying to store
>>>>>>> emojis in text which mariadb/mysql doesn't handle well since it uses 3-byte
>>>>>>> utf-8 by default[0]. Given such cases, I wonder if we'd be better served by
>>>>>>> dropping mariadb/mysql support and going with Postgresql only. Gitlab
>>>>>>> recently came to a similar conclusion[1].
>>>>>>>
>>>>>>> I personally am hesitant to give up being database agnostic but we
>>>>>>> already don't support sqlite. Also, I see some advantages like utilizing
>>>>>>> several Postgresql-only features like extra field types, full text search,
>>>>>>> etc. Also, supporting multiple databases means we'll likely have to write
>>>>>>> db specific code in some places or have plugins that only work with certain
>>>>>>> database types.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> [0]
>>>>>>> https://medium.com/@adamhooper/in-mysql-never-use-utf8-use-utf8mb4-11761243e434
>>>>>>>      https://code.djangoproject.com/ticket/18392
>>>>>>> [1] https://about.gitlab.com/2019/06/27/removing-mysql-support/
>>>>>>>
>>>>>>> David
>>>>>>> _______________________________________________
>>>>>>> Pulp-dev mailing list
>>>>>>> Pulp-dev at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Pulp-dev mailing list
>>>>>> Pulp-dev at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
>>
>> --
>>
>> Mike DePaulo
>>
>> He / Him / His
>>
>> Service Reliability Engineer, Pulp
>>
>> Red Hat <https://www.redhat.com/>
>>
>> IM: mikedep333
>>
>> GPG: 51745404
>> <https://www.redhat.com/>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190715/51be50e9/attachment.htm>