[katello-devel] Bundler vs rpm-gems - in more detail

[Petr Chalupa]

My original email raised a lot of questions, I should wrote it with more 
details. I apology for that. I'll try again and elaborate.

= Goals

1) To be able to implement the solution in a week or so.
2) Not to affect deployment, keep it same way as now.
3) When you are installing development environment on any rpm-based 
system rpm-gems are used even if they are older then gems in our 
gem-repo. (this is borken right now)
4) When you are installing development environment in any non-rpm-base 
system you get installed gems as close as possible to what would you get 
on fedora 16. (you cannot install with bundle install only right now)
   * (optionally including the security patches, lets discuss this later)

= What was discussed, why I think it wont work

== Branch for each platform

* There would be little difference between the branches (only Gemfile or 
Gemfile.lock). Rest of the code will be same.
* It would be too much work to maintain it.

== Gemfile.lock per platform and environment

* We would have to have 8 Gemfile.locks and maintain them all (4 
platforms multiplied by two environments)
* Too much work unless its automated.
* Automation would take time to setup.

== Aeolus solution, bundler_ext [2]

* Dependencies resolution is unaffected so 'bundle install' cannot be 
used to install development dependencies, it would ignore rpm-gems and 
install gems from gem-repo.
* If bundler_ext is used for setup development environment, control over 
which versions are required is lost (requiring is not done by bundler)
* It affects only how requirements are loaded [1], it's useful only for 
deployment.
* (We could use it in deployment to get rid of bundler, but that is out 
of this discussion's scope see Goal.2)

== bundle install --without default

This installs only development gems, but:

* If some of the development gems has a production gem as dependency 
(eg. ActiveSupport), the gem is reinstalled by bundler from a gem-repo 
over the older rpm-gem.
* If only development gems are installed, bundler cannot be used to 
require gems. It will raise LoadError because from bundler's point of 
view production gems are not installed.
* It would have to be combined with bundler_ext to find production 
rpm-gems, so you would loose control over which version are loaded again
* If you forget to add '--without default' when you run 'bundle install' 
unwanted versions are installed.

== vendor/cache

Move all gems to src/vendor/cache (directly into katello/master or as a 
git sub-module) and remove sources form Gemfile completely. 'bundle 
install' installs gems form vendor/cache automatically if there are no 
sources in Gemfile.

* to have .gem files directly in katello/master is ugly
* submodules are real pain
* git is not that good with binary data

= My proposal

== Bundler patch

* it would be loaded in Gemfile, so every tool using bundler will get it
* it would ensure that if rpm-gem is present it's used even if there is 
newer version available in gem-repo
* ensure that locally installed gems are not confused with rpm-gems
* patch is optional, can be turned on/off by ENV variable or by options
* (I am also considering to add optional check that all production gems 
are installed as rpms)

Deployment is unaffected, all dependencies are satisfied by rpm-gems.

When you run bundle install to get development environment on any 
rpm-based machine, all your rpms have priority and they are used. All 
missing development gems are installed from our gem-repo.

On non-rpm-based machine, 'bundle install' installs all gems production 
and development from our gem-repo.

== Gem Repository

Bundler patch will work only if all development and production gems are 
in the gem-repo. To get it fixed quickly 'bundle package' can be used to 
get all the gems. Missing gems can be then uploaded to 
katello-thirdparty (source for our gem-repo)

= Summary

I think that Bundler patch and updating gem-repo to contain all gems is 
the only one which can be done fast and will work on any machine in 
deployment and development. I tried to be thorough but if I missed 
something or if you have better ideas please let me know.

Petr

[1] 
https://github.com/aeolusproject/conductor/blob/master/src/config/application.rb#L21 

[2] https://github.com/aeolus-incubator/bundler_ext