[katello-devel] Bundler vs rpm-gems - in more detail

Thu Aug 23 12:42:02 UTC 2012

On 23/08/12 12:56 PM, Petr Chalupa wrote:
> My original email raised a lot of questions, I should wrote it with 
> more details. I apology for that. I'll try again and elaborate.
>
> = Goals
>
> 1) To be able to implement the solution in a week or so.
> 2) Not to affect deployment, keep it same way as now.
> 3) When you are installing development environment on any rpm-based 
> system rpm-gems are used even if they are older then gems in our 
> gem-repo. (this is borken right now)
> 4) When you are installing development environment in any non-rpm-base 
> system you get installed gems as close as possible to what would you 
> get on fedora 16. (you cannot install with bundle install only right now)
>   * (optionally including the security patches, lets discuss this later)
>
> = What was discussed, why I think it wont work
>
> == Branch for each platform
>
> * There would be little difference between the branches (only Gemfile 
> or Gemfile.lock). Rest of the code will be same.
> * It would be too much work to maintain it.
>
> == Gemfile.lock per platform and environment
>
> * We would have to have 8 Gemfile.locks and maintain them all (4 
> platforms multiplied by two environments)
> * Too much work unless its automated.
> * Automation would take time to setup.
>
> == Aeolus solution, bundler_ext [2]
>
> * Dependencies resolution is unaffected so 'bundle install' cannot be 
> used to install development dependencies, it would ignore rpm-gems and 
> install gems from gem-repo.
> * If bundler_ext is used for setup development environment, control 
> over which versions are required is lost (requiring is not done by 
> bundler)
> * It affects only how requirements are loaded [1], it's useful only 
> for deployment.
> * (We could use it in deployment to get rid of bundler, but that is 
> out of this discussion's scope see Goal.2)
>
> == bundle install --without default
>
> This installs only development gems, but:
>
> * If some of the development gems has a production gem as dependency 
> (eg. ActiveSupport), the gem is reinstalled by bundler from a gem-repo 
> over the older rpm-gem.
> * If only development gems are installed, bundler cannot be used to 
> require gems. It will raise LoadError because from bundler's point of 
> view production gems are not installed.
> * It would have to be combined with bundler_ext to find production 
> rpm-gems, so you would loose control over which version are loaded again
> * If you forget to add '--without default' when you run 'bundle 
> install' unwanted versions are installed.
>
> == vendor/cache
>
> Move all gems to src/vendor/cache (directly into katello/master or as 
> a git sub-module) and remove sources form Gemfile completely. 'bundle 
> install' installs gems form vendor/cache automatically if there are no 
> sources in Gemfile.
>
> * to have .gem files directly in katello/master is ugly
> * submodules are real pain
no need to use sub-modules
> * git is not that good with binary data
>
gems are mostly ruby, very not a lot of binary data.
> = My proposal
>
> == Bundler patch
>
> * it would be loaded in Gemfile, so every tool using bundler will get it
I don't think a single Gemfile is going to solve the issue: we'll need 
to relax version constraints in Gemfile, and add a bunch of ruby-version 
specific gems...
>
> * it would ensure that if rpm-gem is present it's used even if there 
> is newer version available in gem-repo
> * ensure that locally installed gems are not confused with rpm-gems
You didn't explain how you are planning to differentiate between the two 
- system-wide bundler-/ruby-gem-installed gems and rpm-installed gems...
>
> * patch is optional, can be turned on/off by ENV variable or by options
> * (I am also considering to add optional check that all production 
> gems are installed as rpms)
>
> Deployment is unaffected, all dependencies are satisfied by rpm-gems.
>
> When you run bundle install to get development environment on any 
> rpm-based machine, all your rpms have priority and they are used. All 
> missing development gems are installed from our gem-repo.
I think this is the crux of the issue. Once we have a custom gem 
repository, the above step has very little practical value. A ruby gem 
is a ruby gem no matter what place in the system it's installed in.
>
> On non-rpm-based machine, 'bundle install' installs all gems 
> production and development from our gem-repo.
>
> == Gem Repository
>
> Bundler patch will work only if all development and production gems 
> are in the gem-repo. To get it fixed quickly 'bundle package' can be 
> used to get all the gems. Missing gems can be then uploaded to 
> katello-thirdparty (source for our gem-repo)
>
> = Summary
>
> I think that Bundler patch and updating gem-repo to contain all gems 
> is the only one which can be done fast and will work on any machine in 
> deployment and development. I tried to be thorough but if I missed 
> something or if you have better ideas please let me know.
The issue with dependency version variability needs solving. The patch 
itself has little practical value, bundle install from custom repository 
is sufficient. The irony is that you are trying to solve the problem 
Bundler has successfully solved awhile ago. I'm baffled as to why we 
continue to insist on solving the problems the "katello" way.

-d

>
> Petr
>
> [1] 
> https://github.com/aeolusproject/conductor/blob/master/src/config/application.rb#L21 
>
> [2] https://github.com/aeolus-incubator/bundler_ext
>
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel