From pchalupa at redhat.com Mon Dec 3 08:35:37 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Mon, 03 Dec 2012 09:35:37 +0100 Subject: [katello-devel] Katello configuration Message-ID: <50BC6459.8070403@redhat.com> There is a pull request [1] with Katello configuration cleanup. Please take a look at it, if it does not break anything for you. Rpm-build and katello-configure works fine. I'll merge it a few days after updating to latest master Petr [1] https://github.com/Katello/katello/pull/1049 From lzap at redhat.com Mon Dec 3 09:36:23 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 3 Dec 2012 10:36:23 +0100 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50B8D9AB.5070706@redhat.com> References: <1422079349.8432432.1354248399972.JavaMail.root@redhat.com> <50B8BCBB.9010002@redhat.com> <20121130143550.GD2274@lzapx.brq.redhat.com> <50B8CA77.6060002@redhat.com> <20121130154108.GG2274@lzapx.brq.redhat.com> <50B8D9AB.5070706@redhat.com> Message-ID: <20121203093623.GC2428@lzapx.brq.redhat.com> > Step #1 - let's get rid of controller validations. Again, you can only do this for CRUD actions in the controllers. For all other methods we need validations. And we have those, not all our controllers are plain REST-driven. We do have procedure-like actions. Maybe more then we should have, but this is a fact. You say all the validations should be done in models, thus you essentially say let's put all our code logic there. Because it does not make sense to do validations in models while doing anything else (that also requires some level of validations) in controllers or a logic layer. A fact: you need to do validations also outside of models. It is wrong to put all your code logic into models. And unfortunately we have most of it there. Rails follows MVC pattern and what we have is those messy Models-for-everything approach. If you mix this with our orchestration, it is deadly combination. Difficult testability, huge objects with no responsibility split, unable to mock things and so on. Code logic in models works well for small projects and it is de-facto standard in Rails world. People just blindly follow this wrong pattern. So I don't agree with this simplification you offer. We will likely start to refactor our orchestration creating new layer of code logic, which is the right approach to do things. And we should move complex methods like activation key subscription from models there. Then we will need validations on the controller side (or on the newly created logic layer or orchestration layer - depends on how are we gonna name it). So yes, it has something to do with REST. My counter proposal is: #2 Let's get rid of validations where they duplicate model validators, but let's keep them for (usually non-CRUD) methods which need then. We will likely be extending those soon. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 3 09:41:40 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 3 Dec 2012 10:41:40 +0100 Subject: [katello-devel] I had a dream: announcing RPM-based development setup In-Reply-To: <50B8EC65.8040405@redhat.com> References: <20121129111945.GC2291@lzapx.brq.redhat.com> <50B8EC65.8040405@redhat.com> Message-ID: <20121203094140.GD2428@lzapx.brq.redhat.com> I am fine with re-ordering the page. Whatever works for managers :-) LZ On Fri, Nov 30, 2012 at 12:27:01PM -0500, Bryan Kearney wrote: > On 11/29/2012 06:19 AM, Lukas Zapletal wrote: > >Hi all, > > > >since Mirek packaged all our development gems as RPMs, it was one simple > >step to finalize this and offer instructions about how to put production > >instance into development mode very easily. > > > >Today I want to push one another improvement - introduction of bundle.d > >directory (inspiration from Foreman). If you like it, instructions will > >be even more simple (and it will be first step towards modularizing > >Katello). > > > >Note I am not trying to replace your development setups, this can work > >well with git checkouts either on the installed machine or on your > >laptops (you can configure it to connect to the machine). I would say > >this setup is complementary, but I would like to present it as the > >recommended setup for the community, because it's very easy to setup and > >it just works. People can start hacking katello right away, after few > >issued commands. > > > >The development setup was put alive with the bundler_ext change - I will > >blog about it later this week. > > > >https://fedorahosted.org/katello/wiki/DevelopmentSetup#RPM-basedsetup > > > >""" > >This setup installs on top of production mode adding development gems. You can > >enable them and turn Katello into development mode and enjoy all the > >development mode features like autoloading or logging while maintaining the > >environment. Development setup can be used in conjunction with a git checkout > >either on the machine or on developers laptop. The usual practice is to develop > >on a laptop and then using scp/ftp/rsync to overwrite files in > >/usr/share/katello testing the features on the target environment. > > > >This development setup runs under bundler_ext just like production mode does. > >In this mode, Gemfile file is renamed to Gemfile.in and Katello loads all > >dependencies using bundler_ext library rather than bundler. It requires all > >necessary rubygems from the system ignoring versions specified in the Gemfilein > >and if a dependency is missing, it fails. There is no "bundle install" command > >avaiable in this mode and bundler_ext can be put in NOSTRICT mode where it > >ignores missing dependencies (logging error messages) which can be useful for > >development or installation phase (database migration or seed). > > > >This setup works on supported platforms ({{{Fedora/Red Hat and clones}}}) and > >it's the officially maintained by Katello developers - that means we will make > >sure it is always working and we will be able to help you with issues on lists > >and IRC channels. > > > >You need to [wiki:Install install Katello first], please check requirements > >sections and make sure installation is working properly. If you prefer fully > >automatic installation, just run the following on a clean machine: > > > >{{{ > >git clone git://github.com/lzap/ciauciau.git > >echo 'PATH=$PATH:ciauciau/bin' >> ~/.bashrc > >source ~/.bashrc > >ciau -p kt > >}}} > > > >This gives you nightly production installation. Now, you want to enable > >development gems and put Katello into development mode: > > > >{{{ > >yum -y install katello-devel-all > >}}} > > > >Hint: If you only want development gems, install {{{katello-devel}}}, if you > >only need testing gems install {{{katello-devel-test}}}. Possible packages are: > > > > * katello-devel-all (installs all groups bellow) > > * katello-devel (development rubygem group) > > * katello-devel-test (unit testing rubygem group) > > * katello-devel-profiling (profiling rubygem group) > > * katello-devel-jshintrb (javascript analysis rubygem group) > > > >Warning: There is a missing dependency in the katello-devel-jshintrb > >metapackage. It's recommended to install without this package: {{{yum -y > >install katello-devel katello-devel-test katello-devel-profiling}}} until we > >resolve the issue. You will only miss javascript analysis tool which is not > >needed for development. > > > >Uncomment, edit or add the following lines in {{{/etc/sysconfig/katello}}}: > > > >{{{ > >KATELLO_ENV=development > >KATELLO_LOGGING=debug > >BUNDLER_EXT_GROUPS="development debugging test" > >BUNDLER_EXT_NOSTRICT=1 > >}}} > > > >Explanation of the variables: > > > > * KATELLO_ENV - Sets RAILS_ENV variable, basically you turning on autoloading > >and other features of the development mode of Katello. > > * KATELLO_LOGGING - Turns on more verbose logging. Watch out because this is > >very verbose and can load your instance quickly. There are also other logging > >options in the file (SQL logging for instance). > > * BUNDLER_EXT_GROUPS - List of space separated bundler groups you want to > >load. Usually you only need "development test" there. Open > >/usr/share/katello/Gemfile.in to see all groups. > > * BUNDLER_EXT_NOSTRICT=1 - Puts bundler_ext in the NOSTRICT mode, that means > >if it miss a dependency, it will still continue starting up the server. This > >can be very useful when you miss a dependency (e.g. package does not exist yet) > >or you want to test some rubygem on your own. > > > >You may want to review other parameters in that file and also turn off firewall > >and SELinux (this is optional but recommended): > > > >{{{ > >setenforce 0 > >service iptables stop > >}}} > > > >You can restart katello service and enjoy fully development mode (e.g. > >autoloading). Many developers use their IDE via SCP/FTP or rsyncing files or > >simply editing via ssh. > > > >{{{ > >service katello restart > >}}} > > > >Warning: It is not recommended to start the katello server on the foreground as > >root, because this will likely break your installation (pid/temp files will be > >created as root). Use startup script and tail appropriate files at will. Also, > >if you overwrite many files, make sure you check the installation before > >upgrade if you want to return to the RPM version: {{{rpm -qa | grep katello | > >xargs rpm -V}}}. > > > >Advantages of the RPM based development setup are: > > > > * Easy automation - deployment can be as short as 10 minutes with snap-guest. > > * It's close to the production environment. > > * You can switch back to production. > > * Possibility to overwrite with official builds from RPMs. > > * Possibility to upgrade to other version (next day nightly). > > * You can test SELinux. > > * Easy to understand and maintain. > > * Many of the Katello team are using this - easier to find help. > > > I totally dig the ciauciau stuff. However... assume I am a stupid > manager type. I get to the first Q&A about checking out from git, > and you say "Do the needful". This page is about developer setup, > so we should be explicit. If i go look at the rest of hte page for > git I go to the section titled "Set Up Development (using Git) " and > it assuming a slightly different model. What would you think of: > > 1) Kill the sections titled: > > Not-Quite-So-Dirt-Simple-But-Still-Easy Approach > Build Katello from Source > > 2) GIve complete steps for the git checkout. this may be editing the > section (using Git) from above. > > My success criteria would be me doing an install, editing a page, > and seeing it work. > > - bk > > Fedora Hosted > Search: [ ] [Search] > ? Fedora Account Login > ? Help/Guide > ? About Trac > ? Fedora Account Sign Up > ? Preferences > > ? Wiki > ? Timeline > ? Roadmap > ? Browse Source > ? View Tickets > ? Search > > wiki:DevelopmentSetup > > Context Navigation > > ? Start Page > ? Index > ? History > > ??????????????????????????????????????????????????????????????????????????????? > Last modified 5 hours ago Last modified on 11/30/12 11:58:15 > > 1. Installation Instructions > 1. RPM-based setup > 1. Questions and answers > 2. Dirt Simple Approach > 3. Not-Quite-So-Dirt-Simple-But-Still-Easy Approach > 1. Yum Repo Files > 1. Fedora 16 > 2. Red Hat Enterprise Linux 6 > 2. Install the RPMs > 3. If setting up for development, stop the production Katello services > 4. Set Up Development (using Git) > 5. Build Katello from Source > 4. Troubleshooting > 1. Bundle Install error - xyz is missing > 2. Role "katello" does not exists > 3. Candlepin Issues > > Installation Instructions > > ? Dirt Simple Approach > ? Not-Quite-So-Dirt-Simple-But-Still-Easy Approach > ? Build Katello from Source > ? Troubleshooting > > These instructions were written with a minimum installation, so things might be > a little verbose. The following instruction have been codified in a set of > puppet modules. If you would like to use these to gets a new Katello machine > which "Just Works (tm)" you can follow the steps at Install. If not, follow the > steps below. > > RPM-based setup > > This setup installs on top of production mode adding development gems. You can > enable them and turn Katello into development mode and enjoy all the > development mode features like autoloading or logging while maintaining your OS > environment (works on all supported platforms). Development setup can be used > in conjunction with a git checkout either on the machine or on developer's > laptop. The usual practice is to develop on a laptop and then using scp/ftp/ > rsync to overwrite files in /usr/share/katello testing the features on the > target environment. More about this later. > > This development setup runs under bundler_ext extension just like production > mode does. In this mode, Gemfile file distributed as Gemfile.in and Katello > loads all dependencies using bundler_ext library rather than Bundler. It > requires all necessary rubygems from the system ignoring versions specified in > the Gemfile.in and if a dependency is missing, it fails. There is no "bundle > install" command necessary in this mode and bundler_ext can be put in no-strict > mode where it ignores missing dependencies (writing error messages on the > standard error output) which can be useful for development (you want to quickly > try something on a broken installation) or during installation phase (database > migration or seed will not fail if an unimportant dependency is missing). > > This setup works on supported platforms (Fedora/Red Hat and clones) and it's > the officially maintained by Katello developers - that means we will make sure > it is always working and we will be able to help you with issues on lists and > IRC channels. Also note there are various other guides to setup Katello for > development on this wiki page, you can use them as a complementary setup on a > host that was put to development mode using this technique. > > You need to install Katello first, please check requirements sections and make > sure installation is working properly. If you prefer fully automatic > installation of our nightly build, just run the following on a clean machine: > > git clone git://github.com/lzap/ciauciau.git > echo 'PATH=$PATH:ciauciau/bin' >> ~/.bashrc > source ~/.bashrc > ciau -p kt > > There are more options for you to install particular Katello version or > Headpin, see ?https://github.com/lzap/ciauciau. Now, you want to enable > development gems and put Katello into development mode: > > yum -y install katello-devel-all > > Hint: If you only want development gems, install katello-devel, if you only > need testing gems install katello-devel-test. and so on. Possible rubygem group > packages are: > > ? katello-devel-all (installs all groups bellow) > ? katello-devel (development rubygem group) > ? katello-devel-test (unit testing rubygem group) > ? katello-devel-profiling (profiling rubygem group) > ? katello-devel-jshintrb (javascript analysis rubygem group) > > Now you want to put Katello into development mode. Uncomment, edit or add the > following lines in /etc/sysconfig/katello: > > KATELLO_ENV=development > KATELLO_LOGGING=debug > BUNDLER_EXT_GROUPS="development debugging test" > BUNDLER_EXT_NOSTRICT=1 > > Explanation of the variables: > > ? KATELLO_ENV - Sets RAILS_ENV variable, basically it is turning on > autoloading and other features of the development mode of Rails. > ? KATELLO_LOGGING - Turns on more verbose logging. Watch out because logging > in debug level can be very verbose and fill your instance quickly with > hundreds of megabytes. There are also other logging options in the file > (SQL logging for instance). > ? BUNDLER_EXT_GROUPS - List of space separated bundler groups you want to > load. Usually you only need "development test" there. Open /usr/share/ > katello/Gemfile.in to see all groups. > ? BUNDLER_EXT_NOSTRICT=1 - Puts bundler_ext in the NOSTRICT mode, that means > if it miss a dependency, it will still continue starting up the server. > This can be very useful when you miss a dependency (e.g. package does not > exist yet) or you want to test some rubygem on your own. > > You may want to review other parameters in that file and also turn off firewall > and SELinux (this is optional but if you are going to create new files or > overwrite installation with your own version you will likely need at least > SELinux to be turned off): > > setenforce 0 > service iptables stop > > You can restart katello service and enjoy fully development mode (e.g. > autoloading). Many developers use their IDE via SCP/FTP or rsyncing files or > simply editing via ssh using Vim or Emacs. > > service katello restart > > Warning: It is not recommended to start the katello server on the foreground as > root using rails server, because this will likely break your installation (pid/ > temp files will overwritten with incorrect root permissions). Use startup > script and tail appropriate files at will. > > If you overwrite some installed files, make sure you check the installation > before upgrade if you want to return to the RPM version: rpm -qa | grep katello > | xargs rpm -V. Please note upgrades are not supported nightly to nightly, only > latest stable version to nightly. Therefore it is recommended to automate your > development setup installation so you can quickly re-create your environment > without manual intervention. > > Advantages of the RPM based development setup are: > > ? Easy automation - deployment can be as short as 10 minutes with snap-guest. > ? It's close to the production environment - you can "hack" features or fix > bugs easily on a given production setup with a reproducer. > ? You can switch back to production - maybe you want to test your patch under > higher load. > ? Possibility to overwrite with official builds from RPMs - if you overwrite > some files, with some care you can put everything back as it was installed > from RPM. > ? Possibility to upgrade to other version - you can upgrade your instance, > but note this is not supported (usually works). > ? You can test SELinux - maintaining our SELinux policy can be challenging > and it's great to have the option to debug and to hack installation while > SELinux is fully enabled. > ? Easy to understand and maintain - just few commands to remember. > ? Many of the Katello team are using this - easier to find help and support. > > Questions and answers > > Can I checkout our git? > > Sure, you can checkout it in a separate directory, then stop katello instance > and start it from your directory using bundler (downloading dependencies from > rubygems.org). All services are running and has been initialized as well as the > katello database and /etc/katello/katello.yml is also configured correctly, > there are no changes necessary. > > Can I overwrite files in /usr/share/katello? > > You can, actually development mode allows you to change files and Rails server > will automatically load the changes on the fly. You can also rename this > directory and checkout git there, although this is not recommended. You will > need to turn off SELinux or properly relabel the files after this change. If > you need to checkout git, stop katello instance, checkout in a different > directory and start your own instance using bundler. > > How to connect my XYZ IDE? > > If you need to use your IDE, you have several options. You can either mount the > /usr/share/katello somehow, or you can rsynch the changed files there. In both > cases you need to turn off selinux, or make sure files has correct contexts. > > But the preferred way is to run katello locally and configure it to connect to > the backend engines on the host as well as katello database. In this case, you > need to turn off iptables and also reconfigure postgres to listen on all > interfaces: ?https://github.com/lzap/ciauciau/blob/master/steps/ > 499-pg-accept-all.sh In this configuration, you don't even need to install > development dependencies or enable it in the sysconfig file, but it's good to > have it. > > Dirt Simple Approach > > In case you did not read above, see Install for the dirt simple approach. > > Not-Quite-So-Dirt-Simple-But-Still-Easy Approach > > This approach assumes you will be running Katello, Candlepin, and Pulp all > together on a single machine. > > First things first: > > setenforce 0 > Change to read as permissive - to keep over reboots within configuration file - /etc/selinux/config > SELINUX=permissive > yum install wget > > Red Hat Enterprise Linux 6 > > Katello requires the entablement of the [rhel-6-server-optional-rpms] repo. > - If you are using subscription-manager this is found in your /etc/yum.repos.d/redhat.repo file. > Set this repo to: enabled = 1 > - If you are using yum-rhn-plugin this can be enabled using > rhn-channel --add --channel=rhel-x86_64-server-optional-6 > > > Install EPEL for RHEL 6 > > $ rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm > > Yum Repo Files > > You'll need yum repo files for Candlepin, Pulp, and Katello. Those can be > obtained by installing the katello-repos-latest rpm. For example, > > Fedora 16 > > rpm -Uvh http://fedorapeople.org/groups/katello/releases/yum/1.0/Fedora/16/x86_64/katello-repos-latest.rpm > > Red Hat Enterprise Linux 6 > > rpm -Uvh http://fedorapeople.org/groups/katello/releases/yum/1.0/RHEL/6/x86_64/katello-repos-latest.rpm > > Install the RPMs > > Now with the correct repo files we will install everything in one go: > > yum install katello-all > > Once installation is complete, everything will need to be configured. To do so, > please run: > > sudo katello-configure > > katello-configure can take two or three minutes to complete. If everything > completes without error messages, then Katello, along with Candlepin and Pulp, > was configured correctly. You may access your new running instance of Katello > at https:///katello. > > These steps get you a running instance of Katello which is good for testing the > RPMs, etc. > > if you want to be able to develop Katello, please continue > > If setting up for development, stop the production Katello services > > Stop the Katello services > > service katello-jobs stop && service katello stop > > Disable them so they won't start up after a reboot > > systemctl disable katello-jobs.service && systemctl disable katello.service > > Set Up Development (using Git) > > First install Git on your system if it is not already installed > > yum install git > git clone --recursive > > On GitHub, fork the ?Katello project and submit changes via pull requests. > > Once you have a GitHub account, have forked katello, and have a Git repo on > your machine you can then disable the RPM based instances of Katello and then > run Katello from a Rails thin server from within your git checkout. > > There's a few more development libraries that will be needed: > > yum groupinstall "Development Tools" > yum install ruby-devel postgresql-devel sqlite-devel libxml2 libxml2-devel libxslt libxslt-devel > > Make sure your bundle is complete > > cd ~/path/to/katello/git/checkout/src/ > mv /etc/katello/katello.yml /etc/katello/katello.yml.rpm_based > ln -s ~/path/to/katello/git/checkout/src/config/katello.yml /etc/katello/katello.yml > > Install the required Rubygems via Bundler: > > # on Fedora: > yum install katello-devel-all > # everywhere else > bundle install > > Reset all the DBs to run with katello in your git checkout: > > sudo ./script/reset-oath > ./script/katello-reset-dbs development ~/path/to/katello/git/checkout/src/ > > Run Katello from the thin server in your git checkout's src directory > > cd ~/path/to/katello/git/checkout/src/ > export RAILS_RELATIVE_URL_ROOT=/katello > ./script/rails s thin > > If RAILS_RELATIVE_URL_ROOT is not exported, then the Rails server gets stuck > and will not answer URL requests. > > Your development Katello server is now accessible at http://:3000/ > katello > > Run your katello-jobs in dev mode as well: > > cd ~/path/to/katello/git/checkout/src/ > ./script/delayed_job start > > Build Katello from Source > > In order to build Katello from source, steps must be taken to account for the > inclusion of the ConvergeUI git submodule. For the steps below, the assumption > is that you have cloned the Katello source as described above (and that the git > submodule is checked out to the currently pointed-to hash). Also, see > developing with ?ConvergeUI. > > Starting from the base directory, build and install ConvergeUI from HEAD: > > cd src/vendor/converge-ui/ > tito build --test --rpm > rpm -Uvh /tmp/tito/noarch/converge-ui-devel-*.rpm > > Now, change directories back to katello/src and build Katello from HEAD: > > cd ../../ > tito build --test --rpm > > Troubleshooting > > Bundle Install error - xyz is missing > > If you see this message, check the git setup section above. You are missing > some dependnecies, most likely "*-devel." rpms. > > Role "katello" does not exists > > If Katello yells FATAL: role "katello" does not exist, then that means that the > postgres "katello" user has not been created. Run: > > sudo su - postgres -c 'createuser -dls katello' > > Candlepin Issues > > For Candlepin related errors, please see the ?Candlepin troubleshooting section > > Download in other formats: > > ? Plain Text > > ??????????????????????????????????????????????????????????????????????????????? > Trac Powered > > Powered by Trac 0.12.4 > By Edgewall Software. > > Gravatar support by Tracvatar 1.7 > > Visit the Trac open source project at > http://trac.edgewall.org/ > > SECURITY NOTE: file ~/.netrc must not be accessible by others > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From ohadlevy at redhat.com Mon Dec 3 12:59:21 2012 From: ohadlevy at redhat.com (Ohad Levy) Date: Mon, 3 Dec 2012 07:59:21 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50B8D9AB.5070706@redhat.com> Message-ID: <461326763.3208797.1354539561322.JavaMail.root@redhat.com> ----- Original Message ----- | On 30/11/12 03:41 PM, Lukas Zapletal wrote: | > On Fri, Nov 30, 2012 at 03:02:15PM +0000, Dmitri Dolguikh wrote: | >> Repeat after me: validation logic belongs in the model. | > Only in an ideal world, Dmitri. ;-) | > | > Not all our controllers follows REST pattern, and you know that. | > | This has nothing to do with rest - just common sense. If you start | doing | validations in controllers, you will inevitable start moving business | logic to them. | | In our case, instead of perpetuating the problem, let's fix it | instead. | Step #1 - let's get rid of controller validations. | The good thing about the validations is that you can pass them along, for example to a client, so it does not need to make the round trip to figure out he needs another param. I agree that I would prefer the model to raise the error, so should probably turn it off by default? I would also look at https://github.com/bcardarella/client_side_validations for inspirations of how to get the model validations. Ohad | | _______________________________________________ | katello-devel mailing list | katello-devel at redhat.com | https://www.redhat.com/mailman/listinfo/katello-devel | From inecas at redhat.com Mon Dec 3 13:44:03 2012 From: inecas at redhat.com (Ivan Necas) Date: Mon, 3 Dec 2012 08:44:03 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <461326763.3208797.1354539561322.JavaMail.root@redhat.com> Message-ID: <344563507.9582694.1354542243015.JavaMail.root@redhat.com> ----- Original Message ----- > > > ----- Original Message ----- > | On 30/11/12 03:41 PM, Lukas Zapletal wrote: > | > On Fri, Nov 30, 2012 at 03:02:15PM +0000, Dmitri Dolguikh wrote: > | >> Repeat after me: validation logic belongs in the model. > | > Only in an ideal world, Dmitri. ;-) > | > > | > Not all our controllers follows REST pattern, and you know that. > | > > | This has nothing to do with rest - just common sense. If you start > | doing > | validations in controllers, you will inevitable start moving > | business > | logic to them. > | > | In our case, instead of perpetuating the problem, let's fix it > | instead. > | Step #1 - let's get rid of controller validations. > | > > The good thing about the validations is that you can pass them along, > for example to a client, so it does not need to make the round trip > to figure out he needs another param. > > I agree that I would prefer the model to raise the error, so should > probably turn it off by default? +1 for turning it off by default, for this reason. However, I wouldn't recommend to getting rid of them completely. Being able to describe the params with the code has much more value in it than just with plain strings/comments. This was the original intention of the apipie gem. We can just run it with validations turned on when we want to find inconsistencies between documentation and actual code, perhaps automatically with some test suite for that. -- Ivan > > I would also look at > https://github.com/bcardarella/client_side_validations for > inspirations of how to get the model validations. > > Ohad > | > | _______________________________________________ > | katello-devel mailing list > | katello-devel at redhat.com > | https://www.redhat.com/mailman/listinfo/katello-devel > | > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From ehelms at redhat.com Mon Dec 3 14:29:45 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 03 Dec 2012 09:29:45 -0500 Subject: [katello-devel] Current Sprint Features for Katello 1.2 Message-ID: <50BCB759.2090103@redhat.com> All, If you are currently working on a feature or bug fix that you plan to have submitted and merged by the end of the current sprint (December 5th, i.e. to be included in the Katello 1.2 release) please let me know ASAP. Thanks, Eric From lzap at redhat.com Mon Dec 3 14:34:25 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 3 Dec 2012 15:34:25 +0100 Subject: [katello-devel] Current Sprint Features for Katello 1.2 In-Reply-To: <50BCB759.2090103@redhat.com> References: <50BCB759.2090103@redhat.com> Message-ID: <20121203143425.GJ2428@lzapx.brq.redhat.com> I would like to get bundler.d split in. Looking good, all issues fixed, waiting for more ACKs and then I will rename a RPM and it's good to merge. LZ On Mon, Dec 03, 2012 at 09:29:45AM -0500, Eric Helms wrote: > All, > > If you are currently working on a feature or bug fix that you plan > to have submitted and merged by the end of the current sprint > (December 5th, i.e. to be included in the Katello 1.2 release) > please let me know ASAP. > > Thanks, > Eric > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From ehelms at redhat.com Mon Dec 3 14:54:46 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 03 Dec 2012 09:54:46 -0500 Subject: [katello-devel] Apipie and validation. In-Reply-To: <344563507.9582694.1354542243015.JavaMail.root@redhat.com> References: <344563507.9582694.1354542243015.JavaMail.root@redhat.com> Message-ID: <50BCBD36.3050308@redhat.com> We are in the situation of having two different entry points into our model - API and UI controllers. This fact alone makes me feel that we need as much as possible validations at the model layer, otherwise we are just repeating ourselves and opening holes for miss-matches in expectations around our data layer. Further, our data layer should be the end-all, know-all for the /data/. Thus, I would say: - If the validation deals with a direct property of the model (either an attribute or an association), the validation should live in the model - If the validation deals with a value expected specifically by the controller that is somehow involved in massaging of the attributes sent down to the model it should live in the controller I agree our models feel quite large, there is lot of logic and functionality sitting in them currently. That shouldn't be the reason we shy away from putting data validation in that layer, that should be the motivation to organize our models smarter. In the PulpV2 branch we have made some strides to help with this by breaking out all authorization (app/models/authorization) and elasticsearch (app/models/glue/elasticsearch/) functionality into their own files [1]. A good example file to look at is repository.rb [2] https://github.com/Katello/katello/tree/pulpv2/src/app/models https://github.com/Katello/katello/blob/pulpv2/src/app/models/repository.rb - Eric On 12/03/2012 08:44 AM, Ivan Necas wrote: > > ----- Original Message ----- >> >> ----- Original Message ----- >> | On 30/11/12 03:41 PM, Lukas Zapletal wrote: >> | > On Fri, Nov 30, 2012 at 03:02:15PM +0000, Dmitri Dolguikh wrote: >> | >> Repeat after me: validation logic belongs in the model. >> | > Only in an ideal world, Dmitri. ;-) >> | > >> | > Not all our controllers follows REST pattern, and you know that. >> | > >> | This has nothing to do with rest - just common sense. If you start >> | doing >> | validations in controllers, you will inevitable start moving >> | business >> | logic to them. >> | >> | In our case, instead of perpetuating the problem, let's fix it >> | instead. >> | Step #1 - let's get rid of controller validations. >> | >> >> The good thing about the validations is that you can pass them along, >> for example to a client, so it does not need to make the round trip >> to figure out he needs another param. >> >> I agree that I would prefer the model to raise the error, so should >> probably turn it off by default? > +1 for turning it off by default, for this reason. > > However, I wouldn't recommend to getting rid of them completely. Being > able to describe the params with the code has much more value in it than > just with plain strings/comments. This was the original intention of the > apipie gem. We can just run it with validations turned on when we want to > find inconsistencies between documentation and actual code, perhaps automatically > with some test suite for that. > > -- Ivan > >> I would also look at >> https://github.com/bcardarella/client_side_validations for >> inspirations of how to get the model validations. >> >> Ohad >> | >> | _______________________________________________ >> | katello-devel mailing list >> | katello-devel at redhat.com >> | https://www.redhat.com/mailman/listinfo/katello-devel >> | >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrist at redhat.com Mon Dec 3 16:15:06 2012 From: jrist at redhat.com (Jason Rist) Date: Mon, 03 Dec 2012 09:15:06 -0700 Subject: [katello-devel] Current Sprint Features for Katello 1.2 In-Reply-To: <50BCB759.2090103@redhat.com> References: <50BCB759.2090103@redhat.com> Message-ID: <50BCD00A.60308@redhat.com> On 12/03/2012 07:29 AM, Eric Helms wrote: > All, > > If you are currently working on a feature or bug fix that you plan to > have submitted and merged by the end of the current sprint (December > 5th, i.e. to be included in the Katello 1.2 release) please let me know > ASAP. > > Thanks, > Eric > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel I am. Wider tupane. And various UI fixes, including notifications, alchemy coloring, etc. -J -- Jason E. Rist Senior Software Engineer Systems Management and Cloud Enablement Red Hat, Inc. +1.919.754.4048 Freenode: jrist github/identi.ca: knowncitizen From ehelms at redhat.com Mon Dec 3 20:52:43 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 03 Dec 2012 15:52:43 -0500 Subject: [katello-devel] Bash completion file In-Reply-To: <50A68F99.3000405@redhat.com> References: <20121116113003.GN2324@lzapx.brq.redhat.com> <50A656D6.3070303@redhat.com> <20121116161422.GA2324@lzapx.brq.redhat.com> <50A68F99.3000405@redhat.com> Message-ID: <50BD111B.802@redhat.com> While preparing for the Katello 1.2 release, and specifically looking over updating the bash completion script, I ran across this little guy http://furius.ca/optcomplete/ for doing auto-completions when using the optparser in Python. I played around with trying to implement it, and while this worked for the options located here - https://github.com/Katello/katello/blob/master/cli/src/katello/client/cli/base.py#L105 it did not appear to work for all other commands. I attempted to figure out where to place the activation line, or better yet how and if we were adding these commands in a strategic location that would allow usage of this module but got a bit lost and never figured it out. Sending this out to the list in case anyone with more knowledge wanted to take a gander. -Eric On 11/16/2012 02:10 PM, Miroslav Suchy wrote: > On 16.11.2012 17:14, Lukas Zapletal wrote: >> I hope Mirek will come up with something, the last time I checked this, >> all the options to generate this from our opt_parser were quite >> difficult to implement. > > Quite a lot time, lower priority... I would like to not give false hopes. > >> Maybe there is a way... > > Bachelor thesis? > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrist at redhat.com Mon Dec 3 21:09:08 2012 From: jrist at redhat.com (Jason Rist) Date: Mon, 03 Dec 2012 14:09:08 -0700 Subject: [katello-devel] Bash completion file In-Reply-To: <50BD111B.802@redhat.com> References: <20121116113003.GN2324@lzapx.brq.redhat.com> <50A656D6.3070303@redhat.com> <20121116161422.GA2324@lzapx.brq.redhat.com> <50A68F99.3000405@redhat.com> <50BD111B.802@redhat.com> Message-ID: <50BD14F4.7000809@redhat.com> On 12/03/2012 01:52 PM, Eric Helms wrote: > While preparing for the Katello 1.2 release, and specifically looking > over updating the bash completion script, I ran across this little guy > http://furius.ca/optcomplete/ for doing auto-completions when using the > optparser in Python. > > I played around with trying to implement it, and while this worked for > the options located here - > https://github.com/Katello/katello/blob/master/cli/src/katello/client/cli/base.py#L105 > it did not appear to work for all other commands. I attempted to figure > out where to place the activation line, or better yet how and if we were > adding these commands in a strategic location that would allow usage of > this module but got a bit lost and never figured it out. Sending this > out to the list in case anyone with more knowledge wanted to take a gander. > > -Eric > > On 11/16/2012 02:10 PM, Miroslav Suchy wrote: >> On 16.11.2012 17:14, Lukas Zapletal wrote: >>> I hope Mirek will come up with something, the last time I checked this, >>> all the options to generate this from our opt_parser were quite >>> difficult to implement. >> >> Quite a lot time, lower priority... I would like to not give false hopes. >> >>> Maybe there is a way... >> >> Bachelor thesis? >> >> Mirek >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > I thought we already had completion. Tomas might know more... -J -- Jason E. Rist Senior Software Engineer Systems Management and Cloud Enablement Red Hat, Inc. +1.919.754.4048 Freenode: jrist github/identi.ca: knowncitizen From msuchy at redhat.com Tue Dec 4 07:34:57 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Tue, 04 Dec 2012 08:34:57 +0100 Subject: [katello-devel] Bash completion file In-Reply-To: <50BD14F4.7000809@redhat.com> References: <20121116113003.GN2324@lzapx.brq.redhat.com> <50A656D6.3070303@redhat.com> <20121116161422.GA2324@lzapx.brq.redhat.com> <50A68F99.3000405@redhat.com> <50BD111B.802@redhat.com> <50BD14F4.7000809@redhat.com> Message-ID: <50BDA7A1.9080505@redhat.com> On 12/03/2012 10:09 PM, Jason Rist wrote: > I thought we already had completion. But only to common arguments. Completation does not work for every option and is not regulary updated. This can allow us to set up regular synchronization. Nice work Eric. I will check it up. -- Miroslav Suchy Red Hat Systems Management Engineering From pchalupa at redhat.com Tue Dec 4 08:38:22 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Tue, 04 Dec 2012 09:38:22 +0100 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BCBD36.3050308@redhat.com> References: <344563507.9582694.1354542243015.JavaMail.root@redhat.com> <50BCBD36.3050308@redhat.com> Message-ID: <50BDB67E.7030004@redhat.com> I see the model validation and API params validation in controller as two different things. -1 for removing the validation from apipie and controllers. I agree with the lzap. However I felt the duplication in simple crud controller actions when writing api documentation. +1 to fix [1] (the issue inecas mentioned) to get rid of the duplication. -1 for disabling the apipie validations by default. When adding a feature a developer should disable the apipie validation (if needed) to see ActiveRecord errors and to simplify developing. Then apipie validation should be re-enabled and tested if it is up to date. I think task keeping-API-doc-up-to-date should not be done separately from feature development. Petr [1] - https://github.com/Pajk/apipie-rails/issues/67 On 03.12.12 15:54, Eric Helms wrote: > We are in the situation of having two different entry points into our > model - API and UI controllers. This fact alone makes me feel that we > need as much as possible validations at the model layer, otherwise we > are just repeating ourselves and opening holes for miss-matches in > expectations around our data layer. Further, our data layer should be > the end-all, know-all for the /data/. Thus, I would say: > > - If the validation deals with a direct property of the model (either an > attribute or an association), the validation should live in the model > - If the validation deals with a value expected specifically by the > controller that is somehow involved in massaging of the attributes sent > down to the model it should live in the controller > > I agree our models feel quite large, there is lot of logic and > functionality sitting in them currently. That shouldn't be the reason > we shy away from putting data validation in that layer, that should be > the motivation to organize our models smarter. In the PulpV2 branch we > have made some strides to help with this by breaking out all > authorization (app/models/authorization) and elasticsearch > (app/models/glue/elasticsearch/) functionality into their own files > [1]. A good example file to look at is repository.rb [2] > > https://github.com/Katello/katello/tree/pulpv2/src/app/models > > https://github.com/Katello/katello/blob/pulpv2/src/app/models/repository.rb > > - Eric > > On 12/03/2012 08:44 AM, Ivan Necas wrote: >> >> ----- Original Message ----- >>> >>> ----- Original Message ----- >>> | On 30/11/12 03:41 PM, Lukas Zapletal wrote: >>> | > On Fri, Nov 30, 2012 at 03:02:15PM +0000, Dmitri Dolguikh wrote: >>> | >> Repeat after me: validation logic belongs in the model. >>> | > Only in an ideal world, Dmitri. ;-) >>> | > >>> | > Not all our controllers follows REST pattern, and you know that. >>> | > >>> | This has nothing to do with rest - just common sense. If you start >>> | doing >>> | validations in controllers, you will inevitable start moving >>> | business >>> | logic to them. >>> | >>> | In our case, instead of perpetuating the problem, let's fix it >>> | instead. >>> | Step #1 - let's get rid of controller validations. >>> | >>> >>> The good thing about the validations is that you can pass them along, >>> for example to a client, so it does not need to make the round trip >>> to figure out he needs another param. >>> >>> I agree that I would prefer the model to raise the error, so should >>> probably turn it off by default? >> +1 for turning it off by default, for this reason. >> >> However, I wouldn't recommend to getting rid of them completely. Being >> able to describe the params with the code has much more value in it than >> just with plain strings/comments. This was the original intention of the >> apipie gem. We can just run it with validations turned on when we want to >> find inconsistencies between documentation and actual code, perhaps >> automatically >> with some test suite for that. >> >> -- Ivan >> >>> I would also look at >>> https://github.com/bcardarella/client_side_validations for >>> inspirations of how to get the model validations. >>> >>> Ohad >>> | >>> | _______________________________________________ >>> | katello-devel mailing list >>> | katello-devel at redhat.com >>> | https://www.redhat.com/mailman/listinfo/katello-devel >>> | >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From tstrachota at redhat.com Tue Dec 4 08:58:16 2012 From: tstrachota at redhat.com (Tomas Strachota) Date: Tue, 04 Dec 2012 09:58:16 +0100 Subject: [katello-devel] Bash completion file In-Reply-To: <50BDA7A1.9080505@redhat.com> References: <20121116113003.GN2324@lzapx.brq.redhat.com> <50A656D6.3070303@redhat.com> <20121116161422.GA2324@lzapx.brq.redhat.com> <50A68F99.3000405@redhat.com> <50BD111B.802@redhat.com> <50BD14F4.7000809@redhat.com> <50BDA7A1.9080505@redhat.com> Message-ID: <50BDBB28.9080005@redhat.com> On 12/04/2012 08:34 AM, Miroslav Such? wrote: > On 12/03/2012 10:09 PM, Jason Rist wrote: >> I thought we already had completion. > > But only to common arguments. Completation does not work for every > option and is not regulary updated. This can allow us to set up regular > synchronization. > Nice work Eric. I will check it up. > As Mirek said. We have very simple pure bash completion and full completion in "katello shell" mode (python driven) [1]. Optcomplete seems to be nice and simple library. Every command/action in our cli has its own optparser instance. I guess that is the reason why it didn't work with the simplest settings the documentation describes. I briefly checked optcomplete and it seems to have support for such cases and it's possible to define subcommands and solve this issue [2]. On the other hand we already have the completion functionality in shell mode so maybe we could extract it from there, create a separate _katellocomplete script and use it as a bash completor for katello. T. [1] https://github.com/Katello/katello/blob/master/cli/src/katello/client/shell.py#L225 [2] https://hg.furius.ca/public/optcomplete/file/83bf43e3db27/lib/python/optcomplete.py#l191 From dmitri at redhat.com Tue Dec 4 12:10:28 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Tue, 04 Dec 2012 12:10:28 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <20121203093623.GC2428@lzapx.brq.redhat.com> References: <1422079349.8432432.1354248399972.JavaMail.root@redhat.com> <50B8BCBB.9010002@redhat.com> <20121130143550.GD2274@lzapx.brq.redhat.com> <50B8CA77.6060002@redhat.com> <20121130154108.GG2274@lzapx.brq.redhat.com> <50B8D9AB.5070706@redhat.com> <20121203093623.GC2428@lzapx.brq.redhat.com> Message-ID: <50BDE834.9060503@redhat.com> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >> Step #1 - let's get rid of controller validations. > Again, you can only do this for CRUD actions in the controllers. For all > other methods we need validations. And we have those, not all our > controllers are plain REST-driven. We do have procedure-like actions. > Maybe more then we should have, but this is a fact. > > You say all the validations should be done in models, thus you > essentially say let's put all our code logic there. Because it does not > make sense to do validations in models while doing anything else (that > also requires some level of validations) in controllers or a logic > layer. A fact: you need to do validations also outside of models. Domain logic belongs in the model layer. We can discuss how to better factor model layer code, but not where to put it. -d A lot of rationalizations to keep validations in apipie have been put forward, *all* of them attempting to deal with various smells, code or process ones. Let's fix the root issues. Again, Let's get rid of validations in apipie. Refactor the controllers when needed. If apipie is not useful without those validations, let's suspend/stop use of it. -d > > It is wrong to put all your code logic into models. And unfortunately we > have most of it there. Rails follows MVC pattern and what we have is those > messy Models-for-everything approach. If you mix this with our > orchestration, it is deadly combination. Difficult testability, huge > objects with no responsibility split, unable to mock things and so on. > Code logic in models works well for small projects and it is de-facto > standard in Rails world. People just blindly follow this wrong pattern. > > So I don't agree with this simplification you offer. We will likely > start to refactor our orchestration creating new layer of code logic, > which is the right approach to do things. And we should move complex > methods like activation key subscription from models there. Then we will > need validations on the controller side (or on the newly created logic > layer or orchestration layer - depends on how are we gonna name it). > > So yes, it has something to do with REST. My counter proposal is: > > #2 Let's get rid of validations where they duplicate model validators, > but let's keep them for (usually non-CRUD) methods which need then. We > will likely be extending those soon. > From msuchy at redhat.com Tue Dec 4 12:51:24 2012 From: msuchy at redhat.com (=?ISO-8859-2?Q?Miroslav_Such=FD?=) Date: Tue, 04 Dec 2012 13:51:24 +0100 Subject: [katello-devel] Issues on github Message-ID: <50BDF1CC.8010103@redhat.com> Who enabled issues on github? https://github.com/Katello/katello/issues Can I switch it off? -- Miroslav Suchy Red Hat Systems Management Engineering From ericdhelms at gmail.com Tue Dec 4 14:36:54 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Tue, 4 Dec 2012 09:36:54 -0500 Subject: [katello-devel] Cleaning up Branches with Github Message-ID: For those that don't follow the Github blog, be aware of the following both to help you clean-up your forks and not accidentally remove them. https://github.com/blog/1335-tidying-up-after-pull-requests -Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkearney at redhat.com Tue Dec 4 17:08:37 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Tue, 04 Dec 2012 12:08:37 -0500 Subject: [katello-devel] Current Sprint Features for Katello 1.2 In-Reply-To: <50BCB759.2090103@redhat.com> References: <50BCB759.2090103@redhat.com> Message-ID: <50BE2E15.5050909@redhat.com> On 12/03/2012 09:29 AM, Eric Helms wrote: > All, > > If you are currently working on a feature or bug fix that you plan to > have submitted and merged by the end of the current sprint (December > 5th, i.e. to be included in the Katello 1.2 release) please let me know > ASAP. > > Thanks, > Eric > Perhaps this, depending on how bad the pull request gets savaged. https://github.com/Katello/katello/pull/1180 -- bk From jrist at redhat.com Tue Dec 4 20:41:37 2012 From: jrist at redhat.com (Jason Rist) Date: Tue, 04 Dec 2012 13:41:37 -0700 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BDE834.9060503@redhat.com> References: <1422079349.8432432.1354248399972.JavaMail.root@redhat.com> <50B8BCBB.9010002@redhat.com> <20121130143550.GD2274@lzapx.brq.redhat.com> <50B8CA77.6060002@redhat.com> <20121130154108.GG2274@lzapx.brq.redhat.com> <50B8D9AB.5070706@redhat.com> <20121203093623.GC2428@lzapx.brq.redhat.com> <50BDE834.9060503@redhat.com> Message-ID: <50BE6001.5070105@redhat.com> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>> Step #1 - let's get rid of controller validations. >> Again, you can only do this for CRUD actions in the controllers. For all >> other methods we need validations. And we have those, not all our >> controllers are plain REST-driven. We do have procedure-like actions. >> Maybe more then we should have, but this is a fact. >> >> You say all the validations should be done in models, thus you >> essentially say let's put all our code logic there. Because it does not >> make sense to do validations in models while doing anything else (that >> also requires some level of validations) in controllers or a logic >> layer. A fact: you need to do validations also outside of models. > > Domain logic belongs in the model layer. We can discuss how to better > factor model layer code, but not where to put it. > > -d > > A lot of rationalizations to keep validations in apipie have been put > forward, *all* of them attempting to deal with various smells, code or > process ones. Let's fix the root issues. > > Again, Let's get rid of validations in apipie. Refactor the > controllers when needed. If apipie is not useful without those > validations, let's suspend/stop use of it. > > -d > +1 > >> >> It is wrong to put all your code logic into models. And unfortunately we >> have most of it there. Rails follows MVC pattern and what we have is >> those >> messy Models-for-everything approach. If you mix this with our >> orchestration, it is deadly combination. Difficult testability, huge >> objects with no responsibility split, unable to mock things and so on. >> Code logic in models works well for small projects and it is de-facto >> standard in Rails world. People just blindly follow this wrong pattern. >> >> So I don't agree with this simplification you offer. We will likely >> start to refactor our orchestration creating new layer of code logic, >> which is the right approach to do things. And we should move complex >> methods like activation key subscription from models there. Then we will >> need validations on the controller side (or on the newly created logic >> layer or orchestration layer - depends on how are we gonna name it). >> >> So yes, it has something to do with REST. My counter proposal is: >> >> #2 Let's get rid of validations where they duplicate model validators, >> but let's keep them for (usually non-CRUD) methods which need then. We >> will likely be extending those soon. >> > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Jason E. Rist Senior Software Engineer Systems Management and Cloud Enablement Red Hat, Inc. +1.919.754.4048 Freenode: jrist github/identi.ca: knowncitizen From inecas at redhat.com Tue Dec 4 21:47:41 2012 From: inecas at redhat.com (Ivan Necas) Date: Tue, 4 Dec 2012 16:47:41 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BE6001.5070105@redhat.com> Message-ID: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> ----- Original Message ----- > On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > > On 03/12/12 09:36 AM, Lukas Zapletal wrote: > >>> Step #1 - let's get rid of controller validations. > >> Again, you can only do this for CRUD actions in the controllers. > >> For all > >> other methods we need validations. And we have those, not all our > >> controllers are plain REST-driven. We do have procedure-like > >> actions. > >> Maybe more then we should have, but this is a fact. > >> > >> You say all the validations should be done in models, thus you > >> essentially say let's put all our code logic there. Because it > >> does not > >> make sense to do validations in models while doing anything else > >> (that > >> also requires some level of validations) in controllers or a logic > >> layer. A fact: you need to do validations also outside of models. > > > > Domain logic belongs in the model layer. We can discuss how to > > better > > factor model layer code, but not where to put it. > > > > -d > > > > A lot of rationalizations to keep validations in apipie have been > > put > > forward, *all* of them attempting to deal with various smells, code > > or > > process ones. Let's fix the root issues. > > > > Again, Let's get rid of validations in apipie. Refactor the > > controllers when needed. If apipie is not useful without those > > validations, let's suspend/stop use of it. Am I reading it correctly you suggesting getting rid of API documentation? -- Ivan > > > > -d > > > +1 > > > >> > >> It is wrong to put all your code logic into models. And > >> unfortunately we > >> have most of it there. Rails follows MVC pattern and what we have > >> is > >> those > >> messy Models-for-everything approach. If you mix this with our > >> orchestration, it is deadly combination. Difficult testability, > >> huge > >> objects with no responsibility split, unable to mock things and so > >> on. > >> Code logic in models works well for small projects and it is > >> de-facto > >> standard in Rails world. People just blindly follow this wrong > >> pattern. > >> > >> So I don't agree with this simplification you offer. We will > >> likely > >> start to refactor our orchestration creating new layer of code > >> logic, > >> which is the right approach to do things. And we should move > >> complex > >> methods like activation key subscription from models there. Then > >> we will > >> need validations on the controller side (or on the newly created > >> logic > >> layer or orchestration layer - depends on how are we gonna name > >> it). > >> > >> So yes, it has something to do with REST. My counter proposal is: > >> > >> #2 Let's get rid of validations where they duplicate model > >> validators, > >> but let's keep them for (usually non-CRUD) methods which need > >> then. We > >> will likely be extending those soon. > >> > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > -- > Jason E. Rist > Senior Software Engineer > Systems Management and Cloud Enablement > Red Hat, Inc. > +1.919.754.4048 > Freenode: jrist > github/identi.ca: knowncitizen > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Wed Dec 5 08:06:10 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 5 Dec 2012 09:06:10 +0100 Subject: [katello-devel] Cleaning up Branches with Github In-Reply-To: References: Message-ID: <20121205080610.GC2315@lzapx.brq.redhat.com> Just a follow up - then I'd do this on my local checkout: git branch --merged master | grep -v 'master$' | xargs git branch -d (Deletes all branches that has been merged to the master already) LZ On Tue, Dec 04, 2012 at 09:36:54AM -0500, Eric D Helms wrote: > For those that don't follow the Github blog, be aware of the following > both to help you clean-up your forks and not accidentally remove them. > > https://github.com/blog/1335-tidying-up-after-pull-requests > > -Eric > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From pchalupa at redhat.com Wed Dec 5 09:07:44 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Wed, 05 Dec 2012 10:07:44 +0100 Subject: [katello-devel] Apipie and validation. In-Reply-To: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> References: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> Message-ID: <50BF0EE0.10600@redhat.com> I agree that domain validation/consistency checks belongs to the models. Nevertheless definition of the API is other thing entirely. It should be defined and validated on its own with re-usage of information provided by models (domain) to avoid duplications. It should not be delegated to model to deal with it by adding other responsibility. So I strongly suggest to fix the duplication issue [1] and keep things separate. * Apipie for API validations. * Model validations for domain logic. ## Fix Apipie Enable Apipie to read model definitions and validations to set param validations automatically for parts of request representing a domain model, see [1]. + api documentation generated from api specification + api specification is kept up to date (has to be because validation is turned on) + api validation/specification is separated from domain validations/logic ## Remove validations from Apipie + api documentation generated from api specification - api specification is not kept up to date (has to be ensured in an extra task) - no api validation, it's partially handled by models, errors can be ugly, models have another responsibility [1] - https://github.com/Pajk/apipie-rails/issues/67 ## Other notes > The second issue is that apipie validations are stricter than AR ones: array validation assumes that there's an array in place, while AR would accept a nil. These two combined lead to even more false positives during apipie validation. I think be more consistent and strict is a good thing, it prevents confusion. Empty Array is an empty Array not a nil. Petr On 04.12.12 22:47, Ivan Necas wrote: > > > ----- Original Message ----- >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>>> Step #1 - let's get rid of controller validations. >>>> Again, you can only do this for CRUD actions in the controllers. >>>> For all >>>> other methods we need validations. And we have those, not all our >>>> controllers are plain REST-driven. We do have procedure-like >>>> actions. >>>> Maybe more then we should have, but this is a fact. >>>> >>>> You say all the validations should be done in models, thus you >>>> essentially say let's put all our code logic there. Because it >>>> does not >>>> make sense to do validations in models while doing anything else >>>> (that >>>> also requires some level of validations) in controllers or a logic >>>> layer. A fact: you need to do validations also outside of models. >>> >>> Domain logic belongs in the model layer. We can discuss how to >>> better >>> factor model layer code, but not where to put it. >>> >>> -d >>> >>> A lot of rationalizations to keep validations in apipie have been >>> put >>> forward, *all* of them attempting to deal with various smells, code >>> or >>> process ones. Let's fix the root issues. >>> >>> Again, Let's get rid of validations in apipie. Refactor the >>> controllers when needed. If apipie is not useful without those >>> validations, let's suspend/stop use of it. > > Am I reading it correctly you suggesting getting rid of API documentation? > > -- Ivan > >>> >>> -d >>> >> +1 >>> >>>> >>>> It is wrong to put all your code logic into models. And >>>> unfortunately we >>>> have most of it there. Rails follows MVC pattern and what we have >>>> is >>>> those >>>> messy Models-for-everything approach. If you mix this with our >>>> orchestration, it is deadly combination. Difficult testability, >>>> huge >>>> objects with no responsibility split, unable to mock things and so >>>> on. >>>> Code logic in models works well for small projects and it is >>>> de-facto >>>> standard in Rails world. People just blindly follow this wrong >>>> pattern. >>>> >>>> So I don't agree with this simplification you offer. We will >>>> likely >>>> start to refactor our orchestration creating new layer of code >>>> logic, >>>> which is the right approach to do things. And we should move >>>> complex >>>> methods like activation key subscription from models there. Then >>>> we will >>>> need validations on the controller side (or on the newly created >>>> logic >>>> layer or orchestration layer - depends on how are we gonna name >>>> it). >>>> >>>> So yes, it has something to do with REST. My counter proposal is: >>>> >>>> #2 Let's get rid of validations where they duplicate model >>>> validators, >>>> but let's keep them for (usually non-CRUD) methods which need >>>> then. We >>>> will likely be extending those soon. >>>> >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> >> >> -- >> Jason E. Rist >> Senior Software Engineer >> Systems Management and Cloud Enablement >> Red Hat, Inc. >> +1.919.754.4048 >> Freenode: jrist >> github/identi.ca: knowncitizen >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From mgrepl at redhat.com Wed Dec 5 09:14:51 2012 From: mgrepl at redhat.com (Miroslav Grepl) Date: Wed, 05 Dec 2012 10:14:51 +0100 Subject: [katello-devel] Katello, Foreman and SELinux In-Reply-To: <50B75DEC.90902@redhat.com> References: <20121128124624.GF2323@lzapx.brq.redhat.com> <50B60E72.7000704@redhat.com> <50B60F59.4010408@redhat.com> <50B63E6B.9080905@redhat.com> <50B75DEC.90902@redhat.com> Message-ID: <50BF108B.9010108@redhat.com> On 11/29/2012 02:06 PM, Miroslav Grepl wrote: > On 11/28/2012 05:40 PM, Bryan Kearney wrote: >> On 11/28/2012 08:19 AM, Miroslav Grepl wrote: >>> On 11/28/2012 02:15 PM, Miroslav Suchy wrote: >>>> On 28.11.2012 13:46, Lukas Zapletal wrote: >>>>> If we would like to confine thin processes (or perhaps mod_passenger >>>>> processes) for both Katello and Foreman, we would need to extend our >>>>> policy with this and also use the thin_t context as a template: >>>> >>>> Personally I would not spend time on confining thin. >>>> >>>> What about finishing: >>>> https://github.com/Katello/katello/pull/1041 >>>> >>>> We can import selinux-policy package from RHEL6.4 Alpha into rhel6 >>>> thirdparty temporally . And hopefuly finish it. >>>> >>>> This would save us a lot of work. >>>> >>>> Hmm, and even without it... can we use selinux transitions? >>>> >>>> http://danwalsh.livejournal.com/23944.html >>>> >>>> Mirek >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>> The problem is we have different thin policies for RHEL6 against >>> Fedora17+. It just works in RHEL6 because we have >>> >>> optional_policy(` >>> unconfined_domain(thin_t) >>> ') >> Can we add this for F17? > Well, the F17+ thin policy is different. We don't want to make thin_t > as unconfined. It could mask real issues like this and we want to know > if there is a project which uses thin and needs a new thin policy. > > We did the same for aeolus-configserver and it works fine now. So I > believe we can do it also for Foreman together ;-). >> >> -- bk >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel I wrote the following blog which could help you. http://mgrepl.wordpress.com/2012/11/30/how-would-tools-like-paster-work-with-selinux-thin-story/ From inecas at redhat.com Wed Dec 5 09:51:44 2012 From: inecas at redhat.com (Ivan Necas) Date: Wed, 5 Dec 2012 04:51:44 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF0EE0.10600@redhat.com> Message-ID: <2081436589.10734047.1354701104255.JavaMail.root@redhat.com> > [1] - https://github.com/Pajk/apipie-rails/issues/67 > > ## Other notes > > > The second issue is that apipie validations are stricter than AR > ones: array validation assumes that there's an array in place, while > AR > would accept a nil. These two combined lead to even more false > positives > during apipie validation. There is :allow_nil option, that lets you do this. > On 04.12.12 22:47, Ivan Necas wrote: > > > > > > ----- Original Message ----- > >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: > >>>>> Step #1 - let's get rid of controller validations. > >>>> Again, you can only do this for CRUD actions in the controllers. > >>>> For all > >>>> other methods we need validations. And we have those, not all > >>>> our > >>>> controllers are plain REST-driven. We do have procedure-like > >>>> actions. > >>>> Maybe more then we should have, but this is a fact. > >>>> > >>>> You say all the validations should be done in models, thus you > >>>> essentially say let's put all our code logic there. Because it > >>>> does not > >>>> make sense to do validations in models while doing anything else > >>>> (that > >>>> also requires some level of validations) in controllers or a > >>>> logic > >>>> layer. A fact: you need to do validations also outside of > >>>> models. > >>> > >>> Domain logic belongs in the model layer. We can discuss how to > >>> better > >>> factor model layer code, but not where to put it. > >>> > >>> -d > >>> > >>> A lot of rationalizations to keep validations in apipie have been > >>> put > >>> forward, *all* of them attempting to deal with various smells, > >>> code > >>> or > >>> process ones. Let's fix the root issues. > >>> > >>> Again, Let's get rid of validations in apipie. Refactor the > >>> controllers when needed. If apipie is not useful without those > >>> validations, let's suspend/stop use of it. > > > > Am I reading it correctly you suggesting getting rid of API > > documentation? > > > > -- Ivan > > > >>> > >>> -d > >>> > >> +1 > >>> > >>>> > >>>> It is wrong to put all your code logic into models. And > >>>> unfortunately we > >>>> have most of it there. Rails follows MVC pattern and what we > >>>> have > >>>> is > >>>> those > >>>> messy Models-for-everything approach. If you mix this with our > >>>> orchestration, it is deadly combination. Difficult testability, > >>>> huge > >>>> objects with no responsibility split, unable to mock things and > >>>> so > >>>> on. > >>>> Code logic in models works well for small projects and it is > >>>> de-facto > >>>> standard in Rails world. People just blindly follow this wrong > >>>> pattern. > >>>> > >>>> So I don't agree with this simplification you offer. We will > >>>> likely > >>>> start to refactor our orchestration creating new layer of code > >>>> logic, > >>>> which is the right approach to do things. And we should move > >>>> complex > >>>> methods like activation key subscription from models there. Then > >>>> we will > >>>> need validations on the controller side (or on the newly created > >>>> logic > >>>> layer or orchestration layer - depends on how are we gonna name > >>>> it). > >>>> > >>>> So yes, it has something to do with REST. My counter proposal > >>>> is: > >>>> > >>>> #2 Let's get rid of validations where they duplicate model > >>>> validators, > >>>> but let's keep them for (usually non-CRUD) methods which need > >>>> then. We > >>>> will likely be extending those soon. > >>>> > >>> > >>> _______________________________________________ > >>> katello-devel mailing list > >>> katello-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/katello-devel > >> > >> > >> > >> -- > >> Jason E. Rist > >> Senior Software Engineer > >> Systems Management and Cloud Enablement > >> Red Hat, Inc. > >> +1.919.754.4048 > >> Freenode: jrist > >> github/identi.ca: knowncitizen > >> > >> _______________________________________________ > >> katello-devel mailing list > >> katello-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/katello-devel > >> > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From dmitri at redhat.com Wed Dec 5 10:11:49 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Wed, 05 Dec 2012 10:11:49 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF0EE0.10600@redhat.com> References: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> <50BF0EE0.10600@redhat.com> Message-ID: <50BF1DE5.8010606@redhat.com> On 05/12/12 09:07 AM, Petr Chalupa wrote: > I agree that domain validation/consistency checks belongs to the > models. Nevertheless definition of the API is other thing entirely. It > should be defined and validated on its own with re-usage of > information provided by models (domain) to avoid duplications. It > should not be delegated to model to deal with it by adding other > responsibility. This statement is only valid if we are talking about cross-concerns (think authentication, security-related stuff, etc). Everything else will result in duplicated, non-consistent validations. Take a look at current state of validations in foreman api. I don't want to be jumping through hoops, and fixing/adding code/meta-data in multiple places, so just the documentation is up-to-date. It's a process/organizational issue, and tools do not fix those. And, once again, validations belong in the model. Let's get rid of validations in apipie, fix issues we have with domain logic leaking into controllers. > > So I strongly suggest to fix the duplication issue [1] and keep things > separate. > * Apipie for API validations. > * Model validations for domain logic. > > ## Fix Apipie > > Enable Apipie to read model definitions and validations to set param > validations automatically for parts of request representing a domain > model, see [1]. Agree with defining docs based on model (and I would suggest routes). Disagree with everything else. No point in intercepting validations, the most you could do in the controller is map exceptions thrown by the model to http errors/statuses. > > > + api documentation generated from api specification > + api specification is kept up to date (has to be because validation > is turned on) > + api validation/specification is separated from domain validations/logic > > ## Remove validations from Apipie > > + api documentation generated from api specification > - api specification is not kept up to date (has to be ensured in an > extra task) > - no api validation, it's partially handled by models, errors can be > ugly, models have another responsibility Again, tools are not the way to enforce process deficiencies. The last statement is FUD. > > > [1] - https://github.com/Pajk/apipie-rails/issues/67 > > ## Other notes > > > The second issue is that apipie validations are stricter than AR > ones: array validation assumes that there's an array in place, while > AR would accept a nil. These two combined lead to even more false > positives during apipie validation. > > I think be more consistent and strict is a good thing, it prevents > confusion. Empty Array is an empty Array not a nil. It's not good, because depending on how you use foreman, you'll get different results. What's considered valid/correct by foreman proper should be valid by any other means, period. -d > > Petr > > On 04.12.12 22:47, Ivan Necas wrote: >> >> >> ----- Original Message ----- >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>>>> Step #1 - let's get rid of controller validations. >>>>> Again, you can only do this for CRUD actions in the controllers. >>>>> For all >>>>> other methods we need validations. And we have those, not all our >>>>> controllers are plain REST-driven. We do have procedure-like >>>>> actions. >>>>> Maybe more then we should have, but this is a fact. >>>>> >>>>> You say all the validations should be done in models, thus you >>>>> essentially say let's put all our code logic there. Because it >>>>> does not >>>>> make sense to do validations in models while doing anything else >>>>> (that >>>>> also requires some level of validations) in controllers or a logic >>>>> layer. A fact: you need to do validations also outside of models. >>>> >>>> Domain logic belongs in the model layer. We can discuss how to >>>> better >>>> factor model layer code, but not where to put it. >>>> >>>> -d >>>> >>>> A lot of rationalizations to keep validations in apipie have been >>>> put >>>> forward, *all* of them attempting to deal with various smells, code >>>> or >>>> process ones. Let's fix the root issues. >>>> >>>> Again, Let's get rid of validations in apipie. Refactor the >>>> controllers when needed. If apipie is not useful without those >>>> validations, let's suspend/stop use of it. >> >> Am I reading it correctly you suggesting getting rid of API >> documentation? >> >> -- Ivan >> >>>> >>>> -d >>>> >>> +1 >>>> >>>>> >>>>> It is wrong to put all your code logic into models. And >>>>> unfortunately we >>>>> have most of it there. Rails follows MVC pattern and what we have >>>>> is >>>>> those >>>>> messy Models-for-everything approach. If you mix this with our >>>>> orchestration, it is deadly combination. Difficult testability, >>>>> huge >>>>> objects with no responsibility split, unable to mock things and so >>>>> on. >>>>> Code logic in models works well for small projects and it is >>>>> de-facto >>>>> standard in Rails world. People just blindly follow this wrong >>>>> pattern. >>>>> >>>>> So I don't agree with this simplification you offer. We will >>>>> likely >>>>> start to refactor our orchestration creating new layer of code >>>>> logic, >>>>> which is the right approach to do things. And we should move >>>>> complex >>>>> methods like activation key subscription from models there. Then >>>>> we will >>>>> need validations on the controller side (or on the newly created >>>>> logic >>>>> layer or orchestration layer - depends on how are we gonna name >>>>> it). >>>>> >>>>> So yes, it has something to do with REST. My counter proposal is: >>>>> >>>>> #2 Let's get rid of validations where they duplicate model >>>>> validators, >>>>> but let's keep them for (usually non-CRUD) methods which need >>>>> then. We >>>>> will likely be extending those soon. >>>>> >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >>> >>> >>> -- >>> Jason E. Rist >>> Senior Software Engineer >>> Systems Management and Cloud Enablement >>> Red Hat, Inc. >>> +1.919.754.4048 >>> Freenode: jrist >>> github/identi.ca: knowncitizen >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From dmitri at redhat.com Wed Dec 5 10:12:55 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Wed, 05 Dec 2012 10:12:55 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <2081436589.10734047.1354701104255.JavaMail.root@redhat.com> References: <2081436589.10734047.1354701104255.JavaMail.root@redhat.com> Message-ID: <50BF1E27.7000309@redhat.com> On 05/12/12 09:51 AM, Ivan Necas wrote: >> [1] - https://github.com/Pajk/apipie-rails/issues/67 >> >> ## Other notes >> >> > The second issue is that apipie validations are stricter than AR >> ones: array validation assumes that there's an array in place, while >> AR >> would accept a nil. These two combined lead to even more false >> positives >> during apipie validation. > There is :allow_nil option, that lets you do this. There is. But again, this is duplication. -d > >> On 04.12.12 22:47, Ivan Necas wrote: >>> >>> ----- Original Message ----- >>>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>>>>> Step #1 - let's get rid of controller validations. >>>>>> Again, you can only do this for CRUD actions in the controllers. >>>>>> For all >>>>>> other methods we need validations. And we have those, not all >>>>>> our >>>>>> controllers are plain REST-driven. We do have procedure-like >>>>>> actions. >>>>>> Maybe more then we should have, but this is a fact. >>>>>> >>>>>> You say all the validations should be done in models, thus you >>>>>> essentially say let's put all our code logic there. Because it >>>>>> does not >>>>>> make sense to do validations in models while doing anything else >>>>>> (that >>>>>> also requires some level of validations) in controllers or a >>>>>> logic >>>>>> layer. A fact: you need to do validations also outside of >>>>>> models. >>>>> Domain logic belongs in the model layer. We can discuss how to >>>>> better >>>>> factor model layer code, but not where to put it. >>>>> >>>>> -d >>>>> >>>>> A lot of rationalizations to keep validations in apipie have been >>>>> put >>>>> forward, *all* of them attempting to deal with various smells, >>>>> code >>>>> or >>>>> process ones. Let's fix the root issues. >>>>> >>>>> Again, Let's get rid of validations in apipie. Refactor the >>>>> controllers when needed. If apipie is not useful without those >>>>> validations, let's suspend/stop use of it. >>> Am I reading it correctly you suggesting getting rid of API >>> documentation? >>> >>> -- Ivan >>> >>>>> -d >>>>> >>>> +1 >>>>>> It is wrong to put all your code logic into models. And >>>>>> unfortunately we >>>>>> have most of it there. Rails follows MVC pattern and what we >>>>>> have >>>>>> is >>>>>> those >>>>>> messy Models-for-everything approach. If you mix this with our >>>>>> orchestration, it is deadly combination. Difficult testability, >>>>>> huge >>>>>> objects with no responsibility split, unable to mock things and >>>>>> so >>>>>> on. >>>>>> Code logic in models works well for small projects and it is >>>>>> de-facto >>>>>> standard in Rails world. People just blindly follow this wrong >>>>>> pattern. >>>>>> >>>>>> So I don't agree with this simplification you offer. We will >>>>>> likely >>>>>> start to refactor our orchestration creating new layer of code >>>>>> logic, >>>>>> which is the right approach to do things. And we should move >>>>>> complex >>>>>> methods like activation key subscription from models there. Then >>>>>> we will >>>>>> need validations on the controller side (or on the newly created >>>>>> logic >>>>>> layer or orchestration layer - depends on how are we gonna name >>>>>> it). >>>>>> >>>>>> So yes, it has something to do with REST. My counter proposal >>>>>> is: >>>>>> >>>>>> #2 Let's get rid of validations where they duplicate model >>>>>> validators, >>>>>> but let's keep them for (usually non-CRUD) methods which need >>>>>> then. We >>>>>> will likely be extending those soon. >>>>>> >>>>> _______________________________________________ >>>>> katello-devel mailing list >>>>> katello-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> >>>> >>>> -- >>>> Jason E. Rist >>>> Senior Software Engineer >>>> Systems Management and Cloud Enablement >>>> Red Hat, Inc. >>>> +1.919.754.4048 >>>> Freenode: jrist >>>> github/identi.ca: knowncitizen >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From dmitri at redhat.com Wed Dec 5 10:13:58 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Wed, 05 Dec 2012 10:13:58 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> References: <1826296547.10529109.1354657661469.JavaMail.root@redhat.com> Message-ID: <50BF1E66.9080901@redhat.com> On 04/12/12 09:47 PM, Ivan Necas wrote: > > ----- Original Message ----- >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>>> Step #1 - let's get rid of controller validations. >>>> Again, you can only do this for CRUD actions in the controllers. >>>> For all >>>> other methods we need validations. And we have those, not all our >>>> controllers are plain REST-driven. We do have procedure-like >>>> actions. >>>> Maybe more then we should have, but this is a fact. >>>> >>>> You say all the validations should be done in models, thus you >>>> essentially say let's put all our code logic there. Because it >>>> does not >>>> make sense to do validations in models while doing anything else >>>> (that >>>> also requires some level of validations) in controllers or a logic >>>> layer. A fact: you need to do validations also outside of models. >>> Domain logic belongs in the model layer. We can discuss how to >>> better >>> factor model layer code, but not where to put it. >>> >>> -d >>> >>> A lot of rationalizations to keep validations in apipie have been >>> put >>> forward, *all* of them attempting to deal with various smells, code >>> or >>> process ones. Let's fix the root issues. >>> >>> Again, Let's get rid of validations in apipie. Refactor the >>> controllers when needed. If apipie is not useful without those >>> validations, let's suspend/stop use of it. > Am I reading it correctly you suggesting getting rid of API documentation? Only if it's not useful. Or we can't agree that it's useful. Do we have any consumers for docs atm? -d > > -- Ivan > >>> -d >>> >> +1 >>>> It is wrong to put all your code logic into models. And >>>> unfortunately we >>>> have most of it there. Rails follows MVC pattern and what we have >>>> is >>>> those >>>> messy Models-for-everything approach. If you mix this with our >>>> orchestration, it is deadly combination. Difficult testability, >>>> huge >>>> objects with no responsibility split, unable to mock things and so >>>> on. >>>> Code logic in models works well for small projects and it is >>>> de-facto >>>> standard in Rails world. People just blindly follow this wrong >>>> pattern. >>>> >>>> So I don't agree with this simplification you offer. We will >>>> likely >>>> start to refactor our orchestration creating new layer of code >>>> logic, >>>> which is the right approach to do things. And we should move >>>> complex >>>> methods like activation key subscription from models there. Then >>>> we will >>>> need validations on the controller side (or on the newly created >>>> logic >>>> layer or orchestration layer - depends on how are we gonna name >>>> it). >>>> >>>> So yes, it has something to do with REST. My counter proposal is: >>>> >>>> #2 Let's get rid of validations where they duplicate model >>>> validators, >>>> but let's keep them for (usually non-CRUD) methods which need >>>> then. We >>>> will likely be extending those soon. >>>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> >> -- >> Jason E. Rist >> Senior Software Engineer >> Systems Management and Cloud Enablement >> Red Hat, Inc. >> +1.919.754.4048 >> Freenode: jrist >> github/identi.ca: knowncitizen >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> From ohadlevy at redhat.com Wed Dec 5 12:20:25 2012 From: ohadlevy at redhat.com (Ohad Levy) Date: Wed, 5 Dec 2012 07:20:25 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF1DE5.8010606@redhat.com> Message-ID: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> ----- Original Message ----- | On 05/12/12 09:07 AM, Petr Chalupa wrote: | > I agree that domain validation/consistency checks belongs to the | > models. Nevertheless definition of the API is other thing entirely. | > It | > should be defined and validated on its own with re-usage of | > information provided by models (domain) to avoid duplications. It | > should not be delegated to model to deal with it by adding other | > responsibility. | | This statement is only valid if we are talking about cross-concerns | (think authentication, security-related stuff, etc). Everything else | will result in duplicated, non-consistent validations. Take a look at | current state of validations in foreman api. | What do you mean by that? | I don't want to be jumping through hoops, and fixing/adding | code/meta-data in multiple places, so just the documentation is | up-to-date. It's a process/organizational issue, and tools do not fix | those. | | And, once again, validations belong in the model. Let's get rid of | validations in apipie, fix issues we have with domain logic leaking | into | controllers. | | > | > So I strongly suggest to fix the duplication issue [1] and keep | > things | > separate. | > * Apipie for API validations. | > * Model validations for domain logic. | > | > ## Fix Apipie | > | > Enable Apipie to read model definitions and validations to set | > param | > validations automatically for parts of request representing a | > domain | > model, see [1]. | | Agree with defining docs based on model (and I would suggest routes). | Disagree with everything else. No point in intercepting validations, | the | most you could do in the controller is map exceptions thrown by the | model to http errors/statuses. | | > | > | > + api documentation generated from api specification | > + api specification is kept up to date (has to be because | > validation | > is turned on) | > + api validation/specification is separated from domain | > validations/logic | > | > ## Remove validations from Apipie | > | > + api documentation generated from api specification | > - api specification is not kept up to date (has to be ensured in an | > extra task) | > - no api validation, it's partially handled by models, errors can | > be | > ugly, models have another responsibility | | Again, tools are not the way to enforce process deficiencies. The | last | statement is FUD. | | > | > | > [1] - https://github.com/Pajk/apipie-rails/issues/67 | > | > ## Other notes | > | > > The second issue is that apipie validations are stricter than AR | > ones: array validation assumes that there's an array in place, | > while | > AR would accept a nil. These two combined lead to even more false | > positives during apipie validation. | > | > I think be more consistent and strict is a good thing, it prevents | > confusion. Empty Array is an empty Array not a nil. | | It's not good, because depending on how you use foreman, you'll get | different results. What's considered valid/correct by foreman proper | should be valid by any other means, period. | | -d | | > | > Petr | > | > On 04.12.12 22:47, Ivan Necas wrote: | >> | >> | >> ----- Original Message ----- | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: | >>>>>> Step #1 - let's get rid of controller validations. | >>>>> Again, you can only do this for CRUD actions in the | >>>>> controllers. | >>>>> For all | >>>>> other methods we need validations. And we have those, not all | >>>>> our | >>>>> controllers are plain REST-driven. We do have procedure-like | >>>>> actions. | >>>>> Maybe more then we should have, but this is a fact. | >>>>> | >>>>> You say all the validations should be done in models, thus you | >>>>> essentially say let's put all our code logic there. Because it | >>>>> does not | >>>>> make sense to do validations in models while doing anything | >>>>> else | >>>>> (that | >>>>> also requires some level of validations) in controllers or a | >>>>> logic | >>>>> layer. A fact: you need to do validations also outside of | >>>>> models. | >>>> | >>>> Domain logic belongs in the model layer. We can discuss how to | >>>> better | >>>> factor model layer code, but not where to put it. | >>>> | >>>> -d | >>>> | >>>> A lot of rationalizations to keep validations in apipie have | >>>> been | >>>> put | >>>> forward, *all* of them attempting to deal with various smells, | >>>> code | >>>> or | >>>> process ones. Let's fix the root issues. | >>>> | >>>> Again, Let's get rid of validations in apipie. Refactor the | >>>> controllers when needed. If apipie is not useful without those | >>>> validations, let's suspend/stop use of it. | >> | >> Am I reading it correctly you suggesting getting rid of API | >> documentation? | >> | >> -- Ivan | >> | >>>> | >>>> -d | >>>> | >>> +1 | >>>> | >>>>> | >>>>> It is wrong to put all your code logic into models. And | >>>>> unfortunately we | >>>>> have most of it there. Rails follows MVC pattern and what we | >>>>> have | >>>>> is | >>>>> those | >>>>> messy Models-for-everything approach. If you mix this with our | >>>>> orchestration, it is deadly combination. Difficult testability, | >>>>> huge | >>>>> objects with no responsibility split, unable to mock things and | >>>>> so | >>>>> on. | >>>>> Code logic in models works well for small projects and it is | >>>>> de-facto | >>>>> standard in Rails world. People just blindly follow this wrong | >>>>> pattern. | >>>>> | >>>>> So I don't agree with this simplification you offer. We will | >>>>> likely | >>>>> start to refactor our orchestration creating new layer of code | >>>>> logic, | >>>>> which is the right approach to do things. And we should move | >>>>> complex | >>>>> methods like activation key subscription from models there. | >>>>> Then | >>>>> we will | >>>>> need validations on the controller side (or on the newly | >>>>> created | >>>>> logic | >>>>> layer or orchestration layer - depends on how are we gonna name | >>>>> it). | >>>>> | >>>>> So yes, it has something to do with REST. My counter proposal | >>>>> is: | >>>>> | >>>>> #2 Let's get rid of validations where they duplicate model | >>>>> validators, | >>>>> but let's keep them for (usually non-CRUD) methods which need | >>>>> then. We | >>>>> will likely be extending those soon. | >>>>> | >>>> | >>>> _______________________________________________ | >>>> katello-devel mailing list | >>>> katello-devel at redhat.com | >>>> https://www.redhat.com/mailman/listinfo/katello-devel | >>> | >>> | >>> | >>> -- | >>> Jason E. Rist | >>> Senior Software Engineer | >>> Systems Management and Cloud Enablement | >>> Red Hat, Inc. | >>> +1.919.754.4048 | >>> Freenode: jrist | >>> github/identi.ca: knowncitizen | >>> | >>> _______________________________________________ | >>> katello-devel mailing list | >>> katello-devel at redhat.com | >>> https://www.redhat.com/mailman/listinfo/katello-devel | >>> | >> | >> _______________________________________________ | >> katello-devel mailing list | >> katello-devel at redhat.com | >> https://www.redhat.com/mailman/listinfo/katello-devel | >> | > | > _______________________________________________ | > katello-devel mailing list | > katello-devel at redhat.com | > https://www.redhat.com/mailman/listinfo/katello-devel | | _______________________________________________ | katello-devel mailing list | katello-devel at redhat.com | https://www.redhat.com/mailman/listinfo/katello-devel | From ohadlevy at redhat.com Wed Dec 5 12:21:28 2012 From: ohadlevy at redhat.com (Ohad Levy) Date: Wed, 5 Dec 2012 07:21:28 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF1E66.9080901@redhat.com> Message-ID: <221048513.4283101.1354710088141.JavaMail.root@redhat.com> ----- Original Message ----- | On 04/12/12 09:47 PM, Ivan Necas wrote: | > | > ----- Original Message ----- | >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: | >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: | >>>>> Step #1 - let's get rid of controller validations. | >>>> Again, you can only do this for CRUD actions in the controllers. | >>>> For all | >>>> other methods we need validations. And we have those, not all | >>>> our | >>>> controllers are plain REST-driven. We do have procedure-like | >>>> actions. | >>>> Maybe more then we should have, but this is a fact. | >>>> | >>>> You say all the validations should be done in models, thus you | >>>> essentially say let's put all our code logic there. Because it | >>>> does not | >>>> make sense to do validations in models while doing anything else | >>>> (that | >>>> also requires some level of validations) in controllers or a | >>>> logic | >>>> layer. A fact: you need to do validations also outside of | >>>> models. | >>> Domain logic belongs in the model layer. We can discuss how to | >>> better | >>> factor model layer code, but not where to put it. | >>> | >>> -d | >>> | >>> A lot of rationalizations to keep validations in apipie have been | >>> put | >>> forward, *all* of them attempting to deal with various smells, | >>> code | >>> or | >>> process ones. Let's fix the root issues. | >>> | >>> Again, Let's get rid of validations in apipie. Refactor the | >>> controllers when needed. If apipie is not useful without those | >>> validations, let's suspend/stop use of it. | > Am I reading it correctly you suggesting getting rid of API | > documentation? | | Only if it's not useful. Or we can't agree that it's useful. Do we | have | any consumers for docs atm? I would argue that generating a client based on the binding (CLI or even katello) should benefit from the validations as it would safe a roundtrip. Ohad | -d | | > | > -- Ivan | > | >>> -d | >>> | >> +1 | >>>> It is wrong to put all your code logic into models. And | >>>> unfortunately we | >>>> have most of it there. Rails follows MVC pattern and what we | >>>> have | >>>> is | >>>> those | >>>> messy Models-for-everything approach. If you mix this with our | >>>> orchestration, it is deadly combination. Difficult testability, | >>>> huge | >>>> objects with no responsibility split, unable to mock things and | >>>> so | >>>> on. | >>>> Code logic in models works well for small projects and it is | >>>> de-facto | >>>> standard in Rails world. People just blindly follow this wrong | >>>> pattern. | >>>> | >>>> So I don't agree with this simplification you offer. We will | >>>> likely | >>>> start to refactor our orchestration creating new layer of code | >>>> logic, | >>>> which is the right approach to do things. And we should move | >>>> complex | >>>> methods like activation key subscription from models there. Then | >>>> we will | >>>> need validations on the controller side (or on the newly created | >>>> logic | >>>> layer or orchestration layer - depends on how are we gonna name | >>>> it). | >>>> | >>>> So yes, it has something to do with REST. My counter proposal | >>>> is: | >>>> | >>>> #2 Let's get rid of validations where they duplicate model | >>>> validators, | >>>> but let's keep them for (usually non-CRUD) methods which need | >>>> then. We | >>>> will likely be extending those soon. | >>>> | >>> _______________________________________________ | >>> katello-devel mailing list | >>> katello-devel at redhat.com | >>> https://www.redhat.com/mailman/listinfo/katello-devel | >> | >> | >> -- | >> Jason E. Rist | >> Senior Software Engineer | >> Systems Management and Cloud Enablement | >> Red Hat, Inc. | >> +1.919.754.4048 | >> Freenode: jrist | >> github/identi.ca: knowncitizen | >> | >> _______________________________________________ | >> katello-devel mailing list | >> katello-devel at redhat.com | >> https://www.redhat.com/mailman/listinfo/katello-devel | >> | | _______________________________________________ | katello-devel mailing list | katello-devel at redhat.com | https://www.redhat.com/mailman/listinfo/katello-devel | From dmitri at redhat.com Wed Dec 5 12:39:42 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Wed, 05 Dec 2012 12:39:42 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> References: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> Message-ID: <50BF408E.1030609@redhat.com> On 05/12/12 12:20 PM, Ohad Levy wrote: > > ----- Original Message ----- > | On 05/12/12 09:07 AM, Petr Chalupa wrote: > | > I agree that domain validation/consistency checks belongs to the > | > models. Nevertheless definition of the API is other thing entirely. > | > It > | > should be defined and validated on its own with re-usage of > | > information provided by models (domain) to avoid duplications. It > | > should not be delegated to model to deal with it by adding other > | > responsibility. > | > | This statement is only valid if we are talking about cross-concerns > | (think authentication, security-related stuff, etc). Everything else > | will result in duplicated, non-consistent validations. Take a look at > | current state of validations in foreman api. > | > What do you mean by that? Which part? I insist that controllers should not carry any business logic in them, as a corollary to this, no validations should reside in controllers, unless those are used across all controllers, and are outside of business-logic constraints (think aspect-like behaviour). By current state I meant that there are currently inconsistencies (data coming through api are flagged invalid, even though it's valid according to validators on the model) between api and model validations. Config_template has a few, for example (and what started this thread). -d > > | I don't want to be jumping through hoops, and fixing/adding > | code/meta-data in multiple places, so just the documentation is > | up-to-date. It's a process/organizational issue, and tools do not fix > | those. > | > | And, once again, validations belong in the model. Let's get rid of > | validations in apipie, fix issues we have with domain logic leaking > | into > | controllers. > | > | > > | > So I strongly suggest to fix the duplication issue [1] and keep > | > things > | > separate. > | > * Apipie for API validations. > | > * Model validations for domain logic. > | > > | > ## Fix Apipie > | > > | > Enable Apipie to read model definitions and validations to set > | > param > | > validations automatically for parts of request representing a > | > domain > | > model, see [1]. > | > | Agree with defining docs based on model (and I would suggest routes). > | Disagree with everything else. No point in intercepting validations, > | the > | most you could do in the controller is map exceptions thrown by the > | model to http errors/statuses. > | > | > > | > > | > + api documentation generated from api specification > | > + api specification is kept up to date (has to be because > | > validation > | > is turned on) > | > + api validation/specification is separated from domain > | > validations/logic > | > > | > ## Remove validations from Apipie > | > > | > + api documentation generated from api specification > | > - api specification is not kept up to date (has to be ensured in an > | > extra task) > | > - no api validation, it's partially handled by models, errors can > | > be > | > ugly, models have another responsibility > | > | Again, tools are not the way to enforce process deficiencies. The > | last > | statement is FUD. > | > | > > | > > | > [1] - https://github.com/Pajk/apipie-rails/issues/67 > | > > | > ## Other notes > | > > | > > The second issue is that apipie validations are stricter than AR > | > ones: array validation assumes that there's an array in place, > | > while > | > AR would accept a nil. These two combined lead to even more false > | > positives during apipie validation. > | > > | > I think be more consistent and strict is a good thing, it prevents > | > confusion. Empty Array is an empty Array not a nil. > | > | It's not good, because depending on how you use foreman, you'll get > | different results. What's considered valid/correct by foreman proper > | should be valid by any other means, period. > | > | -d > | > | > > | > Petr > | > > | > On 04.12.12 22:47, Ivan Necas wrote: > | >> > | >> > | >> ----- Original Message ----- > | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: > | >>>>>> Step #1 - let's get rid of controller validations. > | >>>>> Again, you can only do this for CRUD actions in the > | >>>>> controllers. > | >>>>> For all > | >>>>> other methods we need validations. And we have those, not all > | >>>>> our > | >>>>> controllers are plain REST-driven. We do have procedure-like > | >>>>> actions. > | >>>>> Maybe more then we should have, but this is a fact. > | >>>>> > | >>>>> You say all the validations should be done in models, thus you > | >>>>> essentially say let's put all our code logic there. Because it > | >>>>> does not > | >>>>> make sense to do validations in models while doing anything > | >>>>> else > | >>>>> (that > | >>>>> also requires some level of validations) in controllers or a > | >>>>> logic > | >>>>> layer. A fact: you need to do validations also outside of > | >>>>> models. > | >>>> > | >>>> Domain logic belongs in the model layer. We can discuss how to > | >>>> better > | >>>> factor model layer code, but not where to put it. > | >>>> > | >>>> -d > | >>>> > | >>>> A lot of rationalizations to keep validations in apipie have > | >>>> been > | >>>> put > | >>>> forward, *all* of them attempting to deal with various smells, > | >>>> code > | >>>> or > | >>>> process ones. Let's fix the root issues. > | >>>> > | >>>> Again, Let's get rid of validations in apipie. Refactor the > | >>>> controllers when needed. If apipie is not useful without those > | >>>> validations, let's suspend/stop use of it. > | >> > | >> Am I reading it correctly you suggesting getting rid of API > | >> documentation? > | >> > | >> -- Ivan > | >> > | >>>> > | >>>> -d > | >>>> > | >>> +1 > | >>>> > | >>>>> > | >>>>> It is wrong to put all your code logic into models. And > | >>>>> unfortunately we > | >>>>> have most of it there. Rails follows MVC pattern and what we > | >>>>> have > | >>>>> is > | >>>>> those > | >>>>> messy Models-for-everything approach. If you mix this with our > | >>>>> orchestration, it is deadly combination. Difficult testability, > | >>>>> huge > | >>>>> objects with no responsibility split, unable to mock things and > | >>>>> so > | >>>>> on. > | >>>>> Code logic in models works well for small projects and it is > | >>>>> de-facto > | >>>>> standard in Rails world. People just blindly follow this wrong > | >>>>> pattern. > | >>>>> > | >>>>> So I don't agree with this simplification you offer. We will > | >>>>> likely > | >>>>> start to refactor our orchestration creating new layer of code > | >>>>> logic, > | >>>>> which is the right approach to do things. And we should move > | >>>>> complex > | >>>>> methods like activation key subscription from models there. > | >>>>> Then > | >>>>> we will > | >>>>> need validations on the controller side (or on the newly > | >>>>> created > | >>>>> logic > | >>>>> layer or orchestration layer - depends on how are we gonna name > | >>>>> it). > | >>>>> > | >>>>> So yes, it has something to do with REST. My counter proposal > | >>>>> is: > | >>>>> > | >>>>> #2 Let's get rid of validations where they duplicate model > | >>>>> validators, > | >>>>> but let's keep them for (usually non-CRUD) methods which need > | >>>>> then. We > | >>>>> will likely be extending those soon. > | >>>>> > | >>>> > | >>>> _______________________________________________ > | >>>> katello-devel mailing list > | >>>> katello-devel at redhat.com > | >>>> https://www.redhat.com/mailman/listinfo/katello-devel > | >>> > | >>> > | >>> > | >>> -- > | >>> Jason E. Rist > | >>> Senior Software Engineer > | >>> Systems Management and Cloud Enablement > | >>> Red Hat, Inc. > | >>> +1.919.754.4048 > | >>> Freenode: jrist > | >>> github/identi.ca: knowncitizen > | >>> > | >>> _______________________________________________ > | >>> katello-devel mailing list > | >>> katello-devel at redhat.com > | >>> https://www.redhat.com/mailman/listinfo/katello-devel > | >>> > | >> > | >> _______________________________________________ > | >> katello-devel mailing list > | >> katello-devel at redhat.com > | >> https://www.redhat.com/mailman/listinfo/katello-devel > | >> > | > > | > _______________________________________________ > | > katello-devel mailing list > | > katello-devel at redhat.com > | > https://www.redhat.com/mailman/listinfo/katello-devel > | > | _______________________________________________ > | katello-devel mailing list > | katello-devel at redhat.com > | https://www.redhat.com/mailman/listinfo/katello-devel > | From dmitri at redhat.com Wed Dec 5 12:46:48 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Wed, 05 Dec 2012 12:46:48 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <221048513.4283101.1354710088141.JavaMail.root@redhat.com> References: <221048513.4283101.1354710088141.JavaMail.root@redhat.com> Message-ID: <50BF4238.5020201@redhat.com> On 05/12/12 12:21 PM, Ohad Levy wrote: > > ----- Original Message ----- > | On 04/12/12 09:47 PM, Ivan Necas wrote: > | > > | > ----- Original Message ----- > | >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > | >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: > | >>>>> Step #1 - let's get rid of controller validations. > | >>>> Again, you can only do this for CRUD actions in the controllers. > | >>>> For all > | >>>> other methods we need validations. And we have those, not all > | >>>> our > | >>>> controllers are plain REST-driven. We do have procedure-like > | >>>> actions. > | >>>> Maybe more then we should have, but this is a fact. > | >>>> > | >>>> You say all the validations should be done in models, thus you > | >>>> essentially say let's put all our code logic there. Because it > | >>>> does not > | >>>> make sense to do validations in models while doing anything else > | >>>> (that > | >>>> also requires some level of validations) in controllers or a > | >>>> logic > | >>>> layer. A fact: you need to do validations also outside of > | >>>> models. > | >>> Domain logic belongs in the model layer. We can discuss how to > | >>> better > | >>> factor model layer code, but not where to put it. > | >>> > | >>> -d > | >>> > | >>> A lot of rationalizations to keep validations in apipie have been > | >>> put > | >>> forward, *all* of them attempting to deal with various smells, > | >>> code > | >>> or > | >>> process ones. Let's fix the root issues. > | >>> > | >>> Again, Let's get rid of validations in apipie. Refactor the > | >>> controllers when needed. If apipie is not useful without those > | >>> validations, let's suspend/stop use of it. > | > Am I reading it correctly you suggesting getting rid of API > | > documentation? > | > | Only if it's not useful. Or we can't agree that it's useful. Do we > | have > | any consumers for docs atm? > > I would argue that generating a client based on the binding (CLI or even katello) should benefit from the validations as it would safe a roundtrip. I'm not saying we don't need validations. I'm saying that mustn't duplicate validations, or constraint them above what model considers valid. Violating this results in code duplication and bizarre behaviour when server can't consume data it produced. among other things. As for client generation, I'm *still* not convinced it's worth it in the long run, and I'm definitely experiencing more work in the short term: atm I need to make changes on the foreman side, make changes in foreman_api, generate a new version of the gem, and only then I'll be able to test the changes. For something that changes a lot (as foreman api currently are) this is hardly optimal, or convenient (esp. taking into account that Katello is the only consumer of these api atm). -d > > Ohad > | -d > | > | > > | > -- Ivan > | > > | >>> -d > | >>> > | >> +1 > | >>>> It is wrong to put all your code logic into models. And > | >>>> unfortunately we > | >>>> have most of it there. Rails follows MVC pattern and what we > | >>>> have > | >>>> is > | >>>> those > | >>>> messy Models-for-everything approach. If you mix this with our > | >>>> orchestration, it is deadly combination. Difficult testability, > | >>>> huge > | >>>> objects with no responsibility split, unable to mock things and > | >>>> so > | >>>> on. > | >>>> Code logic in models works well for small projects and it is > | >>>> de-facto > | >>>> standard in Rails world. People just blindly follow this wrong > | >>>> pattern. > | >>>> > | >>>> So I don't agree with this simplification you offer. We will > | >>>> likely > | >>>> start to refactor our orchestration creating new layer of code > | >>>> logic, > | >>>> which is the right approach to do things. And we should move > | >>>> complex > | >>>> methods like activation key subscription from models there. Then > | >>>> we will > | >>>> need validations on the controller side (or on the newly created > | >>>> logic > | >>>> layer or orchestration layer - depends on how are we gonna name > | >>>> it). > | >>>> > | >>>> So yes, it has something to do with REST. My counter proposal > | >>>> is: > | >>>> > | >>>> #2 Let's get rid of validations where they duplicate model > | >>>> validators, > | >>>> but let's keep them for (usually non-CRUD) methods which need > | >>>> then. We > | >>>> will likely be extending those soon. > | >>>> > | >>> _______________________________________________ > | >>> katello-devel mailing list > | >>> katello-devel at redhat.com > | >>> https://www.redhat.com/mailman/listinfo/katello-devel > | >> > | >> > | >> -- > | >> Jason E. Rist > | >> Senior Software Engineer > | >> Systems Management and Cloud Enablement > | >> Red Hat, Inc. > | >> +1.919.754.4048 > | >> Freenode: jrist > | >> github/identi.ca: knowncitizen > | >> > | >> _______________________________________________ > | >> katello-devel mailing list > | >> katello-devel at redhat.com > | >> https://www.redhat.com/mailman/listinfo/katello-devel > | >> > | > | _______________________________________________ > | katello-devel mailing list > | katello-devel at redhat.com > | https://www.redhat.com/mailman/listinfo/katello-devel > | From lzap at redhat.com Wed Dec 5 13:32:16 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 5 Dec 2012 14:32:16 +0100 Subject: [katello-devel] Gemfile split Message-ID: <20121205133216.GH2315@lzapx.brq.redhat.com> Hello, my Gemfile split PR was finally merged and I am happy to deliver this for Katello 1.2. More in the upcoming blog post, but the main thing is: Gemfile now only lists production gems, all other groups are in bundle.d directory: $ ls bundler.d/ -1 apipie.rb coverage.rb debugging.rb development_boost.rb development.rb foreman.rb checking.rb profiling.rb test.rb The change is not bringing any change in bundler behavior, everything remains the same except the gems are in different files. I also reordered them, cleaned it out, put several TODOs there and also I have split development and test into two separate groups. During development you usually work with bundler while on production setup or RPM-development setup you have bundler_ext enabled. To enable bundler_ext install all RPM dependencies and then rename Gemfile as Gemfile.in - that's it. Rename back and you have bundler back. The most important bit that loads groups was extended a bit: https://gist.github.com/4215458 Short description. With bundler enabled, we load the following groups: production -> default, foreman development -> ditto + development, apipie, development_boost test -> ditto + test, debugging In bundler_ext mode we load them ALL by default so you can install or remove various katello "plugins" giving you gems you want. We currently only have the following devel "plugins": * katello-devel-all (installs all groups bellow) * katello-devel (development rubygem group) * katello-devel-test (unit testing rubygem group) * katello-devel-profiling (profiling rubygem group) * katello-devel-checking (javascript analysis rubygem group) * katello-devel-coverage (test coverage measurement group) * katello-devel-debugging (ruby debugger dependencies) Well yeah, there actually IS the first plugin called "foreman". The bundler.d/foreman.rb is only installed when you install katello-glue-foreman. Without it, katello do not need foreman_api rubygem. There is one exception in the bundler_ext mode - if webmock is found and Rails environment is not "test", we disable it's mocking capabilities because it development or production mode with katello-devel-test package installed it was causing issues (blocking from HTTP queries). Therefore to use test rake tasks in RPM-based dev setup, use this: RAILS_ENV=test rake test xyz ... By the way, this made RPM Development Setup even more simple: https://fedorahosted.org/katello/wiki/DevelopmentSetup The shortest possible path now: 1) Install katello 2) Install katello-devel-all 3) Set KATELLO_ENV=development in the sysconfig file 4) Restart LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From inecas at redhat.com Wed Dec 5 13:43:52 2012 From: inecas at redhat.com (Ivan Necas) Date: Wed, 5 Dec 2012 08:43:52 -0500 (EST) Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF1E66.9080901@redhat.com> Message-ID: <592807782.10893062.1354715032692.JavaMail.root@redhat.com> ----- Original Message ----- > On 04/12/12 09:47 PM, Ivan Necas wrote: > > > > ----- Original Message ----- > >> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: > >>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: > >>>>> Step #1 - let's get rid of controller validations. > >>>> Again, you can only do this for CRUD actions in the controllers. > >>>> For all > >>>> other methods we need validations. And we have those, not all > >>>> our > >>>> controllers are plain REST-driven. We do have procedure-like > >>>> actions. > >>>> Maybe more then we should have, but this is a fact. > >>>> > >>>> You say all the validations should be done in models, thus you > >>>> essentially say let's put all our code logic there. Because it > >>>> does not > >>>> make sense to do validations in models while doing anything else > >>>> (that > >>>> also requires some level of validations) in controllers or a > >>>> logic > >>>> layer. A fact: you need to do validations also outside of > >>>> models. > >>> Domain logic belongs in the model layer. We can discuss how to > >>> better > >>> factor model layer code, but not where to put it. > >>> > >>> -d > >>> > >>> A lot of rationalizations to keep validations in apipie have been > >>> put > >>> forward, *all* of them attempting to deal with various smells, > >>> code > >>> or > >>> process ones. Let's fix the root issues. > >>> > >>> Again, Let's get rid of validations in apipie. Refactor the > >>> controllers when needed. If apipie is not useful without those > >>> validations, let's suspend/stop use of it. > > Am I reading it correctly you suggesting getting rid of API > > documentation? > > Only if it's not useful. Or we can't agree that it's useful. Do we > have any consumers for docs atm? The documentation is planned to be available on the new Foreman site and I consider it really valuable for the project to provide the information. It shows that the developers care. I've also used it just now when scripting something against Foreman and it was subjectively really good feeling to have something I can look at. Anyway, I don't think this is the right mailing list to decide, or even discuss this. -- Ivan > -d > > > > > -- Ivan > > > >>> -d > >>> > >> +1 > >>>> It is wrong to put all your code logic into models. And > >>>> unfortunately we > >>>> have most of it there. Rails follows MVC pattern and what we > >>>> have > >>>> is > >>>> those > >>>> messy Models-for-everything approach. If you mix this with our > >>>> orchestration, it is deadly combination. Difficult testability, > >>>> huge > >>>> objects with no responsibility split, unable to mock things and > >>>> so > >>>> on. > >>>> Code logic in models works well for small projects and it is > >>>> de-facto > >>>> standard in Rails world. People just blindly follow this wrong > >>>> pattern. > >>>> > >>>> So I don't agree with this simplification you offer. We will > >>>> likely > >>>> start to refactor our orchestration creating new layer of code > >>>> logic, > >>>> which is the right approach to do things. And we should move > >>>> complex > >>>> methods like activation key subscription from models there. Then > >>>> we will > >>>> need validations on the controller side (or on the newly created > >>>> logic > >>>> layer or orchestration layer - depends on how are we gonna name > >>>> it). > >>>> > >>>> So yes, it has something to do with REST. My counter proposal > >>>> is: > >>>> > >>>> #2 Let's get rid of validations where they duplicate model > >>>> validators, > >>>> but let's keep them for (usually non-CRUD) methods which need > >>>> then. We > >>>> will likely be extending those soon. > >>>> > >>> _______________________________________________ > >>> katello-devel mailing list > >>> katello-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/katello-devel > >> > >> > >> -- > >> Jason E. Rist > >> Senior Software Engineer > >> Systems Management and Cloud Enablement > >> Red Hat, Inc. > >> +1.919.754.4048 > >> Freenode: jrist > >> github/identi.ca: knowncitizen > >> > >> _______________________________________________ > >> katello-devel mailing list > >> katello-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/katello-devel > >> > > From bkearney at redhat.com Wed Dec 5 14:36:28 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 05 Dec 2012 09:36:28 -0500 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 Message-ID: <50BF5BEC.2030005@redhat.com> I can not comment on a comment, so I figured I would bring them back here: This is about migrating from spacewalk to katello. The pattern which is in the pull request is an ETL (Expoert/Transfrom/Load) process, where the Extrac and Trasform are done in the export steps, and the Load is done by the CLI in the import actions. The main comment from Lzap and Msuchy was around using DB to DB tools as opposed to to using scripts. The main reason for this is that today, katello is 4 data stores and will soon be 5. When we create an activation key, it goes into katello, elastic search, and candlepin. Systems will go into pulp, candlepin, katello, ES, and foreman. If we do postgres to postgres, we will end up having to to write something which says "go into the Katello DB, and push the changes out to all the other systems". TBF, we may need that anyway in the future. Some other questions: * How to deal with None: This is treated as an email column in the CSV/JSON * CSV Sucks! : Yes, but the import portions can be used without the export.. so you can create your data in $YOUR_SPREADSHEET_TOOL and then load them up. * The performance will suck: Could be. But I do not believe we need to migrate over either the package manifest or the action history. I do not think users will care about this. Rather, we need pull over the system list and the their subscriptions. The comment that the export loads all the data into memory is valid. I should fix that. * Has it been tested on real data: Yes, but small datasets. * Why do it in private: I did send out on RFC a while ago. Then figured something running would be easier to discuss. -- bk From lzap at redhat.com Wed Dec 5 15:33:13 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 5 Dec 2012 16:33:13 +0100 Subject: [katello-devel] Katello upgrade and Puppet Message-ID: <20121205153313.GI2315@lzapx.brq.redhat.com> Hi, recent Petr's changes showed that katello-upgrade does not play well with puppet. We have touched this issue several times now: - change touches a puppet template - katello upgrade does this then: - stops services - executes our upgrade scripts - migrates databases - redeploys puppet (katello-configure) When the change in a puppet template is not compatible (e.g. some refactoring), migration fails because Katello won't boot with this (wrong) configuration since codebase was already upgraded with yum. We need to create an upgrade script that somehow corrects the configuration file until it's redeployed via katello configure. Two solutions I see: 1) Call puppet to only deploy configuration files and make this as a first step before anything happens. I have been discussing this with Foreman devs little while back, there are some features of puppet we could use for this but this is not how they should be used. Also seems to be "hacky". 2) Have full upgrade process in puppet. Very challenging, I can't really believe this is even doable - we need to deploy configuration, then execute some steps in particular order, then restart services. One big disadvantage is the upgrade process cannot be interactive anymore I believe - just fire and wait. Opinions? -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 5 15:49:12 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 5 Dec 2012 16:49:12 +0100 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <50BF5BEC.2030005@redhat.com> References: <50BF5BEC.2030005@redhat.com> Message-ID: <20121205154912.GJ2315@lzapx.brq.redhat.com> Bryan, thanks for explanation. I think the biggest issue is the performance. Let's skip monitoring part of Spacewalk and also packages contents (which can be recreated by parsing RPM packages again). What are the largest tables in Spacewalk? We need to know this and test the API-API approach. If the migration level is very high (organizations, channels, groups, templates) then it could be doable, but we need to test this on real environments. -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Wed Dec 5 15:54:03 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 05 Dec 2012 10:54:03 -0500 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <20121205154912.GJ2315@lzapx.brq.redhat.com> References: <50BF5BEC.2030005@redhat.com> <20121205154912.GJ2315@lzapx.brq.redhat.com> Message-ID: <50BF6E1B.9040902@redhat.com> On 12/05/2012 10:49 AM, Lukas Zapletal wrote: > Bryan, > > thanks for explanation. > > I think the biggest issue is the performance. > > Let's skip monitoring part of Spacewalk and also packages contents > (which can be recreated by parsing RPM packages again). What are the > largest tables in Spacewalk? We need to know this and test the API-API > approach. I am going to assume systems would be around 5-10k, and then systems to channels would be 5x that. Does that seem reasonable? > > If the migration level is very high (organizations, channels, groups, > templates) then it could be doable, but we need to test this on real > environments. Agree. is the import or export the performance concern for you? Or, both? -- bk From msuchy at redhat.com Wed Dec 5 16:10:43 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Wed, 05 Dec 2012 17:10:43 +0100 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <50BF5BEC.2030005@redhat.com> References: <50BF5BEC.2030005@redhat.com> Message-ID: <50BF7203.4010506@redhat.com> On 12/05/2012 03:36 PM, Bryan Kearney wrote: > I can not comment on a comment, so I figured I would bring them back here: > > This is about migrating from spacewalk to katello. The pattern which is > in the pull request is an ETL (Expoert/Transfrom/Load) process, where > the Extrac and Trasform are done in the export steps, and the Load is > done by the CLI in the import actions. The main comment from Lzap and > Msuchy was around using DB to DB tools as opposed to to using scripts. > The main reason for this is that today, katello is 4 data stores and > will soon be 5. When we create an activation key, it goes into katello, > elastic search, and candlepin. Systems will go into pulp, candlepin, > katello, ES, and foreman. If we do postgres to postgres, we will end up > having to to write something which says "go into the Katello DB, and > push the changes out to all the other systems". TBF, we may need that > anyway in the future. Yes it can be hard. But I seen instances where were more ten ten thousands channels (and we are importing channels as well, isn't it?) And I would not be surprised if that such huge import over API would take one week. It *can* be reasonably fast. But it *can* be slow as well. And I would bet on the second. If you give me - together with PR some preliminary data, that e.g import of 2000 packages will last 15 mins, then I will clap the hand. But if it will last 8 hours, then we will have problem. And we would have to communicate with those 5 stores no matter how hard implement it would be. Can you run some benchmarks with this first-shot script? > Some other questions: > > * How to deal with None: This is treated as an email column in the CSV/JSON > * CSV Sucks! : Yes, but the import portions can be used without the > export.. so you can create your data in $YOUR_SPREADSHEET_TOOL and then > load them up. > * The performance will suck: Could be. But I do not believe we need to > migrate over either the package manifest or the action history. I do not > think users will care about this. Why do you believe in that? :) I - as sysadmin - would expect that package manifest would be migrated. And to be precise, what is definition of package manifest? List of package which is on machine registered to Katello? > * Why do it in private: I did send out on RFC a while ago. Then figured > something running would be easier to discuss. Yes, I recall it (it was in September) with your proposal on our internal wiki. But it was never updated based on the decision. And the conclusion (at least for me) was Jan idea to use PostgreSQL dump of Spacewalk database as entry point for our import. And as we can see - using Spacewalk XMLRPC call can be tricky, because that call you used "activationkey.listActivationKeys" is available only in Spacewalk 0.6 (Sat 5.3) or higher. So it will not work in migration from older instances. -- Miroslav Suchy Red Hat Systems Management Engineering From bkearney at redhat.com Wed Dec 5 16:22:13 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 05 Dec 2012 11:22:13 -0500 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <50BF7203.4010506@redhat.com> References: <50BF5BEC.2030005@redhat.com> <50BF7203.4010506@redhat.com> Message-ID: <50BF74B5.5020505@redhat.com> On 12/05/2012 11:10 AM, Miroslav Such? wrote: > On 12/05/2012 03:36 PM, Bryan Kearney wrote: >> I can not comment on a comment, so I figured I would bring them back >> here: >> >> This is about migrating from spacewalk to katello. The pattern which is >> in the pull request is an ETL (Expoert/Transfrom/Load) process, where >> the Extrac and Trasform are done in the export steps, and the Load is >> done by the CLI in the import actions. The main comment from Lzap and >> Msuchy was around using DB to DB tools as opposed to to using scripts. >> The main reason for this is that today, katello is 4 data stores and >> will soon be 5. When we create an activation key, it goes into katello, >> elastic search, and candlepin. Systems will go into pulp, candlepin, >> katello, ES, and foreman. If we do postgres to postgres, we will end up >> having to to write something which says "go into the Katello DB, and >> push the changes out to all the other systems". TBF, we may need that >> anyway in the future. > > Yes it can be hard. > But I seen instances where were more ten ten thousands channels (and we > are importing channels as well, isn't it?) > And I would not be surprised if that such huge import over API would > take one week. > It *can* be reasonably fast. But it *can* be slow as well. And I would > bet on the second. > If you give me - together with PR some preliminary data, that e.g import > of 2000 packages will last 15 mins, then I will clap the hand. But if it > will last 8 hours, then we will have problem. And we would have to > communicate with those 5 stores no matter how hard implement it would be. > Can you run some benchmarks with this first-shot script? I will give this a shot and send it out. > >> Some other questions: >> >> * How to deal with None: This is treated as an email column in the >> CSV/JSON >> * CSV Sucks! : Yes, but the import portions can be used without the >> export.. so you can create your data in $YOUR_SPREADSHEET_TOOL and then >> load them up. >> * The performance will suck: Could be. But I do not believe we need to >> migrate over either the package manifest or the action history. I do not >> think users will care about this. > > Why do you believe in that? :) > I - as sysadmin - would expect that package manifest would be migrated. > And to be precise, what is definition of package manifest? List of > package which is on machine registered to Katello? I spoke to customers. They really only want their systems and channels moved over. As for package and action history, they are fine running the old system in report mode if they happen to have reporting requiremnts (HIPPPA orPCI) > >> * Why do it in private: I did send out on RFC a while ago. Then figured >> something running would be easier to discuss. > > Yes, I recall it (it was in September) with your proposal on our > internal wiki. But it was never updated based on the decision. > And the conclusion (at least for me) was Jan idea to use PostgreSQL dump > of Spacewalk database as entry point for our import. > > And as we can see - using Spacewalk XMLRPC call can be tricky, because > that call you used "activationkey.listActivationKeys" is available only > in Spacewalk 0.6 (Sat 5.3) or higher. So it will not work in migration > from older instances. TBF, I had assumed an upgrade to latest and then migrate. -- bk From msuchy at redhat.com Wed Dec 5 16:36:21 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Wed, 05 Dec 2012 17:36:21 +0100 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <20121205154912.GJ2315@lzapx.brq.redhat.com> References: <50BF5BEC.2030005@redhat.com> <20121205154912.GJ2315@lzapx.brq.redhat.com> Message-ID: <50BF7805.9040000@redhat.com> On 12/05/2012 04:49 PM, Lukas Zapletal wrote: > Let's skip monitoring part of Spacewalk and also packages contents > (which can be recreated by parsing RPM packages again). What are the > largest tables in Spacewalk? We need to know this and test the API-API > approach. Not ideally sorted, but here it comes from small instance: RHNCONFIGCONTENT 1.1M RHNERRATABUGLIST 1.1M RHNCHANNELPACKAGE 1.1M RHNERRATAFILECHANNEL 1.1M RHNPACKAGEFILEDELETEQUEUE 1.1M RHNCHANNELNEWESTPACKAGEAUDIT 1.1M RHNCHECKSUM 12M RHNPACKAGE 12M RHNPACKAGENAME 1.5M RHNPACKAGENEVRA 1.5M RHNSERVERHISTORY 1.5M RHNCHANNELNEWESTPACKAGE 1.5M RHNPACKAGECHANGELOGREC 18M RHNSERVERPACKAGE 18M RHNSOURCERPM 1.8M RHNPACKAGEPROVIDES 2.2M RHNPACKAGECAPABILITY 24M RHNDEVICE 2.6M RHNTASKORUN 2.6M RHNSNAPSHOTPACKAGE 36M RHNERRATAFILE 3M RHNERRATA 3M RHNPACKAGECHANGELOGDATA 4.1M RHNPACKAGEREPODATA 48M RHNPACKAGEREQUIRES 4.8M RHNPACKAGEFILE 54M + lots tables with size less then 400K Total 396M -- Miroslav Suchy Red Hat Systems Management Engineering From msuchy at redhat.com Wed Dec 5 16:39:15 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Wed, 05 Dec 2012 17:39:15 +0100 Subject: [katello-devel] Comments on https://github.com/Katello/katello/pull/1180 In-Reply-To: <50BF6E1B.9040902@redhat.com> References: <50BF5BEC.2030005@redhat.com> <20121205154912.GJ2315@lzapx.brq.redhat.com> <50BF6E1B.9040902@redhat.com> Message-ID: <50BF78B3.30900@redhat.com> On 12/05/2012 04:54 PM, Bryan Kearney wrote: > I am going to assume systems would be around 5-10k, and then systems to > channels would be 5x that. Does that seem reasonable? This is average. I think we have to count with worst case, which is IMO; 30k systems, but we can be safe with 10k channels. -- Miroslav Suchy Red Hat Systems Management Engineering From ericdhelms at gmail.com Thu Dec 6 02:20:06 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Wed, 5 Dec 2012 21:20:06 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture Message-ID: They say a picture is worth a thousands words, and after, for lack of a better word, being bugged by the differences in backend service integration I decided to try and spell out the current lay of the land. I have attempted to do this in two ways. The first being a diagram showing how our models, glue and controllers are laid out in relation to backend services with arrows indicating the directional awareness that one entity has with another. For example, an arrow pointing from glue/candlepin to resources/candlepin since glue/candlepin has direct knowledge of the resources but not vice versa. Second, I have attempted to quell out into bullet points how each backend service is currently (and by this I mean as of PulpV2 branch) integrated. Disclaimer: This is my understanding of things and there are places where I could have gotten something wrong or mis-represented that actual implementation Reference Diagram: https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit Note: This is taking into consideration the changes made in the PulpV2 branch and current Candlepin and Foreman work in master *PulpV2* - All Pulp functionality is contained within app/models/glue/pulp - All glue layers are modules that are included into the Katello model if use_pulp = true - Katello Models are treated as the main entity, with Pulp glue layer adding functionality if included - Even non-persisted objects have a top-level representation in Katello (e.g. errata, package, distribution) - Nothing is proxied - Basic functionality is orchestrated (CRUD) - Use of callbacks and hooks to trigger top-level events that can be listened to by glue layer - There is no notion of pulp outside the glue layer *Candlepin* - Contains a single Candlepin only model without corresponding inclusion in a Katello entity (OwnerInfo) - Contains some proxied functionality mainly for subscription manager - Not all Candlepin functionality is contained in glue layer - Basic functionality is orchestrated (CRUD) *Foreman* - Foreman is front and center at all layers - Controllers for API and UI are under foreman/ and namespaced - Models are under foreman/ and are classed as Foreman:: - Katello models that have corresponding Foreman model also get a separate model file besides their glue layer counterpart (i.e. models/foreman/user) - Other models have lightweight representations and are entirely pass through objects down to Foreman but are not top-level Katello entities - Glue layer is fenced off and only included if use_foreman set to true *ElasticSearch* - All ES functionality is contained within app/models/glue/elasticsearch - All glue layers are modules that included into the corresponding Katello model - Adds functionality but does not directly impart itself into the models - Use of callbacks and hooks to trigger top-level events that can be listened to by glue layer I am hoping this will spark some discussion, as I worry that the differences in integration points for each backend service will only make our lives as developers more difficult down the road. -Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From lzap at redhat.com Thu Dec 6 08:56:38 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 6 Dec 2012 09:56:38 +0100 Subject: [katello-devel] Nightly busted Message-ID: <20121206085638.GA2321@lzapx.brq.redhat.com> Hey, after my changes merged and repairs over the night, nightly does not working this morning. I hope this is the latest dependency issue. During katello-jobs startup it fails with: /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in `strict_error': Gem loading error: no such file to load -- ci/reporter/core (RuntimeError) The workaround is to delete group :ci from the Gemfile.in. I am gonna push a fix soon. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Thu Dec 6 09:35:43 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 6 Dec 2012 10:35:43 +0100 Subject: [katello-devel] Nightly busted In-Reply-To: <20121206085638.GA2321@lzapx.brq.redhat.com> References: <20121206085638.GA2321@lzapx.brq.redhat.com> Message-ID: <20121206093543.GB2321@lzapx.brq.redhat.com> Ok I have fixed that, but nightly does not work yet. There is a different issue. Not sure if this is which Eric wrote yesterday, but I am unable to fix this one. LZ On Thu, Dec 06, 2012 at 09:56:38AM +0100, Lukas Zapletal wrote: > Hey, > > after my changes merged and repairs over the night, nightly does not > working this morning. I hope this is the latest dependency issue. During > katello-jobs startup it fails with: > > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > `strict_error': Gem loading error: no such file to load -- > ci/reporter/core (RuntimeError) > > The workaround is to delete group :ci from the Gemfile.in. I am gonna > push a fix soon. > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From gkhachik at redhat.com Thu Dec 6 09:58:56 2012 From: gkhachik at redhat.com (Garik Khachikyan) Date: Thu, 06 Dec 2012 10:58:56 +0100 Subject: [katello-devel] Nightly install fail, katello-jobs log Message-ID: <50C06C60.8010707@redhat.com> Hi all, Seems we need another workaround here? katello-configure fails on katello-jobs start seems: Let me guess: workaround is: `yum -y install rubygem-redcarpet` Then rerun of: `katello-configure --deployment=katello` Garik. # cat /var/log/katello/jobs-startup.log ------------------------ /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in `strict_error': Gem loading error: no such file to load -- redcarpet (RuntimeError) from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:74:in `system_require' from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in `each' from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in `system_require' from /usr/share/katello/config/application.rb:19 from /usr/share/katello/config/environment.rb:2:in `require' from /usr/share/katello/config/environment.rb:2 from script/delayed_job:3:in `require' from script/delayed_job:3 P.S. Doing install of gem redcarpet, you might get another error: so stay tuned! --- /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in `strict_error': Gem loading error: no such file to load -- ci/reporter/core (RuntimeError) from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:74:in `system_require' from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in `each' from /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in `system_require' from /usr/share/katello/config/application.rb:19 from /usr/share/katello/config/environment.rb:2:in `require' from /usr/share/katello/config/environment.rb:2 from script/delayed_job:3:in `require' from script/delayed_job:3 From gkhachik at redhat.com Thu Dec 6 10:06:28 2012 From: gkhachik at redhat.com (Garik Khachikyan) Date: Thu, 06 Dec 2012 11:06:28 +0100 Subject: [katello-devel] Nightly install fail, katello-jobs log In-Reply-To: <50C06C60.8010707@redhat.com> References: <50C06C60.8010707@redhat.com> Message-ID: <50C06E24.2070506@redhat.com> On 06/12/12 10:58, Garik Khachikyan wrote: > Hi all, > > Seems we need another workaround here? > > katello-configure fails on katello-jobs start seems: > > Let me guess: workaround is: `yum -y install rubygem-redcarpet` > > Then rerun of: `katello-configure --deployment=katello` > > Garik. > > # cat /var/log/katello/jobs-startup.log > ------------------------ > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > `strict_error': Gem loading error: no such file to load -- redcarpet > (RuntimeError) > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:74:in > `system_require' > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in > `each' > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in > `system_require' > from /usr/share/katello/config/application.rb:19 > from /usr/share/katello/config/environment.rb:2:in `require' > from /usr/share/katello/config/environment.rb:2 > from script/delayed_job:3:in `require' > from script/delayed_job:3 > > > P.S. > > Doing install of gem redcarpet, you might get another error: so stay > tuned! > --- > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > `strict_error': Gem loading error: no such file to load -- > ci/reporter/core (RuntimeError) > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:74:in > `system_require' > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in > `each' > from > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:53:in > `system_require' > from /usr/share/katello/config/application.rb:19 > from /usr/share/katello/config/environment.rb:2:in `require' > from /usr/share/katello/config/environment.rb:2 > from script/delayed_job:3:in `require' > from script/delayed_job:3 > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel Oh, sorry. Seems duplicated issue - lzap did posted same with workaround: so :ci group removal helps. Confirm from QE side :) Garik From inecas at redhat.com Thu Dec 6 10:28:09 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 6 Dec 2012 05:28:09 -0500 (EST) Subject: [katello-devel] Nightly busted In-Reply-To: <20121206093543.GB2321@lzapx.brq.redhat.com> Message-ID: <693678045.11341921.1354789689470.JavaMail.root@redhat.com> Don't be so secret about the issue and share with us so we can compare our problems :) -- Ivan ----- Original Message ----- > Ok I have fixed that, but nightly does not work yet. There is a > different issue. Not sure if this is which Eric wrote yesterday, but > I > am unable to fix this one. > > LZ > > On Thu, Dec 06, 2012 at 09:56:38AM +0100, Lukas Zapletal wrote: > > Hey, > > > > after my changes merged and repairs over the night, nightly does > > not > > working this morning. I hope this is the latest dependency issue. > > During > > katello-jobs startup it fails with: > > > > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > > `strict_error': Gem loading error: no such file to load -- > > ci/reporter/core (RuntimeError) > > > > The workaround is to delete group :ci from the Gemfile.in. I am > > gonna > > push a fix soon. > > > > -- > > Later, > > > > Lukas "lzap" Zapletal > > #katello #systemengine > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From inecas at redhat.com Thu Dec 6 10:28:46 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 6 Dec 2012 05:28:46 -0500 (EST) Subject: [katello-devel] Nightly busted In-Reply-To: <20121206093543.GB2321@lzapx.brq.redhat.com> Message-ID: <200981027.11342163.1354789726475.JavaMail.root@redhat.com> Btw. after the fix, I was able to katello-configure successfully -- Ivan ----- Original Message ----- > Ok I have fixed that, but nightly does not work yet. There is a > different issue. Not sure if this is which Eric wrote yesterday, but > I > am unable to fix this one. > > LZ > > On Thu, Dec 06, 2012 at 09:56:38AM +0100, Lukas Zapletal wrote: > > Hey, > > > > after my changes merged and repairs over the night, nightly does > > not > > working this morning. I hope this is the latest dependency issue. > > During > > katello-jobs startup it fails with: > > > > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > > `strict_error': Gem loading error: no such file to load -- > > ci/reporter/core (RuntimeError) > > > > The workaround is to delete group :ci from the Gemfile.in. I am > > gonna > > push a fix soon. > > > > -- > > Later, > > > > Lukas "lzap" Zapletal > > #katello #systemengine > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Thu Dec 6 13:02:44 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 6 Dec 2012 14:02:44 +0100 Subject: [katello-devel] Nightly busted In-Reply-To: <200981027.11342163.1354789726475.JavaMail.root@redhat.com> References: <20121206093543.GB2321@lzapx.brq.redhat.com> <200981027.11342163.1354789726475.JavaMail.root@redhat.com> Message-ID: <20121206130244.GC2321@lzapx.brq.redhat.com> Yeah, workaround is in the master already, but I am unable to build new RPM until this fix is ready: https://github.com/Katello/katello/pull/1209 LZ On Thu, Dec 06, 2012 at 05:28:46AM -0500, Ivan Necas wrote: > Btw. after the fix, I was able to katello-configure successfully > > -- Ivan > > ----- Original Message ----- > > Ok I have fixed that, but nightly does not work yet. There is a > > different issue. Not sure if this is which Eric wrote yesterday, but > > I > > am unable to fix this one. > > > > LZ > > > > On Thu, Dec 06, 2012 at 09:56:38AM +0100, Lukas Zapletal wrote: > > > Hey, > > > > > > after my changes merged and repairs over the night, nightly does > > > not > > > working this morning. I hope this is the latest dependency issue. > > > During > > > katello-jobs startup it fails with: > > > > > > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > > > `strict_error': Gem loading error: no such file to load -- > > > ci/reporter/core (RuntimeError) > > > > > > The workaround is to delete group :ci from the Gemfile.in. I am > > > gonna > > > push a fix soon. > > > > > > -- > > > Later, > > > > > > Lukas "lzap" Zapletal > > > #katello #systemengine > > > > > > _______________________________________________ > > > katello-devel mailing list > > > katello-devel at redhat.com > > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > -- > > Later, > > > > Lukas "lzap" Zapletal > > #katello #systemengine > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Thu Dec 6 14:45:21 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 09:45:21 -0500 Subject: [katello-devel] a followup form the demo Message-ID: <50C0AF81.3050706@redhat.com> Dmitri: I did have one question. What would it take to get the fancy text editor (ace.ajax.or_ from foreman into the katello ui? From bkearney at redhat.com Thu Dec 6 14:46:29 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 09:46:29 -0500 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0AF81.3050706@redhat.com> References: <50C0AF81.3050706@redhat.com> Message-ID: <50C0AFC5.6010703@redhat.com> On 12/06/2012 09:45 AM, Bryan Kearney wrote: > Dmitri: > > I did have one question. What would it take to get the fancy text editor > (ace.ajax.or_ from foreman into the katello ui? > sorry ace.ajax.org -- bk From dmitri at redhat.com Thu Dec 6 15:30:40 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Thu, 06 Dec 2012 15:30:40 +0000 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0AFC5.6010703@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> Message-ID: <50C0BA20.5080600@redhat.com> On 06/12/12 02:46 PM, Bryan Kearney wrote: > On 12/06/2012 09:45 AM, Bryan Kearney wrote: >> Dmitri: >> >> I did have one question. What would it take to get the fancy text editor >> (ace.ajax.or_ from foreman into the katello ui? >> > > > sorry ace.ajax.org > > -- bk > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel Good question. Lemme look into this. -d From lzap at redhat.com Thu Dec 6 15:37:02 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 6 Dec 2012 16:37:02 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: References: Message-ID: <20121206153702.GH2321@lzapx.brq.redhat.com> Quick look on the picture shows one thing - folks who were implementing foreman layer took different approach. I'd like to see them to organize deep dive showing us how it works, what is/was the motivation, so we can have further discussion about which approach is better. It's definitely not good to have different approaches for the same things. The picture shows this pretty nicely. ps - why there are arrows from foreman controllers and API to Candlepin? do I miss something for this particular case? Thanks for this, Eric! LZ On Wed, Dec 05, 2012 at 09:20:06PM -0500, Eric D Helms wrote: > They say a picture is worth a thousands words, and after, for lack of a > better word, being bugged by the differences in backend service integration > I decided to try and spell out the current lay of the land. I have > attempted to do this in two ways. The first being a diagram showing how > our models, glue and controllers are laid out in relation to backend > services with arrows indicating the directional awareness that one entity > has with another. For example, an arrow pointing from glue/candlepin to > resources/candlepin since glue/candlepin has direct knowledge of the > resources but not vice versa. Second, I have attempted to quell out into > bullet points how each backend service is currently (and by this I mean as > of PulpV2 branch) integrated. > > Disclaimer: This is my understanding of things and there are places where > I could have gotten something wrong or mis-represented that actual > implementation > > Reference Diagram: > https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > > Note: This is taking into consideration the changes made in the PulpV2 > branch and current Candlepin and Foreman work in master > > > *PulpV2* > > - All Pulp functionality is contained within app/models/glue/pulp > - All glue layers are modules that are included into the Katello model if > use_pulp = true > - Katello Models are treated as the main entity, with Pulp glue layer > adding functionality if included > - Even non-persisted objects have a top-level representation in Katello > (e.g. errata, package, distribution) > - Nothing is proxied > - Basic functionality is orchestrated (CRUD) > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > - There is no notion of pulp outside the glue layer > > > *Candlepin* > > - Contains a single Candlepin only model without corresponding inclusion in > a Katello entity (OwnerInfo) > - Contains some proxied functionality mainly for subscription manager > - Not all Candlepin functionality is contained in glue layer > - Basic functionality is orchestrated (CRUD) > > > *Foreman* > > - Foreman is front and center at all layers > - Controllers for API and UI are under foreman/ and namespaced > - Models are under foreman/ and are classed as Foreman:: > - Katello models that have corresponding Foreman model also get a separate > model file besides their glue layer counterpart (i.e. models/foreman/user) > - Other models have lightweight representations and are entirely pass > through objects down to Foreman but are not top-level Katello entities > - Glue layer is fenced off and only included if use_foreman set to true > > > *ElasticSearch* > > - All ES functionality is contained within app/models/glue/elasticsearch > - All glue layers are modules that included into the corresponding Katello > model > - Adds functionality but does not directly impart itself into the models > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > > > I am hoping this will spark some discussion, as I worry that the > differences in integration points for each backend service will only make > our lives as developers more difficult down the road. > > -Eric > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From thomasmckay at redhat.com Thu Dec 6 15:45:23 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Thu, 6 Dec 2012 10:45:23 -0500 (EST) Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: Message-ID: <1129335304.14876517.1354808723127.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Eric D Helms" > To: katello-devel at redhat.com > Sent: Wednesday, December 5, 2012 9:20:06 PM > Subject: [katello-devel] Katello Backend Service Relationships and Architecture > > > They say a picture is worth a thousands words, and after, for lack of > a better word, being bugged by the differences in backend service > integration I decided to try and spell out the current lay of the > land. I have attempted to do this in two ways. The first being a > diagram showing how our models, glue and controllers are laid out in > relation to backend services with arrows indicating the directional > awareness that one entity has with another. For example, an arrow > pointing from glue/candlepin to resources/candlepin since > glue/candlepin has direct knowledge of the resources but not vice > versa. Second, I have attempted to quell out into bullet points how > each backend service is currently (and by this I mean as of PulpV2 > branch) integrated. > > > Disclaimer: This is my understanding of things and there are places > where I could have gotten something wrong or mis-represented that > actual implementation > > > > Reference Diagram: > https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > > > Note: This is taking into consideration the changes made in the > PulpV2 branch and current Candlepin and Foreman work in master > > > > > PulpV2 > > > - All Pulp functionality is contained within app/models/glue/pulp > - All glue layers are modules that are included into the Katello > model if use_pulp = true > - Katello Models are treated as the main entity, with Pulp glue layer > adding functionality if included > - Even non-persisted objects have a top-level representation in > Katello (e.g. errata, package, distribution) > - Nothing is proxied > - Basic functionality is orchestrated (CRUD) > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > - There is no notion of pulp outside the glue layer > > > > > Candlepin > > > - Contains a single Candlepin only model without corresponding > inclusion in a Katello entity (OwnerInfo) > - Contains some proxied functionality mainly for subscription manager > - Not all Candlepin functionality is contained in glue layer > - Basic functionality is orchestrated (CRUD) > Definitely room for clean-up in the candlepin layer. I would love to see us move to align with the clean pulp v2 integration style. > > > > Foreman > > > - Foreman is front and center at all layers > - Controllers for API and UI are under foreman/ and namespaced > - Models are under foreman/ and are classed as Foreman:: > - Katello models that have corresponding Foreman model also get a > separate model file besides their glue layer counterpart (i.e. > models/foreman/user) > - Other models have lightweight representations and are entirely pass > through objects down to Foreman but are not top-level Katello > entities > - Glue layer is fenced off and only included if use_foreman set to > true > > > > > ElasticSearch > > > - All ES functionality is contained within > app/models/glue/elasticsearch > - All glue layers are modules that included into the corresponding > Katello model > - Adds functionality but does not directly impart itself into the > models > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > > > > > I am hoping this will spark some discussion, as I worry that the > differences in integration points for each backend service will only > make our lives as developers more difficult down the road. > > > -Eric > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ericdhelms at gmail.com Thu Dec 6 15:45:54 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Thu, 6 Dec 2012 10:45:54 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <20121206153702.GH2321@lzapx.brq.redhat.com> References: <20121206153702.GH2321@lzapx.brq.redhat.com> Message-ID: On Thu, Dec 6, 2012 at 10:37 AM, Lukas Zapletal wrote: > Quick look on the picture shows one thing - folks who were implementing > foreman layer took different approach. I'd like to see them to organize > deep dive showing us how it works, what is/was the motivation, so we can > have further discussion about which approach is better. > > It's definitely not good to have different approaches for the same > things. The picture shows this pretty nicely. > > ps - why there are arrows from foreman controllers and API to Candlepin? > do I miss something for this particular case? > The boxes reflect sort of the "filesystem" and "class/module" structure as well as knowledge relationships. So in this case, the "UI" controllers have direct references to the Resources as well as the API controllers for Candlepin. The foreman box inside the controllers might just be too close to the arrow anchor point. -Eric > Thanks for this, Eric! > > LZ > > On Wed, Dec 05, 2012 at 09:20:06PM -0500, Eric D Helms wrote: > > They say a picture is worth a thousands words, and after, for lack of a > > better word, being bugged by the differences in backend service > integration > > I decided to try and spell out the current lay of the land. I have > > attempted to do this in two ways. The first being a diagram showing how > > our models, glue and controllers are laid out in relation to backend > > services with arrows indicating the directional awareness that one entity > > has with another. For example, an arrow pointing from glue/candlepin to > > resources/candlepin since glue/candlepin has direct knowledge of the > > resources but not vice versa. Second, I have attempted to quell out into > > bullet points how each backend service is currently (and by this I mean > as > > of PulpV2 branch) integrated. > > > > Disclaimer: This is my understanding of things and there are places > where > > I could have gotten something wrong or mis-represented that actual > > implementation > > > > Reference Diagram: > > > https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > > > > Note: This is taking into consideration the changes made in the PulpV2 > > branch and current Candlepin and Foreman work in master > > > > > > *PulpV2* > > > > - All Pulp functionality is contained within app/models/glue/pulp > > - All glue layers are modules that are included into the Katello model if > > use_pulp = true > > - Katello Models are treated as the main entity, with Pulp glue layer > > adding functionality if included > > - Even non-persisted objects have a top-level representation in Katello > > (e.g. errata, package, distribution) > > - Nothing is proxied > > - Basic functionality is orchestrated (CRUD) > > - Use of callbacks and hooks to trigger top-level events that can be > > listened to by glue layer > > - There is no notion of pulp outside the glue layer > > > > > > *Candlepin* > > > > - Contains a single Candlepin only model without corresponding inclusion > in > > a Katello entity (OwnerInfo) > > - Contains some proxied functionality mainly for subscription manager > > - Not all Candlepin functionality is contained in glue layer > > - Basic functionality is orchestrated (CRUD) > > > > > > *Foreman* > > > > - Foreman is front and center at all layers > > - Controllers for API and UI are under foreman/ and namespaced > > - Models are under foreman/ and are classed as Foreman:: > > - Katello models that have corresponding Foreman model also get a > separate > > model file besides their glue layer counterpart (i.e. > models/foreman/user) > > - Other models have lightweight representations and are entirely pass > > through objects down to Foreman but are not top-level Katello entities > > - Glue layer is fenced off and only included if use_foreman set to true > > > > > > *ElasticSearch* > > > > - All ES functionality is contained within app/models/glue/elasticsearch > > - All glue layers are modules that included into the corresponding > Katello > > model > > - Adds functionality but does not directly impart itself into the models > > - Use of callbacks and hooks to trigger top-level events that can be > > listened to by glue layer > > > > > > I am hoping this will spark some discussion, as I worry that the > > differences in integration points for each backend service will only make > > our lives as developers more difficult down the road. > > > > -Eric > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paji at redhat.com Thu Dec 6 15:54:31 2012 From: paji at redhat.com (Partha Aji) Date: Thu, 6 Dec 2012 10:54:31 -0500 (EST) Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <1129335304.14876517.1354808723127.JavaMail.root@redhat.com> Message-ID: <1322241480.4852577.1354809271778.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Tom McKay" > To: "Eric D Helms" > Cc: katello-devel at redhat.com > Sent: Thursday, December 6, 2012 10:45:23 AM > Subject: Re: [katello-devel] Katello Backend Service Relationships and Architecture > > > Candlepin > > > > > > - Contains a single Candlepin only model without corresponding > > inclusion in a Katello entity (OwnerInfo) > > - Contains some proxied functionality mainly for subscription > > manager > > - Not all Candlepin functionality is contained in glue layer > > - Basic functionality is orchestrated (CRUD) > > > > Definitely room for clean-up in the candlepin layer. I would love to > see us move to align with the clean pulp v2 integration style. > Candlepin used to have something similar to Runcible, a wrapper library for its calls. We just didn't use em for unknown reasons (may be it was deemed that calling the Rest API directly via url was just as simple as using a wrapper library). Partha From dmitri at redhat.com Thu Dec 6 16:03:17 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Thu, 06 Dec 2012 16:03:17 +0000 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0AFC5.6010703@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> Message-ID: <50C0C1C5.7090101@redhat.com> On 06/12/12 02:46 PM, Bryan Kearney wrote: > On 12/06/2012 09:45 AM, Bryan Kearney wrote: >> Dmitri: >> >> I did have one question. What would it take to get the fancy text editor >> (ace.ajax.or_ from foreman into the katello ui? >> > > > sorry ace.ajax.org > > -- bk > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel I think adding ace editor on the "new" screen should be pretty trivial. Show/modify might present an issue - since we somehow need to integrate with/mimic jeditable behaviour. How important is this? -d From bkearney at redhat.com Thu Dec 6 16:08:42 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 11:08:42 -0500 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0C1C5.7090101@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> <50C0C1C5.7090101@redhat.com> Message-ID: <50C0C30A.3050806@redhat.com> On 12/06/2012 11:03 AM, Dmitri Dolguikh wrote: > On 06/12/12 02:46 PM, Bryan Kearney wrote: >> On 12/06/2012 09:45 AM, Bryan Kearney wrote: >>> Dmitri: >>> >>> I did have one question. What would it take to get the fancy text editor >>> (ace.ajax.or_ from foreman into the katello ui? >>> >> >> >> sorry ace.ajax.org >> >> -- bk >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > I think adding ace editor on the "new" screen should be pretty trivial. > Show/modify might present an issue - since we somehow need to integrate > with/mimic jeditable behaviour. Do we need to mimic it? > > How important is this? Not major, I notice that a big template does not look great. But, I would prefer features over skinning now. -- bk From dmitri at redhat.com Thu Dec 6 16:09:34 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Thu, 06 Dec 2012 16:09:34 +0000 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0C30A.3050806@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> <50C0C1C5.7090101@redhat.com> <50C0C30A.3050806@redhat.com> Message-ID: <50C0C33E.8090101@redhat.com> On 06/12/12 04:08 PM, Bryan Kearney wrote: > On 12/06/2012 11:03 AM, Dmitri Dolguikh wrote: >> On 06/12/12 02:46 PM, Bryan Kearney wrote: >>> On 12/06/2012 09:45 AM, Bryan Kearney wrote: >>>> Dmitri: >>>> >>>> I did have one question. What would it take to get the fancy text >>>> editor >>>> (ace.ajax.or_ from foreman into the katello ui? >>>> >>> >>> >>> sorry ace.ajax.org >>> >>> -- bk >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> I think adding ace editor on the "new" screen should be pretty trivial. >> Show/modify might present an issue - since we somehow need to integrate >> with/mimic jeditable behaviour. > > Do we need to mimic it? if not the look, then behaviour: we need to send updates on focus change/timeout or something like that. > >> >> How important is this? > > Not major, I notice that a big template does not look great. But, I > would prefer features over skinning now. Nods. Let's put it on the backlog? Perhaps mark it as "nice-to-have" feature? -d > > -- bk > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From daviddavis at redhat.com Thu Dec 6 16:54:46 2012 From: daviddavis at redhat.com (David Davis) Date: Thu, 6 Dec 2012 11:54:46 -0500 (EST) Subject: [katello-devel] Multitenancy Message-ID: <1547208147.42088546.1354812886292.JavaMail.root@redhat.com> I watched the recent Railscasts videos on multitenancy and they were quite interesting. For the PostgreSQL one, Bates uses schemas and search paths in postgres to separate out organizations' data. He also uses a shared schema to share data between organizations. They're worth a watch if you have a Railscasts Pro account. http://railscasts.com/episodes/389-multitenancy-with-postgresql http://railscasts.com/episodes/388-multitenancy-with-scopes David From ericdhelms at gmail.com Thu Dec 6 16:58:28 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Thu, 6 Dec 2012 11:58:28 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning Message-ID: Howdy, As I prepare the Katello 1.2 release, I thought I would get the ball rolling on feature planning for Katello 1.3. As it stands, the focus of Katello 1.3 is Pulp V2 integration. However, a lot of other features and work will go on during now and when the Pulp V2 work is ready. So I'd like to get a rough idea of what features we plan to try and get into this next release and potentially any beyond that for the next release nanny to work from. Current rough time frame for Pulp V2 integration is late January, early February time frame and thus this release will be based on feature completion not a hard timed release. Katello 1.3 Features - Pulp V2 integration - Pulp V2 upgrades Katello 1.4 Features -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasmckay at redhat.com Thu Dec 6 17:00:47 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Thu, 6 Dec 2012 12:00:47 -0500 (EST) Subject: [katello-devel] translation of model validation messages Message-ID: <597672650.14925253.1354813247881.JavaMail.root@redhat.com> FYI, many of the validation messages built into the models are not translated. This should be a high priority item in the next iteration. http://guides.rubyonrails.org/i18n.html#translations-for-active-record-models Example when creating an org w/o filling in any values: ?????????: Label can't be blank Label cannot contain characters other than ascii alpha numerals, '_', '-'. Label must contain at least 1 character Label can't be blank Name can't be blank Name must contain at least 1 character From daviddavis at redhat.com Thu Dec 6 17:02:42 2012 From: daviddavis at redhat.com (David Davis) Date: Thu, 6 Dec 2012 12:02:42 -0500 (EST) Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: Message-ID: <255040738.42092279.1354813362963.JavaMail.root@redhat.com> Just out of curiosity, what release are we targeting for content views? Thanks. David ----- Original Message ----- > From: "Eric D Helms" > To: katello-devel at redhat.com > Sent: Thursday, December 6, 2012 11:58:28 AM > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > Howdy, > > As I prepare the Katello 1.2 release, I thought I would get the ball > rolling on feature planning for Katello 1.3. As it stands, the focus > of > Katello 1.3 is Pulp V2 integration. However, a lot of other features > and > work will go on during now and when the Pulp V2 work is ready. So > I'd like > to get a rough idea of what features we plan to try and get into this > next > release and potentially any beyond that for the next release nanny to > work > from. Current rough time frame for Pulp V2 integration is late > January, > early February time frame and thus this release will be based on > feature > completion not a hard timed release. > > Katello 1.3 Features > - Pulp V2 integration > - Pulp V2 upgrades > > > Katello 1.4 Features > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ericdhelms at gmail.com Thu Dec 6 17:04:51 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Thu, 6 Dec 2012 12:04:51 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: <255040738.42092279.1354813362963.JavaMail.root@redhat.com> References: <255040738.42092279.1354813362963.JavaMail.root@redhat.com> Message-ID: On Thu, Dec 6, 2012 at 12:02 PM, David Davis wrote: > Just out of curiosity, what release are we targeting for content views? > Do you have any thoughts around estimated completion time for them (even if a minimally viable version) ? If they can go in when Pulp V2 goes in, I say we target the 1.3 release to include both. -Eric > Thanks. > > David > > ----- Original Message ----- > > From: "Eric D Helms" > > To: katello-devel at redhat.com > > Sent: Thursday, December 6, 2012 11:58:28 AM > > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > > > Howdy, > > > > As I prepare the Katello 1.2 release, I thought I would get the ball > > rolling on feature planning for Katello 1.3. As it stands, the focus > > of > > Katello 1.3 is Pulp V2 integration. However, a lot of other features > > and > > work will go on during now and when the Pulp V2 work is ready. So > > I'd like > > to get a rough idea of what features we plan to try and get into this > > next > > release and potentially any beyond that for the next release nanny to > > work > > from. Current rough time frame for Pulp V2 integration is late > > January, > > early February time frame and thus this release will be based on > > feature > > completion not a hard timed release. > > > > Katello 1.3 Features > > - Pulp V2 integration > > - Pulp V2 upgrades > > > > > > Katello 1.4 Features > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lzap at redhat.com Thu Dec 6 17:16:29 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 6 Dec 2012 18:16:29 +0100 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: References: Message-ID: <20121206171629.GI2321@lzapx.brq.redhat.com> Better foreman support with initial provisioning? LZ On Thu, Dec 06, 2012 at 11:58:28AM -0500, Eric D Helms wrote: > Howdy, > > As I prepare the Katello 1.2 release, I thought I would get the ball > rolling on feature planning for Katello 1.3. As it stands, the focus of > Katello 1.3 is Pulp V2 integration. However, a lot of other features and > work will go on during now and when the Pulp V2 work is ready. So I'd like > to get a rough idea of what features we plan to try and get into this next > release and potentially any beyond that for the next release nanny to work > from. Current rough time frame for Pulp V2 integration is late January, > early February time frame and thus this release will be based on feature > completion not a hard timed release. > > Katello 1.3 Features > - Pulp V2 integration > - Pulp V2 upgrades > > > Katello 1.4 Features > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Thu Dec 6 17:19:17 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 12:19:17 -0500 Subject: [katello-devel] translation of model validation messages In-Reply-To: <597672650.14925253.1354813247881.JavaMail.root@redhat.com> References: <597672650.14925253.1354813247881.JavaMail.root@redhat.com> Message-ID: <50C0D395.3000404@redhat.com> On 12/06/2012 12:00 PM, Tom McKay wrote: > FYI, many of the validation messages built into the models are not translated. This should be a high priority item in the next iteration. > > http://guides.rubyonrails.org/i18n.html#translations-for-active-record-models > > Example when creating an org w/o filling in any values: > > ?????????: > Label can't be blank > Label cannot contain characters other than ascii alpha numerals, '_', '-'. > Label must contain at least 1 character > Label can't be blank > Name can't be blank > Name must contain at least 1 character > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > When does rails give us better l10n friendly validators. The current model is very English Specific... I think the code is: [LABEL NAME] + [SPACE] + [YOUR ERROR MESSAGE] Which can be bad for German and folks who talk like Yoda. -- bk From daviddavis at redhat.com Thu Dec 6 17:29:04 2012 From: daviddavis at redhat.com (David Davis) Date: Thu, 6 Dec 2012 12:29:04 -0500 (EST) Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: Message-ID: <1533735270.42103295.1354814944561.JavaMail.root@redhat.com> I'm not sure of everybody's Christmas plans but at the rate we're going, I think that at the very least we could have the core of content views (ie an MVP) done by mid to late January. Brad, Justin, and Mike does that sound realistic? David ----- Original Message ----- > From: "Eric D Helms" > To: "David Davis" > Cc: katello-devel at redhat.com > Sent: Thursday, December 6, 2012 12:04:51 PM > Subject: Re: [katello-devel] Katello 1.3/1.4 Feature Planning > > > > > > > On Thu, Dec 6, 2012 at 12:02 PM, David Davis < daviddavis at redhat.com > > wrote: > > > Just out of curiosity, what release are we targeting for content > views? > > > > Do you have any thoughts around estimated completion time for them > (even if a minimally viable version) ? If they can go in when Pulp > V2 goes in, I say we target the 1.3 release to include both. > > > -Eric > > > Thanks. > > David > > > > ----- Original Message ----- > > From: "Eric D Helms" < ericdhelms at gmail.com > > > To: katello-devel at redhat.com > > Sent: Thursday, December 6, 2012 11:58:28 AM > > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > > > Howdy, > > > > As I prepare the Katello 1.2 release, I thought I would get the > > ball > > rolling on feature planning for Katello 1.3. As it stands, the > > focus > > of > > Katello 1.3 is Pulp V2 integration. However, a lot of other > > features > > and > > work will go on during now and when the Pulp V2 work is ready. So > > I'd like > > to get a rough idea of what features we plan to try and get into > > this > > next > > release and potentially any beyond that for the next release nanny > > to > > work > > from. Current rough time frame for Pulp V2 integration is late > > January, > > early February time frame and thus this release will be based on > > feature > > completion not a hard timed release. > > > > Katello 1.3 Features > > - Pulp V2 integration > > - Pulp V2 upgrades > > > > > > Katello 1.4 Features > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > From bkearney at redhat.com Thu Dec 6 17:33:45 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 12:33:45 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: <1533735270.42103295.1354814944561.JavaMail.root@redhat.com> References: <1533735270.42103295.1354814944561.JavaMail.root@redhat.com> Message-ID: <50C0D6F9.8010507@redhat.com> Some things I would add (if possible): + New Nav + Tupane Improvements + Custom info via the GUI + Pulp V2 And then for 1.4 + Mod Passenger + Errata Notifications -- bk On 12/06/2012 12:29 PM, David Davis wrote: > I'm not sure of everybody's Christmas plans but at the rate we're going, I think that at the very least we could have the core of content views (ie an MVP) done by mid to late January. > > Brad, Justin, and Mike does that sound realistic? > > David > > ----- Original Message ----- >> From: "Eric D Helms" >> To: "David Davis" >> Cc: katello-devel at redhat.com >> Sent: Thursday, December 6, 2012 12:04:51 PM >> Subject: Re: [katello-devel] Katello 1.3/1.4 Feature Planning >> >> >> >> >> >> >> On Thu, Dec 6, 2012 at 12:02 PM, David Davis < daviddavis at redhat.com >>> wrote: >> >> >> Just out of curiosity, what release are we targeting for content >> views? >> >> >> >> Do you have any thoughts around estimated completion time for them >> (even if a minimally viable version) ? If they can go in when Pulp >> V2 goes in, I say we target the 1.3 release to include both. >> >> >> -Eric >> >> >> Thanks. >> >> David >> >> >> >> ----- Original Message ----- >>> From: "Eric D Helms" < ericdhelms at gmail.com > >>> To: katello-devel at redhat.com >>> Sent: Thursday, December 6, 2012 11:58:28 AM >>> Subject: [katello-devel] Katello 1.3/1.4 Feature Planning >>> >>> Howdy, >>> >>> As I prepare the Katello 1.2 release, I thought I would get the >>> ball >>> rolling on feature planning for Katello 1.3. As it stands, the >>> focus >>> of >>> Katello 1.3 is Pulp V2 integration. However, a lot of other >>> features >>> and >>> work will go on during now and when the Pulp V2 work is ready. So >>> I'd like >>> to get a rough idea of what features we plan to try and get into >>> this >>> next >>> release and potentially any beyond that for the next release nanny >>> to >>> work >>> from. Current rough time frame for Pulp V2 integration is late >>> January, >>> early February time frame and thus this release will be based on >>> feature >>> completion not a hard timed release. >>> >>> Katello 1.3 Features >>> - Pulp V2 integration >>> - Pulp V2 upgrades >>> >>> >>> Katello 1.4 Features >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From thomasmckay at redhat.com Thu Dec 6 17:47:03 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Thu, 6 Dec 2012 12:47:03 -0500 (EST) Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: Message-ID: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Eric D Helms" > To: katello-devel at redhat.com > Sent: Thursday, December 6, 2012 11:58:28 AM > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > > Howdy, > > > As I prepare the Katello 1.2 release, I thought I would get the ball > rolling on feature planning for Katello 1.3. As it stands, the focus > of Katello 1.3 is Pulp V2 integration. However, a lot of other > features and work will go on during now and when the Pulp V2 work is > ready. So I'd like to get a rough idea of what features we plan to > try and get into this next release and potentially any beyond that > for the next release nanny to work from. Current rough time frame > for Pulp V2 integration is late January, early February time frame > and thus this release will be based on feature completion not a hard > timed release. > > > Katello 1.3 Features > - Pulp V2 integration > - Pulp V2 upgrades > > > > > Katello 1.4 Features > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel Reduction of the 564 open BZs. Seriously, if we're not going to fix them let's just mass close. https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 From bkearney at redhat.com Thu Dec 6 18:04:57 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 13:04:57 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: References: Message-ID: <50C0DE49.9070903@redhat.com> On 12/05/2012 09:20 PM, Eric D Helms wrote: > They say a picture is worth a thousands words, and after, for lack of a > better word, being bugged by the differences in backend service > integration I decided to try and spell out the current lay of the land. > I have attempted to do this in two ways. The first being a diagram > showing how our models, glue and controllers are laid out in relation to > backend services with arrows indicating the directional awareness that > one entity has with another. For example, an arrow pointing from > glue/candlepin to resources/candlepin since glue/candlepin has direct > knowledge of the resources but not vice versa. Second, I have attempted > to quell out into bullet points how each backend service is currently > (and by this I mean as of PulpV2 branch) integrated. > > Disclaimer: This is my understanding of things and there are places > where I could have gotten something wrong or mis-represented that actual > implementation > > Reference Diagram: > https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > > Note: This is taking into consideration the changes made in the PulpV2 > branch and current Candlepin and Foreman work in master > > > *PulpV2* > > - All Pulp functionality is contained within app/models/glue/pulp > - All glue layers are modules that are included into the Katello model > if use_pulp = true > - Katello Models are treated as the main entity, with Pulp glue layer > adding functionality if included > - Even non-persisted objects have a top-level representation in Katello > (e.g. errata, package, distribution) > - Nothing is proxied > - Basic functionality is orchestrated (CRUD) > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > - There is no notion of pulp outside the glue layer > > > *Candlepin* > > - Contains a single Candlepin only model without corresponding inclusion > in a Katello entity (OwnerInfo) > - Contains some proxied functionality mainly for subscription manager > - Not all Candlepin functionality is contained in glue layer > - Basic functionality is orchestrated (CRUD) > > > *Foreman* > > - Foreman is front and center at all layers > - Controllers for API and UI are under foreman/ and namespaced > - Models are under foreman/ and are classed as Foreman:: > - Katello models that have corresponding Foreman model also get a > separate model file besides their glue layer counterpart (i.e. > models/foreman/user) > - Other models have lightweight representations and are entirely pass > through objects down to Foreman but are not top-level Katello entities > - Glue layer is fenced off and only included if use_foreman set to true > > > *ElasticSearch* > > - All ES functionality is contained within app/models/glue/elasticsearch > - All glue layers are modules that included into the corresponding > Katello model > - Adds functionality but does not directly impart itself into the models > - Use of callbacks and hooks to trigger top-level events that can be > listened to by glue layer > > > I am hoping this will spark some discussion, as I worry that the > differences in integration points for each backend service will only > make our lives as developers more difficult down the road. > > -Eric > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > I agree with the Deep Dive. Dmitri had expressed similar concerns. Who from ? wants to set this up? -- bk From bbuckingham at redhat.com Thu Dec 6 18:41:21 2012 From: bbuckingham at redhat.com (Brad Buckingham) Date: Thu, 06 Dec 2012 13:41:21 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: <1533735270.42103295.1354814944561.JavaMail.root@redhat.com> References: <1533735270.42103295.1354814944561.JavaMail.root@redhat.com> Message-ID: <50C0E6D1.9030507@redhat.com> IMO we should be able to have some "usable" content view functionality available by the mid-to-late January timeframe. For example, the ability to create definitions, publish views from those definitions, promote/delete content via changesets as well as register/consume content from a view using a client. There is a lot of work to be done to reach that point, but I'd like us to work towards it. We can re-evaluate if it is achievable after we complete the next sprint/iteration. cheers, Brad On 12/06/2012 12:29 PM, David Davis wrote: > I'm not sure of everybody's Christmas plans but at the rate we're going, I think that at the very least we could have the core of content views (ie an MVP) done by mid to late January. > > Brad, Justin, and Mike does that sound realistic? > > David > > ----- Original Message ----- >> From: "Eric D Helms" >> To: "David Davis" >> Cc: katello-devel at redhat.com >> Sent: Thursday, December 6, 2012 12:04:51 PM >> Subject: Re: [katello-devel] Katello 1.3/1.4 Feature Planning >> >> >> >> >> >> >> On Thu, Dec 6, 2012 at 12:02 PM, David Davis < daviddavis at redhat.com >>> wrote: >> >> Just out of curiosity, what release are we targeting for content >> views? >> >> >> >> Do you have any thoughts around estimated completion time for them >> (even if a minimally viable version) ? If they can go in when Pulp >> V2 goes in, I say we target the 1.3 release to include both. >> >> >> -Eric >> >> >> Thanks. >> >> David >> >> >> >> ----- Original Message ----- >>> From: "Eric D Helms" < ericdhelms at gmail.com > >>> To: katello-devel at redhat.com >>> Sent: Thursday, December 6, 2012 11:58:28 AM >>> Subject: [katello-devel] Katello 1.3/1.4 Feature Planning >>> >>> Howdy, >>> >>> As I prepare the Katello 1.2 release, I thought I would get the >>> ball >>> rolling on feature planning for Katello 1.3. As it stands, the >>> focus >>> of >>> Katello 1.3 is Pulp V2 integration. However, a lot of other >>> features >>> and >>> work will go on during now and when the Pulp V2 work is ready. So >>> I'd like >>> to get a rough idea of what features we plan to try and get into >>> this >>> next >>> release and potentially any beyond that for the next release nanny >>> to >>> work >>> from. Current rough time frame for Pulp V2 integration is late >>> January, >>> early February time frame and thus this release will be based on >>> feature >>> completion not a hard timed release. >>> >>> Katello 1.3 Features >>> - Pulp V2 integration >>> - Pulp V2 upgrades >>> >>> >>> Katello 1.4 Features >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ericdhelms at gmail.com Thu Dec 6 18:55:32 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Thu, 6 Dec 2012 13:55:32 -0500 Subject: [katello-devel] Fixing Issues with new Submodule Message-ID: As part of the switch to Alchemy (formerly converge-ui) some path updates for the submodule were made. For older git checkouts, you may get some errors related to attempting to still checkout converge-ui. The following should resolve any issues: 1. Delete the relevant section from .git/config. 2. Run git rm --cached path_to_submodule (no trailing slash). - Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericdhelms at gmail.com Thu Dec 6 19:04:21 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Thu, 6 Dec 2012 14:04:21 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> References: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> Message-ID: I think this would be a great time for a bug-day and am going to organize such. Bug day being, whole team sitting down and pounding through the list of bugs to verify, ask for more information or try and fix. I figure we can get a conference room for a day (in the U.S.) to allow easy discussion since inevitably some people have more information than others in some areas. Team Other side of the Pond -- since the time zone difference is great enough, if you guys would also like to put together a bug day around the same time and if someone from your side would like to volunteer the organization of it that would be great. - Eric On Thu, Dec 6, 2012 at 12:47 PM, Tom McKay wrote: > > > ----- Original Message ----- > > From: "Eric D Helms" > > To: katello-devel at redhat.com > > Sent: Thursday, December 6, 2012 11:58:28 AM > > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > > > > > Howdy, > > > > > > As I prepare the Katello 1.2 release, I thought I would get the ball > > rolling on feature planning for Katello 1.3. As it stands, the focus > > of Katello 1.3 is Pulp V2 integration. However, a lot of other > > features and work will go on during now and when the Pulp V2 work is > > ready. So I'd like to get a rough idea of what features we plan to > > try and get into this next release and potentially any beyond that > > for the next release nanny to work from. Current rough time frame > > for Pulp V2 integration is late January, early February time frame > > and thus this release will be based on feature completion not a hard > > timed release. > > > > > > Katello 1.3 Features > > - Pulp V2 integration > > - Pulp V2 upgrades > > > > > > > > > > Katello 1.4 Features > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > Reduction of the 564 open BZs. Seriously, if we're not going to fix them > let's just mass close. > > > https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkearney at redhat.com Thu Dec 6 19:08:16 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 14:08:16 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: References: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> Message-ID: <50C0ED20.5050409@redhat.com> On 12/06/2012 02:04 PM, Eric D Helms wrote: > I think this would be a great time for a bug-day and am going to > organize such. Bug day being, whole team sitting down and pounding > through the list of bugs to verify, ask for more information or try and > fix. I figure we can get a conference room for a day (in the U.S.) to > allow easy discussion since inevitably some people have more information > than others in some areas. And turn bugs into RFE's so we can clean up the backlog. > > Team Other side of the Pond -- since the time zone difference is great > enough, if you guys would also like to put together a bug day around the > same time and if someone from your side would like to volunteer the > organization of it that would be great. +1 to a single day for both sides.. a 14 hour bug day would be great. -- bk From bbuckingham at redhat.com Thu Dec 6 19:07:53 2012 From: bbuckingham at redhat.com (Brad Buckingham) Date: Thu, 06 Dec 2012 14:07:53 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: References: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> Message-ID: <50C0ED09.6090505@redhat.com> Another thought which might be nice... How about possibly a documentation day (or partial day)? This could be for supporting many forms of documentation (blogs, user docs, manpages (if any needed), screencasts...). Brad On 12/06/2012 02:04 PM, Eric D Helms wrote: > I think this would be a great time for a bug-day and am going to > organize such. Bug day being, whole team sitting down and pounding > through the list of bugs to verify, ask for more information or try > and fix. I figure we can get a conference room for a day (in the > U.S.) to allow easy discussion since inevitably some people have more > information than others in some areas. > > Team Other side of the Pond -- since the time zone difference is great > enough, if you guys would also like to put together a bug day around > the same time and if someone from your side would like to volunteer > the organization of it that would be great. > > - Eric > > > On Thu, Dec 6, 2012 at 12:47 PM, Tom McKay > wrote: > > > > ----- Original Message ----- > > From: "Eric D Helms" > > > To: katello-devel at redhat.com > > Sent: Thursday, December 6, 2012 11:58:28 AM > > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning > > > > > > Howdy, > > > > > > As I prepare the Katello 1.2 release, I thought I would get the ball > > rolling on feature planning for Katello 1.3. As it stands, the focus > > of Katello 1.3 is Pulp V2 integration. However, a lot of other > > features and work will go on during now and when the Pulp V2 work is > > ready. So I'd like to get a rough idea of what features we plan to > > try and get into this next release and potentially any beyond that > > for the next release nanny to work from. Current rough time frame > > for Pulp V2 integration is late January, early February time frame > > and thus this release will be based on feature completion not a hard > > timed release. > > > > > > Katello 1.3 Features > > - Pulp V2 integration > > - Pulp V2 upgrades > > > > > > > > > > Katello 1.4 Features > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > Reduction of the 564 open BZs. Seriously, if we're not going to > fix them let's just mass close. > > https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasmckay at redhat.com Thu Dec 6 19:13:20 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Thu, 6 Dec 2012 14:13:20 -0500 (EST) Subject: [katello-devel] rake gettext:find no longer pulling strings from haml files In-Reply-To: <1236970711.14973959.1354821056855.JavaMail.root@redhat.com> Message-ID: <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> At some point it seems gettext:find stopped working on haml files. The following string does not make it to src/locale/app.pot app/views/systems/_edit.html.haml: - help_message = _("Setting the release version limits content to this version only, preventing newer packages from being available for installation.") Does anyone recall making any changes that may have impacted this? When did we go from gettext to fast_gettext for example? My versions $ rpm -qa | grep gettext gettext-libs-0.17-16.el6.x86_64 rubygem-gettext-2.1.0-5.el6.noarch gettext-0.17-16.el6.x86_64 gettext-devel-0.17-16.el6.x86_64 rubygem-gettext-doc-2.1.0-5.el6.noarch rubygem-fast_gettext-0.5.10-1.el6.noarch rubygem-gettext_i18n_rails-0.2.13-1.el6.noarch From bkearney at redhat.com Thu Dec 6 19:40:43 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 06 Dec 2012 14:40:43 -0500 Subject: [katello-devel] rake gettext:find no longer pulling strings from haml files In-Reply-To: <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> References: <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> Message-ID: <50C0F4BB.5040608@redhat.com> I thought ivan had to do something. -- bk On 12/06/2012 02:13 PM, Tom McKay wrote: > At some point it seems gettext:find stopped working on haml files. The following string does not make it to src/locale/app.pot > > app/views/systems/_edit.html.haml: - help_message = _("Setting the release version limits content to this version only, preventing newer packages from being available for installation.") > > Does anyone recall making any changes that may have impacted this? When did we go from gettext to fast_gettext for example? > > My versions > > $ rpm -qa | grep gettext > gettext-libs-0.17-16.el6.x86_64 > rubygem-gettext-2.1.0-5.el6.noarch > gettext-0.17-16.el6.x86_64 > gettext-devel-0.17-16.el6.x86_64 > rubygem-gettext-doc-2.1.0-5.el6.noarch > rubygem-fast_gettext-0.5.10-1.el6.noarch > rubygem-gettext_i18n_rails-0.2.13-1.el6.noarch > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From thomasmckay at redhat.com Thu Dec 6 19:42:49 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Thu, 6 Dec 2012 14:42:49 -0500 (EST) Subject: [katello-devel] translation of model validation messages In-Reply-To: <597672650.14925253.1354813247881.JavaMail.root@redhat.com> Message-ID: <390077651.14983479.1354822969968.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Tom McKay" > To: "katello-devel" > Sent: Thursday, December 6, 2012 12:00:47 PM > Subject: translation of model validation messages > > FYI, many of the validation messages built into the models are not > translated. This should be a high priority item in the next > iteration. > > http://guides.rubyonrails.org/i18n.html#translations-for-active-record-models > > Example when creating an org w/o filling in any values: > > ?????????: > Label can't be blank > Label cannot contain characters other than ascii alpha numerals, '_', > '-'. > Label must contain at least 1 character > Label can't be blank > Name can't be blank > Name must contain at least 1 character Noticed that config/locale/ja.yml is all in English, not Japanese, which explains why the above is incorrect. Hopefully custom validations wrap messages in _(). From bbuckingham at redhat.com Thu Dec 6 21:57:41 2012 From: bbuckingham at redhat.com (Brad Buckingham) Date: Thu, 06 Dec 2012 16:57:41 -0500 Subject: [katello-devel] Fixing Issues with new Submodule In-Reply-To: References: Message-ID: <50C114D5.2070301@redhat.com> On 12/06/2012 01:55 PM, Eric D Helms wrote: > As part of the switch to Alchemy (formerly converge-ui) some path > updates for the submodule were made. For older git checkouts, you may > get some errors related to attempting to still checkout converge-ui. > The following should resolve any issues: > > 1. Delete the relevant section from |.git/config|. > 2. Run |git rm --cached path_to_submodule| (no trailing slash). > > > - Eric > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel If you are working from katello master and haven't already, you may need to do the following to get 'alchemy': git submodule init git submodule update If you do not, you might see an error similar to the following when attempting to access the server: Missing helper file helpers/alchemy/translation_helper.rb thanks, Brad -------------- next part -------------- An HTML attachment was scrubbed... URL: From inecas at redhat.com Fri Dec 7 07:21:17 2012 From: inecas at redhat.com (Ivan Necas) Date: Fri, 7 Dec 2012 02:21:17 -0500 (EST) Subject: [katello-devel] rake gettext:find no longer pulling strings from haml files In-Reply-To: <50C0F4BB.5040608@redhat.com> Message-ID: <1869114780.11824340.1354864877699.JavaMail.root@redhat.com> ----- Original Message ----- > I thought ivan had to do something. Nothing that dealt with haml and stuff, neither any configuration changed. Only making sure we don't have _("Wrong #{interpolations}") in the strings. -- Ivan > -- bk > > On 12/06/2012 02:13 PM, Tom McKay wrote: > > At some point it seems gettext:find stopped working on haml files. > > The following string does not make it to src/locale/app.pot > > > > app/views/systems/_edit.html.haml: - help_message = > > _("Setting the release version limits content to this version > > only, preventing newer packages from being available for > > installation.") > > > > Does anyone recall making any changes that may have impacted this? > > When did we go from gettext to fast_gettext for example? > > > > My versions > > > > $ rpm -qa | grep gettext > > gettext-libs-0.17-16.el6.x86_64 > > rubygem-gettext-2.1.0-5.el6.noarch > > gettext-0.17-16.el6.x86_64 > > gettext-devel-0.17-16.el6.x86_64 > > rubygem-gettext-doc-2.1.0-5.el6.noarch > > rubygem-fast_gettext-0.5.10-1.el6.noarch > > rubygem-gettext_i18n_rails-0.2.13-1.el6.noarch > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Fri Dec 7 08:54:52 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 7 Dec 2012 09:54:52 +0100 Subject: [katello-devel] rake gettext:find no longer pulling strings from haml files In-Reply-To: <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> References: <1236970711.14973959.1354821056855.JavaMail.root@redhat.com> <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> Message-ID: <20121207085452.GA1673@lzapx.brq.redhat.com> I don't know any details about this, but during my Gemfile review, I have realized that we have both gettext and fast_gettext gems. I did put "TODO" there to investigate this more. AFAIK we do have fast_gettext from the very beginning. It's also mentioned on the I18N wiki (design document). LZ On Thu, Dec 06, 2012 at 02:13:20PM -0500, Tom McKay wrote: > At some point it seems gettext:find stopped working on haml files. The following string does not make it to src/locale/app.pot > > app/views/systems/_edit.html.haml: - help_message = _("Setting the release version limits content to this version only, preventing newer packages from being available for installation.") > > Does anyone recall making any changes that may have impacted this? When did we go from gettext to fast_gettext for example? > > My versions > > $ rpm -qa | grep gettext > gettext-libs-0.17-16.el6.x86_64 > rubygem-gettext-2.1.0-5.el6.noarch > gettext-0.17-16.el6.x86_64 > gettext-devel-0.17-16.el6.x86_64 > rubygem-gettext-doc-2.1.0-5.el6.noarch > rubygem-fast_gettext-0.5.10-1.el6.noarch > rubygem-gettext_i18n_rails-0.2.13-1.el6.noarch > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From pchalupa at redhat.com Fri Dec 7 08:29:47 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Fri, 07 Dec 2012 09:29:47 +0100 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50BF408E.1030609@redhat.com> References: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> <50BF408E.1030609@redhat.com> Message-ID: <50C1A8FB.4000405@redhat.com> On 05.12.12 13:39, Dmitri Dolguikh wrote: > On 05/12/12 12:20 PM, Ohad Levy wrote: >> >> ----- Original Message ----- >> | On 05/12/12 09:07 AM, Petr Chalupa wrote: >> | > I agree that domain validation/consistency checks belongs to the >> | > models. Nevertheless definition of the API is other thing entirely. >> | > It >> | > should be defined and validated on its own with re-usage of >> | > information provided by models (domain) to avoid duplications. It >> | > should not be delegated to model to deal with it by adding other >> | > responsibility. >> | >> | This statement is only valid if we are talking about cross-concerns >> | (think authentication, security-related stuff, etc). Everything else >> | will result in duplicated, non-consistent validations. Take a look at >> | current state of validations in foreman api. >> | >> What do you mean by that? > > Which part? I insist that controllers should not carry any business > logic in them, as a corollary to this, no validations should reside in > controllers, unless those are used across all controllers, and are > outside of business-logic constraints (think aspect-like behaviour). Yes I agree that controllers should not carry any business logic, but they can define an API and they can validate if an user of the API is using it right, meaning validate the request before any other action is taken in domain. The issue is that the API validations are very similar to model validations in simple crud actions and we aren't using the model validation definition to construct API validations dynamically, we have to write them and that is the duplicity. This can be fixed by retrieving the already present information from models and construct API validations dynamically. I would focus fixing [1] soon, after that we won't have any duplicity issue. [1] https://github.com/Pajk/apipie-rails/issues/67 > By current state I meant that there are currently inconsistencies (data > coming through api are flagged invalid, even though it's valid according > to validators on the model) between api and model validations. > Config_template has a few, for example (and what started this thread). Yes this is a problem which will be solved by creating API validations from model validations dynamically. We do not have to disable or stop using the API validations and lose its advantages. > -d >> >> | I don't want to be jumping through hoops, and fixing/adding >> | code/meta-data in multiple places, so just the documentation is >> | up-to-date. It's a process/organizational issue, and tools do not fix >> | those. >> | >> | And, once again, validations belong in the model. Let's get rid of >> | validations in apipie, fix issues we have with domain logic leaking >> | into >> | controllers. >> | >> | > >> | > So I strongly suggest to fix the duplication issue [1] and keep >> | > things >> | > separate. >> | > * Apipie for API validations. >> | > * Model validations for domain logic. >> | > >> | > ## Fix Apipie >> | > >> | > Enable Apipie to read model definitions and validations to set >> | > param >> | > validations automatically for parts of request representing a >> | > domain >> | > model, see [1]. >> | >> | Agree with defining docs based on model (and I would suggest routes). >> | Disagree with everything else. No point in intercepting validations, >> | the >> | most you could do in the controller is map exceptions thrown by the >> | model to http errors/statuses. >> | >> | > >> | > >> | > + api documentation generated from api specification >> | > + api specification is kept up to date (has to be because >> | > validation >> | > is turned on) >> | > + api validation/specification is separated from domain >> | > validations/logic >> | > >> | > ## Remove validations from Apipie >> | > >> | > + api documentation generated from api specification >> | > - api specification is not kept up to date (has to be ensured in an >> | > extra task) >> | > - no api validation, it's partially handled by models, errors can >> | > be >> | > ugly, models have another responsibility >> | >> | Again, tools are not the way to enforce process deficiencies. The >> | last >> | statement is FUD. >> | >> | > >> | > >> | > [1] - https://github.com/Pajk/apipie-rails/issues/67 >> | > >> | > ## Other notes >> | > >> | > > The second issue is that apipie validations are stricter than AR >> | > ones: array validation assumes that there's an array in place, >> | > while >> | > AR would accept a nil. These two combined lead to even more false >> | > positives during apipie validation. >> | > >> | > I think be more consistent and strict is a good thing, it prevents >> | > confusion. Empty Array is an empty Array not a nil. >> | >> | It's not good, because depending on how you use foreman, you'll get >> | different results. What's considered valid/correct by foreman proper >> | should be valid by any other means, period. >> | >> | -d >> | >> | > >> | > Petr >> | > >> | > On 04.12.12 22:47, Ivan Necas wrote: >> | >> >> | >> >> | >> ----- Original Message ----- >> | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >> | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >> | >>>>>> Step #1 - let's get rid of controller validations. >> | >>>>> Again, you can only do this for CRUD actions in the >> | >>>>> controllers. >> | >>>>> For all >> | >>>>> other methods we need validations. And we have those, not all >> | >>>>> our >> | >>>>> controllers are plain REST-driven. We do have procedure-like >> | >>>>> actions. >> | >>>>> Maybe more then we should have, but this is a fact. >> | >>>>> >> | >>>>> You say all the validations should be done in models, thus you >> | >>>>> essentially say let's put all our code logic there. Because it >> | >>>>> does not >> | >>>>> make sense to do validations in models while doing anything >> | >>>>> else >> | >>>>> (that >> | >>>>> also requires some level of validations) in controllers or a >> | >>>>> logic >> | >>>>> layer. A fact: you need to do validations also outside of >> | >>>>> models. >> | >>>> >> | >>>> Domain logic belongs in the model layer. We can discuss how to >> | >>>> better >> | >>>> factor model layer code, but not where to put it. >> | >>>> >> | >>>> -d >> | >>>> >> | >>>> A lot of rationalizations to keep validations in apipie have >> | >>>> been >> | >>>> put >> | >>>> forward, *all* of them attempting to deal with various smells, >> | >>>> code >> | >>>> or >> | >>>> process ones. Let's fix the root issues. >> | >>>> >> | >>>> Again, Let's get rid of validations in apipie. Refactor the >> | >>>> controllers when needed. If apipie is not useful without those >> | >>>> validations, let's suspend/stop use of it. >> | >> >> | >> Am I reading it correctly you suggesting getting rid of API >> | >> documentation? >> | >> >> | >> -- Ivan >> | >> >> | >>>> >> | >>>> -d >> | >>>> >> | >>> +1 >> | >>>> >> | >>>>> >> | >>>>> It is wrong to put all your code logic into models. And >> | >>>>> unfortunately we >> | >>>>> have most of it there. Rails follows MVC pattern and what we >> | >>>>> have >> | >>>>> is >> | >>>>> those >> | >>>>> messy Models-for-everything approach. If you mix this with our >> | >>>>> orchestration, it is deadly combination. Difficult testability, >> | >>>>> huge >> | >>>>> objects with no responsibility split, unable to mock things and >> | >>>>> so >> | >>>>> on. >> | >>>>> Code logic in models works well for small projects and it is >> | >>>>> de-facto >> | >>>>> standard in Rails world. People just blindly follow this wrong >> | >>>>> pattern. >> | >>>>> >> | >>>>> So I don't agree with this simplification you offer. We will >> | >>>>> likely >> | >>>>> start to refactor our orchestration creating new layer of code >> | >>>>> logic, >> | >>>>> which is the right approach to do things. And we should move >> | >>>>> complex >> | >>>>> methods like activation key subscription from models there. >> | >>>>> Then >> | >>>>> we will >> | >>>>> need validations on the controller side (or on the newly >> | >>>>> created >> | >>>>> logic >> | >>>>> layer or orchestration layer - depends on how are we gonna name >> | >>>>> it). >> | >>>>> >> | >>>>> So yes, it has something to do with REST. My counter proposal >> | >>>>> is: >> | >>>>> >> | >>>>> #2 Let's get rid of validations where they duplicate model >> | >>>>> validators, >> | >>>>> but let's keep them for (usually non-CRUD) methods which need >> | >>>>> then. We >> | >>>>> will likely be extending those soon. >> | >>>>> >> | >>>> >> | >>>> _______________________________________________ >> | >>>> katello-devel mailing list >> | >>>> katello-devel at redhat.com >> | >>>> https://www.redhat.com/mailman/listinfo/katello-devel >> | >>> >> | >>> >> | >>> >> | >>> -- >> | >>> Jason E. Rist >> | >>> Senior Software Engineer >> | >>> Systems Management and Cloud Enablement >> | >>> Red Hat, Inc. >> | >>> +1.919.754.4048 >> | >>> Freenode: jrist >> | >>> github/identi.ca: knowncitizen >> | >>> >> | >>> _______________________________________________ >> | >>> katello-devel mailing list >> | >>> katello-devel at redhat.com >> | >>> https://www.redhat.com/mailman/listinfo/katello-devel >> | >>> >> | >> >> | >> _______________________________________________ >> | >> katello-devel mailing list >> | >> katello-devel at redhat.com >> | >> https://www.redhat.com/mailman/listinfo/katello-devel >> | >> >> | > >> | > _______________________________________________ >> | > katello-devel mailing list >> | > katello-devel at redhat.com >> | > https://www.redhat.com/mailman/listinfo/katello-devel >> | >> | _______________________________________________ >> | katello-devel mailing list >> | katello-devel at redhat.com >> | https://www.redhat.com/mailman/listinfo/katello-devel >> | > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From tstrachota at redhat.com Fri Dec 7 09:37:28 2012 From: tstrachota at redhat.com (Tomas Strachota) Date: Fri, 07 Dec 2012 10:37:28 +0100 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C0C33E.8090101@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> <50C0C1C5.7090101@redhat.com> <50C0C30A.3050806@redhat.com> <50C0C33E.8090101@redhat.com> Message-ID: <50C1B8D8.7010202@redhat.com> On 12/06/2012 05:09 PM, Dmitri Dolguikh wrote: > On 06/12/12 04:08 PM, Bryan Kearney wrote: >> On 12/06/2012 11:03 AM, Dmitri Dolguikh wrote: >>> On 06/12/12 02:46 PM, Bryan Kearney wrote: >>>> On 12/06/2012 09:45 AM, Bryan Kearney wrote: >>>>> Dmitri: >>>>> >>>>> I did have one question. What would it take to get the fancy text >>>>> editor >>>>> (ace.ajax.or_ from foreman into the katello ui? >>>>> >>>> >>>> >>>> sorry ace.ajax.org >>>> >>>> -- bk >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >>> I think adding ace editor on the "new" screen should be pretty trivial. >>> Show/modify might present an issue - since we somehow need to integrate >>> with/mimic jeditable behaviour. >> >> Do we need to mimic it? > > if not the look, then behaviour: we need to send updates on focus > change/timeout or something like that. Do we really have to make it such complicated? Wouldn't a simple form and a save button be just fine? We already use this approach on other places where we use ajax-saved jeditable for text inputs and more complicated elements are saved via classic forms. > >> >>> >>> How important is this? >> >> Not major, I notice that a big template does not look great. But, I >> would prefer features over skinning now. > > Nods. Let's put it on the backlog? Perhaps mark it as "nice-to-have" > feature? > -d > >> >> -- bk >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From pchalupa at redhat.com Fri Dec 7 09:50:11 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Fri, 07 Dec 2012 10:50:11 +0100 Subject: [katello-devel] Katello upgrade and Puppet In-Reply-To: <20121205153313.GI2315@lzapx.brq.redhat.com> References: <20121205153313.GI2315@lzapx.brq.redhat.com> Message-ID: <50C1BBD3.3070602@redhat.com> On 05.12.12 16:33, Lukas Zapletal wrote: > Hi, > > recent Petr's changes showed that katello-upgrade does not play well > with puppet. We have touched this issue several times now: > > - change touches a puppet template > - katello upgrade does this then: > - stops services > - executes our upgrade scripts > - migrates databases > - redeploys puppet (katello-configure) > > When the change in a puppet template is not compatible (e.g. some > refactoring), migration fails because Katello won't boot with this > (wrong) configuration since codebase was already upgraded with yum. > We need to create an upgrade script that somehow corrects the > configuration file until it's redeployed via katello configure. > > Two solutions I see: > > 1) Call puppet to only deploy configuration files and make this as a > first step before anything happens. I have been discussing this with > Foreman devs little while back, there are some features of puppet we > could use for this but this is not how they should be used. Also seems > to be "hacky". I would look into this one, because it would fix the problem with old config for me and I could merge configuration cleanup. > 2) Have full upgrade process in puppet. Very challenging, I can't really > believe this is even doable - we need to deploy configuration, then > execute some steps in particular order, then restart services. One big > disadvantage is the upgrade process cannot be interactive anymore I > believe - just fire and wait. > > Opinions? > Anyway I think we have issues with katello-configuration and katello-upgrade too often. Maybe we should spend some tome on finding a better solution. From dmitri at redhat.com Fri Dec 7 10:00:29 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Fri, 07 Dec 2012 10:00:29 +0000 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C1B8D8.7010202@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> <50C0C1C5.7090101@redhat.com> <50C0C30A.3050806@redhat.com> <50C0C33E.8090101@redhat.com> <50C1B8D8.7010202@redhat.com> Message-ID: <50C1BE3D.1060202@redhat.com> On 07/12/12 09:37 AM, Tomas Strachota wrote: > On 12/06/2012 05:09 PM, Dmitri Dolguikh wrote: >> On 06/12/12 04:08 PM, Bryan Kearney wrote: >>> On 12/06/2012 11:03 AM, Dmitri Dolguikh wrote: >>>> On 06/12/12 02:46 PM, Bryan Kearney wrote: >>>>> On 12/06/2012 09:45 AM, Bryan Kearney wrote: >>>>>> Dmitri: >>>>>> >>>>>> I did have one question. What would it take to get the fancy text >>>>>> editor >>>>>> (ace.ajax.or_ from foreman into the katello ui? >>>>>> >>>>> >>>>> >>>>> sorry ace.ajax.org >>>>> >>>>> -- bk >>>>> >>>>> _______________________________________________ >>>>> katello-devel mailing list >>>>> katello-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> >>>> I think adding ace editor on the "new" screen should be pretty >>>> trivial. >>>> Show/modify might present an issue - since we somehow need to >>>> integrate >>>> with/mimic jeditable behaviour. >>> >>> Do we need to mimic it? >> >> if not the look, then behaviour: we need to send updates on focus >> change/timeout or something like that. > > Do we really have to make it such complicated? Wouldn't a simple form > and a save button be just fine? > We already use this approach on other places where we use ajax-saved > jeditable for text inputs and more complicated elements are saved via > classic forms. I'll leave it to UI folks to make the ruling on this one, but to me that would look pretty inconsistent, not to mention weird. Either the whole form is submitted via with 'submit' button click, or the whole screen is using jeditable approach... -d > >> >>> >>>> >>>> How important is this? >>> >>> Not major, I notice that a big template does not look great. But, I >>> would prefer features over skinning now. >> >> Nods. Let's put it on the backlog? Perhaps mark it as "nice-to-have" >> feature? >> -d >> >>> >>> -- bk >>> >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From dmitri at redhat.com Fri Dec 7 10:10:41 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Fri, 07 Dec 2012 10:10:41 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50C1A8FB.4000405@redhat.com> References: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> <50BF408E.1030609@redhat.com> <50C1A8FB.4000405@redhat.com> Message-ID: <50C1C0A1.5040207@redhat.com> On 07/12/12 08:29 AM, Petr Chalupa wrote: > > > On 05.12.12 13:39, Dmitri Dolguikh wrote: >> On 05/12/12 12:20 PM, Ohad Levy wrote: >>> >>> ----- Original Message ----- >>> | On 05/12/12 09:07 AM, Petr Chalupa wrote: >>> | > I agree that domain validation/consistency checks belongs to the >>> | > models. Nevertheless definition of the API is other thing entirely. >>> | > It >>> | > should be defined and validated on its own with re-usage of >>> | > information provided by models (domain) to avoid duplications. It >>> | > should not be delegated to model to deal with it by adding other >>> | > responsibility. >>> | >>> | This statement is only valid if we are talking about cross-concerns >>> | (think authentication, security-related stuff, etc). Everything else >>> | will result in duplicated, non-consistent validations. Take a look at >>> | current state of validations in foreman api. >>> | >>> What do you mean by that? >> >> Which part? I insist that controllers should not carry any business >> logic in them, as a corollary to this, no validations should reside in >> controllers, unless those are used across all controllers, and are >> outside of business-logic constraints (think aspect-like behaviour). > > Yes I agree that controllers should not carry any business logic, but > they can define an API and they can validate if an user of the API is > using it right, meaning validate the request before any other action > is taken in domain. I have no clue what you mean by "validating API". At all. Nada. Again: it's ok to validate cross-converns in controllers, those are outside of model/business logic. Everything else is duplicating/relocating business logic into the controllers. It's not apipie's responsibility to enforce validations. Drop this functionality, or I'll be asking the team next to remove apipie *completely*, as its developers are not being reasonable. > The issue is that the API validations are very similar to model > validations in simple crud actions and we aren't using the model > validation definition to construct API validations dynamically, we > have to write them and that is the duplicity. This can be fixed by > retrieving the already present information from models and construct > API validations dynamically. > > I would focus fixing [1] soon, after that we won't have any duplicity > issue. Don't we have more pressing stuff to do? > > [1] https://github.com/Pajk/apipie-rails/issues/67 > >> By current state I meant that there are currently inconsistencies (data >> coming through api are flagged invalid, even though it's valid according >> to validators on the model) between api and model validations. >> Config_template has a few, for example (and what started this thread). > > Yes this is a problem which will be solved by creating API validations > from model validations dynamically. We do not have to disable or stop > using the API validations and lose its advantages. This makes no sense. At best you'd be running model validations in controllers. This is wasted effort, see my reasons above. Please stop this madness. -d > >> -d >>> >>> | I don't want to be jumping through hoops, and fixing/adding >>> | code/meta-data in multiple places, so just the documentation is >>> | up-to-date. It's a process/organizational issue, and tools do not fix >>> | those. >>> | >>> | And, once again, validations belong in the model. Let's get rid of >>> | validations in apipie, fix issues we have with domain logic leaking >>> | into >>> | controllers. >>> | >>> | > >>> | > So I strongly suggest to fix the duplication issue [1] and keep >>> | > things >>> | > separate. >>> | > * Apipie for API validations. >>> | > * Model validations for domain logic. >>> | > >>> | > ## Fix Apipie >>> | > >>> | > Enable Apipie to read model definitions and validations to set >>> | > param >>> | > validations automatically for parts of request representing a >>> | > domain >>> | > model, see [1]. >>> | >>> | Agree with defining docs based on model (and I would suggest routes). >>> | Disagree with everything else. No point in intercepting validations, >>> | the >>> | most you could do in the controller is map exceptions thrown by the >>> | model to http errors/statuses. >>> | >>> | > >>> | > >>> | > + api documentation generated from api specification >>> | > + api specification is kept up to date (has to be because >>> | > validation >>> | > is turned on) >>> | > + api validation/specification is separated from domain >>> | > validations/logic >>> | > >>> | > ## Remove validations from Apipie >>> | > >>> | > + api documentation generated from api specification >>> | > - api specification is not kept up to date (has to be ensured in an >>> | > extra task) >>> | > - no api validation, it's partially handled by models, errors can >>> | > be >>> | > ugly, models have another responsibility >>> | >>> | Again, tools are not the way to enforce process deficiencies. The >>> | last >>> | statement is FUD. >>> | >>> | > >>> | > >>> | > [1] - https://github.com/Pajk/apipie-rails/issues/67 >>> | > >>> | > ## Other notes >>> | > >>> | > > The second issue is that apipie validations are stricter than AR >>> | > ones: array validation assumes that there's an array in place, >>> | > while >>> | > AR would accept a nil. These two combined lead to even more false >>> | > positives during apipie validation. >>> | > >>> | > I think be more consistent and strict is a good thing, it prevents >>> | > confusion. Empty Array is an empty Array not a nil. >>> | >>> | It's not good, because depending on how you use foreman, you'll get >>> | different results. What's considered valid/correct by foreman proper >>> | should be valid by any other means, period. >>> | >>> | -d >>> | >>> | > >>> | > Petr >>> | > >>> | > On 04.12.12 22:47, Ivan Necas wrote: >>> | >> >>> | >> >>> | >> ----- Original Message ----- >>> | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>> | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>> | >>>>>> Step #1 - let's get rid of controller validations. >>> | >>>>> Again, you can only do this for CRUD actions in the >>> | >>>>> controllers. >>> | >>>>> For all >>> | >>>>> other methods we need validations. And we have those, not all >>> | >>>>> our >>> | >>>>> controllers are plain REST-driven. We do have procedure-like >>> | >>>>> actions. >>> | >>>>> Maybe more then we should have, but this is a fact. >>> | >>>>> >>> | >>>>> You say all the validations should be done in models, thus you >>> | >>>>> essentially say let's put all our code logic there. Because it >>> | >>>>> does not >>> | >>>>> make sense to do validations in models while doing anything >>> | >>>>> else >>> | >>>>> (that >>> | >>>>> also requires some level of validations) in controllers or a >>> | >>>>> logic >>> | >>>>> layer. A fact: you need to do validations also outside of >>> | >>>>> models. >>> | >>>> >>> | >>>> Domain logic belongs in the model layer. We can discuss how to >>> | >>>> better >>> | >>>> factor model layer code, but not where to put it. >>> | >>>> >>> | >>>> -d >>> | >>>> >>> | >>>> A lot of rationalizations to keep validations in apipie have >>> | >>>> been >>> | >>>> put >>> | >>>> forward, *all* of them attempting to deal with various smells, >>> | >>>> code >>> | >>>> or >>> | >>>> process ones. Let's fix the root issues. >>> | >>>> >>> | >>>> Again, Let's get rid of validations in apipie. Refactor the >>> | >>>> controllers when needed. If apipie is not useful without those >>> | >>>> validations, let's suspend/stop use of it. >>> | >> >>> | >> Am I reading it correctly you suggesting getting rid of API >>> | >> documentation? >>> | >> >>> | >> -- Ivan >>> | >> >>> | >>>> >>> | >>>> -d >>> | >>>> >>> | >>> +1 >>> | >>>> >>> | >>>>> >>> | >>>>> It is wrong to put all your code logic into models. And >>> | >>>>> unfortunately we >>> | >>>>> have most of it there. Rails follows MVC pattern and what we >>> | >>>>> have >>> | >>>>> is >>> | >>>>> those >>> | >>>>> messy Models-for-everything approach. If you mix this with our >>> | >>>>> orchestration, it is deadly combination. Difficult testability, >>> | >>>>> huge >>> | >>>>> objects with no responsibility split, unable to mock things and >>> | >>>>> so >>> | >>>>> on. >>> | >>>>> Code logic in models works well for small projects and it is >>> | >>>>> de-facto >>> | >>>>> standard in Rails world. People just blindly follow this wrong >>> | >>>>> pattern. >>> | >>>>> >>> | >>>>> So I don't agree with this simplification you offer. We will >>> | >>>>> likely >>> | >>>>> start to refactor our orchestration creating new layer of code >>> | >>>>> logic, >>> | >>>>> which is the right approach to do things. And we should move >>> | >>>>> complex >>> | >>>>> methods like activation key subscription from models there. >>> | >>>>> Then >>> | >>>>> we will >>> | >>>>> need validations on the controller side (or on the newly >>> | >>>>> created >>> | >>>>> logic >>> | >>>>> layer or orchestration layer - depends on how are we gonna name >>> | >>>>> it). >>> | >>>>> >>> | >>>>> So yes, it has something to do with REST. My counter proposal >>> | >>>>> is: >>> | >>>>> >>> | >>>>> #2 Let's get rid of validations where they duplicate model >>> | >>>>> validators, >>> | >>>>> but let's keep them for (usually non-CRUD) methods which need >>> | >>>>> then. We >>> | >>>>> will likely be extending those soon. >>> | >>>>> >>> | >>>> >>> | >>>> _______________________________________________ >>> | >>>> katello-devel mailing list >>> | >>>> katello-devel at redhat.com >>> | >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>> | >>> >>> | >>> >>> | >>> >>> | >>> -- >>> | >>> Jason E. Rist >>> | >>> Senior Software Engineer >>> | >>> Systems Management and Cloud Enablement >>> | >>> Red Hat, Inc. >>> | >>> +1.919.754.4048 >>> | >>> Freenode: jrist >>> | >>> github/identi.ca: knowncitizen >>> | >>> >>> | >>> _______________________________________________ >>> | >>> katello-devel mailing list >>> | >>> katello-devel at redhat.com >>> | >>> https://www.redhat.com/mailman/listinfo/katello-devel >>> | >>> >>> | >> >>> | >> _______________________________________________ >>> | >> katello-devel mailing list >>> | >> katello-devel at redhat.com >>> | >> https://www.redhat.com/mailman/listinfo/katello-devel >>> | >> >>> | > >>> | > _______________________________________________ >>> | > katello-devel mailing list >>> | > katello-devel at redhat.com >>> | > https://www.redhat.com/mailman/listinfo/katello-devel >>> | >>> | _______________________________________________ >>> | katello-devel mailing list >>> | katello-devel at redhat.com >>> | https://www.redhat.com/mailman/listinfo/katello-devel >>> | >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From bkearney at redhat.com Fri Dec 7 12:26:22 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Fri, 07 Dec 2012 07:26:22 -0500 Subject: [katello-devel] a followup form the demo In-Reply-To: <50C1BE3D.1060202@redhat.com> References: <50C0AF81.3050706@redhat.com> <50C0AFC5.6010703@redhat.com> <50C0C1C5.7090101@redhat.com> <50C0C30A.3050806@redhat.com> <50C0C33E.8090101@redhat.com> <50C1B8D8.7010202@redhat.com> <50C1BE3D.1060202@redhat.com> Message-ID: <50C1E06E.2090706@redhat.com> On 12/07/2012 05:00 AM, Dmitri Dolguikh wrote: > On 07/12/12 09:37 AM, Tomas Strachota wrote: >> On 12/06/2012 05:09 PM, Dmitri Dolguikh wrote: >>> On 06/12/12 04:08 PM, Bryan Kearney wrote: >>>> On 12/06/2012 11:03 AM, Dmitri Dolguikh wrote: >>>>> On 06/12/12 02:46 PM, Bryan Kearney wrote: >>>>>> On 12/06/2012 09:45 AM, Bryan Kearney wrote: >>>>>>> Dmitri: >>>>>>> >>>>>>> I did have one question. What would it take to get the fancy text >>>>>>> editor >>>>>>> (ace.ajax.or_ from foreman into the katello ui? >>>>>>> >>>>>> >>>>>> >>>>>> sorry ace.ajax.org >>>>>> >>>>>> -- bk >>>>>> >>>>>> _______________________________________________ >>>>>> katello-devel mailing list >>>>>> katello-devel at redhat.com >>>>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>>> >>>>> I think adding ace editor on the "new" screen should be pretty >>>>> trivial. >>>>> Show/modify might present an issue - since we somehow need to >>>>> integrate >>>>> with/mimic jeditable behaviour. >>>> >>>> Do we need to mimic it? >>> >>> if not the look, then behaviour: we need to send updates on focus >>> change/timeout or something like that. >> >> Do we really have to make it such complicated? Wouldn't a simple form >> and a save button be just fine? >> We already use this approach on other places where we use ajax-saved >> jeditable for text inputs and more complicated elements are saved via >> classic forms. > > I'll leave it to UI folks to make the ruling on this one, but to me that > would look pretty inconsistent, not to mention weird. Either the whole > form is submitted via with 'submit' button click, or the whole screen is > using jeditable approach... > > -d I am fine leaving it to them, and delaying. But.. the amount of text did not look good in the demo. The new tu-pane may solve that tho. as Dmitri suggested, it is on the backlog. -- bk From ericdhelms at gmail.com Fri Dec 7 13:45:54 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Fri, 7 Dec 2012 08:45:54 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: <50C0ED09.6090505@redhat.com> References: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> <50C0ED09.6090505@redhat.com> Message-ID: Two others that crossed my mind: - Getting on the Rails 3.2 stack (to be congruent with Aeolus) - Splitting up the codebase into focused repositories On Thu, Dec 6, 2012 at 2:07 PM, Brad Buckingham wrote: > Another thought which might be nice... How about possibly a > documentation day (or partial day)? This could be for supporting many > forms of documentation (blogs, user docs, manpages (if any needed), > screencasts...). > > Brad > > > On 12/06/2012 02:04 PM, Eric D Helms wrote: > > I think this would be a great time for a bug-day and am going to organize > such. Bug day being, whole team sitting down and pounding through the list > of bugs to verify, ask for more information or try and fix. I figure we > can get a conference room for a day (in the U.S.) to allow easy discussion > since inevitably some people have more information than others in some > areas. > > Team Other side of the Pond -- since the time zone difference is great > enough, if you guys would also like to put together a bug day around the > same time and if someone from your side would like to volunteer the > organization of it that would be great. > > - Eric > > > On Thu, Dec 6, 2012 at 12:47 PM, Tom McKay wrote: > >> >> >> ----- Original Message ----- >> > From: "Eric D Helms" >> > To: katello-devel at redhat.com >> > Sent: Thursday, December 6, 2012 11:58:28 AM >> > Subject: [katello-devel] Katello 1.3/1.4 Feature Planning >> > >> > >> > Howdy, >> > >> > >> > As I prepare the Katello 1.2 release, I thought I would get the ball >> > rolling on feature planning for Katello 1.3. As it stands, the focus >> > of Katello 1.3 is Pulp V2 integration. However, a lot of other >> > features and work will go on during now and when the Pulp V2 work is >> > ready. So I'd like to get a rough idea of what features we plan to >> > try and get into this next release and potentially any beyond that >> > for the next release nanny to work from. Current rough time frame >> > for Pulp V2 integration is late January, early February time frame >> > and thus this release will be based on feature completion not a hard >> > timed release. >> > >> > >> > Katello 1.3 Features >> > - Pulp V2 integration >> > - Pulp V2 upgrades >> > >> > >> > >> > >> > Katello 1.4 Features >> > _______________________________________________ >> > katello-devel mailing list >> > katello-devel at redhat.com >> > https://www.redhat.com/mailman/listinfo/katello-devel >> >> >> Reduction of the 564 open BZs. Seriously, if we're not going to fix them >> let's just mass close. >> >> >> https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 >> > > > > _______________________________________________ > katello-devel mailing listkatello-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From morazi at redhat.com Fri Dec 7 14:40:50 2012 From: morazi at redhat.com (Mike Orazi) Date: Fri, 07 Dec 2012 09:40:50 -0500 Subject: [katello-devel] Katello upgrade and Puppet In-Reply-To: <50C1BBD3.3070602@redhat.com> References: <20121205153313.GI2315@lzapx.brq.redhat.com> <50C1BBD3.3070602@redhat.com> Message-ID: <50C1FFF2.80104@redhat.com> On 12/07/2012 04:50 AM, Petr Chalupa wrote: > > > On 05.12.12 16:33, Lukas Zapletal wrote: >> Hi, >> >> recent Petr's changes showed that katello-upgrade does not play well >> with puppet. We have touched this issue several times now: >> >> - change touches a puppet template >> - katello upgrade does this then: >> - stops services >> - executes our upgrade scripts >> - migrates databases >> - redeploys puppet (katello-configure) >> >> When the change in a puppet template is not compatible (e.g. some >> refactoring), migration fails because Katello won't boot with this >> (wrong) configuration since codebase was already upgraded with yum. >> We need to create an upgrade script that somehow corrects the >> configuration file until it's redeployed via katello configure. >> >> Two solutions I see: >> >> 1) Call puppet to only deploy configuration files and make this as a >> first step before anything happens. I have been discussing this with >> Foreman devs little while back, there are some features of puppet we >> could use for this but this is not how they should be used. Also seems >> to be "hacky". > > I would look into this one, because it would fix the problem with old > config for me and I could merge configuration cleanup. > >> 2) Have full upgrade process in puppet. Very challenging, I can't really >> believe this is even doable - we need to deploy configuration, then >> execute some steps in particular order, then restart services. One big >> disadvantage is the upgrade process cannot be interactive anymore I >> believe - just fire and wait. >> >> Opinions? >> > > Anyway I think we have issues with katello-configuration and > katello-upgrade too often. Maybe we should spend some tome on finding a > better solution. > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel Please keep the updates flowing for any big changes to katello-configuration/upgrade as the aeolus variants try to seem somewhat similar so that folks who use both have some amount of similarity with the configuration & upgrade cycle on both projects. I've cc:ed a few folks involved in configure/upgrade on the aeolus side who might have some input regarding the problems you are facing presently or would probably like to be involved in any ongoing conversations involving fundamental changes to configuration and updating of the applications. Thanks, Mike From ericdhelms at gmail.com Fri Dec 7 14:54:03 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Fri, 7 Dec 2012 09:54:03 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <1129335304.14876517.1354808723127.JavaMail.root@redhat.com> References: <1129335304.14876517.1354808723127.JavaMail.root@redhat.com> Message-ID: On Thu, Dec 6, 2012 at 10:45 AM, Tom McKay wrote: > > > ----- Original Message ----- > > From: "Eric D Helms" > > To: katello-devel at redhat.com > > Sent: Wednesday, December 5, 2012 9:20:06 PM > > Subject: [katello-devel] Katello Backend Service Relationships and > Architecture > > > > > > They say a picture is worth a thousands words, and after, for lack of > > a better word, being bugged by the differences in backend service > > integration I decided to try and spell out the current lay of the > > land. I have attempted to do this in two ways. The first being a > > diagram showing how our models, glue and controllers are laid out in > > relation to backend services with arrows indicating the directional > > awareness that one entity has with another. For example, an arrow > > pointing from glue/candlepin to resources/candlepin since > > glue/candlepin has direct knowledge of the resources but not vice > > versa. Second, I have attempted to quell out into bullet points how > > each backend service is currently (and by this I mean as of PulpV2 > > branch) integrated. > > > > > > Disclaimer: This is my understanding of things and there are places > > where I could have gotten something wrong or mis-represented that > > actual implementation > > > > > > > > Reference Diagram: > > > https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > > > > > > Note: This is taking into consideration the changes made in the > > PulpV2 branch and current Candlepin and Foreman work in master > > > > > > > > > > PulpV2 > > > > > > - All Pulp functionality is contained within app/models/glue/pulp > > - All glue layers are modules that are included into the Katello > > model if use_pulp = true > > - Katello Models are treated as the main entity, with Pulp glue layer > > adding functionality if included > > - Even non-persisted objects have a top-level representation in > > Katello (e.g. errata, package, distribution) > > - Nothing is proxied > > - Basic functionality is orchestrated (CRUD) > > - Use of callbacks and hooks to trigger top-level events that can be > > listened to by glue layer > > - There is no notion of pulp outside the glue layer > > > > > > > > > > Candlepin > > > > > > - Contains a single Candlepin only model without corresponding > > inclusion in a Katello entity (OwnerInfo) > > - Contains some proxied functionality mainly for subscription manager > > - Not all Candlepin functionality is contained in glue layer > > - Basic functionality is orchestrated (CRUD) > > > > Definitely room for clean-up in the candlepin layer. I would love to see > us move to align with the clean pulp v2 integration style. > > From what we learned cleaning-up the Pulp layer for V2 integration, doing the same for the Candlepin side shouldn't be too difficult or time-consuming. -Eric > > > > > > > > Foreman > > > > > > - Foreman is front and center at all layers > > - Controllers for API and UI are under foreman/ and namespaced > > - Models are under foreman/ and are classed as Foreman:: > > - Katello models that have corresponding Foreman model also get a > > separate model file besides their glue layer counterpart (i.e. > > models/foreman/user) > > - Other models have lightweight representations and are entirely pass > > through objects down to Foreman but are not top-level Katello > > entities > > - Glue layer is fenced off and only included if use_foreman set to > > true > > > > > > > > > > ElasticSearch > > > > > > - All ES functionality is contained within > > app/models/glue/elasticsearch > > - All glue layers are modules that included into the corresponding > > Katello model > > - Adds functionality but does not directly impart itself into the > > models > > - Use of callbacks and hooks to trigger top-level events that can be > > listened to by glue layer > > > > > > > > > > I am hoping this will spark some discussion, as I worry that the > > differences in integration points for each backend service will only > > make our lives as developers more difficult down the road. > > > > > > -Eric > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericdhelms at gmail.com Fri Dec 7 16:16:42 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Fri, 7 Dec 2012 11:16:42 -0500 Subject: [katello-devel] Katello 1.2 Release Candidate Message-ID: Howdy All, The first release candidate for Katello 1.2 is now available. There are candidates for: Fedora 16 - http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm Fedora 17 - http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm RHEL 6 - http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm Repos: http://fedorapeople.org/groups/katello/releases/yum/1.2/ Installation and Upgrade Instructions: https://fedorahosted.org/katello/wiki/Install Report Bugs: https://fedorahosted.org/katello/wiki/ReportErrors If you have the bandwidth, please test installations, upgrades and run any test suites you may have or have access to. It may also help to reply to this email claiming a particular test path to help spread out and cover more testing ground. If there are install, or usage breaking bugs that are found, they will follow the method of being fixed upstream in master and cherry-picked into the 1.2 branch. Thanks, Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkearney at redhat.com Fri Dec 7 16:35:17 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Fri, 07 Dec 2012 11:35:17 -0500 Subject: [katello-devel] Katello 1.3/1.4 Feature Planning In-Reply-To: References: <1169493747.14939710.1354816023733.JavaMail.root@redhat.com> <50C0ED09.6090505@redhat.com> Message-ID: <50C21AC5.4070207@redhat.com> On 12/07/2012 08:45 AM, Eric D Helms wrote: > Two others that crossed my mind: > > - Getting on the Rails 3.2 stack (to be congruent with Aeolus) I like this. > - Splitting up the codebase into focused repositories This one worries me. Seems like lots of destabilzation for marginal gain. -- bk From daviddavis at redhat.com Fri Dec 7 17:02:42 2012 From: daviddavis at redhat.com (David Davis) Date: Fri, 7 Dec 2012 12:02:42 -0500 (EST) Subject: [katello-devel] Rails 3.2 features In-Reply-To: <261716780.42796153.1354898542107.JavaMail.root@redhat.com> Message-ID: <1965415828.42808420.1354899762103.JavaMail.root@redhat.com> I just wanted to make a list of features that we'll get upgrading to Rails 3.2. I'm sending these out to inform everyone but I think it's worth reiterating that it's going to be painful to upgrade. It won't be as bad as going from Rails 2 to Rails 3 but it's not going to be easy either. Without futher ado... * Probably the biggest thing is the asset pipeline. It does everything from compiling languages (like SASS) to uglifying js code. Check out more info at http://guides.rubyonrails.org/asset_pipeline.html * New migration features. You no longer have to write up/down methods for the most part. Instead you just write "change" methods. More info at http://guides.rubyonrails.org/migrations.html * Multiple levels of has_many :through * Create key value stores in the database (http://blog.rawonrails.com/2012/02/using-activerecordstore-with-rails-32.html) * New active record methods like pluck (http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-pluck) * Scoped mass assignment. Helpful for locking down attributes per permission. (http://launchware.com/articles/whats-new-in-edge-scoped-mass-assignment-in-rails-3-1) * Slow query logger http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/. I've used http://explain.depesz.com/ to help analyze query explain output from postgres. * Encrypted fields (passwords, etc) http://railscasts.com/episodes/270-authentication-in-rails-3-1 * New generator stuff. You can do "rails g scaffold Person name email" instead of "rails g scaffold Person name:string email:string". Or if you want to be really crafty: rails g scaffold Person name:index email:uniq. David From daviddavis at redhat.com Fri Dec 7 17:03:20 2012 From: daviddavis at redhat.com (David Davis) Date: Fri, 7 Dec 2012 12:03:20 -0500 (EST) Subject: [katello-devel] Rails 3.2 features Message-ID: <18062358.42808689.1354899800009.JavaMail.root@redhat.com> I just wanted to make a list of features that we'll get upgrading to Rails 3.2. I'm sending these out to inform everyone but I think it's worth reiterating that it's going to be painful to upgrade. It won't be as bad as going from Rails 2 to Rails 3 but it's not going to be easy either. Without futher ado... * Probably the biggest thing is the asset pipeline. It does everything from compiling languages (like SASS) to uglifying js code. Check out more info at http://guides.rubyonrails.org/asset_pipeline.html * New migration features. You no longer have to write up/down methods for the most part. Instead you just write "change" methods. More info at http://guides.rubyonrails.org/migrations.html * Multiple levels of has_many :through * Create key value stores in the database (http://blog.rawonrails.com/2012/02/using-activerecordstore-with-rails-32.html) * New active record methods like pluck (http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-pluck) * Scoped mass assignment. Helpful for locking down attributes per permission. (http://launchware.com/articles/whats-new-in-edge-scoped-mass-assignment-in-rails-3-1) * Slow query logger http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/. I've used http://explain.depesz.com/ to help analyze query explain output from postgres. * Encrypted fields (passwords, etc) http://railscasts.com/episodes/270-authentication-in-rails-3-1 * New generator stuff. You can do "rails g scaffold Person name email" instead of "rails g scaffold Person name:string email:string". Or if you want to be really crafty: rails g scaffold Person name:index email:uniq. David David From daviddavis at redhat.com Fri Dec 7 17:15:15 2012 From: daviddavis at redhat.com (David Davis) Date: Fri, 7 Dec 2012 12:15:15 -0500 (EST) Subject: [katello-devel] Rails 3.2 features In-Reply-To: <18062358.42808689.1354899800009.JavaMail.root@redhat.com> Message-ID: <211697786.42812901.1354900515970.JavaMail.root@redhat.com> I thought I'd send a second email in case you guys happened to miss the first one. David ----- Original Message ----- > From: "David Davis" > To: katello-devel at redhat.com > Sent: Friday, December 7, 2012 12:03:20 PM > Subject: [katello-devel] Rails 3.2 features > > I just wanted to make a list of features that we'll get upgrading to > Rails 3.2. I'm sending these out to inform everyone but I think it's > worth reiterating that it's going to be painful to upgrade. It won't > be as bad as going from Rails 2 to Rails 3 but it's not going to be > easy either. Without futher ado... > > * Probably the biggest thing is the asset pipeline. It does > everything from compiling languages (like SASS) to uglifying js > code. Check out more info at > http://guides.rubyonrails.org/asset_pipeline.html > * New migration features. You no longer have to write up/down methods > for the most part. Instead you just write "change" methods. More > info at http://guides.rubyonrails.org/migrations.html > * Multiple levels of has_many :through > * Create key value stores in the database > (http://blog.rawonrails.com/2012/02/using-activerecordstore-with-rails-32.html) > * New active record methods like pluck > (http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-pluck) > * Scoped mass assignment. Helpful for locking down attributes per > permission. > (http://launchware.com/articles/whats-new-in-edge-scoped-mass-assignment-in-rails-3-1) > * Slow query logger > http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/. > I've used http://explain.depesz.com/ to help analyze query explain > output from postgres. > * Encrypted fields (passwords, etc) > http://railscasts.com/episodes/270-authentication-in-rails-3-1 > * New generator stuff. You can do "rails g scaffold Person name > email" instead of "rails g scaffold Person name:string > email:string". Or if you want to be really crafty: rails g scaffold > Person name:index email:uniq. > > David > > David > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From ewoud+katello at kohlvanwijngaarden.nl Fri Dec 7 18:31:35 2012 From: ewoud+katello at kohlvanwijngaarden.nl (Ewoud Kohl van Wijngaarden) Date: Fri, 7 Dec 2012 19:31:35 +0100 Subject: [katello-devel] Fixing Issues with new Submodule In-Reply-To: <50C114D5.2070301@redhat.com> References: <50C114D5.2070301@redhat.com> Message-ID: <20121207183135.GG16243@bogey.xentower.nl> On Thu, Dec 06, 2012 at 04:57:41PM -0500, Brad Buckingham wrote: > git submodule init > git submodule update For the lazy like me there's also: git submodule update --init From mmccune at redhat.com Fri Dec 7 18:45:32 2012 From: mmccune at redhat.com (Mike McCune) Date: Fri, 07 Dec 2012 10:45:32 -0800 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: References: Message-ID: <50C2394C.3050106@redhat.com> On 12/07/2012 08:16 AM, Eric D Helms wrote: > Howdy All, > > The first release candidate for Katello 1.2 is now available. There are > candidates for: > > Fedora 16 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm > > Fedora 17 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm > > RHEL 6 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm > > Repos: > http://fedorapeople.org/groups/katello/releases/yum/1.2/ > > Installation and Upgrade Instructions: > https://fedorahosted.org/katello/wiki/Install > > Report Bugs: > https://fedorahosted.org/katello/wiki/ReportErrors > > > If you have the bandwidth, please test installations, upgrades and run > any test suites you may have or have access to. It may also help to > reply to this email claiming a particular test path to help spread out > and cover more testing ground. > > If there are install, or usage breaking bugs that are found, they will > follow the method of being fixed upstream in master and cherry-picked > into the 1.2 branch. > some initial notes: 1) I noticed the repo file for 1.2 still points at the nightly URL: [root at g-12070948 ~]# rpm -q katello-repos katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch [root at g-12070948 ~]# grep baseurl /etc/yum.repos.d/katello.repo baseurl=http://fedorapeople.org/groups/katello/releases/yum/nightly/RHEL/$releasever/$basearch/ baseurl=http://fedorapeople.org/groups/katello/releases/source/nightly/RHEL/$releasever/ 2) katello-devel-all fails to install on F17: --> Finished Dependency Resolution Error: Package: rubygem-jshintrb-0.2.1-2.fc17.noarch (katello) Requires: rubygem(multi_json) >= 1.3 Installed: rubygem-multi_json-1.0.3-5.fc17.noarch (@fedora) rubygem(multi_json) = 1.0.3 You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest 3) katello-configure failed on EL6: Starting Katello configuration The top-level log file is [/var/log/katello/katello-configure-20121207-102354/main.log] Creating Candlepin database user ############################################################ ... OK Creating Katello database user ############################################################ ... OK Creating Katello database ############################################################ ... OK Populating Katello database schema ############################################################ ... OK Initializing Katello data ############################################################ ... OK err: /Stage[main]/Katello::Service/Service[katello-jobs]/ensure: change from stopped to running failed: Could not start Service[katello-jobs]: Execution of '/sbin/service katello-jobs start' returned 1: at /usr/share/katello/install/puppet/modules/katello/manifests/service.pp:21 which looks like it is that cp/reporter/core error that I thought got removed from the Gemfile: > Exiting! /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in `strict_error': Gem loading error: no such file to load -- ci/reporter/core (RuntimeError) Gemfile.in: # The following groups are not packaged as RPMs and are used only in Bundler mode group :ci do # needed by hudson gem 'ci_reporter', '~> 1.7.2', :require => 'ci/reporter/core' end -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From ericdhelms at gmail.com Fri Dec 7 18:48:22 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Fri, 7 Dec 2012 13:48:22 -0500 Subject: [katello-devel] Rails 3.2 features In-Reply-To: <211697786.42812901.1354900515970.JavaMail.root@redhat.com> References: <18062358.42808689.1354899800009.JavaMail.root@redhat.com> <211697786.42812901.1354900515970.JavaMail.root@redhat.com> Message-ID: My previous experience porting a 3.0 app to a 3.2 application wasn't too shabby in the end. I only had to change application.rb and the rest was all asset pipeline related which just involved mostly moving files around and creating manifests for the Javacript files. -Eric On Fri, Dec 7, 2012 at 12:15 PM, David Davis wrote: > I thought I'd send a second email in case you guys happened to miss the > first one. > > David > > ----- Original Message ----- > > From: "David Davis" > > To: katello-devel at redhat.com > > Sent: Friday, December 7, 2012 12:03:20 PM > > Subject: [katello-devel] Rails 3.2 features > > > > I just wanted to make a list of features that we'll get upgrading to > > Rails 3.2. I'm sending these out to inform everyone but I think it's > > worth reiterating that it's going to be painful to upgrade. It won't > > be as bad as going from Rails 2 to Rails 3 but it's not going to be > > easy either. Without futher ado... > > > > * Probably the biggest thing is the asset pipeline. It does > > everything from compiling languages (like SASS) to uglifying js > > code. Check out more info at > > http://guides.rubyonrails.org/asset_pipeline.html > > * New migration features. You no longer have to write up/down methods > > for the most part. Instead you just write "change" methods. More > > info at http://guides.rubyonrails.org/migrations.html > > * Multiple levels of has_many :through > > * Create key value stores in the database > > ( > http://blog.rawonrails.com/2012/02/using-activerecordstore-with-rails-32.html > ) > > * New active record methods like pluck > > ( > http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-pluck > ) > > * Scoped mass assignment. Helpful for locking down attributes per > > permission. > > ( > http://launchware.com/articles/whats-new-in-edge-scoped-mass-assignment-in-rails-3-1 > ) > > * Slow query logger > > > http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/. > > I've used http://explain.depesz.com/ to help analyze query explain > > output from postgres. > > * Encrypted fields (passwords, etc) > > http://railscasts.com/episodes/270-authentication-in-rails-3-1 > > * New generator stuff. You can do "rails g scaffold Person name > > email" instead of "rails g scaffold Person name:string > > email:string". Or if you want to be really crafty: rails g scaffold > > Person name:index email:uniq. > > > > David > > > > David > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From omaciel at redhat.com Fri Dec 7 19:12:18 2012 From: omaciel at redhat.com (Og Maciel) Date: Fri, 7 Dec 2012 14:12:18 -0500 (EST) Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C2394C.3050106@redhat.com> Message-ID: <1872798003.20901748.1354907538489.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Mike McCune" > To: katello-devel at redhat.com > Sent: Friday, December 7, 2012 1:45:32 PM > Subject: Re: [katello-devel] Katello 1.2 Release Candidate > > 3) katello-configure failed on EL6: > > Starting Katello configuration > The top-level log file is > [/var/log/katello/katello-configure-20121207-102354/main.log] > Creating Candlepin database user > ############################################################ ... OK > Creating Katello database user > ############################################################ ... OK > Creating Katello database > ############################################################ ... OK > Populating Katello database schema > ############################################################ ... OK > Initializing Katello data > ############################################################ ... OK > err: /Stage[main]/Katello::Service/Service[katello-jobs]/ensure: > change > from stopped to running failed: Could not start > Service[katello-jobs]: > Execution of '/sbin/service katello-jobs start' returned 1: at > /usr/share/katello/install/puppet/modules/katello/manifests/service.pp:21 > > which looks like it is that cp/reporter/core error that I thought got > removed from the Gemfile: hmmmm this worked for me on a RHEL 6.3 system -- Og Maciel Senior QA Engineer Red Hat, Inc. +1.646.707.7723 irc: omaciel From bkearney at redhat.com Fri Dec 7 19:19:31 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Fri, 07 Dec 2012 14:19:31 -0500 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C2394C.3050106@redhat.com> References: <50C2394C.3050106@redhat.com> Message-ID: <50C24143.9060207@redhat.com> On 12/07/2012 01:45 PM, Mike McCune wrote: > On 12/07/2012 08:16 AM, Eric D Helms wrote: >> Howdy All, >> >> The first release candidate for Katello 1.2 is now available. There are >> candidates for: >> >> Fedora 16 - >> http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm >> >> >> Fedora 17 - >> http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm >> >> >> RHEL 6 - >> http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm >> >> >> Repos: >> http://fedorapeople.org/groups/katello/releases/yum/1.2/ >> >> Installation and Upgrade Instructions: >> https://fedorahosted.org/katello/wiki/Install >> >> Report Bugs: >> https://fedorahosted.org/katello/wiki/ReportErrors >> >> >> If you have the bandwidth, please test installations, upgrades and run >> any test suites you may have or have access to. It may also help to >> reply to this email claiming a particular test path to help spread out >> and cover more testing ground. >> >> If there are install, or usage breaking bugs that are found, they will >> follow the method of being fixed upstream in master and cherry-picked >> into the 1.2 branch. >> > > > some initial notes: > > > 1) I noticed the repo file for 1.2 still points at the nightly URL: > > [root at g-12070948 ~]# rpm -q katello-repos > katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch > [root at g-12070948 ~]# grep baseurl /etc/yum.repos.d/katello.repo > baseurl=http://fedorapeople.org/groups/katello/releases/yum/nightly/RHEL/$releasever/$basearch/ > > baseurl=http://fedorapeople.org/groups/katello/releases/source/nightly/RHEL/$releasever/ > > > > 2) katello-devel-all fails to install on F17: > > --> Finished Dependency Resolution > Error: Package: rubygem-jshintrb-0.2.1-2.fc17.noarch (katello) > Requires: rubygem(multi_json) >= 1.3 > Installed: rubygem-multi_json-1.0.3-5.fc17.noarch (@fedora) > rubygem(multi_json) = 1.0.3 > You could try using --skip-broken to work around the problem > You could try running: rpm -Va --nofiles --nodigest > > 3) katello-configure failed on EL6: > > Starting Katello configuration > The top-level log file is > [/var/log/katello/katello-configure-20121207-102354/main.log] > Creating Candlepin database user > ############################################################ ... OK > Creating Katello database user > ############################################################ ... OK > Creating Katello database > ############################################################ ... OK > Populating Katello database schema > ############################################################ ... OK > Initializing Katello data > ############################################################ ... OK > err: /Stage[main]/Katello::Service/Service[katello-jobs]/ensure: change > from stopped to running failed: Could not start Service[katello-jobs]: > Execution of '/sbin/service katello-jobs start' returned 1: at > /usr/share/katello/install/puppet/modules/katello/manifests/service.pp:21 > > which looks like it is that cp/reporter/core error that I thought got > removed from the Gemfile: > >> Exiting! > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > `strict_error': Gem loading error: no such file to load -- > ci/reporter/core (RuntimeError) > Same on Fedora 16. -- bk From thomasmckay at redhat.com Fri Dec 7 22:20:47 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Fri, 7 Dec 2012 17:20:47 -0500 (EST) Subject: [katello-devel] rake gettext:find no longer pulling strings from haml files In-Reply-To: <1560831486.14974647.1354821200169.JavaMail.root@redhat.com> Message-ID: <624473279.15562031.1354918847891.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Tom McKay" > To: "katello-devel" > Sent: Thursday, December 6, 2012 2:13:20 PM > Subject: rake gettext:find no longer pulling strings from haml files > > At some point it seems gettext:find stopped working on haml files. > The following string does not make it to src/locale/app.pot > > app/views/systems/_edit.html.haml: - help_message = > _("Setting the release version limits content to this version only, > preventing newer packages from being available for installation.") > > Does anyone recall making any changes that may have impacted this? > When did we go from gettext to fast_gettext for example? > > My versions > > $ rpm -qa | grep gettext > gettext-libs-0.17-16.el6.x86_64 > rubygem-gettext-2.1.0-5.el6.noarch > gettext-0.17-16.el6.x86_64 > gettext-devel-0.17-16.el6.x86_64 > rubygem-gettext-doc-2.1.0-5.el6.noarch > rubygem-fast_gettext-0.5.10-1.el6.noarch > rubygem-gettext_i18n_rails-0.2.13-1.el6.noarch > The problem I experienced was due to the version of ruby_parser used. Using 2.3.1 and all strings are pulled as expected from the haml files. Using 3.0.4 or 3.1.0 and no strings from any haml files are extracted. It is not clear to me what is wrong. From daviddavis at redhat.com Sat Dec 8 15:35:02 2012 From: daviddavis at redhat.com (David Davis) Date: Sat, 8 Dec 2012 10:35:02 -0500 (EST) Subject: [katello-devel] My development workflow In-Reply-To: <1466406852.42926989.1354912173032.JavaMail.root@redhat.com> Message-ID: <25780420.44477494.1354980902681.JavaMail.root@redhat.com> I just made a screencast of how to develop using vim, tmux, and vimux. It's pretty useful for running tests (Rspec, MiniTest, nosetests) without having to leave the text editor. https://vimeo.com/55122881 David From lzap at redhat.com Mon Dec 10 08:07:31 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 09:07:31 +0100 Subject: [katello-devel] My development workflow In-Reply-To: <25780420.44477494.1354980902681.JavaMail.root@redhat.com> References: <1466406852.42926989.1354912173032.JavaMail.root@redhat.com> <25780420.44477494.1354980902681.JavaMail.root@redhat.com> Message-ID: <20121210080731.GA1721@lzapx.brq.redhat.com> Nice. If you don't use screen/tmux and like to have IDE-like terminal (maybe on a different screen or monitor), check out my VIDE program I wrote a while ago: https://github.com/lzap/vide Works with Vim or other editors too. LZ On Sat, Dec 08, 2012 at 10:35:02AM -0500, David Davis wrote: > I just made a screencast of how to develop using vim, tmux, and vimux. It's pretty useful for running tests (Rspec, MiniTest, nosetests) without having to leave the text editor. > > https://vimeo.com/55122881 > > David > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 09:04:42 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 10:04:42 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C2394C.3050106@redhat.com> References: <50C2394C.3050106@redhat.com> Message-ID: <20121210090442.GB1721@lzapx.brq.redhat.com> > 2) katello-devel-all fails to install on F17: > > --> Finished Dependency Resolution > Error: Package: rubygem-jshintrb-0.2.1-2.fc17.noarch (katello) > Requires: rubygem(multi_json) >= 1.3 > Installed: rubygem-multi_json-1.0.3-5.fc17.noarch (@fedora) > rubygem(multi_json) = 1.0.3 > You could try using --skip-broken to work around the problem > You could try running: rpm -Va --nofiles --nodigest Are we releasing Katello 1.2 for Fedora 17? Because Foreman will not start, installer will eventually fail AFAIK. > 3) katello-configure failed on EL6: > > Starting Katello configuration > The top-level log file is > [/var/log/katello/katello-configure-20121207-102354/main.log] > Creating Candlepin database user > ############################################################ ... OK > Creating Katello database user > ############################################################ ... OK > Creating Katello database > ############################################################ ... OK > Populating Katello database schema > ############################################################ ... OK > Initializing Katello data > ############################################################ ... OK > err: /Stage[main]/Katello::Service/Service[katello-jobs]/ensure: > change from stopped to running failed: Could not start > Service[katello-jobs]: Execution of '/sbin/service katello-jobs > start' returned 1: at /usr/share/katello/install/puppet/modules/katello/manifests/service.pp:21 > > which looks like it is that cp/reporter/core error that I thought > got removed from the Gemfile: > > >Exiting! > /usr/lib/ruby/gems/1.8/gems/bundler_ext-0.2.0/lib/aeolus/ext/bundler_ext.rb:48:in > `strict_error': Gem loading error: no such file to load -- > ci/reporter/core (RuntimeError) > Gemfile.in: > > # The following groups are not packaged as RPMs and are used only in > Bundler mode > group :ci do > # needed by hudson > gem 'ci_reporter', '~> 1.7.2', :require => 'ci/reporter/core' > end This has been already fixed in master last week, I am unable to reproduce both with Nightly and 1.2 on RHEL 6.3. The section was already moved into bundler.d/build.rb: http://github.com/Katello/katello/commit/e3400be0767f85e358ec7aeabfcd3d0f0f30c96e It might not get into 1.2 branch. When did you branch Eric? I am not finished with all my mails yet, but I did not see any mail in this regard. Speaking about katello-devel-* packages, there were some problems with two ruby native libraries. Mirek was this sorted out? It might miss branching too. Trying fedoras now. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 09:11:06 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 10:11:06 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: References: Message-ID: <20121210091106.GC1721@lzapx.brq.redhat.com> Created pad: https://pad-katello.rhcloud.com/p/katello-1.2rc1-testing LZ On Fri, Dec 07, 2012 at 11:16:42AM -0500, Eric D Helms wrote: > Howdy All, > > The first release candidate for Katello 1.2 is now available. There are > candidates for: > > Fedora 16 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm > > Fedora 17 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm > > RHEL 6 - > http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm > > Repos: > http://fedorapeople.org/groups/katello/releases/yum/1.2/ > > Installation and Upgrade Instructions: > https://fedorahosted.org/katello/wiki/Install > > Report Bugs: > https://fedorahosted.org/katello/wiki/ReportErrors > > > If you have the bandwidth, please test installations, upgrades and run any > test suites you may have or have access to. It may also help to reply to > this email claiming a particular test path to help spread out and cover > more testing ground. > > If there are install, or usage breaking bugs that are found, they will > follow the method of being fixed upstream in master and cherry-picked into > the 1.2 branch. > > Thanks, > Eric > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 09:46:40 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 10:46:40 +0100 Subject: [katello-devel] Katello upgrade and Puppet In-Reply-To: <50C1FFF2.80104@redhat.com> References: <20121205153313.GI2315@lzapx.brq.redhat.com> <50C1BBD3.3070602@redhat.com> <50C1FFF2.80104@redhat.com> Message-ID: <20121210094640.GD1721@lzapx.brq.redhat.com> Will do. FYI there was a refactoring of katello-upgrade script about month ago. I see you are using the older version of it. It resolved several issues with had, but still this approach is not perfect (described in this thread). LZ On Fri, Dec 07, 2012 at 09:40:50AM -0500, Michael Orazi wrote: > On 12/07/2012 04:50 AM, Petr Chalupa wrote: > > > > > >On 05.12.12 16:33, Lukas Zapletal wrote: > >>Hi, > >> > >>recent Petr's changes showed that katello-upgrade does not play well > >>with puppet. We have touched this issue several times now: > >> > >>- change touches a puppet template > >>- katello upgrade does this then: > >> - stops services > >> - executes our upgrade scripts > >> - migrates databases > >> - redeploys puppet (katello-configure) > >> > >>When the change in a puppet template is not compatible (e.g. some > >>refactoring), migration fails because Katello won't boot with this > >>(wrong) configuration since codebase was already upgraded with yum. > >>We need to create an upgrade script that somehow corrects the > >>configuration file until it's redeployed via katello configure. > >> > >>Two solutions I see: > >> > >>1) Call puppet to only deploy configuration files and make this as a > >>first step before anything happens. I have been discussing this with > >>Foreman devs little while back, there are some features of puppet we > >>could use for this but this is not how they should be used. Also seems > >>to be "hacky". > > > >I would look into this one, because it would fix the problem with old > >config for me and I could merge configuration cleanup. > > > >>2) Have full upgrade process in puppet. Very challenging, I can't really > >>believe this is even doable - we need to deploy configuration, then > >>execute some steps in particular order, then restart services. One big > >>disadvantage is the upgrade process cannot be interactive anymore I > >>believe - just fire and wait. > >> > >>Opinions? > >> > > > >Anyway I think we have issues with katello-configuration and > >katello-upgrade too often. Maybe we should spend some tome on finding a > >better solution. > > > >_______________________________________________ > >katello-devel mailing list > >katello-devel at redhat.com > >https://www.redhat.com/mailman/listinfo/katello-devel > > > Please keep the updates flowing for any big changes to > katello-configuration/upgrade as the aeolus variants try to seem > somewhat similar so that folks who use both have some amount of > similarity with the configuration & upgrade cycle on both projects. > > I've cc:ed a few folks involved in configure/upgrade on the aeolus > side who might have some input regarding the problems you are facing > presently or would probably like to be involved in any ongoing > conversations involving fundamental changes to configuration and > updating of the applications. > > Thanks, > Mike > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 11:57:33 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 12:57:33 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <20121210091106.GC1721@lzapx.brq.redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> Message-ID: <20121210115733.GH1721@lzapx.brq.redhat.com> So I have tested all platforms and it's working for me (I did not use katello-repos) except: - katello-devel-all won't work (restart error due to dependency issue) - installation on F17 fails (acts_as_reportable dep missing) It's on the ehterpad there. LZ On Mon, Dec 10, 2012 at 10:11:06AM +0100, Lukas Zapletal wrote: > Created pad: > > https://pad-katello.rhcloud.com/p/katello-1.2rc1-testing > > LZ > > On Fri, Dec 07, 2012 at 11:16:42AM -0500, Eric D Helms wrote: > > Howdy All, > > > > The first release candidate for Katello 1.2 is now available. There are > > candidates for: > > > > Fedora 16 - > > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm > > > > Fedora 17 - > > http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm > > > > RHEL 6 - > > http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm > > > > Repos: > > http://fedorapeople.org/groups/katello/releases/yum/1.2/ > > > > Installation and Upgrade Instructions: > > https://fedorahosted.org/katello/wiki/Install > > > > Report Bugs: > > https://fedorahosted.org/katello/wiki/ReportErrors > > > > > > If you have the bandwidth, please test installations, upgrades and run any > > test suites you may have or have access to. It may also help to reply to > > this email claiming a particular test path to help spread out and cover > > more testing ground. > > > > If there are install, or usage breaking bugs that are found, they will > > follow the method of being fixed upstream in master and cherry-picked into > > the 1.2 branch. > > > > Thanks, > > Eric > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From msuchy at redhat.com Mon Dec 10 12:40:48 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Mon, 10 Dec 2012 13:40:48 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: References: Message-ID: <50C5D850.5010405@redhat.com> RHEL6: Test suite Config Templates -------------------------------------------------------------------------------- config_template try create not valid [ OK ] config_template create [ OK ] config_template create snippet [ OK ] config_template info [ FAILED ] config_template info --name=a_P8jkxrHlOP0K error: %d format: a number is required, not NoneType (more in the log file /var/log/katello/client.log) config_template update name [ FAILED ] config_template update --name=a_P8jkxrHlOP0K --new_name=b_P8jkxrHlOP0K Foreman::ConfigTemplate with id '24' is invalid: - Snippet Invalid parameter 'snippet' value nil: Must be 'true' or 'false' config_template update template [ FAILED ] config_template update --name=b_P8jkxrHlOP0K --template=new test template Foreman::ConfigTemplate with id 'b_P8jkxrHlOP0K' not found config_template list [ OK ] config_template try to delete old name [ FAILED ] config_template delete --name=a_P8jkxrHlOP0K Successfuly deleted Config Template [ a_P8jkxrHlOP0K ] config_template delete [ FAILED ] config_template delete --name=b_P8jkxrHlOP0K Foreman::ConfigTemplate with id 'b_P8jkxrHlOP0K' not found config_template delete snippet [ OK ] Fedora 16: AVC messages found during installation. The same CLI test fails as above Fedora 17: Non-fatal POSTTRANS scriptlet failure in rpm package pulp-selinux-server-1.1.12-1.fc17.noarch Fail: AVC messages found during installation CLI test: 312 tests, 289 failed -- Miroslav Suchy Red Hat Systems Management Engineering From lzap at redhat.com Mon Dec 10 13:59:06 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 14:59:06 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C5D850.5010405@redhat.com> References: <50C5D850.5010405@redhat.com> Message-ID: <20121210135906.GI1721@lzapx.brq.redhat.com> On Mon, Dec 10, 2012 at 01:40:48PM +0100, Miroslav Suchy wrote: > RHEL6: > > > Test suite Config Templates Known issue, Dmitri working on it I believe. > Fedora 16: > AVC messages found during installation. > The same CLI test fails as above The same here. > Fedora 17: > Non-fatal POSTTRANS scriptlet failure in rpm package > pulp-selinux-server-1.1.12-1.fc17.noarch > Fail: AVC messages found during installation > CLI test: 312 tests, 289 failed I was not able to install. My error is on the pad. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 14:14:16 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 15:14:16 +0100 Subject: [katello-devel] Registering pre-registered system (uh) Message-ID: <20121210141416.GJ1721@lzapx.brq.redhat.com> Okay, strange subject but... Well, I have added new system using CLI/UI with name "XYZ" and then registered a system using rhsm under this name. New system entry is created while I'd expected the one I have created is reused. Is this correct behavior? I'd expect the entry in Katello to be reused. That would question the New System button in Katello completely. ps - the same for hostname as name -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Mon Dec 10 14:20:47 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Mon, 10 Dec 2012 09:20:47 -0500 Subject: [katello-devel] Registering pre-registered system (uh) In-Reply-To: <20121210141416.GJ1721@lzapx.brq.redhat.com> References: <20121210141416.GJ1721@lzapx.brq.redhat.com> Message-ID: <50C5EFBF.9050301@redhat.com> On 12/10/2012 09:14 AM, Lukas Zapletal wrote: > Okay, strange subject but... > > Well, I have added new system using CLI/UI with name "XYZ" and then > registered a system using rhsm under this name. New system entry is > created while I'd expected the one I have created is > reused. > > Is this correct behavior? We do have a duplicate issue. The only way to work around this is to use the UUID during registration, which will pull down the latest version of the consumer with the same UUID. > > I'd expect the entry in Katello to be reused. That would question the > New System button in Katello completely. > > ps - the same for hostname as name > We have the same "duplicate system" issue which spacewalk has, unfortunately. -- bk From gkhachik at redhat.com Mon Dec 10 14:18:56 2012 From: gkhachik at redhat.com (Garik Khachikyan) Date: Mon, 10 Dec 2012 15:18:56 +0100 Subject: [katello-devel] Registering pre-registered system (uh) In-Reply-To: <20121210141416.GJ1721@lzapx.brq.redhat.com> References: <20121210141416.GJ1721@lzapx.brq.redhat.com> Message-ID: <50C5EF50.8020504@redhat.com> On 10/12/12 15:14, Lukas Zapletal wrote: > Okay, strange subject but... > > Well, I have added new system using CLI/UI with name "XYZ" and then > registered a system using rhsm under this name. New system entry is > created while I'd expected the one I have created is > reused. > > Is this correct behavior? > > I'd expect the entry in Katello to be reused. That would question the > New System button in Katello completely. > > ps - the same for hostname as name > I think, it is already long time that we allow to prepare system duplicate names within org and (even) environment. So yes - it looks ok. For reusing there is: --consumerid in RHSM register call. Hope it helps. Garik From cperry at redhat.com Mon Dec 10 14:37:55 2012 From: cperry at redhat.com (Cliff Perry) Date: Mon, 10 Dec 2012 14:37:55 +0000 Subject: [katello-devel] Registering pre-registered system (uh) In-Reply-To: <50C5EFBF.9050301@redhat.com> References: <20121210141416.GJ1721@lzapx.brq.redhat.com> <50C5EFBF.9050301@redhat.com> Message-ID: <50C5F3C3.5030302@redhat.com> On 12/10/2012 02:20 PM, Bryan Kearney wrote: > On 12/10/2012 09:14 AM, Lukas Zapletal wrote: >> Okay, strange subject but... >> >> Well, I have added new system using CLI/UI with name "XYZ" and then >> registered a system using rhsm under this name. New system entry is >> created while I'd expected the one I have created is >> reused. >> >> Is this correct behavior? > > We do have a duplicate issue. The only way to work around this is to use > the UUID during registration, which will pull down the latest version of > the consumer with the same UUID. > Any way to make this happen automagically? (my guess is new code in both client and server). - I don't quite know/understand why you would allow to manually create a system profile within UI, prior to the attempted registration of said system. But the correct way today to do this, seems to be by documenting the usage of --consumerid at time of registering system with rhsm. >> >> I'd expect the entry in Katello to be reused. That would question the >> New System button in Katello completely. >> >> ps - the same for hostname as name >> > > We have the same "duplicate system" issue which spacewalk has, > unfortunately. Yes. I know today the rhsm client code does a good job in trying to prevent duplicates. I guess, the biggest kicker is, what is the use case in allowing the creation in UI prior to registration. And then if that use case is valid, is there a way to get it to do the right thing and not create a duplicate. Cliff > > -- bk > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From bkearney at redhat.com Mon Dec 10 14:43:55 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Mon, 10 Dec 2012 09:43:55 -0500 Subject: [katello-devel] Registering pre-registered system (uh) In-Reply-To: <50C5F3C3.5030302@redhat.com> References: <20121210141416.GJ1721@lzapx.brq.redhat.com> <50C5EFBF.9050301@redhat.com> <50C5F3C3.5030302@redhat.com> Message-ID: <50C5F52B.3070205@redhat.com> On 12/10/2012 09:37 AM, Cliff Perry wrote: > On 12/10/2012 02:20 PM, Bryan Kearney wrote: >> On 12/10/2012 09:14 AM, Lukas Zapletal wrote: >>> Okay, strange subject but... >>> >>> Well, I have added new system using CLI/UI with name "XYZ" and then >>> registered a system using rhsm under this name. New system entry is >>> created while I'd expected the one I have created is >>> reused. >>> >>> Is this correct behavior? >> >> We do have a duplicate issue. The only way to work around this is to use >> the UUID during registration, which will pull down the latest version of >> the consumer with the same UUID. >> > Any way to make this happen automagically? (my guess is new code in both > client and server). > > - I don't quite know/understand why you would allow to manually create > a system profile within UI, prior to the attempted registration of said > system. But the correct way today to do this, seems to be by documenting > the usage of --consumerid at time of registering system with rhsm. My guess is that this is background for the design on how to register a system record and then provision it via foreman and have it all "just work" > >>> >>> I'd expect the entry in Katello to be reused. That would question the >>> New System button in Katello completely. >>> >>> ps - the same for hostname as name >>> >> >> We have the same "duplicate system" issue which spacewalk has, >> unfortunately. > Yes. I know today the rhsm client code does a good job in trying to > prevent duplicates. > > I guess, the biggest kicker is, what is the use case in allowing the > creation in UI prior to registration. And then if that use case is > valid, is there a way to get it to do the right thing and not create a > duplicate. Provisioning. You create the system record, rack the machine, turn it on, and Katello "does the needful". This is how foreman works today. -- bk From ehelms at redhat.com Mon Dec 10 15:01:37 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 10 Dec 2012 10:01:37 -0500 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <20121210115733.GH1721@lzapx.brq.redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> Message-ID: <50C5F951.8000204@redhat.com> On 12/10/2012 06:57 AM, Lukas Zapletal wrote: > So I have tested all platforms and it's working for me (I did not use > katello-repos) except: > > - katello-devel-all won't work (restart error due to dependency issue) > - installation on F17 fails (acts_as_reportable dep missing) I hit this second issue myself and it appears to stem from fastercsv missing on F17 installations. I got past this error once I manually yum installed rubygem-fastercsv, and after looking over the changelog for Ruport, fastercsv looks like it was not included as a dependency for ruby > 1.9. Mirek can you speak to this as it looks like you added this? After manually installing rubygem-fastercsv, I hit an SSL issue during db_seed with Foreman. I think this is potentially a known issue with SSL support for F17 - I'll leave Mike to comment since I believe he mentioned this to me. As a whole, the F17 support is experimental and may not make the final release, but I wanted it to be present for testing. And if the issues are SSL related, is there a work around that we could release F17 support without SSL for now? -Eric > It's on the ehterpad there. > > LZ > > On Mon, Dec 10, 2012 at 10:11:06AM +0100, Lukas Zapletal wrote: >> Created pad: >> >> https://pad-katello.rhcloud.com/p/katello-1.2rc1-testing >> >> LZ >> >> On Fri, Dec 07, 2012 at 11:16:42AM -0500, Eric D Helms wrote: >>> Howdy All, >>> >>> The first release candidate for Katello 1.2 is now available. There are >>> candidates for: >>> >>> Fedora 16 - >>> http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.3-1.fc16.noarch.rpm >>> >>> Fedora 17 - >>> http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.3-1.fc17.noarch.rpm >>> >>> RHEL 6 - >>> http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.1-1.git.34.5c113ae.el6.noarch.rpm >>> >>> Repos: >>> http://fedorapeople.org/groups/katello/releases/yum/1.2/ >>> >>> Installation and Upgrade Instructions: >>> https://fedorahosted.org/katello/wiki/Install >>> >>> Report Bugs: >>> https://fedorahosted.org/katello/wiki/ReportErrors >>> >>> >>> If you have the bandwidth, please test installations, upgrades and run any >>> test suites you may have or have access to. It may also help to reply to >>> this email claiming a particular test path to help spread out and cover >>> more testing ground. >>> >>> If there are install, or usage breaking bugs that are found, they will >>> follow the method of being fixed upstream in master and cherry-picked into >>> the 1.2 branch. >>> >>> Thanks, >>> Eric >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> -- >> Later, >> >> Lukas "lzap" Zapletal >> #katello #systemengine >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel From lzap at redhat.com Mon Dec 10 15:06:19 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 16:06:19 +0100 Subject: [katello-devel] Registering pre-registered system (uh) In-Reply-To: <50C5EFBF.9050301@redhat.com> References: <20121210141416.GJ1721@lzapx.brq.redhat.com> <50C5EFBF.9050301@redhat.com> Message-ID: <20121210150619.GM1721@lzapx.brq.redhat.com> On Mon, Dec 10, 2012 at 09:20:47AM -0500, Bryan Kearney wrote: > We do have a duplicate issue. The only way to work around this is to > use the UUID during registration, which will pull down the latest > version of the consumer with the same UUID. Ok this was what I was looking for! LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 10 15:08:05 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 10 Dec 2012 16:08:05 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C5F951.8000204@redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> Message-ID: <20121210150805.GN1721@lzapx.brq.redhat.com> On Mon, Dec 10, 2012 at 10:01:37AM -0500, Eric Helms wrote: > As a whole, the F17 support is experimental and may not make the > final release, but I wanted it to be present for testing. And if > the issues are SSL related, is there a work around that we could > release F17 support without SSL for now? > +1 I am also for "tech preview" release. Maybe holding it a bit until we build the last missing bit (which is Foreman F17 support). On the other hand it would be nice to have it, at least for users to check Katello 1.2 out. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From gkhachik at redhat.com Mon Dec 10 15:37:16 2012 From: gkhachik at redhat.com (Garik Khachikyan) Date: Mon, 10 Dec 2012 16:37:16 +0100 Subject: [katello-devel] Katello upgrade - for client systems? Message-ID: <50C601AC.3000107@redhat.com> Hi all. My $subj question could sound a little bit naive but: Are there any instructions of upgrading client systems to the latest? Is it suffice saying: "just run `yum -y update subscription-manager python-rhsm katello-agent katello-cli*` and you're good to go?" Would be there all the configuration and certificates remain unchanged? Thanks in advance. Garik From pchalupa at redhat.com Mon Dec 10 16:09:00 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Mon, 10 Dec 2012 17:09:00 +0100 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50C1C0A1.5040207@redhat.com> References: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> <50BF408E.1030609@redhat.com> <50C1A8FB.4000405@redhat.com> <50C1C0A1.5040207@redhat.com> Message-ID: <50C6091C.8080703@redhat.com> I'll try to explain better my reasoning. ## Facts I see: 1. As Bryan said we have to have API documentation. 2. API documentation is loosing value when it's outdated. 3. API request should be validated and a client should get readable error when its format isn't right. 4. A client is talking to API. 5. Models should be hidden from client. 6. Business logic should be captured and enforced by models. 7. An API request validation can be more complex or simpler than a model validation. (complex example: batch update of resources; simple example: update of a model where only a subset of attributes is exposed) 8. It's not a model responsibility to know how it's exposed via API. 9. Duplication is bad. ## Implications: Points 5,6,7,8 implies that there should be another layer between client and business logic, which is controllers. Controllers with help of Apipie are defining the API. To avoid duplication (9) there has to be one source of information how the API is specified (which is current done by Apipie's DSL in controllers). (Models are not suitable for this because of 7. and 8.) API specification is used both for generating API documentation and request validation. That ties them together for documentation or validation not to become outdated. There is a duplication (9) between validation definition of modes and API specification of request parts representing models. This can be fixed by introducing helper to Apipie's DSL which will get the information form models and reuse it for desired attributes. The example how the helper would look like: # in a controller api :POST, "/users", "Create an user" # use :only option to specify which attributes to use, # others are ignored param_by_model :user, :only => [:disabled, :email, :password, :username, :default_environment_id] do param :disabled, :bool # override default behavior if needed end def create # ... end The helper only adds simple validations that: param is present, param has correct type, param is not unknown. This ensures that models will get correct hash of attributes, which I think is the main purpose to ensure that model methods are called with valid data. More complex validations are done by model. To comply with 3,4,5 ActiveRecord::RecordInvalid should be handled same way as Apipie::ParamInvalid (easily doable by .rescue_from). ## Todo There are not many tasks left to do: - add the helper - report ActiveRecord::RecordInvalid in same way as Apipie::ParamInvalid - re-enable Apipie validation ## Disadvantages of disabling Apipie validations - Outdated documentation unless it's periodically updated and checked - less useful errors messages to a client - duplication (API documentation vs model validations, and other checks in controllers) - model errors are exposed to a client - danger of adding another responsibilities to our models to handle this On 07.12.12 11:10, Dmitri Dolguikh wrote: > On 07/12/12 08:29 AM, Petr Chalupa wrote: >> >> >> On 05.12.12 13:39, Dmitri Dolguikh wrote: >>> On 05/12/12 12:20 PM, Ohad Levy wrote: >>>> >>>> ----- Original Message ----- >>>> | On 05/12/12 09:07 AM, Petr Chalupa wrote: >>>> | > I agree that domain validation/consistency checks belongs to the >>>> | > models. Nevertheless definition of the API is other thing entirely. >>>> | > It >>>> | > should be defined and validated on its own with re-usage of >>>> | > information provided by models (domain) to avoid duplications. It >>>> | > should not be delegated to model to deal with it by adding other >>>> | > responsibility. >>>> | >>>> | This statement is only valid if we are talking about cross-concerns >>>> | (think authentication, security-related stuff, etc). Everything else >>>> | will result in duplicated, non-consistent validations. Take a look at >>>> | current state of validations in foreman api. >>>> | >>>> What do you mean by that? >>> >>> Which part? I insist that controllers should not carry any business >>> logic in them, as a corollary to this, no validations should reside in >>> controllers, unless those are used across all controllers, and are >>> outside of business-logic constraints (think aspect-like behaviour). >> >> Yes I agree that controllers should not carry any business logic, but >> they can define an API and they can validate if an user of the API is >> using it right, meaning validate the request before any other action >> is taken in domain. > > I have no clue what you mean by "validating API". At all. Nada. I mean validation of incoming JSON requests that they have correct structure. > > Again: it's ok to validate cross-converns in controllers, those are > outside of model/business logic. Everything else is > duplicating/relocating business logic into the controllers. What I am talking about is not duplication or relocation, it's a simple re-usage of information. Business logic would be still described and enforced by models. > It's not apipie's responsibility to enforce validations. Drop this > functionality, or I'll be asking the team next to remove apipie > *completely*, as its developers are not being reasonable. This is only a cosmetic issue of placement. If you insist you can extract API request validations to another rubygem. > >> The issue is that the API validations are very similar to model >> validations in simple crud actions and we aren't using the model >> validation definition to construct API validations dynamically, we >> have to write them and that is the duplicity. This can be fixed by >> retrieving the already present information from models and construct >> API validations dynamically. >> >> I would focus fixing [1] soon, after that we won't have any duplicity >> issue. > > Don't we have more pressing stuff to do? > >> >> [1] https://github.com/Pajk/apipie-rails/issues/67 >> >>> By current state I meant that there are currently inconsistencies (data >>> coming through api are flagged invalid, even though it's valid according >>> to validators on the model) between api and model validations. >>> Config_template has a few, for example (and what started this thread). >> >> Yes this is a problem which will be solved by creating API validations >> from model validations dynamically. We do not have to disable or stop >> using the API validations and lose its advantages. > > This makes no sense. At best you'd be running model validations in > controllers. This is wasted effort, see my reasons above. Please stop > this madness. Please be patient, this discussion is not fun for me either, but I would not continue in it unless I wasn't thinking that I have something valuable to say. I tried to explain at the top. > -d > >> >>> -d >>>> >>>> | I don't want to be jumping through hoops, and fixing/adding >>>> | code/meta-data in multiple places, so just the documentation is >>>> | up-to-date. It's a process/organizational issue, and tools do not fix >>>> | those. >>>> | >>>> | And, once again, validations belong in the model. Let's get rid of >>>> | validations in apipie, fix issues we have with domain logic leaking >>>> | into >>>> | controllers. >>>> | >>>> | > >>>> | > So I strongly suggest to fix the duplication issue [1] and keep >>>> | > things >>>> | > separate. >>>> | > * Apipie for API validations. >>>> | > * Model validations for domain logic. >>>> | > >>>> | > ## Fix Apipie >>>> | > >>>> | > Enable Apipie to read model definitions and validations to set >>>> | > param >>>> | > validations automatically for parts of request representing a >>>> | > domain >>>> | > model, see [1]. >>>> | >>>> | Agree with defining docs based on model (and I would suggest routes). >>>> | Disagree with everything else. No point in intercepting validations, >>>> | the >>>> | most you could do in the controller is map exceptions thrown by the >>>> | model to http errors/statuses. >>>> | >>>> | > >>>> | > >>>> | > + api documentation generated from api specification >>>> | > + api specification is kept up to date (has to be because >>>> | > validation >>>> | > is turned on) >>>> | > + api validation/specification is separated from domain >>>> | > validations/logic >>>> | > >>>> | > ## Remove validations from Apipie >>>> | > >>>> | > + api documentation generated from api specification >>>> | > - api specification is not kept up to date (has to be ensured in an >>>> | > extra task) >>>> | > - no api validation, it's partially handled by models, errors can >>>> | > be >>>> | > ugly, models have another responsibility >>>> | >>>> | Again, tools are not the way to enforce process deficiencies. The >>>> | last >>>> | statement is FUD. >>>> | >>>> | > >>>> | > >>>> | > [1] - https://github.com/Pajk/apipie-rails/issues/67 >>>> | > >>>> | > ## Other notes >>>> | > >>>> | > > The second issue is that apipie validations are stricter than AR >>>> | > ones: array validation assumes that there's an array in place, >>>> | > while >>>> | > AR would accept a nil. These two combined lead to even more false >>>> | > positives during apipie validation. >>>> | > >>>> | > I think be more consistent and strict is a good thing, it prevents >>>> | > confusion. Empty Array is an empty Array not a nil. >>>> | >>>> | It's not good, because depending on how you use foreman, you'll get >>>> | different results. What's considered valid/correct by foreman proper >>>> | should be valid by any other means, period. >>>> | >>>> | -d >>>> | >>>> | > >>>> | > Petr >>>> | > >>>> | > On 04.12.12 22:47, Ivan Necas wrote: >>>> | >> >>>> | >> >>>> | >> ----- Original Message ----- >>>> | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>>> | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>> | >>>>>> Step #1 - let's get rid of controller validations. >>>> | >>>>> Again, you can only do this for CRUD actions in the >>>> | >>>>> controllers. >>>> | >>>>> For all >>>> | >>>>> other methods we need validations. And we have those, not all >>>> | >>>>> our >>>> | >>>>> controllers are plain REST-driven. We do have procedure-like >>>> | >>>>> actions. >>>> | >>>>> Maybe more then we should have, but this is a fact. >>>> | >>>>> >>>> | >>>>> You say all the validations should be done in models, thus you >>>> | >>>>> essentially say let's put all our code logic there. Because it >>>> | >>>>> does not >>>> | >>>>> make sense to do validations in models while doing anything >>>> | >>>>> else >>>> | >>>>> (that >>>> | >>>>> also requires some level of validations) in controllers or a >>>> | >>>>> logic >>>> | >>>>> layer. A fact: you need to do validations also outside of >>>> | >>>>> models. >>>> | >>>> >>>> | >>>> Domain logic belongs in the model layer. We can discuss how to >>>> | >>>> better >>>> | >>>> factor model layer code, but not where to put it. >>>> | >>>> >>>> | >>>> -d >>>> | >>>> >>>> | >>>> A lot of rationalizations to keep validations in apipie have >>>> | >>>> been >>>> | >>>> put >>>> | >>>> forward, *all* of them attempting to deal with various smells, >>>> | >>>> code >>>> | >>>> or >>>> | >>>> process ones. Let's fix the root issues. >>>> | >>>> >>>> | >>>> Again, Let's get rid of validations in apipie. Refactor the >>>> | >>>> controllers when needed. If apipie is not useful without those >>>> | >>>> validations, let's suspend/stop use of it. >>>> | >> >>>> | >> Am I reading it correctly you suggesting getting rid of API >>>> | >> documentation? >>>> | >> >>>> | >> -- Ivan >>>> | >> >>>> | >>>> >>>> | >>>> -d >>>> | >>>> >>>> | >>> +1 >>>> | >>>> >>>> | >>>>> >>>> | >>>>> It is wrong to put all your code logic into models. And >>>> | >>>>> unfortunately we >>>> | >>>>> have most of it there. Rails follows MVC pattern and what we >>>> | >>>>> have >>>> | >>>>> is >>>> | >>>>> those >>>> | >>>>> messy Models-for-everything approach. If you mix this with our >>>> | >>>>> orchestration, it is deadly combination. Difficult testability, >>>> | >>>>> huge >>>> | >>>>> objects with no responsibility split, unable to mock things and >>>> | >>>>> so >>>> | >>>>> on. >>>> | >>>>> Code logic in models works well for small projects and it is >>>> | >>>>> de-facto >>>> | >>>>> standard in Rails world. People just blindly follow this wrong >>>> | >>>>> pattern. >>>> | >>>>> >>>> | >>>>> So I don't agree with this simplification you offer. We will >>>> | >>>>> likely >>>> | >>>>> start to refactor our orchestration creating new layer of code >>>> | >>>>> logic, >>>> | >>>>> which is the right approach to do things. And we should move >>>> | >>>>> complex >>>> | >>>>> methods like activation key subscription from models there. >>>> | >>>>> Then >>>> | >>>>> we will >>>> | >>>>> need validations on the controller side (or on the newly >>>> | >>>>> created >>>> | >>>>> logic >>>> | >>>>> layer or orchestration layer - depends on how are we gonna name >>>> | >>>>> it). >>>> | >>>>> >>>> | >>>>> So yes, it has something to do with REST. My counter proposal >>>> | >>>>> is: >>>> | >>>>> >>>> | >>>>> #2 Let's get rid of validations where they duplicate model >>>> | >>>>> validators, >>>> | >>>>> but let's keep them for (usually non-CRUD) methods which need >>>> | >>>>> then. We >>>> | >>>>> will likely be extending those soon. >>>> | >>>>> >>>> | >>>> >>>> | >>>> _______________________________________________ >>>> | >>>> katello-devel mailing list >>>> | >>>> katello-devel at redhat.com >>>> | >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> | >>> >>>> | >>> >>>> | >>> >>>> | >>> -- >>>> | >>> Jason E. Rist >>>> | >>> Senior Software Engineer >>>> | >>> Systems Management and Cloud Enablement >>>> | >>> Red Hat, Inc. >>>> | >>> +1.919.754.4048 >>>> | >>> Freenode: jrist >>>> | >>> github/identi.ca: knowncitizen >>>> | >>> >>>> | >>> _______________________________________________ >>>> | >>> katello-devel mailing list >>>> | >>> katello-devel at redhat.com >>>> | >>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> | >>> >>>> | >> >>>> | >> _______________________________________________ >>>> | >> katello-devel mailing list >>>> | >> katello-devel at redhat.com >>>> | >> https://www.redhat.com/mailman/listinfo/katello-devel >>>> | >> >>>> | > >>>> | > _______________________________________________ >>>> | > katello-devel mailing list >>>> | > katello-devel at redhat.com >>>> | > https://www.redhat.com/mailman/listinfo/katello-devel >>>> | >>>> | _______________________________________________ >>>> | katello-devel mailing list >>>> | katello-devel at redhat.com >>>> | https://www.redhat.com/mailman/listinfo/katello-devel >>>> | >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From daviddavis at redhat.com Mon Dec 10 16:50:39 2012 From: daviddavis at redhat.com (David Davis) Date: Mon, 10 Dec 2012 11:50:39 -0500 (EST) Subject: [katello-devel] Ruby meetup tomorrow night Message-ID: <858524662.46694149.1355158239284.JavaMail.root@redhat.com> I brought it up in the Scrum but I figured I'd send out a link to those interested. Again, tomorrow night I'll be presenting Minitest at the Ruby meetup in Raleigh. It's right across the street from the old Red Hat building. There'll be a bunch of other gems too so if you're interested in learning more about gems, Minitest, or Ruby, it's worth coming. http://www.meetup.com/raleighrb/events/91671692/ Thanks! David From msuchy at redhat.com Mon Dec 10 16:58:09 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Mon, 10 Dec 2012 17:58:09 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C5F951.8000204@redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> Message-ID: <50C614A1.3070803@redhat.com> On 12/10/2012 04:01 PM, Eric Helms wrote: > I got past this error once I manually yum installed rubygem-fastercsv, > and after looking over the changelog for Ruport, fastercsv looks like it > was not included as a dependency for ruby > 1.9. Mirek can you speak to > this as it looks like you added this? fastercsv should not be needed. And if you install it using steps documented on our wiki, it is not required. If you are using devel setup and you have fastercsv in some Gemfile, it should be removed. Of course only on F17. It should be preserved on F16 and EL6. -- Miroslav Suchy Red Hat Systems Management Engineering From msuchy at redhat.com Mon Dec 10 17:04:10 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Mon, 10 Dec 2012 18:04:10 +0100 Subject: [katello-devel] Katello upgrade - for client systems? In-Reply-To: <50C601AC.3000107@redhat.com> References: <50C601AC.3000107@redhat.com> Message-ID: <50C6160A.3010204@redhat.com> On 12/10/2012 04:37 PM, Garik Khachikyan wrote: > Are there any instructions of upgrading client systems to the latest? > > Is it suffice saying: "just run `yum -y update subscription-manager > python-rhsm katello-agent katello-cli*` and you're good to go?" always: yum upgrade :) > Would be there all the configuration and certificates remain unchanged? Configuration *can* change. Although it will not be probably case of rhsm. I always use: rpmconf -a after upgrade, to find changes in configuration and to resolve them. -- Miroslav Suchy Red Hat Systems Management Engineering From dmitri at redhat.com Mon Dec 10 17:55:51 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Mon, 10 Dec 2012 17:55:51 +0000 Subject: [katello-devel] Apipie and validation. In-Reply-To: <50C6091C.8080703@redhat.com> References: <493334260.4282191.1354710025318.JavaMail.root@redhat.com> <50BF408E.1030609@redhat.com> <50C1A8FB.4000405@redhat.com> <50C1C0A1.5040207@redhat.com> <50C6091C.8080703@redhat.com> Message-ID: <50C62227.1040008@redhat.com> On 10/12/12 04:09 PM, Petr Chalupa wrote: > I'll try to explain better my reasoning. > > ## Facts I see: > > 1. As Bryan said we have to have API documentation. > 2. API documentation is loosing value when it's outdated. > 3. API request should be validated and a client should get readable > error when its format isn't right. a nitpick: not just the format of data, but anything that renders the value invalid/illegal. > 4. A client is talking to API. > 5. Models should be hidden from client. Don't like the phrasing. Resources != models; one model (or its parts) can be present in multiple resources. > > 6. Business logic should be captured and enforced by models. > 7. An API request validation can be more complex or simpler than a > model validation. (complex example: batch update of resources; simple > example: update of a model where only a subset of attributes is exposed) Agreed with the premise, but not with the conclusion. Batch of models is a model itself, governed by business rules, and should be modeled as such. A subset of attributes is still constrained by the business rules of the complete set of attributes. A valid example is (pls. excuse the silliness): api exposes an attribute as Base64-encoded, while the model uses string. It's up to the controller to convert from one representation to another, and raise errors if that conversion fails. The controller doesn't look into the attribute itself to validate the value - that's model's job. > > 8. It's not a model responsibility to know how it's exposed via API. Contollers define representation, composition, etc of data represented in models. Nothing else. They have nothing to do with validation of models - data supplied through API ultimately ends up in model. Model still needs to validate the data. > 9. Duplication is bad. > > ## Implications: > > Points 5,6,7,8 implies that there should be another layer between > client and business logic, which is controllers. Controllers with help > of Apipie are defining the API. > > To avoid duplication (9) there has to be one source of information how > the API is specified (which is current done by Apipie's DSL in > controllers). (Models are not suitable for this because of 7. and 8.) > API specification is used both for generating API documentation and > request validation. That ties them together for documentation or > validation not to become outdated. Once again, apipie is a documentation tool, which tries to be too many things. API (in rails world) is defined by the contents of routes.rb, nothing else. Validity of data is defined by model validators. Period. > > There is a duplication (9) between validation definition of modes and > API specification of request parts representing models. This can be > fixed by introducing helper to Apipie's DSL which will get the > information form models and reuse it for desired attributes. > > The example how the helper would look like: > > # in a controller > api :POST, "/users", "Create an user" > # use :only option to specify which attributes to use, > # others are ignored > param_by_model :user, > :only => [:disabled, :email, :password, :username, > :default_environment_id] do > param :disabled, :bool # override default behavior if needed > end > def create > # ... > end > > The helper only adds simple validations that: param is present, param > has correct type, param is not unknown. This ensures that models will > get correct hash of attributes, which I think is the main purpose to > ensure that model methods are called with valid data. More complex > validations are done by model. Don't. Models can ensure they are populated with correct data. At best you are duplicating rules captured by the models. At worst you'll be introducing inconsistencies/bugs. Another thing to keep in mind that "simple" validations tend to turn into complex ones very quickly - the best thing we (api developers) can do is to return meaningful errors that point to the source of the problem. > > To comply with 3,4,5 ActiveRecord::RecordInvalid should be handled > same way as Apipie::ParamInvalid (easily doable by .rescue_from). > > ## Todo > > There are not many tasks left to do: > - add the helper > - report ActiveRecord::RecordInvalid in same way as Apipie::ParamInvalid > - re-enable Apipie validation Or, do nothing of the above and adopt the scrum solution to this organizational problem (pls. see below). > > ## Disadvantages of disabling Apipie validations > > - Outdated documentation unless it's periodically updated and checked I have a suggestion: Add another completion criteria: no story is completed until docs have been updated. > - less useful errors messages to a client FUD. Model error messages can be as good as we need them to be. > - duplication (API documentation vs model validations, and other > checks in controllers) FUD, or plain wrong. Model can handle all the validations we need. apipie is the source of duplication in this case. > - model errors are exposed to a client Perfect! Remember, rails error messages point to attributes that are in error. > - danger of adding another responsibilities to our models to handle this It's not a danger - it's model's responsibility. Again, now that the validation is off, it should stay off. -d > > On 07.12.12 11:10, Dmitri Dolguikh wrote: >> On 07/12/12 08:29 AM, Petr Chalupa wrote: >>> >>> >>> On 05.12.12 13:39, Dmitri Dolguikh wrote: >>>> On 05/12/12 12:20 PM, Ohad Levy wrote: >>>>> >>>>> ----- Original Message ----- >>>>> | On 05/12/12 09:07 AM, Petr Chalupa wrote: >>>>> | > I agree that domain validation/consistency checks belongs to the >>>>> | > models. Nevertheless definition of the API is other thing >>>>> entirely. >>>>> | > It >>>>> | > should be defined and validated on its own with re-usage of >>>>> | > information provided by models (domain) to avoid duplications. It >>>>> | > should not be delegated to model to deal with it by adding other >>>>> | > responsibility. >>>>> | >>>>> | This statement is only valid if we are talking about cross-concerns >>>>> | (think authentication, security-related stuff, etc). Everything >>>>> else >>>>> | will result in duplicated, non-consistent validations. Take a >>>>> look at >>>>> | current state of validations in foreman api. >>>>> | >>>>> What do you mean by that? >>>> >>>> Which part? I insist that controllers should not carry any business >>>> logic in them, as a corollary to this, no validations should reside in >>>> controllers, unless those are used across all controllers, and are >>>> outside of business-logic constraints (think aspect-like behaviour). >>> >>> Yes I agree that controllers should not carry any business logic, but >>> they can define an API and they can validate if an user of the API is >>> using it right, meaning validate the request before any other action >>> is taken in domain. >> >> I have no clue what you mean by "validating API". At all. Nada. > > I mean validation of incoming JSON requests that they have correct > structure. > >> >> Again: it's ok to validate cross-converns in controllers, those are >> outside of model/business logic. Everything else is >> duplicating/relocating business logic into the controllers. > > What I am talking about is not duplication or relocation, it's a > simple re-usage of information. Business logic would be still > described and enforced by models. > >> It's not apipie's responsibility to enforce validations. Drop this >> functionality, or I'll be asking the team next to remove apipie >> *completely*, as its developers are not being reasonable. > > This is only a cosmetic issue of placement. If you insist you can > extract API request validations to another rubygem. > >> >>> The issue is that the API validations are very similar to model >>> validations in simple crud actions and we aren't using the model >>> validation definition to construct API validations dynamically, we >>> have to write them and that is the duplicity. This can be fixed by >>> retrieving the already present information from models and construct >>> API validations dynamically. >>> >>> I would focus fixing [1] soon, after that we won't have any duplicity >>> issue. >> >> Don't we have more pressing stuff to do? >> >>> >>> [1] https://github.com/Pajk/apipie-rails/issues/67 >>> >>>> By current state I meant that there are currently inconsistencies >>>> (data >>>> coming through api are flagged invalid, even though it's valid >>>> according >>>> to validators on the model) between api and model validations. >>>> Config_template has a few, for example (and what started this thread). >>> >>> Yes this is a problem which will be solved by creating API validations >>> from model validations dynamically. We do not have to disable or stop >>> using the API validations and lose its advantages. >> >> This makes no sense. At best you'd be running model validations in >> controllers. This is wasted effort, see my reasons above. Please stop >> this madness. > > Please be patient, this discussion is not fun for me either, but I > would not continue in it unless I wasn't thinking that I have > something valuable to say. I tried to explain at the top. > >> -d >> >>> >>>> -d >>>>> >>>>> | I don't want to be jumping through hoops, and fixing/adding >>>>> | code/meta-data in multiple places, so just the documentation is >>>>> | up-to-date. It's a process/organizational issue, and tools do >>>>> not fix >>>>> | those. >>>>> | >>>>> | And, once again, validations belong in the model. Let's get rid of >>>>> | validations in apipie, fix issues we have with domain logic leaking >>>>> | into >>>>> | controllers. >>>>> | >>>>> | > >>>>> | > So I strongly suggest to fix the duplication issue [1] and keep >>>>> | > things >>>>> | > separate. >>>>> | > * Apipie for API validations. >>>>> | > * Model validations for domain logic. >>>>> | > >>>>> | > ## Fix Apipie >>>>> | > >>>>> | > Enable Apipie to read model definitions and validations to set >>>>> | > param >>>>> | > validations automatically for parts of request representing a >>>>> | > domain >>>>> | > model, see [1]. >>>>> | >>>>> | Agree with defining docs based on model (and I would suggest >>>>> routes). >>>>> | Disagree with everything else. No point in intercepting >>>>> validations, >>>>> | the >>>>> | most you could do in the controller is map exceptions thrown by the >>>>> | model to http errors/statuses. >>>>> | >>>>> | > >>>>> | > >>>>> | > + api documentation generated from api specification >>>>> | > + api specification is kept up to date (has to be because >>>>> | > validation >>>>> | > is turned on) >>>>> | > + api validation/specification is separated from domain >>>>> | > validations/logic >>>>> | > >>>>> | > ## Remove validations from Apipie >>>>> | > >>>>> | > + api documentation generated from api specification >>>>> | > - api specification is not kept up to date (has to be ensured >>>>> in an >>>>> | > extra task) >>>>> | > - no api validation, it's partially handled by models, errors can >>>>> | > be >>>>> | > ugly, models have another responsibility >>>>> | >>>>> | Again, tools are not the way to enforce process deficiencies. The >>>>> | last >>>>> | statement is FUD. >>>>> | >>>>> | > >>>>> | > >>>>> | > [1] - https://github.com/Pajk/apipie-rails/issues/67 >>>>> | > >>>>> | > ## Other notes >>>>> | > >>>>> | > > The second issue is that apipie validations are stricter >>>>> than AR >>>>> | > ones: array validation assumes that there's an array in place, >>>>> | > while >>>>> | > AR would accept a nil. These two combined lead to even more false >>>>> | > positives during apipie validation. >>>>> | > >>>>> | > I think be more consistent and strict is a good thing, it >>>>> prevents >>>>> | > confusion. Empty Array is an empty Array not a nil. >>>>> | >>>>> | It's not good, because depending on how you use foreman, you'll get >>>>> | different results. What's considered valid/correct by foreman >>>>> proper >>>>> | should be valid by any other means, period. >>>>> | >>>>> | -d >>>>> | >>>>> | > >>>>> | > Petr >>>>> | > >>>>> | > On 04.12.12 22:47, Ivan Necas wrote: >>>>> | >> >>>>> | >> >>>>> | >> ----- Original Message ----- >>>>> | >>> On Tue 04 Dec 2012 05:10:28 AM MST, Dmitri Dolguikh wrote: >>>>> | >>>> On 03/12/12 09:36 AM, Lukas Zapletal wrote: >>>>> | >>>>>> Step #1 - let's get rid of controller validations. >>>>> | >>>>> Again, you can only do this for CRUD actions in the >>>>> | >>>>> controllers. >>>>> | >>>>> For all >>>>> | >>>>> other methods we need validations. And we have those, not all >>>>> | >>>>> our >>>>> | >>>>> controllers are plain REST-driven. We do have procedure-like >>>>> | >>>>> actions. >>>>> | >>>>> Maybe more then we should have, but this is a fact. >>>>> | >>>>> >>>>> | >>>>> You say all the validations should be done in models, thus >>>>> you >>>>> | >>>>> essentially say let's put all our code logic there. >>>>> Because it >>>>> | >>>>> does not >>>>> | >>>>> make sense to do validations in models while doing anything >>>>> | >>>>> else >>>>> | >>>>> (that >>>>> | >>>>> also requires some level of validations) in controllers or a >>>>> | >>>>> logic >>>>> | >>>>> layer. A fact: you need to do validations also outside of >>>>> | >>>>> models. >>>>> | >>>> >>>>> | >>>> Domain logic belongs in the model layer. We can discuss how to >>>>> | >>>> better >>>>> | >>>> factor model layer code, but not where to put it. >>>>> | >>>> >>>>> | >>>> -d >>>>> | >>>> >>>>> | >>>> A lot of rationalizations to keep validations in apipie have >>>>> | >>>> been >>>>> | >>>> put >>>>> | >>>> forward, *all* of them attempting to deal with various smells, >>>>> | >>>> code >>>>> | >>>> or >>>>> | >>>> process ones. Let's fix the root issues. >>>>> | >>>> >>>>> | >>>> Again, Let's get rid of validations in apipie. Refactor the >>>>> | >>>> controllers when needed. If apipie is not useful without those >>>>> | >>>> validations, let's suspend/stop use of it. >>>>> | >> >>>>> | >> Am I reading it correctly you suggesting getting rid of API >>>>> | >> documentation? >>>>> | >> >>>>> | >> -- Ivan >>>>> | >> >>>>> | >>>> >>>>> | >>>> -d >>>>> | >>>> >>>>> | >>> +1 >>>>> | >>>> >>>>> | >>>>> >>>>> | >>>>> It is wrong to put all your code logic into models. And >>>>> | >>>>> unfortunately we >>>>> | >>>>> have most of it there. Rails follows MVC pattern and what we >>>>> | >>>>> have >>>>> | >>>>> is >>>>> | >>>>> those >>>>> | >>>>> messy Models-for-everything approach. If you mix this with >>>>> our >>>>> | >>>>> orchestration, it is deadly combination. Difficult >>>>> testability, >>>>> | >>>>> huge >>>>> | >>>>> objects with no responsibility split, unable to mock >>>>> things and >>>>> | >>>>> so >>>>> | >>>>> on. >>>>> | >>>>> Code logic in models works well for small projects and it is >>>>> | >>>>> de-facto >>>>> | >>>>> standard in Rails world. People just blindly follow this >>>>> wrong >>>>> | >>>>> pattern. >>>>> | >>>>> >>>>> | >>>>> So I don't agree with this simplification you offer. We will >>>>> | >>>>> likely >>>>> | >>>>> start to refactor our orchestration creating new layer of >>>>> code >>>>> | >>>>> logic, >>>>> | >>>>> which is the right approach to do things. And we should move >>>>> | >>>>> complex >>>>> | >>>>> methods like activation key subscription from models there. >>>>> | >>>>> Then >>>>> | >>>>> we will >>>>> | >>>>> need validations on the controller side (or on the newly >>>>> | >>>>> created >>>>> | >>>>> logic >>>>> | >>>>> layer or orchestration layer - depends on how are we gonna >>>>> name >>>>> | >>>>> it). >>>>> | >>>>> >>>>> | >>>>> So yes, it has something to do with REST. My counter proposal >>>>> | >>>>> is: >>>>> | >>>>> >>>>> | >>>>> #2 Let's get rid of validations where they duplicate model >>>>> | >>>>> validators, >>>>> | >>>>> but let's keep them for (usually non-CRUD) methods which need >>>>> | >>>>> then. We >>>>> | >>>>> will likely be extending those soon. >>>>> | >>>>> >>>>> | >>>> >>>>> | >>>> _______________________________________________ >>>>> | >>>> katello-devel mailing list >>>>> | >>>> katello-devel at redhat.com >>>>> | >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>>> | >>> >>>>> | >>> >>>>> | >>> >>>>> | >>> -- >>>>> | >>> Jason E. Rist >>>>> | >>> Senior Software Engineer >>>>> | >>> Systems Management and Cloud Enablement >>>>> | >>> Red Hat, Inc. >>>>> | >>> +1.919.754.4048 >>>>> | >>> Freenode: jrist >>>>> | >>> github/identi.ca: knowncitizen >>>>> | >>> >>>>> | >>> _______________________________________________ >>>>> | >>> katello-devel mailing list >>>>> | >>> katello-devel at redhat.com >>>>> | >>> https://www.redhat.com/mailman/listinfo/katello-devel >>>>> | >>> >>>>> | >> >>>>> | >> _______________________________________________ >>>>> | >> katello-devel mailing list >>>>> | >> katello-devel at redhat.com >>>>> | >> https://www.redhat.com/mailman/listinfo/katello-devel >>>>> | >> >>>>> | > >>>>> | > _______________________________________________ >>>>> | > katello-devel mailing list >>>>> | > katello-devel at redhat.com >>>>> | > https://www.redhat.com/mailman/listinfo/katello-devel >>>>> | >>>>> | _______________________________________________ >>>>> | katello-devel mailing list >>>>> | katello-devel at redhat.com >>>>> | https://www.redhat.com/mailman/listinfo/katello-devel >>>>> | >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ericdhelms at gmail.com Tue Dec 11 01:44:34 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Mon, 10 Dec 2012 20:44:34 -0500 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C614A1.3070803@redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> <50C614A1.3070803@redhat.com> Message-ID: The Ruport RPM has been updated in the 1.2 repositories to remove a dependency on fastercsv in the gemspec on Ruby > 1.9 systems. This means that on Fedora 17 systems, Katello should install properly, and as long as the following are run (due to SElinux issues with the F17 install): setenforce 0 service iptables stop On Mon, Dec 10, 2012 at 11:58 AM, Miroslav Such? wrote: > On 12/10/2012 04:01 PM, Eric Helms wrote: > >> I got past this error once I manually yum installed rubygem-fastercsv, >> and after looking over the changelog for Ruport, fastercsv looks like it >> was not included as a dependency for ruby > 1.9. Mirek can you speak to >> this as it looks like you added this? >> > > fastercsv should not be needed. And if you install it using steps > documented on our wiki, it is not required. > > If you are using devel setup and you have fastercsv in some Gemfile, it > should be removed. Of course only on F17. It should be preserved on F16 and > EL6. > > > -- > Miroslav Suchy > Red Hat Systems Management Engineering > > ______________________________**_________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/**mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmccune at redhat.com Tue Dec 11 05:22:33 2012 From: mmccune at redhat.com (Mike McCune) Date: Mon, 10 Dec 2012 21:22:33 -0800 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> <50C614A1.3070803@redhat.com> Message-ID: <50C6C319.6040309@redhat.com> this is correct, I tested the clean install and it works fine, see: https://pad-katello.rhcloud.com/p/katello-1.2rc1-testing On 12/10/2012 05:44 PM, Eric D Helms wrote: > The Ruport RPM has been updated in the 1.2 repositories to remove a > dependency on fastercsv in the gemspec on Ruby > 1.9 systems. This > means that on Fedora 17 systems, Katello should install properly, and as > long as the following are run (due to SElinux issues with the F17 install): > > setenforce 0 > service iptables stop > > > On Mon, Dec 10, 2012 at 11:58 AM, Miroslav Such? > wrote: > > On 12/10/2012 04:01 PM, Eric Helms wrote: > > I got past this error once I manually yum installed > rubygem-fastercsv, > and after looking over the changelog for Ruport, fastercsv looks > like it > was not included as a dependency for ruby > 1.9. Mirek can you > speak to > this as it looks like you added this? > > > fastercsv should not be needed. And if you install it using steps > documented on our wiki, it is not required. > > If you are using devel setup and you have fastercsv in some Gemfile, > it should be removed. Of course only on F17. It should be preserved > on F16 and EL6. > > > -- > Miroslav Suchy > Red Hat Systems Management Engineering > > _________________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/__mailman/listinfo/katello-devel > > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From lzap at redhat.com Tue Dec 11 09:06:14 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Tue, 11 Dec 2012 10:06:14 +0100 Subject: [katello-devel] Katello upgrade - for client systems? In-Reply-To: <50C601AC.3000107@redhat.com> References: <50C601AC.3000107@redhat.com> Message-ID: <20121211090614.GB1719@lzapx.brq.redhat.com> This is documented, you are supposed to upgrade systems manually for SE 1.1. > Would be there all the configuration and certificates remain unchanged? Yes, it is working, except for the port. It is all documented. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Tue Dec 11 14:16:28 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Tue, 11 Dec 2012 15:16:28 +0100 Subject: [katello-devel] Upgrade error during reindex Message-ID: <20121211141628.GE1719@lzapx.brq.redhat.com> Hello, I have a BZ https://bugzilla.redhat.com/show_bug.cgi?id=885261 with error during upgrade: ** Invoke db:migrate (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute db:migrate ** Invoke db:schema:dump (first_time) ** Invoke environment ** Execute db:schema:dump ** Invoke reindex (first_time) ** Invoke environment (first_time) ** Execute environment ** Invoke clear_search_indices (first_time) ** Execute clear_search_indices ** Execute reindex rake aborted! Couldn't find Organization with ID=3 /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:304:in `find_one' /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:289:in `find_with_ids' /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:107:in `find' /usr/share/katello/app/models/provider.rb:112:in `organization' /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `send' /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in `method_missing' /usr/share/katello/app/models/product.rb:105:in `organization' /usr/share/katello/app/models/product.rb:78:in `extended_index_attrs' /usr/share/katello/app/models/indexed_model.rb:140:in `send' /usr/share/katello/app/models/indexed_model.rb:140:in `to_indexed_json' /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:296:in `convert_document_to_json' /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:76:in `bulk_store' /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:68:in `map' /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:68:in `bulk_store' /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:118:in `import' /usr/share/katello/lib/tasks/reindex.rake:11 /usr/share/katello/lib/tasks/reindex.rake:8:in `each' /usr/share/katello/lib/tasks/reindex.rake:8 /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:205:in `call' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:205:in `execute' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:200:in `each' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:200:in `execute' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:158:in `invoke_with_call_chain' /usr/lib/ruby/1.8/monitor.rb:242:in `synchronize' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:151:in `invoke_with_call_chain' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:144:in `invoke' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:112:in `invoke_task' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in `each' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:129:in `standard_exception_handling' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:84:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:62:in `run' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:129:in `standard_exception_handling' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:59:in `run' /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/bin/rake:32 /usr/bin/rake:19:in `load' /usr/bin/rake:19 Tasks: TOP => reindex Search Indices cleared. Re-indexing ActivationKey Re-indexing Changeset Re-indexing DeletionChangeset Re-indexing Filter Re-indexing GpgKey Re-indexing Job Re-indexing MarketingProduct Reporter said there were several manifest imports that failed. Is there any known error that would leave database in inconsistent state? Also, I think we are indexing both MarketingProduct and Product which is not necessary, I guess. Any opinions about that? LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From thomasmckay at redhat.com Tue Dec 11 14:21:53 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Tue, 11 Dec 2012 09:21:53 -0500 (EST) Subject: [katello-devel] Upgrade error during reindex In-Reply-To: <20121211141628.GE1719@lzapx.brq.redhat.com> Message-ID: <1500952894.17078323.1355235713099.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Lukas Zapletal" > To: katello-devel at redhat.com > Sent: Tuesday, December 11, 2012 9:16:28 AM > Subject: [katello-devel] Upgrade error during reindex > > Hello, > > I have a BZ > > https://bugzilla.redhat.com/show_bug.cgi?id=885261 > > with error during upgrade: > > ** Invoke db:migrate (first_time) > ** Invoke environment (first_time) > ** Execute environment > ** Execute db:migrate > ** Invoke db:schema:dump (first_time) > ** Invoke environment > ** Execute db:schema:dump > ** Invoke reindex (first_time) > ** Invoke environment (first_time) > ** Execute environment > ** Invoke clear_search_indices (first_time) > ** Execute clear_search_indices > ** Execute reindex > rake aborted! > Couldn't find Organization with ID=3 > /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:304:in > `find_one' > /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:289:in > `find_with_ids' > /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/relation/finder_methods.rb:107:in > `find' > /usr/share/katello/app/models/provider.rb:112:in `organization' > /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in > `send' > /usr/lib/ruby/gems/1.8/gems/activerecord-3.0.10/lib/active_record/associations/association_proxy.rb:222:in > `method_missing' > /usr/share/katello/app/models/product.rb:105:in `organization' > /usr/share/katello/app/models/product.rb:78:in `extended_index_attrs' > /usr/share/katello/app/models/indexed_model.rb:140:in `send' > /usr/share/katello/app/models/indexed_model.rb:140:in > `to_indexed_json' > /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:296:in > `convert_document_to_json' > /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:76:in > `bulk_store' > /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:68:in `map' > /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:68:in > `bulk_store' > /usr/lib/ruby/gems/1.8/gems/tire-0.3.12/lib/tire/index.rb:118:in > `import' > /usr/share/katello/lib/tasks/reindex.rake:11 > /usr/share/katello/lib/tasks/reindex.rake:8:in `each' > /usr/share/katello/lib/tasks/reindex.rake:8 > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:205:in `call' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:205:in > `execute' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:200:in `each' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:200:in > `execute' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:158:in > `invoke_with_call_chain' > /usr/lib/ruby/1.8/monitor.rb:242:in `synchronize' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:151:in > `invoke_with_call_chain' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/task.rb:144:in > `invoke' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:112:in > `invoke_task' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in > `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in > `each' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:90:in > `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:129:in > `standard_exception_handling' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:84:in > `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:62:in > `run' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:129:in > `standard_exception_handling' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/lib/rake/application.rb:59:in > `run' > /usr/lib/ruby/gems/1.8/gems/rake-0.9.2/bin/rake:32 > /usr/bin/rake:19:in `load' > /usr/bin/rake:19 > Tasks: TOP => reindex > Search Indices cleared. > Re-indexing ActivationKey > Re-indexing Changeset > Re-indexing DeletionChangeset > Re-indexing Filter > Re-indexing GpgKey > Re-indexing Job > Re-indexing MarketingProduct > > Reporter said there were several manifest imports that failed. Is > there > any known error that would leave database in inconsistent state? > > > Also, I think we are indexing both MarketingProduct and Product which > is > not necessary, I guess. Any opinions about that? > > LZ > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > The indexing is failing because there are orphan "Red Hat" providers without an organization. My guess was that deleting an org is not properly cleaning up all the dependencies. Indexing both MarketingProduct and Product is legitimate and useful. From lzap at redhat.com Tue Dec 11 15:24:46 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Tue, 11 Dec 2012 16:24:46 +0100 Subject: [katello-devel] Dangerous katello-configure, planes in kitchen etc. Message-ID: <20121211152446.GF1719@lzapx.brq.redhat.com> Hello, so after investigation of logs, I have found out dangerous issue about katello-configure. Suppose you install with the following command: # katello-configure --org-name="MyOrg" --other-options ... Then you fill some data, work with your instance and then you re-execute katello-configure, which is considered safe: # katello-configure (Note the format of the commend - I will refer to this command later on) Then you work with the system again, everything seems to be working, until one process - the upgrade: # katello-upgrade In this case, upgrade process will break, because for some reason it tries to re-seed database with ACME_Corporation (default org name) creating another Red Hat provider etc. Other question, not important for us, is why it does try to re-seed, but anyway there are other things that can break like candlepin migration which would fail without proper candlepin db password setting. The issue: I said (on the list, on demos and maybe several other places including kitchen) that it is SAFE to run katello-configure multiple times. I have been working on improving our puppet codebase. But there is one big phat BUT: You MUST provide the --answer-file option! # katello-configure --answer-file=/etc/katello/katello-configure.conf Not providing it DELETES all your previous settings which does not break things immediately, but during things like upgrades etc. This is even more dangerous. A pilot forgets to turn deicing switch on during landing checklist and plane does not crash immediately - but ice starts to cumulate on wings and ... THEREFORE I *highly* recommend to change the behavior of katello-configure: 1) Automatically load answer file 2) New option --purge-answers that deletes the answer file so one can start over with clean table (good for tests etc but not for production setups). Fully documented with big warning. Opinions? The BZ that helped me found this issue is: https://bugzilla.redhat.com/show_bug.cgi?id=885261 -- Later, Lukas "lzap" Zapletal #katello #systemengine From inecas at redhat.com Tue Dec 11 15:39:21 2012 From: inecas at redhat.com (Ivan Necas) Date: Tue, 11 Dec 2012 10:39:21 -0500 (EST) Subject: [katello-devel] Dangerous katello-configure, planes in kitchen etc. In-Reply-To: <20121211152446.GF1719@lzapx.brq.redhat.com> Message-ID: <1080551424.14148024.1355240361415.JavaMail.root@redhat.com> ----- Original Message ----- > Hello, > > so after investigation of logs, I have found out dangerous issue > about > katello-configure. Suppose you install with the following command: > > # katello-configure --org-name="MyOrg" --other-options ... > > Then you fill some data, work with your instance and then you > re-execute > katello-configure, which is considered safe: > > # katello-configure > > (Note the format of the commend - I will refer to this command later > on) > > Then you work with the system again, everything seems to be working, > until one process - the upgrade: > > # katello-upgrade > > In this case, upgrade process will break, because for some reason it > tries to re-seed database with ACME_Corporation (default org name) > creating another Red Hat provider etc. Other question, not important > for > us, is why it does try to re-seed, but anyway there are other things > that can break like candlepin migration which would fail without > proper > candlepin db password setting. > > The issue: I said (on the list, on demos and maybe several other > places > including kitchen) that it is SAFE to run katello-configure multiple > times. I have been working on improving our puppet codebase. But > there > is one big phat BUT: > > You MUST provide the --answer-file option! > > # katello-configure --answer-file=/etc/katello/katello-configure.conf > > Not providing it DELETES all your previous settings which does not > break > things immediately, but during things like upgrades etc. This is even > more dangerous. A pilot forgets to turn deicing switch on during > landing > checklist and plane does not crash immediately - but ice starts to > cumulate on wings and ... > > THEREFORE > > I *highly* recommend to change the behavior of katello-configure: > > 1) Automatically load answer file > 2) New option --purge-answers that deletes the answer file so one can > start over with clean table (good for tests etc but not for > production > setups). Fully documented with big warning. > > Opinions? +1 for automatic deicing -- Ivan > > The BZ that helped me found this issue is: > > https://bugzilla.redhat.com/show_bug.cgi?id=885261 > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From msuchy at redhat.com Tue Dec 11 17:39:24 2012 From: msuchy at redhat.com (Miroslav Suchy) Date: Tue, 11 Dec 2012 18:39:24 +0100 Subject: [katello-devel] Dangerous katello-configure, planes in kitchen etc. In-Reply-To: <20121211152446.GF1719@lzapx.brq.redhat.com> References: <20121211152446.GF1719@lzapx.brq.redhat.com> Message-ID: <50C76FCC.7030005@redhat.com> On 11.12.2012 16:24, Lukas Zapletal wrote: > 1) Automatically load answer file Honestly I though that this is already implemented. And that /etc/katello/katello-configure.conf is loaded if exists. +100 for this one. > 2) New option --purge-answers that deletes the answer file so one can > start over with clean table (good for tests etc but not for production > setups). Fully documented with big warning. And is it really necessary? I would say if we print at begginging that we are reusing katello-configure.conf, then sysadmin can stop the script. Remove that file and start again. I would personally find that easier than remember another command line option. Something like # katello-configure .... [normal installation] # katello-configure Note: Reusing previously created answer file: /etc/katello/katello-configure.conf .... [normal installation] Mirek From ericdhelms at gmail.com Tue Dec 11 19:30:14 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Tue, 11 Dec 2012 14:30:14 -0500 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: <50C6C319.6040309@redhat.com> References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> <50C614A1.3070803@redhat.com> <50C6C319.6040309@redhat.com> Message-ID: The error trying to restart after installing katello-devel-all - http://pastie.org/5505656 appears to only affect katello-jobs and not the katello web service. Investigating closer, this appears to be related to trying to run bundler_ext under strict mode even if BUNDLER_EXT_NOSTRICT=1 is set in /etc/sysconfig/katello This indicates to me that the katello-jobs service doesn't respect or incorporate that particular setting in sysconfig. I don't believe this a blocker for the 1.2 release, as you can uninstall (or not install) the katello-devel-checking RPM. I will leave this till morning EST tomorrow for any other tinkering or attempts to fix, and unless any other major issues arise will begin building the signed RPMs and issuing the final release. - Eric On Tue, Dec 11, 2012 at 12:22 AM, Mike McCune wrote: > this is correct, I tested the clean install and it works fine, see: > > https://pad-katello.rhcloud.**com/p/katello-1.2rc1-testing > > > > On 12/10/2012 05:44 PM, Eric D Helms wrote: > >> The Ruport RPM has been updated in the 1.2 repositories to remove a >> dependency on fastercsv in the gemspec on Ruby > 1.9 systems. This >> means that on Fedora 17 systems, Katello should install properly, and as >> long as the following are run (due to SElinux issues with the F17 >> install): >> >> setenforce 0 >> service iptables stop >> >> >> On Mon, Dec 10, 2012 at 11:58 AM, Miroslav Such? > > wrote: >> >> On 12/10/2012 04:01 PM, Eric Helms wrote: >> >> I got past this error once I manually yum installed >> rubygem-fastercsv, >> and after looking over the changelog for Ruport, fastercsv looks >> like it >> was not included as a dependency for ruby > 1.9. Mirek can you >> speak to >> this as it looks like you added this? >> >> >> fastercsv should not be needed. And if you install it using steps >> documented on our wiki, it is not required. >> >> If you are using devel setup and you have fastercsv in some Gemfile, >> it should be removed. Of course only on F17. It should be preserved >> on F16 and EL6. >> >> >> -- >> Miroslav Suchy >> Red Hat Systems Management Engineering >> >> ______________________________**___________________ >> katello-devel mailing list >> katello-devel at redhat.com >> > >> https://www.redhat.com/__**mailman/listinfo/katello-devel >> >> **> >> >> >> >> >> >> ______________________________**_________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/**mailman/listinfo/katello-devel >> > > -- > Mike McCune > mmccune AT redhat.com > Red Hat Engineering | Portland, OR > Systems Management | 650-254-4248 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehelms at redhat.com Tue Dec 11 19:57:32 2012 From: ehelms at redhat.com (Eric Helms) Date: Tue, 11 Dec 2012 14:57:32 -0500 Subject: [katello-devel] Request for User Workflows and Example Stories Message-ID: <50C7902C.5030108@redhat.com> Howdy All, As part of the recent feedback gathered from the community, one item that was mentioned was: "A need for example user stories that show how to accomplish real world tasks with proposed features and stories to help developers better understand what aspects of a story are important to users and who the users or personas are. Further, to provide examples to new users of workflows and scenarios that can be accomplished with Katello." To the above end, I am requesting from both developers and community users the following to develop user workflows, personas and example stories in as much detail as you can muster. These workflows, personas and stories will be developed further and produced in an easily reference-able and readable documentation. These will also help drive in driving the development of future screencasts and features. - Scenarios and Workflows that you use in your common day and week with Katello - Scenarios and Workflows you envision classes of users performing with Katello - Simple and Complex workflows Where possible, please include the following for each Scenario/Workflow: * User/Persona performing the workflow (e.g. sys admin, IT dev op user) * End goal of performing the Scenario/Workflow * Steps (generic or specific) taken to complete some Scenario/Workflow * Entities created or needed to perform Scenario/Workflow Thanks for any time you can spare contributing. -Eric From lbrindle at redhat.com Tue Dec 11 22:50:31 2012 From: lbrindle at redhat.com (Lana Brindley) Date: Wed, 12 Dec 2012 09:50:31 +1100 Subject: [katello-devel] Request for User Workflows and Example Stories In-Reply-To: <50C7902C.5030108@redhat.com> References: <50C7902C.5030108@redhat.com> Message-ID: <50C7B8B7.2080909@redhat.com> On 12/12/2012 06:57 AM, Eric Helms wrote: > Howdy All, > > As part of the recent feedback gathered from the community, one item > that was mentioned was: > Hi Eric, Any examples you get, would you please be able to cc the documentation team in? That way we can integrate them into the enterprise docs as well. L -- Lana Brindley Senior Content Author Engineering Content Services Any word you have to hunt for in a thesaurus is the wrong word. There are no exceptions to this rule. - Stephen King From lzap at redhat.com Wed Dec 12 08:49:24 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 09:49:24 +0100 Subject: [katello-devel] Dangerous katello-configure, planes in kitchen etc. In-Reply-To: <50C76FCC.7030005@redhat.com> References: <20121211152446.GF1719@lzapx.brq.redhat.com> <50C76FCC.7030005@redhat.com> Message-ID: <20121212084924.GB1895@lzapx.brq.redhat.com> On Tue, Dec 11, 2012 at 06:39:24PM +0100, Miroslav Suchy wrote: > >2) New option --purge-answers that deletes the answer file so one can > >start over with clean table (good for tests etc but not for production > >setups). Fully documented with big warning. > > And is it really necessary? I would say if we print at begginging > that we are reusing katello-configure.conf, then sysadmin can stop > the script. Remove that file and start again. I would personally > find that easier than remember another command line option. > Ok, I like this too. Will do this way. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 12 08:56:29 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 09:56:29 +0100 Subject: [katello-devel] Katello 1.2 Release Candidate In-Reply-To: References: <20121210091106.GC1721@lzapx.brq.redhat.com> <20121210115733.GH1721@lzapx.brq.redhat.com> <50C5F951.8000204@redhat.com> <50C614A1.3070803@redhat.com> <50C6C319.6040309@redhat.com> Message-ID: <20121212085629.GC1895@lzapx.brq.redhat.com> On Tue, Dec 11, 2012 at 02:30:14PM -0500, Eric D Helms wrote: > The error trying to restart after installing katello-devel-all - > http://pastie.org/5505656 > appears to only affect katello-jobs and not the katello web service. > Investigating closer, this appears to be related to trying to run > bundler_ext under strict mode even if BUNDLER_EXT_NOSTRICT=1 is set in > /etc/sysconfig/katello > > This indicates to me that the katello-jobs service doesn't respect or > incorporate that particular setting in sysconfig. I don't believe this a > blocker for the 1.2 release, as you can uninstall (or not install) the > katello-devel-checking RPM. I will leave this till morning EST tomorrow > for any other tinkering or attempts to fix, and unless any other major > issues arise will begin building the signed RPMs and issuing the final > release. This has been fixed already: https://github.com/Katello/katello/pull/1228 I will look on the jshintrb error now if I can make a quick fix. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From jmagen at redhat.com Wed Dec 12 09:13:37 2012 From: jmagen at redhat.com (Joseph Magen) Date: Wed, 12 Dec 2012 04:13:37 -0500 (EST) Subject: [katello-devel] Foreman API v2 guidelines and design In-Reply-To: <1094683748.21886062.1355303447734.JavaMail.root@redhat.com> Message-ID: <143241186.21890499.1355303617888.JavaMail.root@redhat.com> Hi all, I am interested in hearing from you about the design and guidelines for best consuming the Foreman API v2. Some questions are: 1) What to include when returning list of objects vs. single object? 2) When to return child/parent relationship objects vs. return child/parent id's only vs. return child/parent id's and name's vs. don't return child/parent details (as it can be returned in separate call) 3) When to include timestamps "created_at, updated_at" in each response. Is this information used? For the sake of discussion, let's take hostgroups. The current v1 response for a list of hostgroups AND a specific hostgroup is the same. It includes all database fields including parent_id's (domain_id, evironment_id, subnet_id, operatingsystem_id), plus array of child id's (for puppletclass), plus child objects (for parameters). I don't know if all this information is necessary for each call. Would it be better to break it done in minimal responses that will be quicker. V1) same response for /api/hostgroups AND /api/hostgroups/8. No nested routes. "hostgroup": { "name": "PG-Database", "environment_id": 1, "subnet_id": 1, "label": "Base/PG-Database", "puppetclass_ids": [ 1, 6, 5, 100, 77, 229 ], "operatingsystem_id": 6, "ancestry": "1", "domain_id": 6, "parameters": { "test_pg_param": "111" }, "id": 8 } V2) minimal responses depending on api call /api/hostgroups - returns list of hostgroups with minimal response (id, name, label) /api/hostgroups/8 - returns database fields of hostgroup id 8 (same as v1, but without puppetpclass_ids and parameters in response) /api/hostgroups/8/puppetclasses - returns puppetclass objects of hostgroup id 8 /api/hostgroups/8/parameters - returns parameter objects of hostgroup id 8 /api/hostgroups/8/domain - returns domain object of hostgroup id 8 /api/hostgroups/8/environment - returns environment object of hostgroup id 8 /api/hostgroups/8/operatingsystem - returns operatingsystem object of hostgroup id 8 /api/hostgroups/8/subnet - returns subnet object of hostgroup id 8 /api/hostgroups/8/ancestor_ids - returns array of hostgroup ids in ancestry of hostgroup id 8 /api/hostgroups/8/otheraction - any other action Note: /api/hostgroups/8/domain would return the same response as /api/domains/6 Regards, Joseph From ehelms at redhat.com Wed Dec 12 14:25:23 2012 From: ehelms at redhat.com (Eric Helms) Date: Wed, 12 Dec 2012 09:25:23 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 Message-ID: <50C893D3.8090503@redhat.com> Howdy All, There are a large number of open bugs, some dating back well into last year on the Katello plate. Roughly speaking: https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 I am proposing 12/18/2012 unless some major reason not to use that date comes along. For those in the U.S. time zones, I will organize. For those working in Europe, would someone volunteer to coordinate for that side of the pond? I am attempting to also give enough lead time so that developers can prepare themselves and environments to spend the entire day together closing, fixing, asking for more information and generally bringing down our open bug count. - Eric From bbuckingham at redhat.com Wed Dec 12 14:27:13 2012 From: bbuckingham at redhat.com (Brad Buckingham) Date: Wed, 12 Dec 2012 09:27:13 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C893D3.8090503@redhat.com> References: <50C893D3.8090503@redhat.com> Message-ID: <50C89441.5010208@redhat.com> On 12/12/2012 09:25 AM, Eric Helms wrote: > Howdy All, > > There are a large number of open bugs, some dating back well into last > year on the Katello plate. Roughly speaking: > > https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 > > > I am proposing 12/18/2012 unless some major reason not to use that > date comes along. For those in the U.S. time zones, I will organize. > For those working in Europe, would someone volunteer to coordinate for > that side of the pond? > > I am attempting to also give enough lead time so that developers can > prepare themselves and environments to spend the entire day together > closing, fixing, asking for more information and generally bringing > down our open bug count. > > - Eric > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel +1 to the proposed date Brad From daviddavis at redhat.com Wed Dec 12 14:38:41 2012 From: daviddavis at redhat.com (David Davis) Date: Wed, 12 Dec 2012 09:38:41 -0500 (EST) Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C89441.5010208@redhat.com> Message-ID: <1607045833.48168121.1355323121365.JavaMail.root@redhat.com> +1 David ----- Original Message ----- > From: "Brad Buckingham" > To: katello-devel at redhat.com > Sent: Wednesday, December 12, 2012 9:27:13 AM > Subject: Re: [katello-devel] Katello Bug Day 2012 - 12/18/2012 > > On 12/12/2012 09:25 AM, Eric Helms wrote: > > Howdy All, > > > > There are a large number of open bugs, some dating back well into > > last > > year on the Katello plate. Roughly speaking: > > > > https://bugzilla.redhat.com/buglist.cgi?quicksearch=NEW%2CASSIGNED%20product%3A%22katello%22%2C%22CloudForms%20System%20Engine%22%2C%22Subscription%20Asset%20Manager%22&list_id=912048 > > > > > > I am proposing 12/18/2012 unless some major reason not to use that > > date comes along. For those in the U.S. time zones, I will > > organize. > > For those working in Europe, would someone volunteer to coordinate > > for > > that side of the pond? > > > > I am attempting to also give enough lead time so that developers > > can > > prepare themselves and environments to spend the entire day > > together > > closing, fixing, asking for more information and generally bringing > > down our open bug count. > > > > - Eric > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > +1 to the proposed date > > Brad > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From pchalupa at redhat.com Wed Dec 12 14:49:17 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Wed, 12 Dec 2012 15:49:17 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50C0DE49.9070903@redhat.com> References: <50C0DE49.9070903@redhat.com> Message-ID: <50C8996D.7080801@redhat.com> I am not sure if I understand it correctly, so I've decided to send my version to make sure that we are on the same track. ## Legend - black edges are representing knowledge dependency as before - red represents modules and direction in which they are included (Interpretation id: what knowledge ActiveRecord model gains when a module is included) - green represents services - 1) in reality there is one step skipped, instantiation of an object from foreman_api gem and creating a singleton, see [1] ## Where I see differences - Candlepin doesn't have a Rubygem with Ruby bindings, they are placed in /lib - In Foreman there is one additional layer "Remote resource representation". Objects of the layer are replacing lazy_accessors and they are introducing missing separation between data from Katello and remote service. - There are no controllers with full CRUD to manipulate with non-Katello data in Pulp or Candlepin. There are in Foreman and they are using added representation of remote resources. ## Implementation notes - All classes in "Remote resource representation" are children of Resources::ForemanModel which is representation of any Foreman remote resource. Resources::ForemanModel is inheriting from Resources::AbstractModel which is representation of any remote resource. (AbstractModel could also by used for Candlepin and Pulp) More documentation is on wiki [2] If you have any additional questions about the Foreman please send me an email and I'll prepare answers for deep dive tomorrow. Petr [1] https://github.com/Katello/katello/blob/pulpv2/src/lib/resources/foreman.rb [2] https://fedorahosted.org/katello/wiki/ForemanIntegrationAPI ## Other resources There were at least 3 long threads about this topic I think. I didn't have luck finding them but if you want I can try harder. On 06.12.12 19:04, Bryan Kearney wrote: > On 12/05/2012 09:20 PM, Eric D Helms wrote: >> They say a picture is worth a thousands words, and after, for lack of a >> better word, being bugged by the differences in backend service >> integration I decided to try and spell out the current lay of the land. >> I have attempted to do this in two ways. The first being a diagram >> showing how our models, glue and controllers are laid out in relation to >> backend services with arrows indicating the directional awareness that >> one entity has with another. For example, an arrow pointing from >> glue/candlepin to resources/candlepin since glue/candlepin has direct >> knowledge of the resources but not vice versa. Second, I have attempted >> to quell out into bullet points how each backend service is currently >> (and by this I mean as of PulpV2 branch) integrated. >> >> Disclaimer: This is my understanding of things and there are places >> where I could have gotten something wrong or mis-represented that actual >> implementation >> >> Reference Diagram: >> https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit >> >> >> Note: This is taking into consideration the changes made in the PulpV2 >> branch and current Candlepin and Foreman work in master >> >> >> *PulpV2* >> >> - All Pulp functionality is contained within app/models/glue/pulp >> - All glue layers are modules that are included into the Katello model >> if use_pulp = true >> - Katello Models are treated as the main entity, with Pulp glue layer >> adding functionality if included >> - Even non-persisted objects have a top-level representation in Katello >> (e.g. errata, package, distribution) >> - Nothing is proxied >> - Basic functionality is orchestrated (CRUD) >> - Use of callbacks and hooks to trigger top-level events that can be >> listened to by glue layer >> - There is no notion of pulp outside the glue layer >> >> >> *Candlepin* >> >> - Contains a single Candlepin only model without corresponding inclusion >> in a Katello entity (OwnerInfo) >> - Contains some proxied functionality mainly for subscription manager >> - Not all Candlepin functionality is contained in glue layer >> - Basic functionality is orchestrated (CRUD) >> >> >> *Foreman* >> >> - Foreman is front and center at all layers >> - Controllers for API and UI are under foreman/ and namespaced >> - Models are under foreman/ and are classed as Foreman:: >> - Katello models that have corresponding Foreman model also get a >> separate model file besides their glue layer counterpart (i.e. >> models/foreman/user) >> - Other models have lightweight representations and are entirely pass >> through objects down to Foreman but are not top-level Katello entities >> - Glue layer is fenced off and only included if use_foreman set to true >> >> >> *ElasticSearch* >> >> - All ES functionality is contained within app/models/glue/elasticsearch >> - All glue layers are modules that included into the corresponding >> Katello model >> - Adds functionality but does not directly impart itself into the models >> - Use of callbacks and hooks to trigger top-level events that can be >> listened to by glue layer >> >> >> I am hoping this will spark some discussion, as I worry that the >> differences in integration points for each backend service will only >> make our lives as developers more difficult down the road. >> >> -Eric >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > > I agree with the Deep Dive. Dmitri had expressed similar concerns. Who > from ? wants to set this up? > > -- bk > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: graph.dot Type: application/msword Size: 1908 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graph.pdf Type: application/pdf Size: 20402 bytes Desc: not available URL: From lzap at redhat.com Wed Dec 12 15:08:33 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 16:08:33 +0100 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C893D3.8090503@redhat.com> References: <50C893D3.8090503@redhat.com> Message-ID: <20121212150832.GH1895@lzapx.brq.redhat.com> On Wed, Dec 12, 2012 at 09:25:23AM -0500, Eric Helms wrote: > I am proposing 12/18/2012 unless some major reason not to use that > date comes along. For those in the U.S. time zones, I will > organize. For those working in Europe, would someone volunteer to > coordinate for that side of the pond? I can do it. Are we gonna use some Keyword to mark those which was already discussed/triaged? LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Wed Dec 12 15:09:44 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 12 Dec 2012 10:09:44 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <20121212150832.GH1895@lzapx.brq.redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> Message-ID: <50C89E38.6020106@redhat.com> On 12/12/2012 10:08 AM, Lukas Zapletal wrote: > On Wed, Dec 12, 2012 at 09:25:23AM -0500, Eric Helms wrote: >> I am proposing 12/18/2012 unless some major reason not to use that >> date comes along. For those in the U.S. time zones, I will >> organize. For those working in Europe, would someone volunteer to >> coordinate for that side of the pond? > > I can do it. > > Are we gonna use some Keyword to mark those which was already > discussed/triaged? > > LZ > > My hope is that folks will take some, try and re-produce, and then close or try and fix. We could list all the bugs on a public etherpad some place. -- bk From lzap at redhat.com Wed Dec 12 15:11:18 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 16:11:18 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50C8996D.7080801@redhat.com> References: <50C0DE49.9070903@redhat.com> <50C8996D.7080801@redhat.com> Message-ID: <20121212151118.GI1895@lzapx.brq.redhat.com> On Wed, Dec 12, 2012 at 03:49:17PM +0100, Petr Chalupa wrote: > If you have any additional questions about the Foreman please send > me an email and I'll prepare answers for deep dive tomorrow. I am the most interested in comparing the Foreman and our current approach among each other. Benefits, drawbacks, differences. -- Later, Lukas "lzap" Zapletal #katello #systemengine From bbuckingham at redhat.com Wed Dec 12 15:13:47 2012 From: bbuckingham at redhat.com (Brad Buckingham) Date: Wed, 12 Dec 2012 10:13:47 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <20121212150832.GH1895@lzapx.brq.redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> Message-ID: <50C89F2B.1020405@redhat.com> On 12/12/2012 10:08 AM, Lukas Zapletal wrote: > On Wed, Dec 12, 2012 at 09:25:23AM -0500, Eric Helms wrote: >> I am proposing 12/18/2012 unless some major reason not to use that >> date comes along. For those in the U.S. time zones, I will >> organize. For those working in Europe, would someone volunteer to >> coordinate for that side of the pond? > I can do it. > > Are we gonna use some Keyword to mark those which was already > discussed/triaged? > > LZ > > Can we use the BZ states for that? As soon as a developer picks a bug to investigate, they should move it to ASSIGNED. So, assigned can be considered 'taken'. If they find it is no longer a bug, they can change state appropriately. If they find they need more information, they can request more info. This basically means that all bugs in 'NEW' state are available to be taken. We should probably begin first with those that are triaged, since we have a good bucket to begin with. From lzap at redhat.com Wed Dec 12 15:14:53 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 16:14:53 +0100 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C89E38.6020106@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89E38.6020106@redhat.com> Message-ID: <20121212151453.GJ1895@lzapx.brq.redhat.com> On Wed, Dec 12, 2012 at 10:09:44AM -0500, Bryan Kearney wrote: > On 12/12/2012 10:08 AM, Lukas Zapletal wrote: > >On Wed, Dec 12, 2012 at 09:25:23AM -0500, Eric Helms wrote: > >>I am proposing 12/18/2012 unless some major reason not to use that > >>date comes along. For those in the U.S. time zones, I will > >>organize. For those working in Europe, would someone volunteer to > >>coordinate for that side of the pond? > > > >I can do it. > > > >Are we gonna use some Keyword to mark those which was already > >discussed/triaged? > > > >LZ > > > > > My hope is that folks will take some, try and re-produce, and then > close or try and fix. > > We could list all the bugs on a public etherpad some place. Ok let's assume once bug is ASSIGNED then you will not squash it. And we will flip as many bugs as we think we can do in one day. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From thomasmckay at redhat.com Wed Dec 12 15:28:01 2012 From: thomasmckay at redhat.com (Tom McKay) Date: Wed, 12 Dec 2012 10:28:01 -0500 (EST) Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50C8996D.7080801@redhat.com> Message-ID: <2092760419.17592890.1355326081251.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Petr Chalupa" > To: katello-devel at redhat.com > Sent: Wednesday, December 12, 2012 9:49:17 AM > Subject: Re: [katello-devel] Katello Backend Service Relationships and Architecture > > I am not sure if I understand it correctly, so I've decided to send > my > version to make sure that we are on the same track. > > ## Legend > > - black edges are representing knowledge dependency as before > - red represents modules and direction in which they are included > (Interpretation id: what knowledge ActiveRecord model gains when a > module is included) > - green represents services > - 1) in reality there is one step skipped, instantiation of an object > from foreman_api gem and creating a singleton, see [1] > > ## Where I see differences > > - Candlepin doesn't have a Rubygem with Ruby bindings, they are > placed > in /lib > - In Foreman there is one additional layer "Remote resource > representation". Objects of the layer are replacing lazy_accessors > and > they are introducing missing separation between data from Katello and > remote service. > - There are no controllers with full CRUD to manipulate with > non-Katello > data in Pulp or Candlepin. There are in Foreman and they are using > added > representation of remote resources. > > ## Implementation notes > > - All classes in "Remote resource representation" are children of > Resources::ForemanModel which is representation of any Foreman remote > resource. Resources::ForemanModel is inheriting from > Resources::AbstractModel which is representation of any remote > resource. > (AbstractModel could also by used for Candlepin and Pulp) > > More documentation is on wiki [2] > > If you have any additional questions about the Foreman please send me > an > email and I'll prepare answers for deep dive tomorrow. > > Petr > > [1] > https://github.com/Katello/katello/blob/pulpv2/src/lib/resources/foreman.rb > [2] https://fedorahosted.org/katello/wiki/ForemanIntegrationAPI > > ## Other resources > > There were at least 3 long threads about this topic I think. I didn't > have luck finding them but if you want I can try harder. > > > On 06.12.12 19:04, Bryan Kearney wrote: > > On 12/05/2012 09:20 PM, Eric D Helms wrote: > >> They say a picture is worth a thousands words, and after, for lack > >> of a > >> better word, being bugged by the differences in backend service > >> integration I decided to try and spell out the current lay of the > >> land. > >> I have attempted to do this in two ways. The first being a > >> diagram > >> showing how our models, glue and controllers are laid out in > >> relation to > >> backend services with arrows indicating the directional awareness > >> that > >> one entity has with another. For example, an arrow pointing from > >> glue/candlepin to resources/candlepin since glue/candlepin has > >> direct > >> knowledge of the resources but not vice versa. Second, I have > >> attempted > >> to quell out into bullet points how each backend service is > >> currently > >> (and by this I mean as of PulpV2 branch) integrated. > >> > >> Disclaimer: This is my understanding of things and there are > >> places > >> where I could have gotten something wrong or mis-represented that > >> actual > >> implementation > >> > >> Reference Diagram: > >> https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit > >> > >> > >> Note: This is taking into consideration the changes made in the > >> PulpV2 > >> branch and current Candlepin and Foreman work in master > >> > >> > >> *PulpV2* > >> > >> - All Pulp functionality is contained within app/models/glue/pulp > >> - All glue layers are modules that are included into the Katello > >> model > >> if use_pulp = true > >> - Katello Models are treated as the main entity, with Pulp glue > >> layer > >> adding functionality if included > >> - Even non-persisted objects have a top-level representation in > >> Katello > >> (e.g. errata, package, distribution) > >> - Nothing is proxied > >> - Basic functionality is orchestrated (CRUD) > >> - Use of callbacks and hooks to trigger top-level events that can > >> be > >> listened to by glue layer > >> - There is no notion of pulp outside the glue layer > >> > >> > >> *Candlepin* > >> > >> - Contains a single Candlepin only model without corresponding > >> inclusion > >> in a Katello entity (OwnerInfo) > >> - Contains some proxied functionality mainly for subscription > >> manager > >> - Not all Candlepin functionality is contained in glue layer > >> - Basic functionality is orchestrated (CRUD) > >> > >> > >> *Foreman* > >> > >> - Foreman is front and center at all layers > >> - Controllers for API and UI are under foreman/ and namespaced > >> - Models are under foreman/ and are classed as Foreman:: > >> - Katello models that have corresponding Foreman model also get a > >> separate model file besides their glue layer counterpart (i.e. > >> models/foreman/user) > >> - Other models have lightweight representations and are entirely > >> pass > >> through objects down to Foreman but are not top-level Katello > >> entities > >> - Glue layer is fenced off and only included if use_foreman set to > >> true > >> > >> > >> *ElasticSearch* > >> > >> - All ES functionality is contained within > >> app/models/glue/elasticsearch > >> - All glue layers are modules that included into the corresponding > >> Katello model > >> - Adds functionality but does not directly impart itself into the > >> models > >> - Use of callbacks and hooks to trigger top-level events that can > >> be > >> listened to by glue layer > >> > >> > >> I am hoping this will spark some discussion, as I worry that the > >> differences in integration points for each backend service will > >> only > >> make our lives as developers more difficult down the road. > >> > >> -Eric > >> > >> > >> _______________________________________________ > >> katello-devel mailing list > >> katello-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/katello-devel > >> > > > > I agree with the Deep Dive. Dmitri had expressed similar concerns. > > Who > > from ? wants to set this up? > > > > -- bk > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel My biggest issue is that Foreman stuff is all over the place. It was not done following the existing "glue" implementation of pulp and candlepin. As I've said, I really don't have a stake in this decision except for the fact that headpin shares the code base. The pain we've experienced was mostly due with packaging of the headpin RPMs. I am just largely disappointed that the foreman integration didn't join the project as a member with the goal of iterating all the backend services to something better. Instead it just landed with a thump, squashing everything in its way. From mmccune at redhat.com Wed Dec 12 15:32:36 2012 From: mmccune at redhat.com (Mike McCune) Date: Wed, 12 Dec 2012 07:32:36 -0800 Subject: [katello-devel] Dangerous katello-configure, planes in kitchen etc. In-Reply-To: <50C76FCC.7030005@redhat.com> References: <20121211152446.GF1719@lzapx.brq.redhat.com> <50C76FCC.7030005@redhat.com> Message-ID: <50C8A394.6020902@redhat.com> On 12/11/2012 09:39 AM, Miroslav Suchy wrote: > On 11.12.2012 16:24, Lukas Zapletal wrote: >> 1) Automatically load answer file > > Honestly I though that this is already implemented. And that > /etc/katello/katello-configure.conf is loaded if exists. > > +100 for this one. > >> 2) New option --purge-answers that deletes the answer file so one can >> start over with clean table (good for tests etc but not for production >> setups). Fully documented with big warning. > > And is it really necessary? I would say if we print at begginging that > we are reusing katello-configure.conf, then sysadmin can stop the > script. Remove that file and start again. I would personally find that > easier than remember another command line option. > > Something like > > # katello-configure > .... [normal installation] > > # katello-configure > Note: Reusing previously created answer file: > /etc/katello/katello-configure.conf > .... [normal installation] > > Mirek > the above sounds the most sensible and doesn't overwrite anything in an unexpected way. +1 -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From mmccune at redhat.com Wed Dec 12 15:46:36 2012 From: mmccune at redhat.com (Mike McCune) Date: Wed, 12 Dec 2012 07:46:36 -0800 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C89F2B.1020405@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> Message-ID: <50C8A6DC.6020008@redhat.com> On 12/12/2012 07:13 AM, Brad Buckingham wrote: > On 12/12/2012 10:08 AM, Lukas Zapletal wrote: >> On Wed, Dec 12, 2012 at 09:25:23AM -0500, Eric Helms wrote: >>> I am proposing 12/18/2012 unless some major reason not to use that >>> date comes along. For those in the U.S. time zones, I will >>> organize. For those working in Europe, would someone volunteer to >>> coordinate for that side of the pond? >> I can do it. >> >> Are we gonna use some Keyword to mark those which was already >> discussed/triaged? >> >> LZ >> >> > Can we use the BZ states for that? > > As soon as a developer picks a bug to investigate, they should move it > to ASSIGNED. So, assigned can be considered 'taken'. If they find it > is no longer a bug, they can change state appropriately. If they find > they need more information, they can request more info. we may have to reset all the existing bugs that are already in ASSIGNED back to NEW that people might not actively be working on (12 bugs) : http://tinyurl.com/aqfznmb not a big deal but something to consider. > > This basically means that all bugs in 'NEW' state are available to be > taken. We should probably begin first with those that are triaged, > since we have a good bucket to begin with. +1, also means we should work to at least triage all the un-triaged bugs (~145 today) before bug day. That would help get a much better picture of the bugs we need to work to resolve. Mike -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From msuchy at redhat.com Wed Dec 12 16:11:50 2012 From: msuchy at redhat.com (=?ISO-8859-1?Q?Miroslav_Such=FD?=) Date: Wed, 12 Dec 2012 17:11:50 +0100 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C8A6DC.6020008@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> Message-ID: <50C8ACC6.3040105@redhat.com> On 12/12/2012 04:46 PM, Mike McCune wrote: > we may have to reset all the existing bugs that are already in ASSIGNED > back to NEW that people might not actively be working on (12 bugs) -1 -- Miroslav Suchy Red Hat Systems Management Engineering From mmccune at redhat.com Wed Dec 12 17:39:31 2012 From: mmccune at redhat.com (Mike McCune) Date: Wed, 12 Dec 2012 09:39:31 -0800 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C8ACC6.3040105@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> Message-ID: <50C8C153.7050907@redhat.com> On 12/12/2012 08:11 AM, Miroslav Such? wrote: > On 12/12/2012 04:46 PM, Mike McCune wrote: >> we may have to reset all the existing bugs that are already in ASSIGNED >> back to NEW that people might not actively be working on (12 bugs) > > -1 > if it was 100+ ASSIGNED bugs, I think I'd agree that we would want to reset them to NEW but since it is only 12 we can leave it alone. so .. -1 to my own suggestion :) Mike -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From bkearney at redhat.com Wed Dec 12 18:26:19 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 12 Dec 2012 13:26:19 -0500 Subject: [katello-devel] Foreman Registration Design Message-ID: <50C8CC4B.5040106@redhat.com> Lzap: I edited your write up a bit. You can see it at [1]. Next steps, lets flush out the design and user stories. I think between you and Dominic we are close Finally, I think it would be good to have a 1 hour review with the team. -- bk [1] https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign From lzap at redhat.com Wed Dec 12 19:06:22 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 12 Dec 2012 20:06:22 +0100 Subject: [katello-devel] Redcarpet and development mode Message-ID: <20121212190622.GB27952@lzapx.brq.redhat.com> Hey, for some reason, Redcarpet segfaults for me when I try start Katello in development mode. This is very limiting for me, is there a workaround? I think this was working, not sure what is wrong. Thanks -- Later, Lukas "lzap" Zapletal #katello #systemengine From mmccune at redhat.com Wed Dec 12 20:10:28 2012 From: mmccune at redhat.com (Mike McCune) Date: Wed, 12 Dec 2012 12:10:28 -0800 Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <20121212190622.GB27952@lzapx.brq.redhat.com> References: <20121212190622.GB27952@lzapx.brq.redhat.com> Message-ID: <50C8E4B4.10206@redhat.com> On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > Hey, > > for some reason, Redcarpet segfaults for me when I try start Katello in > development mode. This is very limiting for me, is there a workaround? > I think this was working, not sure what is wrong. > > Thanks > I get the same on EL 6.3, haven't found a workaround yet. -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From ehelms at redhat.com Thu Dec 13 03:19:34 2012 From: ehelms at redhat.com (Eric Helms) Date: Wed, 12 Dec 2012 22:19:34 -0500 (EST) Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50C8996D.7080801@redhat.com> Message-ID: <91740383.17807534.1355368774620.JavaMail.root@redhat.com> My intent for creating the diagram and bring to light the various implementation details surrounding each service was not to raise the ActiveResource debate again or to question the Foreman integration. As my diagram showed, Candlepin has connections and touches almost every layer as did PulpV1 integration. Rather, my goal is to bring up a larger conversation about the Katello architecture as a whole going forward. We are at a point where we are migrating to a completely new version of one backend service and re-writing a lot of the Katello bits aroun d it (Pulp) and simultaneously integrating a brand new backend engine (Foreman). I am of the opinion, and I think based on replies that others echo this, that we should have a consistent architecture for the integration of backend services into Katello. No matter what ends up being the choice, consistency is key. If there are no other agenda items for the Deep Dive tomorrow, perhaps we can take that time to discuss the state of each backend service integration wise and how we'd like to achieve consistency and a separation of concerns throughout the Katello layers. -Eric ----- Original Message ----- From: "Petr Chalupa" To: katello-devel at redhat.com Sent: Wednesday, December 12, 2012 9:49:17 AM Subject: Re: [katello-devel] Katello Backend Service Relationships and Architecture I am not sure if I understand it correctly, so I've decided to send my version to make sure that we are on the same track. ## Legend - black edges are representing knowledge dependency as before - red represents modules and direction in which they are included (Interpretation id: what knowledge ActiveRecord model gains when a module is included) - green represents services - 1) in reality there is one step skipped, instantiation of an object from foreman_api gem and creating a singleton, see [1] ## Where I see differences - Candlepin doesn't have a Rubygem with Ruby bindings, they are placed in /lib - In Foreman there is one additional layer "Remote resource representation". Objects of the layer are replacing lazy_accessors and they are introducing missing separation between data from Katello and remote service. - There are no controllers with full CRUD to manipulate with non-Katello data in Pulp or Candlepin. There are in Foreman and they are using added representation of remote resources. ## Implementation notes - All classes in "Remote resource representation" are children of Resources::ForemanModel which is representation of any Foreman remote resource. Resources::ForemanModel is inheriting from Resources::AbstractModel which is representation of any remote resource. (AbstractModel could also by used for Candlepin and Pulp) More documentation is on wiki [2] If you have any additional questions about the Foreman please send me an email and I'll prepare answers for deep dive tomorrow. Petr [1] https://github.com/Katello/katello/blob/pulpv2/src/lib/resources/foreman.rb [2] https://fedorahosted.org/katello/wiki/ForemanIntegrationAPI ## Other resources There were at least 3 long threads about this topic I think. I didn't have luck finding them but if you want I can try harder. On 06.12.12 19:04, Bryan Kearney wrote: > On 12/05/2012 09:20 PM, Eric D Helms wrote: >> They say a picture is worth a thousands words, and after, for lack of a >> better word, being bugged by the differences in backend service >> integration I decided to try and spell out the current lay of the land. >> I have attempted to do this in two ways. The first being a diagram >> showing how our models, glue and controllers are laid out in relation to >> backend services with arrows indicating the directional awareness that >> one entity has with another. For example, an arrow pointing from >> glue/candlepin to resources/candlepin since glue/candlepin has direct >> knowledge of the resources but not vice versa. Second, I have attempted >> to quell out into bullet points how each backend service is currently >> (and by this I mean as of PulpV2 branch) integrated. >> >> Disclaimer: This is my understanding of things and there are places >> where I could have gotten something wrong or mis-represented that actual >> implementation >> >> Reference Diagram: >> https://docs.google.com/drawings/d/1lK-BozdUbTBJPnvSv40uCtzy4_Y1qUZCIJesq_pqsHY/edit >> >> >> Note: This is taking into consideration the changes made in the PulpV2 >> branch and current Candlepin and Foreman work in master >> >> >> *PulpV2* >> >> - All Pulp functionality is contained within app/models/glue/pulp >> - All glue layers are modules that are included into the Katello model >> if use_pulp = true >> - Katello Models are treated as the main entity, with Pulp glue layer >> adding functionality if included >> - Even non-persisted objects have a top-level representation in Katello >> (e.g. errata, package, distribution) >> - Nothing is proxied >> - Basic functionality is orchestrated (CRUD) >> - Use of callbacks and hooks to trigger top-level events that can be >> listened to by glue layer >> - There is no notion of pulp outside the glue layer >> >> >> *Candlepin* >> >> - Contains a single Candlepin only model without corresponding inclusion >> in a Katello entity (OwnerInfo) >> - Contains some proxied functionality mainly for subscription manager >> - Not all Candlepin functionality is contained in glue layer >> - Basic functionality is orchestrated (CRUD) >> >> >> *Foreman* >> >> - Foreman is front and center at all layers >> - Controllers for API and UI are under foreman/ and namespaced >> - Models are under foreman/ and are classed as Foreman:: >> - Katello models that have corresponding Foreman model also get a >> separate model file besides their glue layer counterpart (i.e. >> models/foreman/user) >> - Other models have lightweight representations and are entirely pass >> through objects down to Foreman but are not top-level Katello entities >> - Glue layer is fenced off and only included if use_foreman set to true >> >> >> *ElasticSearch* >> >> - All ES functionality is contained within app/models/glue/elasticsearch >> - All glue layers are modules that included into the corresponding >> Katello model >> - Adds functionality but does not directly impart itself into the models >> - Use of callbacks and hooks to trigger top-level events that can be >> listened to by glue layer >> >> >> I am hoping this will spark some discussion, as I worry that the >> differences in integration points for each backend service will only >> make our lives as developers more difficult down the road. >> >> -Eric >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > > I agree with the Deep Dive. Dmitri had expressed similar concerns. Who > from ? wants to set this up? > > -- bk > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel _______________________________________________ katello-devel mailing list katello-devel at redhat.com https://www.redhat.com/mailman/listinfo/katello-devel From lzap at redhat.com Thu Dec 13 09:49:13 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 10:49:13 +0100 Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <50C8E4B4.10206@redhat.com> References: <20121212190622.GB27952@lzapx.brq.redhat.com> <50C8E4B4.10206@redhat.com> Message-ID: <20121213094913.GB4688@lzapx.brq.redhat.com> On Wed, Dec 12, 2012 at 12:10:28PM -0800, Michael McCune wrote: > On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > >Hey, > > > >for some reason, Redcarpet segfaults for me when I try start Katello in > >development mode. This is very limiting for me, is there a workaround? > >I think this was working, not sure what is wrong. > > > >Thanks > > > > I get the same on EL 6.3, haven't found a workaround yet. Redcarpet is not used in production mode, so subject should be maybe: Redcarpet does not work on RHEL 6.3 at all. There is a bunch of bugreports on their site, about 10 segfaults. I wonder if we have chosen the right dependency. ps - When choosing a dependency, always select the one without any native code. This is an old good Java rule. No optimalization when we do not need it. Mixing interpreted and native code is challenging. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From inecas at redhat.com Thu Dec 13 10:02:51 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 13 Dec 2012 05:02:51 -0500 (EST) Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <20121213094913.GB4688@lzapx.brq.redhat.com> Message-ID: <1766838152.15232612.1355392971434.JavaMail.root@redhat.com> ----- Original Message ----- > On Wed, Dec 12, 2012 at 12:10:28PM -0800, Michael McCune wrote: > > On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > > >Hey, > > > > > >for some reason, Redcarpet segfaults for me when I try start > > >Katello in > > >development mode. This is very limiting for me, is there a > > >workaround? > > >I think this was working, not sure what is wrong. > > > > > >Thanks > > > > > > > I get the same on EL 6.3, haven't found a workaround yet. > > Redcarpet is not used in production mode, so subject should be maybe: > > Redcarpet does not work on RHEL 6.3 at all. > > There is a bunch of bugreports on their site, about 10 segfaults. I > wonder if we have chosen the right dependency. > > ps - When choosing a dependency, always select the one without any > native code. This is an old good Java rule. No optimalization when we > do > not need it. Mixing interpreted and native code is challenging. That's why I've emphased on making this dependency only development level. Switching to another library that conversts markdown to html is really easy: implementing a 10 lines class (including empty lines:) like this https://github.com/Pajk/apipie-rails/blob/master/lib/apipie/markup.rb#L19 The reasons there is RedCarpet used as default are: - large user base (github uses it for rendering their stuff, what else you can wish) - already packaged for Fedora - it worked :) I will try different renderer -- Ivan > > LZ > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From msuchy at redhat.com Thu Dec 13 10:11:04 2012 From: msuchy at redhat.com (Miroslav Suchy) Date: Thu, 13 Dec 2012 11:11:04 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50C8CC4B.5040106@redhat.com> References: <50C8CC4B.5040106@redhat.com> Message-ID: <50C9A9B8.9060706@redhat.com> On 12.12.2012 19:26, Bryan Kearney wrote: > [1] https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > SysAdmin? racks server hardware and notes MAC Address. When I had to provision HW in past, I never knew the MAC address before booting. Probably Dell, IBM or Fujitsu is writing it somewhere on box. But the commodity hardware did not have written it on box. And I do not recall it on Dell either, but I'm not 100% sure. What I like on Satellite was, that you can create activation key and then assign it to kickstart profile. And you can then boot up machine with that kickstart profile: Ie. tftp record would be: LABEL rhel60.x86_64.web-server.ks KERNEL yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz APPEND initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 And when provisioning machine I would just choose boot from PXE and in menu choose label rhel60.x86_64.server.ks and it will be provisioned using kickstart profile rhel-x86_64-webserver-6 and use associated activation key (which mean i.e. install rhel6 in noninteractive mode, install httpd and deploy previously created configuration keys) Can we add this use case to this Design as well? Mirek From inecas at redhat.com Thu Dec 13 10:24:28 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 13 Dec 2012 05:24:28 -0500 (EST) Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <1766838152.15232612.1355392971434.JavaMail.root@redhat.com> Message-ID: <402912280.15247175.1355394268527.JavaMail.root@redhat.com> ----- Original Message ----- > > > ----- Original Message ----- > > On Wed, Dec 12, 2012 at 12:10:28PM -0800, Michael McCune wrote: > > > On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > > > >Hey, > > > > > > > >for some reason, Redcarpet segfaults for me when I try start > > > >Katello in > > > >development mode. This is very limiting for me, is there a > > > >workaround? > > > >I think this was working, not sure what is wrong. > > > > > > > >Thanks > > > > > > > > > > I get the same on EL 6.3, haven't found a workaround yet. > > > > Redcarpet is not used in production mode, so subject should be > > maybe: > > > > Redcarpet does not work on RHEL 6.3 at all. > > > > There is a bunch of bugreports on their site, about 10 segfaults. I > > wonder if we have chosen the right dependency. > > > > ps - When choosing a dependency, always select the one without any > > native code. This is an old good Java rule. No optimalization when > > we > > do > > not need it. Mixing interpreted and native code is challenging. > > That's why I've emphased on making this dependency only development > level. > Switching to another library that conversts markdown to html is > really easy: > implementing a 10 lines class (including empty lines:) like this > > https://github.com/Pajk/apipie-rails/blob/master/lib/apipie/markup.rb#L19 > > The reasons there is RedCarpet used as default are: > > - large user base (github uses it for rendering their stuff, what > else you can wish) > - already packaged for Fedora > - it worked :) > > I will try different renderer >From fast survey I've made here is the winner: https://www.ruby-toolbox.com/projects/maruku Features: - converts markdown to html (quite expected:) - pure ruby - packaged for fedora http://koji.fedoraproject.org/koji/packageinfo?packageID=10063 Expect PR and RPMs for f16 and epel in the afternoon. -- Ivan > > -- Ivan > > > > > LZ > > > > -- > > Later, > > > > Lukas "lzap" Zapletal > > #katello #systemengine > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Thu Dec 13 10:35:29 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 11:35:29 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <91740383.17807534.1355368774620.JavaMail.root@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> Message-ID: <20121213103529.GC4688@lzapx.brq.redhat.com> On Wed, Dec 12, 2012 at 10:19:34PM -0500, Eric Helms wrote: > My intent for creating the diagram and bring to light the various implementation details surrounding each service was not to raise the ActiveResource debate again or to question the Foreman integration. As my diagram showed, Candlepin has connections and touches almost every layer as did PulpV1 integration. Rather, my goal is to bring up a larger conversation about the Katello architecture as a whole going forward. We are at a point where we are migrating to a completely new version of one backend service and re-writing a lot of the Katello bits aroun d it (Pulp) and simultaneously integrating a brand new backend engine (Foreman). I am of the opinion, and I think based on replies that others echo this, that we should have a consistent architecture for the integration of backend services into Katello. No matter what ends up being the choice, consistency is key. > > If there are no other agenda items for the Deep Dive tomorrow, perhaps we can take that time to discuss the state of each backend service integration wise and how we'd like to achieve consistency and a separation of concerns throughout the Katello layers. Speaking about integration, I was collecting items in regard to our orchestration and integration issues for several months now: https://fedorahosted.org/katello/wiki/OrchestrationNG I am not sure if this is topic to raise now (definitely not a deep dive topic), but eventually I would like to talk about it. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Thu Dec 13 10:57:24 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 11:57:24 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50C9A9B8.9060706@redhat.com> References: <50C8CC4B.5040106@redhat.com> <50C9A9B8.9060706@redhat.com> Message-ID: <20121213105724.GD4688@lzapx.brq.redhat.com> On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: > On 12.12.2012 19:26, Bryan Kearney wrote: > >[1] https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > > > > SysAdmin? racks server hardware and notes MAC Address. > > When I had to provision HW in past, I never knew the MAC address > before booting. Probably Dell, IBM or Fujitsu is writing it > somewhere on box. But the commodity hardware did not have written it > on box. And I do not recall it on Dell either, but I'm not 100% > sure. > > What I like on Satellite was, that you can create activation key and > then assign it to kickstart profile. And you can then boot up > machine with that kickstart profile: > > Ie. tftp record would be: > LABEL rhel60.x86_64.web-server.ks > KERNEL > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz > APPEND initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 > > > And when provisioning machine I would just choose boot from PXE and > in menu choose label rhel60.x86_64.server.ks and it will be > provisioned using kickstart profile rhel-x86_64-webserver-6 and use > associated activation key (which mean i.e. install rhel6 in > noninteractive mode, install httpd and deploy previously created > configuration keys) > > Can we add this use case to this Design as well? So basically the way it works is for a particular timeframe, you are making your TFTP server to serve one same configuration (kickstart) for all machines, right? Well, this can only work for small companies. It's more Foreman related question. I think you can do it with Foreman right now (manually editing the template). LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Thu Dec 13 10:59:00 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 11:59:00 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50C9A9B8.9060706@redhat.com> References: <50C8CC4B.5040106@redhat.com> <50C9A9B8.9060706@redhat.com> Message-ID: <20121213105900.GE4688@lzapx.brq.redhat.com> Oh I read it properly now - you manually select the option from the PXE boot menu. I think this is doable in Foreman and we could eventually support it. LZ On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: > On 12.12.2012 19:26, Bryan Kearney wrote: > >[1] https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > > > > SysAdmin? racks server hardware and notes MAC Address. > > When I had to provision HW in past, I never knew the MAC address > before booting. Probably Dell, IBM or Fujitsu is writing it > somewhere on box. But the commodity hardware did not have written it > on box. And I do not recall it on Dell either, but I'm not 100% > sure. > > What I like on Satellite was, that you can create activation key and > then assign it to kickstart profile. And you can then boot up > machine with that kickstart profile: > > Ie. tftp record would be: > LABEL rhel60.x86_64.web-server.ks > KERNEL > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz > APPEND initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 > > > And when provisioning machine I would just choose boot from PXE and > in menu choose label rhel60.x86_64.server.ks and it will be > provisioned using kickstart profile rhel-x86_64-webserver-6 and use > associated activation key (which mean i.e. install rhel6 in > noninteractive mode, install httpd and deploy previously created > configuration keys) > > Can we add this use case to this Design as well? > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From pchalupa at redhat.com Thu Dec 13 11:35:30 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Thu, 13 Dec 2012 12:35:30 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <20121212151118.GI1895@lzapx.brq.redhat.com> References: <50C0DE49.9070903@redhat.com> <50C8996D.7080801@redhat.com> <20121212151118.GI1895@lzapx.brq.redhat.com> Message-ID: <50C9BD82.5020800@redhat.com> On 12.12.12 16:11, Lukas Zapletal wrote: > On Wed, Dec 12, 2012 at 03:49:17PM +0100, Petr Chalupa wrote: >> If you have any additional questions about the Foreman please send >> me an email and I'll prepare answers for deep dive tomorrow. > > I am the most interested in comparing the Foreman and our current > approach among each other. Benefits, drawbacks, differences. > This is how I see it, I hope its comprehensive. Please comment if I've missed something. ## Benefits By introducing other layer of objects to represent remote resources we've got: - less fat models - better responsibility separation - objects to talk to when crud operations are needed on a remote resource (e.g. Foreman::ArchitectureController and its Api relative are doing that) - Resources::AbstractModel is prepared to usage in other back-ends (if we decide to use it) ## Drawbacks - additional layer which is not present in other backends - inconsistency - headpin packaging (I am not exactly sure what went wrong there.) ## Differences I am not aware of any other differences than the ones I've mentioned in previous email. From inecas at redhat.com Thu Dec 13 12:17:42 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 13 Dec 2012 07:17:42 -0500 (EST) Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <402912280.15247175.1355394268527.JavaMail.root@redhat.com> Message-ID: <494287028.15303673.1355401062094.JavaMail.root@redhat.com> The switch from redcarpet to maruku is in this PR. I'm waiting for the gems to build in koji. I will do one final scratch build for this PR once the deps are ready, my local tests went fine. https://github.com/Katello/katello/pull/1249 -- Ivan ----- Original Message ----- > > > ----- Original Message ----- > > > > > > ----- Original Message ----- > > > On Wed, Dec 12, 2012 at 12:10:28PM -0800, Michael McCune wrote: > > > > On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > > > > >Hey, > > > > > > > > > >for some reason, Redcarpet segfaults for me when I try start > > > > >Katello in > > > > >development mode. This is very limiting for me, is there a > > > > >workaround? > > > > >I think this was working, not sure what is wrong. > > > > > > > > > >Thanks > > > > > > > > > > > > > I get the same on EL 6.3, haven't found a workaround yet. > > > > > > Redcarpet is not used in production mode, so subject should be > > > maybe: > > > > > > Redcarpet does not work on RHEL 6.3 at all. > > > > > > There is a bunch of bugreports on their site, about 10 segfaults. > > > I > > > wonder if we have chosen the right dependency. > > > > > > ps - When choosing a dependency, always select the one without > > > any > > > native code. This is an old good Java rule. No optimalization > > > when > > > we > > > do > > > not need it. Mixing interpreted and native code is challenging. > > > > That's why I've emphased on making this dependency only development > > level. > > Switching to another library that conversts markdown to html is > > really easy: > > implementing a 10 lines class (including empty lines:) like this > > > > https://github.com/Pajk/apipie-rails/blob/master/lib/apipie/markup.rb#L19 > > > > The reasons there is RedCarpet used as default are: > > > > - large user base (github uses it for rendering their stuff, what > > else you can wish) > > - already packaged for Fedora > > - it worked :) > > > > I will try different renderer > > >From fast survey I've made here is the winner: > > https://www.ruby-toolbox.com/projects/maruku > > Features: > > - converts markdown to html (quite expected:) > - pure ruby > - packaged for fedora > http://koji.fedoraproject.org/koji/packageinfo?packageID=10063 > > Expect PR and RPMs for f16 and epel in the afternoon. > > -- Ivan > > > > > -- Ivan > > > > > > > > LZ > > > > > > -- > > > Later, > > > > > > Lukas "lzap" Zapletal > > > #katello #systemengine > > > > > > _______________________________________________ > > > katello-devel mailing list > > > katello-devel at redhat.com > > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From bkearney at redhat.com Thu Dec 13 13:25:55 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 13 Dec 2012 08:25:55 -0500 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50C9A9B8.9060706@redhat.com> References: <50C8CC4B.5040106@redhat.com> <50C9A9B8.9060706@redhat.com> Message-ID: <50C9D763.6020504@redhat.com> On 12/13/2012 05:11 AM, Miroslav Suchy wrote: > On 12.12.2012 19:26, Bryan Kearney wrote: >> [1] https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > > > > SysAdmin? racks server hardware and notes MAC Address. > > When I had to provision HW in past, I never knew the MAC address before > booting. Probably Dell, IBM or Fujitsu is writing it somewhere on box. > But the commodity hardware did not have written it on box. And I do not > recall it on Dell either, but I'm not 100% sure. > > What I like on Satellite was, that you can create activation key and > then assign it to kickstart profile. And you can then boot up machine > with that kickstart profile: > > Ie. tftp record would be: > LABEL rhel60.x86_64.web-server.ks > KERNEL > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz > APPEND > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 > > > And when provisioning machine I would just choose boot from PXE and in > menu choose label rhel60.x86_64.server.ks and it will be provisioned > using kickstart profile rhel-x86_64-webserver-6 and use associated > activation key (which mean i.e. install rhel6 in noninteractive mode, > install httpd and deploy previously created configuration keys) > > Can we add this use case to this Design as well? > > Mirek Pulling in Ohad.. I was attempting to reflect what Foreman does today. I may have gotten things wrong. -- bk From ohadlevy at redhat.com Thu Dec 13 13:52:41 2012 From: ohadlevy at redhat.com (Ohad Levy) Date: Thu, 13 Dec 2012 08:52:41 -0500 (EST) Subject: [katello-devel] Foreman Registration Design In-Reply-To: <20121213105724.GD4688@lzapx.brq.redhat.com> Message-ID: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> ----- Original Message ----- | On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: | > On 12.12.2012 19:26, Bryan Kearney wrote: | > >[1] | > >https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign | > | > | > > SysAdmin? racks server hardware and notes MAC Address. | > | > When I had to provision HW in past, I never knew the MAC address | > before booting. Probably Dell, IBM or Fujitsu is writing it | > somewhere on box. But the commodity hardware did not have written | > it | > on box. And I do not recall it on Dell either, but I'm not 100% | > sure. | > Right! this is exactly where the auto discovery work we doing now, where an unknown machine boots into a mini OS, that reports the inventory value back to foreman, so later on (or automatically) you can approve that host and get it rebooting and auto building. of course that VM has no issue like that (as the mac is auto assigned while creating the vm). | > What I like on Satellite was, that you can create activation key | > and | > then assign it to kickstart profile. And you can then boot up | > machine with that kickstart profile: | > | > Ie. tftp record would be: | > LABEL rhel60.x86_64.web-server.ks | > KERNEL | > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz | > APPEND | > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img | > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 | > | > | > And when provisioning machine I would just choose boot from PXE and | > in menu choose label rhel60.x86_64.server.ks and it will be | > provisioned using kickstart profile rhel-x86_64-webserver-6 and use | > associated activation key (which mean i.e. install rhel6 in | > noninteractive mode, install httpd and deploy previously created | > configuration keys) | > | > Can we add this use case to this Design as well? | | So basically the way it works is for a particular timeframe, you are | making your TFTP server to serve one same configuration (kickstart) | for | all machines, right? | | Well, this can only work for small companies. It's more Foreman | related | question. I think you can do it with Foreman right now (manually | editing | the template). You can do it now, Foreman supports Hostgroup based deployment (aka profiles) it does not manage the machine afterwards (with regard to puppet certificates etc, as there is no hostname etc). | | LZ | | -- | Later, | | Lukas "lzap" Zapletal | #katello #systemengine | | _______________________________________________ | katello-devel mailing list | katello-devel at redhat.com | https://www.redhat.com/mailman/listinfo/katello-devel | From bkearney at redhat.com Thu Dec 13 13:55:07 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 13 Dec 2012 08:55:07 -0500 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> Message-ID: <50C9DE3B.8050601@redhat.com> On 12/13/2012 08:52 AM, Ohad Levy wrote: > > > ----- Original Message ----- > | On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: > | > On 12.12.2012 19:26, Bryan Kearney wrote: > | > >[1] > | > >https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > | > > | > > | > > SysAdmin? racks server hardware and notes MAC Address. > | > > | > When I had to provision HW in past, I never knew the MAC address > | > before booting. Probably Dell, IBM or Fujitsu is writing it > | > somewhere on box. But the commodity hardware did not have written > | > it > | > on box. And I do not recall it on Dell either, but I'm not 100% > | > sure. > | > > Right! this is exactly where the auto discovery work we doing now, where an unknown machine boots into > a mini OS, that reports the inventory value back to foreman, so later on (or automatically) you can > approve that host and get it rebooting and auto building. > > of course that VM has no issue like that (as the mac is auto assigned while creating the vm). > > > | > What I like on Satellite was, that you can create activation key > | > and > | > then assign it to kickstart profile. And you can then boot up > | > machine with that kickstart profile: > | > > | > Ie. tftp record would be: > | > LABEL rhel60.x86_64.web-server.ks > | > KERNEL > | > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz > | > APPEND > | > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img > | > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 > | > > | > > | > And when provisioning machine I would just choose boot from PXE and > | > in menu choose label rhel60.x86_64.server.ks and it will be > | > provisioned using kickstart profile rhel-x86_64-webserver-6 and use > | > associated activation key (which mean i.e. install rhel6 in > | > noninteractive mode, install httpd and deploy previously created > | > configuration keys) > | > > | > Can we add this use case to this Design as well? > | > | So basically the way it works is for a particular timeframe, you are > | making your TFTP server to serve one same configuration (kickstart) > | for > | all machines, right? > | > | Well, this can only work for small companies. It's more Foreman > | related > | question. I think you can do it with Foreman right now (manually > | editing > | the template). > > You can do it now, Foreman supports Hostgroup based deployment (aka profiles) > > it does not manage the machine afterwards (with regard to puppet certificates etc, as there is no hostname etc). In this case, you would need to manually register the machine afterwards... yes? -- bk From ohadlevy at redhat.com Thu Dec 13 14:01:58 2012 From: ohadlevy at redhat.com (Ohad Levy) Date: Thu, 13 Dec 2012 09:01:58 -0500 (EST) Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50C9DE3B.8050601@redhat.com> Message-ID: <1885480426.613959.1355407318429.JavaMail.root@redhat.com> ----- Original Message ----- | On 12/13/2012 08:52 AM, Ohad Levy wrote: | > | > | > ----- Original Message ----- | > | On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: | > | > On 12.12.2012 19:26, Bryan Kearney wrote: | > | > >[1] | > | > >https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign | > | > | > | > | > | > > SysAdmin? racks server hardware and notes MAC Address. | > | > | > | > When I had to provision HW in past, I never knew the MAC | > | > address | > | > before booting. Probably Dell, IBM or Fujitsu is writing it | > | > somewhere on box. But the commodity hardware did not have | > | > written | > | > it | > | > on box. And I do not recall it on Dell either, but I'm not 100% | > | > sure. | > | > | > Right! this is exactly where the auto discovery work we doing now, | > where an unknown machine boots into | > a mini OS, that reports the inventory value back to foreman, so | > later on (or automatically) you can | > approve that host and get it rebooting and auto building. | > | > of course that VM has no issue like that (as the mac is auto | > assigned while creating the vm). | > | > | > | > What I like on Satellite was, that you can create activation | > | > key | > | > and | > | > then assign it to kickstart profile. And you can then boot up | > | > machine with that kickstart profile: | > | > | > | > Ie. tftp record would be: | > | > LABEL rhel60.x86_64.web-server.ks | > | > KERNEL | > | > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz | > | > APPEND | > | > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img | > | > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 | > | > | > | > | > | > And when provisioning machine I would just choose boot from PXE | > | > and | > | > in menu choose label rhel60.x86_64.server.ks and it will be | > | > provisioned using kickstart profile rhel-x86_64-webserver-6 and | > | > use | > | > associated activation key (which mean i.e. install rhel6 in | > | > noninteractive mode, install httpd and deploy previously | > | > created | > | > configuration keys) | > | > | > | > Can we add this use case to this Design as well? | > | | > | So basically the way it works is for a particular timeframe, you | > | are | > | making your TFTP server to serve one same configuration | > | (kickstart) | > | for | > | all machines, right? | > | | > | Well, this can only work for small companies. It's more Foreman | > | related | > | question. I think you can do it with Foreman right now (manually | > | editing | > | the template). | > | > You can do it now, Foreman supports Hostgroup based deployment (aka | > profiles) | > | > it does not manage the machine afterwards (with regard to puppet | > certificates etc, as there is no hostname etc). | | In this case, you would need to manually register the machine | afterwards... yes? Yes, but, hostnames etc are always an issue, most people i know simply trash it and reinstall. AFAIK, most people copy paste the mac address (they boot it once/ into bios etc, capture the mac and edit the host in foreman) | | -- bk | | | _______________________________________________ | katello-devel mailing list | katello-devel at redhat.com | https://www.redhat.com/mailman/listinfo/katello-devel | From lzap at redhat.com Thu Dec 13 14:49:08 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 15:49:08 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing Message-ID: <20121213144908.GG4688@lzapx.brq.redhat.com> Guys, now for two days I am working on a nasty bug. When you delete an organization, its red hat provider (by default created with name "Red Hat") is not deleted. If you have imported any manifests there or synced content, it is not deleted too. But the organization itself gets deleted from our database. And there is more - orchestration is somehow broken, so things are also not deleted in backend engines. Even if I fix the provider deletion, deletion orchestration just does not work. Mainly because organization deletion was not working for a while. I think this is typical error that shows major weak points of our orchestration code being tightly coupled with models. Organization deletion was refactored to be a background job, because it can take a long time to delete. The implementation is a bit hacky - each organization has a task_id flag and when it is set to non-nil value, organization is hidden with a default_scope. That means once background task starts, it is immediately invisible to both UI and CLI. This should be a separate process (method or something in our business logic code), that would be started either directly or via delayed_jobs. When quality engineers were testing Katello as a "black box", everything was looking good. But inside there was a background job that (due to our bug) deleted only organization from Katello database leaving all providers, products, repositories and stuff there. Basically it only deleted one record from our database and then it stopped. We have our unit tests that are able to reveal errors in units, and QA have their system tests which tests Katello as a project. But we are missing one important thing - integration tests. Something similar to PulpV2 VCR test suite that is able to test all required HTTP REST calls were made. De-facto standard in enterprise integration is the very similar approach of "recording" interactions between systems and then making stubs and comparing against results. By the way, I have been trained on a software that is called Green Hat (it's proprietary but funny name, right). User story: As a dev, I want decent integration test suite for all backend engines The bug also points on our orchestration - because it is tightly coupled with models, we have designed the orchestration deletion that hacky way. The proper and logical approach is to start a process (or at least a Ruby code bit) that has a procedural structure and does all necessary things in simple steps - like one function or several sub-function calls. In the EI world, these are processes and sub-processes. But since our orchestration is hooked into Katello database, we tend to rely on it for things that should definitely not be written as updates or deletes in our database. This example also shows how important is ability to write some orchestration in one-way messaging pattern (katello integration is nothing else than message handling between backend systems). For example deletion is a typical one-way process that should either finished or suspended until someone investigates what is wrong resuming it or cancelling. This makes recovery much more easier. This is not my invention, but standard approach for most integration projects. In short, katello orchestration should be a separate component/service with independent parties: Katello, Candlepin, Pulp, Foreman. And it should be able to work online or as a background service allowing request-reply or one-way MEPs (message exchange patterns). There are existing solutions like JBoss Drools, Apache Camel, Apache ServiceMix - all Java based. As I don't see feasible to integrate with those, I need to insist on adding tasks that would change way how orchestration works today. Recovery from data inconsistency bugs is _very_ expensive. Actually there are enterprises that have offerings solely dedicated to this topic. User stories: As a dev, I want to detach orchestration from models As a dev, I want to have clean and consistent orchestration code Design out: Process-like orchestration with various MEPs We are not done yet! There is more. We have lots of database hooks and validations. For this particular deletion, there are before_delete hooks and in those it is not sufficient to return false if there is a problem (validation issue or general error). We must throw an exception, otherwise Rails will not rollback the whole transaction. User stories: As a dev, I want all callbacks to be validated to throw errors when transaction should be rolled back As I will only fix for the particular BZ and I will continue investigation about what is wrong, what data was or was not deleted in each particular backend engine and prepare some kind of migration script that will correct data inconsistencies, we should add those onto our backlog, because I will only fix this particular (org deletion) case. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From msuchy at redhat.com Thu Dec 13 15:14:33 2012 From: msuchy at redhat.com (Miroslav Suchy) Date: Thu, 13 Dec 2012 16:14:33 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121213144908.GG4688@lzapx.brq.redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> Message-ID: <50C9F0D9.9040203@redhat.com> On 13.12.2012 15:49, Lukas Zapletal wrote: > But inside there was a background job that (due to our bug) deleted only > organization from Katello database leaving all providers, products, > repositories and stuff there. Basically it only deleted one record from > our database and then it stopped. /me mumble: "It would not happen with foreign keys!" User story: As a dev, I want foreign keys, so db watch integrity for us. Mirek From ehelms at redhat.com Thu Dec 13 15:15:17 2012 From: ehelms at redhat.com (Eric Helms) Date: Thu, 13 Dec 2012 10:15:17 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50C9BD82.5020800@redhat.com> References: <50C0DE49.9070903@redhat.com> <50C8996D.7080801@redhat.com> <20121212151118.GI1895@lzapx.brq.redhat.com> <50C9BD82.5020800@redhat.com> Message-ID: <50C9F105.4030900@redhat.com> Continuing the trend of using pictures to try to express architecture since that typically is the easiest way, I have attempted to produce three different forward looking architecture diagrams. To the best of my understanding of the various potential architecture designs, the links below are intended to show what the Katello architecture would look like if all backend services were re-structured to fit the given design with associated relationships. Notes: * Arrows indicate direction of knowledge ( A -> B meaning A knows about B but not vice versa) * Attempted to incorporate Foreman only, Pulp only, Candlepin only, Katello only and joint models in each diagram to show the integration points Designs: Remote Resources: https://docs.google.com/drawings/d/17C-1xxIh1iuHPfKcfV1Q7s0X579uzhBxhMVWkkw17vU/edit Glue Layer: https://docs.google.com/drawings/d/1QD5VFgEncwhnCg6Jt7Kd8ViR2dHjkmFc5J0CKJJBpzk/edit Glue Layer + Remote Resources Layer: https://docs.google.com/drawings/d/1vVhePWvq_ktnRLf0o3-WhKD2i9dpngcjd1oMNrxIiMM/edit -Eric On 12/13/2012 06:35 AM, Petr Chalupa wrote: > > > On 12.12.12 16:11, Lukas Zapletal wrote: >> On Wed, Dec 12, 2012 at 03:49:17PM +0100, Petr Chalupa wrote: >>> If you have any additional questions about the Foreman please send >>> me an email and I'll prepare answers for deep dive tomorrow. >> >> I am the most interested in comparing the Foreman and our current >> approach among each other. Benefits, drawbacks, differences. >> > > This is how I see it, I hope its comprehensive. Please comment if I've > missed something. > > ## Benefits > > By introducing other layer of objects to represent remote resources > we've got: > - less fat models > - better responsibility separation > - objects to talk to when crud operations are needed on a remote > resource (e.g. Foreman::ArchitectureController and its Api relative > are doing that) > - Resources::AbstractModel is prepared to usage in other back-ends (if > we decide to use it) > > ## Drawbacks > > - additional layer which is not present in other backends - inconsistency > - headpin packaging (I am not exactly sure what went wrong there.) > > ## Differences > > I am not aware of any other differences than the ones I've mentioned > in previous email. > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lzap at redhat.com Thu Dec 13 15:37:22 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 16:37:22 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <50C9F0D9.9040203@redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> Message-ID: <20121213153722.GJ4688@lzapx.brq.redhat.com> On Thu, Dec 13, 2012 at 04:14:33PM +0100, Miroslav Suchy wrote: > User story: > As a dev, I want foreign keys, so db watch integrity for us. +1 -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Thu Dec 13 15:44:37 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 13 Dec 2012 16:44:37 +0100 Subject: [katello-devel] Redcarpet and development mode In-Reply-To: <494287028.15303673.1355401062094.JavaMail.root@redhat.com> References: <402912280.15247175.1355394268527.JavaMail.root@redhat.com> <494287028.15303673.1355401062094.JavaMail.root@redhat.com> Message-ID: <20121213154437.GK4688@lzapx.brq.redhat.com> Thanks for this! LZ On Thu, Dec 13, 2012 at 07:17:42AM -0500, Ivan Necas wrote: > The switch from redcarpet to maruku is in this PR. I'm waiting for the gems to build > in koji. I will do one final scratch build for this PR once the deps are ready, > my local tests went fine. > > https://github.com/Katello/katello/pull/1249 > > -- Ivan > ----- Original Message ----- > > > > > > ----- Original Message ----- > > > > > > > > > ----- Original Message ----- > > > > On Wed, Dec 12, 2012 at 12:10:28PM -0800, Michael McCune wrote: > > > > > On 12/12/2012 11:06 AM, Lukas Zapletal wrote: > > > > > >Hey, > > > > > > > > > > > >for some reason, Redcarpet segfaults for me when I try start > > > > > >Katello in > > > > > >development mode. This is very limiting for me, is there a > > > > > >workaround? > > > > > >I think this was working, not sure what is wrong. > > > > > > > > > > > >Thanks > > > > > > > > > > > > > > > > I get the same on EL 6.3, haven't found a workaround yet. > > > > > > > > Redcarpet is not used in production mode, so subject should be > > > > maybe: > > > > > > > > Redcarpet does not work on RHEL 6.3 at all. > > > > > > > > There is a bunch of bugreports on their site, about 10 segfaults. > > > > I > > > > wonder if we have chosen the right dependency. > > > > > > > > ps - When choosing a dependency, always select the one without > > > > any > > > > native code. This is an old good Java rule. No optimalization > > > > when > > > > we > > > > do > > > > not need it. Mixing interpreted and native code is challenging. > > > > > > That's why I've emphased on making this dependency only development > > > level. > > > Switching to another library that conversts markdown to html is > > > really easy: > > > implementing a 10 lines class (including empty lines:) like this > > > > > > https://github.com/Pajk/apipie-rails/blob/master/lib/apipie/markup.rb#L19 > > > > > > The reasons there is RedCarpet used as default are: > > > > > > - large user base (github uses it for rendering their stuff, what > > > else you can wish) > > > - already packaged for Fedora > > > - it worked :) > > > > > > I will try different renderer > > > > >From fast survey I've made here is the winner: > > > > https://www.ruby-toolbox.com/projects/maruku > > > > Features: > > > > - converts markdown to html (quite expected:) > > - pure ruby > > - packaged for fedora > > http://koji.fedoraproject.org/koji/packageinfo?packageID=10063 > > > > Expect PR and RPMs for f16 and epel in the afternoon. > > > > -- Ivan > > > > > > > > -- Ivan > > > > > > > > > > > LZ > > > > > > > > -- > > > > Later, > > > > > > > > Lukas "lzap" Zapletal > > > > #katello #systemengine > > > > > > > > _______________________________________________ > > > > katello-devel mailing list > > > > katello-devel at redhat.com > > > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > > > > > > > _______________________________________________ > > > katello-devel mailing list > > > katello-devel at redhat.com > > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From adprice at redhat.com Thu Dec 13 15:53:29 2012 From: adprice at redhat.com (Adam Price) Date: Thu, 13 Dec 2012 10:53:29 -0500 (EST) Subject: [katello-devel] cli localization In-Reply-To: <1252065108.47363777.1355413705398.JavaMail.root@redhat.com> Message-ID: <1919940931.47365558.1355414009262.JavaMail.root@redhat.com> hey folks, I spent a fair amount of time over the last few days trying to fix some localization on the CLI. this includes: - finding unlocalized strings in the output - localizing all column names from the CLI output - unifying "Could (not) do something to object [ %s ]" to have brackets in all places it contains a lot of changes, so it would be awesome if I could get a couple ACKs from people. watch for spelling errors or incorrect naming, etc. https://github.com/Katello/katello/pull/1253 -- Adam Price adprice at redhat dot com http://github.com/komidore64 http://www.ohloh.net/accounts/komidore64 From bkearney at redhat.com Thu Dec 13 16:34:16 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 13 Dec 2012 11:34:16 -0500 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> Message-ID: <50CA0388.9000907@redhat.com> On 12/13/2012 08:52 AM, Ohad Levy wrote: > > > ----- Original Message ----- > | On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: > | > On 12.12.2012 19:26, Bryan Kearney wrote: > | > >[1] > | > >https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign > | > > | > > | > > SysAdmin? racks server hardware and notes MAC Address. > | > > | > When I had to provision HW in past, I never knew the MAC address > | > before booting. Probably Dell, IBM or Fujitsu is writing it > | > somewhere on box. But the commodity hardware did not have written > | > it > | > on box. And I do not recall it on Dell either, but I'm not 100% > | > sure. > | > > Right! this is exactly where the auto discovery work we doing now, where an unknown machine boots into > a mini OS, that reports the inventory value back to foreman, so later on (or automatically) you can > approve that host and get it rebooting and auto building. > > of course that VM has no issue like that (as the mac is auto assigned while creating the vm). > > > | > What I like on Satellite was, that you can create activation key > | > and > | > then assign it to kickstart profile. And you can then boot up > | > machine with that kickstart profile: > | > > | > Ie. tftp record would be: > | > LABEL rhel60.x86_64.web-server.ks > | > KERNEL > | > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz > | > APPEND > | > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img > | > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 > | > > | > > | > And when provisioning machine I would just choose boot from PXE and > | > in menu choose label rhel60.x86_64.server.ks and it will be > | > provisioned using kickstart profile rhel-x86_64-webserver-6 and use > | > associated activation key (which mean i.e. install rhel6 in > | > noninteractive mode, install httpd and deploy previously created > | > configuration keys) > | > > | > Can we add this use case to this Design as well? > | > | So basically the way it works is for a particular timeframe, you are > | making your TFTP server to serve one same configuration (kickstart) > | for > | all machines, right? > | > | Well, this can only work for small companies. It's more Foreman > | related > | question. I think you can do it with Foreman right now (manually > | editing > | the template). > > You can do it now, Foreman supports Hostgroup based deployment (aka profiles) > > it does not manage the machine afterwards (with regard to puppet certificates etc, as there is no hostname etc). > Added this to the doc, is it correct? ==== Component Outline Based Provisioning ==== Use case scenario: 1. Sysadmin creates a Component Outline (Host Group) and selects it to be public. 1. Sysadmin racks a machine, and does a NetBoot. 1. Sysadmin selects the profile, which provides a kickstart to build the machine 1. After boot, sysadmin registers the machine with subscription manager (see above) With the Open Question: For Component Outline Provisioning, where does environment fit in? From inecas at redhat.com Thu Dec 13 16:44:25 2012 From: inecas at redhat.com (Ivan Necas) Date: Thu, 13 Dec 2012 11:44:25 -0500 (EST) Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <50C9F0D9.9040203@redhat.com> Message-ID: <16769562.15491221.1355417065500.JavaMail.root@redhat.com> ----- Original Message ----- > On 13.12.2012 15:49, Lukas Zapletal wrote: > > But inside there was a background job that (due to our bug) deleted > > only > > organization from Katello database leaving all providers, products, > > repositories and stuff there. Basically it only deleted one record > > from > > our database and then it stopped. > > /me mumble: "It would not happen with foreign keys!" while agreeing it would help a lot discovering this kind of issue (and +1000 from me), we still need to thing about keeping the whole infra in consistency, where out database is just a subset, things like pulp going down in middle of org deletion might lead to some undefined state, as well as candlepin or foreman. So yes, it will help, and no it won't solve it all. -- Ivan > > User story: > As a dev, I want foreign keys, so db watch integrity for us. > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From daviddavis at redhat.com Thu Dec 13 16:47:27 2012 From: daviddavis at redhat.com (David Davis) Date: Thu, 13 Dec 2012 11:47:27 -0500 (EST) Subject: [katello-devel] How to have TravisCI run all your code Message-ID: <1086289986.49044371.1355417247048.JavaMail.root@redhat.com> These steps will help you setup TravisCI so that it runs all your branches. It's like having your own personal instance of Jenkins. 1. Got to https://travis-ci.org/ and log in with Github 2. Click on your name in the top right and then accounts 3. Find your_name/katello and flip the switch to 'on' 4. Enjoy automated testing every time you push to github! David From jrist at redhat.com Thu Dec 13 17:17:42 2012 From: jrist at redhat.com (Jason Rist) Date: Thu, 13 Dec 2012 10:17:42 -0700 Subject: [katello-devel] How to have TravisCI run all your code In-Reply-To: <1086289986.49044371.1355417247048.JavaMail.root@redhat.com> References: <1086289986.49044371.1355417247048.JavaMail.root@redhat.com> Message-ID: <50CA0DB6.4050107@redhat.com> On Thu 13 Dec 2012 09:47:27 AM MST, David Davis wrote: > These steps will help you setup TravisCI so that it runs all your > branches. It's like having your own personal instance of Jenkins. > > 1. Got to https://travis-ci.org/ and log in with Github > > 2. Click on your name in the top right and then accounts > > 3. Find your_name/katello and flip the switch to 'on' > > 4. Enjoy automated testing every time you push to github! > > David > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel I've done this as well and it's saved me several times :) +100 -- Jason E. Rist Senior Software Engineer Systems Management and Cloud Enablement Red Hat, Inc. +1.919.754.4048 Freenode: jrist github/identi.ca: knowncitizen From bkearney at redhat.com Thu Dec 13 20:03:00 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 13 Dec 2012 15:03:00 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <20121213103529.GC4688@lzapx.brq.redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> Message-ID: <50CA3474.90301@redhat.com> On 12/13/2012 05:35 AM, Lukas Zapletal wrote: > On Wed, Dec 12, 2012 at 10:19:34PM -0500, Eric Helms wrote: >> My intent for creating the diagram and bring to light the various implementation details surrounding each service was not to raise the ActiveResource debate again or to question the Foreman integration. As my diagram showed, Candlepin has connections and touches almost every layer as did PulpV1 integration. Rather, my goal is to bring up a larger conversation about the Katello architecture as a whole going forward. We are at a point where we are migrating to a completely new version of one backend service and re-writing a lot of the Katello bits aroun d it (Pulp) and simultaneously integrating a brand new backend engine (Foreman). I am of the opinion, and I think based on replies that others echo this, that we should have a consistent architecture for the integration of backend services into Katello. No matter what ends up being the choice, consistency is key. >> >> If there are no other agenda items for the Deep Dive tomorrow, perhaps we can take that time to discuss the state of each backend service integration wise and how we'd like to achieve consistency and a separation of concerns throughout the Katello layers. > > Speaking about integration, I was collecting items in regard to our > orchestration and integration issues for several months now: > > https://fedorahosted.org/katello/wiki/OrchestrationNG > > I am not sure if this is topic to raise now (definitely not a deep dive > topic), but eventually I would like to talk about it. > > LZ > I took a read of the thread, and asked Eric for a quick "talk to me like I am a 2 year old" review. This is what I came away with. The original glue layer attempted to enforce the following practices: * The controller layer (/controllers and /controllers/api) should not be aware of the backend engine providing the implementation. * Any object which the controller would interact with should live in the model tier (/models), since that is standard rails * A glue layer (/model/glue) would enhance the model tier and handle reading/writing to the backend. * All transport to the backend systems is external (in /lib/resources) So, things which go against this pattern in the code base (Master) today include: A) Controllers calling the glue layer directly (see packages_controller.rb) B) Model objects which are backend specific (see /models/candlepin/*, and models/foreman/*) C) Model objects not living in /app/models. D) App specific namespaces (see controllers/foreman) Eric, have I understood the main issues correctly? My guess is that (C) would be the larger issue in the current codebase, since it would require a level of abstrction be added into the existing foreman and candlepin paths. -- bk From ehelms at redhat.com Thu Dec 13 21:53:06 2012 From: ehelms at redhat.com (Eric Helms) Date: Thu, 13 Dec 2012 16:53:06 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CA3474.90301@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> Message-ID: <50CA4E42.3030105@redhat.com> On 12/13/2012 03:03 PM, Bryan Kearney wrote: > On 12/13/2012 05:35 AM, Lukas Zapletal wrote: >> On Wed, Dec 12, 2012 at 10:19:34PM -0500, Eric Helms wrote: >>> My intent for creating the diagram and bring to light the various >>> implementation details surrounding each service was not to raise the >>> ActiveResource debate again or to question the Foreman integration. >>> As my diagram showed, Candlepin has connections and touches almost >>> every layer as did PulpV1 integration. Rather, my goal is to bring >>> up a larger conversation about the Katello architecture as a whole >>> going forward. We are at a point where we are migrating to a >>> completely new version of one backend service and re-writing a lot >>> of the Katello bits aroun d it (Pulp) and simultaneously integrating >>> a brand new backend engine (Foreman). I am of the opinion, and I >>> think based on replies that others echo this, that we should have a >>> consistent architecture for the integration of backend services into >>> Katello. No matter what ends up being the choice, consistency is key. >>> >>> If there are no other agenda items for the Deep Dive tomorrow, >>> perhaps we can take that time to discuss the state of each backend >>> service integration wise and how we'd like to achieve consistency >>> and a separation of concerns throughout the Katello layers. >> >> Speaking about integration, I was collecting items in regard to our >> orchestration and integration issues for several months now: >> >> https://fedorahosted.org/katello/wiki/OrchestrationNG >> >> I am not sure if this is topic to raise now (definitely not a deep dive >> topic), but eventually I would like to talk about it. >> >> LZ >> > I took a read of the thread, and asked Eric for a quick "talk to me > like I am a 2 year old" review. This is what I came away with. > > The original glue layer attempted to enforce the following practices: > > * The controller layer (/controllers and /controllers/api) should not > be aware of the backend engine providing the implementation. > * Any object which the controller would interact with should live in > the model tier (/models), since that is standard rails > * A glue layer (/model/glue) would enhance the model tier and handle > reading/writing to the backend. > * All transport to the backend systems is external (in /lib/resources) > > So, things which go against this pattern in the code base (Master) > today include: > > A) Controllers calling the glue layer directly (see > packages_controller.rb) > B) Model objects which are backend specific (see /models/candlepin/*, > and models/foreman/*) > C) Model objects not living in /app/models. > D) App specific namespaces (see controllers/foreman) > > Eric, have I understood the main issues correctly? My guess is that > (C) would be the larger issue in the current codebase, since it would > require a level of abstrction be added into the existing foreman and > candlepin paths. Yep - I think you have nicely summarized the biggest issues around architecture inconsistency as they stand in Master. > > -- bk > > > > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ehelms at redhat.com Thu Dec 13 21:57:55 2012 From: ehelms at redhat.com (Eric Helms) Date: Thu, 13 Dec 2012 16:57:55 -0500 Subject: [katello-devel] Katello 1.2 RC2 Message-ID: <50CA4F63.6070401@redhat.com> Howdy All, This second release candidate for Katello 1.2 is now available and hopefully represents the final release candidate as these are now fully signed RPMS. There have been no major code changes since RC1; however, if anyone has any bandwidth over the next ~24 hours to test install and/or update with these RPMs for sanity please do. There are candidates for: Fedora 16 -http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/16/x86_64/katello-repos-1.2.4-1.fc16.noarch.rpm Fedora 17 - http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora/17/x86_64/katello-repos-1.2.4-1.fc17.noarch.rpm RHEL 6 - http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL/6/x86_64/katello-repos-1.2.4-1.el6.noarch.rpm Repos: http://fedorapeople.org/groups/katello/releases/yum/1.2/ Installation and Upgrade Instructions: https://fedorahosted.org/katello/wiki/Install Report Bugs: https://fedorahosted.org/katello/wiki/ReportErrors Thanks, Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmccune at redhat.com Thu Dec 13 23:40:18 2012 From: mmccune at redhat.com (Mike McCune) Date: Thu, 13 Dec 2012 15:40:18 -0800 Subject: [katello-devel] Katello 1.2 RC2 In-Reply-To: <50CA4F63.6070401@redhat.com> References: <50CA4F63.6070401@redhat.com> Message-ID: <50CA6762.4000606@redhat.com> On 12/13/2012 01:57 PM, Eric Helms wrote: > Howdy All, > > This second release candidate for Katello 1.2 is now available and > hopefully represents the final release candidate as these are now fully > signed RPMS. There have been no major code changes since RC1; however, > if anyone has any bandwidth over the next ~24 hours to test install > and/or update with these RPMs for sanity please do. There are candidates > for: notes and status: https://pad-katello.rhcloud.com/p/katello-1.2rc2-testing Mike -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From inecas at redhat.com Fri Dec 14 08:06:33 2012 From: inecas at redhat.com (Ivan Necas) Date: Fri, 14 Dec 2012 03:06:33 -0500 (EST) Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121213144908.GG4688@lzapx.brq.redhat.com> Message-ID: <834056342.15763660.1355472393919.JavaMail.root@redhat.com> ----- Original Message ----- > Guys, > > now for two days I am working on a nasty bug. When you delete an > organization, its red hat provider (by default created with name "Red > Hat") is not deleted. If you have imported any manifests there or > synced > content, it is not deleted too. But the organization itself gets > deleted > from our database. And there is more - orchestration is somehow > broken, > so things are also not deleted in backend engines. Even if I fix the > provider deletion, deletion orchestration just does not work. Mainly > because organization deletion was not working for a while. > > I think this is typical error that shows major weak points of our > orchestration code being tightly coupled with models. Organization > deletion was refactored to be a background job, because it can take a > long time to delete. > > The implementation is a bit hacky - each organization has a task_id > flag > and when it is set to non-nil value, organization is hidden with a > default_scope. That means once background task starts, it is > immediately > invisible to both UI and CLI. > > This should be a separate process (method or something in our > business > logic code), that would be started either directly or via > delayed_jobs. > When quality engineers were testing Katello as a "black box", > everything > was looking good. > > But inside there was a background job that (due to our bug) deleted > only > organization from Katello database leaving all providers, products, > repositories and stuff there. Basically it only deleted one record > from > our database and then it stopped. I'm not 100% sure this happened every time an org was deleted. In that case, we wouldn't be able to run system tests more times in a row. > > We have our unit tests that are able to reveal errors in units, and > QA > have their system tests which tests Katello as a project. But we are > missing one important thing - integration tests. Something similar to > PulpV2 VCR test suite that is able to test all required HTTP REST > calls > were made. De-facto standard in enterprise integration is the very > similar approach of "recording" interactions between systems and then > making stubs and comparing against results. By the way, I have been > trained on a software that is called Green Hat (it's proprietary but > funny name, right). > > User story: > > As a dev, I want decent integration test suite for all backend > engines +1 > > The bug also points on our orchestration - because it is tightly > coupled > with models, we have designed the orchestration deletion that hacky > way. > The proper and logical approach is to start a process (or at least a > Ruby code bit) that has a procedural structure and does all necessary > things in simple steps - like one function or several sub-function > calls. In the EI world, these are processes and sub-processes. > > But since our orchestration is hooked into Katello database, we tend > to > rely on it for things that should definitely not be written as > updates > or deletes in our database. > > This example also shows how important is ability to write some > orchestration in one-way messaging pattern (katello integration is > nothing else than message handling between backend systems). For > example > deletion is a typical one-way process that should either finished or > suspended until someone investigates what is wrong resuming it or > cancelling. This makes recovery much more easier. This is not my > invention, but standard approach for most integration projects. > > In short, katello orchestration should be a separate > component/service > with independent parties: Katello, Candlepin, Pulp, Foreman. And it > should be able to work online or as a background service allowing > request-reply or one-way MEPs (message exchange patterns). > > There are existing solutions like JBoss Drools, Apache Camel, Apache > ServiceMix - all Java based. As I don't see feasible to integrate > with > those, I need to insist on adding tasks that would change way how > orchestration works today. With JRuby, reusing this tools might be feasible, right? > > Recovery from data inconsistency bugs is _very_ expensive. Actually > there are enterprises that have offerings solely dedicated to this > topic. > > User stories: > > As a dev, I want to detach orchestration from models > As a dev, I want to have clean and consistent orchestration code > Design out: Process-like orchestration with various MEPs > > We are not done yet! There is more. We have lots of database hooks > and > validations. For this particular deletion, there are before_delete > hooks > and in those it is not sufficient to return false if there is a > problem > (validation issue or general error). We must throw an exception, > otherwise Rails will not rollback the whole transaction. > > User stories: > > As a dev, I want all callbacks to be validated to throw errors when > transaction should be rolled back > > As I will only fix for the particular BZ and I will continue We also need better testing for edge cases > investigation about what is wrong, what data was or was not deleted > in > each particular backend engine and prepare some kind of migration > script > that will correct data inconsistencies, we should add those onto our > backlog, because I will only fix this particular (org deletion) case. We also need better testing for edge cases. E.g. what happens when I delete an organization and restart all the services in the mean time. How can I recover? Trying to solve this issues can point us the the weak points (which will be probably very similar to those described in this thread.) I agree neither black box testing nor unit tests will help us much here. Unit test because of the isolation, black box because everything might seem working at the moment and knowledge of the code base can help finding the issue. Stories: As a integrator, I want to have the edge cases automated. As a integrator, I want to provide standardized ways for recovering from various non-standard situations. When talking about keeping consistency between the systems, the first obvious step, as Mirek pointed out, is to finally introduce foreign keys to keep the consistency in our own database. This should be #1 priority, if we don't want to spend the rest of our lives fixing bugs like this one. -- Ivan > > LZ > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Fri Dec 14 09:05:00 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 14 Dec 2012 10:05:00 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CA3474.90301@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> Message-ID: <20121214090500.GD1653@lzapx.brq.redhat.com> On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: > The original glue layer attempted to enforce the following practices: > > * The controller layer (/controllers and /controllers/api) should > not be aware of the backend engine providing the implementation. > * Any object which the controller would interact with should live in > the model tier (/models), since that is standard rails > * A glue layer (/model/glue) would enhance the model tier and handle > reading/writing to the backend. > * All transport to the backend systems is external (in /lib/resources) I know this thread is about Foreman vs others, but since you replied to my mail I need to write this over and over again. With the current orchestration: We tightly couple model and orchestration while orchestration should be independent. Katello model is just one of the parties we want to integrate with. We can only orchestrate things that have resources in our database and for all the others we need to "hack" alternate models to get this working. We tend to do long running orchestration by hacking model classes (like org.task_id) which is weird - we are abusing models to do orchestration work. Those things are mixed. Our codebase is difficult to understand - we are building chains of callbacks that are deeply hidden in our model objects. Integration proces is a simple function with several statements while we spread it across multiple classes. Errors in our model lead to errors in orchestration. While data inconsistency in our katello db are really bad, they are much easier to fix comparing with inconsistencies across multiple backend engines. Our orchestration code is pretty untestable as a standalone component and the whole codebase is error prone. Katello models are error prone because we have so much code logic in there, so many hooks, callbacks and validations needed for orchestration. They don't need to be there. Our unit tests are either testing models with orchestration turned off or models and orchestration with turned it on. No separation here, tests are mixed into each other. This approach came from foreman where it works pretty well, but Foreman's orchestration is more different from Katello than we thought it will be. I said it like twenty times and I can say it again: Let's move our orchestration out of our models. Let's refactor this while we still have some chance to do this. Now is the time, once we agree this is the "best" approach and maybe rewrite foreman part into this pattern, there is no easy way out. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Fri Dec 14 09:14:12 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 14 Dec 2012 10:14:12 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <16769562.15491221.1355417065500.JavaMail.root@redhat.com> References: <50C9F0D9.9040203@redhat.com> <16769562.15491221.1355417065500.JavaMail.root@redhat.com> Message-ID: <20121214091412.GE1653@lzapx.brq.redhat.com> On Thu, Dec 13, 2012 at 11:44:25AM -0500, Ivan Necas wrote: > we still need to thing about keeping the whole infra in consistency, where out database > is just a subset, things like pulp going down in middle of org deletion might lead > to some undefined state, as well as candlepin or foreman. But at least we get notified something is wrong. Without foreign keys, it breaks the same way - the only difference is we sometimes do not notice it. I agree, we need to take orchestration out of our models. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From inecas at redhat.com Fri Dec 14 09:20:12 2012 From: inecas at redhat.com (Ivan Necas) Date: Fri, 14 Dec 2012 04:20:12 -0500 (EST) Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <20121214090500.GD1653@lzapx.brq.redhat.com> Message-ID: <195005560.15796514.1355476812071.JavaMail.root@redhat.com> ----- Original Message ----- > On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: > > The original glue layer attempted to enforce the following > > practices: > > > > * The controller layer (/controllers and /controllers/api) should > > not be aware of the backend engine providing the implementation. > > * Any object which the controller would interact with should live > > in > > the model tier (/models), since that is standard rails > > * A glue layer (/model/glue) would enhance the model tier and > > handle > > reading/writing to the backend. > > * All transport to the backend systems is external (in > > /lib/resources) > > I know this thread is about Foreman vs others, but since you replied > to > my mail I need to write this over and over again. With the current > orchestration: > > We tightly couple model and orchestration while orchestration should > be independent. Katello model is just one of the parties we want to > integrate with. > > We can only orchestrate things that have resources in our database > and > for all the others we need to "hack" alternate models to get this > working. > > We tend to do long running orchestration by hacking model classes > (like > org.task_id) which is weird - we are abusing models to do > orchestration > work. Those things are mixed. > > Our codebase is difficult to understand - we are building chains of > callbacks that are deeply hidden in our model objects. Integration > proces is a simple function with several statements while we spread > it > across multiple classes. > > Errors in our model lead to errors in orchestration. While data > inconsistency in our katello db are really bad, they are much easier > to > fix comparing with inconsistencies across multiple backend engines. > > Our orchestration code is pretty untestable as a standalone component > and the whole codebase is error prone. > > Katello models are error prone because we have so much code logic in > there, so many hooks, callbacks and validations needed for > orchestration. They don't need to be there. > > Our unit tests are either testing models with orchestration turned > off > or models and orchestration with turned it on. No separation here, > tests > are mixed into each other. > > This approach came from foreman where it works pretty well, but > Foreman's orchestration is more different from Katello than we > thought > it will be. Just note on the difference: Foreman orchestrates mainly with the smart proxy, and the difference is the proxy barely stores any data, far from what we do in our subsystems. Also not long-running tasks are involved in the Foreman orchestration. And also, if something goes from in Foreman orchestration, the fix consists mainly form editing some config file for dns/dhcp/tftp, which can be done by the sys admin, and in ideal case he know what is doing. When something does wrong while calling CP, Pulp or Foreman from Katello, there is not so straight way to fix that. > > I said it like twenty times and I can say it again: Let's move our > orchestration out of our models. Let's refactor this while we still > have > some chance to do this. Now is the time, once we agree this is the > "best" approach and maybe rewrite foreman part into this pattern, > there > is no easy way out. +1 > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Fri Dec 14 09:23:52 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 14 Dec 2012 10:23:52 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <834056342.15763660.1355472393919.JavaMail.root@redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <834056342.15763660.1355472393919.JavaMail.root@redhat.com> Message-ID: <20121214092352.GF1653@lzapx.brq.redhat.com> On Fri, Dec 14, 2012 at 03:06:33AM -0500, Ivan Necas wrote: > With JRuby, reusing this tools might be feasible, right? It does not depend on which platform are we running Katello on, integration with such a service is usually external - via an API like web-service, REST service or messaging. Engine is a standalone process usually. I don't thing this is acceptable from the management point of view. But I think it is feasible to refactor this. > We also need better testing for edge cases. E.g. what happens when I delete > an organization and restart all the services in the mean time. How can I recover? > Trying to solve this issues can point us the the weak points (which will be probably > very similar to those described in this thread.) > > I agree neither black box testing nor unit tests will help us much here. Unit test because > of the isolation, black box because everything might seem working at the moment and > knowledge of the code base can help finding the issue. > > > Stories: > > As a integrator, I want to have the edge cases automated. > As a integrator, I want to provide standardized ways for recovering from various non-standard situations. > > When talking about keeping consistency between the systems, the first obvious step, as Mirek pointed out, is to finally introduce foreign keys > to keep the consistency in our own database. This should be #1 priority, if we don't want to spend the rest of our lives fixing bugs like this > one. Well, with decent integration approach you don't really need this - data changes should be one-way only. If any process changing our database gets stuck, you get notified and you can easily make it finish (e.g. after you restart services involved). The main advantage of this approach is you (as a caller) are not responsible error recovery anymore. Then you don't need to focus so much on testing edge (error) cases. But if we stick with our current "web server process rules them all" approach, then we would need that. Many more tests. I'd start with implementing integration oriented tests (that should come with PulpV2 and minitest/webmock/facorygirl/vcr merge). -- Later, Lukas "lzap" Zapletal #katello #systemengine From msuchy at redhat.com Fri Dec 14 09:33:31 2012 From: msuchy at redhat.com (Miroslav Suchy) Date: Fri, 14 Dec 2012 10:33:31 +0100 Subject: [katello-devel] Katello 1.2 RC2 In-Reply-To: <50CA4F63.6070401@redhat.com> References: <50CA4F63.6070401@redhat.com> Message-ID: <50CAF26B.8020100@redhat.com> On 13.12.2012 22:57, Eric Helms wrote: > Howdy All, > > This second release candidate for Katello 1.2 is now available and > hopefully represents the final release candidate as these are now fully > signed RPMS. There have been no major code changes since RC1; however, > if anyone has any bandwidth over the next ~24 hours to test install > and/or update with these RPMs for sanity please do. There are candidates > for: After installation of katello-devel-all and katello restart it crash within dozen seconds. It is caused by bad rubygem-jshintrb. This is already solved in master. Just tag latest rubygem-jshintrb into katello-1.2-* tags, sign it and create new repo. Mirek From tstrachota at redhat.com Fri Dec 14 09:49:23 2012 From: tstrachota at redhat.com (Tomas Strachota) Date: Fri, 14 Dec 2012 10:49:23 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121213153722.GJ4688@lzapx.brq.redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> <20121213153722.GJ4688@lzapx.brq.redhat.com> Message-ID: <50CAF623.1030402@redhat.com> On 12/13/2012 04:37 PM, Lukas Zapletal wrote: > On Thu, Dec 13, 2012 at 04:14:33PM +0100, Miroslav Suchy wrote: >> User story: >> As a dev, I want foreign keys, so db watch integrity for us. > > +1 > +1000 to both foreign keys and decoupled orchestration. I think all proposed tasks are steps in the right direction to make our orchestration more solid. T. From inecas at redhat.com Fri Dec 14 10:43:35 2012 From: inecas at redhat.com (Ivan Necas) Date: Fri, 14 Dec 2012 05:43:35 -0500 (EST) Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121214092352.GF1653@lzapx.brq.redhat.com> Message-ID: <257727672.15843341.1355481815496.JavaMail.root@redhat.com> ----- Original Message ----- > On Fri, Dec 14, 2012 at 03:06:33AM -0500, Ivan Necas wrote: > > With JRuby, reusing this tools might be feasible, right? > > It does not depend on which platform are we running Katello on, > integration with such a service is usually external - via an API like > web-service, REST service or messaging. Engine is a standalone > process > usually. > > I don't thing this is acceptable from the management point of view. > But > I think it is feasible to refactor this. > > > We also need better testing for edge cases. E.g. what happens when > > I delete > > an organization and restart all the services in the mean time. How > > can I recover? > > Trying to solve this issues can point us the the weak points (which > > will be probably > > very similar to those described in this thread.) > > > > I agree neither black box testing nor unit tests will help us much > > here. Unit test because > > of the isolation, black box because everything might seem working > > at the moment and > > knowledge of the code base can help finding the issue. > > > > > > Stories: > > > > As a integrator, I want to have the edge cases automated. > > As a integrator, I want to provide standardized ways for recovering > > from various non-standard situations. > > > > When talking about keeping consistency between the systems, the > > first obvious step, as Mirek pointed out, is to finally introduce > > foreign keys > > to keep the consistency in our own database. This should be #1 > > priority, if we don't want to spend the rest of our lives fixing > > bugs like this > > one. > > Well, with decent integration approach you don't really need this - > data > changes should be one-way only. If any process changing our database > gets stuck, you get notified and you can easily make it finish (e.g. > after you restart services involved). > > The main advantage of this approach is you (as a caller) are not > responsible error recovery anymore. Then you don't need to focus so > much > on testing edge (error) cases. And who is responsible for that then? -- Ivan > > But if we stick with our current "web server process rules them all" > approach, then we would need that. Many more tests. I'd start with > implementing integration oriented tests (that should come with PulpV2 > and minitest/webmock/facorygirl/vcr merge). > > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From pchalupa at redhat.com Fri Dec 14 13:51:54 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Fri, 14 Dec 2012 14:51:54 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <20121214090500.GD1653@lzapx.brq.redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> <20121214090500.GD1653@lzapx.brq.redhat.com> Message-ID: <50CB2EFA.9030909@redhat.com> On 14.12.12 10:05, Lukas Zapletal wrote: > On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: >> The original glue layer attempted to enforce the following practices: >> >> * The controller layer (/controllers and /controllers/api) should >> not be aware of the backend engine providing the implementation. >> * Any object which the controller would interact with should live in >> the model tier (/models), since that is standard rails >> * A glue layer (/model/glue) would enhance the model tier and handle >> reading/writing to the backend. >> * All transport to the backend systems is external (in /lib/resources) > > I know this thread is about Foreman vs others, but since you replied to > my mail I need to write this over and over again. With the current > orchestration: > > We tightly couple model and orchestration while orchestration should > be independent. Katello model is just one of the parties we want to > integrate with. > > We can only orchestrate things that have resources in our database and > for all the others we need to "hack" alternate models to get this > working. > > We tend to do long running orchestration by hacking model classes (like > org.task_id) which is weird - we are abusing models to do orchestration > work. Those things are mixed. > > Our codebase is difficult to understand - we are building chains of > callbacks that are deeply hidden in our model objects. Integration > proces is a simple function with several statements while we spread it > across multiple classes. > > Errors in our model lead to errors in orchestration. While data > inconsistency in our katello db are really bad, they are much easier to > fix comparing with inconsistencies across multiple backend engines. > > Our orchestration code is pretty untestable as a standalone component > and the whole codebase is error prone. > > Katello models are error prone because we have so much code logic in > there, so many hooks, callbacks and validations needed for > orchestration. They don't need to be there. > > Our unit tests are either testing models with orchestration turned off > or models and orchestration with turned it on. No separation here, tests > are mixed into each other. > > This approach came from foreman where it works pretty well, but > Foreman's orchestration is more different from Katello than we thought > it will be. > > I said it like twenty times and I can say it again: Let's move our > orchestration out of our models. Let's refactor this while we still have > some chance to do this. Now is the time, once we agree this is the > "best" approach and maybe rewrite foreman part into this pattern, there > is no easy way out. > I agree with Lukas. Our orchestration is fragile and doesn't feel good. # What I think we can do right now We could at least start cleaning up the current orchestration code without changing its design. I think we would benefit a lot if we just removed the model-tangle by replacing modules with classes. see attachment ## Image legend - blue represents gems - all black nodes are classes, no modules - nodes with full line are present classes - dashed nodes are classes which aren't implemented - black full edge represents oriented knowledge - dashed edges represents inheritance You can modify and regenerate the pdf with graphwiz: `dot graph_orchest.dot -Tpdf graph_orchest.dot.pdf` ## Class legend of new classes AbstractKatelloRecord: Abstract parent of all katello ActiveRecord models AbstractOrchestratedRecord: Abstract parent of all katello ActiveRecord models with any orchestration. Glue::Abstract, Glue::Foreman::Abstract, Glue::Pulp::Abstract: abstract parent orchestration classes for all, foreman and pulp Relation between Glue and ActiveRecords objects: will be very similar to a observer pattern. An AbstractOrchestratedRecord descendant (e.g. User instance) will send events (generated by ActiveRecord callbacks) to any number of registered Glue::Abstract descendants (e.g. Glue::Foreman::User instance). ## Benefits are: - separated logic - not so fat ActiveRecord models - Glue object has its own namespace, no collisions inside model - Orchestration switching off/on is simple as un/registering glue objects from AbstractOrchestratedRecord descendants # And for future I think we should start exploring other solutions. I like Lukas's idea to build orchestration around processes (ruote [1]) very much. [1] http://ruote.rubyforge.org/ Petr # Notes How to define Abstract class from ActiveRecord::Base: class AbstractKatelloRecord < ActiveRecord::Base self.abstract_class = true # ... end This class then doesn't have table, is considered abstract by ActiveRecord and doesn't break STI. -------------- next part -------------- A non-text attachment was scrubbed... Name: graph_orchest.dot Type: application/msword Size: 1631 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graph_orchest.dot.pdf Type: application/pdf Size: 21087 bytes Desc: not available URL: From bkearney at redhat.com Fri Dec 14 15:55:04 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Fri, 14 Dec 2012 10:55:04 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CB2EFA.9030909@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> <20121214090500.GD1653@lzapx.brq.redhat.com> <50CB2EFA.9030909@redhat.com> Message-ID: <50CB4BD8.60705@redhat.com> On 12/14/2012 08:51 AM, Petr Chalupa wrote: > > > On 14.12.12 10:05, Lukas Zapletal wrote: >> On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: >>> The original glue layer attempted to enforce the following practices: >>> >>> * The controller layer (/controllers and /controllers/api) should >>> not be aware of the backend engine providing the implementation. >>> * Any object which the controller would interact with should live in >>> the model tier (/models), since that is standard rails >>> * A glue layer (/model/glue) would enhance the model tier and handle >>> reading/writing to the backend. >>> * All transport to the backend systems is external (in /lib/resources) >> >> I know this thread is about Foreman vs others, but since you replied to >> my mail I need to write this over and over again. With the current >> orchestration: >> >> We tightly couple model and orchestration while orchestration should >> be independent. Katello model is just one of the parties we want to >> integrate with. >> >> We can only orchestrate things that have resources in our database and >> for all the others we need to "hack" alternate models to get this >> working. >> >> We tend to do long running orchestration by hacking model classes (like >> org.task_id) which is weird - we are abusing models to do orchestration >> work. Those things are mixed. >> >> Our codebase is difficult to understand - we are building chains of >> callbacks that are deeply hidden in our model objects. Integration >> proces is a simple function with several statements while we spread it >> across multiple classes. >> >> Errors in our model lead to errors in orchestration. While data >> inconsistency in our katello db are really bad, they are much easier to >> fix comparing with inconsistencies across multiple backend engines. >> >> Our orchestration code is pretty untestable as a standalone component >> and the whole codebase is error prone. >> >> Katello models are error prone because we have so much code logic in >> there, so many hooks, callbacks and validations needed for >> orchestration. They don't need to be there. >> >> Our unit tests are either testing models with orchestration turned off >> or models and orchestration with turned it on. No separation here, tests >> are mixed into each other. >> >> This approach came from foreman where it works pretty well, but >> Foreman's orchestration is more different from Katello than we thought >> it will be. >> >> I said it like twenty times and I can say it again: Let's move our >> orchestration out of our models. Let's refactor this while we still have >> some chance to do this. Now is the time, once we agree this is the >> "best" approach and maybe rewrite foreman part into this pattern, there >> is no easy way out. >> > > I agree with Lukas. Our orchestration is fragile and doesn't feel good. > > # What I think we can do right now > > We could at least start cleaning up the current orchestration code > without changing its design. I think we would benefit a lot if we just > removed the model-tangle by replacing modules with classes. > > see attachment > > ## Image legend > > - blue represents gems > - all black nodes are classes, no modules > - nodes with full line are present classes > - dashed nodes are classes which aren't implemented > - black full edge represents oriented knowledge > - dashed edges represents inheritance > > You can modify and regenerate the pdf with graphwiz: > `dot graph_orchest.dot -Tpdf graph_orchest.dot.pdf` > > ## Class legend of new classes > > AbstractKatelloRecord: Abstract parent of all katello ActiveRecord models > > AbstractOrchestratedRecord: Abstract parent of all katello ActiveRecord > models with any orchestration. > > Glue::Abstract, Glue::Foreman::Abstract, Glue::Pulp::Abstract: abstract > parent orchestration classes for all, foreman and pulp > > Relation between Glue and ActiveRecords objects: will be very similar to > a observer pattern. An AbstractOrchestratedRecord descendant (e.g. User > instance) will send events (generated by ActiveRecord callbacks) to any > number of registered Glue::Abstract descendants (e.g. > Glue::Foreman::User instance). > > ## Benefits are: > - separated logic > - not so fat ActiveRecord models > - Glue object has its own namespace, no collisions inside model > - Orchestration switching off/on is simple as un/registering glue > objects from AbstractOrchestratedRecord descendants > > # And for future > > I think we should start exploring other solutions. I like Lukas's idea > to build orchestration around processes (ruote [1]) very much. > > [1] http://ruote.rubyforge.org/ > > Petr > > # Notes > > How to define Abstract class from ActiveRecord::Base: > > class AbstractKatelloRecord < ActiveRecord::Base > self.abstract_class = true > # ... > end > > This class then doesn't have table, is considered abstract by > ActiveRecord and doesn't break STI. > > > > I assume in picture that Glue::Pulp::User and Pulp::User would be concrete classes, not abstract. -- bk From pchalupa at redhat.com Fri Dec 14 15:59:21 2012 From: pchalupa at redhat.com (Petr Chalupa) Date: Fri, 14 Dec 2012 16:59:21 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CB4BD8.60705@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> <20121214090500.GD1653@lzapx.brq.redhat.com> <50CB2EFA.9030909@redhat.com> <50CB4BD8.60705@redhat.com> Message-ID: <50CB4CD9.7050508@redhat.com> On 14.12.12 16:55, Bryan Kearney wrote: > On 12/14/2012 08:51 AM, Petr Chalupa wrote: >> >> >> On 14.12.12 10:05, Lukas Zapletal wrote: >>> On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: >>>> The original glue layer attempted to enforce the following practices: >>>> >>>> * The controller layer (/controllers and /controllers/api) should >>>> not be aware of the backend engine providing the implementation. >>>> * Any object which the controller would interact with should live in >>>> the model tier (/models), since that is standard rails >>>> * A glue layer (/model/glue) would enhance the model tier and handle >>>> reading/writing to the backend. >>>> * All transport to the backend systems is external (in /lib/resources) >>> >>> I know this thread is about Foreman vs others, but since you replied to >>> my mail I need to write this over and over again. With the current >>> orchestration: >>> >>> We tightly couple model and orchestration while orchestration should >>> be independent. Katello model is just one of the parties we want to >>> integrate with. >>> >>> We can only orchestrate things that have resources in our database and >>> for all the others we need to "hack" alternate models to get this >>> working. >>> >>> We tend to do long running orchestration by hacking model classes (like >>> org.task_id) which is weird - we are abusing models to do orchestration >>> work. Those things are mixed. >>> >>> Our codebase is difficult to understand - we are building chains of >>> callbacks that are deeply hidden in our model objects. Integration >>> proces is a simple function with several statements while we spread it >>> across multiple classes. >>> >>> Errors in our model lead to errors in orchestration. While data >>> inconsistency in our katello db are really bad, they are much easier to >>> fix comparing with inconsistencies across multiple backend engines. >>> >>> Our orchestration code is pretty untestable as a standalone component >>> and the whole codebase is error prone. >>> >>> Katello models are error prone because we have so much code logic in >>> there, so many hooks, callbacks and validations needed for >>> orchestration. They don't need to be there. >>> >>> Our unit tests are either testing models with orchestration turned off >>> or models and orchestration with turned it on. No separation here, tests >>> are mixed into each other. >>> >>> This approach came from foreman where it works pretty well, but >>> Foreman's orchestration is more different from Katello than we thought >>> it will be. >>> >>> I said it like twenty times and I can say it again: Let's move our >>> orchestration out of our models. Let's refactor this while we still have >>> some chance to do this. Now is the time, once we agree this is the >>> "best" approach and maybe rewrite foreman part into this pattern, there >>> is no easy way out. >>> >> >> I agree with Lukas. Our orchestration is fragile and doesn't feel good. >> >> # What I think we can do right now >> >> We could at least start cleaning up the current orchestration code >> without changing its design. I think we would benefit a lot if we just >> removed the model-tangle by replacing modules with classes. >> >> see attachment >> >> ## Image legend >> >> - blue represents gems >> - all black nodes are classes, no modules >> - nodes with full line are present classes >> - dashed nodes are classes which aren't implemented >> - black full edge represents oriented knowledge >> - dashed edges represents inheritance >> >> You can modify and regenerate the pdf with graphwiz: >> `dot graph_orchest.dot -Tpdf graph_orchest.dot.pdf` >> >> ## Class legend of new classes >> >> AbstractKatelloRecord: Abstract parent of all katello ActiveRecord models >> >> AbstractOrchestratedRecord: Abstract parent of all katello ActiveRecord >> models with any orchestration. >> >> Glue::Abstract, Glue::Foreman::Abstract, Glue::Pulp::Abstract: abstract >> parent orchestration classes for all, foreman and pulp >> >> Relation between Glue and ActiveRecords objects: will be very similar to >> a observer pattern. An AbstractOrchestratedRecord descendant (e.g. User >> instance) will send events (generated by ActiveRecord callbacks) to any >> number of registered Glue::Abstract descendants (e.g. >> Glue::Foreman::User instance). >> >> ## Benefits are: >> - separated logic >> - not so fat ActiveRecord models >> - Glue object has its own namespace, no collisions inside model >> - Orchestration switching off/on is simple as un/registering glue >> objects from AbstractOrchestratedRecord descendants >> >> # And for future >> >> I think we should start exploring other solutions. I like Lukas's idea >> to build orchestration around processes (ruote [1]) very much. >> >> [1] http://ruote.rubyforge.org/ >> >> Petr >> >> # Notes >> >> How to define Abstract class from ActiveRecord::Base: >> >> class AbstractKatelloRecord < ActiveRecord::Base >> self.abstract_class = true >> # ... >> end >> >> This class then doesn't have table, is considered abstract by >> ActiveRecord and doesn't break STI. >> >> >> >> > > I assume in picture that Glue::Pulp::User and Pulp::User would be > concrete classes, not abstract. > Yes they would be concrete classes. The dashing around a class means that they do not exist. Classes with full edge are in master already. > -- bk > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From lzap at redhat.com Fri Dec 14 17:05:59 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 14 Dec 2012 18:05:59 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <257727672.15843341.1355481815496.JavaMail.root@redhat.com> References: <20121214092352.GF1653@lzapx.brq.redhat.com> <257727672.15843341.1355481815496.JavaMail.root@redhat.com> Message-ID: <20121214170559.GK1653@lzapx.brq.redhat.com> On Fri, Dec 14, 2012 at 05:43:35AM -0500, Ivan Necas wrote: > > The main advantage of this approach is you (as a caller) are not > > responsible error recovery anymore. Then you don't need to focus so > > much > > on testing edge (error) cases. > > And who is responsible for that then? Caller can be sure, that the data change will eventually be processed. The process can get stuck for some time, but after manual intervention, it is processed. The advantage is you don't need to do error recovery, which is nearly impossible in systems that does not implement distributed transactions. Compensations do not need to be done, you just fire it and forget. This approach is less error prone, and it can be used for all the data changes. But for most of it - yes. -- Later, Lukas "lzap" Zapletal #katello #systemengine From dcleal at redhat.com Fri Dec 14 17:22:30 2012 From: dcleal at redhat.com (Dominic Cleal) Date: Fri, 14 Dec 2012 17:22:30 +0000 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50CA0388.9000907@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> Message-ID: <50CB6056.8020407@redhat.com> On 13/12/12 16:34, Bryan Kearney wrote: > On 12/13/2012 08:52 AM, Ohad Levy wrote: >> >> >> ----- Original Message ----- >> | On Thu, Dec 13, 2012 at 11:11:04AM +0100, Miroslav Suchy wrote: >> | > On 12.12.2012 19:26, Bryan Kearney wrote: >> | > >[1] >> | > >https://fedorahosted.org/katello/wiki/ForemanRegistrationDesign >> | > >> | > >> | > > SysAdmin? racks server hardware and notes MAC Address. >> | > >> | > When I had to provision HW in past, I never knew the MAC address >> | > before booting. Probably Dell, IBM or Fujitsu is writing it >> | > somewhere on box. But the commodity hardware did not have written >> | > it >> | > on box. And I do not recall it on Dell either, but I'm not 100% >> | > sure. >> | > >> Right! this is exactly where the auto discovery work we doing now, where an unknown machine boots into >> a mini OS, that reports the inventory value back to foreman, so later on (or automatically) you can >> approve that host and get it rebooting and auto building. >> >> of course that VM has no issue like that (as the mac is auto assigned while creating the vm). >> >> >> | > What I like on Satellite was, that you can create activation key >> | > and >> | > then assign it to kickstart profile. And you can then boot up >> | > machine with that kickstart profile: >> | > >> | > Ie. tftp record would be: >> | > LABEL rhel60.x86_64.web-server.ks >> | > KERNEL >> | > yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/vmlinuz >> | > APPEND >> | > initrd=yourhostname.redhat.com::linux-install/rhel-x86_64-server-6/initrd.img >> | > ks=http://yourhostname.redhat.com/ks/cfg/org/1/label/rhel-x86_64-webserver-6 >> | > >> | > >> | > And when provisioning machine I would just choose boot from PXE and >> | > in menu choose label rhel60.x86_64.server.ks and it will be >> | > provisioned using kickstart profile rhel-x86_64-webserver-6 and use >> | > associated activation key (which mean i.e. install rhel6 in >> | > noninteractive mode, install httpd and deploy previously created >> | > configuration keys) >> | > >> | > Can we add this use case to this Design as well? >> | >> | So basically the way it works is for a particular timeframe, you are >> | making your TFTP server to serve one same configuration (kickstart) >> | for >> | all machines, right? >> | >> | Well, this can only work for small companies. It's more Foreman >> | related >> | question. I think you can do it with Foreman right now (manually >> | editing >> | the template). >> >> You can do it now, Foreman supports Hostgroup based deployment (aka profiles) >> >> it does not manage the machine afterwards (with regard to puppet certificates etc, as there is no hostname etc). >> > > Added this to the doc, is it correct? > > ==== Component Outline Based Provisioning ==== > > Use case scenario: > > 1. Sysadmin creates a Component Outline (Host Group) and selects it to > be public. > 1. Sysadmin racks a machine, and does a NetBoot. > 1. Sysadmin selects the profile, which provides a kickstart to build > the machine > 1. After boot, sysadmin registers the machine with subscription > manager (see above) Sysadmin will also need to name the machine (hostnames are unique in Foreman). > With the Open Question: > > For Component Outline Provisioning, where does environment fit in? Wouldn't hostgroups, organisations and locations (or their Katello equivalents) have the same problem? Where would new hosts live by default? We're going to have a few bits of bootstrap data required for a client-initiated registration, e.g. - puppetmaster host - environment - organisation - location Unless these can all be automatically derived somehow, or the user prompted for them all at registration time, it may need a different model. How about treating them like a discovered machine (the work Greg's doing in Foreman), then having a way to "convert" them to a managed host and request the extra data from the user? The remaining host data (OS, domain, subnet etc) that would usually be required for a full managed registration in Foreman would then be picked up on the first Puppet run. -- Dominic Cleal Red Hat Engineering From jlabocki at redhat.com Fri Dec 14 19:41:29 2012 From: jlabocki at redhat.com (James Labocki) Date: Fri, 14 Dec 2012 14:41:29 -0500 (EST) Subject: [katello-devel] Subscription Manifest upload for Provider 'Red Hat' fails with traceback Message-ID: <1407327088.18759237.1355514089889.JavaMail.root@redhat.com> I opened the following bug - https://bugzilla.redhat.com/show_bug.cgi?id=886987 . If someone needs access to the system where it is taking place please let me know. Description of problem: I receive the following error (see bug for screenshot) when uploading a subscription manifest to the Red Hat provider in System Engine 1.1.12-22. Version-Release number of selected component (if applicable): System Engine 1.1.12-22 How reproducible: 1. Download Manifest from access.redhat.com. 2. Upload Manifest into organization in System Engine. 3. Upon successful upload, go back to distributor at access.redhat.com and add another subscription to the distributor. 4. Download Manifest from access.redhat.com and upload into system engine. 5. Receive error (see bug for screenshot) Actual results: Receive error (see bug for screenshot) Expected results: Subscription manifest should upload successfully at step #4 and new subscriptions should be available within the organization. From ehelms at redhat.com Fri Dec 14 20:24:37 2012 From: ehelms at redhat.com (Eric Helms) Date: Fri, 14 Dec 2012 15:24:37 -0500 Subject: [katello-devel] Katello 1.2 Released Message-ID: <50CB8B05.2030900@redhat.com> = Katello 1.2 Release Notes = We are happy to announce the release 1.2 of Katello, a systems management solution. Katello is here to help you take control of your software and your systems in an easy-to-use and scalable manner. Offering a modern web user interface and API, Katello can pull content from remote repositories into isolated environments, make subscriptions management easier and provide provisioning at scale. The download locations are * http://fedorapeople.org/groups/katello/releases/yum/1.2/RHEL * http://fedorapeople.org/groups/katello/releases/yum/1.2/Fedora with client repositories under: * http://fedorapeople.org/groups/katello/releases/yum/1.2-client/RHEL * http://fedorapeople.org/groups/katello/releases/yum/1.2-client/Fedora For fresh installations, please use steps from * https://fedorahosted.org/katello/wiki/Install To upgrade your installation, please follow steps from * https://fedorahosted.org/katello/wiki/Upgrade == Features & Enhancements in Katello 1.2 == * Architectures, Config Templates and Domains support added to API and CLI * Subnets and Domains can now be created and managed within the UI * More robust custom info and defaults * Use of bundler.d for production installations * CLI bash completion file updates * Fedora 17 support Notable bug fixes: * 876896, 876911, 878355, 878750, 874502, 874510 - Fixed panel-name/new-link overlap * 877947 - clear-es - added index clean up after import and after delete manifest * 878693 - [RFE] Selecting multiple systems does not give me any action * 875609 - hypervisor - allow hypervisors to successfully register and list in katello * 874280 - terminology changes for consistency across subman, candlepin, etc * 845620 - [RFE] Improve messaging around results of setting the yStream * 853445 - correctly determine the affected repos after deletion * 864936 - products - labelize name on create entry * 873302 - Environments do not populate when adding a new user without full admin * 871086 - Changes to respond with template validation errors as bad requests instead of internal server errors. * 750660 - System packages list doesn't allow you to search for a package installed on the system * 869380 - confirm-delete - add confirmation message before deleting manifest * 865472 - system groups - fix auto-complete on add of systems to groups * 862997 - On content search page, during repository comparison, clicking the show more button will now properly load more data for packages and errata. * 806383 - [RFE] As the SE administrator I want to see all active and scheduled sync tasks for all organizations in one place * 847002 - Fixes rendering issue in IE9 for nested content search results. * 847002 - Fix for IE9 Changeset Environment Selector API Enhancements and Changes: * 866359 - API: /consumers/{id}/entitlements returns incorrect data and Content-Type header * 866995: Fix the status API so that it is exposed correctly for rhsm. * 871086 - Changes to respond with template validation errors as bad requests instead of internal server errors. * 874280 - terminology changes for consistency across subman, candlepin, etc * Added API documentation for architectures * Added API documentation for config_templates * Added API documentation for domains * Default custom info for systems by org * Ensure that the name and label is unique across all all orgs * Foreman domains added to CLI client * Foreman's Config Templates added to CLI client. * Subnets API added == Contributors == * Adam Price * Adam Saleh * Brad Buckingham * Bryan Kearney * David Davis * Dmitri Dolguikh * Eric D Helms * Gerhard Stoeckel * Ivan Necas * Jakub Hadvig * Jason E. Rist * Jeff Weiss * Jordan OMara * Justin Sherrill * Luk?s( Zapletal * Martin Bac(ovsk? * Mike !McCune * Miroslav Such? * Og B. Maciel * Partha Aji * Pavel Pokorn? * Petr Chalupa * Tomas Strachota * Tom !McKay https://fedorahosted.org/katello/wiki/ContributorList == Some statistics == In Katello 1.2, we've seen * 68 bugs fixed * 714 changesets committed * 725 commits done == Known Issues == === No Fedora 17 SELinux Support === The Fedora 17 1.2 installation does not support SELinux, please set the following: {{{ $ service iptables stop $ setenforce 0 }}} == Reporting errors == To report issues with Katello, please use mailing list [https://lists.fedorahosted.org/mailman/listinfo/katello katello at lists.fedorahosted.org] to reach the user community. We might ask you to file bugzilla at https://bugzilla.redhat.com/enter_bug.cgi?product=Katello with more details or full logs. Thank you for using Katello. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmccune at redhat.com Fri Dec 14 20:43:13 2012 From: mmccune at redhat.com (Mike McCune) Date: Fri, 14 Dec 2012 12:43:13 -0800 Subject: [katello-devel] Subscription Manifest upload for Provider 'Red Hat' fails with traceback In-Reply-To: <1407327088.18759237.1355514089889.JavaMail.root@redhat.com> References: <1407327088.18759237.1355514089889.JavaMail.root@redhat.com> Message-ID: <50CB8F61.50009@redhat.com> On 12/14/2012 11:41 AM, James Labocki wrote: > I opened the following bug - https://bugzilla.redhat.com/show_bug.cgi?id=886987 . If someone needs access to the system where it is taking place please let me know. > > Description of problem: > I receive the following error (see bug for screenshot) when uploading a subscription manifest to the Red Hat provider in System Engine 1.1.12-22. > > Version-Release number of selected component (if applicable): > System Engine 1.1.12-22 > > > How reproducible: > 1. Download Manifest from access.redhat.com. > 2. Upload Manifest into organization in System Engine. > 3. Upon successful upload, go back to distributor at access.redhat.com and add another subscription to the distributor. > 4. Download Manifest from access.redhat.com and upload into system engine. > 5. Receive error (see bug for screenshot) > > Actual results: > Receive error (see bug for screenshot) > > Expected results: > Subscription manifest should upload successfully at step #4 and new subscriptions should be available within the organization. > dupe of this bug which I just uploaded a set of hotfix packages : https://bugzilla.redhat.com/show_bug.cgi?id=882411#c8 errata should be avail in January with the fix Mike From mbacovsk at redhat.com Mon Dec 17 09:52:33 2012 From: mbacovsk at redhat.com (Martin Bacovsky) Date: Mon, 17 Dec 2012 10:52:33 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121213153722.GJ4688@lzapx.brq.redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> <20121213153722.GJ4688@lzapx.brq.redhat.com> Message-ID: <50CEEB61.4060608@redhat.com> On 12/13/2012 04:37 PM, Lukas Zapletal wrote: > On Thu, Dec 13, 2012 at 04:14:33PM +0100, Miroslav Suchy wrote: >> User story: >> As a dev, I want foreign keys, so db watch integrity for us. > +1 > Missing foreign keys in Katello were the first thing that scared me when I started on the project. I didn't yet accustomed to the idea that the rails way is the right way in this case, so +1 from me. Since when I tried to understand and debug our orchestration for the first time I have a feeling that it is a bit unclear and fragile way to handle communication between services. Today I came across article about messaging [1] and to me it seems a way to go. I know it would be a *big* change, but I think it is worth to at least consider it, especially when we have messaging broker (QPID) already set up in katello. Pros I can see: - decoupling of our codebase - easier to maintain and test, separate modules, consistent orchestration for all subsystems - debugging - with the right tools we can have detailed orchestration monitoring for free - consistency - support for transactions, proven messaging patterns would help us to keep data consistent across subsystems. Like DB server is good at keeping data healthy (FKs, constraints, transactions), similarly message broker is good at keeping communication between system parts healthy. (lets learn from our previous mistakes) - scalability - adding new components doesn't pollute our backend that much; adding more instances of Foreman would be possible; Katello distributed on more physical machines would be easy Cons I can see: - probably big changes in the code - probably not enough experience with messaging in the team Ideas? [1] http://www.rubyinside.com/why-rubyists-should-care-about-messaging-a-high-level-intro-5017.html From lzap at redhat.com Mon Dec 17 11:36:28 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 17 Dec 2012 12:36:28 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <50CEEB61.4060608@redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> <20121213153722.GJ4688@lzapx.brq.redhat.com> <50CEEB61.4060608@redhat.com> Message-ID: <20121217113628.GD1698@lzapx.brq.redhat.com> On Mon, Dec 17, 2012 at 10:52:33AM +0100, Martin Bacovsky wrote: > to handle communication between services. Today I came across > article about messaging [1] and to me it seems a way to go. I know > it would be a *big* change, but I think it is worth to at least > consider it, especially when we have messaging broker (QPID) already > set up in katello. Well, messaging is not the only way to orchestrate things, but it looks like a good option since developers trying to achieve orchestration tend to write state machines or queue-based processors. I see the biggest advantage in easy de-coupling. We need to get orchestration out of the model, out of Katello completely. With messaging between Katello and OrchestrationNG, it will be possible to run many lightweight Ruby processes/threads for orchestration without Rails in the memory. Having standalone broker also allows services to communicate directly to each other - for request-reply read-only lookups. > - consistency - support for transactions, proven messaging patterns > would help us to keep data consistent across subsystems. Like DB > server is good at keeping data healthy (FKs, constraints, > transactions), similarly message broker is good at keeping > communication between system parts healthy. (lets learn from our > previous mistakes) While messaging systems (including AMQP-based) supports both local and distributed transactions, I'd rather stick with transaction-compensation support. We would need to have distributed transaction support to be implemented in all three backend engines which is not feasible right now. Transaction compensation can be easier to implement if we strictly divide messaging exchanges. There are many other advantages like performance, durability, better error recovery and other things. The ultimate goal should be to drop REST via HTTP and integrate using REST via AMQP. But for the first phase, Katello still can have simple bridge between AMQP and HTTP for each backend engine. -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Mon Dec 17 13:29:24 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Mon, 17 Dec 2012 14:29:24 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50CB6056.8020407@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> Message-ID: <20121217132924.GE1698@lzapx.brq.redhat.com> On Fri, Dec 14, 2012 at 05:22:30PM +0000, Dominic Cleal wrote: > We're going to have a few bits of bootstrap data required for a > client-initiated registration, e.g. > - puppetmaster host > - environment > - organisation > - location > > Unless these can all be automatically derived somehow, or the user > prompted for them all at registration time, it may need a different > model. How about treating them like a discovered machine (the work > Greg's doing in Foreman), then having a way to "convert" them to a > managed host and request the extra data from the user? Users need to pass environment into subscription-manager to register as well as organization. Location can be added here if it is not possible to deduct it from organization/environment/user triple. Puppetmaster host can be auto-deployed during installation of katello certificate - users download an RPM from katello which contains katello certificate and configures rhsm.conf file to point to the proper hostname. I think we can do the same for Puppet master. -- Later, Lukas "lzap" Zapletal #katello #systemengine From dmitri at redhat.com Mon Dec 17 13:45:59 2012 From: dmitri at redhat.com (Dmitri Dolguikh) Date: Mon, 17 Dec 2012 13:45:59 +0000 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <20121217113628.GD1698@lzapx.brq.redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> <20121213153722.GJ4688@lzapx.brq.redhat.com> <50CEEB61.4060608@redhat.com> <20121217113628.GD1698@lzapx.brq.redhat.com> Message-ID: <50CF2217.1020607@redhat.com> On 17/12/12 11:36 AM, Lukas Zapletal wrote: > On Mon, Dec 17, 2012 at 10:52:33AM +0100, Martin Bacovsky wrote: >> to handle communication between services. Today I came across >> article about messaging [1] and to me it seems a way to go. I know >> it would be a *big* change, but I think it is worth to at least >> consider it, especially when we have messaging broker (QPID) already >> set up in katello. > Well, messaging is not the only way to orchestrate things, but it looks > like a good option since developers trying to achieve orchestration tend > to write state machines or queue-based processors. > > I see the biggest advantage in easy de-coupling. We need to get > orchestration out of the model, out of Katello completely. With > messaging between Katello and OrchestrationNG, it will be possible to > run many lightweight Ruby processes/threads for orchestration without > Rails in the memory. > > Having standalone broker also allows services to communicate directly to > each other - for request-reply read-only lookups. > >> - consistency - support for transactions, proven messaging patterns >> would help us to keep data consistent across subsystems. Like DB >> server is good at keeping data healthy (FKs, constraints, >> transactions), similarly message broker is good at keeping >> communication between system parts healthy. (lets learn from our >> previous mistakes) > While messaging systems (including AMQP-based) supports both local and > distributed transactions, I'd rather stick with transaction-compensation > support. We would need to have distributed transaction support to be > implemented in all three backend engines which is not feasible right > now. Transaction compensation can be easier to implement if we strictly > divide messaging exchanges. > > There are many other advantages like performance, durability, better > error recovery and other things. The ultimate goal should be to drop > REST via HTTP and integrate using REST via AMQP. But for the first phase, > Katello still can have simple bridge between AMQP and HTTP for each > backend engine. > I agree with the general sentiment expressed so far - I think we need to revise our approach to orchestration. Let's avoid architectural discussions in this thread, but rather concentrate on what we need from the new engine, stuff like: Crash-only semantics (this is pretty big - as things stand right now we won't recover from any crash should it happen in the middle of an orchestration) Simpler definition of orchestrations Ability to orchestrate any operation, whether it includes AR model or not Ability to recover from certain types of failures (500, 503 type of errors) Easier testability of orchestrations As a side note, I think we need to dispose of the idea of synchronous orchestrations altogether (it makes error recovery possible without breaking the semantics of the call among other things), and aim for eventual consistency between Katello (primary) and its subsystems. -d From ehelms at redhat.com Mon Dec 17 13:48:49 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 08:48:49 -0500 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CB4CD9.7050508@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> <20121214090500.GD1653@lzapx.brq.redhat.com> <50CB2EFA.9030909@redhat.com> <50CB4BD8.60705@redhat.com> <50CB4CD9.7050508@redhat.com> Message-ID: <50CF22C1.1080109@redhat.com> As Lukas has pointed out, this discussion and the next generation orchestration discussion go hand in hand. Both affect the overall architecture of Katello and the move towards a consistent implementation up and down the stack from controllers to backend services. In my opinion, we would be better served by deciding upon a single architectural design and working towards implementation rather than trying to patch up what we have now (since what we have now currently works, but lacks robustness and consistency). The key things I have picked up from the discussion that are in need of resolution and potentially design (and that I have attempted to summarize the different schools of thought on): * How to do OrchestrationNG - Best practice and method for performing orchestration - How this new methodology for doing orchestration fits into the architecture - How Katello entities and glue layer interact with this orchestration * Lazy accessors vs. Remote resources - Lazy accessors are the original method of defining a set of attributes in the glue layer that access the object only when an attribute is requested - Remote resources encompass using a thin model to represent what the resource looks like in the backend service * Backend namespacing and proxied entities vs. top level Katello entities for every model - Keeping backend resources and functionality relegated to the glue layer and lazy accessors/remote resource, and treating all entities as Katello entities that may use a backend service to do the heavy lifting - Exposing proxied entities (those without an ActiveRecord model) to controllers and namespacing at each level of the application * Glue layer interactions with base Katello model: modules vs. classes - Treating glue layer as modules that enhance the base Katello model - Creating a glue layer class for each entity that functions as an observer of the base Katello model - Eric On 12/14/2012 10:59 AM, Petr Chalupa wrote: > > > On 14.12.12 16:55, Bryan Kearney wrote: >> On 12/14/2012 08:51 AM, Petr Chalupa wrote: >>> >>> >>> On 14.12.12 10:05, Lukas Zapletal wrote: >>>> On Thu, Dec 13, 2012 at 03:03:00PM -0500, Bryan Kearney wrote: >>>>> The original glue layer attempted to enforce the following practices: >>>>> >>>>> * The controller layer (/controllers and /controllers/api) should >>>>> not be aware of the backend engine providing the implementation. >>>>> * Any object which the controller would interact with should live in >>>>> the model tier (/models), since that is standard rails >>>>> * A glue layer (/model/glue) would enhance the model tier and handle >>>>> reading/writing to the backend. >>>>> * All transport to the backend systems is external (in >>>>> /lib/resources) >>>> >>>> I know this thread is about Foreman vs others, but since you >>>> replied to >>>> my mail I need to write this over and over again. With the current >>>> orchestration: >>>> >>>> We tightly couple model and orchestration while orchestration should >>>> be independent. Katello model is just one of the parties we want to >>>> integrate with. >>>> >>>> We can only orchestrate things that have resources in our database and >>>> for all the others we need to "hack" alternate models to get this >>>> working. >>>> >>>> We tend to do long running orchestration by hacking model classes >>>> (like >>>> org.task_id) which is weird - we are abusing models to do >>>> orchestration >>>> work. Those things are mixed. >>>> >>>> Our codebase is difficult to understand - we are building chains of >>>> callbacks that are deeply hidden in our model objects. Integration >>>> proces is a simple function with several statements while we spread it >>>> across multiple classes. >>>> >>>> Errors in our model lead to errors in orchestration. While data >>>> inconsistency in our katello db are really bad, they are much >>>> easier to >>>> fix comparing with inconsistencies across multiple backend engines. >>>> >>>> Our orchestration code is pretty untestable as a standalone component >>>> and the whole codebase is error prone. >>>> >>>> Katello models are error prone because we have so much code logic in >>>> there, so many hooks, callbacks and validations needed for >>>> orchestration. They don't need to be there. >>>> >>>> Our unit tests are either testing models with orchestration turned off >>>> or models and orchestration with turned it on. No separation here, >>>> tests >>>> are mixed into each other. >>>> >>>> This approach came from foreman where it works pretty well, but >>>> Foreman's orchestration is more different from Katello than we thought >>>> it will be. >>>> >>>> I said it like twenty times and I can say it again: Let's move our >>>> orchestration out of our models. Let's refactor this while we still >>>> have >>>> some chance to do this. Now is the time, once we agree this is the >>>> "best" approach and maybe rewrite foreman part into this pattern, >>>> there >>>> is no easy way out. >>>> >>> >>> I agree with Lukas. Our orchestration is fragile and doesn't feel good. >>> >>> # What I think we can do right now >>> >>> We could at least start cleaning up the current orchestration code >>> without changing its design. I think we would benefit a lot if we just >>> removed the model-tangle by replacing modules with classes. >>> >>> see attachment >>> >>> ## Image legend >>> >>> - blue represents gems >>> - all black nodes are classes, no modules >>> - nodes with full line are present classes >>> - dashed nodes are classes which aren't implemented >>> - black full edge represents oriented knowledge >>> - dashed edges represents inheritance >>> >>> You can modify and regenerate the pdf with graphwiz: >>> `dot graph_orchest.dot -Tpdf graph_orchest.dot.pdf` >>> >>> ## Class legend of new classes >>> >>> AbstractKatelloRecord: Abstract parent of all katello ActiveRecord >>> models >>> >>> AbstractOrchestratedRecord: Abstract parent of all katello ActiveRecord >>> models with any orchestration. >>> >>> Glue::Abstract, Glue::Foreman::Abstract, Glue::Pulp::Abstract: >>> abstract >>> parent orchestration classes for all, foreman and pulp >>> >>> Relation between Glue and ActiveRecords objects: will be very >>> similar to >>> a observer pattern. An AbstractOrchestratedRecord descendant (e.g. User >>> instance) will send events (generated by ActiveRecord callbacks) to any >>> number of registered Glue::Abstract descendants (e.g. >>> Glue::Foreman::User instance). >>> >>> ## Benefits are: >>> - separated logic >>> - not so fat ActiveRecord models >>> - Glue object has its own namespace, no collisions inside model >>> - Orchestration switching off/on is simple as un/registering glue >>> objects from AbstractOrchestratedRecord descendants >>> >>> # And for future >>> >>> I think we should start exploring other solutions. I like Lukas's idea >>> to build orchestration around processes (ruote [1]) very much. >>> >>> [1] http://ruote.rubyforge.org/ >>> >>> Petr >>> >>> # Notes >>> >>> How to define Abstract class from ActiveRecord::Base: >>> >>> class AbstractKatelloRecord < ActiveRecord::Base >>> self.abstract_class = true >>> # ... >>> end >>> >>> This class then doesn't have table, is considered abstract by >>> ActiveRecord and doesn't break STI. >>> >>> >>> >>> >> >> I assume in picture that Glue::Pulp::User and Pulp::User would be >> concrete classes, not abstract. >> > > Yes they would be concrete classes. The dashing around a class means > that they do not exist. Classes with full edge are in master already. > >> -- bk >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From ehelms at redhat.com Mon Dec 17 14:37:55 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 09:37:55 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50C8C153.7050907@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> Message-ID: <50CF2E43.7040407@redhat.com> This is a reminder that tomorrow is Katello Bug Day 2012 - Fix some bugs before the Mayans fix you. GOAL: Resolve as many bugs as possible. Suggestions: An environment set up and configured for development with master. Plan of Attack: - Start with bugs attached to your name - If you run out of bugs, start pulling from team mates local to you - Find low hanging fruit first * Easily verifiable bugs * Bugs that need info * Bugs that are irrelevant * Bugs with short fixes * Identify potential RFE bugs and re-name them * Identify RFE bugs worthy of list discussion Lukas is coordinating Team Euro Eric is coordinating Team Dollar - Eric From ehelms at redhat.com Mon Dec 17 14:51:42 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 09:51:42 -0500 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans Message-ID: <50CF317E.3020508@redhat.com> Katello 1.3 - PulpV2 Release Nanny: Justin Sherrill Completion Goal: PulpV2 migration merged to master Features: - Pulp V2 Integration - Pulp V2 upgrades - Content Views - Navigation Updates - Custom Info UI - Tupane 2.0 - Significant bug reduction count (564 current) - Initial Provisioning - Rails 3.2 Integration - Alchemy Forms Katello 1.4 - ?? Release Nanny: ?? Completion Goal: ?? Features: - Mod Passenger integration - Errata Notifications From daviddavis at redhat.com Mon Dec 17 15:17:46 2012 From: daviddavis at redhat.com (David Davis) Date: Mon, 17 Dec 2012 10:17:46 -0500 (EST) Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF317E.3020508@redhat.com> Message-ID: <745248395.53760976.1355757466292.JavaMail.root@redhat.com> For 1.4, what if we target content views and then I can be the nanny? Thanks. David ----- Original Message ----- > From: "Eric Helms" > To: katello-devel at redhat.com > Sent: Monday, December 17, 2012 9:51:42 AM > Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans > > Katello 1.3 - PulpV2 > > Release Nanny: Justin Sherrill > > Completion Goal: PulpV2 migration merged to master > > Features: > - Pulp V2 Integration > - Pulp V2 upgrades > - Content Views > - Navigation Updates > - Custom Info UI > - Tupane 2.0 > - Significant bug reduction count (564 current) > - Initial Provisioning > - Rails 3.2 Integration > - Alchemy Forms > > > Katello 1.4 - ?? > > Release Nanny: ?? > > Completion Goal: ?? > > Features: > - Mod Passenger integration > - Errata Notifications > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From ehelms at redhat.com Mon Dec 17 15:24:32 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 10:24:32 -0500 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <745248395.53760976.1355757466292.JavaMail.root@redhat.com> References: <745248395.53760976.1355757466292.JavaMail.root@redhat.com> Message-ID: <50CF3930.1020708@redhat.com> On 12/17/2012 10:17 AM, David Davis wrote: > For 1.4, what if we target content views and then I can be the nanny? Do you see a decent enough time gap between 1.3 being released (~end of Jan to mid Feb) and Content Views being ready? I wouldn't think we'd want to release 1.3 and then two weeks later release a 1.4. -Eric > Thanks. > > David > > ----- Original Message ----- >> From: "Eric Helms" >> To: katello-devel at redhat.com >> Sent: Monday, December 17, 2012 9:51:42 AM >> Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans >> >> Katello 1.3 - PulpV2 >> >> Release Nanny: Justin Sherrill >> >> Completion Goal: PulpV2 migration merged to master >> >> Features: >> - Pulp V2 Integration >> - Pulp V2 upgrades >> - Content Views >> - Navigation Updates >> - Custom Info UI >> - Tupane 2.0 >> - Significant bug reduction count (564 current) >> - Initial Provisioning >> - Rails 3.2 Integration >> - Alchemy Forms >> >> >> Katello 1.4 - ?? >> >> Release Nanny: ?? >> >> Completion Goal: ?? >> >> Features: >> - Mod Passenger integration >> - Errata Notifications >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> From daviddavis at redhat.com Mon Dec 17 15:26:45 2012 From: daviddavis at redhat.com (David Davis) Date: Mon, 17 Dec 2012 10:26:45 -0500 (EST) Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF3930.1020708@redhat.com> Message-ID: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> Oh, I didn't see that content views were already included in 1.3 so that's fine. I'll still volunteer to be the 1.4 nanny though unless someone else wants to. David ----- Original Message ----- > From: "Eric Helms" > To: "David Davis" > Cc: katello-devel at redhat.com > Sent: Monday, December 17, 2012 10:24:32 AM > Subject: Re: [katello-devel] Katello 1.3 and 1.4 Release Plans > > On 12/17/2012 10:17 AM, David Davis wrote: > > For 1.4, what if we target content views and then I can be the > > nanny? > Do you see a decent enough time gap between 1.3 being released (~end > of > Jan to mid Feb) and Content Views being ready? I wouldn't think we'd > want to release 1.3 and then two weeks later release a 1.4. > > -Eric > > > Thanks. > > > > David > > > > ----- Original Message ----- > >> From: "Eric Helms" > >> To: katello-devel at redhat.com > >> Sent: Monday, December 17, 2012 9:51:42 AM > >> Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans > >> > >> Katello 1.3 - PulpV2 > >> > >> Release Nanny: Justin Sherrill > >> > >> Completion Goal: PulpV2 migration merged to master > >> > >> Features: > >> - Pulp V2 Integration > >> - Pulp V2 upgrades > >> - Content Views > >> - Navigation Updates > >> - Custom Info UI > >> - Tupane 2.0 > >> - Significant bug reduction count (564 current) > >> - Initial Provisioning > >> - Rails 3.2 Integration > >> - Alchemy Forms > >> > >> > >> Katello 1.4 - ?? > >> > >> Release Nanny: ?? > >> > >> Completion Goal: ?? > >> > >> Features: > >> - Mod Passenger integration > >> - Errata Notifications > >> > >> _______________________________________________ > >> katello-devel mailing list > >> katello-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/katello-devel > >> > > From bkearney at redhat.com Sun Dec 16 22:27:38 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Mon, 17 Dec 2012 10:27:38 +1200 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> References: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> Message-ID: <50CE4ADA.5010309@redhat.com> On 12/18/2012 03:26 AM, David Davis wrote: > Oh, I didn't see that content views were already included in 1.3 so that's fine. I'll still volunteer to be the 1.4 nanny though unless someone else wants to. > > David > Dow e want to go back across the pond for 1.3? -- bk From ehelms at redhat.com Mon Dec 17 15:47:24 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 10:47:24 -0500 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CE4ADA.5010309@redhat.com> References: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> <50CE4ADA.5010309@redhat.com> Message-ID: <50CF3E8C.9090301@redhat.com> On 12/16/2012 05:27 PM, Bryan Kearney wrote: > On 12/18/2012 03:26 AM, David Davis wrote: >> Oh, I didn't see that content views were already included in 1.3 so >> that's fine. I'll still volunteer to be the 1.4 nanny though unless >> someone else wants to. >> >> David >> > > Dow e want to go back across the pond for 1.3? Do you mean 1.4? As it stands, the first two releases were done by the guys in Europe, so after my nanny-ing for 1.2, we are still unbalanced. After 1.3, we should go to back and forth across the pond every other release. - Eric > > -- bk > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From bkearney at redhat.com Sun Dec 16 23:14:20 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Mon, 17 Dec 2012 11:14:20 +1200 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF3E8C.9090301@redhat.com> References: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> <50CE4ADA.5010309@redhat.com> <50CF3E8C.9090301@redhat.com> Message-ID: <50CE55CC.20705@redhat.com> On 12/18/2012 03:47 AM, Eric Helms wrote: > On 12/16/2012 05:27 PM, Bryan Kearney wrote: >> On 12/18/2012 03:26 AM, David Davis wrote: >>> Oh, I didn't see that content views were already included in 1.3 so >>> that's fine. I'll still volunteer to be the 1.4 nanny though unless >>> someone else wants to. >>> >>> David >>> >> >> Dow e want to go back across the pond for 1.3? > Do you mean 1.4? As it stands, the first two releases were done by the > guys in Europe, so after my nanny-ing for 1.2, we are still unbalanced. > After 1.3, we should go to back and forth across the pond every other > release. Ok.. 1.4 across the pond. thanks! Just want to make sure we are sharing the fun! -- bk From vondruch at redhat.com Mon Dec 17 17:14:56 2012 From: vondruch at redhat.com (=?ISO-8859-1?Q?V=EDt_Ondruch?=) Date: Mon, 17 Dec 2012 18:14:56 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF317E.3020508@redhat.com> References: <50CF317E.3020508@redhat.com> Message-ID: <50CF5310.7050006@redhat.com> Dne 17.12.2012 15:51, Eric Helms napsal(a): > > - Rails 3.2 Integration Rails 4.0 integration? V?t From ehelms at redhat.com Mon Dec 17 18:19:55 2012 From: ehelms at redhat.com (Eric Helms) Date: Mon, 17 Dec 2012 13:19:55 -0500 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF5310.7050006@redhat.com> References: <50CF317E.3020508@redhat.com> <50CF5310.7050006@redhat.com> Message-ID: <50CF624B.5010803@redhat.com> On 12/17/2012 12:14 PM, V?t Ondruch wrote: > Dne 17.12.2012 15:51, Eric Helms napsal(a): >> >> - Rails 3.2 Integration > > Rails 4.0 integration? When is this slated for release? Any idea on the planned support for it with regards to Fedora and/or RHEL? Our goal was to be on par, Rails stack wise, with some other Rails projects such as Aeolus and OpenShift. Thus targeting 3.2. -Eric > > V?t > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From daviddavis at redhat.com Mon Dec 17 18:34:28 2012 From: daviddavis at redhat.com (David Davis) Date: Mon, 17 Dec 2012 13:34:28 -0500 (EST) Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CF624B.5010803@redhat.com> Message-ID: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> There's no release date yet for 4.0. David ----- Original Message ----- > From: "Eric Helms" > To: katello-devel at redhat.com > Sent: Monday, December 17, 2012 1:19:55 PM > Subject: Re: [katello-devel] Katello 1.3 and 1.4 Release Plans > > On 12/17/2012 12:14 PM, V?t Ondruch wrote: > > Dne 17.12.2012 15:51, Eric Helms napsal(a): > >> > >> - Rails 3.2 Integration > > > > Rails 4.0 integration? > When is this slated for release? Any idea on the planned support for > it > with regards to Fedora and/or RHEL? Our goal was to be on par, Rails > stack wise, with some other Rails projects such as Aeolus and > OpenShift. Thus targeting 3.2. > > -Eric > > > > V?t > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From jlabocki at redhat.com Mon Dec 17 22:19:00 2012 From: jlabocki at redhat.com (James Labocki) Date: Mon, 17 Dec 2012 17:19:00 -0500 (EST) Subject: [katello-devel] Fwd: CFSE question In-Reply-To: <1271494272.19453284.1355782614923.JavaMail.root@redhat.com> Message-ID: <1319214909.19453555.1355782740034.JavaMail.root@redhat.com> I seem to recall discussions last week regarding repositories missing in the CDN. Does anyone know if Ben's problem (see fpaste below) is new or if it is a duplicate and we already have a bug open? -James ----- Forwarded Message ----- > From: "Ben Breard" > To: "James Labocki" , "Robert Proffitt" > Sent: Monday, December 17, 2012 5:16:54 PM > Subject: Re: CFSE question > > http://fpaste.org/1Og8/ > > Maybe something is up w/ the CDN. Strange. > > Ben Breard, RHCA, RHCDS, RHCE, RHCVA > Sr. Inside Solutions Architect, Red Hat, Inc. > Office: 972-892-4512 > Mobile: 972-816-9081 > > ----- Original Message ----- > From: "Ben Breard" > To: "James Labocki" , "Robert Proffitt" > > Sent: Monday, December 17, 2012 4:00:26 PM > Subject: CFSE question > > James, > > I just checked on the sync of the system engine box in PHX. The large > channels sync'ed properly, but none of the smaller ones did. Check > out the screenshot and let me know what I'm doing wrong. I think > Solberg ran into something similar when he setup your wordpress > demo. > > Cheers, > > Ben Breard, RHCA, RHCDS, RHCE, RHCVA > Sr. Inside Solutions Architect, Red Hat, Inc. > Office: 972-892-4512 > Mobile: 972-816-9081 > > From ericdhelms at gmail.com Tue Dec 18 02:20:41 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Mon, 17 Dec 2012 21:20:41 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: <50CF2E43.7040407@redhat.com> References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: Quick note or two: Please remember to switch to ASSIGNED once you take responsibility for a bug. This will also help in allowing others to find bugs to tackle after their bucket is emptied. If you manage to clean out your bugs (or get down to multi-day style bugs) the following buckets can also be tackled: Bug Bucket: http://tinyurl.com/bshll5s https://bugzilla.redhat.com/buglist.cgi?list_id=952479&emailtype1=substring&emailassigned_to1=1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&email1=katello-bugs&product=CloudForms%20System%20Engine&product=Katello RFEs: - For determining if they are still valid - For deciding if any should be worked for next release - For finding any that could use a design spike http://tinyurl.com/csnhe23 * * https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=allwords&list_id=952465&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=anywordssubstr&product=CloudForms%20System%20Engine&product=Katello On Mon, Dec 17, 2012 at 9:37 AM, Eric Helms wrote: > This is a reminder that tomorrow is Katello Bug Day 2012 - Fix some bugs > before the Mayans fix you. > > GOAL: Resolve as many bugs as possible. > > Suggestions: An environment set up and configured for development with > master. > > Plan of Attack: > - Start with bugs attached to your name > - If you run out of bugs, start pulling from team mates local to you > - Find low hanging fruit first > * Easily verifiable bugs > * Bugs that need info > * Bugs that are irrelevant > * Bugs with short fixes > * Identify potential RFE bugs and re-name them > * Identify RFE bugs worthy of list discussion > > Lukas is coordinating Team Euro > Eric is coordinating Team Dollar > > - Eric > > > ______________________________**_________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/**mailman/listinfo/katello-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericdhelms at gmail.com Tue Dec 18 03:32:36 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Mon, 17 Dec 2012 22:32:36 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: One last bit, our starting benchmark: 586 New or Assigned bugs https://bugzilla.redhat.com/buglist.cgi?list_id=952603&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager 478 Non-RFE bugs https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=nowords&list_id=952609&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager 108 RFEs https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=anywords&list_id=952615&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=allwordssubstr&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager Couple of Reminders: * Move fixed bugs to Post once an associated PR is merged to master * Please adjust approriate flags: - Devel ACK if not set - Request QA acks where needed On Mon, Dec 17, 2012 at 9:20 PM, Eric D Helms wrote: > Quick note or two: > > Please remember to switch to ASSIGNED once you take responsibility for a > bug. This will also help in allowing others to find bugs to tackle after > their bucket is emptied. > > If you manage to clean out your bugs (or get down to multi-day style bugs) > the following buckets can also be tackled: > > Bug Bucket: > > http://tinyurl.com/bshll5s > > > https://bugzilla.redhat.com/buglist.cgi?list_id=952479&emailtype1=substring&emailassigned_to1=1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&email1=katello-bugs&product=CloudForms%20System%20Engine&product=Katello > > > > RFEs: > - For determining if they are still valid > - For deciding if any should be worked for next release > - For finding any that could use a design spike > > http://tinyurl.com/csnhe23 > * > * > > https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=allwords&list_id=952465&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=anywordssubstr&product=CloudForms%20System%20Engine&product=Katello > > > On Mon, Dec 17, 2012 at 9:37 AM, Eric Helms wrote: > >> This is a reminder that tomorrow is Katello Bug Day 2012 - Fix some bugs >> before the Mayans fix you. >> >> GOAL: Resolve as many bugs as possible. >> >> Suggestions: An environment set up and configured for development with >> master. >> >> Plan of Attack: >> - Start with bugs attached to your name >> - If you run out of bugs, start pulling from team mates local to you >> - Find low hanging fruit first >> * Easily verifiable bugs >> * Bugs that need info >> * Bugs that are irrelevant >> * Bugs with short fixes >> * Identify potential RFE bugs and re-name them >> * Identify RFE bugs worthy of list discussion >> >> Lukas is coordinating Team Euro >> Eric is coordinating Team Dollar >> >> - Eric >> >> >> ______________________________**_________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/**mailman/listinfo/katello-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vondruch at redhat.com Tue Dec 18 08:28:57 2012 From: vondruch at redhat.com (=?UTF-8?B?VsOtdCBPbmRydWNo?=) Date: Tue, 18 Dec 2012 09:28:57 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> References: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> Message-ID: <50D02949.5020305@redhat.com> This is the latest information I've heard: http://weblog.rubyonrails.org/2012/11/21/the-people-behind-rails-4/#comment-716087815 However, count with me. We are thinking about Rails 4.0 in F19, but I am not sure it is going to happen, since the Alpha freeze is currently planned at beginning of March [1], which is pretty soon and there is lot to do yet. So let's say there is 50% chance to get Rails 4 into Fedora 19. But if we miss F19, then we want to have them in F20 for sure and Alpha freeze of F20 happens somewhere around beginning of September. At that time, you should be done with Rails 4.0 support to be able to ship Katello in Fedora. If I consider that Rails 3.2 was released at January 20, 2012 (a year ago), I am not sure how you want to catch Rails 4.0 train in F20, not even speaking about F19. V?t [1] https://fedorahosted.org/fesco/ticket/966 Dne 17.12.2012 19:34, David Davis napsal(a): > There's no release date yet for 4.0. > > David > > ----- Original Message ----- >> From: "Eric Helms" >> To: katello-devel at redhat.com >> Sent: Monday, December 17, 2012 1:19:55 PM >> Subject: Re: [katello-devel] Katello 1.3 and 1.4 Release Plans >> >> On 12/17/2012 12:14 PM, V?t Ondruch wrote: >>> Dne 17.12.2012 15:51, Eric Helms napsal(a): >>>> - Rails 3.2 Integration >>> Rails 4.0 integration? >> When is this slated for release? Any idea on the planned support for >> it >> with regards to Fedora and/or RHEL? Our goal was to be on par, Rails >> stack wise, with some other Rails projects such as Aeolus and >> OpenShift. Thus targeting 3.2. >> >> -Eric >>> V?t >>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From dcleal at redhat.com Tue Dec 18 11:25:31 2012 From: dcleal at redhat.com (Dominic Cleal) Date: Tue, 18 Dec 2012 11:25:31 +0000 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <20121217132924.GE1698@lzapx.brq.redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> Message-ID: <50D052AB.9050707@redhat.com> On 17/12/12 13:29, Lukas Zapletal wrote: > On Fri, Dec 14, 2012 at 05:22:30PM +0000, Dominic Cleal wrote: >> We're going to have a few bits of bootstrap data required for a >> client-initiated registration, e.g. >> - puppetmaster host >> - environment >> - organisation >> - location >> >> Unless these can all be automatically derived somehow, or the user >> prompted for them all at registration time, it may need a different >> model. How about treating them like a discovered machine (the work >> Greg's doing in Foreman), then having a way to "convert" them to a >> managed host and request the extra data from the user? > > Users need to pass environment into subscription-manager to register as > well as organization. Location can be added here if it is not possible > to deduct it from organization/environment/user triple. Ok, that works. > Puppetmaster host can be auto-deployed during installation of katello > certificate - users download an RPM from katello which contains katello > certificate and configures rhsm.conf file to point to the proper > hostname. I think we can do the same for Puppet master. I suppose the assumption for now is that the puppetmaster will always be the Katello host - there won't any support for splitting up and multiplying the number of puppetmaster hosts yet? At that point, the hostname may need to be more dynamic than as delivered in the RPM. -- Dominic Cleal Red Hat Engineering From lzap at redhat.com Tue Dec 18 12:58:55 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Tue, 18 Dec 2012 13:58:55 +0100 Subject: [katello-devel] Bug Squashing Day - Etherpad Message-ID: <20121218125854.GB1687@lzapx.brq.redhat.com> Heads up, we have set a new etherpad page for tracking todays bug squashing. Located here: https://pad-katello.rhcloud.com/p/bug-squashing-dec2012 Not reading mails today, just bugs :-) LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Tue Dec 18 17:00:41 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Tue, 18 Dec 2012 18:00:41 +0100 Subject: [katello-devel] Error when running unit tests: uninitialized constant Resources::Foreman::ForemanApi Message-ID: <20121218170041.GC1687@lzapx.brq.redhat.com> Guys, I noticed when I run these tests I always get this error while on Travis and Hudson it seems they are okay. What can be possibly wrong? Maybe a configuration is missing in my katello.yml? Thanks LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From ericdhelms at gmail.com Wed Dec 19 00:38:52 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Tue, 18 Dec 2012 19:38:52 -0500 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: Initial post bug-day stats: Pull Requests opened: 56 Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM before: 478 Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM after: 397 # of Bugs moved to ASSIGNED: 42 # of Bugs moved to POST: 21 # of Bugs moved to CLOSED: 53 # of RFEs created: 2 If we assume that the number of bugs moved to ASSIGNED represent un-merged bugs with PRs, then roughly speaking we dealt with ~120 bugs!! Please spend time tomorrow reviewing, commenting and ACKing the bug related pull requests. Thanks for everybody's participation today, hopefully it was fun. We will try and do this once a month to continue pushing down our bug count. - Eric On Mon, Dec 17, 2012 at 10:32 PM, Eric D Helms wrote: > One last bit, our starting benchmark: > > 586 New or Assigned bugs > > https://bugzilla.redhat.com/buglist.cgi?list_id=952603&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager > > > 478 Non-RFE bugs > > https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=nowords&list_id=952609&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager > > 108 RFEs > > https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=anywords&list_id=952615&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=allwordssubstr&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager > > Couple of Reminders: > * Move fixed bugs to Post once an associated PR is merged to master > * Please adjust approriate flags: > - Devel ACK if not set > - Request QA acks where needed > > > On Mon, Dec 17, 2012 at 9:20 PM, Eric D Helms wrote: > >> Quick note or two: >> >> Please remember to switch to ASSIGNED once you take responsibility for a >> bug. This will also help in allowing others to find bugs to tackle after >> their bucket is emptied. >> >> If you manage to clean out your bugs (or get down to multi-day style >> bugs) the following buckets can also be tackled: >> >> Bug Bucket: >> >> http://tinyurl.com/bshll5s >> >> >> https://bugzilla.redhat.com/buglist.cgi?list_id=952479&emailtype1=substring&emailassigned_to1=1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&email1=katello-bugs&product=CloudForms%20System%20Engine&product=Katello >> >> >> >> RFEs: >> - For determining if they are still valid >> - For deciding if any should be worked for next release >> - For finding any that could use a design spike >> >> http://tinyurl.com/csnhe23 >> * >> * >> >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=allwords&list_id=952465&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=anywordssubstr&product=CloudForms%20System%20Engine&product=Katello >> >> >> On Mon, Dec 17, 2012 at 9:37 AM, Eric Helms wrote: >> >>> This is a reminder that tomorrow is Katello Bug Day 2012 - Fix some bugs >>> before the Mayans fix you. >>> >>> GOAL: Resolve as many bugs as possible. >>> >>> Suggestions: An environment set up and configured for development with >>> master. >>> >>> Plan of Attack: >>> - Start with bugs attached to your name >>> - If you run out of bugs, start pulling from team mates local to you >>> - Find low hanging fruit first >>> * Easily verifiable bugs >>> * Bugs that need info >>> * Bugs that are irrelevant >>> * Bugs with short fixes >>> * Identify potential RFE bugs and re-name them >>> * Identify RFE bugs worthy of list discussion >>> >>> Lukas is coordinating Team Euro >>> Eric is coordinating Team Dollar >>> >>> - Eric >>> >>> >>> ______________________________**_________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/**mailman/listinfo/katello-devel >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jrist at redhat.com Wed Dec 19 01:29:29 2012 From: jrist at redhat.com (Jason Rist) Date: Tue, 18 Dec 2012 20:29:29 -0500 (EST) Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: I thought it was fun. Thanks for doing the stats, Eric! Mike - we should include these stats at the sprint review! -J On Dec 18, 2012, at 17:39, Eric D Helms wrote: > Initial post bug-day stats: > > Pull Requests opened: 56 > Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM before: 478 > Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM after: 397 > > # of Bugs moved to ASSIGNED: 42 > # of Bugs moved to POST: 21 > # of Bugs moved to CLOSED: 53 > > # of RFEs created: 2 > > If we assume that the number of bugs moved to ASSIGNED represent un-merged bugs with PRs, then roughly speaking we dealt with ~120 bugs!! > > Please spend time tomorrow reviewing, commenting and ACKing the bug related pull requests. > > > Thanks for everybody's participation today, hopefully it was fun. We will try and do this once a month to continue pushing down our bug count. > > - Eric > > > On Mon, Dec 17, 2012 at 10:32 PM, Eric D Helms wrote: >> One last bit, our starting benchmark: >> >> 586 New or Assigned bugs >> https://bugzilla.redhat.com/buglist.cgi?list_id=952603&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> >> 478 Non-RFE bugs >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=nowords&list_id=952609&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> 108 RFEs >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=anywords&list_id=952615&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=allwordssubstr&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> Couple of Reminders: >> * Move fixed bugs to Post once an associated PR is merged to master >> * Please adjust approriate flags: >> - Devel ACK if not set >> - Request QA acks where needed >> >> >> On Mon, Dec 17, 2012 at 9:20 PM, Eric D Helms wrote: >>> Quick note or two: >>> >>> Please remember to switch to ASSIGNED once you take responsibility for a bug. This will also help in allowing others to find bugs to tackle after their bucket is emptied. >>> >>> If you manage to clean out your bugs (or get down to multi-day style bugs) the following buckets can also be tackled: >>> >>> Bug Bucket: >>> >>> http://tinyurl.com/bshll5s >>> >>> https://bugzilla.redhat.com/buglist.cgi?list_id=952479&emailtype1=substring&emailassigned_to1=1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&email1=katello-bugs&product=CloudForms%20System%20Engine&product=Katello >>> >>> >>> >>> RFEs: >>> - For determining if they are still valid >>> - For deciding if any should be worked for next release >>> - For finding any that could use a design spike >>> >>> http://tinyurl.com/csnhe23 >>> >>> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=allwords&list_id=952465&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=anywordssubstr&product=CloudForms%20System%20Engine&product=Katello >>> >>> >>> On Mon, Dec 17, 2012 at 9:37 AM, Eric Helms wrote: >>>> This is a reminder that tomorrow is Katello Bug Day 2012 - Fix some bugs before the Mayans fix you. >>>> >>>> GOAL: Resolve as many bugs as possible. >>>> >>>> Suggestions: An environment set up and configured for development with master. >>>> >>>> Plan of Attack: >>>> - Start with bugs attached to your name >>>> - If you run out of bugs, start pulling from team mates local to you >>>> - Find low hanging fruit first >>>> * Easily verifiable bugs >>>> * Bugs that need info >>>> * Bugs that are irrelevant >>>> * Bugs with short fixes >>>> * Identify potential RFE bugs and re-name them >>>> * Identify RFE bugs worthy of list discussion >>>> >>>> Lukas is coordinating Team Euro >>>> Eric is coordinating Team Dollar >>>> >>>> - Eric >>>> >>>> >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmccune at redhat.com Wed Dec 19 04:10:58 2012 From: mmccune at redhat.com (Mike McCune) Date: Tue, 18 Dec 2012 20:10:58 -0800 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: <50D13E52.5080509@redhat.com> absolutely, great work all On 12/18/2012 05:29 PM, Jason Rist wrote: > I thought it was fun. Thanks for doing the stats, Eric! Mike - we should > include these stats at the sprint review! > > -J > > On Dec 18, 2012, at 17:39, Eric D Helms > wrote: > >> Initial post bug-day stats: >> >> Pull Requests opened: 56 >> Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM before: 478 >> Bug count (NEW, ASSIGNED) for Katello, CFSE and SAM after: 397 >> >> # of Bugs moved to ASSIGNED: 42 >> # of Bugs moved to POST: 21 >> # of Bugs moved to CLOSED: 53 >> >> # of RFEs created: 2 >> >> If we assume that the number of bugs moved to ASSIGNED represent >> un-merged bugs with PRs, then roughly speaking we dealt with ~120 bugs!! >> >> Please spend time tomorrow reviewing, commenting and ACKing the bug >> related pull requests. >> >> >> Thanks for everybody's participation today, hopefully it was fun. We >> will try and do this once a month to continue pushing down our bug count. >> >> - Eric >> >> >> On Mon, Dec 17, 2012 at 10:32 PM, Eric D Helms > > wrote: >> >> One last bit, our starting benchmark: >> >> 586 New or Assigned bugs >> https://bugzilla.redhat.com/buglist.cgi?list_id=952603&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> >> >> 478 Non-RFE bugs >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=nowords&list_id=952609&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> >> 108 RFEs >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=anywords&list_id=952615&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=allwordssubstr&product=CloudForms%20System%20Engine&product=Katello&product=Subscription%20Asset%20Manager >> >> >> Couple of Reminders: >> * Move fixed bugs to Post once an associated PR is merged to master >> * Please adjust approriate flags: >> - Devel ACK if not set >> - Request QA acks where needed >> >> >> On Mon, Dec 17, 2012 at 9:20 PM, Eric D Helms >> > wrote: >> >> Quick note or two: >> >> Please remember to switch to ASSIGNED once you take >> responsibility for a bug. This will also help in allowing >> others to find bugs to tackle after their bucket is emptied. >> >> If you manage to clean out your bugs (or get down to multi-day >> style bugs) the following buckets can also be tackled: >> >> Bug Bucket: >> >> http://tinyurl.com/bshll5s >> >> https://bugzilla.redhat.com/buglist.cgi?list_id=952479&emailtype1=substring&emailassigned_to1=1&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&email1=katello-bugs&product=CloudForms%20System%20Engine&product=Katello >> >> >> >> >> RFEs: >> - For determining if they are still valid >> - For deciding if any should be worked for next release >> - For finding any that could use a design spike >> >> http://tinyurl.com/csnhe23 >> * >> * >> https://bugzilla.redhat.com/buglist.cgi?keywords=FutureFeature%2C%20&keywords_type=allwords&list_id=952465&short_desc=RFE&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&short_desc_type=anywordssubstr&product=CloudForms%20System%20Engine&product=Katello >> >> >> >> On Mon, Dec 17, 2012 at 9:37 AM, Eric Helms > > wrote: >> >> This is a reminder that tomorrow is Katello Bug Day 2012 - >> Fix some bugs before the Mayans fix you. >> >> GOAL: Resolve as many bugs as possible. >> >> Suggestions: An environment set up and configured for >> development with master. >> >> Plan of Attack: >> - Start with bugs attached to your name >> - If you run out of bugs, start pulling from team mates >> local to you >> - Find low hanging fruit first >> * Easily verifiable bugs >> * Bugs that need info >> * Bugs that are irrelevant >> * Bugs with short fixes >> * Identify potential RFE bugs and re-name them >> * Identify RFE bugs worthy of list discussion >> >> Lukas is coordinating Team Euro >> Eric is coordinating Team Dollar >> >> - Eric >> >> >> _________________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/__mailman/listinfo/katello-devel >> >> >> >> >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650-254-4248 From lzap at redhat.com Wed Dec 19 09:20:12 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 10:20:12 +0100 Subject: [katello-devel] Organization deletion bug, orchestration, testing In-Reply-To: <50CF2217.1020607@redhat.com> References: <20121213144908.GG4688@lzapx.brq.redhat.com> <50C9F0D9.9040203@redhat.com> <20121213153722.GJ4688@lzapx.brq.redhat.com> <50CEEB61.4060608@redhat.com> <20121217113628.GD1698@lzapx.brq.redhat.com> <50CF2217.1020607@redhat.com> Message-ID: <20121219092012.GC2074@lzapx.brq.redhat.com> Sure, if you dont mind putting this on this page: https://fedorahosted.org/katello/wiki/OrchestrationNG I would be happy. LZ On Mon, Dec 17, 2012 at 01:45:59PM +0000, Dmitri Dolguikh wrote: > On 17/12/12 11:36 AM, Lukas Zapletal wrote: > >On Mon, Dec 17, 2012 at 10:52:33AM +0100, Martin Bacovsky wrote: > >>to handle communication between services. Today I came across > >>article about messaging [1] and to me it seems a way to go. I know > >>it would be a *big* change, but I think it is worth to at least > >>consider it, especially when we have messaging broker (QPID) already > >>set up in katello. > >Well, messaging is not the only way to orchestrate things, but it looks > >like a good option since developers trying to achieve orchestration tend > >to write state machines or queue-based processors. > > > >I see the biggest advantage in easy de-coupling. We need to get > >orchestration out of the model, out of Katello completely. With > >messaging between Katello and OrchestrationNG, it will be possible to > >run many lightweight Ruby processes/threads for orchestration without > >Rails in the memory. > > > >Having standalone broker also allows services to communicate directly to > >each other - for request-reply read-only lookups. > > > >>- consistency - support for transactions, proven messaging patterns > >>would help us to keep data consistent across subsystems. Like DB > >>server is good at keeping data healthy (FKs, constraints, > >>transactions), similarly message broker is good at keeping > >>communication between system parts healthy. (lets learn from our > >>previous mistakes) > >While messaging systems (including AMQP-based) supports both local and > >distributed transactions, I'd rather stick with transaction-compensation > >support. We would need to have distributed transaction support to be > >implemented in all three backend engines which is not feasible right > >now. Transaction compensation can be easier to implement if we strictly > >divide messaging exchanges. > > > >There are many other advantages like performance, durability, better > >error recovery and other things. The ultimate goal should be to drop > >REST via HTTP and integrate using REST via AMQP. But for the first phase, > >Katello still can have simple bridge between AMQP and HTTP for each > >backend engine. > > > > I agree with the general sentiment expressed so far - I think we > need to revise our approach to orchestration. > > Let's avoid architectural discussions in this thread, but rather > concentrate on what we need from the new engine, stuff like: > > Crash-only semantics (this is pretty big - as things stand right now > we won't recover from any crash should it happen in the middle of an > orchestration) > Simpler definition of orchestrations > Ability to orchestrate any operation, whether it includes AR model or not > Ability to recover from certain types of failures (500, 503 type of errors) > Easier testability of orchestrations > > As a side note, I think we need to dispose of the idea of > synchronous orchestrations altogether (it makes error recovery > possible without breaking the semantics of the call among other > things), and aim for eventual consistency between Katello (primary) > and its subsystems. > > -d > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 09:27:30 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 10:27:30 +0100 Subject: [katello-devel] Fwd: CFSE question In-Reply-To: <1319214909.19453555.1355782740034.JavaMail.root@redhat.com> References: <1271494272.19453284.1355782614923.JavaMail.root@redhat.com> <1319214909.19453555.1355782740034.JavaMail.root@redhat.com> Message-ID: <20121219092730.GD2074@lzapx.brq.redhat.com> If you are able to reproduce on that box, is it possible to extract the certificate from the manifest and use wget/curl to download the file to confirm CDN is okay? https://fedorahosted.org/katello/wiki/GuideDebugCertificates Listing is disabled on the CDN, just use our pre-prod environment to locate the metadata.xml and then try the curl command bellow with cert/key extracted from the manifest file to fetch it. LZ On Mon, Dec 17, 2012 at 05:19:00PM -0500, James Labocki wrote: > I seem to recall discussions last week regarding repositories missing in the CDN. Does anyone know if Ben's problem (see fpaste below) is new or if it is a duplicate and we already have a bug open? > > -James > > ----- Forwarded Message ----- > > From: "Ben Breard" > > To: "James Labocki" , "Robert Proffitt" > > Sent: Monday, December 17, 2012 5:16:54 PM > > Subject: Re: CFSE question > > > > http://fpaste.org/1Og8/ > > > > Maybe something is up w/ the CDN. Strange. > > > > Ben Breard, RHCA, RHCDS, RHCE, RHCVA > > Sr. Inside Solutions Architect, Red Hat, Inc. > > Office: 972-892-4512 > > Mobile: 972-816-9081 > > > > ----- Original Message ----- > > From: "Ben Breard" > > To: "James Labocki" , "Robert Proffitt" > > > > Sent: Monday, December 17, 2012 4:00:26 PM > > Subject: CFSE question > > > > James, > > > > I just checked on the sync of the system engine box in PHX. The large > > channels sync'ed properly, but none of the smaller ones did. Check > > out the screenshot and let me know what I'm doing wrong. I think > > Solberg ran into something similar when he setup your wordpress > > demo. > > > > Cheers, > > > > Ben Breard, RHCA, RHCDS, RHCE, RHCVA > > Sr. Inside Solutions Architect, Red Hat, Inc. > > Office: 972-892-4512 > > Mobile: 972-816-9081 > > > > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 09:33:56 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 10:33:56 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50CE55CC.20705@redhat.com> References: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> <50CE4ADA.5010309@redhat.com> <50CF3E8C.9090301@redhat.com> <50CE55CC.20705@redhat.com> Message-ID: <20121219093356.GE2074@lzapx.brq.redhat.com> We are actually fighting for the release nanny. I am happy. If there is no volunteer across the pond, I can do it. But I'd rather see few ;-) Ok, if you want to be RN, make one step forward! LZ On Mon, Dec 17, 2012 at 11:14:20AM +1200, Bryan Kearney wrote: > On 12/18/2012 03:47 AM, Eric Helms wrote: > >On 12/16/2012 05:27 PM, Bryan Kearney wrote: > >>On 12/18/2012 03:26 AM, David Davis wrote: > >>>Oh, I didn't see that content views were already included in 1.3 so > >>>that's fine. I'll still volunteer to be the 1.4 nanny though unless > >>>someone else wants to. > >>> > >>>David > >>> > >> > >>Dow e want to go back across the pond for 1.3? > >Do you mean 1.4? As it stands, the first two releases were done by the > >guys in Europe, so after my nanny-ing for 1.2, we are still unbalanced. > >After 1.3, we should go to back and forth across the pond every other > >release. > > Ok.. 1.4 across the pond. thanks! Just want to make sure we are > sharing the fun! > > -- bk > > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 09:34:50 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 10:34:50 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <50D02949.5020305@redhat.com> References: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> <50D02949.5020305@redhat.com> Message-ID: <20121219093450.GF2074@lzapx.brq.redhat.com> Rails devs are crazy people. Btw what is gonna be in RHEL7? LZ On Tue, Dec 18, 2012 at 09:28:57AM +0100, Vit Ondruch wrote: > This is the latest information I've heard: http://weblog.rubyonrails.org/2012/11/21/the-people-behind-rails-4/#comment-716087815 > > However, count with me. We are thinking about Rails 4.0 in F19, but > I am not sure it is going to happen, since the Alpha freeze is > currently planned at beginning of March [1], which is pretty soon > and there is lot to do yet. So let's say there is 50% chance to get > Rails 4 into Fedora 19. > > But if we miss F19, then we want to have them in F20 for sure and > Alpha freeze of F20 happens somewhere around beginning of September. > At that time, you should be done with Rails 4.0 support to be able > to ship Katello in Fedora. > > If I consider that Rails 3.2 was released at January 20, 2012 (a > year ago), I am not sure how you want to catch Rails 4.0 train in > F20, not even speaking about F19. > > > V?t > > > [1] https://fedorahosted.org/fesco/ticket/966 > > > > > > Dne 17.12.2012 19:34, David Davis napsal(a): > >There's no release date yet for 4.0. > > > >David > > > >----- Original Message ----- > >>From: "Eric Helms" > >>To: katello-devel at redhat.com > >>Sent: Monday, December 17, 2012 1:19:55 PM > >>Subject: Re: [katello-devel] Katello 1.3 and 1.4 Release Plans > >> > >>On 12/17/2012 12:14 PM, V?t Ondruch wrote: > >>>Dne 17.12.2012 15:51, Eric Helms napsal(a): > >>>> - Rails 3.2 Integration > >>>Rails 4.0 integration? > >>When is this slated for release? Any idea on the planned support for > >>it > >>with regards to Fedora and/or RHEL? Our goal was to be on par, Rails > >>stack wise, with some other Rails projects such as Aeolus and > >>OpenShift. Thus targeting 3.2. > >> > >>-Eric > >>>V?t > >>> > >>>_______________________________________________ > >>>katello-devel mailing list > >>>katello-devel at redhat.com > >>>https://www.redhat.com/mailman/listinfo/katello-devel > >>_______________________________________________ > >>katello-devel mailing list > >>katello-devel at redhat.com > >>https://www.redhat.com/mailman/listinfo/katello-devel > >> > >_______________________________________________ > >katello-devel mailing list > >katello-devel at redhat.com > >https://www.redhat.com/mailman/listinfo/katello-devel > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 09:38:02 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 10:38:02 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50D052AB.9050707@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> Message-ID: <20121219093802.GG2074@lzapx.brq.redhat.com> On Tue, Dec 18, 2012 at 11:25:31AM +0000, Dominic Cleal wrote: > I suppose the assumption for now is that the puppetmaster will always be > the Katello host - there won't any support for splitting up and > multiplying the number of puppetmaster hosts yet? At that point, the > hostname may need to be more dynamic than as delivered in the RPM. > Well I mean we would just add puppetmaster bits into the katello-cert rpm. This wiki explains what we actually do: https://fedorahosted.org/katello/wiki/GuideSystemRegistrationClient#Option1preferredmethod:UsingthebootstrapRPM and here you can see the RPM post inst script: https://github.com/Katello/katello/blob/master/katello-configure/modules/certs/templates/rhsm-katello-reconfigure.erb The RPM is generated during configuration of Katello, everything in there is dynamic (and can be also re-generated with reapplying katello-configure = puppet). LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 10:08:26 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 11:08:26 +0100 Subject: [katello-devel] Katello Bug Day 2012 - 12/18/2012 In-Reply-To: References: <50C893D3.8090503@redhat.com> <20121212150832.GH1895@lzapx.brq.redhat.com> <50C89F2B.1020405@redhat.com> <50C8A6DC.6020008@redhat.com> <50C8ACC6.3040105@redhat.com> <50C8C153.7050907@redhat.com> <50CF2E43.7040407@redhat.com> Message-ID: <20121219100826.GI2074@lzapx.brq.redhat.com> Just a heads up: We are not using POST anymore. There was a discussion on the list and the wiki shows MODIFIED too: https://engineering.redhat.com/trac/systemengine/wiki/BugzillaProcess#StateFlowForCommunityBugs Please use MODIFIED for community bugs. For SystemEngine we should use POST only when we need it (it's not the timeframe today :-) I have been doing MODIFIED yesterday - note for the next release nanny. LZ -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 10:47:00 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 11:47:00 +0100 Subject: [katello-devel] Katello Backend Service Relationships and Architecture In-Reply-To: <50CF22C1.1080109@redhat.com> References: <50C8996D.7080801@redhat.com> <91740383.17807534.1355368774620.JavaMail.root@redhat.com> <20121213103529.GC4688@lzapx.brq.redhat.com> <50CA3474.90301@redhat.com> <20121214090500.GD1653@lzapx.brq.redhat.com> <50CB2EFA.9030909@redhat.com> <50CB4BD8.60705@redhat.com> <50CB4CD9.7050508@redhat.com> <50CF22C1.1080109@redhat.com> Message-ID: <20121219104700.GJ2074@lzapx.brq.redhat.com> On Mon, Dec 17, 2012 at 08:48:49AM -0500, Eric Helms wrote: > * Lazy accessors vs. Remote resources > - Lazy accessors are the original method of defining a set of > attributes in the glue layer that access the object only when an > attribute is requested > - Remote resources encompass using a thin model to represent > what the resource looks like in the backend service I think this is a bit out of the orchestration code, but as you say it goes hand in hand. IMHO lazy accessors is great idea and it works for read only fields - it's very convenient to have the possibility to fetch these without additional code. And it should go through our new integration layer - whatever it is - as well as data changes. In all architectures I have been working on we used to pass EVERYTHING through the integration layer (the bus), so everything was logged in the message store. Big advantage - when you are solving an issue, you are very often interested what data does a backend engine return. And since you have also read-only request-reply messages stored in the log, you can see it immediately (and identify a bug in an orchestration transformation where a value is changed improperly for example). Please note integration and orchestration are two different things in this architecture. Orchestration box lives in the integration box and the only way to interact with orchestration is via integration (bus). And all the parties involved need to use that integration layer too. And there is audit, log, routing and other services that we can offer if we stick with this rule - not from the day one maybe, but possibility to have it is a big win. It *may* look scary, but we do not want to implement full-blown ESB, we just need to stick with basic attributes of ESB (Enterprise Service Bus) to have all the possibilities, architecture and design patterns which are possible for ESBs. Typical ESB uses reliable messaging for inside communication between components, have routing and orchestration engine inside to do transformations and orchestration, have several other services like audit or message store and adapters for integration with other systems (backend services in our case). They can be web services, REST or just messaging bridges. Katello would be good candidate for direct communication via messages while backend engines could be integrated via REST HTTP adapters for the first phase. ESB projects and products are usually too big for us, even "lightweight" projects like Apache ServiceMix (FUSE now acquired by Red Hat). And that is the reason why I'd prefer to build something light with the same architecture. > * Backend namespacing and proxied entities vs. top level Katello > entities for every model > - Keeping backend resources and functionality relegated to the > glue layer and lazy accessors/remote resource, and treating all > entities as Katello entities that may use a backend service to do > the heavy lifting > - Exposing proxied entities (those without an ActiveRecord > model) to controllers and namespacing at each level of the > application If we are planning to change our integration and orchestration architecture, I think ActiveResource will disappear, because we will need to create a thin layer that would send message (either request or just notification) through the integration layer. Only the integration layer will be able to contact the backend services, either via REST HTTP or via other means. But this is very important point - integration layer should be the glue point, the single point of contact for all parties. And this is something I will need to insist for OrchestrationNG (ONG from now on). If we agree this is the right way to do it, there will be no proxied entities - you just ask for a resource state through integration layer (or send change there). The goal is to move from spaghetti integration to robust and centralized integration. I think your pictures shows this very clearly that is is the time to at least start designing change like this. > * Glue layer interactions with base Katello model: modules vs. classes > - Treating glue layer as modules that enhance the base Katello model > - Creating a glue layer class for each entity that functions as > an observer of the base Katello model Orchestration must be separate from our integration layer. Advantages are described on the ONG wiki page - like testability, loose coupling and other things. We need to totally separate this, it needs a surgery. -- Later, Lukas "lzap" Zapletal #katello #systemengine From vondruch at redhat.com Wed Dec 19 10:50:24 2012 From: vondruch at redhat.com (=?ISO-8859-1?Q?V=EDt_Ondruch?=) Date: Wed, 19 Dec 2012 11:50:24 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <20121219093450.GF2074@lzapx.brq.redhat.com> References: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> <50D02949.5020305@redhat.com> <20121219093450.GF2074@lzapx.brq.redhat.com> Message-ID: <50D19BF0.1020706@redhat.com> ATM, there are no Rails and I don't believe they will ever be part of base RHEL, since RHEL has much longer lifespan then Rails. But they will be probably available as a software collection. What version, that is up to discussion I guess. V?t Dne 19.12.2012 10:34, Lukas Zapletal napsal(a): > Rails devs are crazy people. > > Btw what is gonna be in RHEL7? > > LZ > > On Tue, Dec 18, 2012 at 09:28:57AM +0100, Vit Ondruch wrote: >> This is the latest information I've heard: http://weblog.rubyonrails.org/2012/11/21/the-people-behind-rails-4/#comment-716087815 >> >> However, count with me. We are thinking about Rails 4.0 in F19, but >> I am not sure it is going to happen, since the Alpha freeze is >> currently planned at beginning of March [1], which is pretty soon >> and there is lot to do yet. So let's say there is 50% chance to get >> Rails 4 into Fedora 19. >> >> But if we miss F19, then we want to have them in F20 for sure and >> Alpha freeze of F20 happens somewhere around beginning of September. >> At that time, you should be done with Rails 4.0 support to be able >> to ship Katello in Fedora. >> >> If I consider that Rails 3.2 was released at January 20, 2012 (a >> year ago), I am not sure how you want to catch Rails 4.0 train in >> F20, not even speaking about F19. >> >> >> V?t >> >> >> [1] https://fedorahosted.org/fesco/ticket/966 >> >> >> >> >> >> Dne 17.12.2012 19:34, David Davis napsal(a): >>> There's no release date yet for 4.0. >>> >>> David >>> >>> ----- Original Message ----- >>>> From: "Eric Helms" >>>> To: katello-devel at redhat.com >>>> Sent: Monday, December 17, 2012 1:19:55 PM >>>> Subject: Re: [katello-devel] Katello 1.3 and 1.4 Release Plans >>>> >>>> On 12/17/2012 12:14 PM, V?t Ondruch wrote: >>>>> Dne 17.12.2012 15:51, Eric Helms napsal(a): >>>>>> - Rails 3.2 Integration >>>>> Rails 4.0 integration? >>>> When is this slated for release? Any idea on the planned support for >>>> it >>>> with regards to Fedora and/or RHEL? Our goal was to be on par, Rails >>>> stack wise, with some other Rails projects such as Aeolus and >>>> OpenShift. Thus targeting 3.2. >>>> >>>> -Eric >>>>> V?t >>>>> >>>>> _______________________________________________ >>>>> katello-devel mailing list >>>>> katello-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> _______________________________________________ >>>> katello-devel mailing list >>>> katello-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/katello-devel >>>> >>> _______________________________________________ >>> katello-devel mailing list >>> katello-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/katello-devel >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel From msuchy at redhat.com Wed Dec 19 11:45:50 2012 From: msuchy at redhat.com (Miroslav Suchy) Date: Wed, 19 Dec 2012 12:45:50 +0100 Subject: [katello-devel] Katello's year 2012 in numbers Message-ID: <50D1A8EE.4050804@redhat.com> Let me allow to summarize year of 2012. I gathered some numbers for you: == Code == We committed 5072 changesets to Katello http://lzap.fedorapeople.org/stats/katello/activity.html#commits_by_year Top contributor is Miroslav Such? with 592 commits (728 when including merges). https://github.com/Katello/katello/graphs/contributors?from=2012-01-12&to=2012-12-31&type=c We resolved 1720 Bugs. https://bugzilla.redhat.com/buglist.cgi?bug_status=CLOSED&chfield=bug_status&chfieldfrom=2012-01-01&chfieldto=Now&classification=Red%20Hat&list_id=959476&product=CloudForms%20System%20Engine&query_format=advanced&order=changeddate%2Cbug_id%20DESC&limit=0 https://bugzilla.redhat.com/buglist.cgi?list_id=959488&classification=Community&chfieldto=Now&query_format=advanced&chfield=bug_status&chfieldfrom=2012-01-01&bug_status=CLOSED&product=Katello == Project cost == This is wild estimation, but just guess: Codebase: 99,723 lines (including code and mark up) Effort (est.): 25 Person Years Avg. Salary: $55000 year Estimated cost of project:$ 1,365,353 http://www.ohloh.net/p/katello == Releases == We released 3 releases during year 2012: 1.0, 1.1, 1.2 https://freecode.com/projects/katello/releases == IRC == We have 281 users on #katello. And most active is jrist with 9898 lines written. cliff brings happiness to the world as 24.4% of his lines contained smiling faces. :) I'm looking forward to see you in new year :) Mirek From mmorsi at redhat.com Wed Dec 19 12:19:35 2012 From: mmorsi at redhat.com (Mo Morsi) Date: Wed, 19 Dec 2012 07:19:35 -0500 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <20121219093450.GF2074@lzapx.brq.redhat.com> References: <627381028.53953668.1355769268558.JavaMail.root@redhat.com> <50D02949.5020305@redhat.com> <20121219093450.GF2074@lzapx.brq.redhat.com> Message-ID: <50D1B0D7.5070601@redhat.com> On 12/19/2012 04:34 AM, Lukas Zapletal wrote: > Rails devs are crazy people. > > +1 :-) Still think it'll be a while before the Fedora / Ruby community has enough pull w/ upstream ruby communities to push for some stability w/ their APIs. But our numbers grow by the day and I'm hoping soon we'll be a force to be reckoned with. -Mo From lzap at redhat.com Wed Dec 19 12:31:16 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 13:31:16 +0100 Subject: [katello-devel] Katello 1.3 and 1.4 Release Plans In-Reply-To: <20121219093356.GE2074@lzapx.brq.redhat.com> References: <1537662547.53780588.1355758005584.JavaMail.root@redhat.com> <50CE4ADA.5010309@redhat.com> <50CF3E8C.9090301@redhat.com> <50CE55CC.20705@redhat.com> <20121219093356.GE2074@lzapx.brq.redhat.com> Message-ID: <20121219123116.GM2074@lzapx.brq.redhat.com> Scratch this, I thought we are already expecting 1.4. Too early. Yeah, 1.4 is our turn. LZ On Wed, Dec 19, 2012 at 10:33:56AM +0100, Lukas Zapletal wrote: > We are actually fighting for the release nanny. > > I am happy. > > If there is no volunteer across the pond, I can do it. But I'd rather > see few ;-) > > Ok, if you want to be RN, make one step forward! > > LZ > > On Mon, Dec 17, 2012 at 11:14:20AM +1200, Bryan Kearney wrote: > > On 12/18/2012 03:47 AM, Eric Helms wrote: > > >On 12/16/2012 05:27 PM, Bryan Kearney wrote: > > >>On 12/18/2012 03:26 AM, David Davis wrote: > > >>>Oh, I didn't see that content views were already included in 1.3 so > > >>>that's fine. I'll still volunteer to be the 1.4 nanny though unless > > >>>someone else wants to. > > >>> > > >>>David > > >>> > > >> > > >>Dow e want to go back across the pond for 1.3? > > >Do you mean 1.4? As it stands, the first two releases were done by the > > >guys in Europe, so after my nanny-ing for 1.2, we are still unbalanced. > > >After 1.3, we should go to back and forth across the pond every other > > >release. > > > > Ok.. 1.4 across the pond. thanks! Just want to make sure we are > > sharing the fun! > > > > -- bk > > > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From omaciel at redhat.com Wed Dec 19 14:22:57 2012 From: omaciel at redhat.com (Og Maciel) Date: Wed, 19 Dec 2012 09:22:57 -0500 (EST) Subject: [katello-devel] Katello's year 2012 in numbers In-Reply-To: <50D1A8EE.4050804@redhat.com> Message-ID: <1240561154.41199276.1355926977224.JavaMail.root@redhat.com> I also see 203 BZs where I was the QA Contact, but it felt like more than that :) -- Og Maciel Senior QA Engineer Red Hat, Inc. +1.646.707.7723 irc: omaciel From lzap at redhat.com Wed Dec 19 14:51:48 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 15:51:48 +0100 Subject: [katello-devel] Katello's year 2012 in numbers In-Reply-To: <50D1A8EE.4050804@redhat.com> References: <50D1A8EE.4050804@redhat.com> Message-ID: <20121219145148.GR2074@lzapx.brq.redhat.com> Shared, thanks for this! LZ On Wed, Dec 19, 2012 at 12:45:50PM +0100, Miroslav Suchy wrote: > Let me allow to summarize year of 2012. I gathered some numbers for you: > > == Code == > > We committed 5072 changesets to Katello > > http://lzap.fedorapeople.org/stats/katello/activity.html#commits_by_year > > > Top contributor is Miroslav Such? with 592 commits (728 when > including merges). > > https://github.com/Katello/katello/graphs/contributors?from=2012-01-12&to=2012-12-31&type=c > > > We resolved 1720 Bugs. > > https://bugzilla.redhat.com/buglist.cgi?bug_status=CLOSED&chfield=bug_status&chfieldfrom=2012-01-01&chfieldto=Now&classification=Red%20Hat&list_id=959476&product=CloudForms%20System%20Engine&query_format=advanced&order=changeddate%2Cbug_id%20DESC&limit=0 > > https://bugzilla.redhat.com/buglist.cgi?list_id=959488&classification=Community&chfieldto=Now&query_format=advanced&chfield=bug_status&chfieldfrom=2012-01-01&bug_status=CLOSED&product=Katello > > > == Project cost == > > This is wild estimation, but just guess: > Codebase: 99,723 lines (including code and mark up) > Effort (est.): 25 Person Years > Avg. Salary: $55000 year > Estimated cost of project:$ 1,365,353 > http://www.ohloh.net/p/katello > > > == Releases == > > We released 3 releases during year 2012: 1.0, 1.1, 1.2 > https://freecode.com/projects/katello/releases > > == IRC == > > We have 281 users on #katello. > And most active is jrist with 9898 lines written. > cliff brings happiness to the world as 24.4% of his lines contained > smiling faces. :) > > > > I'm looking forward to see you in new year :) > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Wed Dec 19 15:13:03 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 16:13:03 +0100 Subject: [katello-devel] Katello's year 2012 in numbers In-Reply-To: <50D1A8EE.4050804@redhat.com> References: <50D1A8EE.4050804@redhat.com> Message-ID: <20121219151303.GT2074@lzapx.brq.redhat.com> I have added this kind of statistics to be done by Release nanny: https://engineering.redhat.com/trac/kalpana/wiki/HowToReleaseKatello LZ On Wed, Dec 19, 2012 at 12:45:50PM +0100, Miroslav Suchy wrote: > Let me allow to summarize year of 2012. I gathered some numbers for you: > > == Code == > > We committed 5072 changesets to Katello > > http://lzap.fedorapeople.org/stats/katello/activity.html#commits_by_year > > > Top contributor is Miroslav Such? with 592 commits (728 when > including merges). > > https://github.com/Katello/katello/graphs/contributors?from=2012-01-12&to=2012-12-31&type=c > > > We resolved 1720 Bugs. > > https://bugzilla.redhat.com/buglist.cgi?bug_status=CLOSED&chfield=bug_status&chfieldfrom=2012-01-01&chfieldto=Now&classification=Red%20Hat&list_id=959476&product=CloudForms%20System%20Engine&query_format=advanced&order=changeddate%2Cbug_id%20DESC&limit=0 > > https://bugzilla.redhat.com/buglist.cgi?list_id=959488&classification=Community&chfieldto=Now&query_format=advanced&chfield=bug_status&chfieldfrom=2012-01-01&bug_status=CLOSED&product=Katello > > > == Project cost == > > This is wild estimation, but just guess: > Codebase: 99,723 lines (including code and mark up) > Effort (est.): 25 Person Years > Avg. Salary: $55000 year > Estimated cost of project:$ 1,365,353 > http://www.ohloh.net/p/katello > > > == Releases == > > We released 3 releases during year 2012: 1.0, 1.1, 1.2 > https://freecode.com/projects/katello/releases > > == IRC == > > We have 281 users on #katello. > And most active is jrist with 9898 lines written. > cliff brings happiness to the world as 24.4% of his lines contained > smiling faces. :) > > > > I'm looking forward to see you in new year :) > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From jweiss at redhat.com Wed Dec 19 15:19:03 2012 From: jweiss at redhat.com (Jeff Weiss) Date: Wed, 19 Dec 2012 10:19:03 -0500 Subject: [katello-devel] Katello's year 2012 in numbers In-Reply-To: <50D1A8EE.4050804@redhat.com> References: <50D1A8EE.4050804@redhat.com> Message-ID: <877goeqdco.fsf@blinky.jweiss.com> Miroslav Suchy writes: > Let me allow to summarize year of 2012. I gathered some numbers for you: > > == Code == > > We committed 5072 changesets to Katello > > http://lzap.fedorapeople.org/stats/katello/activity.html#commits_by_year > > > Top contributor is Miroslav Such? with 592 commits (728 when including > merges). > > https://github.com/Katello/katello/graphs/contributors?from=2012-01-12&to=2012-12-31&type=c > > > We resolved 1720 Bugs. > > https://bugzilla.redhat.com/buglist.cgi?bug_status=CLOSED&chfield=bug_status&chfieldfrom=2012-01-01&chfieldto=Now&classification=Red%20Hat&list_id=959476&product=CloudForms%20System%20Engine&query_format=advanced&order=changeddate%2Cbug_id%20DESC&limit=0 > > https://bugzilla.redhat.com/buglist.cgi?list_id=959488&classification=Community&chfieldto=Now&query_format=advanced&chfield=bug_status&chfieldfrom=2012-01-01&bug_status=CLOSED&product=Katello > > > == Project cost == > > This is wild estimation, but just guess: > Codebase: 99,723 lines (including code and mark up) > Effort (est.): 25 Person Years > Avg. Salary: $55000 year > Estimated cost of project:$ 1,365,353 > http://www.ohloh.net/p/katello > Strange, github says it's 450k lines: https://github.com/Katello/katello/graphs/code-frequency I'm sure it's less than that, just a quick wc on filetypes .haml, .rb, .py, and .js : [jweiss at blinky src]$ find .. -regex '.*\.haml$\|.*\.js$\|.*\.rb$\|.*\.py$\|.*\.pp$' | xargs wc -l 124360 total I'm not really sure what makes up the rest, I know there's filetypes I'm forgetting, and that images and such probably don't count. > > == Releases == > > We released 3 releases during year 2012: 1.0, 1.1, 1.2 > https://freecode.com/projects/katello/releases > > == IRC == > > We have 281 users on #katello. > And most active is jrist with 9898 lines written. > cliff brings happiness to the world as 24.4% of his lines contained > smiling faces. :) > > > > I'm looking forward to see you in new year :) > > Mirek > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From lzap at redhat.com Wed Dec 19 15:35:54 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Wed, 19 Dec 2012 16:35:54 +0100 Subject: [katello-devel] Why do we hide orgs being deleted Message-ID: <20121219153554.GV2074@lzapx.brq.redhat.com> Hey, why do we do this? Can't we just make a notification that organization is being deleted and leave it there? At least the way how we do this today using default model scope is ugly and I see hacks in the code when we need to deal with organization references (unscoping it). Wouldn't be better just to hide organization in the list of organizations in the controllers (UI/API) not breaking the rest of the application? Or is the default scope necessary for different reason (orchestration)? Opinions? -- Later, Lukas "lzap" Zapletal #katello #systemengine From jsherril at redhat.com Wed Dec 19 15:43:28 2012 From: jsherril at redhat.com (Justin Sherrill) Date: Wed, 19 Dec 2012 10:43:28 -0500 Subject: [katello-devel] Why do we hide orgs being deleted In-Reply-To: <20121219153554.GV2074@lzapx.brq.redhat.com> References: <20121219153554.GV2074@lzapx.brq.redhat.com> Message-ID: <50D1E0A0.3050804@redhat.com> On 12/19/2012 10:35 AM, Lukas Zapletal wrote: > Hey, > > why do we do this? Can't we just make a notification that organization > is being deleted and leave it there? > > At least the way how we do this today using default model scope is ugly > and I see hacks in the code when we need to deal with organization > references (unscoping it). Wouldn't be better just to hide organization > in the list of organizations in the controllers (UI/API) not breaking > the rest of the application? > > Or is the default scope necessary for different reason (orchestration)? > > Opinions? > So originally org deletion was made asynronous because it was taking too long (up to 10 minutes or so iirc). We wanted to make sure that anyone within the org being deleted was logged out as soon as possible, and that the org didn't show up within other parts of the application while it was being deleted. For example: For an org in the process of being deleted you wouldn't want to: * create permissions based on that org * create environments in that org * set a users default org to this org Probably a few more. Instead of hacking all of those locations (and probably missing a few), it seemed easier to make a default scope as there were fewer places that we would want to 'know about' an org being deleted than where we would want to ignore them. This did not seem fairly troublesome at the time, however if it is causing issues, I'm not opposed to changing it. There is no orchestration reason for it. -Justin From omaciel at redhat.com Wed Dec 19 15:51:55 2012 From: omaciel at redhat.com (Og Maciel) Date: Wed, 19 Dec 2012 10:51:55 -0500 (EST) Subject: [katello-devel] Why do we hide orgs being deleted In-Reply-To: <50D1E0A0.3050804@redhat.com> Message-ID: <1397956442.41284865.1355932315961.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Justin Sherrill" > To: katello-devel at redhat.com > Sent: Wednesday, December 19, 2012 10:43:28 AM > Subject: Re: [katello-devel] Why do we hide orgs being deleted > > For an org in the process of being deleted you wouldn't want to: > * create permissions based on that org > * create environments in that org > * set a users default org to this org > > Probably a few more. * Be able to re-create the same org while all of its components (repos for instance) were still around... > > Instead of hacking all of those locations (and probably missing a > few), > it seemed easier to make a default scope as there were fewer places > that > we would want to 'know about' an org being deleted than where we > would > want to ignore them. This did not seem fairly troublesome at the > time, > however if it is causing issues, I'm not opposed to changing it. > > There is no orchestration reason for it. > > -Justin > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > -- Og Maciel Senior QA Engineer Red Hat, Inc. +1.646.707.7723 irc: omaciel From dcleal at redhat.com Wed Dec 19 16:32:12 2012 From: dcleal at redhat.com (Dominic Cleal) Date: Wed, 19 Dec 2012 16:32:12 +0000 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <20121219093802.GG2074@lzapx.brq.redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> Message-ID: <50D1EC0C.70805@redhat.com> On 19/12/12 09:38, Lukas Zapletal wrote: > On Tue, Dec 18, 2012 at 11:25:31AM +0000, Dominic Cleal wrote: >> I suppose the assumption for now is that the puppetmaster will always be >> the Katello host - there won't any support for splitting up and >> multiplying the number of puppetmaster hosts yet? At that point, the >> hostname may need to be more dynamic than as delivered in the RPM. >> > > Well I mean we would just add puppetmaster bits into the katello-cert > rpm. This wiki explains what we actually do: > > https://fedorahosted.org/katello/wiki/GuideSystemRegistrationClient#Option1preferredmethod:UsingthebootstrapRPM > > and here you can see the RPM post inst script: > > https://github.com/Katello/katello/blob/master/katello-configure/modules/certs/templates/rhsm-katello-reconfigure.erb > > The RPM is generated during configuration of Katello, everything in > there is dynamic (and can be also re-generated with reapplying > katello-configure = puppet). This assumes a single puppetmaster though specified in the RPM upfront. Foreman supports any number of puppetmasters, which is specified on the host (and might be populated from the hostgroup) and could be installed alongside Foreman or on other hosts with smart proxies. How flexible will Katello be about supported Foreman and Puppet architectures to start with? -- Dominic Cleal Red Hat Engineering From lzap at redhat.com Thu Dec 20 08:49:42 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Thu, 20 Dec 2012 09:49:42 +0100 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50D1EC0C.70805@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> <50D1EC0C.70805@redhat.com> Message-ID: <20121220084942.GA1660@lzapx.brq.redhat.com> On Wed, Dec 19, 2012 at 04:32:12PM +0000, Dominic Cleal wrote: > This assumes a single puppetmaster though specified in the RPM upfront. > Foreman supports any number of puppetmasters, which is specified on the > host (and might be populated from the hostgroup) and could be installed > alongside Foreman or on other hosts with smart proxies. What you mean by multiple puppetmasters? I have to admit I am still not done with my Puppet book, but does it make really sense? Isn't added value of Puppet one centralized place for all the configuration? > How flexible will Katello be about supported Foreman and Puppet > architectures to start with? I would say, the first version with Foreman integrated will only support one instance (I mean - everything on one box), but we should definitely think to the future. Can you elaborate what issues do you see with this? -- Later, Lukas "lzap" Zapletal #katello #systemengine From bkearney at redhat.com Thu Dec 20 13:20:56 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 20 Dec 2012 08:20:56 -0500 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50D1EC0C.70805@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> <50D1EC0C.70805@redhat.com> Message-ID: <50D310B8.9020108@redhat.com> On 12/19/2012 11:32 AM, Dominic Cleal wrote: > On 19/12/12 09:38, Lukas Zapletal wrote: >> On Tue, Dec 18, 2012 at 11:25:31AM +0000, Dominic Cleal wrote: >>> I suppose the assumption for now is that the puppetmaster will always be >>> the Katello host - there won't any support for splitting up and >>> multiplying the number of puppetmaster hosts yet? At that point, the >>> hostname may need to be more dynamic than as delivered in the RPM. >>> >> >> Well I mean we would just add puppetmaster bits into the katello-cert >> rpm. This wiki explains what we actually do: >> >> https://fedorahosted.org/katello/wiki/GuideSystemRegistrationClient#Option1preferredmethod:UsingthebootstrapRPM >> >> and here you can see the RPM post inst script: >> >> https://github.com/Katello/katello/blob/master/katello-configure/modules/certs/templates/rhsm-katello-reconfigure.erb >> >> The RPM is generated during configuration of Katello, everything in >> there is dynamic (and can be also re-generated with reapplying >> katello-configure = puppet). > > This assumes a single puppetmaster though specified in the RPM upfront. > Foreman supports any number of puppetmasters, which is specified on the > host (and might be populated from the hostgroup) and could be installed > alongside Foreman or on other hosts with smart proxies. > > How flexible will Katello be about supported Foreman and Puppet > architectures to start with? > The goal is to follow hte foreman model. Have installers to set up 1-n proxies and 1-n puppet masters on different machines, and then use the katello-ui (which in turn uses the foreman api) to register these proxies and puppet masters. Pulp 2.0 will allows us to migrate puppet content. So, we will then be able to build up content in these puppet masters using content views and promotions. -- bk From bkearney at redhat.com Thu Dec 20 13:21:45 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 20 Dec 2012 08:21:45 -0500 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <20121220084942.GA1660@lzapx.brq.redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> <50D1EC0C.70805@redhat.com> <20121220084942.GA1660@lzapx.brq.redhat.com> Message-ID: <50D310E9.4070704@redhat.com> On 12/20/2012 03:49 AM, Lukas Zapletal wrote: > On Wed, Dec 19, 2012 at 04:32:12PM +0000, Dominic Cleal wrote: >> This assumes a single puppetmaster though specified in the RPM upfront. >> Foreman supports any number of puppetmasters, which is specified on the >> host (and might be populated from the hostgroup) and could be installed >> alongside Foreman or on other hosts with smart proxies. > > What you mean by multiple puppetmasters? I have to admit I am still not > done with my Puppet book, but does it make really sense? Isn't added > value of Puppet one centralized place for all the configuration? It is part a scaling thing, and part a deployment model. You can assume many puppet masters deployed for every katello server. -- bk From dcleal at redhat.com Thu Dec 20 14:12:59 2012 From: dcleal at redhat.com (Dominic Cleal) Date: Thu, 20 Dec 2012 14:12:59 +0000 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <20121220084942.GA1660@lzapx.brq.redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> <50D1EC0C.70805@redhat.com> <20121220084942.GA1660@lzapx.brq.redhat.com> Message-ID: <50D31CEB.70100@redhat.com> On 20/12/12 08:49, Lukas Zapletal wrote: > On Wed, Dec 19, 2012 at 04:32:12PM +0000, Dominic Cleal wrote: >> This assumes a single puppetmaster though specified in the RPM upfront. >> Foreman supports any number of puppetmasters, which is specified on the >> host (and might be populated from the hostgroup) and could be installed >> alongside Foreman or on other hosts with smart proxies. > > What you mean by multiple puppetmasters? I have to admit I am still not > done with my Puppet book, but does it make really sense? Isn't added > value of Puppet one centralized place for all the configuration? You're right, you would still want a central source of configuration. The typical use for having multiple puppetmaster servers is to increase performance and serve a larger number of clients. Serving Puppet clients can be quite CPU and I/O intensive, as catalog compilations can take tens of seconds (tying up a web server worker) as it loads and parses .pp files. Other uses would be for organisational or network topology splits (say, a puppetmaster in each datacentre), or for environments, since Puppet doesn't cope well with different versions of certain features in each environment on the same master. The modules deployed to these masters would usually come from a single source still, such as directly from git or packages. The single source of content for Katello would be Pulp. >> How flexible will Katello be about supported Foreman and Puppet >> architectures to start with? > > I would say, the first version with Foreman integrated will only support > one instance (I mean - everything on one box), but we should definitely > think to the future. > > Can you elaborate what issues do you see with this? I think that's sensible. I would imagine that the scalability and performance problems I mention above aren't going to be unique to puppetmasters (Pulp comes to mind, you want content near clients) and that in due course we may want to support multiple instances of certain applications to cater for increasing numbers of clients. If the intention is to only support everything-in-a-box for now, then the puppetmaster is similar to other Katello components in that it should get installed alongside Katello, Foreman and the Foreman smart proxy. We should manage its config, but then perhaps leave it to the user to add module content. In the future, we could support "satellite" puppetmasters being installed (with a Foreman smart proxy) to solve any of the problems listed above. Deploying additional instances could be done entirely through puppet modules. -- Dominic Cleal Red Hat Engineering From dcleal at redhat.com Thu Dec 20 14:42:13 2012 From: dcleal at redhat.com (Dominic Cleal) Date: Thu, 20 Dec 2012 14:42:13 +0000 Subject: [katello-devel] Foreman Registration Design In-Reply-To: <50D310B8.9020108@redhat.com> References: <1155691569.609675.1355406761916.JavaMail.root@redhat.com> <50CA0388.9000907@redhat.com> <50CB6056.8020407@redhat.com> <20121217132924.GE1698@lzapx.brq.redhat.com> <50D052AB.9050707@redhat.com> <20121219093802.GG2074@lzapx.brq.redhat.com> <50D1EC0C.70805@redhat.com> <50D310B8.9020108@redhat.com> Message-ID: <50D323C5.9020603@redhat.com> On 20/12/12 13:20, Bryan Kearney wrote: > On 12/19/2012 11:32 AM, Dominic Cleal wrote: >> On 19/12/12 09:38, Lukas Zapletal wrote: >>> The RPM is generated during configuration of Katello, everything in >>> there is dynamic (and can be also re-generated with reapplying >>> katello-configure = puppet). >> >> This assumes a single puppetmaster though specified in the RPM upfront. >> Foreman supports any number of puppetmasters, which is specified on the >> host (and might be populated from the hostgroup) and could be installed >> alongside Foreman or on other hosts with smart proxies. >> >> How flexible will Katello be about supported Foreman and Puppet >> architectures to start with? >> > The goal is to follow hte foreman model. Have installers to set up 1-n > proxies and 1-n puppet masters on different machines, and then use the > katello-ui (which in turn uses the foreman api) to register these > proxies and puppet masters. As there are Puppet classes for the proxy, puppet::server and Katello itself, I think this could be a fairly generic mechanism, not specific to puppetmasters. You could build additional Katello instances simply from your main Katello instance by adding the relevant modules for the components (puppetmaster/foreman/pulp etc) to the host, plus some SSL/CSR magic for service certificates. Or build them with katello-configure and register them as extensions to the Katello host. I'm not sure if the smart proxy bits need exposing to the user in order to install additional puppetmasters. > Pulp 2.0 will allows us to migrate puppet content. So, we will then be > able to build up content in these puppet masters using content views and > promotions. This will be very exciting for the Puppet community. -- Dominic Cleal Red Hat Engineering From jweiss at redhat.com Thu Dec 20 18:40:52 2012 From: jweiss at redhat.com (Jeff Weiss) Date: Thu, 20 Dec 2012 13:40:52 -0500 Subject: [katello-devel] src/lib/navigation/*.rb : keys must be unique Message-ID: <87y5gspnwr.fsf@blinky.jweiss.com> These are the files that define the menu navigation. Each entry has a :key, you cannot have duplicate keys anywhere else that whole dir, but they keep getting added with duplicates. The problem with dupes is that it produces HTML where multiple elements in the DOM have the same id attribute. That's invalid HTML, and breaks selenium automated tests, because it can't uniquely identify the menu items. If anyone could write a unit test that checks this, that would be awesome. I will submit a PR to correct the latest round of dupes. -Jeff From kybaker at redhat.com Thu Dec 20 22:09:48 2012 From: kybaker at redhat.com (Kyle Baker) Date: Thu, 20 Dec 2012 17:09:48 -0500 (EST) Subject: [katello-devel] High Fidelity Mock-ups and Specs for Foreman Integration In-Reply-To: <424177145.3244849.1356040291505.JavaMail.root@redhat.com> Message-ID: <900126190.3249081.1356041388435.JavaMail.root@redhat.com> I have updated the Katello wiki with high fidelity mockups of the Foreman integration wireframes as well as specs to assist with implementation. The high fidelity mockups are intended to show the wireframes as they would exist in the UI. There has been no additional changes of functionality or layout which had been previously integrated in the wireframes. I have created high fidelity mockups and specs for each unique element focusing primarily on visuals. Refer to the wireframes for all content and interactions for the entire Foreman integration design. All Foreman integration designs as well as specs for those can be seen here - https://fedorahosted.org/katello/wiki/Foreman%20Integration Kyle Baker --- UX Team Desk 978 392 3116 IRC kylebaker From msuchy at redhat.com Fri Dec 21 09:16:06 2012 From: msuchy at redhat.com (=?ISO-8859-2?Q?Miroslav_Such=FD?=) Date: Fri, 21 Dec 2012 10:16:06 +0100 Subject: [katello-devel] Katello 1.2 component in BZ Message-ID: <50D428D6.2010609@redhat.com> Whoever have enought power - please add Katello 1.2 component in BZ. Last we have there is 1.1. Thanks -- Miroslav Suchy Red Hat Systems Management Engineering From lzap at redhat.com Fri Dec 21 09:20:44 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 21 Dec 2012 10:20:44 +0100 Subject: [katello-devel] Katello 1.2 component in BZ In-Reply-To: <50D428D6.2010609@redhat.com> References: <50D428D6.2010609@redhat.com> Message-ID: <20121221092044.GB1639@lzapx.brq.redhat.com> Maybe even more... LZ On Fri, Dec 21, 2012 at 10:16:06AM +0100, Miroslav Suchy wrote: > Whoever have enought power - please add Katello 1.2 component in BZ. > Last we have there is 1.1. > > Thanks > -- > Miroslav Suchy > Red Hat Systems Management Engineering > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From lzap at redhat.com Fri Dec 21 09:25:58 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 21 Dec 2012 10:25:58 +0100 Subject: [katello-devel] High Fidelity Mock-ups and Specs for Foreman Integration In-Reply-To: <900126190.3249081.1356041388435.JavaMail.root@redhat.com> References: <424177145.3244849.1356040291505.JavaMail.root@redhat.com> <900126190.3249081.1356041388435.JavaMail.root@redhat.com> Message-ID: <20121221092558.GC1639@lzapx.brq.redhat.com> Just a quick note not related to the UI: On the Organization screen, I can see DHCP/DNS/TFT/ to be part of an organization. Are we assuming one organization can only have one particular network service? I would expect a Network or something on the Organization page. Or are these just a mockups without real content in it? LZ On Thu, Dec 20, 2012 at 05:09:48PM -0500, Kyle Baker wrote: > I have updated the Katello wiki with high fidelity mockups of the Foreman integration wireframes as well as specs to assist with implementation. The high fidelity mockups are intended to show the wireframes as they would exist in the UI. There has been no additional changes of functionality or layout which had been previously integrated in the wireframes. I have created high fidelity mockups and specs for each unique element focusing primarily on visuals. Refer to the wireframes for all content and interactions for the entire Foreman integration design. > > All Foreman integration designs as well as specs for those can be seen here - https://fedorahosted.org/katello/wiki/Foreman%20Integration > > Kyle Baker > --- > UX Team > Desk 978 392 3116 > IRC kylebaker > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel -- Later, Lukas "lzap" Zapletal #katello #systemengine From cperry at redhat.com Fri Dec 21 10:12:35 2012 From: cperry at redhat.com (Cliff Perry) Date: Fri, 21 Dec 2012 10:12:35 +0000 Subject: [katello-devel] Katello 1.2 component in BZ In-Reply-To: <20121221092044.GB1639@lzapx.brq.redhat.com> References: <50D428D6.2010609@redhat.com> <20121221092044.GB1639@lzapx.brq.redhat.com> Message-ID: <50D43613.9090202@redhat.com> On 12/21/2012 09:20 AM, Lukas Zapletal wrote: > Maybe even more... > > LZ Add to the Nanny guide duties a task to poke /someone/ to add next version. I have added: 1.2 1.3 To Katello versions. Let me know if you need anything, now or in the future :) BTW - not sure if you really want the 1.x and 2.0 versions, but they have bugs associated with those versions, maybe RFE's or BZ's punted for future. Cliff Version Active Bugs Action 1.0 Yes 188 Delete 1.1 Yes 10 Delete 1.2 Yes 0 Delete 1.3 Yes 0 Delete 1.X Yes 32 Delete 2.0 Yes 28 Delete > > On Fri, Dec 21, 2012 at 10:16:06AM +0100, Miroslav Suchy wrote: >> Whoever have enought power - please add Katello 1.2 component in BZ. >> Last we have there is 1.1. >> >> Thanks >> -- >> Miroslav Suchy >> Red Hat Systems Management Engineering >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel > From lzap at redhat.com Fri Dec 21 12:05:13 2012 From: lzap at redhat.com (Lukas Zapletal) Date: Fri, 21 Dec 2012 13:05:13 +0100 Subject: [katello-devel] Keeping our installer "stable" Message-ID: <20121221120512.GG1639@lzapx.brq.redhat.com> Hey, I noticed today that our installer is not "stable" anymore (not sure the proper wording from the Puppet world). Basically, all subsequent executions with the same parameters should not do any configuration changes - Puppet should only check that everything is deployed properly and return immediately (in 1-3 seconds). Currently nightly is trying to reconfigure foreman and do several other things. So the rule of thumb here is: When you are working on installer, make sure it works correctly AND try to run it several times with the same parameters - it should not do anything. https://bugzilla.redhat.com/show_bug.cgi?id=889488 I will fix this bug next year, but keep this in mind please. I have been refactoring a LOT of Puppet code some time ago to make it "stable". Thanks -- Later, Lukas "lzap" Zapletal #katello #systemengine From ericdhelms at gmail.com Fri Dec 21 13:23:17 2012 From: ericdhelms at gmail.com (Eric D Helms) Date: Fri, 21 Dec 2012 08:23:17 -0500 Subject: [katello-devel] Rails 3.2 features In-Reply-To: References: <18062358.42808689.1354899800009.JavaMail.root@redhat.com> <211697786.42812901.1354900515970.JavaMail.root@redhat.com> Message-ID: I have recently spent some time working getting Katello up and running on the Rails 3.2 stack and thought I would give a status update and point people to the work in the event they wanted to explore or try it out. Branch: https://github.com/Katello/katello/tree/rails32 You should be able to install a normal Katello git checkout, switch to the branch and using RVM get the sever up and running. Ping me if you run into issues attempting to try it. Status: Green: Asset Pipeline - fully integrated in both development and production environments, assets moved to proper locations Alchemy - submodule entirely removed from project, Alchemy now used as a gem and Rails engine Tupane - required defining tupane layouts in the views themselves (much cleaner way of doing it anyhow) Yellow: Spec - Not fully flushed out or tested Install - see above Red: Spec tests - Fixes for a lot of test failures, still a number of failures exist, largely around the use of "update_attributes" System Tests - Not run yet UI Testing - items create and update, however, seeing validation messages returned upon successful create of an object (multiple requests being made) Any help trying to tackle or investigate the issues listed above would be greatly appreciated. Please reply here or ping me if you try to tackle one or the other to prevent overlap where possible. I had previously been pushing directly into the branch due to the large size and testing nature of some of the work. I plan to start pull requesting into the branch from here on out. - Eric On Fri, Dec 7, 2012 at 1:48 PM, Eric D Helms wrote: > My previous experience porting a 3.0 app to a 3.2 application wasn't too > shabby in the end. I only had to change application.rb and the rest was > all asset pipeline related which just involved mostly moving files around > and creating manifests for the Javacript files. > > -Eric > > > On Fri, Dec 7, 2012 at 12:15 PM, David Davis wrote: > >> I thought I'd send a second email in case you guys happened to miss the >> first one. >> >> David >> >> ----- Original Message ----- >> > From: "David Davis" >> > To: katello-devel at redhat.com >> > Sent: Friday, December 7, 2012 12:03:20 PM >> > Subject: [katello-devel] Rails 3.2 features >> > >> > I just wanted to make a list of features that we'll get upgrading to >> > Rails 3.2. I'm sending these out to inform everyone but I think it's >> > worth reiterating that it's going to be painful to upgrade. It won't >> > be as bad as going from Rails 2 to Rails 3 but it's not going to be >> > easy either. Without futher ado... >> > >> > * Probably the biggest thing is the asset pipeline. It does >> > everything from compiling languages (like SASS) to uglifying js >> > code. Check out more info at >> > http://guides.rubyonrails.org/asset_pipeline.html >> > * New migration features. You no longer have to write up/down methods >> > for the most part. Instead you just write "change" methods. More >> > info at http://guides.rubyonrails.org/migrations.html >> > * Multiple levels of has_many :through >> > * Create key value stores in the database >> > ( >> http://blog.rawonrails.com/2012/02/using-activerecordstore-with-rails-32.html >> ) >> > * New active record methods like pluck >> > ( >> http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-pluck >> ) >> > * Scoped mass assignment. Helpful for locking down attributes per >> > permission. >> > ( >> http://launchware.com/articles/whats-new-in-edge-scoped-mass-assignment-in-rails-3-1 >> ) >> > * Slow query logger >> > >> http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/ >> . >> > I've used http://explain.depesz.com/ to help analyze query explain >> > output from postgres. >> > * Encrypted fields (passwords, etc) >> > http://railscasts.com/episodes/270-authentication-in-rails-3-1 >> > * New generator stuff. You can do "rails g scaffold Person name >> > email" instead of "rails g scaffold Person name:string >> > email:string". Or if you want to be really crafty: rails g scaffold >> > Person name:index email:uniq. >> > >> > David >> > >> > David >> > >> > _______________________________________________ >> > katello-devel mailing list >> > katello-devel at redhat.com >> > https://www.redhat.com/mailman/listinfo/katello-devel >> > >> >> _______________________________________________ >> katello-devel mailing list >> katello-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/katello-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jweiss at redhat.com Fri Dec 21 13:32:21 2012 From: jweiss at redhat.com (Jeff Weiss) Date: Fri, 21 Dec 2012 08:32:21 -0500 Subject: [katello-devel] Keeping our installer "stable" In-Reply-To: <20121221120512.GG1639@lzapx.brq.redhat.com> References: <20121221120512.GG1639@lzapx.brq.redhat.com> Message-ID: <87obhnpm3e.fsf@blinky.jweiss.com> Lukas Zapletal writes: > Hey, > > I noticed today that our installer is not "stable" anymore (not sure the > proper wording from the Puppet world). Basically, all subsequent > executions with the same parameters should not do any configuration > changes - Puppet should only check that everything is deployed properly > and return immediately (in 1-3 seconds). I think the word you're looking for is idempotent: http://en.wikipedia.org/wiki/Idempotence > > Currently nightly is trying to reconfigure foreman and do several other > things. So the rule of thumb here is: > > When you are working on installer, make sure it works correctly AND try > to run it several times with the same parameters - it should not do > anything. > > https://bugzilla.redhat.com/show_bug.cgi?id=889488 > > I will fix this bug next year, but keep this in mind please. I have been > refactoring a LOT of Puppet code some time ago to make it "stable". > > Thanks > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel From kybaker at redhat.com Fri Dec 21 14:23:05 2012 From: kybaker at redhat.com (Kyle Baker) Date: Fri, 21 Dec 2012 09:23:05 -0500 (EST) Subject: [katello-devel] High Fidelity Mock-ups and Specs for Foreman Integration In-Reply-To: <20121221092558.GC1639@lzapx.brq.redhat.com> Message-ID: <2080259777.3473489.1356099785624.JavaMail.root@redhat.com> ----- Original Message ----- > Just a quick note not related to the UI: > > On the Organization screen, I can see DHCP/DNS/TFT/ to be part of an > organization. Are we assuming one organization can only have one > particular network service? > > I would expect a Network or something on the Organization page. > > Or are these just a mockups without real content in it? As part of PM's usecase for server containers there is the concept of inheritance. You can set the network settings in the org and they will be inherited by the server container as default, but under server container details you can change them to what ever you want. The default is merely a suggestion in each services dropdown. There is a tab in the org detail that says 'Services'. This is where the default network settings are configured. In the main navigation (in the context of an org) the user can still configure all Smart Proxies and Subnets separately as they do in Foreman today. > > LZ > > On Thu, Dec 20, 2012 at 05:09:48PM -0500, Kyle Baker wrote: > > I have updated the Katello wiki with high fidelity mockups of the > > Foreman integration wireframes as well as specs to assist with > > implementation. The high fidelity mockups are intended to show the > > wireframes as they would exist in the UI. There has been no > > additional changes of functionality or layout which had been > > previously integrated in the wireframes. I have created high > > fidelity mockups and specs for each unique element focusing > > primarily on visuals. Refer to the wireframes for all content and > > interactions for the entire Foreman integration design. > > > > All Foreman integration designs as well as specs for those can be > > seen here - > > https://fedorahosted.org/katello/wiki/Foreman%20Integration > > > > Kyle Baker > > --- > > UX Team > > Desk 978 392 3116 > > IRC kylebaker > > > > _______________________________________________ > > katello-devel mailing list > > katello-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/katello-devel > > -- > Later, > > Lukas "lzap" Zapletal > #katello #systemengine > > _______________________________________________ > katello-devel mailing list > katello-devel at redhat.com > https://www.redhat.com/mailman/listinfo/katello-devel > From bkearney at redhat.com Fri Dec 21 14:42:33 2012 From: bkearney at redhat.com (Bryan Kearney) Date: Fri, 21 Dec 2012 09:42:33 -0500 Subject: [katello-devel] High Fidelity Mock-ups and Specs for Foreman Integration In-Reply-To: <2080259777.3473489.1356099785624.JavaMail.root@redhat.com> References: <2080259777.3473489.1356099785624.JavaMail.root@redhat.com> Message-ID: <50D47559.6040809@redhat.com> On 12/21/2012 09:23 AM, Kyle Baker wrote: > > > ----- Original Message ----- >> Just a quick note not related to the UI: >> >> On the Organization screen, I can see DHCP/DNS/TFT/ to be part of an >> organization. Are we assuming one organization can only have one >> particular network service? >> >> I would expect a Network or something on the Organization page. >> >> Or are these just a mockups without real content in it? > > As part of PM's usecase for server containers there is the concept of inheritance. You can set the network settings in the org and they will be inherited by the server container as default, but under server container details you can change them to what ever you want. The default is merely a suggestion in each services dropdown. There is a tab in the org detail that says 'Services'. This is where the default network settings are configured. In the main navigation (in the context of an org) the user can still configure all Smart Proxies and Subnets separately as they do in Foreman today. Lukas: Where there are differences, we we will want to start with what foreman does today and then evolve to the differences over time. First goal, integrate the current foreman process into katello to enable provisioning. -- bk From daviddavis at redhat.com Fri Dec 21 15:56:19 2012 From: daviddavis at redhat.com (David Davis) Date: Fri, 21 Dec 2012 10:56:19 -0500 (EST) Subject: [katello-devel] Gemfile.lock files In-Reply-To: <1615006422.57145458.1356105331478.JavaMail.root@redhat.com> Message-ID: <1379622925.57146711.1356105379133.JavaMail.root@redhat.com> Is anyone using the Gemfile.locks in the gemfiles directory? If not, I am thinking of removing them since they're not up to date and they're a huge pain to keep updated. David From mmccune at redhat.com Fri Dec 21 16:17:28 2012 From: mmccune at redhat.com (Mike McCune) Date: Fri, 21 Dec 2012 08:17:28 -0800 Subject: [katello-devel] Katello 1.2 component in BZ In-Reply-To: <50D43613.9090202@redhat.com> References: <50D428D6.2010609@redhat.com> <20121221092044.GB1639@lzapx.brq.redhat.com> <50D43613.9090202@redhat.com> Message-ID: <50D48B98.60605@redhat.com> On 12/21/2012 02:12 AM, Cliff Perry wrote: > On 12/21/2012 09:20 AM, Lukas Zapletal wrote: >> Maybe even more... >> >> LZ > > Add to the Nanny guide duties a task to poke /someone/ to add next version. > > I have added: > 1.2 > 1.3 > > To Katello versions. > > Let me know if you need anything, now or in the future :) > > BTW - not sure if you really want the 1.x and 2.0 versions, but they > have bugs associated with those versions, maybe RFE's or BZ's punted for > future. > > Cliff likewise, I have BZ admin rights and I would have done this if Cliff didn't beat me to the punch :) From jsherril at redhat.com Fri Dec 21 16:44:56 2012 From: jsherril at redhat.com (Justin Sherrill) Date: Fri, 21 Dec 2012 11:44:56 -0500 Subject: [katello-devel] scrum status 12/21 Message-ID: <50D49208.4040305@redhat.com> Previously: Continued working on repo discovery (creation of repos specifically) Today: Finish up repo discovery Do ethics training From mmorsi at redhat.com Fri Dec 21 17:15:58 2012 From: mmorsi at redhat.com (Mo Morsi) Date: Fri, 21 Dec 2012 12:15:58 -0500 Subject: [katello-devel] Gemfile.lock files In-Reply-To: <1379622925.57146711.1356105379133.JavaMail.root@redhat.com> References: <1379622925.57146711.1356105379133.JavaMail.root@redhat.com> Message-ID: <50D4994E.5000000@redhat.com> On 12/21/2012 10:56 AM, David Davis wrote: > Is anyone using the Gemfile.locks in the gemfiles directory? If not, I am thinking of removing them since they're not up to date and they're a huge pain to keep updated. > > David > > I've noticed a few upstream projects that have done this recently. Not all of em of course, but its not too uncommon to just see a Gemfile (if that). Good riddance IMO, never was a fan of the bundler approach. I often delete the Gemfile.lock from a project I checkout as one of the first things (even for stuff like aeolus and isitfedoraruby) Just my 2 cents. -Mo