From icon at fedoraproject.org Thu Nov 1 02:36:18 2007 From: icon at fedoraproject.org (Konstantin Ryabitsev) Date: Wed, 31 Oct 2007 22:36:18 -0400 Subject: Php why must your apps suck so? In-Reply-To: <1193869479.28356.8.camel@ignacio.lan> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> Message-ID: On 10/31/07, Ignacio Vazquez-Abrams wrote: > > One thing Fedora has is expertise in writing SELinux policy. A working > > SELinux policy would be a good contribution to an upstream. > > SELinux can't help with XSS attacks. Well, in all fairness -- XSS and similar attacks are equally as present in any other programming environment. It's easy to be sloppy with user input in any language. Regards, -- Konstantin Ryabitsev Montr?al, Qu?bec From amolarakh at gmail.com Thu Nov 1 06:50:54 2007 From: amolarakh at gmail.com (amolarakh) Date: Thu, 01 Nov 2007 12:20:54 +0530 Subject: Introduction to List Message-ID: <4729774E.3040205@gmail.com> Hello list, I am Amol Arakh Basically I am Computer Engineer,and i am end user of RedHat Linux OS since RedHat 9.0 Launched,. Currently I am working as Linux System Administrator in VPCOE,Baramati,Maharashtra,INDIA (An Pronouned Educational Institute) since last 2-3 years. In my Network Currently Most of Clients are working on Fedora Core 6.0. I am proud to say that for the installation purpose I am using one of Famous Network Installation System named as DRBL,for Network in our Institute. I have hands on Practice on Configuration of IPTABLEs,SQUID,Samba,APACHE,vsFTPD,PAM configuration,Packaging,Taking Backups. Thanks, Warm Regards, Amol Arakh -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From thinklinux.ssh at gmail.com Thu Nov 1 08:26:42 2007 From: thinklinux.ssh at gmail.com (susmit shannigrahi) Date: Thu, 1 Nov 2007 13:56:42 +0530 Subject: Introduction to List In-Reply-To: <4729774E.3040205@gmail.com> References: <4729774E.3040205@gmail.com> Message-ID: > I am Amol Arakh. Welcome to the list. A few questions,please find them inline. > In my Network Currently Most of Clients are working on Fedora Core 6.0. Nice. > I am proud to say that for the installation purpose I am using one of > Famous Network Installation System named as DRBL,for Network in our > Institute. Cobbler ? And how do you update? Do you have a local mirror? Please let us know. -- Warm Regards, Susmit. ====================================== ssh 0x86DD170A http://www.fedoraproject.org/wiki/SusmitShannigrahi ====================================== From mastahnke at gmail.com Thu Nov 1 12:59:32 2007 From: mastahnke at gmail.com (Michael Stahnke) Date: Thu, 1 Nov 2007 07:59:32 -0500 Subject: Php why must your apps suck so? In-Reply-To: References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> Message-ID: <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> Again, blaming the lanaguage doesn't make a ton of sense. If you're worried about XSS, audit the code. If you're worried about buffer attacks, run SELinux. The list goes on. These same security measures should be taken with any application in any lanaguage. Just because we have some PHP-haters out there, doesn't really mean it sucks. Sure it's easy to write bad code in. So is Bash. Should we ban bash from all Fedora systems? I can write all sorts of junk with it. stahnma From skvidal at fedoraproject.org Thu Nov 1 13:02:47 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Thu, 01 Nov 2007 09:02:47 -0400 Subject: Php why must your apps suck so? In-Reply-To: <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> Message-ID: <1193922167.2654.0.camel@cutter> On Thu, 2007-11-01 at 07:59 -0500, Michael Stahnke wrote: > Again, blaming the lanaguage doesn't make a ton of sense. If you're > worried about XSS, audit the code. If you're worried about buffer > attacks, run SELinux. The list goes on. > > These same security measures should be taken with any application in > any lanaguage. Just because we have some PHP-haters out there, > doesn't really mean it sucks. Sure it's easy to write bad code in. > So is Bash. Should we ban bash from all Fedora systems? I can write > all sorts of junk with it. > If someone was writing a public-facing non-authenticated application in bash, then yes, I would not recommend using it, too. -sv From jkeating at redhat.com Thu Nov 1 13:31:01 2007 From: jkeating at redhat.com (Jesse Keating) Date: Thu, 1 Nov 2007 09:31:01 -0400 Subject: Reminder of change freeze In-Reply-To: <472905B8.70307@redhat.com> References: <472905B8.70307@redhat.com> Message-ID: <20071101093101.00c67d5a@redhat.com> On Wed, 31 Oct 2007 17:46:16 -0500 Mike McGrath wrote: > Just a reminder that after tomorrow night no changes are allowed of > any kind in Fedora's infrastructure without discussing it on the list > first. This includes software upgrades, config changes, etc. > > Exceptions are outages / issues. > Changes already in the ticketing system related to the release (like > getting the Fedora 8 content generated and put on the website) In the future can we align this with the final freeze? I know it lengthens the period when no changes can go in, but it also prevents things from going badly right as we are trying to get the last mile of testing in. Thoughts? -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From mmcgrath at redhat.com Thu Nov 1 13:24:18 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 01 Nov 2007 08:24:18 -0500 Subject: Reminder of change freeze In-Reply-To: <20071101093101.00c67d5a@redhat.com> References: <472905B8.70307@redhat.com> <20071101093101.00c67d5a@redhat.com> Message-ID: <4729D382.3010401@redhat.com> Jesse Keating wrote: > On Wed, 31 Oct 2007 17:46:16 -0500 > Mike McGrath wrote: > > >> Just a reminder that after tomorrow night no changes are allowed of >> any kind in Fedora's infrastructure without discussing it on the list >> first. This includes software upgrades, config changes, etc. >> >> Exceptions are outages / issues. >> Changes already in the ticketing system related to the release (like >> getting the Fedora 8 content generated and put on the website) >> > > In the future can we align this with the final freeze? I know it > lengthens the period when no changes can go in, but it also prevents > things from going badly right as we are trying to get the last mile of > testing in. > Right now it is set for one week before release day, what time line would you prefer? -Mike From jkeating at redhat.com Thu Nov 1 14:03:47 2007 From: jkeating at redhat.com (Jesse Keating) Date: Thu, 1 Nov 2007 10:03:47 -0400 Subject: Reminder of change freeze In-Reply-To: <4729D382.3010401@redhat.com> References: <472905B8.70307@redhat.com> <20071101093101.00c67d5a@redhat.com> <4729D382.3010401@redhat.com> Message-ID: <20071101100347.2fb13ba9@redhat.com> On Thu, 01 Nov 2007 08:24:18 -0500 Mike McGrath wrote: > Right now it is set for one week before release day, what time line > would you prefer? I'd say align it with the final freeze date of the release. That's the critical few weeks where we bugfix all the remaining stuff, get release candidates out and the final tree synced to the mirror. Any delays during those weeks can have disastrous results regarding the ship date. One week before release day is actually mostly quiet time, letting the mirrors pick up the bits and maybe doing the first set of updates push for the new release. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From mmcgrath at redhat.com Thu Nov 1 16:29:02 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 01 Nov 2007 11:29:02 -0500 Subject: Sysctl on the proxy servers Message-ID: <4729FECE.6050805@redhat.com> I'd like to discuss this at the meeting today, here are the optimizations as they stand for our proxy boxes. Its ticket #222: # Kernel sysctl configuration file for Red Hat Linux # # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and # sysctl.conf(5) for more details. # Controls IP packet forwarding net.ipv4.ip_forward = 0 # Controls source route verification net.ipv4.conf.default.rp_filter = 1 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 1 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 # Ensure connection tracking isn't limiting our connections net.ipv4.ip_conntrack_max=6553600 # Allow higher than default file descriptors fs.file-max=4947900 # How many pages to free at a time vm.page-cluster = 7 # Try to always keep this amount free vm.min_free_kbytes = 10000 # Allow system to be a swappier than normal when it needs to be for caching server vm.swappiness = 60 # Security, protects against TIME WAIT attacks net.ipv4.tcp_rfc1337 = 1 # Security, protects against SYN floods net.ipv4.tcp_syncookies = 1 # Lower keep alive time on the edge proxies net.ipv4.tcp_keepalive_time = 300 # Limit tcp orphans #net.ipv4.tcp_max_orphans = 1000 # Give the network stack access to more memory for queueing net.core.rmem_default = 262144 net.core.rmem_max = 262144 From a.badger at gmail.com Thu Nov 1 17:28:52 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 01 Nov 2007 10:28:52 -0700 Subject: Php why must your apps suck so? In-Reply-To: <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> Message-ID: <472A0CD4.90401@gmail.com> Michael Stahnke wrote: > Again, blaming the lanaguage doesn't make a ton of sense. If you're > worried about XSS, audit the code. If you're worried about buffer > attacks, run SELinux. The list goes on. > We could audit the code of everything that goes into infrastructure but our manpower would spend quite a bit of time on that (every new service and every update would have to be reviewed....) Can we do this? Could we setup a SIG which was interested enough in security to do this? (Such a sub-project would have a benefit to Fedora as a whole, not just infrastructure... but it isn't a trivial task.) If not we have to play the stock market on this one. 1) What's past performance been like? 2) How's the current management doing at identifying and removing security problems? For #1, compare the number of CVEs_ in mediawiki to moin and drupal to zope+plone: 2007 2006 2005 moin 5 0 0 mediawiki 7 5 12 drupal 36 37 8 zope(plone) 1(+0) 2(+3) 1(+0) Now we all know that numbers can be misleading but still this seems to highlight something for me: there are projects which care about security and there are projects which tack it on as an after thought. No matter how much work we put into security locally (SELinux, mod_security, code auditing), we don't want to be using a project which belongs to the latter camp. *Sending security patches upstream doesn't help if upstream will just introduce a new batch of security issues in their next release.* PS: Purely on the basis of these numbers I'd be led to believe that replacing moin with mediawiki would be acceptable. Knowing the mismatch in priorities between us and upstream moin, this might be a benefit to us (especially if we have some expert help tuning an SELinux policy and setup mod_security for it). Drupal, though, just makes me feel like we'd be trying to keep water in a sieve by putting it in a trash bag. PPS: we're currently creating SELinux policies for our machines that allow us to run in enforcing mode... Depending on the timeframe we're looking at for deploying that, we might need to create an app server that is specifically going to run SELinux from day one; implementing these new services there while we transition the old services over. .. CVEs: CVEs were simply searched for by project name in http://nvd.nist.gov/nvd.cfm?advancedsearch and counted. No attempt was made to find duplicates, or invalid CVEs for any of the projects. -Toshio From mastahnke at gmail.com Thu Nov 1 18:00:29 2007 From: mastahnke at gmail.com (Michael Stahnke) Date: Thu, 1 Nov 2007 13:00:29 -0500 Subject: Php why must your apps suck so? In-Reply-To: <472A0CD4.90401@gmail.com> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> <472A0CD4.90401@gmail.com> Message-ID: <7874d9dd0711011100r6c6563f8haac815a39ea52355@mail.gmail.com> > identifying and removing security problems? > > For #1, compare the number of CVEs_ in mediawiki to moin and drupal to > zope+plone: > 2007 2006 2005 > moin 5 0 0 > mediawiki 7 5 12 > > drupal 36 37 8 > zope(plone) 1(+0) 2(+3) 1(+0) > > Now we all know that numbers can be misleading but still this seems to > highlight something for me: there are projects which care about security > and there are projects which tack it on as an after thought. No matter > how much work we put into security locally (SELinux, mod_security, code > auditing), we don't want to be using a project which belongs to the > latter camp. *Sending security patches upstream doesn't help if > upstream will just introduce a new batch of security issues in their > next release.* Some of the numbers might have to do with install-base size also. I realize you did qualify your statment, but I thought it should be called out explicitly. I know of dozens of mediawiki sites I use nearly everyday, whereas moin, I know of one. Also, why is mediawiki ok for 108 and et.redhat.com but not for fedora? I would think some type of review/assesment was done for those sites. I am not trying to troll and/or flame, I really am just curious. stahnma From a.badger at gmail.com Thu Nov 1 18:31:31 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 01 Nov 2007 11:31:31 -0700 Subject: Php why must your apps suck so? In-Reply-To: <7874d9dd0711011100r6c6563f8haac815a39ea52355@mail.gmail.com> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> <472A0CD4.90401@gmail.com> <7874d9dd0711011100r6c6563f8haac815a39ea52355@mail.gmail.com> Message-ID: <472A1B83.9030301@gmail.com> Michael Stahnke wrote: >> identifying and removing security problems? >> >> For #1, compare the number of CVEs_ in mediawiki to moin and drupal to >> zope+plone: >> 2007 2006 2005 >> moin 5 0 0 >> mediawiki 7 5 12 >> >> drupal 36 37 8 >> zope(plone) 1(+0) 2(+3) 1(+0) >> > > >> Now we all know that numbers can be misleading but still this seems to >> highlight something for me: there are projects which care about security >> and there are projects which tack it on as an after thought. No matter >> how much work we put into security locally (SELinux, mod_security, code >> auditing), we don't want to be using a project which belongs to the >> latter camp. *Sending security patches upstream doesn't help if >> upstream will just introduce a new batch of security issues in their >> next release.* > > Some of the numbers might have to do with install-base size also. I > realize you did qualify your statment, but I thought it should be > called out explicitly. I know of dozens of mediawiki sites I use > nearly everyday, whereas moin, I know of one. Also, why is mediawiki > ok for 108 and et.redhat.com but not for fedora? I would think some > type of review/assesment was done for those sites. > The first sentence of my next paragraph is important here: ''' PS: Purely on the basis of these numbers I'd be led to believe that replacing moin with mediawiki would be acceptable. [...] ''' ;-) In my mind, I drew the line between drupal and the rest of the projects in that group. In plone+zope's worst year, it still had 7x less CVEs while mediawiki is pretty close to moin (1.4x). I didn't want to write it in the paragraph you quoted because making that judgement drags in install base (as you mention) which I don't have any numbers for. -Toshio From flatfender at gmail.com Thu Nov 1 19:24:06 2007 From: flatfender at gmail.com (Flatfender) Date: Thu, 1 Nov 2007 15:24:06 -0400 Subject: Php why must your apps suck so? In-Reply-To: <472A1B83.9030301@gmail.com> References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> <472A0CD4.90401@gmail.com> <7874d9dd0711011100r6c6563f8haac815a39ea52355@mail.gmail.com> <472A1B83.9030301@gmail.com> Message-ID: I've been lurking for awhile, but haven't thrown my hat into the ring for any projects yet. I'd be willing to help with Drupal or Mediawiki, both of which I run internally for my present employer. Matt Pusateri On 11/1/07, Toshio Kuratomi wrote: > Michael Stahnke wrote: > >> identifying and removing security problems? > >> > >> For #1, compare the number of CVEs_ in mediawiki to moin and drupal to > >> zope+plone: > >> 2007 2006 2005 > >> moin 5 0 0 > >> mediawiki 7 5 12 > >> > >> drupal 36 37 8 > >> zope(plone) 1(+0) 2(+3) 1(+0) > >> > > > > > >> Now we all know that numbers can be misleading but still this seems to > >> highlight something for me: there are projects which care about security > >> and there are projects which tack it on as an after thought. No matter > >> how much work we put into security locally (SELinux, mod_security, code > >> auditing), we don't want to be using a project which belongs to the > >> latter camp. *Sending security patches upstream doesn't help if > >> upstream will just introduce a new batch of security issues in their > >> next release.* > > > > Some of the numbers might have to do with install-base size also. I > > realize you did qualify your statment, but I thought it should be > > called out explicitly. I know of dozens of mediawiki sites I use > > nearly everyday, whereas moin, I know of one. Also, why is mediawiki > > ok for 108 and et.redhat.com but not for fedora? I would think some > > type of review/assesment was done for those sites. > > > > The first sentence of my next paragraph is important here: > ''' > PS: Purely on the basis of these numbers I'd be led to believe that > replacing moin with mediawiki would be acceptable. [...] > ''' > > ;-) > > In my mind, I drew the line between drupal and the rest of the projects > in that group. In plone+zope's worst year, it still had 7x less CVEs > while mediawiki is pretty close to moin (1.4x). I didn't want to write > it in the paragraph you quoted because making that judgement drags in > install base (as you mention) which I don't have any numbers for. > > -Toshio > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From rayvd at bludgeon.org Thu Nov 1 19:46:00 2007 From: rayvd at bludgeon.org (Ray Van Dolson) Date: Thu, 1 Nov 2007 12:46:00 -0700 Subject: Moin and notifications In-Reply-To: <20071018214258.GA26843@imperial.ac.uk> References: <4715137C.3010701@redhat.com> <200710162317.17470.opensource@till.name> <47152CA1.8070600@redhat.com> <47154107.6010301@gmail.com> <20071018214258.GA26843@imperial.ac.uk> Message-ID: <20071101194600.GA1183@bludgeon.org> On Thu, Oct 18, 2007 at 10:42:58PM +0100, Kostas Georgiou wrote: > > >It'd require patches to moin. IE: We'd either have to get rid of all of > > >the users or gut the mailing section of the wiki I guess. Just a > > >reminder to those that aren't familiar with it. Moin iterates over > > >every user who has page watch lists (currently in the thousands) to find > > >out who to notify. As the number of our users increases and as the > > >number of the pages they watch increase moin gets slower and slower on > > >page saves. > > > > > It would be doable though. When I looked ~ a year ago, there were only > > two places in the code that would need to be hacked to replace the email > > subsystem. I imagine we could find a single point in the code to hack > > if we just made finding the subscribed pages function return > > 'fedora-wiki-commits-list at r.c' instead of parsing each user's config > > file and returning that list. > > With a quick look at the code it seems quite simple to make the changes, > the only issue that I can see is if there are any pages that have acls > that don't allow all users to view them. > It would be fairly simple to fix -- or at least band-aid. I had done some initial testing and work on this, but unfortunately I moved in real life and work started taking up more time. A somewhat brief overview of the problem and profiler output of where the slowdowns are is listed here: http://wiki.bludgeon.org/FedoraProject/MoinOptimize At this point I don't know where upstream is at implementing fixes that might help this out. I hope to take a look at this again at some point, but someone else is welcome to as well. Seems like it would be much less work than switching to a completely different wiki platform. Although there is something to be said for the Moin team perhaps being focused on more small-scale sites whereas MediaWiki's dev team has to handle a site of much larger scale :) Ray From kwade at redhat.com Fri Nov 2 00:26:36 2007 From: kwade at redhat.com (Karsten Wade) Date: Thu, 01 Nov 2007 17:26:36 -0700 Subject: wiki madness Message-ID: <1193963197.4307.34.camel@erato.phig.org> So ... I'm seeing the comments fly around that replacing Moin Moin is the solution to solving our Wiki woes. For your consideration, I submit the following details. Thus we can all possess the same set of facts to work from. I'm not arguing for or against a tooling change. I just don't want Fedora Documentation to end up in a worse position. :) 1. Because of the long-standing Python bias of the Fedora Project, and the fact that Moin Moin is a popular and active Python-based wiki, Fedora Docs has long accepted this tool and adapted tools and practices to fit. Changing out this wiki engine is more than a technical exercise, and it affects content creation across the Project in the same way that replacing the buildsystem did for packaging. * Wiki formatting that can convert to DocBook -- processes * Wiki to DocBook conversions -- two GSoC projects; small tooling; processes * Release note creation process has small ties to Moin Moin way of doing things * Other WikiEditing processes that are tied to Moin Moin way of doing things 2. We have committed two Summer of Code internships to improving facets of Moin Moin for Fedora Docs, all this work happening upstream. However, this work has not been picked up entirely in the upstream tree, so hasn't made it into any releases. We have been suffering from not having a Python coder who can maintain the Moin Moin DocBook XML code. If we replace Moin Moin with another wiki, can we be guaranteed of Fedora Infrastructure support to replace those toolings? And do that in whatever language the replacement is written in? * Granted, some wiki communities are very active, so we might be able to grow a group of contributors around a useful plugin or two, where maybe we aren't seeing that in Moin Moin I guess the bottom line is around the challenges we've had in not being able to get code in the upstream, or having enough knowledge of the upstream to know when it is safe or proper to run the beta of the next release. I'd rather see Fedora Infrastructure or Websites commit to being involved more in the upstream, and fixing our problems there. If you decide that switching upstreams would do that, I reckon we'll need to get a deep requirements doc out of Fedora Documentation. - Karsten -- Karsten Wade, Developer Community Mgr. Dev Fu : http://developer.redhatmagazine.com Fedora : http://quaid.fedorapeople.org gpg key : AD0E0C41 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ricky at fedoraproject.org Fri Nov 2 02:21:18 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Thu, 1 Nov 2007 22:21:18 -0400 Subject: Meeting Log - 2007-11-01 Message-ID: <20071102022118.GE23471@Max.example.com> 15:55 * jima runs and hides 15:58 -!- fchiulli [i=824c4010 at gateway/web/cgi-irc/ircatwork.com/x-a6b81cd860eb6651] has joined #fedora-meeting 16:00 < jima> AIEEE NOOO 16:00 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Who's here? 16:00 < mmcgrath> Ping 16:00 < mmcgrath> who's around? 16:01 < markhsa> Mark H 16:01 < paulobanon_> me 16:01 < markhsa> me 16:01 < f13> I'm here 16:01 < fchiulli> fchiulli is listening 16:01 * jeremy is around-ish 16:01 * quaid is lurking 16:01 < bklemm> bklemm is listening in too 16:01 * jima imagines fchiulli and bklemm are new to irc :) 16:02 * jima 16:02 < mmcgrath> spevack: abadger1999 poelcat mbacovsk ricky ivazquez warren skvidal lmacken dgilmore: ping 16:02 < fchiulli> Give that man a cigar. 16:02 < mmcgrath> I know some of them won't be in today. 16:02 < skvidal> pong 16:02 < abadger1999> pong 16:02 < warren> pong 16:02 < poelcat> mmcgrath: pong 16:02 < jima> dgilmore was busy as heck earlier, not sure about now. 16:03 -!- giarc_w [i=hidden-u at gnat.asiscan.com] has joined #fedora-meeting 16:03 < lmacken> pong 16:03 < mmcgrath> Welp, lets get started then. 16:03 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Release is go! 16:03 < mmcgrath> .tiny https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?status=new&status=assigned&status=reopened&milestone=Fedora+8 16:03 < mmcgrathbot> mmcgrath: http://tinyurl.com/292qq8 16:04 < mmcgrath> So yeah, AFAIK we will release Fedora 8 on the 8th next week. 16:04 < mmcgrath> Is anyone confused about what they should or shouldn't be doing at this point? 16:04 < jima> should: looking for bugs 16:04 < spoleeba> mmcgrath, duck and cover? 16:04 < warren> meh 16:04 < jima> shouldn't: changing infrastructure without asking :) 16:05 < skvidal> jima: wait? what? 16:05 < skvidal> should NOT 16:05 < jima> spoleeba: i thought we did that every day ;) 16:05 < skvidal> oh shit 16:05 < abadger1999> :-) 16:05 * mmcgrath smacks skvidal 16:05 -!- nim-nim [n=nim-nim at fedora/nim-nim] has quit "Leaving." 16:05 < skvidal> mmcgrath: hey, I wasn't the one who brought proxy4 online yesterday! :) 16:05 < mmcgrath> Just a reminder, ANOTHER one :) After tonight, no changes to infrastructure without contacting the list first. 16:05 < jima> skvidal: ...? 16:05 < skvidal> jima: I'm making fun of mmcgrath 16:06 < mmcgrath> jesse also had a wish to extend this period for next release. I'm generally in favor of that though I'd like to see hwo this change freeze goes. 16:06 * dgilmore is sitting on the cold floor of the colo 16:06 < mmcgrath> There are a few critical things I'd like to talk about. 16:06 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure - mmcgrath (schedule) 16:06 < jima> hmm, should i be getting approval to work on ticket #220? :) 16:06 < mmcgrath> jima: lets get to that in a bit 16:06 < jima> 'k 16:06 -!- nim-nim [n=nim-nim at fedora/nim-nim] has joined #fedora-meeting 16:07 < mmcgrath> So, if I've done my job right, I won't need to be around for this release. Having said that. I'm going to try to be. 16:07 < paulobanon_> no changes no fun! 16:07 < mmcgrath> skvidal is going to be the point person during the release, even if I am around. 16:07 < skvidal> mmcgrath: do not be around 16:07 < skvidal> seriously - don't do it 16:07 < mmcgrath> :) 16:07 < jima> skvidal: see, you shouldn't have made fun of him -- he just got payback. 16:07 < skvidal> we'll deal w/o you :) 16:07 < mmcgrath> So here's my schedule. 16:07 < skvidal> jima: I [was] volunteered 16:08 < mmcgrath> I'm off tomorrow and will largely be unavailable until Monday. 16:08 < jima> skvidal: exactly my point. ;) 16:08 < jima> mmcgrath: have fun! :) 16:08 < mmcgrath> I'll be taking half days Monday - Thursday (release day) and on the 9th I'll be gone. 16:08 < mmcgrath> just gone. Like I wasn't ever here. 16:08 < jima> damn ninja. 16:08 < mmcgrath> and I'll be back for good on the 13th. 16:08 < jeremy> who's mmcgrath? 16:08 < skvidal> mmcgrath: wow, like keyser sose 16:08 < mmcgrath> Hopefully with a tan. Though its likely I'll burst into flames the first time I come in contact with the sun. 16:09 < skvidal> KEYSER SOSE KEYSER SOSE! 16:09 < mmcgrath> :) 16:09 < jima> i think anything bursts into flame when it comes even *close* to the sun. 16:09 < mmcgrath> So mark it down and remember that we talked about it in the meeting. 16:09 < mmcgrath> Next topic! 16:09 < paulobanon_> wedding topic ?! 16:09 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Current nown oddities. 16:09 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Current known oddities. 16:09 < mmcgrath> damn 16:09 < mmcgrath> Ok. 16:09 < jima> known oddity: "nown" is missing a 'k' 16:09 < skvidal> so, next week after we remove all of mmcgrath's accounts what will we be doing 16:09 < skvidal> ? 16:09 < skvidal> oh 16:09 < skvidal> damn 16:10 < skvidal> wrong channel 16:10 < jima> skvidal: that was for #fedora-evil, huh? 16:10 < mmcgrath> heh 16:10 < skvidal> it was 16:10 < mmcgrath> https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/222 16:10 < mmcgrath> .ticket 222 16:10 < mmcgrathbot> mmcgrath: #222 (sysctl on the proxy servers) - Fedora Infrastructure - Trac 16:10 < mmcgrath> I should have had this done a week ago but testing it and being busy with other stuff put it off. 16:10 < mmcgrath> Long story short. 16:10 -!- stahnma_ [n=stahnma at c-76-18-178-254.hsd1.tn.comcast.net] has joined #fedora-meeting 16:10 < yingbull> I should probably say hi - I was around earlier in the fall, but got a bad case of the $DAYJOB, and had fedora account issues that slowed me down getting involved (and without anything tasked to me yet, I just sorta faded off). Back around now, tho. So... hi. ;-) 16:10 < mmcgrath> The sysctl configuration optimizations have been done on all the proxy servers. 16:10 < mmcgrath> yingbull: welcome! 16:11 < yingbull> made it to this meeting at least, hopefully its a good start. :-) Sorry to interrupt. 16:11 < mmcgrath> So here's the scoop, we've got optimized proxy servers. I've done all the testing I can to ensure that we are getting peak performance from them. 16:11 * jima waves to yingbull 16:11 < jima> mmcgrath: seems like a clever idea, yeah. 16:11 < jima> is that all the proxies or just 1-2? 16:11 < mmcgrath> Having said that, I don't feel they're tested enough to just deploy. So to mitigate this risk my *optimized* config is in /root/sysctl.conf and the normal config thats been deployed for many months is in /etc/sysctl.conf 16:11 < mmcgrath> jima: all of them. 16:12 < mmcgrath> So if anyone sees the proxy servers getting overwhelmed, and feel it might be tuned wrong, just run "sysctl -p" 16:12 < mmcgrath> if you want to go back to the 'optimized configs' run "sysctl -p /root/sysctl.conf" 16:12 < mmcgrath> profit. 16:12 * jima almosts asks "where's a copy of the optimized one in puppet?" then remembers mmcgrath provided us with a copy. 16:12 < mmcgrath> Thats an odditiy I thought I'd mention. 16:12 -!- clarkbw [i=clarkbw at nat/redhat/x-ae3597e3eeab15df] has joined #fedora-meeting 16:12 < mmcgrath> since it doesn't make any obvious sense. 16:13 -!- londo [n=georgiou at 82.133.49.59] has quit Read error: 104 (Connection reset by peer) 16:13 -!- clarkbw [i=clarkbw at nat/redhat/x-ae3597e3eeab15df] has quit Read error: 104 (Connection reset by peer) 16:13 < mmcgrath> Anyone have any questions on that? Did anyone see anything obviously wrong with the sysctl.conf I sent to the list? 16:13 < lmacken> looked sane to me 16:13 < jeremy> seem fine to me 16:13 < mmcgrath> ok, moving on. 16:14 < mmcgrath> Oddity number 2. 16:14 -!- rdieter is now known as rdieter_away 16:14 < mmcgrath> The wiki notifications today. 16:14 < mmcgrath> We've been seeing higher than normal wiki notifications as of today. 16:14 < mmcgrath> I don't think the wiki is any worse off today then it was yesterday. I think its because we've been monitoring the wiki with caching enabled and I recently made that cache more aggressive than normal. 16:15 < mmcgrath> The notifications we're getting are from the only non-cached page. 16:15 < mmcgrath> Which means A) the wiki is unresponsive at times. 16:15 < mmcgrath> but B) I've not witnessed this myself, only seen the nagios alerts. 16:16 < mmcgrath> I'm watching this closely and trying to tune the wiki a bit but so far nothing has made the load any better. 16:16 < mmcgrath> Long story short, the caching I added sees between a 50% and 75% hit rate. 16:16 < jima> that's good. 16:16 < mmcgrath> that means only 50% - 25% of the requests actually hit the app server. 16:16 < mmcgrath> this is something we did not have for the F7 release. 16:16 -!- clarkbw [i=clarkbw at nat/redhat/x-29205887cfe9d537] has joined #fedora-meeting 16:16 < mmcgrath> The bulk of our work will be done by the proxy servers this time around. 16:17 < jima> but god help the actual wiki for the hits that do get through. :| 16:17 < mmcgrath> of which we have 4, the 2 new ones have more ram and are x86_64 boxes. So I think we're in good shape. 16:17 < mmcgrath> The wiki, as always, concerns me. 16:17 -!- k0k [n=k0k at fedora/k0k] has quit Read error: 110 (Connection timed out) 16:17 < mmcgrath> It just does. 16:17 * jima concurs 16:17 < mmcgrath> I've moved app3 off of xen1 and on to xen2 to dedicate more resources to xen1 (which currently contains the wiki and one proxy server) 16:17 < jeremy> we should probably just be aggressive about making "important" pages static and for the rest, if the wiki goes down, the wiki goes down 16:17 < poelcat> skvidal: is the problems I had yesterday app1 and with updating wiki pages is fixed? 16:17 < mmcgrath> unfortunately thats the best setup I've found so far. 16:18 < skvidal> poelcat: for certain values of fixed, yes 16:18 < mmcgrath> Any questions about the wiki right now? 16:19 < jima> mmcgrath: yes, but my questions keep timing out. 16:19 < mmcgrath> I've also turned off the translate app for app1 for now. 16:19 * mmcgrath smacks jima. 16:19 < mmcgrath> especially over the next many days. If you have ANY problems with the wiki, please email me the time, what you were trying to do and your IP address. 16:20 < mmcgrath> Ok, moving on. 16:20 < mmcgrath> mdomsch is not around but all that is left on that end is to disable the primary mirrors initially (download.fedora.redhat.com) from the mirror list and public list. 16:20 * jima scratches head 16:21 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Specific tickets open. 16:21 < jima> disable them from mirrorlist? isn't that tweakable via mirrormanager? 16:21 < mmcgrath> .ticket 93 16:21 < mmcgrathbot> mmcgrath: #93 (Fedora 8) - Fedora Infrastructure - Trac 16:21 < mmcgrath> We'll close that when everything is done 16:21 < mmcgrath> .ticket 97 16:21 < mmcgrathbot> mmcgrath: #97 (Fedora 8 Website) - Fedora Infrastructure - Trac 16:21 < mmcgrath> Ricky and I have been working on this together, expect to see the F8 website by the end of the day but with the F7 graphics and such. 16:21 < mmcgrath> We'll have a special build for F8 for the mirrors by the end of the day on Monday. 16:21 < mmcgrath> .ticket 99 16:21 < mmcgrathbot> mmcgrath: #99 (Mirror Coordination) - Fedora Infrastructure - Trac 16:22 < mmcgrath> Currently assigned to mdomsch either he or jesse keating will likely notify the mirrors when the bits are ready. 16:22 < mmcgrath> .ticket 100 16:22 < mmcgrathbot> mmcgrath: #100 (Web Content on the mirrors) - Fedora Infrastructure - Trac 16:22 < mmcgrath> already discussed, this will be done with ticket #97 16:22 < mmcgrath> .ticket 101 16:22 < mmcgrathbot> mmcgrath: #101 (Fedora Infrastructure Change Freeze) - Fedora Infrastructure - Trac 16:22 < mmcgrath> I'll close this ticket AFTER tonight. 16:22 < mmcgrath> ticket .137 16:22 < mmcgrath> .ticket 137 16:22 < mmcgrath> :) 16:22 < mmcgrathbot> mmcgrath: #137 (Sponsorship prominence) - Fedora Infrastructure - Trac 16:23 < mmcgrath> This is actually in good shape and live. Its linked at the bottom of the page http://fedoraproject.org/sponsors 16:23 < mmcgrath> .ticket 210 16:23 < mmcgrathbot> mmcgrath: #210 (Disable F8 in Mirrormanager) - Fedora Infrastructure - Trac 16:23 < notting> g 16:24 < mmcgrath> This is also currently assigned to mdomsch (who's not here) but I'll follow up with him to make sure this is done correctly when the bits go 16:24 < mmcgrath> .ticket 211 16:24 < mmcgrathbot> mmcgrath: #211 (Enable Bodhi Updates) - Fedora Infrastructure - Trac 16:24 < mmcgrath> lmacken: ? 16:24 < mmcgrath> This is assigned to you, still all good? 16:24 < lmacken> shouldn't be a problem 16:24 < mmcgrath> any ETA? 16:25 < lmacken> it's just a bitflip.. whenever we want to unleash 16:25 -!- stahnma_ is now known as stahnma 16:25 < mmcgrath> k, I'll leave that up to you and releng then :) 16:25 < mmcgrath> .ticket 54 16:25 < mmcgrathbot> mmcgrath: #54 (Postfix Server) - Fedora Infrastructure - Trac 16:25 < mmcgrath> This, unfortunately, will probably be the only ticket we didn't completely finish. Mostly an -ENOTIME and a -EALREADYHAVESOMETHING 16:26 < mmcgrath> bastion's been fine, this was a bigger priority when the mailman list stuff was being discussed and we had OTRS. 16:26 < mmcgrath> and the last ticket 16:26 < mmcgrath> .ticket 136 16:26 < mmcgrathbot> mmcgrath: #136 (Translated Website) - Fedora Infrastructure - Trac 16:26 < mmcgrath> We've 'frozen' the website and the translation team is currently at work on it 16:26 < jima> yay for translators! 16:26 * mmcgrath notes 16:26 < mmcgrath> http://translate.fedoraproject.org/module/fedora-web 16:27 < mmcgrath> I think this is actually a big deal. 16:27 < mmcgrath> but the final 'freeze' will be on monday night. 16:27 < mmcgrath> So really, we're in good shape. 16:27 < mmcgrath> Hopefully we won't have to juggle much at release time. 16:27 < mmcgrath> Anyone have any questions or concerns regarding me, our infrastructure, stuff not staying up or getting overloaded? 16:28 < mmcgrath> skvidal: you've got to have questions about stuff right? 16:29 < skvidal> mostly i'm reading 16:29 -!- EvilBob [n=bob at fedora/pdpc.sustaining.BobJensen] has quit Remote closed the connection 16:29 * mmcgrath also notes http://fedoraproject.org/wiki/Infrastructure/SOP/Release 16:29 < mmcgrath> still exists and is valid. 16:29 < skvidal> but right now my primary thoughts are making sure we have all the steps down for dynamically making static pages when we need them 16:29 < mmcgrath> skvidal: we should make an SOP for that. 16:30 < mmcgrath> skvidal: OH, thats actually a good point. 16:30 < skvidal> but we can't get to the SOP :) 16:30 < skvidal> so it's better just to have a note 16:30 < skvidal> so i'm thinking 16:30 < mmcgrath> *if* the wiki gets overloaded and you can't get to it, log into RH VPN and go to app2.fedora.phx.redhat.com/wiki/ 16:30 < mmcgrath> since we can't balance the wiki its always on one or the other, just go to the other :) 16:30 < skvidal> 1. save the page in firefox 16:30 < mmcgrath> skvidal: do it with wget. 16:30 < skvidal> 2. upload the files to the static path on the proxies 16:31 < skvidal> wget, in the past, didn't grab css for me 16:31 < mmcgrath> hmm, k, we can test that after the meeting. 16:31 < skvidal> okie doke 16:31 < mmcgrath> I'd also encourage everyone to become familiar with our caching setup. especially the headers. 16:32 < mmcgrath> .headers http://fedoraproject.org/wiki/ 16:32 < mmcgrathbot> mmcgrath: apptime: D=77335, proxytime: D=167438, expires: Thu, 01 Nov 2007 21:32:01 GMT, vary: Cookie,User-Agent,Accept-Language, connection: close, server: Apache/2.2.3 (Red Hat), appserver: (null), proxyserver: proxy3.fedoraproject.org, date: Thu, 01 Nov 2007 20:32:01 GMT, content-type: text/html; charset=utf-8 16:32 * mmcgrath notes proxy and app time 16:32 < mmcgrath> .headers http://fedoraproject.org/wiki/ 16:32 < mmcgrathbot> mmcgrath: apptime: D=77335, content-length: 13644, proxytime: D=87, expires: Thu, 01 Nov 2007 21:32:01 GMT, vary: Cookie,User-Agent,Accept-Language, connection: close, server: Apache/2.2.3 (Red Hat), appserver: (null), proxyserver: (null), date: Thu, 01 Nov 2007 20:32:15 GMT, content-type: text/html; charset=utf-8, age: 15 16:32 * mmcgrath notes "age" this time. 16:32 < mmcgrath> stuff like that is important for understanding why USER A) gets cached copy and why user B) doesn't. 16:33 < mmcgrath> If you want to flush all the cache, just "rm -rf /srv/web/cache/*" on the proxy box. 16:33 -!- londo [n=georgiou at 82.133.49.59] has joined #fedora-meeting 16:33 < mmcgrath> blah blah. 16:33 < mmcgrath> Ok, if no one has any questions about anything else I'll open the floor? 16:33 < skvidal> mmcgrath: one more question 16:33 < skvidal> on the proxy boxes 16:33 < skvidal> we have /srv/web/static-web 16:34 < skvidal> is that path old/broken? 16:34 < mmcgrath> That is the old path. 16:34 * mmcgrath notes proxy4 16:34 < mmcgrath> it doesn't have it. 16:34 < skvidal> right - I was more thinking of cleaning that up 16:34 < mmcgrath> 16:34 < mmcgrath> I thought I'd moved that to /tmp/ 16:35 < mmcgrath> perhaps I moved it back during the conversion. 16:35 < mmcgrath> but yeah, that should now be "/srv/web/fedoraproject.org/" or "/srv/web/start.fedoraproject.org/" 16:35 < skvidal> looks like you copied it to temp :) 16:35 < skvidal> but it never got nuked 16:35 * mmcgrath also notes that AFAIK, search.fedoraproejct.org won't be ready. 16:36 < mmcgrath> but thats not really our concern, just pointing it out. 16:36 < yingbull> my bit: I'll want something to do soon. I need to fix my fedora account, but will want something to work on to get started. Else I'm likely to let $DAYJOB eat my $NIGHTLIFE, and without anything depending/waiting on me, fade away again. :-) Anyone got something that I can thing them about once my fedora account is working? 16:36 < mmcgrath> yingbull: how's your mysql skills? 16:36 -!- clarkbw [i=clarkbw at nat/redhat/x-29205887cfe9d537] has quit Read error: 110 (Connection timed out) 16:37 < yingbull> mmcgrath: not bad, haven't had a huge need for it lately, but I did work with it before. 16:37 < yingbull> Used it for a big web-app at the last place I worked. 16:37 < mmcgrath> yingbull: ping me after the meeting in #fedora-admin, we'll find something definitive for you. 16:37 < yingbull> Will do. Save me from $DAYJOB, please! :-) 16:37 < mmcgrath> heh 16:37 < mmcgrath> ok 16:37 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor 16:38 < mmcgrath> Does anyone have anything specific they'd like to discuss? 16:38 < mmcgrath> I'd encourage everyone to start thinking of F9 goals. 16:38 * jima muses 16:39 < mmcgrath> Anyone have anything else? If not we'll close the meeting. 16:39 < jima> mmcgrath: ticket 220? 16:40 < mmcgrath> .ticket 220 16:40 < mmcgrathbot> mmcgrath: #220 (nagios monitoring for puppet1) - Fedora Infrastructure - Trac 16:40 < mmcgrath> jima: ahh, yeah I just wanted to make sure it was beign monitored. 16:40 < jima> it's not. next question? :) 16:41 < mmcgrath> well, can we make it monitored? ICMP, disk, puppetmaster, http. 16:41 < mmcgrath> standard stuff. 16:41 < jima> is it running nrpe? 16:41 < mmcgrath> it should be. 16:41 < jima> yep, it is. 16:42 < jima> should i make those changes to nagios? no conflicts with the infrastructure freeze? 16:42 < mmcgrath> jima: I'd put that as a low priority but if you can't get it done tonight, I'd wait until after the release. 16:42 < jima> 'k. 16:42 < nim-nim> ticket 215 anyone? 16:42 < mmcgrath> .ticket 215 16:42 < mmcgrathbot> mmcgrath: #215 (New mailing list request: fedora-fonts-bugs-list) - Fedora Infrastructure - Trac 16:42 < mmcgrath> nim-nim: that one's on me. I'll put that request in now. 16:43 < nim-nim> mmcgrath: thanks! 16:43 * mmcgrath has been a little busy :) 16:43 * nim-nim understands 16:43 < mmcgrath> I'd also like to get the rest of the hosted repos created that are in the queue by tonight. 16:43 < nim-nim> still, nim-nim needs to get his SIG in shape too 16:44 < nim-nim> many thanks! 16:44 -!- JSchmitt [n=s4504kr at fedora/JSchmitt] has quit "Konversation terminated!" 16:44 < mmcgrath> Ok, anyone have anything else? 16:45 < mmcgrath> If not I'll close the meeting in 30. 16:45 < mmcgrath> 15 16:45 < mmcgrath> 5 16:45 < mmcgrath> Good luck to us all next week :) 16:46 -!- mmcgrath changed the topic of #fedora-meeting to: Infrasturcture -- Meeting Closed. 16:46 -!- fchiulli [i=824c4010 at gateway/web/cgi-irc/ircatwork.com/x-a6b81cd860eb6651] has left #fedora-meeting [] 16:46 < mmcgrath> Thanks for coming everyone. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From ivazqueznet at gmail.com Fri Nov 2 04:49:31 2007 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Fri, 02 Nov 2007 00:49:31 -0400 Subject: wiki madness In-Reply-To: <1193963197.4307.34.camel@erato.phig.org> References: <1193963197.4307.34.camel@erato.phig.org> Message-ID: <1193978971.3271.21.camel@ignacio.lan> On Thu, 2007-11-01 at 17:26 -0700, Karsten Wade wrote: > So ... I'm seeing the comments fly around that replacing Moin Moin is > the solution to solving our Wiki woes. For your consideration, I submit > the following details. Thus we can all possess the same set of facts to > work from. I'm not arguing for or against a tooling change. I just > don't want Fedora Documentation to end up in a worse position. :) Absolutely. We still haven't decided on a plan of attack though; it's probably best to wait until we've settled down after F8's release before making any big plans. As I see it, we have 3 options: 1) Continue to work with Moin 2) Migrate to another wiki 3) Roll our own TurboGears wiki Option 1: Continue to work with Moin Advantages: - Existing code base - Existing developer base - No need to transcribe content Disadvantages: - Upstream seems uncooperative Option 2: Migrate to another wiki Advantages: - Existing code base - Existing developer base - Current problems with Moin may not exist Disadvantages: - Content may need to be transcribed - Workflow/process may differ - Existing GSoC work may be useless Unknowns: - Attitude of upstream Option 3: Roll our own TurboGears wiki Advantages: - We're in control (workflow, features, upstream, etc.) - Integration with FAS is trivial Disadvantages: - Almost no current specifications - Minimal existing code (based on requirements) - Smaller developer base - Content many need to be transcribed (based on requirements) - GSoC work may need retooling (based on requirements) If I've left anything out then please feel free to add. -- Ignacio Vazquez-Abrams PLEASE don't CC me; I'm already subscribed -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From cra at WPI.EDU Fri Nov 2 05:06:06 2007 From: cra at WPI.EDU (Chuck Anderson) Date: Fri, 2 Nov 2007 01:06:06 -0400 Subject: wiki madness In-Reply-To: <1193978971.3271.21.camel@ignacio.lan> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> Message-ID: <20071102050606.GN25455@angus.ind.WPI.EDU> On Fri, Nov 02, 2007 at 12:49:31AM -0400, Ignacio Vazquez-Abrams wrote: > Option 3: Roll our own TurboGears wiki > > Advantages: > - We're in control (workflow, features, upstream, etc.) > - Integration with FAS is trivial > > Disadvantages: > - Almost no current specifications > - Minimal existing code (based on requirements) > - Smaller developer base > - Content many need to be transcribed (based on requirements) > - GSoC work may need retooling (based on requirements) Won't there be performance problems with a TurboGears-based wiki? I thought MirrorManager was having issues with TG performance and had to enable form-data caching to get acceptable performance at the cost of possibly stale data. I don't know the details behind it, but that was the reason I was given for why when you edit forms in MM it sometimes returns old pre-edit field values. From fedora at leemhuis.info Fri Nov 2 06:00:42 2007 From: fedora at leemhuis.info (Thorsten Leemhuis) Date: Fri, 02 Nov 2007 07:00:42 +0100 Subject: wiki madness In-Reply-To: <1193963197.4307.34.camel@erato.phig.org> References: <1193963197.4307.34.camel@erato.phig.org> Message-ID: <472ABD0A.3050200@leemhuis.info> On 02.11.2007 01:26, Karsten Wade wrote: > So ... I'm seeing the comments fly around that replacing Moin Moin is > the solution to solving our Wiki woes. For your consideration, I submit > the following details. Thus we can all possess the same set of facts to > work from. I'm not arguing for or against a tooling change. I just > don't want Fedora Documentation to end up in a worse position. :) Here are my 2 cent: Lot's of our contributors (Docs, Packaging, ...) are used to moin moin and know how it works. We want them to improve our docs and help in the wiki -- by changing the wiki we force them to learn something new and that creates a non-trivial "entry" burden, which I think we should avoid the best we can if there isn't a good reasons. Sure, most easy stuff will work similar in another wiki, but there is some more advanced stuff that won't. IOW: Work around the problem (by disabling individual subscriptions and instead sending everything to a mailinglist which people can subscribe to and filter) is of course some work for someone (?), but it's likely a lot less work in total then for hundreds of people to learn a different wiki. CU knurd (?) -- no, I'm not volunteering. We all have different areas of the project where we work on, and I have enough on my todo-list already for those areas. In fact I can live with the current situation. From kwade at redhat.com Fri Nov 2 06:42:49 2007 From: kwade at redhat.com (Karsten Wade) Date: Thu, 01 Nov 2007 23:42:49 -0700 Subject: wiki madness In-Reply-To: <1193978971.3271.21.camel@ignacio.lan> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> Message-ID: <1193985769.4307.100.camel@erato.phig.org> On Fri, 2007-11-02 at 00:49 -0400, Ignacio Vazquez-Abrams wrote: > Option 1: Continue to work with Moin > ... > Disadvantages: > - Upstream seems uncooperative Sorry, I worded my email poorly; I meant to go back over that part and make sure I wasn't unintentionally maligning the Moin Moin project. They're very cooperative, afaic, but they have resources with specific skills and interests. Just because we want to shoe-horn in better DocBook support and man page editing features doesn't mean they *should* accept it. My point is that Fedora Project could find a way to step-up more with contributing to the Moin Project, as part of the method of getting our changes accepted (and knowing when and where to run beta versions, how to keep our one-off changes in sync with the tree, etc.) MediaWiki has Wikipedia as a main constituent user that drives some features, right? Who does that for Moin? Could that be fedoraproject.org/wiki? Or are we really not that big of an install/userbase? - Karsten -- Karsten Wade, Developer Community Mgr. Dev Fu : http://developer.redhatmagazine.com Fedora : http://quaid.fedorapeople.org gpg key : AD0E0C41 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From ivazqueznet at gmail.com Fri Nov 2 07:51:51 2007 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Fri, 02 Nov 2007 03:51:51 -0400 Subject: wiki madness In-Reply-To: <1193985769.4307.100.camel@erato.phig.org> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <1193985769.4307.100.camel@erato.phig.org> Message-ID: <1193989911.3271.29.camel@ignacio.lan> On Thu, 2007-11-01 at 23:42 -0700, Karsten Wade wrote: > On Fri, 2007-11-02 at 00:49 -0400, Ignacio Vazquez-Abrams wrote: > > > Option 1: Continue to work with Moin > > > ... > > > Disadvantages: > > - Upstream seems uncooperative > > Sorry, I worded my email poorly; I meant to go back over that part and > make sure I wasn't unintentionally maligning the Moin Moin project. It's not what you wrote that led me to write that. Rather, it's a general feeling I've been picking up on when inquiring about Moin's specific problems. > They're very cooperative, afaic, but they have resources with specific > skills and interests. Just because we want to shoe-horn in better > DocBook support and man page editing features doesn't mean they *should* > accept it. My concern is that if they're unsympathetic regarding the minor changes we need (e.g., ACLs), how are they going to react to the larger changes we're going to come up with (e.g., user DB in a RDBMS)? If we're going to have to fork Moin *anyways*, then why don't we start with a specification appropriate for our needs in the first place? > My point is that Fedora Project could find a way to step-up more with > contributing to the Moin Project, as part of the method of getting our > changes accepted (and knowing when and where to run beta versions, how > to keep our one-off changes in sync with the tree, etc.) > > MediaWiki has Wikipedia as a main constituent user that drives some > features, right? Who does that for Moin? Could that be > fedoraproject.org/wiki? Or are we really not that big of an > install/userbase? Fedora has one of if not the biggest Moin installations in existence. And it's collapsing under our weight. I'm sure that Moin's developers have their reasons for doing things the way that they do, and I respect them for that, but if it's not working out for us then what are we supposed to do? -- Ignacio Vazquez-Abrams PLEASE don't CC me; I'm already subscribed -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From vpivaini at cs.helsinki.fi Fri Nov 2 09:25:03 2007 From: vpivaini at cs.helsinki.fi (Ville-Pekka Vainio) Date: Fri, 2 Nov 2007 11:25:03 +0200 Subject: wiki madness In-Reply-To: <1193978971.3271.21.camel@ignacio.lan> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> Message-ID: <200711021125.03452.vpivaini@cs.helsinki.fi> Ignacio Vazquez-Abrams wrote: > Option 1: Continue to work with Moin > ... > Disadvantages: > - Upstream seems uncooperative I wouldn't actually say so. The thing is, they have quite a small core team who have been working for quite a while to get 1.6 released. They just don't have the manpower at this point to fix our issues, especially when a lot of Moin users never face the same problems our wiki has. In my experience they are cooperative and interested - they have for example offered ideas and possible solutions to the subscription problem - but if we want fixes soon, we need to do them ourselves. -- Ville-Pekka Vainio From jkeating at redhat.com Fri Nov 2 10:23:11 2007 From: jkeating at redhat.com (Jesse Keating) Date: Fri, 2 Nov 2007 06:23:11 -0400 Subject: Another Xen guest lockup Message-ID: <20071102062311.5c19fdd0@redhat.com> At some point xen4 went down last night. 22:37 koji time was the last time the guest running on it (xenbuilder2) checked in to koji. It was building sbcl, which I think was the thing that was building last time a builder went down. Perhaps this package is triggering something bad? -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From jkeating at redhat.com Fri Nov 2 10:58:51 2007 From: jkeating at redhat.com (Jesse Keating) Date: Fri, 2 Nov 2007 06:58:51 -0400 Subject: Another Xen guest lockup In-Reply-To: <20071102062311.5c19fdd0@redhat.com> References: <20071102062311.5c19fdd0@redhat.com> Message-ID: <20071102065851.24945c77@redhat.com> On Fri, 2 Nov 2007 06:23:11 -0400 Jesse Keating wrote: > At some point xen4 went down last night. 22:37 koji time was the last > time the guest running on it (xenbuilder2) checked in to koji. It was > building sbcl, which I think was the thing that was building last time > a builder went down. Perhaps this package is triggering something > bad? Looking at this build... the x86_64 buildArch task used a number of buildroots: xenbuilder4 at 04:57 xenbuilder1 at 13:50 xenbuilder4 at 14:15 xenbuilder2 at 14:40 xenbuilder2 at 14:57 xenbuilder2 at 15:09 xenbuilder2 at 16:25 xenbuilder2 at 22:28 xenbuilder2 at 03:20 the next day. Do these match up to the xen failures we were seeing yesterday? If so, I think the blame completely rests with the x86_64 build of sbcl and really should be investigated. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From Matt_Domsch at dell.com Fri Nov 2 13:40:12 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 08:40:12 -0500 Subject: wiki madness In-Reply-To: <20071102050606.GN25455@angus.ind.WPI.EDU> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> Message-ID: <20071102134012.GA16220@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 01:06:06AM -0400, Chuck Anderson wrote: > On Fri, Nov 02, 2007 at 12:49:31AM -0400, Ignacio Vazquez-Abrams wrote: > > Option 3: Roll our own TurboGears wiki > > > > Advantages: > > - We're in control (workflow, features, upstream, etc.) > > - Integration with FAS is trivial > > > > Disadvantages: > > - Almost no current specifications > > - Minimal existing code (based on requirements) > > - Smaller developer base > > - Content many need to be transcribed (based on requirements) > > - GSoC work may need retooling (based on requirements) > > Won't there be performance problems with a TurboGears-based wiki? I > thought MirrorManager was having issues with TG performance and had to > enable form-data caching to get acceptable performance at the cost of > possibly stale data. I don't know the details behind it, but that was > the reason I was given for why when you edit forms in MM it sometimes > returns old pre-edit field values. correct. Now, my TG-foo may not strong enough to overcome this, but perhaps it's easier for others? I'm still not convinced to throw the baby out with the bathwater. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From sundaram at fedoraproject.org Fri Nov 2 14:13:20 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Fri, 02 Nov 2007 19:43:20 +0530 Subject: wiki madness In-Reply-To: <1193978971.3271.21.camel@ignacio.lan> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> Message-ID: <472B3080.5090905@fedoraproject.org> Ignacio Vazquez-Abrams wrote: > On Thu, 2007-11-01 at 17:26 -0700, Karsten Wade wrote: >> So ... I'm seeing the comments fly around that replacing Moin Moin is >> the solution to solving our Wiki woes. For your consideration, I submit >> the following details. Thus we can all possess the same set of facts to >> work from. I'm not arguing for or against a tooling change. I just >> don't want Fedora Documentation to end up in a worse position. :) > > Absolutely. We still haven't decided on a plan of attack though; it's > probably best to wait until we've settled down after F8's release before > making any big plans. > > As I see it, we have 3 options: > > 1) Continue to work with Moin > 2) Migrate to another wiki > 3) Roll our own TurboGears wiki > 4) Fork Moin. Rahul From sundaram at fedoraproject.org Fri Nov 2 14:17:04 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Fri, 02 Nov 2007 19:47:04 +0530 Subject: Php why must your apps suck so? In-Reply-To: References: <471E52B7.9070503@redhat.com> <7a41c4bc0710231313n7276a88ayd831153c98608bfd@mail.gmail.com> <471FC94D.6080406@gmail.com> <1193843131.5314.16.camel@erato.phig.org> <1193869479.28356.8.camel@ignacio.lan> <7874d9dd0711010559r1ef80f7ao91f243aada72ea58@mail.gmail.com> <472A0CD4.90401@gmail.com> <7874d9dd0711011100r6c6563f8haac815a39ea52355@mail.gmail.com> <472A1B83.9030301@gmail.com> Message-ID: <472B3160.3020801@fedoraproject.org> Flatfender wrote: > I've been lurking for awhile, but haven't thrown my hat into the ring > for any projects yet. I'd be willing to help with Drupal or > Mediawiki, both of which I run internally for my present employer. > > Matt Pusateri Drupal is being evaluated for news.fedoraproject.org. Hop in #fedora-admin and ask for details if you are interested in helping out. Rahul From mmcgrath at redhat.com Fri Nov 2 14:54:50 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Fri, 02 Nov 2007 09:54:50 -0500 Subject: [Fwd: ** PROBLEM alert - app1.fedora.phx.redhat.com/Puppet is CRITICAL **] Message-ID: <472B3A3A.1020001@redhat.com> I had disabled this yesterday and forgot to re-enable it. If no one has any complaints, can someone re-enable it this afternoon? (change freeze and all...) -Mike -------------- next part -------------- An embedded message was scrubbed... From: Nagios Monitoring User Subject: ** PROBLEM alert - app1.fedora.phx.redhat.com/Puppet is CRITICAL ** Date: Fri, 2 Nov 2007 07:48:33 -0700 Size: 2110 URL: From mmcgrath at redhat.com Fri Nov 2 15:26:54 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Fri, 02 Nov 2007 10:26:54 -0500 Subject: wiki madness In-Reply-To: <20071102050606.GN25455@angus.ind.WPI.EDU> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> Message-ID: <472B41BE.7000206@redhat.com> Chuck Anderson wrote: > On Fri, Nov 02, 2007 at 12:49:31AM -0400, Ignacio Vazquez-Abrams wrote: > >> Option 3: Roll our own TurboGears wiki >> >> Advantages: >> - We're in control (workflow, features, upstream, etc.) >> - Integration with FAS is trivial >> >> Disadvantages: >> - Almost no current specifications >> - Minimal existing code (based on requirements) >> - Smaller developer base >> - Content many need to be transcribed (based on requirements) >> - GSoC work may need retooling (based on requirements) >> > > Won't there be performance problems with a TurboGears-based wiki? I > thought MirrorManager was having issues with TG performance and had to > enable form-data caching to get acceptable performance at the cost of > possibly stale data. I don't know the details behind it, but that was > the reason I was given for why when you edit forms in MM it sometimes > returns old pre-edit field values. > Correct me if I'm wrong mdomsch, but I think the performance issues were with the database having to hit it with every request. The same would happen with any other framework that was coded the same way. -Mike From Matt_Domsch at dell.com Fri Nov 2 15:39:15 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 10:39:15 -0500 Subject: mirrormanager priority lists for F8 and higher Message-ID: <20071102153915.GB16220@auslistsprd01.us.dell.com> NO, I'm not applying this now. This is for review only right now. From sundaram at fedoraproject.org Fri Nov 2 15:38:39 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Fri, 02 Nov 2007 21:08:39 +0530 Subject: mirrormanager priority lists for F8 and higher In-Reply-To: <20071102153915.GB16220@auslistsprd01.us.dell.com> References: <20071102153915.GB16220@auslistsprd01.us.dell.com> Message-ID: <472B447F.9070904@fedoraproject.org> Matt Domsch wrote: > NO, I'm not applying this now. This is for review only right now. > Did you mean to attach something? Rahul From Matt_Domsch at dell.com Fri Nov 2 15:42:42 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 10:42:42 -0500 Subject: mirrormanager priority lists for F8 and higher In-Reply-To: <20071102153915.GB16220@auslistsprd01.us.dell.com> References: <20071102153915.GB16220@auslistsprd01.us.dell.com> Message-ID: <20071102154242.GC16220@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 10:39:15AM -0500, Matt Domsch wrote: > NO, I'm not applying this now. This is for review only right now. From skvidal at fedoraproject.org Fri Nov 2 15:43:54 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 02 Nov 2007 11:43:54 -0400 Subject: mirrormanager priority lists for F8 and higher In-Reply-To: <20071102154242.GC16220@auslistsprd01.us.dell.com> References: <20071102153915.GB16220@auslistsprd01.us.dell.com> <20071102154242.GC16220@auslistsprd01.us.dell.com> Message-ID: <1194018234.2654.25.camel@cutter> On Fri, 2007-11-02 at 10:42 -0500, Matt Domsch wrote: > On Fri, Nov 02, 2007 at 10:39:15AM -0500, Matt Domsch wrote: > > NO, I'm not applying this now. This is for review only right now. > did you mean to attach something again? -sv From Matt_Domsch at dell.com Fri Nov 2 15:45:51 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 10:45:51 -0500 Subject: mirrormanager priority lists for F8 and higher In-Reply-To: <1194018234.2654.25.camel@cutter> References: <20071102153915.GB16220@auslistsprd01.us.dell.com> <20071102154242.GC16220@auslistsprd01.us.dell.com> <1194018234.2654.25.camel@cutter> Message-ID: <20071102154551.GD16220@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 11:43:54AM -0400, seth vidal wrote: > > On Fri, Nov 02, 2007 at 10:39:15AM -0500, Matt Domsch wrote: > > > NO, I'm not applying this now. This is for review only right now. > > did you mean to attach something again? It was inline, not an attachment. Mailman ate it, twice. :-( http://domsch.com/linux/fedora/0001-SCHEMA-CHANGE-mirrorlist-return-a-longer-ordered-l.patch -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From a.badger at gmail.com Fri Nov 2 17:32:59 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 02 Nov 2007 10:32:59 -0700 Subject: wiki madness In-Reply-To: <472B3080.5090905@fedoraproject.org> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <472B3080.5090905@fedoraproject.org> Message-ID: <472B5F4B.5030807@gmail.com> Rahul Sundaram wrote: > Ignacio Vazquez-Abrams wrote: >> As I see it, we have 3 options: >> >> 1) Continue to work with Moin >> 2) Migrate to another wiki >> 3) Roll our own TurboGears wiki >> > > 4) Fork Moin. > As a subpoint to this -- There's several ways we could fork moin. 1) take their code and only add the changes we need that they are unwilling to take. We are already doing this on an extremely small scale with the hierarchical acls patch and the async notification patch. 2) make major changes to the codebase, for instance tying it to a DB backend. 3) take individual components that we like from moin and port those to another framework (for instance, taking their syntax parser and porting it to a turboGears backend). -Toshio From a.badger at gmail.com Fri Nov 2 18:06:11 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 02 Nov 2007 11:06:11 -0700 Subject: wiki madness In-Reply-To: <20071102050606.GN25455@angus.ind.WPI.EDU> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> Message-ID: <472B6713.3040406@gmail.com> Chuck Anderson wrote: > Won't there be performance problems with a TurboGears-based wiki? I > thought MirrorManager was having issues with TG performance and had to > enable form-data caching to get acceptable performance at the cost of > possibly stale data. I don't know the details behind it, but that was > the reason I was given for why when you edit forms in MM it sometimes > returns old pre-edit field values. > We might have performance issues but I'm confident they'll be different performance issues than we're currently experiencing ;-) The issues we're running into with moin right now are largely caused by Moin's philosophy of having to run off the filesystem, not a db. This means 1) we're unable to spread the load among multiple different app servers so we are constrained to a single server's memory and CPU resources, 2) it makes multiple views of data much harder than it needs to (in the subscription list case, Moin has to walk the filesystem, finding each user's prefs file, parsing it for a watchlist, if the watchlist exists, checking if the page and page categories are in that watchlist, and finally being able to send the notification. With a db, we'd have a separate table for the watchlist and have indexes for the userid and the pagename. Searching for a page wouldn't have to open a file for every single one of our users, instead it would access a single table and pull out the users which were in the watchlist.) With MirrorManager I know we've had memory and db query speed issues trying to serve the mirrorlist directly from the TG app. I wasn't aware that mirrormanager was having trouble keeping up with it's management functionality, Matt is that still true or is caching a leftover from when the two functions were combined? I don't think the wiki is as busy as mirrormanager's mirrorlist serving but it would definitely be busier than any of our other apps. We'd have to take a close look at this to see where we're bottlenecking in currently deployed code before deciding if we want to migrate. -Toshio From a.badger at gmail.com Fri Nov 2 18:15:11 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 02 Nov 2007 11:15:11 -0700 Subject: wiki madness In-Reply-To: <200711021125.03452.vpivaini@cs.helsinki.fi> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <200711021125.03452.vpivaini@cs.helsinki.fi> Message-ID: <472B692F.4070405@gmail.com> Ville-Pekka Vainio wrote: > Ignacio Vazquez-Abrams wrote: >> Option 1: Continue to work with Moin >> ... >> Disadvantages: >> - Upstream seems uncooperative > > I wouldn't actually say so. The thing is, they have quite a small core team > who have been working for quite a while to get 1.6 released. They just don't > have the manpower at this point to fix our issues, especially when a lot of > Moin users never face the same problems our wiki has. > > In my experience they are cooperative and interested - they have for example > offered ideas and possible solutions to the subscription problem - but if we > want fixes soon, we need to do them ourselves. > Have you successfully gotten patches applied? My impression has been that they are very willing to talk about our problems but not been very open to our approaches to fixing them. It could be a misperception or miscommunication but the project seems very cathedral-ish with the unfortunate addition that the dev team isn't focused on large sites and therefore has suboptimal solutions in mind for our problems. -Toshio From Matt_Domsch at dell.com Fri Nov 2 19:58:08 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 14:58:08 -0500 Subject: wiki madness In-Reply-To: <472B6713.3040406@gmail.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> Message-ID: <20071102195808.GA18984@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 11:06:11AM -0700, Toshio Kuratomi wrote: > Chuck Anderson wrote: > > >Won't there be performance problems with a TurboGears-based wiki? I > >thought MirrorManager was having issues with TG performance and had to > >enable form-data caching to get acceptable performance at the cost of > >possibly stale data. I don't know the details behind it, but that was > >the reason I was given for why when you edit forms in MM it sometimes > >returns old pre-edit field values. > > > We might have performance issues but I'm confident they'll be different > performance issues than we're currently experiencing ;-) > > The issues we're running into with moin right now are largely caused by > Moin's philosophy of having to run off the filesystem, not a db. This > means 1) we're unable to spread the load among multiple different app > servers so we are constrained to a single server's memory and CPU > resources, 2) it makes multiple views of data much harder than it needs > to (in the subscription list case, Moin has to walk the filesystem, > finding each user's prefs file, parsing it for a watchlist, if the > watchlist exists, checking if the page and page categories are in that > watchlist, and finally being able to send the notification. With a db, > we'd have a separate table for the watchlist and have indexes for the > userid and the pagename. Searching for a page wouldn't have to open a > file for every single one of our users, instead it would access a single > table and pull out the users which were in the watchlist.) > > With MirrorManager I know we've had memory and db query speed issues > trying to serve the mirrorlist directly from the TG app. I wasn't aware > that mirrormanager was having trouble keeping up with it's management > functionality, Matt is that still true or is caching a leftover from > when the two functions were combined? I'm sure it's still true, it predated having any mirrorlist functionality at all. The short story is, TG (well, SQLObject) either caches data very aggresively, so you can see stale data on changes, or not at all, so each field read in each row results in a DB query. Even with object.sync() calls scattered through the UI actions like I did, leaving caching enabled we do still see stale data on occasion. Disabling caching, generating the UI pages or certainly the publiclist pages takes _forever_, hundreds of thousands of small DB queries. Maybe SQLAlchemy has a better caching mechanism, I don't know. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From skid at linux.md Fri Nov 2 20:37:16 2007 From: skid at linux.md (Serghey Rodin) Date: Fri, 02 Nov 2007 22:37:16 +0200 Subject: Let me introduce myself Message-ID: <472B8A7C.6060906@linux.md> Hello guys, my name is Serghey Rodin and I am from capital of Moldova, Chishinau. I would like to be a part of fedora-project heart. Im not sure what to say here :) I like Fedora and I like GNU/Linux very much:) Regarding my working experience: I worked in best Moldovian Internet Provider (StarNet http://starnet.md), in outsourcing company Remsys (http://remsys.net), Web Development company Ceonex (http://ceonex.com). I have big experience in building failover clusters, security stuff, monitoring huge networks and so. Here is some of my skills: [ OS types ] Fedora, RHEL, CentOS, Debian, SuSE FreeBSD [ Web Servers] Nginx Apache Lighttpd LSWS [ DB ] PostgreSQL MySQL [ Languages ] C (not so good, but I can write something small if its needed) PHP / HTML CGI / PERL (not so good, actually I can only troubleshoot code) BASH (Im a guru :)) ) SQL [ Hosting Panel's and VPS software ] OpenVZ Virtuozzo Xen Vserver HSP/Plesk cPanel DirectAdmin [ Network Equipment ] Cisco Foundry 3Com Dell Alteon Thats not all, I'm just trying to show that I can be helpfull. I will automate just about everything: backups, OS install, system updates, etc... I am natural multi-tasker I can handle interrupts while fluidly switching between several projects Please feel free to ask any questions. Thanks -- With Best Regards Serghey Rodin From a.badger at gmail.com Fri Nov 2 22:07:55 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 02 Nov 2007 15:07:55 -0700 Subject: wiki madness In-Reply-To: <20071102195808.GA18984@auslistsprd01.us.dell.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> Message-ID: <472B9FBB.9080602@gmail.com> Matt Domsch wrote: > On Fri, Nov 02, 2007 at 11:06:11AM -0700, Toshio Kuratomi wrote: >> Chuck Anderson wrote: >> >>> Won't there be performance problems with a TurboGears-based wiki? I >>> thought MirrorManager was having issues with TG performance and had to >>> enable form-data caching to get acceptable performance at the cost of >>> possibly stale data. I don't know the details behind it, but that was >>> the reason I was given for why when you edit forms in MM it sometimes >>> returns old pre-edit field values. >>> >> We might have performance issues but I'm confident they'll be different >> performance issues than we're currently experiencing ;-) >> >> The issues we're running into with moin right now are largely caused by >> Moin's philosophy of having to run off the filesystem, not a db. This >> means 1) we're unable to spread the load among multiple different app >> servers so we are constrained to a single server's memory and CPU >> resources, 2) it makes multiple views of data much harder than it needs >> to (in the subscription list case, Moin has to walk the filesystem, >> finding each user's prefs file, parsing it for a watchlist, if the >> watchlist exists, checking if the page and page categories are in that >> watchlist, and finally being able to send the notification. With a db, >> we'd have a separate table for the watchlist and have indexes for the >> userid and the pagename. Searching for a page wouldn't have to open a >> file for every single one of our users, instead it would access a single >> table and pull out the users which were in the watchlist.) >> >> With MirrorManager I know we've had memory and db query speed issues >> trying to serve the mirrorlist directly from the TG app. I wasn't aware >> that mirrormanager was having trouble keeping up with it's management >> functionality, Matt is that still true or is caching a leftover from >> when the two functions were combined? > > I'm sure it's still true, it predated having any mirrorlist > functionality at all. > > The short story is, TG (well, SQLObject) either caches data very > aggresively, so you can see stale data on changes, or not at all, so > each field read in each row results in a DB query. Even with > object.sync() calls scattered through the UI actions like I did, > leaving caching enabled we do still see stale data on occasion. > Disabling caching, generating the UI pages or certainly the publiclist > pages takes _forever_, hundreds of thousands of small DB queries. > > Maybe SQLAlchemy has a better caching mechanism, I don't know. > I've just taken an extremely quick look at this and I don't know where the stale data problem is coming from, but it does look like SQLObject could make more db calls than SQLAlchemy even with caching on. The first part of this is okay:: In [30]: import model In [31]: sites = model.Site.select(orderBy='name') In [32]: for site in sites: ....: pass ....: 1/Select : SELECT site.id, site.name, site.password, site.org_url, site.private, site.admin_active, site.user_active, site.created_at, site.created_by, site.all_sites_can_pull_from_me, site.downstream_comments FROM site WHERE 1 = 1 ORDER BY name 1/QueryR : SELECT site.id, site.name, site.password, site.org_url, site.private, site.admin_active, site.user_active, site.created_at, site.created_by, site.all_sites_can_pull_from_me, site.downstream_comments FROM site WHERE 1 = 1 ORDER BY name 1/COMMIT : auto This second part is inefficient:: In [33]: site.hosts 1/QueryAll: SELECT id FROM host WHERE site_id = (173) 1/QueryR : SELECT id FROM host WHERE site_id = (173) 1/COMMIT : auto 1/QueryAll: SELECT id FROM host WHERE site_id = (173) 1/QueryR : SELECT id FROM host WHERE site_id = (173) 1/COMMIT : auto Out[33]: [Snip values of site.hosts] In [34]: site.hosts 1/QueryAll: SELECT id FROM host WHERE site_id = (173) 1/QueryR : SELECT id FROM host WHERE site_id = (173) 1/COMMIT : auto 1/QueryAll: SELECT id FROM host WHERE site_id = (173) 1/QueryR : SELECT id FROM host WHERE site_id = (173) 1/COMMIT : auto Out[34]: The list of hosts is retrieved from the db each time the variable is accessed even though caching is enabled. This will make a difference if you access a variable more than once, for instance, printing all the site.hosts.name in a menu of links at the top of the page and then looping through site.hosts to print out a complete record for each. For the stale data problem I'd have to know how to reproduce it. Is the data stale when two people are editing the same information? Is it stale on a page refresh? Etc. -Toshio From Matt_Domsch at dell.com Sat Nov 3 00:59:25 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 2 Nov 2007 19:59:25 -0500 Subject: wiki madness In-Reply-To: <472B9FBB.9080602@gmail.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> <472B9FBB.9080602@gmail.com> Message-ID: <20071103005925.GA6839@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 03:07:55PM -0700, Toshio Kuratomi wrote: > For the stale data problem I'd have to know how to reproduce it. Is the > data stale when two people are editing the same information? Is it > stale on a page refresh? Etc. It's stale on page refresh. Yesterday, I changed the name of a Site using the web UI. I hit "submit", and the change is committed and the page refreshed automatically. The Site name was unchanged (old value) in the text box. I waited a few seconds, and reloaded the page, and the Site name was changed to the new value. In controller.py, Site.update() calls site.sync() at the end of the update call, and the next call will be to Site.get() which should have the new value you'd think. -Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From a.badger at gmail.com Sat Nov 3 04:06:04 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 02 Nov 2007 21:06:04 -0700 Subject: wiki madness In-Reply-To: <20071103005925.GA6839@auslistsprd01.us.dell.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> <472B9FBB.9080602@gmail.com> <20071103005925.GA6839@auslistsprd01.us.dell.com> Message-ID: <472BF3AC.8020005@gmail.com> Matt Domsch wrote: > It's stale on page refresh. Yesterday, I changed the name of a Site > using the web UI. I hit "submit", and the change is committed and the > page refreshed automatically. The Site name was unchanged (old value) > in the text box. I waited a few seconds, and reloaded the page, and > the Site name was changed to the new value. > > In controller.py, Site.update() calls site.sync() at the end of the > update call, and the next call will be to Site.get() which should have > the new value you'd think. > Matt and I looked at this and we have a theory of what's going wrong as well as a fix. I tried to replicate this on publictest1 without success. When I tried to replicate it on the app servers it returned stale data to me every time. The theory is that since the app servers are alternating in sending the page, we're ending up with this sequence of events: 1) app3 serves the site form. 2) User fills out form and submits 3) app4 processes the form results (including the call to site.sync()) and raises a redirect back to the form. 4) app3 gets the request and pulls the stale data out of its cache because site.sync() wasn't called on this server. I'm attaching a patch that invalidates the site cache before serving the page. This should fix the stale cache but we'll have to check whether it brings back the performance issues that require turning on the cache in the first place. (It should be better than not having caching on at all but it still might not be acceptable. Needs testing.) Also, the other data on the forms probably needs to have similar calls made to prevent stale data. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: mirrorman-controllers.patch Type: text/x-patch Size: 526 bytes Desc: not available URL: From Matt_Domsch at dell.com Sat Nov 3 12:38:08 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Sat, 3 Nov 2007 07:38:08 -0500 Subject: wiki madness In-Reply-To: <472BF3AC.8020005@gmail.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> <472B9FBB.9080602@gmail.com> <20071103005925.GA6839@auslistsprd01.us.dell.com> <472BF3AC.8020005@gmail.com> Message-ID: <20071103123808.GC6839@auslistsprd01.us.dell.com> On Fri, Nov 02, 2007 at 09:06:04PM -0700, Toshio Kuratomi wrote: > 1) app3 serves the site form. > 2) User fills out form and submits > 3) app4 processes the form results (including the call to site.sync()) > and raises a redirect back to the form. > 4) app3 gets the request and pulls the stale data out of its cache > because site.sync() wasn't called on this server. Another option that dawns on me, that also matches Mike's goals for the applications servers, would be to change the balancer. Instead of evenly balancing between app3 and app4, we set the weight on app3 to be very very high, like 2^32-1 or whatever the max is, and set the weight for app4 to be 1. That will redirect the traffic to app3 all the time, except when it's offline, in which case it'll redirect to app4, yes? -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From claudemir.daluz at redlinks.com.br Sun Nov 4 15:48:40 2007 From: claudemir.daluz at redlinks.com.br (claudemir.daluz at redlinks.com.br) Date: Sun, 4 Nov 2007 13:48:40 -0200 (BRST) Subject: Contribution in List Message-ID: <61691.201.9.147.120.1194191320.squirrel@www.redlinks.com.br> Hi List, My name is Claudemir da Luz Jr. And I work with the Platform Linux are 7 years. I spent by various distros as in the beginning with Slackware, RedHat, Conectiva (distribution Brasileira), Mandriva. I have extensive experience in administration of networks. I have skills to develop kernel modules, kernel hacking and drivers. When I am creating RPM packages for Bash, perl and new software. Depending on the subject I have a good knowledge in the language C. Already with libnewt and several other libs. And finally ... Linux is in my life to a long time. And I would like to dedicate a part of it to the Fedora Project. Big hugs to all, Claudemir Jr. From gerold at lugd.org Sun Nov 4 20:15:34 2007 From: gerold at lugd.org (Gerold Kassube) Date: Sun, 04 Nov 2007 21:15:34 +0100 Subject: [Fwd: [FOSDEM] Call for Developer Rooms + Stands] Message-ID: <1194207334.19160.23.camel@F7NB.homenet.local> -------- Weitergeleitete Nachricht -------- > Von: Pascal Bleser > Antwort an: FOSDEM visitors > An: fosdem at sophos.fosdem.org > Betreff: [FOSDEM] Call for Developer Rooms + Stands > Datum: Sun, 04 Nov 2007 21:01:23 +0100 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > ... starts now: http://fosdem.org/2008/call_for_devrooms > > Deadline: Monday 2007-11-26, 23:59 CET > > For further information/questions/..., please contact devrooms at fosdem.org > > For projects that have already sent a request, please do send it again, > including the information as asked for on the URL above. > > cheers > - -- > -o) Pascal Bleser http://www.fosdem.org > /\\ FOSDEM 2008 :: 23 + 24 February 2008 in Brussels > _\_v Free and Opensource Software Developers European Meeting > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.4-svn0 (GNU/Linux) > Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org > > iD8DBQFHLiUTr3NMWliFcXcRAgHNAJ9KvshC62d+JClcNhxi/jS8zQM23wCeN702 > 5xElr/ycDGcJAAVKJjQ4Hbg= > =RQz2 > -----END PGP SIGNATURE----- > _______________________________________________ > FOSDEM mailing list > FOSDEM at lists.fosdem.org > http://lists.fosdem.org/mailman/listinfo/fosdem > -- Gerold Kassube -Vorstandsvorsitzender- Linux Usergroup L?rrach e.V. Marie-Curie-Strasse 8 79539 L?rrach _ ASCII ribbon campaign ( ) - against HTML email X & vCards / \ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Dies ist ein digital signierter Nachrichtenteil URL: From Matt_Domsch at dell.com Mon Nov 5 13:21:52 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 5 Nov 2007 07:21:52 -0600 Subject: 2 mirrorlist changes for this week Message-ID: <20071105132152.GG20324@auslistsprd01.us.dell.com> Vasile, admin of repo.fedoramd.org, noticed two bugs in the mirrorlist script. 1) some users aren't being resolved by GeoIP. Turns out, we have two different copies of the GeoIP.dat database on app3 and app4. I want to update both to have this month's copy. We should also figure out how to automate getting this file once a month and publishing to our app servers. 2) If a user isn't resolved by GeoIP, they will be given the global list, but without excluding mirrors who have an Exclusive Country set (e.g. Vasile's server really only wants traffic from .md users, but he sees global users). This requires a change to mirrorlist_server.py trim_by_client_country() which I haven't made yet, but will look into doing so. We should make this change to provide better accurate clients to our mirrors to keep their expenses down. I'll post the patch here for review before implementing. Objections? -Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From jima at beer.tclug.org Mon Nov 5 14:24:07 2007 From: jima at beer.tclug.org (Jima) Date: Mon, 5 Nov 2007 08:24:07 -0600 (CST) Subject: 2 mirrorlist changes for this week In-Reply-To: <20071105132152.GG20324@auslistsprd01.us.dell.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> Message-ID: On Mon, 5 Nov 2007, Matt Domsch wrote: > Objections? None from me. :-) Jima From paulo.banon at googlemail.com Mon Nov 5 14:32:31 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Mon, 5 Nov 2007 15:32:31 +0100 Subject: 2 mirrorlist changes for this week In-Reply-To: References: <20071105132152.GG20324@auslistsprd01.us.dell.com> Message-ID: <7a41c4bc0711050632q7494392dqa88bf8bdf7d1b116@mail.gmail.com> Matt, I think these are reasonable changes that will improve the 'user experience', so also no objections from me :) Paulo On Nov 5, 2007 3:24 PM, Jima wrote: > On Mon, 5 Nov 2007, Matt Domsch wrote: > > Objections? > > None from me. :-) > > Jima > > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From mmcgrath at redhat.com Mon Nov 5 15:41:40 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 09:41:40 -0600 Subject: 2 mirrorlist changes for this week In-Reply-To: <20071105132152.GG20324@auslistsprd01.us.dell.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> Message-ID: <472F39B4.9040806@redhat.com> Matt Domsch wrote: > Vasile, admin of repo.fedoramd.org, noticed two bugs in the mirrorlist > script. > > 1) some users aren't being resolved by GeoIP. Turns out, we have two > different copies of the GeoIP.dat database on app3 and app4. I > want to update both to have this month's copy. We should also > figure out how to automate getting this file once a month and > publishing to our app servers. > This sounds very low risk, I'd say you could do this during the freeze without issue. > 2) If a user isn't resolved by GeoIP, they will be given the global > list, but without excluding mirrors who have an Exclusive Country > set (e.g. Vasile's server really only wants traffic from .md users, > but he sees global users). This requires a change to > mirrorlist_server.py trim_by_client_country() which I haven't made > yet, but will look into doing so. We should make this change to > provide better accurate clients to our mirrors to keep their > expenses down. I'll post the patch here for review before implementing. > Were you planning on implementing this pre or post F8 launch? -Mike From Matt_Domsch at dell.com Mon Nov 5 15:49:58 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 5 Nov 2007 09:49:58 -0600 Subject: 2 mirrorlist changes for this week In-Reply-To: <472F39B4.9040806@redhat.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> <472F39B4.9040806@redhat.com> Message-ID: <20071105154958.GA13061@auslistsprd01.us.dell.com> On Mon, Nov 05, 2007 at 09:41:40AM -0600, Mike McGrath wrote: > >2) If a user isn't resolved by GeoIP, they will be given the global > > list, but without excluding mirrors who have an Exclusive Country > > set (e.g. Vasile's server really only wants traffic from .md users, > > but he sees global users). This requires a change to > > mirrorlist_server.py trim_by_client_country() which I haven't made > > yet, but will look into doing so. We should make this change to > > provide better accurate clients to our mirrors to keep their > > expenses down. I'll post the patch here for review before implementing. > > > > Were you planning on implementing this pre or post F8 launch? > ideally pre-F8 launch, so then one-click downloaders don't get sent to mirrors that don't want them. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From Matt_Domsch at dell.com Mon Nov 5 15:54:01 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 5 Nov 2007 09:54:01 -0600 Subject: 2 mirrorlist changes for this week In-Reply-To: <472F39B4.9040806@redhat.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> <472F39B4.9040806@redhat.com> Message-ID: <20071105155401.GB13061@auslistsprd01.us.dell.com> On Mon, Nov 05, 2007 at 09:41:40AM -0600, Mike McGrath wrote: > Matt Domsch wrote: > >Vasile, admin of repo.fedoramd.org, noticed two bugs in the mirrorlist > >script. > > > >1) some users aren't being resolved by GeoIP. Turns out, we have two > > different copies of the GeoIP.dat database on app3 and app4. I > > want to update both to have this month's copy. We should also > > figure out how to automate getting this file once a month and > > publishing to our app servers. > > > > This sounds very low risk, I'd say you could do this during the freeze > without issue. OK, this is done now, and I've restarted the mirrorlist_server apps on app3 and app4. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From mmcgrath at redhat.com Mon Nov 5 16:09:30 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 10:09:30 -0600 Subject: wiki madness In-Reply-To: <20071103123808.GC6839@auslistsprd01.us.dell.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> <472B9FBB.9080602@gmail.com> <20071103005925.GA6839@auslistsprd01.us.dell.com> <472BF3AC.8020005@gmail.com> <20071103123808.GC6839@auslistsprd01.us.dell.com> Message-ID: <472F403A.9000703@redhat.com> Matt Domsch wrote: > On Fri, Nov 02, 2007 at 09:06:04PM -0700, Toshio Kuratomi wrote: > >> 1) app3 serves the site form. >> 2) User fills out form and submits >> 3) app4 processes the form results (including the call to site.sync()) >> and raises a redirect back to the form. >> 4) app3 gets the request and pulls the stale data out of its cache >> because site.sync() wasn't called on this server. >> > > Another option that dawns on me, that also matches Mike's goals for > the applications servers, would be to change the balancer. Instead of > evenly balancing between app3 and app4, we set the weight on app3 to > be very very high, like 2^32-1 or whatever the max is, and set the > weight for app4 to be 1. That will redirect the traffic to app3 all > the time, except when it's offline, in which case it'll redirect to > app4, yes? > I've tried stuff like that in the past, its sub-optimal and hacky :) Why are we even using cache for MM? Its pretty low traffic compared to the mirrorlist. -Mike From a.badger at gmail.com Mon Nov 5 16:15:07 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 05 Nov 2007 08:15:07 -0800 Subject: 2 mirrorlist changes for this week In-Reply-To: <20071105132152.GG20324@auslistsprd01.us.dell.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> Message-ID: <472F418B.8080001@gmail.com> Matt Domsch wrote: > 2) If a user isn't resolved by GeoIP, they will be given the global > list, but without excluding mirrors who have an Exclusive Country > set (e.g. Vasile's server really only wants traffic from .md users, > but he sees global users). This requires a change to > mirrorlist_server.py trim_by_client_country() which I haven't made > yet, but will look into doing so. We should make this change to > provide better accurate clients to our mirrors to keep their > expenses down. I'll post the patch here for review before implementing. I'm not sure how invasive this will be but it does sound like something we want to keep our mirrors happy. Maybe seeing the patch first will be better for deciding if it's too big a change during the freeze. -Toshio From mmcgrath at redhat.com Mon Nov 5 16:24:56 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 10:24:56 -0600 Subject: Contribution in List In-Reply-To: <61691.201.9.147.120.1194191320.squirrel@www.redlinks.com.br> References: <61691.201.9.147.120.1194191320.squirrel@www.redlinks.com.br> Message-ID: <472F43D8.4010603@redhat.com> claudemir.daluz at redlinks.com.br wrote: > Hi List, > > My name is Claudemir da Luz Jr. And I work with the Platform Linux are 7 > years. > > I spent by various distros as in the beginning with Slackware, RedHat, > Conectiva (distribution Brasileira), Mandriva. > > I have extensive experience in administration of networks. > > I have skills to develop kernel modules, kernel hacking and drivers. > > When I am creating RPM packages for Bash, perl and new software. > > Depending on the subject I have a good knowledge in the language C. > > Already with libnewt and several other libs. And finally ... > > Linux is in my life to a long time. And I would like to dedicate a part of > it to the Fedora Project. > > > Big hugs to all, > > > Claudemir Jr. Welcome Claudemir! Is there a specific project or task you're interested in working on? Infrastructure is just one team to which you can contribute. -Mike From mmcgrath at redhat.com Mon Nov 5 16:57:46 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 10:57:46 -0600 Subject: Minor change to logger Message-ID: <472F4B8A.2000004@redhat.com> I'd like to make some changes to the logger, I think this is very low risk. Basically after changing IP's for the mirror list, we're getting inaccurate results as we're now using twice the number of servers: https://admin.fedoraproject.org/cacti/graph.php?action=view&rra_id=all&local_graph_id=263 Any objections? -Mike From claudemir.daluz at redlinks.com.br Mon Nov 5 17:00:07 2007 From: claudemir.daluz at redlinks.com.br (claudemir.daluz at redlinks.com.br) Date: Mon, 5 Nov 2007 15:00:07 -0200 (BRST) Subject: Contribution in List In-Reply-To: <472F43D8.4010603@redhat.com> References: <61691.201.9.147.120.1194191320.squirrel@www.redlinks.com.br> <472F43D8.4010603@redhat.com> Message-ID: <60462.201.9.168.206.1194282007.squirrel@www.redlinks.com.br> > claudemir.daluz at redlinks.com.br wrote: >> Hi List, >> >> My name is Claudemir da Luz Jr. And I work with the Platform Linux are 7 >> years. >> >> I spent by various distros as in the beginning with Slackware, RedHat, >> Conectiva (distribution Brasileira), Mandriva. >> >> I have extensive experience in administration of networks. >> >> I have skills to develop kernel modules, kernel hacking and drivers. >> >> When I am creating RPM packages for Bash, perl and new software. >> >> Depending on the subject I have a good knowledge in the language C. >> >> Already with libnewt and several other libs. And finally ... >> >> Linux is in my life to a long time. And I would like to dedicate a part >> of >> it to the Fedora Project. >> >> >> Big hugs to all, >> >> >> Claudemir Jr. > > Welcome Claudemir! Is there a specific project or task you're > interested in working on? Infrastructure is just one team to which you > can contribute. > > -Mike > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > Thanks Mike, My Interesting are: - Kernel Modules and Drivers - Portability - Packaging - Programming: C, perl, GTK - Debugger & Core well... I would like to contribute. And I want to devote part of my time to the Fedora community. Regards, Claudemir P. da Luz Jr. From mmcgrath at redhat.com Mon Nov 5 17:22:16 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 11:22:16 -0600 Subject: Another change during the change freeze Message-ID: <472F5148.3000007@redhat.com> I just noticed that app4 is actually mis-configured for the VPN. It's ip address is incorrectly set to 192.168.0.6 instead of 192.168.1.6. I'd like to make this change after lunch. Risk: Low Issues: This is causing proxy3 to ONLY contact app3 for mirrors list and as a result is not redundant. Details: All of the PHX servers communicate with app4 directly, not over the VPN. Only proxy3 in Denver uses the VPN to contact app4 and cannot do so because app4 is not listening to the correct IP. Any objections? -Mike From Matt_Domsch at dell.com Mon Nov 5 17:34:30 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 5 Nov 2007 11:34:30 -0600 Subject: 2 mirrorlist changes for this week In-Reply-To: <472F418B.8080001@gmail.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> <472F418B.8080001@gmail.com> Message-ID: <20071105173430.GA28038@auslistsprd01.us.dell.com> On Mon, Nov 05, 2007 at 08:15:07AM -0800, Toshio Kuratomi wrote: > Matt Domsch wrote: > > >2) If a user isn't resolved by GeoIP, they will be given the global > > list, but without excluding mirrors who have an Exclusive Country > > set (e.g. Vasile's server really only wants traffic from .md users, > > but he sees global users). This requires a change to > > mirrorlist_server.py trim_by_client_country() which I haven't made > > yet, but will look into doing so. We should make this change to > > provide better accurate clients to our mirrors to keep their > > expenses down. I'll post the patch here for review before implementing. > > I'm not sure how invasive this will be but it does sound like something > we want to keep our mirrors happy. Maybe seeing the patch first will > be better for deciding if it's too big a change during the freeze. Here's the resulting function. I think this is very clear code now, and it works for me tested against live data on pt1. def trim_by_client_country(hostresults, clientCountry): results = [] for hostid, hcurl in hostresults: if hostid not in host_country_allowed_cache or \ clientCountry in host_country_allowed_cache[hostid]: results.append((hostid, hcurl)) return results Patch follows. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux diff --git a/mirrors/server-scripts/mirrorlist_server.py b/mirrors/server-scripts/mirrorlist_server.py index cf703ff..c7e17e0 100755 --- a/mirrors/server-scripts/mirrorlist_server.py +++ b/mirrors/server-scripts/mirrorlist_server.py @@ -82,18 +82,11 @@ def client_in_host_allowed(clientCountry, hostID): def trim_by_client_country(hostresults, clientCountry): - if clientCountry is None: - return hostresults - results = [] - for hostid, hcurl in hostresults: - if not host_country_allowed_cache.has_key(hostid): + if hostid not in host_country_allowed_cache or \ + clientCountry in host_country_allowed_cache[hostid]: results.append((hostid, hcurl)) - else: - if clientCountry in host_country_allowed_cache[hostid]: - results.append((hostid, hcurl)) - return results From skvidal at fedoraproject.org Mon Nov 5 17:40:53 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 05 Nov 2007 12:40:53 -0500 Subject: Another change during the change freeze In-Reply-To: <472F5148.3000007@redhat.com> References: <472F5148.3000007@redhat.com> Message-ID: <1194284453.2654.104.camel@cutter> On Mon, 2007-11-05 at 11:22 -0600, Mike McGrath wrote: > I just noticed that app4 is actually mis-configured for the VPN. It's > ip address is incorrectly set to 192.168.0.6 instead of 192.168.1.6. > I'd like to make this change after lunch. > > Risk: Low > Issues: This is causing proxy3 to ONLY contact app3 for mirrors list > and as a result is not redundant. > Details: All of the PHX servers communicate with app4 directly, not > over the VPN. Only proxy3 in Denver uses the VPN to contact app4 and > cannot do so because app4 is not listening to the correct IP. > > Any objections? none here. -sv From dennis at ausil.us Mon Nov 5 17:46:59 2007 From: dennis at ausil.us (Dennis Gilmore) Date: Mon, 5 Nov 2007 12:46:59 -0500 Subject: Another change during the change freeze In-Reply-To: <472F5148.3000007@redhat.com> References: <472F5148.3000007@redhat.com> Message-ID: <200711051147.01166.dennis@ausil.us> On Monday 05 November 2007 11:22:16 am Mike McGrath wrote: > I just noticed that app4 is actually mis-configured for the VPN. It's > ip address is incorrectly set to 192.168.0.6 instead of 192.168.1.6. > I'd like to make this change after lunch. > > Risk: Low > Issues: This is causing proxy3 to ONLY contact app3 for mirrors list > and as a result is not redundant. > Details: All of the PHX servers communicate with app4 directly, not > over the VPN. Only proxy3 in Denver uses the VPN to contact app4 and > cannot do so because app4 is not listening to the correct IP. > > Any objections? > > -Mike sounds fine to me From mmcgrath at redhat.com Mon Nov 5 18:06:25 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 05 Nov 2007 12:06:25 -0600 Subject: 2 mirrorlist changes for this week In-Reply-To: <20071105173430.GA28038@auslistsprd01.us.dell.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> <472F418B.8080001@gmail.com> <20071105173430.GA28038@auslistsprd01.us.dell.com> Message-ID: <472F5BA1.2060408@redhat.com> Matt Domsch wrote: > On Mon, Nov 05, 2007 at 08:15:07AM -0800, Toshio Kuratomi wrote: > >> Matt Domsch wrote: >> >> >>> 2) If a user isn't resolved by GeoIP, they will be given the global >>> list, but without excluding mirrors who have an Exclusive Country >>> set (e.g. Vasile's server really only wants traffic from .md users, >>> but he sees global users). This requires a change to >>> mirrorlist_server.py trim_by_client_country() which I haven't made >>> yet, but will look into doing so. We should make this change to >>> provide better accurate clients to our mirrors to keep their >>> expenses down. I'll post the patch here for review before implementing. >>> >> I'm not sure how invasive this will be but it does sound like something >> we want to keep our mirrors happy. Maybe seeing the patch first will >> be better for deciding if it's too big a change during the freeze. >> > > Here's the resulting function. I think this is very clear code now, > and it works for me tested against live data on pt1. > > def trim_by_client_country(hostresults, clientCountry): > results = [] > for hostid, hcurl in hostresults: > if hostid not in host_country_allowed_cache or \ > clientCountry in host_country_allowed_cache[hostid]: > results.append((hostid, hcurl)) > return results > > Patch follows. > +1 from me, can I get atleast a +1 from someone else in sysadmin-main? Its tested, low risk (as in we can roll back if there's any issues) -Mike From jima at beer.tclug.org Mon Nov 5 18:26:48 2007 From: jima at beer.tclug.org (Jima) Date: Mon, 5 Nov 2007 12:26:48 -0600 (CST) Subject: Another change during the change freeze In-Reply-To: <472F5148.3000007@redhat.com> References: <472F5148.3000007@redhat.com> Message-ID: On Mon, 5 Nov 2007, Mike McGrath wrote: > I just noticed that app4 is actually mis-configured for the VPN. It's ip > address is incorrectly set to 192.168.0.6 instead of 192.168.1.6. I'd like > to make this change after lunch. I'd say go for it if you haven't already (err, get email commits ever get re-enabled?). Jima From katzj at redhat.com Mon Nov 5 18:47:10 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 05 Nov 2007 13:47:10 -0500 Subject: 2 mirrorlist changes for this week In-Reply-To: <472F5BA1.2060408@redhat.com> References: <20071105132152.GG20324@auslistsprd01.us.dell.com> <472F418B.8080001@gmail.com> <20071105173430.GA28038@auslistsprd01.us.dell.com> <472F5BA1.2060408@redhat.com> Message-ID: <1194288430.31380.18.camel@localhost.localdomain> On Mon, 2007-11-05 at 12:06 -0600, Mike McGrath wrote: > Matt Domsch wrote: > > On Mon, Nov 05, 2007 at 08:15:07AM -0800, Toshio Kuratomi wrote: > >> Matt Domsch wrote: > >>> 2) If a user isn't resolved by GeoIP, they will be given the global > >>> list, but without excluding mirrors who have an Exclusive Country > >>> set (e.g. Vasile's server really only wants traffic from .md users, > >>> but he sees global users). This requires a change to > >>> mirrorlist_server.py trim_by_client_country() which I haven't made > >>> yet, but will look into doing so. We should make this change to > >>> provide better accurate clients to our mirrors to keep their > >>> expenses down. I'll post the patch here for review before implementing. > >>> > >> I'm not sure how invasive this will be but it does sound like something > >> we want to keep our mirrors happy. Maybe seeing the patch first will > >> be better for deciding if it's too big a change during the freeze. > >> > > > > Here's the resulting function. I think this is very clear code now, > > and it works for me tested against live data on pt1. > > > > def trim_by_client_country(hostresults, clientCountry): > > results = [] > > for hostid, hcurl in hostresults: > > if hostid not in host_country_allowed_cache or \ > > clientCountry in host_country_allowed_cache[hostid]: > > results.append((hostid, hcurl)) > > return results > > > > Patch follows. > > > > +1 from me, can I get atleast a +1 from someone else in sysadmin-main? > Its tested, low risk (as in we can roll back if there's any issues) Looks straight-forward enough to me too. And getting it in place today does give us the time to know if something's wrong before Thursday. So go go go! :-) Jeremy From a.badger at gmail.com Mon Nov 5 22:47:09 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 05 Nov 2007 14:47:09 -0800 Subject: wiki madness In-Reply-To: <472F403A.9000703@redhat.com> References: <1193963197.4307.34.camel@erato.phig.org> <1193978971.3271.21.camel@ignacio.lan> <20071102050606.GN25455@angus.ind.WPI.EDU> <472B6713.3040406@gmail.com> <20071102195808.GA18984@auslistsprd01.us.dell.com> <472B9FBB.9080602@gmail.com> <20071103005925.GA6839@auslistsprd01.us.dell.com> <472BF3AC.8020005@gmail.com> <20071103123808.GC6839@auslistsprd01.us.dell.com> <472F403A.9000703@redhat.com> Message-ID: <472F9D6D.6070903@gmail.com> Mike McGrath wrote: > Matt Domsch wrote: >> On Fri, Nov 02, 2007 at 09:06:04PM -0700, Toshio Kuratomi wrote: >> >>> 1) app3 serves the site form. >>> 2) User fills out form and submits >>> 3) app4 processes the form results (including the call to >>> site.sync()) and raises a redirect back to the form. >>> 4) app3 gets the request and pulls the stale data out of its cache >>> because site.sync() wasn't called on this server. >>> >> >> Another option that dawns on me, that also matches Mike's goals for >> the applications servers, would be to change the balancer. Instead of >> evenly balancing between app3 and app4, we set the weight on app3 to >> be very very high, like 2^32-1 or whatever the max is, and set the >> weight for app4 to be 1. That will redirect the traffic to app3 all >> the time, except when it's offline, in which case it'll redirect to >> app4, yes? >> > I've tried stuff like that in the past, its sub-optimal and hacky :) > Why are we even using cache for MM? Its pretty low traffic compared to > the mirrorlist. > I'm tempted to say "because SQLObject sucks" but I'd have to do more analysis to know that SQLAlchemy wouldn't also suffer from this problem :-) Basically, SQLObject without caching enabled makes a fresh call to the database for every access of an object which is persisted in the database. Iterate through a list of records more than once (once to convert a value to unicode, the second to output a menu, the third to output the record itself,... etc) and you hit the database innumerable times. SQLObject with caching is more efficient. From what I've seen, SQLAlchemy is even more efficient than that. I think SQLAlchemy is also designed to be immune to this stale cache problem but I'd want to test it to make sure. The PackageDB tries to avoid making redirect calls so it might be different design decisions in the apps rather than SO/SA differences that allow it to avoid this problem. -Toshio From jim at meyering.net Tue Nov 6 15:45:41 2007 From: jim at meyering.net (Jim Meyering) Date: Tue, 06 Nov 2007 16:45:41 +0100 Subject: git tweak request: display URLs on all summary pages Message-ID: <87d4unfkd6.fsf@rho.meyering.net> Hi, I visited http://git.fedoraproject.org/?p=hosted/livecd (all others are the same) and noticed that the usual list of clonable git://, ssh://, http:// URLs is not displayed. Can someone here fix that? Here's how: Determine what file gitweb.cgi is using for its config. (probably just /etc/gitweb.conf), and add to or change that file so that this line appears: @git_base_url_list = (qw(git://git.fedoraproject.org/git/hosted ssh://git.fedoraproject.org/git/hosted); Or maybe it will automatically append the "/hosted/PROJ" suffix, in which case, use this: @git_base_url_list = (qw(git://git.fedoraproject.org/git ssh://git.fedoraproject.org/git); You can add an http:// one, too, if that works. Regards, Jim From mmcgrath at redhat.com Tue Nov 6 17:13:31 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 06 Nov 2007 11:13:31 -0600 Subject: Change freeze ambiguity Message-ID: <4730A0BB.9040203@redhat.com> For the next release I plan on making a change freeze page because it has dawned on me that there are other day-to-day things that require changes but that I wouldn't consider part of the change freeze. For example, creating a new hosted site. We have a couple of open tickets still regarding hosted sites. I can't think of any reason to stop doing tasks such as the hosted tickets, anyone disagree or have additional concerns, comments? -Mike From katzj at redhat.com Tue Nov 6 19:31:34 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 06 Nov 2007 14:31:34 -0500 Subject: git tweak request: display URLs on all summary pages In-Reply-To: <87d4unfkd6.fsf@rho.meyering.net> References: <87d4unfkd6.fsf@rho.meyering.net> Message-ID: <1194377494.31380.74.camel@localhost.localdomain> On Tue, 2007-11-06 at 16:45 +0100, Jim Meyering wrote: > I visited http://git.fedoraproject.org/?p=hosted/livecd > (all others are the same) and noticed that the usual list of > clonable git://, ssh://, http:// URLs is not displayed. > > Can someone here fix that? Should be all set now Jeremy From jim at meyering.net Tue Nov 6 19:36:30 2007 From: jim at meyering.net (Jim Meyering) Date: Tue, 06 Nov 2007 20:36:30 +0100 Subject: git tweak request: display URLs on all summary pages In-Reply-To: <1194377494.31380.74.camel@localhost.localdomain> (Jeremy Katz's message of "Tue, 06 Nov 2007 14:31:34 -0500") References: <87d4unfkd6.fsf@rho.meyering.net> <1194377494.31380.74.camel@localhost.localdomain> Message-ID: <87ir4fdv41.fsf@rho.meyering.net> Jeremy Katz wrote: > On Tue, 2007-11-06 at 16:45 +0100, Jim Meyering wrote: >> I visited http://git.fedoraproject.org/?p=hosted/livecd >> (all others are the same) and noticed that the usual list of >> clonable git://, ssh://, http:// URLs is not displayed. >> >> Can someone here fix that? > > Should be all set now Much better. Thanks. Does the lack of http:// URLs mean they're not supported? I've heard a few complaints, in other contexts, from people unable to use git:// due to firewalls, so http:// access is essential for them. From jkeating at redhat.com Tue Nov 6 23:03:46 2007 From: jkeating at redhat.com (Jesse Keating) Date: Tue, 6 Nov 2007 18:03:46 -0500 Subject: Rebooting builders back to disabled selinux Message-ID: <20071106180346.700573fc@redhat.com> Having the builders with selinux enabled (even in permissive) is causing problems with rawhide composes. Until we can research the problems more fully, I have decided to disable selinux on the builders. This will allow rawhide to be installable once more, which is going to be important in the next few days. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From jkeating at redhat.com Tue Nov 6 23:19:58 2007 From: jkeating at redhat.com (Jesse Keating) Date: Tue, 6 Nov 2007 18:19:58 -0500 Subject: Rebooting builders back to disabled selinux In-Reply-To: <20071106180346.700573fc@redhat.com> References: <20071106180346.700573fc@redhat.com> Message-ID: <20071106181958.7ba9f216@redhat.com> On Tue, 6 Nov 2007 18:03:46 -0500 Jesse Keating wrote: > Having the builders with selinux enabled (even in permissive) is > causing problems with rawhide composes. Until we can research the > problems more fully, I have decided to disable selinux on the > builders. This will allow rawhide to be installable once more, which > is going to be important in the next few days. Only a few hosts needed to be rebooted to have selinux disabled once more. ppc2, xenbuilder1,2,4. So far I've rebooted all but xenbuilder4. Currently it's disabled in Koji so that it will not take on any new jobs. I'm going to let it finish the jobs it has running and then reboot it. Luke made the appropriate puppet change so that selinux will remain disabled on the builders. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From katzj at redhat.com Wed Nov 7 02:57:00 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 06 Nov 2007 21:57:00 -0500 Subject: Rebooting builders back to disabled selinux In-Reply-To: <20071106180346.700573fc@redhat.com> References: <20071106180346.700573fc@redhat.com> Message-ID: <1194404220.31380.84.camel@localhost.localdomain> On Tue, 2007-11-06 at 18:03 -0500, Jesse Keating wrote: > Having the builders with selinux enabled (even in permissive) is > causing problems with rawhide composes. Until we can research the > problems more fully, I have decided to disable selinux on the > builders. This will allow rawhide to be installable once more, which > is going to be important in the next few days. Just to give a little bit more information on what's going on -- When we build the second stage for anaconda, we're now using yum rather than rpm2cpio (this change occurred a few months ago). This means that rpmlib wants to set file contexts on the files as it looks like SELinux is enabled. Unfortunately, the current rawhide policy defines a type for some of the files which the FC6 policy doesn't know about. So trying to set those contexts fails. Which then leads to packages failing to install and things fall down. The likely fix is probably going to involve using the hacky fake libselinux from mock when building the anaconda chroot. But needs some more investigation and poking. Now that Jesse's tracked it down to a cause, it should be easy enough to install an FC6 box and figure something out Jeremy From Matt_Domsch at dell.com Wed Nov 7 12:54:47 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Wed, 7 Nov 2007 06:54:47 -0600 Subject: d.f.r.c marked private in MM now Message-ID: <20071107125447.GB31486@auslistsprd01.us.dell.com> I marked d.f.r.c as private this morning, so in ~15 minutes it should be dropped from the mirrorlists. There are plenty of active mirrors now so no worries. We can change this back next week. -Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From mmcgrath at redhat.com Wed Nov 7 17:07:13 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 07 Nov 2007 11:07:13 -0600 Subject: Release and meeting tomorrow Message-ID: <4731F0C1.6070309@redhat.com> Reminder! We have a release tomorrow and also a meeting. In the US we had a time change so many of you will have the meeting earlier than normal. http://timeanddate.com/worldclock/fixedtime.html?month=11&day=8&year=2007&hour=20&min=0&sec=0&p1=0 -Mike From mmcgrath at redhat.com Wed Nov 7 19:46:58 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 07 Nov 2007 13:46:58 -0600 Subject: Bad drive on app4 Message-ID: <47321632.2000105@redhat.com> We've got a bad drive on app4, this box is scheduled to be removed soon but hasn't been yet and, is not under warranty. I consider this to be of emergency in importance, and I will be finding a different location to build app4 immediately. Getting the new app server up is of medium risk but not having it up for tomorrow is of HIGH risk as it would cause all mirror traffic to go through app3, and only app3. -Mike From ricky at fedoraproject.org Wed Nov 7 20:07:23 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Wed, 7 Nov 2007 15:07:23 -0500 Subject: syncStatic.sh Change Message-ID: <20071107200723.GF6263@Max.example.com> I had to make a last-minute change to syncStatic.sh - the counter image said "1 days" instead of "1 day." Since I had frozen it to a single commit, it'll just wget the updated image from fedorapeople.org- of course, I'll remove this before pushing the F8 site. Next time, I'll branch fedora-web instead of locking to a commit to make these sorts of changes much easier. Thanks, Ricky -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From lmacken at redhat.com Wed Nov 7 21:20:26 2007 From: lmacken at redhat.com (Luke Macken) Date: Wed, 7 Nov 2007 16:20:26 -0500 Subject: yum depsolver patch for releng1 Message-ID: <20071107212026.GD11043@crow.myhome.westell.com> As most are probably aware, we are hitting a bug[0] in the yum depsolver when trying to mash dist-f7-updates-testing. Tim Lauridsen wrote a patch[1] that seems to fix the problem and allows us to compose updates-testing again (WOOOOOOOO!). Tim sent the patch to yum-devel for review, but in the mean time I'd be happy to apply this to the yum on releng1 so we can get these repos back under testing. What do you guys think? luke [0]: https://bugzilla.redhat.com/show_bug.cgi?id=360291 [1]: https://bugzilla.redhat.com/show_bug.cgi?id=360291#c24 From mmcgrath at redhat.com Wed Nov 7 22:24:43 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 07 Nov 2007 16:24:43 -0600 Subject: Bad drive on app4 In-Reply-To: <47321632.2000105@redhat.com> References: <47321632.2000105@redhat.com> Message-ID: <47323B2B.10701@redhat.com> Mike McGrath wrote: > We've got a bad drive on app4, this box is scheduled to be removed > soon but hasn't been yet and, is not under warranty. I consider this > to be of emergency in importance, and I will be finding a different > location to build app4 immediately. Getting the new app server up is > of medium risk but not having it up for tomorrow is of HIGH risk as it > would cause all mirror traffic to go through app3, and only app3. > > -Mike Ok, the new app4 is up and on xen4. xenbuilder2 is no more (we'll bring it back after the release). We hit a strange puppet bug we've never seen before. We are still hitting this bug, as a result to do a manual run on puppet1 you must run: puppetd -t --server=puppet1.fedora.phx.redhat.com For some reason app4 does not work with puppet1 like the rest of the hosts. There's a certificate error of some kind. The good news is that we now have way more capacity for app4 now then we did on the i386 box it was on. -Mike From lmacken at redhat.com Wed Nov 7 23:55:04 2007 From: lmacken at redhat.com (Luke Macken) Date: Wed, 7 Nov 2007 18:55:04 -0500 Subject: yum depsolver patch for releng1 In-Reply-To: <20071107212026.GD11043@crow.myhome.westell.com> References: <20071107212026.GD11043@crow.myhome.westell.com> Message-ID: <20071107235504.GE11043@crow.myhome.westell.com> On Wed, Nov 07, 2007 at 04:20:26PM -0500, Luke Macken wrote: > As most are probably aware, we are hitting a bug[0] in the yum depsolver > when trying to mash dist-f7-updates-testing. Tim Lauridsen wrote a > patch[1] that seems to fix the problem and allows us to compose > updates-testing again (WOOOOOOOO!). > > Tim sent the patch to yum-devel for review, but in the mean time I'd be > happy to apply this to the yum on releng1 so we can get these repos > back under testing. What do you guys think? Also worth noting that this patch does not break any of yums unittests: python test/alltests.py ......................................................................................................................... ---------------------------------------------------------------------- Ran 121 tests in 0.286s OK luke From katzj at redhat.com Thu Nov 8 00:28:04 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 07 Nov 2007 19:28:04 -0500 Subject: yum depsolver patch for releng1 In-Reply-To: <20071107235504.GE11043@crow.myhome.westell.com> References: <20071107212026.GD11043@crow.myhome.westell.com> <20071107235504.GE11043@crow.myhome.westell.com> Message-ID: <1194481684.11300.1.camel@localhost.localdomain> On Wed, 2007-11-07 at 18:55 -0500, Luke Macken wrote: > On Wed, Nov 07, 2007 at 04:20:26PM -0500, Luke Macken wrote: > > As most are probably aware, we are hitting a bug[0] in the yum depsolver > > when trying to mash dist-f7-updates-testing. Tim Lauridsen wrote a > > patch[1] that seems to fix the problem and allows us to compose > > updates-testing again (WOOOOOOOO!). > > > > Tim sent the patch to yum-devel for review, but in the mean time I'd be > > happy to apply this to the yum on releng1 so we can get these repos > > back under testing. What do you guys think? > > Also worth noting that this patch does not break any of yums unittests: But given that the tests are really only starting to be written, counting on them as the only indicator is perhaps not wise. I've got the patch flagged to look at later tonight or tomorrow Jeremy From katzj at redhat.com Thu Nov 8 16:03:53 2007 From: katzj at redhat.com (Jeremy Katz) Date: Thu, 08 Nov 2007 11:03:53 -0500 Subject: yum depsolver patch for releng1 In-Reply-To: <1194481684.11300.1.camel@localhost.localdomain> References: <20071107212026.GD11043@crow.myhome.westell.com> <20071107235504.GE11043@crow.myhome.westell.com> <1194481684.11300.1.camel@localhost.localdomain> Message-ID: <1194537833.23584.2.camel@localhost.localdomain> On Wed, 2007-11-07 at 19:28 -0500, Jeremy Katz wrote: > On Wed, 2007-11-07 at 18:55 -0500, Luke Macken wrote: > > On Wed, Nov 07, 2007 at 04:20:26PM -0500, Luke Macken wrote: > > > As most are probably aware, we are hitting a bug[0] in the yum depsolver > > > when trying to mash dist-f7-updates-testing. Tim Lauridsen wrote a > > > patch[1] that seems to fix the problem and allows us to compose > > > updates-testing again (WOOOOOOOO!). > > > > > > Tim sent the patch to yum-devel for review, but in the mean time I'd be > > > happy to apply this to the yum on releng1 so we can get these repos > > > back under testing. What do you guys think? > > > > Also worth noting that this patch does not break any of yums unittests: > > But given that the tests are really only starting to be written, > counting on them as the only indicator is perhaps not wise. I've got > the patch flagged to look at later tonight or tomorrow Okay, after looking closer, the patch seems sane to me Jeremy From lmacken at redhat.com Thu Nov 8 16:59:07 2007 From: lmacken at redhat.com (Luke Macken) Date: Thu, 8 Nov 2007 11:59:07 -0500 Subject: yum depsolver patch for releng1 In-Reply-To: <20071107212026.GD11043@crow.myhome.westell.com> References: <20071107212026.GD11043@crow.myhome.westell.com> Message-ID: <20071108165907.GH11043@crow.myhome.westell.com> On Wed, Nov 07, 2007 at 04:20:26PM -0500, Luke Macken wrote: > As most are probably aware, we are hitting a bug[0] in the yum depsolver > when trying to mash dist-f7-updates-testing. Tim Lauridsen wrote a > patch[1] that seems to fix the problem and allows us to compose > updates-testing again (WOOOOOOOO!). > > Tim sent the patch to yum-devel for review, but in the mean time I'd be > happy to apply this to the yum on releng1 so we can get these repos > back under testing. What do you guys think? Both Jeremy and Florian approved the patch, so I went ahead and patched yum on releng1 and mashed f7-updates-testing. So we should be back in action, after 16 days of updates-testing downtime. luke From Matt_Domsch at dell.com Thu Nov 8 18:31:37 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Thu, 8 Nov 2007 12:31:37 -0600 Subject: F8 postmortem Message-ID: <20071108183137.GA17706@auslistsprd01.us.dell.com> Here are a few thoughts on the F8 release, from the mirror manager perspective. :-) Handoff from rel-eng to IS to put on masters worked great, all the bits were landed by Monday morning. Content on the masters wasn't hardlinked at first, which caused some churn and unnecessary downloading. A tree that was hardlinked from rawhide, carrying only i386 and x86_64, would have downloaded only 12GB instead of 109GB for the full unhardlinked tree. Let's be sure the tree is hardlinked. Permissions on dirs/files on the mirror should be revisited. All directories should be 0750 and files should be 0640 before the bitflip, to prevent leaks. vsftpd will serve a file with a known name and perms 0644 even if the directory or one above it is 0750. Apache won't. Let's be sure to use these permissions. Release day bitflip was 30 minutes later than expected, which caused churn for our mirroradmins. Can this be a scheduled cronjob? I'd like a way to be notified automatically when changes, such as the bitflip or content landing, has actually been snapmirrored everywhere. After this message is received, I'd then like to be able to send email to the mirror admins. Is this possible? all in all it's been smooth, just a few hiccups. Thanks, Matt Fedora Mirror Wrangler -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From mmcgrath at redhat.com Thu Nov 8 19:02:48 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 08 Nov 2007 13:02:48 -0600 Subject: F8 postmortem In-Reply-To: <20071108183137.GA17706@auslistsprd01.us.dell.com> References: <20071108183137.GA17706@auslistsprd01.us.dell.com> Message-ID: <47335D58.3080509@redhat.com> Matt Domsch wrote: > Here are a few thoughts on the F8 release, from the mirror manager > perspective. :-) > > > Handoff from rel-eng to IS to put on masters worked great, all the > bits were landed by Monday morning. > > Content on the masters wasn't hardlinked at first, which caused some > churn and unnecessary downloading. A tree that was hardlinked from rawhide, > carrying only i386 and x86_64, would have downloaded only 12GB instead > of 109GB for the full unhardlinked tree. Let's be sure the tree is hardlinked. > > Permissions on dirs/files on the mirror should be revisited. > All directories should be 0750 and files should be 0640 before the > bitflip, to prevent leaks. vsftpd will serve a file with a known name > and perms 0644 even if the directory or one above it is 0750. Apache > won't. Let's be sure to use these permissions. > > Release day bitflip was 30 minutes later than expected, which caused > churn for our mirroradmins. Can this be a scheduled cronjob? > Additionally, we need to flip the bit before we release it on the website. -Mike From smooge at gmail.com Thu Nov 8 19:31:43 2007 From: smooge at gmail.com (Stephen John Smoogen) Date: Thu, 8 Nov 2007 12:31:43 -0700 Subject: F8 postmortem In-Reply-To: <20071108183137.GA17706@auslistsprd01.us.dell.com> References: <20071108183137.GA17706@auslistsprd01.us.dell.com> Message-ID: <80d7e4090711081131k507cf97dkd0fa6d983aa72101@mail.gmail.com> On Nov 8, 2007 11:31 AM, Matt Domsch wrote: > Here are a few thoughts on the F8 release, from the mirror manager > perspective. :-) > As a very very former mirror manager, I wanted to say that Fedora has done a heck of a job on getting this working and this was very painless compared to a long time ago. I want to say that the BIGGEST WIN is that by using a community model of getting software together, hardware requirements, and various tracking software Fedora was able to get a trust model working between the masters and tier 1/tier2 etc mirrors that was hard if not impossible a long time ago. Thanks. -- Stephen J Smoogen. -- CSIRT/Linux System Administrator How far that little candle throws his beams! So shines a good deed in a naughty world. = Shakespeare. "The Merchant of Venice" From fantec at proxad.net Thu Nov 8 20:21:31 2007 From: fantec at proxad.net (Francois Petillon) Date: Thu, 08 Nov 2007 21:21:31 +0100 Subject: F8 postmortem In-Reply-To: <20071108183137.GA17706@auslistsprd01.us.dell.com> References: <20071108183137.GA17706@auslistsprd01.us.dell.com> Message-ID: <47336FCB.3010207@proxad.net> Matt Domsch wrote: > Permissions on dirs/files on the mirror should be revisited. > All directories should be 0750 and files should be 0640 before the > bitflip, to prevent leaks. vsftpd will serve a file with a known name > and perms 0644 even if the directory or one above it is 0750. Apache > won't. Let's be sure to use these permissions. I disagree. This is typically a server setup issue, not a permission issue. If vsftpd serves such files, it means it has the right to access the directory (so it is run with the same UID than rsync or it is in the same group). If the files are group readable, then technically, vsftpd has the right to read them just like it has the right to access the directories path. Doing 0640 on files will block vsftpd access if and only if the admin has enabled anon_world_readable_only. I would advocate for a release root-only bitfliped to get updates as simple as possible. As admins are usually asked to schedule a atjob to run a rsync/chmod at release date/time, KISS... ;-) If you really want to avoid leaks, then perhaps you should test mirrors with a special directory to reproduce usual release rights and check from time to time if this directory contents are unreadable. Fran?ois From cweyl at alumni.drew.edu Fri Nov 9 01:17:11 2007 From: cweyl at alumni.drew.edu (Chris Weyl) Date: Thu, 8 Nov 2007 17:17:11 -0800 Subject: RFE: torrents.fp.o -> torrent.fp.o? Message-ID: <7dd7ab490711081717q306301ep9bf2d599de08d9e@mail.gmail.com> >From the "d'oh!" category: I tend to go reflexively to torrents.fedoraproject.org rather than torrent.fp.o, always resulting in a momentary twinge of panic when I get the "server not found" alert. Any chance we can setup a "torrents" subdomain and just have it redirect (dns alias, apache, whatever) to torrent? My brain would appreciate it :) Thanks- -Chris -- Chris Weyl Ex astris, scientia From skvidal at fedoraproject.org Fri Nov 9 02:14:42 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Thu, 08 Nov 2007 21:14:42 -0500 Subject: RFE: torrents.fp.o -> torrent.fp.o? In-Reply-To: <7dd7ab490711081717q306301ep9bf2d599de08d9e@mail.gmail.com> References: <7dd7ab490711081717q306301ep9bf2d599de08d9e@mail.gmail.com> Message-ID: <1194574482.15170.51.camel@cutter> On Thu, 2007-11-08 at 17:17 -0800, Chris Weyl wrote: > >From the "d'oh!" category: I tend to go reflexively to > torrents.fedoraproject.org rather than torrent.fp.o, always resulting > in a momentary twinge of panic when I get the "server not found" > alert. > > Any chance we can setup a "torrents" subdomain and just have it > redirect (dns alias, apache, whatever) to torrent? > > My brain would appreciate it :) > This is done. -sv From wtogami at redhat.com Fri Nov 9 03:17:30 2007 From: wtogami at redhat.com (Warren Togami) Date: Thu, 08 Nov 2007 22:17:30 -0500 Subject: Mirror Manager Backup Message-ID: <4733D14A.10500@redhat.com> Just a quick thought after tonight's outage. If *everything* is down, the most important thing to have up is the mirrormanager since all of our users depend on it on a regular basis. It might be a relatively easy thing to periodically replicate the entire mirrormanager database over to an external box (like @ serverbeach) so we can repoint DNS and bring it back online at moment's notice. Just a thought. Warren Togami wtogami at redhat.com From cweyl at alumni.drew.edu Fri Nov 9 03:18:11 2007 From: cweyl at alumni.drew.edu (Chris Weyl) Date: Thu, 8 Nov 2007 19:18:11 -0800 Subject: RFE: torrents.fp.o -> torrent.fp.o? In-Reply-To: <1194574482.15170.51.camel@cutter> References: <7dd7ab490711081717q306301ep9bf2d599de08d9e@mail.gmail.com> <1194574482.15170.51.camel@cutter> Message-ID: <7dd7ab490711081918x5c55a1a4pc0308a4dfa9a9953@mail.gmail.com> On 11/8/07, seth vidal wrote: > On Thu, 2007-11-08 at 17:17 -0800, Chris Weyl wrote: > > Any chance we can setup a "torrents" subdomain and just have it > > redirect (dns alias, apache, whatever) to torrent? > > This is done. Thanks :) -- Chris Weyl Ex astris, scientia From jonathansteffan at gmail.com Fri Nov 9 03:25:46 2007 From: jonathansteffan at gmail.com (Jonathan Steffan) Date: Thu, 08 Nov 2007 20:25:46 -0700 Subject: Mirror Manager Backup In-Reply-To: <4733D14A.10500@redhat.com> References: <4733D14A.10500@redhat.com> Message-ID: <4733D33A.1070302@gmail.com> Warren Togami wrote: > Just a quick thought after tonight's outage. > > If *everything* is down, the most important thing to have up is the > mirrormanager since all of our users depend on it on a regular basis. > > It might be a relatively easy thing to periodically replicate the > entire mirrormanager database over to an external box (like @ > serverbeach) so we can repoint DNS and bring it back online at > moment's notice. +1 -- Jonathan Steffan daMaestro GPG Fingerprint: 93A2 3E2F DC26 5570 3472 5B16 AD12 6CE7 0D86 AF59 From mmcgrath at redhat.com Fri Nov 9 03:52:27 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 08 Nov 2007 21:52:27 -0600 Subject: Mirror Manager Backup In-Reply-To: <4733D14A.10500@redhat.com> References: <4733D14A.10500@redhat.com> Message-ID: <4733D97B.5090800@redhat.com> Warren Togami wrote: > Just a quick thought after tonight's outage. > > If *everything* is down, the most important thing to have up is the > mirrormanager since all of our users depend on it on a regular basis. Mirrormanager is perfectly designed in this respect. Long story short, everything that has anything to do with distribution needs to not be in a Red Hat colo. Certainly not as the SPOF. This will require a few external pieces (including a backup vpn server, which openVPN makes pretty easy to do). -Mike From skvidal at fedoraproject.org Fri Nov 9 03:57:49 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Thu, 08 Nov 2007 22:57:49 -0500 Subject: Mirror Manager Backup In-Reply-To: <4733D97B.5090800@redhat.com> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> Message-ID: <1194580669.15170.74.camel@cutter> On Thu, 2007-11-08 at 21:52 -0600, Mike McGrath wrote: > Warren Togami wrote: > > Just a quick thought after tonight's outage. > > > > If *everything* is down, the most important thing to have up is the > > mirrormanager since all of our users depend on it on a regular basis. > > Mirrormanager is perfectly designed in this respect. Long story short, > everything that has anything to do with distribution needs to not be in > a Red Hat colo. Certainly not as the SPOF. This will require a few > external pieces (including a backup vpn server, which openVPN makes > pretty easy to do). > The infrastructure we need to support: build: koji, plague, local mirrors for building, cvs, bodhi, pkgdb distribution: mirrormanager, mirrormaster, staging, torrent primary site: fp.o, wiki, docs, start.fp.o meta services all of the above will or do require: vpn, fas[2], puppet, infrastructure-cvs, dns, mail Does that sound right? -sv From skvidal at fedoraproject.org Fri Nov 9 04:21:16 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Thu, 08 Nov 2007 23:21:16 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194580669.15170.74.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> Message-ID: <1194582076.15170.76.camel@cutter> On Thu, 2007-11-08 at 22:57 -0500, seth vidal wrote: > On Thu, 2007-11-08 at 21:52 -0600, Mike McGrath wrote: > > Warren Togami wrote: > > > Just a quick thought after tonight's outage. > > > > > > If *everything* is down, the most important thing to have up is the > > > mirrormanager since all of our users depend on it on a regular basis. > > > > Mirrormanager is perfectly designed in this respect. Long story short, > > everything that has anything to do with distribution needs to not be in > > a Red Hat colo. Certainly not as the SPOF. This will require a few > > external pieces (including a backup vpn server, which openVPN makes > > pretty easy to do). > > > > The infrastructure we need to support: > > build: koji, plague, local mirrors for building, cvs, bodhi, pkgdb > > distribution: mirrormanager, mirrormaster, staging, torrent > > primary site: fp.o, wiki, docs, start.fp.o > > meta services all of the above will or do require: > vpn, fas[2], puppet, infrastructure-cvs, dns, mail meta services additions: backups and then mostly unrelated services: fedorapeople.org, planet.fedoraproject.org, hosted.fedoraproject.org -sv From paulo.banon at googlemail.com Fri Nov 9 08:16:05 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Fri, 9 Nov 2007 09:16:05 +0100 Subject: Mirror Manager Backup In-Reply-To: <1194582076.15170.76.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> Message-ID: <7a41c4bc0711090016v223e6e2di2aeac7a82270503a@mail.gmail.com> Part of the Primary site, will be hosted in Germany (right now Telia/ProIO are just waiting for our server). We still have 1/2 rack to use, and from my personal experience working with Telia and ProIO i don't remember having 1 "unscheduled" outage in that colo for the last year. Paulo On Nov 9, 2007 5:21 AM, seth vidal wrote: > > On Thu, 2007-11-08 at 22:57 -0500, seth vidal wrote: > > On Thu, 2007-11-08 at 21:52 -0600, Mike McGrath wrote: > > > Warren Togami wrote: > > > > Just a quick thought after tonight's outage. > > > > > > > > If *everything* is down, the most important thing to have up is the > > > > mirrormanager since all of our users depend on it on a regular basis. > > > > > > Mirrormanager is perfectly designed in this respect. Long story short, > > > everything that has anything to do with distribution needs to not be in > > > a Red Hat colo. Certainly not as the SPOF. This will require a few > > > external pieces (including a backup vpn server, which openVPN makes > > > pretty easy to do). > > > > > > > The infrastructure we need to support: > > > > build: koji, plague, local mirrors for building, cvs, bodhi, pkgdb > > > > distribution: mirrormanager, mirrormaster, staging, torrent > > > > primary site: fp.o, wiki, docs, start.fp.o > > > > meta services all of the above will or do require: > > vpn, fas[2], puppet, infrastructure-cvs, dns, mail > > meta services additions: backups > > and then mostly unrelated services: > fedorapeople.org, planet.fedoraproject.org, hosted.fedoraproject.org > > > -sv > > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From santosp at fedoraproject.org Fri Nov 9 09:12:35 2007 From: santosp at fedoraproject.org (Paulo Santos) Date: Fri, 9 Nov 2007 10:12:35 +0100 Subject: Wiki Message-ID: <7a41c4bc0711090112l3806b14m86fb7da8ea739731@mail.gmail.com> Hi all, Now that everything calmed down, i've removed app2 from the wiki loadbalancer so that the editing problems go away. It should be picked up deployed at the next puppet run Please let us know if you still find any problem. Paulo From skvidal at fedoraproject.org Fri Nov 9 13:25:52 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 09 Nov 2007 08:25:52 -0500 Subject: Wiki In-Reply-To: <7a41c4bc0711090112l3806b14m86fb7da8ea739731@mail.gmail.com> References: <7a41c4bc0711090112l3806b14m86fb7da8ea739731@mail.gmail.com> Message-ID: <1194614752.15170.78.camel@cutter> On Fri, 2007-11-09 at 10:12 +0100, Paulo Santos wrote: > Hi all, > > Now that everything calmed down, i've removed app2 from the wiki > loadbalancer so that the editing problems go away. > It should be picked up deployed at the next puppet run > > Please let us know if you still find any problem. > Thanks. -sv From Matt_Domsch at dell.com Fri Nov 9 13:43:10 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Fri, 9 Nov 2007 07:43:10 -0600 Subject: Mirror Manager Backup In-Reply-To: <7a41c4bc0711090016v223e6e2di2aeac7a82270503a@mail.gmail.com> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <7a41c4bc0711090016v223e6e2di2aeac7a82270503a@mail.gmail.com> Message-ID: <20071109134310.GA6274@auslistsprd01.us.dell.com> On Fri, Nov 09, 2007 at 09:16:05AM +0100, Paulo Santos wrote: > Part of the Primary site, will be hosted in Germany (right now > Telia/ProIO are just waiting for our server). > We still have 1/2 rack to use, and from my personal experience working > with Telia and ProIO i don't remember having 1 "unscheduled" outage in > that colo for the last year. Seth said he'd get additional app servers put together (starting today) at serverbeach. Once we've got the additional equipment in Germany adding high-priority app servers there too would be very good. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From katzj at redhat.com Fri Nov 9 13:43:17 2007 From: katzj at redhat.com (Jeremy Katz) Date: Fri, 09 Nov 2007 08:43:17 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194580669.15170.74.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> Message-ID: <1194615797.20915.26.camel@localhost.localdomain> On Thu, 2007-11-08 at 22:57 -0500, seth vidal wrote: > On Thu, 2007-11-08 at 21:52 -0600, Mike McGrath wrote: > > Warren Togami wrote: > > > Just a quick thought after tonight's outage. > > > > > > If *everything* is down, the most important thing to have up is the > > > mirrormanager since all of our users depend on it on a regular basis. > > > > Mirrormanager is perfectly designed in this respect. Long story short, > > everything that has anything to do with distribution needs to not be in > > a Red Hat colo. Certainly not as the SPOF. This will require a few > > external pieces (including a backup vpn server, which openVPN makes > > pretty easy to do). > > The infrastructure we need to support: [snip] > Does that sound right? That seems like a reasonable list. The other thing to keep in mind, though, is that some things can be lower on the list for trying to duplicate. eg, if the colo is down, the fact that a lot of the developer infrastructure is down is fairly acceptable I think. Now if the colo is down because a hole opened in the earth and the DC has fallen to the center of the earth, then it's another question. But then we're talking substantial disaster recovery and not just avoiding SPOF for our users during colo downtimes Jeremy From notting at redhat.com Fri Nov 9 15:09:59 2007 From: notting at redhat.com (Bill Nottingham) Date: Fri, 9 Nov 2007 10:09:59 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194582076.15170.76.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> Message-ID: <20071109150959.GA24638@nostromo.devel.redhat.com> seth vidal (skvidal at fedoraproject.org) said: > > build: koji, plague, local mirrors for building, cvs, bodhi, pkgdb > > > > distribution: mirrormanager, mirrormaster, staging, torrent > > > > primary site: fp.o, wiki, docs, start.fp.o > > > > meta services all of the above will or do require: > > vpn, fas[2], puppet, infrastructure-cvs, dns, mail > > meta services additions: backups > > and then mostly unrelated services: > fedorapeople.org, planet.fedoraproject.org, hosted.fedoraproject.org If I was planning strategy, the priority for offsite/redundancy would be: 1) mirrormanager 2) torrent 3) start.fp.o, docs, fp.o 4) mirrormaster 5) hosted 6) wiki 7) fedorapeople/planet 8) build stuff (meta-services sprinkled in as necessary) Bill From skvidal at fedoraproject.org Fri Nov 9 16:46:06 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 09 Nov 2007 11:46:06 -0500 Subject: Mirror Manager Backup In-Reply-To: <20071109150959.GA24638@nostromo.devel.redhat.com> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> Message-ID: <1194626766.15170.84.camel@cutter> On Fri, 2007-11-09 at 10:09 -0500, Bill Nottingham wrote: > seth vidal (skvidal at fedoraproject.org) said: > > > build: koji, plague, local mirrors for building, cvs, bodhi, pkgdb > > > > > > distribution: mirrormanager, mirrormaster, staging, torrent > > > > > > primary site: fp.o, wiki, docs, start.fp.o > > > > > > meta services all of the above will or do require: > > > vpn, fas[2], puppet, infrastructure-cvs, dns, mail > > > > meta services additions: backups > > > > and then mostly unrelated services: > > fedorapeople.org, planet.fedoraproject.org, hosted.fedoraproject.org > > If I was planning strategy, the priority for offsite/redundancy would be: > > 1) mirrormanager > 2) torrent > 3) start.fp.o, docs, fp.o > 4) mirrormaster > 5) hosted > 6) wiki > 7) fedorapeople/planet > 8) build stuff > > (meta-services sprinkled in as necessary) > the importance order is something separate. I was just listing the service so we can create interdependency charts. fo example: mirrormanager is useless w/o fas :) -sv From paulo.banon at googlemail.com Fri Nov 9 16:52:06 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Fri, 9 Nov 2007 17:52:06 +0100 Subject: Wiki In-Reply-To: <1194614752.15170.78.camel@cutter> References: <7a41c4bc0711090112l3806b14m86fb7da8ea739731@mail.gmail.com> <1194614752.15170.78.camel@cutter> Message-ID: <7a41c4bc0711090852g4b5e0fei2d738f238008d060@mail.gmail.com> This change was reverted back so that we have more performance during the weekend. Basically the writes will be "kaput", let's see how this goes :) Paulo On Nov 9, 2007 2:25 PM, seth vidal wrote: > > > On Fri, 2007-11-09 at 10:12 +0100, Paulo Santos wrote: > > Hi all, > > > > Now that everything calmed down, i've removed app2 from the wiki > > loadbalancer so that the editing problems go away. > > It should be picked up deployed at the next puppet run > > > > Please let us know if you still find any problem. > > > > Thanks. > -sv > > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From jkeating at redhat.com Fri Nov 9 16:53:46 2007 From: jkeating at redhat.com (Jesse Keating) Date: Fri, 9 Nov 2007 11:53:46 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194626766.15170.84.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> <1194626766.15170.84.camel@cutter> Message-ID: <20071109115346.4b83ea2a@redhat.com> On Fri, 09 Nov 2007 11:46:06 -0500 seth vidal wrote: > fo example: mirrormanager is useless w/o fas :) There are degrees of usefulness. Surely you don't need fas just to respond to mirrorlist cgi requests, or provide a listing of last known good mirrors for direct downloading... -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From katzj at redhat.com Fri Nov 9 17:16:53 2007 From: katzj at redhat.com (Jeremy Katz) Date: Fri, 09 Nov 2007 12:16:53 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194626766.15170.84.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> <1194626766.15170.84.camel@cutter> Message-ID: <1194628613.20915.40.camel@localhost.localdomain> On Fri, 2007-11-09 at 11:46 -0500, seth vidal wrote: > the importance order is something separate. I was just listing the > service so we can create interdependency charts. > > fo example: mirrormanager is useless w/o fas :) The managemetn side of mirrormanager is. But the "serve out the existing mirror information" side is perfectly fine without FAS (and realistically, is the part that more of us think of when we think MM) Jeremy From skvidal at fedoraproject.org Fri Nov 9 17:20:22 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Fri, 09 Nov 2007 12:20:22 -0500 Subject: Mirror Manager Backup In-Reply-To: <1194628613.20915.40.camel@localhost.localdomain> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> <1194626766.15170.84.camel@cutter> <1194628613.20915.40.camel@localhost.localdomain> Message-ID: <1194628822.15170.91.camel@cutter> On Fri, 2007-11-09 at 12:16 -0500, Jeremy Katz wrote: > On Fri, 2007-11-09 at 11:46 -0500, seth vidal wrote: > > the importance order is something separate. I was just listing the > > service so we can create interdependency charts. > > > > fo example: mirrormanager is useless w/o fas :) > > The managemetn side of mirrormanager is. But the "serve out the > existing mirror information" side is perfectly fine without FAS (and > realistically, is the part that more of us think of when we think MM) in my mind: mirrormanger - the management side mirrorlist - the actual mirrorlists -sv From jima at beer.tclug.org Fri Nov 9 19:09:30 2007 From: jima at beer.tclug.org (Jima) Date: Fri, 9 Nov 2007 13:09:30 -0600 (CST) Subject: Mirror Manager Backup In-Reply-To: <1194628822.15170.91.camel@cutter> References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> <1194626766.15170.84.camel@cutter> <1194628613.20915.40.camel@localhost.localdomain> <1194628822.15170.91.camel@cutter> Message-ID: On Fri, 9 Nov 2007, seth vidal wrote: > On Fri, 2007-11-09 at 12:16 -0500, Jeremy Katz wrote: >> The managemetn side of mirrormanager is. But the "serve out the >> existing mirror information" side is perfectly fine without FAS (and >> realistically, is the part that more of us think of when we think MM) > > in my mind: > mirrormanger - the management side > mirrorlist - the actual mirrorlists Ditto. I do agree, though, that we could probably make mirrorlist redundant without necessarily making mirrormanager so, and that would fulfill most of our needs. Jima From tchung at fedoraproject.org Sun Nov 11 02:36:29 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sat, 10 Nov 2007 18:36:29 -0800 Subject: UnicodeEncodeError is back In-Reply-To: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> Message-ID: <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> On 11/10/07, Thomas Chung wrote: > Hello, > > I'm getting UnicodeEncodeError message on following page: > > http://fedoraproject.org/wiki/Artwork/CDArt It's been 6 hours since I reported. FYI, here is the SOP for this issue. http://fedoraproject.org/wiki/Infrastructure/SOP/wiki#UnicodeEncodeError Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From jkeating at redhat.com Sun Nov 11 13:39:42 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 11 Nov 2007 08:39:42 -0500 Subject: UnicodeEncodeError is back In-Reply-To: <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> Message-ID: <20071111083942.01ef6c72@redhat.com> On Sat, 10 Nov 2007 18:36:29 -0800 "Thomas Chung" wrote: > On 11/10/07, Thomas Chung wrote: > > Hello, > > > > I'm getting UnicodeEncodeError message on following page: > > > > http://fedoraproject.org/wiki/Artwork/CDArt > > It's been 6 hours since I reported. FYI, here is the SOP for this > issue. > http://fedoraproject.org/wiki/Infrastructure/SOP/wiki#UnicodeEncodeError > Regards, -- Sorry for the delay, I think I have it all cleaned up. Let me know if you run across more pages with errors. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From kanarip at kanarip.com Sun Nov 11 14:12:24 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 11 Nov 2007 15:12:24 +0100 Subject: UnicodeEncodeError is back In-Reply-To: <20071111083942.01ef6c72@redhat.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> <20071111083942.01ef6c72@redhat.com> Message-ID: <47370DC8.2060607@kanarip.com> Jesse Keating wrote: > On Sat, 10 Nov 2007 18:36:29 -0800 > "Thomas Chung" wrote: > >> On 11/10/07, Thomas Chung wrote: >>> Hello, >>> >>> I'm getting UnicodeEncodeError message on following page: >>> >>> http://fedoraproject.org/wiki/Artwork/CDArt >> It's been 6 hours since I reported. FYI, here is the SOP for this >> issue. >> http://fedoraproject.org/wiki/Infrastructure/SOP/wiki#UnicodeEncodeError >> Regards, -- > > Sorry for the delay, I think I have it all cleaned up. Let me know if > you run across more pages with errors. > http://fedoraproject.org/wiki/PackageMaintainers/Policy had been reported by 'blarney' to have the same error. Kind regards, Jeroen van Meeuwen -kanarip From jkeating at redhat.com Sun Nov 11 14:17:30 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 11 Nov 2007 09:17:30 -0500 Subject: UnicodeEncodeError is back In-Reply-To: <47370DC8.2060607@kanarip.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> <20071111083942.01ef6c72@redhat.com> <47370DC8.2060607@kanarip.com> Message-ID: <20071111091730.70e24b63@redhat.com> On Sun, 11 Nov 2007 15:12:24 +0100 Jeroen van Meeuwen wrote: > http://fedoraproject.org/wiki/PackageMaintainers/Policy had been > reported by 'blarney' to have the same error. Which I fixed at the same time, as you would know if you actually checked it after I sent my message. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From kanarip at kanarip.com Sun Nov 11 15:40:50 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 11 Nov 2007 16:40:50 +0100 Subject: UnicodeEncodeError is back In-Reply-To: <20071111091730.70e24b63@redhat.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> <20071111083942.01ef6c72@redhat.com> <47370DC8.2060607@kanarip.com> <20071111091730.70e24b63@redhat.com> Message-ID: <47372282.9020401@kanarip.com> Jesse Keating wrote: > On Sun, 11 Nov 2007 15:12:24 +0100 > Jeroen van Meeuwen wrote: > >> http://fedoraproject.org/wiki/PackageMaintainers/Policy had been >> reported by 'blarney' to have the same error. > > > Which I fixed at the same time, as you would know if you actually > checked it after I sent my message. > I'm sorry to have troubled you; I'm now getting the UnicodeError on http://fedoraproject.org/wiki/Features/JigdoRelease Kind regards, Jeroen van Meeuwen -kanarip From tchung at fedoraproject.org Sat Nov 10 19:56:21 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sat, 10 Nov 2007 11:56:21 -0800 Subject: UnicodeEncodeError is back Message-ID: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> Hello, I'm getting UnicodeEncodeError message on following page: http://fedoraproject.org/wiki/Artwork/CDArt Thanks -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From jkeating at redhat.com Sun Nov 11 16:00:16 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 11 Nov 2007 11:00:16 -0500 Subject: UnicodeEncodeError is back In-Reply-To: <47372282.9020401@kanarip.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> <20071111083942.01ef6c72@redhat.com> <47370DC8.2060607@kanarip.com> <20071111091730.70e24b63@redhat.com> <47372282.9020401@kanarip.com> Message-ID: <20071111110016.04ee62ec@redhat.com> On Sun, 11 Nov 2007 16:40:50 +0100 Jeroen van Meeuwen wrote: > I'm sorry to have troubled you; I'm now getting the UnicodeError on > > http://fedoraproject.org/wiki/Features/JigdoRelease Fixed. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From lmacken at redhat.com Sun Nov 11 16:30:46 2007 From: lmacken at redhat.com (Luke Macken) Date: Sun, 11 Nov 2007 11:30:46 -0500 Subject: comps corruption.. again. Message-ID: <20071111163046.GQ11043@crow.myhome.westell.com> https://bugzilla.redhat.com/show_bug.cgi?id=374801 Once again, the checksums match up when they hit /mnt/koji/mash/updates, but from there they get synced out to our mirrors corrupted. We may be experiencing some corruption on the master mirror, but I don't believe I have the access to verify. luke From jkeating at redhat.com Sun Nov 11 16:48:26 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 11 Nov 2007 11:48:26 -0500 Subject: comps corruption.. again. In-Reply-To: <20071111163046.GQ11043@crow.myhome.westell.com> References: <20071111163046.GQ11043@crow.myhome.westell.com> Message-ID: <20071111114826.0c20c091@redhat.com> On Sun, 11 Nov 2007 11:30:46 -0500 Luke Macken wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=374801 > > Once again, the checksums match up when they > hit /mnt/koji/mash/updates, but from there they get synced out to our > mirrors corrupted. We may be experiencing some corruption on the > master mirror, but I don't believe I have the access to verify. we are. It got corrupted either in route to RDU or after the fact in RDU, but in such a way that the size and timestamp didn't change, so rsync wouldn't pick up new copies. I manually touched the file and synced a correct copy down that will filter out to the mirrors. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From dennis at ausil.us Sun Nov 11 23:10:46 2007 From: dennis at ausil.us (Dennis Gilmore) Date: Sun, 11 Nov 2007 17:10:46 -0600 Subject: FC-6 box rebuild's Message-ID: <200711111710.50737.dennis@ausil.us> One of the takeaway items from the meeting on thursday was the need to rebuild all boxes that are currently running FC-6 before FC-6 is EOL'd Right now we know we need to rebuild the buildsystem. which includes all builders and the koji hub. Some of the app servers need rebuilding. We need to test the applications running on FC-6 on their new platform wether it is RHEL5 or F-8 right now i would like to do the buildsystem on the weekend of Dec 1-2. Volunteers to help would be appreciated as well as identifying what else needs migration and testing Dennis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From jkeating at redhat.com Sun Nov 11 23:46:29 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 11 Nov 2007 18:46:29 -0500 Subject: FC-6 box rebuild's In-Reply-To: <200711111710.50737.dennis@ausil.us> References: <200711111710.50737.dennis@ausil.us> Message-ID: <20071111184629.7a9eecd7@redhat.com> On Sun, 11 Nov 2007 17:10:46 -0600 Dennis Gilmore wrote: > right now i would like to do the buildsystem on the weekend of Dec > 1-2. Volunteers to help would be appreciated as well as identifying > what else needs migration and testing Upgrading the buildsystem OSs would be a great time to get a new build of koji on to them. We should coordinate this with koji upstream folks to get any changes in prior to the refresh, and consider that locked down for the Fedora 9 cycle. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From ricky at fedoraproject.org Mon Nov 12 02:20:41 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Sun, 11 Nov 2007 21:20:41 -0500 Subject: Meeting Log - 2007-11-08 Message-ID: <20071112022041.GG28779@Max.nyc1.dsl.speakeasy.net> 15:01 * jima 15:01 * ricky 15:01 -!- Netsplit zelazny.freenode.net <-> irc.freenode.net quits: yingbull 15:02 < abadger1999> Grr 15:02 < ricky> Crazy netsplits? 15:02 -!- Netsplit over, joins: yingbull 15:02 < abadger1999> Yesh. making me crazy 15:02 -!- giarc_w [i=hidden-u at gnat.asiscan.com] has joined #fedora-meeting 15:02 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting time - Who's here? 15:03 * jima 15:03 * ricky 15:03 < fchiulli> frank 15:03 * mmcgrath is here 15:03 -!- jmtaylor [n=jason at c-76-112-115-86.hsd1.mi.comcast.net] has joined #fedora-meeting 15:04 * dgilmore is here 15:04 < jcollie> hello 15:04 * jima has to leave early 15:04 < mmcgrath> skvidal: paulobanon: lmacken: others? 15:04 < mmcgrath> jcollie: yo 15:05 < skvidal> pong 15:05 < jima> and i may have to wander off to do Real Work(tm). 15:05 < jcollie> real work sucks 15:05 * jcollie is jealous of mmcgrath dgilmore and abadger1999 :) 15:05 * nirik is in the rabble seats. 15:05 < jima> nirik: we're all rabble here. :P 15:06 < dgilmore> jima: why ? 15:06 < dgilmore> jcollie: why ? 15:06 < mmcgrath> Alrighty, lets get started. 15:06 * nirik might be interested in doing more infrastructure stuff someday...perhaps when pkgdb gets rid of the need to process cvsadmin requests. ;) 15:06 < jcollie> dgilmore: your new job of course :) (congrats by the way) 15:06 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets (the Release) 15:06 < dgilmore> jcollie: :) thanks 15:07 -!- CyberSpy [n=cyberspy at cpe-065-191-024-225.nc.res.rr.com] has joined #fedora-meeting 15:07 * bpepple lurks in the background. 15:07 < ricky> Mass ticket-closing? :) 15:09 < mmcgrath> https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?status=new&status=assigned&status=reopened&milestone=Fedora+8 15:10 < dgilmore> mmcgrath: i think there all done :) 15:12 < skvidal> okay 15:12 < skvidal> so 15:12 < skvidal> mmcgrath has been netsplit into oblivion 15:12 < skvidal> or disconnected 15:12 < skvidal> so 15:12 < skvidal> ya 15:13 -!- Netsplit zelazny.freenode.net <-> irc.freenode.net quits: fab_away, gregdek, lancelan, Standbye, mmcgrathbot, couf_afk, clarkbw, so_solid_moo, mbacovsk, mmcgrath, (+2 more, use /NETSPLIT to show all of them) 15:13 < jima> oh man. 15:13 < skvidal> he just jabbered me to say 'well that sucks' 15:13 < jcollie> uhm 15:13 < skvidal> see 15:13 < dgilmore> :P 15:13 < ricky> Aha, there it goes. 15:13 < ricky> Hehe. 15:13 < jcollie> fedora needs a jabber server 15:13 < ricky> +1 to that idea :) 15:13 < jima> or irc.fedoraproject.org ;) 15:13 < skvidal> +1 to jabber 15:13 < dgilmore> skvidal: +1 to jabber 15:13 < ricky> Some projects have irc.project.org CNAME to Freenode. 15:13 < skvidal> tho a lot of red hat folks are bizarrely addicted to irc 15:13 -!- Netsplit over, joins: mmcgrathbot, paulobanon, mmcgrath, so_solid_moo, couf_afk, fab_away, Standbye, clarkbw, mbacovsk, gregdek (+2 more) 15:13 < dgilmore> conference rooms rock 15:14 < notting> skvidal: does 'too lazy to learn jabber' count? 15:14 < tibbs|h> notting: +1 15:14 < skvidal> notting: no 15:14 * jima hasn't done much with jabber lately (although he needs to get back into it) 15:14 < dgilmore> notting: theres nothing to learn 15:14 < mmcgrath> What did I miss? 15:14 < skvidal> mmcgrath: we're opening a jabber server 15:14 < ricky> 15:13:12 < jcollie> fedora needs a jabber server 15:14 < dgilmore> mmcgrath: jabber.fp.o 15:15 < mmcgrath> :: sigh :: 15:15 < skvidal> *love* 15:15 < mmcgrath> Ok, anyway, hurray on the release. 15:15 < dgilmore> mmcgrath: so looks like we got all those tickets done 15:15 < dgilmore> :) 15:15 < mmcgrath> we need to make sure the mirrors are ready *before* we switch the website. 15:15 < dgilmore> congrats to everyone that got things done 15:16 < skvidal> dgilmore: hey what about those of us who just got in the way? 15:16 < ricky> And we need to double check website links before too :/ 15:16 < skvidal> dgilmore: don't we get a congrats, too? 15:16 < dgilmore> skvidal: thats why i watched 15:16 < dgilmore> not to get in the way 15:16 < dgilmore> skvidal: nope you get a slap upside your head :) 15:16 < skvidal> thanks 15:16 < skvidal> dgilmore: next fudcon you have to carry spot 15:17 < dgilmore> skvidal: i did that last one 15:17 < mmcgrath> Anyone have anything else to say about the release? 15:17 < skvidal> dgilmore: it's your job now :) 15:17 < skvidal> mmcgrath: yay? 15:17 < dgilmore> mmcgrath: we seem to be getting better at it 15:17 < skvidal> mmcgrath: fedora++? 15:17 < jcollie> werewolf rocks 15:17 < skvidal> forward the fedora? 15:17 < dgilmore> skvidal: thanks 15:17 < skvidal> Zod is pissed? 15:17 < dgilmore> Zod might go on the warpath 15:17 < jcollie> in the land of strange coincidences, my 6yo son chose to be a werewolf for halloween this year 15:17 < dgilmore> i say give it a month then we huke him 15:17 < ricky> Heh. 15:18 < dgilmore> s/huke/nuke/ 15:18 < skvidal> dgilmore: you huke 15:18 < dgilmore> skvidal: i just make people puke 15:18 < skvidal> dgilmore: esp spot 15:18 < jima> argh! 15:18 < skvidal> I think we lost some people again 15:18 * spot ralfs 15:18 -!- G__ [n=njones at wikipedia/NigelJ] has joined #fedora-meeting 15:18 * dgilmore picks spot up 15:19 < jima> skvidal: my connection hosed for about two minutes, yeah. 15:19 < dgilmore> mike is dead in the water again 15:19 * lmacken rolls in late 15:19 * ricky is curious about SELinux coolness. (saw that ticket from lmacken) 15:19 < ricky> Well, speak of the devil... 15:19 < lmacken> yeah.. 15:19 < dgilmore> so We made some good strides this release 15:19 < lmacken> it's probably not going to be a very easy task 15:20 * mmcgrath thinks so too. 15:20 < mmcgrath> Ok, it doesn't seem anyone has anything else. 15:20 < lmacken> most of our machines should be in permissive mode.. although for some odd reason that caused issues with mock, so we disabled it 15:20 * ricky plans to leave SELinux enabled on his computer for the first time this relase :) 15:20 < dgilmore> One thing we need to plan for is rebuilding the builders 15:20 < lmacken> we'll just have to keep an eye out for violations, and fix them as they come.. until we are satisfied that nothing will explode 15:20 < abadger1999> When are we going to unfreeze? 15:20 -!- k0k [n=k0k at fedora/k0k] has joined #fedora-meeting 15:21 < dgilmore> abadger1999: i say give it a week 15:21 * ricky double-checks slashdot for the article... 15:21 < dgilmore> mmcgrath: im thinking we should rebuild the builders in 3 weeks time 15:21 < dgilmore> do it on a weekend 15:22 < mmcgrath> No doubt, this release went even better then F7 did and way way better then FC6 15:22 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor 15:22 < mmcgrath> See I said it'd be quick :) 15:22 < mmcgrath> Anyone have anything they'd like / want to talk about? 15:22 -!- mmcgrath [n=mmcgrath at 74-136-11-144.dhcp.insightbb.com] has quit "leaving" 15:22 < ricky> Guess not. 15:22 -!- mmcgrath [n=mmcgrath at 74-136-11-144.dhcp.insightbb.com] has joined #fedora-meeting 15:22 < skvidal> hahah 15:23 < dgilmore> mmcgrath: just builders 15:24 < skvidal> dgilmore: what do we use to rebuild the builders? 15:24 < skvidal> dgilmore: do we need to first build builder builders? 15:24 < dgilmore> skvidal: either RHEL5 or F-8 15:24 < skvidal> and then how do we make those?! 15:24 < dgilmore> depending on the python bug 15:24 < skvidal> we'd need to build builder builder builders! 15:24 * mmcgrath votes RHEL5 if we can get away with it. 15:24 < skvidal> +1 15:25 * ricky wonders how we'll be moving away from FC6.. 15:25 < dgilmore> mmcgrath: RHEL5 will be similiar to what we use today 15:25 < abadger1999> +1 15:25 < jcollie> +1 on RHEL5 15:26 < jcollie> unless there's going to be a Werewolf LTS edition :) 15:26 < dgilmore> so we should do it the weekend of Dec1 and 2 15:27 -!- rdieter_away is now known as rdieter 15:27 < dgilmore> we should consider what to do with hammer2 and xenbuilder1 they have been in use since not long after Fedora Extras 3 started 15:28 * jcollie accepts hardware donations :) 15:28 * jima too :) 15:28 < dgilmore> they are dual opteron 142's i think 15:28 < dgilmore> with 3gb ram 15:28 * jima would take 'em ;) 15:29 < jcollie> same here :) 15:29 < dgilmore> penguin computing 1u boxes 15:29 * jima has one of those, but it's only dual xeon 15:29 < jima> (w/ 2gb ram) 15:29 < mmcgrath> dgilmore: Once the bladeservers are in place, I'd think we could use them for test servers or additional expansion until they eventually die and get replaced. 15:29 < dgilmore> skvidal: do you remember when we got em 15:30 < dgilmore> mmcgrath: :) any word on that 15:30 < jima> okay, i'm off for the day -- everyone have a nice rest-of-day/evening! 15:30 < dgilmore> should we start beating people up 15:30 < dgilmore> seeya jima 15:30 < ricky> See you. 15:30 < skvidal> umm 15:30 < skvidal> yes 15:30 -!- G [n=njones at 202-154-149-26.ubs-dynamic.connections.net.nz] has quit Connection timed out 15:30 < skvidal> gafton was in charge 15:30 < skvidal> so 15:30 < skvidal> a looooooooooooong time ago 15:30 < dgilmore> it was awhile ago 15:31 < dgilmore> ppc1 was gotten at the same time 15:31 < dgilmore> or shortly after 15:31 -!- hircus [n=michel at dhcp-cs-244-140.cs.indiana.edu] has joined #fedora-meeting 15:31 -!- fchiulli [i=824c400e at gateway/web/cgi-irc/ircatwork.com/session] has left #fedora-meeting [] 15:32 < dgilmore> so we need to annouce the buildsys downtime soon give a few weeks notice 15:32 < dgilmore> do we have other systems on FC-6 we need to look at migrating 15:33 < ricky> Aren't app3/4 FC6? 15:35 < abadger1999> ricky: Yes. We have to migrate those as well. 15:35 < dgilmore> anyways lets talk about this on the list and get an action plan together 15:35 < dgilmore> for now lets finish the meeting and relax for a minute 15:36 < ricky> Sound good to me :) 15:36 < abadger1999> I think we have all the packages necessary to run the TG apps on RHEL5 built. Now we need to test mirrormanager, pkgdb, smolt, etc on publictest10 (RHEL5 box) and then migrate. 15:36 < ricky> Nothing depends on python 2.4? 15:36 < dgilmore> abadger1999: :) 15:36 < dgilmore> ricky: rhel5 is python 2.4 15:37 < abadger1999> RHEL5 and FC6 both use python2.4 15:37 < ricky> Oop, mixed that up. 15:38 -!- giarc_w [i=hidden-u at gnat.asiscan.com] has quit "Leaving" 15:38 < dgilmore> so for now 15:38 * jcollie loves foisting salesdroids off on my boss 15:38 < dgilmore> ===Meeting Over ==== 15:38 < dgilmore> thanks yall -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From paulo.banon at googlemail.com Sun Nov 11 15:02:37 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Sun, 11 Nov 2007 16:02:37 +0100 Subject: UnicodeEncodeError is back In-Reply-To: <20071111091730.70e24b63@redhat.com> References: <369bce3b0711101156q413e3da1qb1f479990424780c@mail.gmail.com> <369bce3b0711101836l100dd837hf4e59c8d04bcb015@mail.gmail.com> <20071111083942.01ef6c72@redhat.com> <47370DC8.2060607@kanarip.com> <20071111091730.70e24b63@redhat.com> Message-ID: <7a41c4bc0711110702ld4c8f82m1758bb77beda8596@mail.gmail.com> This is the consequence of both app servers supporting the wiki. Until we use only one again, we will have this errors. Paulo On Nov 11, 2007 3:17 PM, Jesse Keating wrote: > On Sun, 11 Nov 2007 15:12:24 +0100 > Jeroen van Meeuwen wrote: > > > http://fedoraproject.org/wiki/PackageMaintainers/Policy had been > > reported by 'blarney' to have the same error. > > > Which I fixed at the same time, as you would know if you actually > checked it after I sent my message. > > > -- > Jesse Keating > Fedora -- All my bits are free, are yours? > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > > From tchung at fedoraproject.org Mon Nov 12 04:16:22 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 20:16:22 -0800 Subject: Wiki ERROR ! In-Reply-To: <4737896C.4020406@skynet.be> References: <4737896C.4020406@skynet.be> Message-ID: <369bce3b0711112016w650a5f2en7ff45baa5c2aa455@mail.gmail.com> On 11/11/07, david vantongerloo wrote: > What now ? > http://fedoraproject.org/wiki/DavidVantongerloo > > Almost ok !! > > Now i get an Big ERROR page... > This is an ongoing issue with our current wiki system. I'll forward your message to Fedora Infrastructure Team. Next time, you can report this issue (UnicodeEncodeError) to fedora-infrastructure-list at redhat.com Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From ricky at fedoraproject.org Mon Nov 12 04:55:04 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Sun, 11 Nov 2007 23:55:04 -0500 Subject: Wiki ERROR ! In-Reply-To: <369bce3b0711112016w650a5f2en7ff45baa5c2aa455@mail.gmail.com> References: <4737896C.4020406@skynet.be> <369bce3b0711112016w650a5f2en7ff45baa5c2aa455@mail.gmail.com> Message-ID: <20071112045504.GI28779@Max.nyc1.dsl.speakeasy.net> On Sun, Nov 11, 2007 at 08:16:22PM -0800, Thomas Chung wrote: > On 11/11/07, david vantongerloo wrote: > > What now ? > > http://fedoraproject.org/wiki/DavidVantongerloo > > > > Almost ok !! > > > > Now i get an Big ERROR page... > > > > This is an ongoing issue with our current wiki system. > I'll forward your message to Fedora Infrastructure Team. > Next time, you can report this issue (UnicodeEncodeError) to > fedora-infrastructure-list at redhat.com This should be fixed now. Thanks for letting us know, Ricky -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From Matt_Domsch at dell.com Sat Nov 10 14:21:03 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Sat, 10 Nov 2007 08:21:03 -0600 Subject: Mirror Manager Backup In-Reply-To: References: <4733D14A.10500@redhat.com> <4733D97B.5090800@redhat.com> <1194580669.15170.74.camel@cutter> <1194582076.15170.76.camel@cutter> <20071109150959.GA24638@nostromo.devel.redhat.com> <1194626766.15170.84.camel@cutter> <1194628613.20915.40.camel@localhost.localdomain> <1194628822.15170.91.camel@cutter> Message-ID: <20071110142103.GA15880@auslistsprd01.us.dell.com> On Fri, Nov 09, 2007 at 01:09:30PM -0600, Jima wrote: > On Fri, 9 Nov 2007, seth vidal wrote: > >On Fri, 2007-11-09 at 12:16 -0500, Jeremy Katz wrote: > >>The managemetn side of mirrormanager is. But the "serve out the > >>existing mirror information" side is perfectly fine without FAS (and > >>realistically, is the part that more of us think of when we think MM) > > > >in my mind: > >mirrormanger - the management side > >mirrorlist - the actual mirrorlists > > Ditto. > I do agree, though, that we could probably make mirrorlist redundant > without necessarily making mirrormanager so, and that would fulfill most > of our needs. Right. In practice, mirrorlist doesn't need regular access to the database, and in case we can't reach the database, can continue to offer slighly stale data (e.g. >1 hour old, which right now it never is). Which should be just fine. Mike wanted me too stop doing queries on app4, and just pull the data that's queried from app3. Now that we're talking about more app servers for the mirrorlist, this makes even more sense, and I'll look to do this. We will need app3 to generate the data, and then copy it to the other app servers. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From tchung at fedoraproject.org Mon Nov 12 05:09:12 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 21:09:12 -0800 Subject: UnicodeEncodeError - FWN/Beats/Artwork In-Reply-To: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> References: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> Message-ID: <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> On 11/11/07, Thomas Chung wrote: > http://fedoraproject.org/wiki/FWN/Beats/Artwork Also in http://fedoraproject.org/wiki/FWN/Beats/ArtworkTemp Thanks. -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From tchung at fedoraproject.org Mon Nov 12 04:47:49 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 20:47:49 -0800 Subject: UnicodeEncodeError - FWN/Beats/Artwork Message-ID: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> http://fedoraproject.org/wiki/FWN/Beats/Artwork Thanks. -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From ricky at fedoraproject.org Mon Nov 12 05:17:09 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Mon, 12 Nov 2007 00:17:09 -0500 Subject: UnicodeEncodeError - FWN/Beats/Artwork In-Reply-To: <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> References: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> Message-ID: <20071112051709.GJ28779@Max.nyc1.dsl.speakeasy.net> On Sun, Nov 11, 2007 at 09:09:12PM -0800, Thomas Chung wrote: > Also in http://fedoraproject.org/wiki/FWN/Beats/ArtworkTemp Fixed. Thanks, Ricky -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tchung at fedoraproject.org Mon Nov 12 06:06:16 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 22:06:16 -0800 Subject: UnicodeEncodeError - FWN/Beats/Artwork In-Reply-To: <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> References: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> Message-ID: <369bce3b0711112206r6b75d7aey495e2975b380ca7b@mail.gmail.com> On 11/11/07, Thomas Chung wrote: > On 11/11/07, Thomas Chung wrote: > > http://fedoraproject.org/wiki/FWN/Beats/Artwork > > Also in http://fedoraproject.org/wiki/FWN/Beats/ArtworkTemp Sorry guys, the list is growing. http://fedoraproject.org/wiki/FWN/Beats/Marketing Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From tchung at fedoraproject.org Mon Nov 12 06:45:08 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 22:45:08 -0800 Subject: UnicodeEncodeError - FWN/Beats/PlanetFedora In-Reply-To: <369bce3b0711112243x3344d98g2adc6e4d5883e7a2@mail.gmail.com> References: <369bce3b0711112243x3344d98g2adc6e4d5883e7a2@mail.gmail.com> Message-ID: <369bce3b0711112245j73002984r2611a9a1e357872@mail.gmail.com> On 11/11/07, Thomas Chung wrote: > Sorry guys, > http://fedoraproject.org/wiki/FWN/Beats/PlanetFedora oh man, another one. :( http://fedoraproject.org/wiki/FWN/Beats -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From tchung at fedoraproject.org Mon Nov 12 06:16:20 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 22:16:20 -0800 Subject: UnicodeEncodeError - FWN/Beats/Artwork In-Reply-To: <20071112051709.GJ28779@Max.nyc1.dsl.speakeasy.net> References: <369bce3b0711112047w27c98f38wb42fc5e3d4268e87@mail.gmail.com> <369bce3b0711112109t2d3da9di648a1a5d2422f094@mail.gmail.com> <20071112051709.GJ28779@Max.nyc1.dsl.speakeasy.net> Message-ID: <369bce3b0711112216i45124206x7c33e397634c013f@mail.gmail.com> On 11/11/07, Ricky Zhou wrote: > On Sun, Nov 11, 2007 at 09:09:12PM -0800, Thomas Chung wrote: > > Also in http://fedoraproject.org/wiki/FWN/Beats/ArtworkTemp > Fixed. > > Thanks, > Ricky Thank you Ricky. -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From tchung at fedoraproject.org Mon Nov 12 06:43:53 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Sun, 11 Nov 2007 22:43:53 -0800 Subject: UnicodeEncodeError - FWN/Beats/PlanetFedora Message-ID: <369bce3b0711112243x3344d98g2adc6e4d5883e7a2@mail.gmail.com> Sorry guys, http://fedoraproject.org/wiki/FWN/Beats/PlanetFedora -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From tchung at fedoraproject.org Mon Nov 12 08:32:14 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Mon, 12 Nov 2007 00:32:14 -0800 Subject: UnicodeEncodeError Queue Message-ID: <369bce3b0711120032l26185d19h8fd61805e68b5cb4@mail.gmail.com> I think we should have queue page for UnicodeEncodeError :) http://fedoraproject.org/wiki/UnicodeEncodeError Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From ivazqueznet at gmail.com Mon Nov 12 08:47:10 2007 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Mon, 12 Nov 2007 03:47:10 -0500 Subject: UnicodeEncodeError Queue In-Reply-To: <369bce3b0711120032l26185d19h8fd61805e68b5cb4@mail.gmail.com> References: <369bce3b0711120032l26185d19h8fd61805e68b5cb4@mail.gmail.com> Message-ID: <1194857230.3390.0.camel@ignacio.lan> On Mon, 2007-11-12 at 00:32 -0800, Thomas Chung wrote: > I think we should have queue page for UnicodeEncodeError :) I think we should fix the problem. Unfortunately that's easier said than done... -- Ignacio Vazquez-Abrams PLEASE don't CC me; I'm already subscribed -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From tchung at fedoraproject.org Mon Nov 12 08:32:14 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Mon, 12 Nov 2007 00:32:14 -0800 Subject: UnicodeEncodeError Queue Message-ID: <369bce3b0711120032l26185d19h8fd61805e68b5cb4@mail.gmail.com> I think we should have queue page for UnicodeEncodeError :) http://fedoraproject.org/wiki/UnicodeEncodeError Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From sundaram at fedoraproject.org Mon Nov 12 11:18:18 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Mon, 12 Nov 2007 16:48:18 +0530 Subject: Another unicode error Message-ID: <4738367A.7090601@fedoraproject.org> Hi Can someone fix this? http://fedoraproject.org/wiki/SIGs/Games Rahul From paulo.banon at googlemail.com Mon Nov 12 14:28:03 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Mon, 12 Nov 2007 14:28:03 +0000 Subject: Wiki In-Reply-To: <7a41c4bc0711090852g4b5e0fei2d738f238008d060@mail.gmail.com> References: <7a41c4bc0711090112l3806b14m86fb7da8ea739731@mail.gmail.com> <1194614752.15170.78.camel@cutter> <7a41c4bc0711090852g4b5e0fei2d738f238008d060@mail.gmail.com> Message-ID: <7a41c4bc0711120628o7a23f03fy9f5021b088556a17@mail.gmail.com> I've disabled app2 on loadbalancer now, it should pickup the change on the next puppet run. Paulo On Nov 9, 2007 4:52 PM, Paulo Santos wrote: > This change was reverted back so that we have more performance during > the weekend. > Basically the writes will be "kaput", let's see how this goes :) > > > Paulo > > > On Nov 9, 2007 2:25 PM, seth vidal wrote: > > > > > > On Fri, 2007-11-09 at 10:12 +0100, Paulo Santos wrote: > > > Hi all, > > > > > > Now that everything calmed down, i've removed app2 from the wiki > > > loadbalancer so that the editing problems go away. > > > It should be picked up deployed at the next puppet run > > > > > > Please let us know if you still find any problem. > > > > > > > Thanks. > > -sv > > > > > > _______________________________________________ > > Fedora-infrastructure-list mailing list > > Fedora-infrastructure-list at redhat.com > > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > > > From Matt_Domsch at dell.com Wed Nov 14 05:17:12 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Tue, 13 Nov 2007 23:17:12 -0600 Subject: blog praise for Infrastructure / Web teams Message-ID: <20071114051712.GB32042@auslistsprd01.us.dell.com> Good work teams! http://blog.linuxtoday.com/blog/archives/071113-111653.html [snip] November 13, 2007 Spinning a New Kind of Distro When Fedora 8 came out last week, the first thing that came to mind was this was the first release of Fedora in recent memory that didn't have some sort of glitch associated with it. You know the kind: early mirrored versions come out, someone gets wise and then posts it on Digg or Slashdot, people start pulling down ISOs, chaos ensues. This time, however, there was none of that. Announcements went up, mirror sites snapped on, and Fedora 8's first day was smoothly done. This may seem like a weird thing to point out, but I think it offers us a glimpse of the overall efficiency that's becoming more of a factor in the Fedora Project overall. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From blasphemous.mourning at gmail.com Wed Nov 14 21:35:02 2007 From: blasphemous.mourning at gmail.com (Timothy Roberts) Date: Wed, 14 Nov 2007 16:35:02 -0500 Subject: Join Infrastructure Team? Message-ID: Hello, I am interested in joining the Fedora Infrastructure team. Experience: Administration and installation of multiple home desktop and server Gentoo Linux boxes. Configuration of PaX and SELinux on Gentoo Hardened Servers. Sorry that it's all Gentoo box experience, that's just what I used at home prior to the Fedora switch. I'm also very interested in gaining for *NIX admin experience =] So if I can help out it'd be a great learning experience. From skvidal at fedoraproject.org Wed Nov 14 21:39:03 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Wed, 14 Nov 2007 16:39:03 -0500 Subject: Join Infrastructure Team? In-Reply-To: References: Message-ID: <1195076343.23955.87.camel@cutter> On Wed, 2007-11-14 at 16:35 -0500, Timothy Roberts wrote: > Hello, I am interested in joining the Fedora Infrastructure team. > Experience: > Administration and installation of multiple home desktop and server > Gentoo Linux boxes. > Configuration of PaX and SELinux on Gentoo Hardened Servers. > > Sorry that it's all Gentoo box experience, that's just what I used at > home prior to the Fedora switch. > I'm also very interested in gaining for *NIX admin experience =] So if > I can help out it'd be a great learning experience. > Come by #fedora-admin on irc.freenode.org and we can help you find your way to what needs doing. -sv From ricky at fedoraproject.org Thu Nov 15 20:35:57 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Thu, 15 Nov 2007 15:35:57 -0500 Subject: Meeting Log - 2007-11-15 Message-ID: <20071115203557.GA32335@Max.nyc1.dsl.speakeasy.net> 14:59 < skvidal> who wants to have a meeting? 14:59 -!- jcollie [n=jcollie at 161.210.6.122] has joined #fedora-meeting 14:59 -!- kwizart [n=kwizart at fedora/kwizart] has joined #fedora-meeting 14:59 < GeroldKa> what kind of meeting skvidal ? 14:59 * ricky 15:00 < skvidal> fedora-infrastructure meeting :) 15:00 < GeroldKa> ah OK 15:00 < GeroldKa> that's not my $favorite meeting today :-/ 15:01 < abadger1999> Yes, let's. 15:01 < nnm> yep 15:01 -!- MrBawb [i=abob at guppy.drown.org] has joined #fedora-meeting 15:02 * ricky copies from an old log: < mmcgrath> dgilmore jeremy abadger1999 jima paulobanon mbonnet f13 mdomsch warren ricky skvidal ivazquez people I forgot: Ping! 15:02 * mdomsch needs to fix the calendar again after DST change 15:02 < skvidal> ricky: useful :) 15:02 * skvidal is here 15:02 < skvidal> oh crap, I'm supposedly in charge for this week 15:02 * lmacken 15:02 < skvidal> okie doke 15:02 < abadger1999> ha ha 15:02 < jcollie> yo 15:02 < abadger1999> sucker :-) 15:02 -!- G_ [n=njones at wikipedia/NigelJ] has joined #fedora-meeting 15:02 < warren> Even though I fixed the bug, I still fear crashes at this time. 15:02 < skvidal> let's see what the certs say 15:02 < skvidal> s/certs/tickets/ 15:02 -!- clarkbw [i=clarkbw at nat/redhat/x-3b1efb9013e9f272] has quit "shocks the conscience" 15:03 < ricky> https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?status=new&status=assigned&status=reopened&component=Hosted+Projects&order=priority 15:03 < ricky> Oops. 15:03 < ricky> I missed: https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?status=new&status=assigned&status=reopened&group=milestone&keywords=%7EMeeting&order=priority 15:04 < skvidal> so 15:04 < skvidal> guess what 15:04 < skvidal> we're still low on disk space 15:04 -!- hircus [n=michel at dhcp-cs-244-140.cs.indiana.edu] has quit "Ex-Chat" 15:04 < jeremy> shocker 15:04 < skvidal> on the plus side we do have the serverbeach servers and the rest of today, for me, is to get at least one of them online enough for us to use 15:04 < jeremy> go seth go! 15:04 < skvidal> now, once it's all xen and happy I need some suggestions on what we need to get over there first 15:05 < f13> I'm in another meeting :( 15:05 < ricky> Woohoo! 15:05 < mdomsch> skvidal, mirrorlist requests 15:05 < skvidal> we need to have a new app server 15:05 < ricky> So we said mirrormanager (or list at least) 15:05 < mdomsch> e.g. mirrors.fp.o 15:05 < skvidal> mdomsch: can our app servers talk to any db they need or to fas w/o having to be in phx? 15:06 < mdomsch> skvidal, if they can reach db1 over the vpn, then yes 15:06 < mdomsch> if they can't, no dice 15:06 < skvidal> mdomsch: yes, well the vpn is the other issue 15:06 < abadger1999> s/db1/db2/ 15:06 < mdomsch> ok 15:06 < ricky> Can't seem to ping db1/2 from proxy3, at least. 15:06 < mdomsch> w/o the vpn, all we can do is scp the mirrorlist data out to the app servers not in phx 15:07 < skvidal> okay then it seems like I have to either find that password/key for the CA for the vpn 15:07 < mdomsch> fwiw, I'm ok with copying the data out from app3 -> app4 and new serverbeach appX 15:07 < skvidal> but for the long run this needs to be documented 15:08 -!- halfline_ [i=rstrode at nat/redhat/x-d3d704dad926c3de] has joined #fedora-meeting 15:08 < skvidal> setting up a vpn client isn't hard - but the CA signing is difficult :) 15:08 < skvidal> okay, next thing 15:08 < skvidal> ricky: what's the good word? did glezos have to jet? 15:08 * nirik recomends the use of the openvpn 'easy-rsa' scripts. Makes it very easy to handle openvpn certs. 15:09 < skvidal> nirik: yes, that's true 15:09 < skvidal> one problem 15:09 < ricky> skvidal: For transifex? Yeah, I'm waiting a bit on glezos, but I may have to do some digging about that error myself. 15:09 < skvidal> they don't work without the password on the CA 15:09 < nirik> sure. Thats gonna be needed no matter what. ;) 15:09 < skvidal> ricky: okay, thanks - it'd be good to get that back up sooner than later 15:09 -!- mclasen [n=mclasen at c-24-218-44-243.hsd1.ma.comcast.net] has quit "There must be some way out of here." 15:09 < ricky> Definitely. 15:09 < skvidal> ricky: is there anything I can do to help? 15:10 < lmacken> ricky: what error ? 15:10 -!- nim-nim [n=nim-nim at fedora/nim-nim] has joined #fedora-meeting 15:10 < skvidal> mdomsch, abadger1999, ricky, lmacken: what else can we do with what's in serverbeach? 15:11 < ricky> When a translation is submitted, it gives a UnicodeDecodeError. 15:11 -!- JSchmitt [n=s4504kr at p54B12824.dip0.t-ipconnect.de] has joined #fedora-meeting 15:11 < mdomsch> skvidal, another proxy 15:11 < skvidal> I know mike had some specific plans, and that's fine - I just wanted to see if there was anything else pressing 15:11 < lmacken> ricky: fun! I love those 15:11 < mdomsch> skvidal, maybe smolt backend 15:11 < mdomsch> since those boxes are beefy 15:11 < ricky> Heh- funny thing is- it doesn't happen on publictest5 - just app3. 15:11 < skvidal> mdomsch: yah - I'm going to see if I can find my spreadsheet 15:11 -!- wolfy [n=lonewolf at fedora/wolfy] has left #fedora-meeting ["Okay, who stopped the payment on my reality check?"] 15:11 < skvidal> mdomsch: indeed and smolt should be outside of phx,imo, too 15:12 < ricky> Maybe it's an effect of the sqlite => mysql migration. 15:12 < skvidal> okay, we can allocate them in xen once this is happy 15:12 < skvidal> okay next item 15:12 < skvidal> bacula 15:12 < skvidal> we're hitting the wall on bacula 15:12 < skvidal> and I want to know if there is anyone who is more versed in bacula than me 15:13 < skvidal> b/c I have some questions if there are 15:14 -!- G [n=njones at wikipedia/NigelJ] has quit Connection timed out 15:14 < ricky> Guess not at the moment :) 15:14 -!- bpepple [n=bpepple at adsl-75-60-205-159.dsl.wotnoh.sbcglobal.net] has quit "Ex-Chat" 15:15 -!- bpepple [n=bpepple at adsl-75-60-205-159.dsl.wotnoh.sbcglobal.net] has joined #fedora-meeting 15:15 < skvidal> okay 15:15 < skvidal> anything else? b/c if not we can close this early and I'll see if I can be useful 15:15 < ricky> Perhaps #bacula would be able to help a bit, if we can't find somebody. 15:15 < skvidal> ricky: yah - I know - I'll sort it out - I just wish there was more room for errors :) 15:17 -!- smooge [n=smooge at int.smoogespace.com] has quit "-ENOCAFFEINE" 15:17 < skvidal> okay seems like it is quiet, too quiet 15:17 < abadger1999> smolt would benefit from having a local database but I don't think we want to set that up without some thought. 15:18 < mdomsch> be useful 15:18 < skvidal> anythin else? 15:18 < warren> how is the storage going 15:18 < warren> for koji 15:18 < warren> after the purging 15:18 < skvidal> warren: no word since mike is away 15:18 < mdomsch> jima, are your map stats now unique IPs ? 15:18 < warren> ok 15:18 < skvidal> I've not heard anything new from max 15:18 < f13> purging is going well 15:18 < f13> we have another round of purges that will hit in a week or so 15:19 < f13> and that will give us a good indication if we look at the initial purge and the second p urge at what our expected growth will be over time. 15:19 < skvidal> purging is good 15:19 < jima> mdomsch: mirrorlist? yes. 15:19 < ricky> Hm, I don't know anything about this, but would db balancing help smolt? 15:19 < f13> we're obviously growing, but not as much as before. 15:19 < ricky> (Or is that not where the bottleneck is?) 15:20 < abadger1999> ricky: It might. But that's also something that I'd want to think about before doing... replication makes things more complex. 15:20 < f13> I also have word from my manager that once we have that idea of growth, and project out a year, we have budget to get needed storage, and needed backup system. 15:20 < ricky> Aha. 15:20 < f13> so when Mike gets back, I'll work with him on that. 15:21 < abadger1999> And I don't even know if we've added more indexes to the db tables yet. (which would be a much simpler fix to the db bottleneck.) 15:22 -!- JSchmitt [n=s4504kr at fedora/JSchmitt] has quit "Konversation terminated!" 15:23 < ricky> Wasn't there a db person in #fedora-admin working with smolt recently? 15:24 * dgilmore is here now 15:24 < abadger1999> There were a few. loupgaroublond -- do you know if the those db changes were implemented for smolt? 15:26 < ricky> And anybody have an update on anything at http://fedoraproject.org/wiki/Infrastructure/Schedule? 15:31 -!- mether [i=ask at fedora/mether] has quit Read error: 145 (Connection timed out) 15:33 < skvidal> I think we're closed out at this point, actually 15:34 < ricky> OK then :) Sending the log. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From buytenh at wantstofly.org Mon Nov 19 11:01:21 2007 From: buytenh at wantstofly.org (Lennert Buytenhek) Date: Mon, 19 Nov 2007 12:01:21 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? Message-ID: <20071119110121.GA24667@xi.wantstofly.org> Hi, For a while now, I've been maintaining a git conversion of the Fedora CVS tree, pulling in a copy of the CVS tree via rsync, and running some local scripts to convert that to git, incrementally updating the git tree as commits are made to the CVS tree. (For more background info, see here:) https://www.redhat.com/archives/fedora-devel-list/2007-November/msg00561.html I think most of the issues with the conversion have been worked out, and I'd like to make this available to the World in some way. I was wondering whether it makes sense to host something like this on Fedora infrastructure. Note that this is _not_ a proposal to replace CVS by git. The git tree is currently a read-only (slave) version of the CVS tree, and I expect it to stay that way for some time. But even though Fedora isn't switching VCSes at this point, I think it would still make sense to have git/hg/random-other-VCS conversions of the Fedora CVS tree publically available, for a number of reasons: - Give package maintainers the option of working with their favorite VCS for local development (while continuing to use CVS when committing things upstream.) - All the advantages of other version control systems over CVS, e.g.: - Give people the opportunity to pull a local copy of the entire tree or parts of the tree for local browsing of packages and their history without having to go through the server (CVS doesn't support this, although you _could_ just rsync the entire CVS tree to your local machine...) - Allow stacking commits, reverting commits, merging commits, splitting commits, reordering commits, etc., before the changes are pushed into the CVS tree and become final. - Allow easy maintaining of local branches of packages. What would be needed to host this on Fedora infrastructure: - Some disk space. The size of the converted git tree is about 725 megabytes after packing, but for experimenting it would be good to have a bit more space available, say, 10G or so. - Open ports. For browsing the git tree via the web, port 80 access would be needed, and for allowing people to clone the tree over git://, port 9418 access would be useful. - Read-only access to the ,v files in the CVS tree, say, over NFS. Ideas? thanks, Lennert From skvidal at fedoraproject.org Tue Nov 20 03:18:43 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 19 Nov 2007 22:18:43 -0500 Subject: remote vnc-installs Message-ID: <1195528723.30278.11.camel@cutter> Here's the steps I took to reinstall the serverbeach servers: 0. make sure the hosts ip is allowed to access the repos on infrastructure. and make sure puppet allows the host to connect to its fileserver - in the fileserver.conf file in cvs 1. login to existing system as root 2. run: urlgrabber \ http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/vmlinuz \ /boot/vmlinuz-install urlgrabber \ http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/initrd.img \ /boot/initrd-install.img 3. run this command +/- ip address changes and path changes for the kickstart config grubby --add-kernel=/boot/vmlinuz-install \ --args="ks=http://skvidal.fedorapeople.org/hidden/sb1.ks lang= \ devfs=nomount ksdevice=link selinux=0 ip=64.34.163.94 \ gateway=64.34.163.65 netmask=255.255.255.192 \ dns=64.34.160.92,64.34.160.76" --title="install el5" \ --initrd=/boot/initrd-install.img 4. run 'grub' and type: savedefault --default=0 --once quit 5. reboot the system. Watch it via ping, when it starts pinging again after the install run: vncview hostip:1 type in the vnc password - in this case 'vncinstall' That's it. -sv From mmcgrath at redhat.com Tue Nov 20 03:23:47 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 19 Nov 2007 21:23:47 -0600 Subject: Fedora 8 - A release in review. Message-ID: <47425343.9050703@redhat.com> For those on the list that don't actively participate in discussion, Fedora Infrastructure uses time-lines roughly related to the releases that come out. This works out well for us as major changes to our infrastructure typically are aligned well with a release. So in saying that, here's a quick look back on stuff we did for fedora 8. This list is FAR from exhaustive, but gives a good view: Fedora 8 Website: A lot of stuff changed on the Fedora 8 website this time around. Not as drastic of a change as last time in terms of looks, but a lot of changes in how we build it. It has already been translated into many languages. Its been replicated to multiple locations (with one or two more to come). This served us well when PHX fell off the planet during our release. DB1 upgrade: DB1 was our only DB server for a long time. We upgraded the OS and versions of Postgres and Mysql. Then split it up into two boxes. One on DB1 (MySQL) and one on DB2 (Postgres). The two send snapshots of eachother around. Its also worth noting that DB2 is now on better hardware and is virtualized and run over iscsi. Two clicks to live CD: A simple and obvious thing to do but took coordination and work to actually implement. It has made our systems more user friendly and allowed us to completely re-do what download.fedoraproject.org is. Major props to Matt Domsch on that for creating the ability in Mirror Manager. Change Freeze: We officially had a change freeze this time around. Aside from some failed hardware. It went well. We're gaining more discipline in our infrastructure while still remaining adaptable to change. A tricky task. But I hope to see more of it during Fedora 9. Hosted: We implemented a hosted solution (still not quite "official" but being worked on). Think about hosted for a moment. We haven't announced it and basically word of mouth has made it what it is already. We support multiple source control mechanisms and its possibilities are quite limitless. Ticketing system: Otrs is gone, Trac replaced it. Hurray! We actually use this system. Its easy to use and its how I'm getting this list of things we completed :) (note to someone that happens upon this. OTRS is a great tool, just the wrong tool for our needs) New backup system: We're using bacula now instead of backuppc. Its still got some bugs and is in desperate need of a proper storage backend. But it is something we setup and got working. Package Database: Toshio wrote a package database where there just was none before. It's very useful and will continue to become more useful as we integrate our systems. fedorapeople.org: Amazing how popular this turned out to be. People host code, scm's, all sorts of stuff. Its been quite useful. Ibiblio mirror: A small thing but has been useful. We have an ibiblio mirror. It's proved we can run one outside of Red Hat's control and it's even allowed us to fix up some mistakes we made quicker then syncing from Phoenix. Proxy Caching: Probably one of THE most useful (for Fedora Infrastructure) parts of our release this time around has been proxy caching. It has had a major impact on how often our application servers actually serve pages. We've even made things easier on the wiki. Open Infrastructure: We did do some major reorganization during the F8 release. We have more groups then ever, more sponsors then ever and more contributors then ever in Infrastructure. We continue to need to expand a bit and harden our actual procedures but the team is strong and thats good. Voip: Our asterisk server was up and running in time for FudCon. I should remind everyone that this went from "nothingness" to an open working solution in a matter of days and ready for our virtual fudcon. Major props to dgilmore, jcollie and skvidal. In Fedora 9 we'll build some more dedicated boxes to this. Hopefully it can become a major helpful tool for training, meetings, etc. VPN: In response to our decentralization we've decided to implement a vpn solution. Much of it is in place with more to come as we create more machines. Respins: spins.fedoraproject.org is up and running. It's still very basic but as we get more time, we'll be implementing more and more as our new torrent box comes online. start.fedoraproject.org: Well, our end is up anyway :) translate.fedoraproject.org: A brand new for us website that, when combined with transifex, will allow us to bring our software to more people then ever. maps: how cool are maps? http://fedoraproject.org/maps/ We got'em and they're great! Upgrades and other stuff: Even after adding all of this stuff, we've made a lot of our current systems better through virtualization and upgrades. Ok, I'm done. If there's something I've forgotten that you'd like to add. It's a list, reply! I'm incredibly happy with all the work we've done in the last 6 months. Stay tuned for the "Fedora 9 - A release in preview". Start thinking of things that are important to you and what you'd like to see done for Fedora 9. We'll discuss it in that thread, then at the next meeting. -Mike From kwade at redhat.com Tue Nov 20 08:25:09 2007 From: kwade at redhat.com (Karsten Wade) Date: Tue, 20 Nov 2007 00:25:09 -0800 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <20071119110121.GA24667@xi.wantstofly.org> References: <20071119110121.GA24667@xi.wantstofly.org> Message-ID: <1195547109.13156.145.camel@erato.phig.org> On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: > The git tree is currently a read-only (slave) version of the CVS > tree, and I expect it to stay that way for some time. But even though > Fedora isn't switching VCSes at this point, I think it would still > make sense to have git/hg/random-other-VCS conversions of the Fedora > CVS tree publically available ... +1 to the general idea. Are you interested in building and maintaining the solution in Fedora Infrastructure? Just asking as a bystander ... :) - Karsten -- Karsten Wade, Developer Community Mgr. Dev Fu : http://developer.redhatmagazine.com Fedora : http://quaid.fedorapeople.org gpg key : AD0E0C41 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: From amolbagde at yahoo.com Tue Nov 20 08:42:32 2007 From: amolbagde at yahoo.com (amol bagde) Date: Tue, 20 Nov 2007 00:42:32 -0800 (PST) Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? Message-ID: <560460.64368.qm@web63414.mail.re1.yahoo.com> Hi, Yes i m interesting in that. But one thing i want to say about me that i m a fresher. What ever i can i will. Regards, Amol. ----- Original Message ---- From: Karsten Wade To: fedora-infrastructure-list at redhat.com Sent: Tuesday, November 20, 2007 1:55:09 PM Subject: Re: hosting git conversion of Fedora CVS tree on fedora infrastructure? On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: > The git tree is currently a read-only (slave) version of the CVS > tree, and I expect it to stay that way for some time. But even though > Fedora isn't switching VCSes at this point, I think it would still > make sense to have git/hg/random-other-VCS conversions of the Fedora > CVS tree publically available ... +1 to the general idea. Are you interested in building and maintaining the solution in Fedora Infrastructure? Just asking as a bystander ... :) - Karsten -- Karsten Wade, Developer Community Mgr. Dev Fu : http://developer.redhatmagazine.com Fedora : http://quaid.fedorapeople.org gpg key : AD0E0C41 ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmcgrath at redhat.com Tue Nov 20 14:39:20 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 20 Nov 2007 08:39:20 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <1195547109.13156.145.camel@erato.phig.org> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> Message-ID: <4742F198.8030202@redhat.com> Karsten Wade wrote: > On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: > > >> The git tree is currently a read-only (slave) version of the CVS >> tree, and I expect it to stay that way for some time. But even though >> Fedora isn't switching VCSes at this point, I think it would still >> make sense to have git/hg/random-other-VCS conversions of the Fedora >> CVS tree publically available ... >> > > +1 to the general idea. > > Are you interested in building and maintaining the solution in Fedora > Infrastructure? Just asking as a bystander ... :) > What problem are we trying to solve by doing this or what added functionality will we get that people can't get by just downloading a snapshot and importing it into git themselves? -Mike From jim at meyering.net Tue Nov 20 14:54:53 2007 From: jim at meyering.net (Jim Meyering) Date: Tue, 20 Nov 2007 15:54:53 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <20071119110121.GA24667@xi.wantstofly.org> (Lennert Buytenhek's message of "Mon, 19 Nov 2007 12:01:21 +0100") References: <20071119110121.GA24667@xi.wantstofly.org> Message-ID: <87zlx96k76.fsf@rho.meyering.net> Lennert Buytenhek wrote: > For a while now, I've been maintaining a git conversion of the > Fedora CVS tree, pulling in a copy of the CVS tree via rsync, and > running some local scripts to convert that to git, incrementally > updating the git tree as commits are made to the CVS tree. > > (For more background info, see here:) > > https://www.redhat.com/archives/fedora-devel-list/2007-November/msg00561.html Nice! Maintaining an accurate cvs->git mirror is not trivial (I know, from experience with the full repositories of coreutils, emacs, libc, and a few more). If you can do it well and keep it accurate, then it'd make a fine and very welcome service. I've found that the initial conversion works best with cvsparse. It's _much_ more efficient than git-cvsimport, and more reliable, too. From katzj at redhat.com Tue Nov 20 15:43:01 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 20 Nov 2007 10:43:01 -0500 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <4742F198.8030202@redhat.com> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> <4742F198.8030202@redhat.com> Message-ID: <1195573381.4552.11.camel@localhost.localdomain> On Tue, 2007-11-20 at 08:39 -0600, Mike McGrath wrote: > Karsten Wade wrote: > > On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: > >> The git tree is currently a read-only (slave) version of the CVS > >> tree, and I expect it to stay that way for some time. But even though > >> Fedora isn't switching VCSes at this point, I think it would still > >> make sense to have git/hg/random-other-VCS conversions of the Fedora > >> CVS tree publically available ... > > > > +1 to the general idea. > > > > Are you interested in building and maintaining the solution in Fedora > > Infrastructure? Just asking as a bystander ... :) > > What problem are we trying to solve by doing this or what added > functionality will we get that people can't get by just downloading a > snapshot and importing it into git themselves? Downloading a snapshot of what? The entire repo? Do we have that actually available? Even if so, it's still not an incremental grab. Not that I'm fully sold on mirroring our pkgcvs data across multiple SCMs, both from a disk space and a maintenance perspective. Jeremy From mmcgrath at redhat.com Tue Nov 20 15:51:04 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 20 Nov 2007 09:51:04 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <1195573381.4552.11.camel@localhost.localdomain> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> <4742F198.8030202@redhat.com> <1195573381.4552.11.camel@localhost.localdomain> Message-ID: <47430268.5010008@redhat.com> Jeremy Katz wrote: > On Tue, 2007-11-20 at 08:39 -0600, Mike McGrath wrote: > >> Karsten Wade wrote: >> >>> On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: >>> >>>> The git tree is currently a read-only (slave) version of the CVS >>>> tree, and I expect it to stay that way for some time. But even though >>>> Fedora isn't switching VCSes at this point, I think it would still >>>> make sense to have git/hg/random-other-VCS conversions of the Fedora >>>> CVS tree publically available ... >>>> >>> +1 to the general idea. >>> >>> Are you interested in building and maintaining the solution in Fedora >>> Infrastructure? Just asking as a bystander ... :) >>> >> What problem are we trying to solve by doing this or what added >> functionality will we get that people can't get by just downloading a >> snapshot and importing it into git themselves? >> > > Downloading a snapshot of what? The entire repo? Do we have that > actually available? Even if so, it's still not an incremental grab. > > I think the closest is webfiles: http://cvs.fedoraproject.org/webfiles/ -Mike From jim at meyering.net Tue Nov 20 16:01:40 2007 From: jim at meyering.net (Jim Meyering) Date: Tue, 20 Nov 2007 17:01:40 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87zlx96k76.fsf@rho.meyering.net> (Jim Meyering's message of "Tue, 20 Nov 2007 15:54:53 +0100") References: <20071119110121.GA24667@xi.wantstofly.org> <87zlx96k76.fsf@rho.meyering.net> Message-ID: <87oddo7vob.fsf@rho.meyering.net> Jim Meyering wrote: ... > Maintaining an accurate cvs->git mirror is not trivial (I know, from > experience with the full repositories of coreutils, emacs, libc, and a > few more). If you can do it well and keep it accurate, then it'd make > a fine and very welcome service. > > I've found that the initial conversion works best with cvsparse. s/cvsparse/parsecvs/ FYI, sources here git-clone git://people.freedesktop.org/~keithp/parsecvs > It's _much_ more efficient than git-cvsimport, and more reliable, too. From buytenh at wantstofly.org Tue Nov 20 19:45:22 2007 From: buytenh at wantstofly.org (Lennert Buytenhek) Date: Tue, 20 Nov 2007 20:45:22 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87oddo7vob.fsf@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <87zlx96k76.fsf@rho.meyering.net> <87oddo7vob.fsf@rho.meyering.net> Message-ID: <20071120194522.GA10722@xi.wantstofly.org> On Tue, Nov 20, 2007 at 05:01:40PM +0100, Jim Meyering wrote: > > Maintaining an accurate cvs->git mirror is not trivial (I know, from > > experience with the full repositories of coreutils, emacs, libc, and a > > few more). If you can do it well and keep it accurate, then it'd make > > a fine and very welcome service. > > > > I've found that the initial conversion works best with cvsparse. > > s/cvsparse/parsecvs/ That's what I've been using. From buytenh at wantstofly.org Tue Nov 20 19:45:33 2007 From: buytenh at wantstofly.org (Lennert Buytenhek) Date: Tue, 20 Nov 2007 20:45:33 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <1195547109.13156.145.camel@erato.phig.org> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> Message-ID: <20071120194533.GB10722@xi.wantstofly.org> On Tue, Nov 20, 2007 at 12:25:09AM -0800, Karsten Wade wrote: > > The git tree is currently a read-only (slave) version of the CVS > > tree, and I expect it to stay that way for some time. But even though > > Fedora isn't switching VCSes at this point, I think it would still > > make sense to have git/hg/random-other-VCS conversions of the Fedora > > CVS tree publically available ... > > +1 to the general idea. > > Are you interested in building and maintaining the solution in Fedora > Infrastructure? Just asking as a bystander ... :) Yep. From buytenh at wantstofly.org Tue Nov 20 20:02:10 2007 From: buytenh at wantstofly.org (Lennert Buytenhek) Date: Tue, 20 Nov 2007 21:02:10 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <4742F198.8030202@redhat.com> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> <4742F198.8030202@redhat.com> Message-ID: <20071120200210.GC10722@xi.wantstofly.org> On Tue, Nov 20, 2007 at 08:39:20AM -0600, Mike McGrath wrote: > >>The git tree is currently a read-only (slave) version of the CVS > >>tree, and I expect it to stay that way for some time. But even though > >>Fedora isn't switching VCSes at this point, I think it would still > >>make sense to have git/hg/random-other-VCS conversions of the Fedora > >>CVS tree publically available ... > > > >+1 to the general idea. > > > >Are you interested in building and maintaining the solution in Fedora > >Infrastructure? Just asking as a bystander ... :) > > What problem are we trying to solve by doing this or what added > functionality will we get that people can't get by just downloading > a snapshot and importing it into git themselves? As far as I can see, the wegfiles snapshots are CVS checkouts, and don't contain history information. As such, you don't get the ability to view history locally, or to easily work with multiple branches, or some of the other things I described here: https://www.redhat.com/archives/fedora-devel-list/2007-November/msg00561.html (If you wanted to perform the conversion with history on your local machine, you'd have to rsync the entire Fedora CVS tree, weighing in at about 250k files.) From jeff at ocjtech.us Tue Nov 20 20:34:27 2007 From: jeff at ocjtech.us (Jeffrey Ollie) Date: Tue, 20 Nov 2007 14:34:27 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <4742F198.8030202@redhat.com> References: <20071119110121.GA24667@xi.wantstofly.org> <1195547109.13156.145.camel@erato.phig.org> <4742F198.8030202@redhat.com> Message-ID: <935ead450711201234k592e1aacm230ff0ab1247dd7a@mail.gmail.com> On 11/20/07, Mike McGrath wrote: > Karsten Wade wrote: > > On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote: > > > >> The git tree is currently a read-only (slave) version of the CVS > >> tree, and I expect it to stay that way for some time. But even though > >> Fedora isn't switching VCSes at this point, I think it would still > >> make sense to have git/hg/random-other-VCS conversions of the Fedora > >> CVS tree publically available ... > > > > +1 to the general idea. > > > > Are you interested in building and maintaining the solution in Fedora > > Infrastructure? Just asking as a bystander ... :) > > What problem are we trying to solve by doing this I see it as a POC for a future VCS alternative. Now that Koji is getting some support for building from a VCS other than CVS I we are getting to the point where people can actually try something other than CVS for maintaining packages, rather than just talk about it. > or what added > functionality will we get that people can't get by just downloading a > snapshot and importing it into git themselves? The structure and the size of the package CVS makes it difficult for someone to casually make their own git mirror. Jeff From poelstra at redhat.com Wed Nov 21 17:07:50 2007 From: poelstra at redhat.com (John Poelstra) Date: Wed, 21 Nov 2007 09:07:50 -0800 Subject: remote vnc-installs In-Reply-To: <1195528723.30278.11.camel@cutter> References: <1195528723.30278.11.camel@cutter> Message-ID: <474465E6.2060603@redhat.com> seth vidal said the following on 11/19/2007 07:18 PM Pacific Time: > Here's the steps I took to reinstall the serverbeach servers: > Thanks for posting stuff like this! I've been trying to do a setup like this on my home testing network for a while, but kept running into problems. I really appreciate how the Infrastructure team publishes things like this on a regular basis! John From skvidal at fedoraproject.org Wed Nov 21 17:22:26 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Wed, 21 Nov 2007 12:22:26 -0500 Subject: remote vnc-installs In-Reply-To: <474465E6.2060603@redhat.com> References: <1195528723.30278.11.camel@cutter> <474465E6.2060603@redhat.com> Message-ID: <1195665746.2291.4.camel@cutter> On Wed, 2007-11-21 at 09:07 -0800, John Poelstra wrote: > seth vidal said the following on 11/19/2007 07:18 PM Pacific Time: > > Here's the steps I took to reinstall the serverbeach servers: > > > > Thanks for posting stuff like this! I've been trying to do a setup like this on my home testing network for a while, but kept running into problems. > > I really appreciate how the Infrastructure team publishes things like this on a regular basis! > vnc kickstarts are nice b/c if you want to do any step manually it lets you have that option and you get the extra power of the graphical installer vs the somewhat hurky text-mode installer. I just wish there was a way via vnc to see all of the vts during the install. -sv From jkeating at redhat.com Wed Nov 21 17:34:13 2007 From: jkeating at redhat.com (Jesse Keating) Date: Wed, 21 Nov 2007 12:34:13 -0500 Subject: remote vnc-installs In-Reply-To: <1195665746.2291.4.camel@cutter> References: <1195528723.30278.11.camel@cutter> <474465E6.2060603@redhat.com> <1195665746.2291.4.camel@cutter> Message-ID: <20071121123413.22e08b3f@redhat.com> On Wed, 21 Nov 2007 12:22:26 -0500 seth vidal wrote: > I just wish there was a way via vnc to see all of the vts during the > install. There is if you're doing a virt install and exporting the guest via vnc, but not if you're just doing anaconda via vnc. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From skvidal at fedoraproject.org Wed Nov 21 17:34:21 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Wed, 21 Nov 2007 12:34:21 -0500 Subject: remote vnc-installs In-Reply-To: <20071121123413.22e08b3f@redhat.com> References: <1195528723.30278.11.camel@cutter> <474465E6.2060603@redhat.com> <1195665746.2291.4.camel@cutter> <20071121123413.22e08b3f@redhat.com> Message-ID: <1195666461.2291.6.camel@cutter> On Wed, 2007-11-21 at 12:34 -0500, Jesse Keating wrote: > On Wed, 21 Nov 2007 12:22:26 -0500 > seth vidal wrote: > > > I just wish there was a way via vnc to see all of the vts during the > > install. > > There is if you're doing a virt install and exporting the guest via > vnc, but not if you're just doing anaconda via vnc. right - for bare metal, not virt. -sv From Matt_Domsch at Dell.com Wed Nov 21 17:52:34 2007 From: Matt_Domsch at Dell.com (Matt_Domsch at Dell.com) Date: Wed, 21 Nov 2007 11:52:34 -0600 Subject: lockbox daily email 6MB? Message-ID: For those receiving this daily 6MB email from lockbox, is it really necessary? :-) -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From skvidal at fedoraproject.org Wed Nov 21 17:53:11 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Wed, 21 Nov 2007 12:53:11 -0500 Subject: lockbox daily email 6MB? In-Reply-To: References: Message-ID: <1195667591.2291.8.camel@cutter> On Wed, 2007-11-21 at 11:52 -0600, Matt_Domsch at Dell.com wrote: > For those receiving this daily 6MB email from lockbox, is it really necessary? :-) > it's on the list to be nuked from orbit by trimming out the crap. -sv From mmcgrath at redhat.com Wed Nov 21 18:09:17 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 21 Nov 2007 12:09:17 -0600 Subject: lockbox daily email 6MB? In-Reply-To: References: Message-ID: <4744744D.3070706@redhat.com> Matt_Domsch at Dell.com wrote: > For those receiving this daily 6MB email from lockbox, is it really necessary? :-) > FWIW: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 -Mike From lmacken at redhat.com Wed Nov 21 18:30:56 2007 From: lmacken at redhat.com (Luke Macken) Date: Wed, 21 Nov 2007 13:30:56 -0500 Subject: lockbox daily email 6MB? In-Reply-To: <1195667591.2291.8.camel@cutter> References: <1195667591.2291.8.camel@cutter> Message-ID: <20071121183056.GH17076@crow.myhome.westell.com> On Wed, Nov 21, 2007 at 12:53:11PM -0500, seth vidal wrote: > > On Wed, 2007-11-21 at 11:52 -0600, Matt_Domsch at Dell.com wrote: > > For those receiving this daily 6MB email from lockbox, is it really necessary? :-) > > > > it's on the list to be nuked from orbit by trimming out the crap. We're planning to redesign the way we handle this stuff. This will probably entail adding some new feature to epylog. https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 luke From cweyl at alumni.drew.edu Wed Nov 21 20:12:35 2007 From: cweyl at alumni.drew.edu (Chris Weyl) Date: Wed, 21 Nov 2007 12:12:35 -0800 Subject: remote vnc-installs In-Reply-To: <1195528723.30278.11.camel@cutter> References: <1195528723.30278.11.camel@cutter> Message-ID: <7dd7ab490711211212k40c86367p8e318dc933b54e62@mail.gmail.com> On Nov 19, 2007 7:18 PM, seth vidal wrote: > Here's the steps I took to reinstall the serverbeach servers: > > 0. make sure the hosts ip is allowed to access the repos on > infrastructure. and make sure puppet allows the host to connect to its > fileserver - in the fileserver.conf file in cvs > > 1. login to existing system as root > 2. run: > urlgrabber \ > http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/vmlinuz \ /boot/vmlinuz-install I don't follow this list closely, but are we now using RHEL for Fedora infrastructure? Weren't we using Fedora to run the Fedora infrastructure (for obvious reasons)? Why the change? -Chris -- Chris Weyl Ex astris, scientia From a.badger at gmail.com Wed Nov 21 20:23:54 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 21 Nov 2007 12:23:54 -0800 Subject: remote vnc-installs In-Reply-To: <7dd7ab490711211212k40c86367p8e318dc933b54e62@mail.gmail.com> References: <1195528723.30278.11.camel@cutter> <7dd7ab490711211212k40c86367p8e318dc933b54e62@mail.gmail.com> Message-ID: <474493DA.4080100@gmail.com> Chris Weyl wrote: > I don't follow this list closely, but are we now using RHEL for Fedora > infrastructure? Weren't we using Fedora to run the Fedora > infrastructure (for obvious reasons)? Why the change? > We've used a mixture of RHEL and Fedora for as long as I can remember. In general RHEL fits our needs better because it gives us a longer period before it goes EOL which means we don't have to reinstal the servers once a year. When we have a need for a feature that isn't present in RHEL we use Fedora for the service, usually with the idea that we'll migrate the service to RHEL when RHEL rebases to a sufficiently current Fedora. -Toshio From skvidal at fedoraproject.org Wed Nov 21 20:21:01 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Wed, 21 Nov 2007 15:21:01 -0500 Subject: remote vnc-installs In-Reply-To: <7dd7ab490711211212k40c86367p8e318dc933b54e62@mail.gmail.com> References: <1195528723.30278.11.camel@cutter> <7dd7ab490711211212k40c86367p8e318dc933b54e62@mail.gmail.com> Message-ID: <1195676461.2291.10.camel@cutter> On Wed, 2007-11-21 at 12:12 -0800, Chris Weyl wrote: > On Nov 19, 2007 7:18 PM, seth vidal wrote: > > Here's the steps I took to reinstall the serverbeach servers: > > > > 0. make sure the hosts ip is allowed to access the repos on > > infrastructure. and make sure puppet allows the host to connect to its > > fileserver - in the fileserver.conf file in cvs > > > > 1. login to existing system as root > > 2. run: > > urlgrabber \ > > http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/vmlinuz \ /boot/vmlinuz-install > > I don't follow this list closely, but are we now using RHEL for Fedora > infrastructure? Weren't we using Fedora to run the Fedora > infrastructure (for obvious reasons)? Why the change? we've been using a mix based on the service and service-life-time. -sv From mmcgrath at redhat.com Wed Nov 21 21:03:16 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 21 Nov 2007 15:03:16 -0600 Subject: Fedora 9 - A release preview Message-ID: <47449D14.20108@redhat.com> So what do we want to see by the Fedora 9 release? Here's a list I'd like to see: 1) Remove all of our FC6 boxes (either by upgrade or move to RHEL) 2) Separate Test infrastructure - Right now we have people using test boxes that connect to production databases and information. This needs to stop. (I understand luke and dgilmore have had some discussions about this already) 3) Finalized backup solution and koji share (we are out of room for both backups and koji) 4) Further hardening of our systems. Implementing puppet has done a lot of good but there's more that needs to be done. This includes making sure all boxes come up as expected on reboot. Ensuring we have some sort of management system in place for our xen guests that can run on multiple hosts. 5) Further system replication - Anything related to distribution or the primary website (including docs, and mirrors) should be able to run while PHX is down. Its already pretty close to that. 6) New torrent server 7) New collaboration servers 8) Move hosted out of PHX and on to new server beach systems. This will likely include creating a new "hosted" group. 9) FAS2 - This will be a big project and is one I'm hoping to accomplish prior to F9 test1. Ricky has done some great work with it, we'll see what it takes to finish it off. 10) Better systems integration - Many of our systems now support different rss feeds and such. We can more easily integrate these systems together with groups, koji, pkgdb, FAS2, bodhi, you name it. 11) Fewer new systems - This goes along with all the stuff we did to get F7 and F8 ready. For F9 I'd like to see the team take a bit more time taking each project to focus on that last 10%. Its always harder then it seems and with the FAS switchover I feel its important. :: whew :: I'd like to spend time at next week's meeting (I'll likely not be at this weeks meeting) talking about what is important for the individuals in the rest of the team to get done. After that meeting we'll have a F9 milestone in place and populated with tickets. What else do you guys want to do in the next 6 months? Whats important to you not only as a Fedora Infrastructure member, but as a contributor? -Mike From katzj at redhat.com Wed Nov 21 21:32:23 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 21 Nov 2007 16:32:23 -0500 Subject: Fedora 9 - A release preview In-Reply-To: <47449D14.20108@redhat.com> References: <47449D14.20108@redhat.com> Message-ID: <1195680743.4552.121.camel@localhost.localdomain> On Wed, 2007-11-21 at 15:03 -0600, Mike McGrath wrote: > 8) Move hosted out of PHX and on to new server beach systems. This will > likely include creating a new "hosted" group. Somewhat related is that it'd be good to build up new machines per-SCM as opposed to the current fun with chroots on cvs.fedoraproject.org. I should get back to the work I started for replacing git.fedoraproject.org as a first one to tackle Jeremy From mmcgrath at redhat.com Wed Nov 21 21:35:54 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 21 Nov 2007 15:35:54 -0600 Subject: Fedora 9 - A release preview In-Reply-To: <1195680743.4552.121.camel@localhost.localdomain> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> Message-ID: <4744A4BA.2040502@redhat.com> Jeremy Katz wrote: > On Wed, 2007-11-21 at 15:03 -0600, Mike McGrath wrote: > >> 8) Move hosted out of PHX and on to new server beach systems. This will >> likely include creating a new "hosted" group. >> > > Somewhat related is that it'd be good to build up new machines per-SCM > as opposed to the current fun with chroots on cvs.fedoraproject.org. I > should get back to the work I started for replacing > git.fedoraproject.org as a first one to tackle > That's something worth talking about. As it is we run hg, mercurial and git in the same chroot (we're not looking to go with another chroot) but I think we could just run it on the machine, there's a script + ssh_key setup in those chroots for security. I think they'd be just as affective out of a chroot on the hosted boxes. I also wouldn't mind moving the locations from: git://git.fedoraproject.org/git/hosted/fedora-infrastructure.git/ to git://git.fedoraproject.org/fedora-infrastructure.git/ From dennis at ausil.us Wed Nov 21 21:40:36 2007 From: dennis at ausil.us (Dennis Gilmore) Date: Wed, 21 Nov 2007 15:40:36 -0600 Subject: Fedora 9 - A release preview In-Reply-To: <47449D14.20108@redhat.com> References: <47449D14.20108@redhat.com> Message-ID: <200711211540.37230.dennis@ausil.us> On Wednesday 21 November 2007, Mike McGrath wrote: > 9) FAS2 - This will be a big project and is one I'm hoping to accomplish > prior to F9 test1. Ricky has done some great work with it, we'll see > what it takes to finish it off. this will be good to see it finalised > 10) Better systems integration - Many of our systems now support > different rss feeds and such. We can more easily integrate these > systems together with groups, koji, pkgdb, FAS2, bodhi, you name it. use ssl auth as an option across all web based apps. > 11) Fewer new systems - This goes along with all the stuff we did to get > F7 and F8 ready. For F9 I'd like to see the team take a bit more time > taking each project to focus on that last 10%. Its always harder then > it seems and with the FAS switchover I feel its important. > > :: whew :: > > I'd like to spend time at next week's meeting (I'll likely not be at > this weeks meeting) talking about what is important for the individuals > in the rest of the team to get done. After that meeting we'll have a F9 > milestone in place and populated with tickets. > > What else do you guys want to do in the next 6 months? better integration with downstream from us projects. like OLPC not sure exactly how and what yet. get a concrete plan in place to evaluate the use of cvs for package maintenance. and evaluate if we move to something else at all, with a plan to have it implemented for F10 if the decision is to move away from cvs > Whats important to you not only as a Fedora Infrastructure member, but > as a contributor? From katzj at redhat.com Wed Nov 21 21:55:51 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 21 Nov 2007 16:55:51 -0500 Subject: Fedora 9 - A release preview In-Reply-To: <4744A4BA.2040502@redhat.com> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> Message-ID: <1195682151.4552.127.camel@localhost.localdomain> On Wed, 2007-11-21 at 15:35 -0600, Mike McGrath wrote: > Jeremy Katz wrote: > > On Wed, 2007-11-21 at 15:03 -0600, Mike McGrath wrote: > >> 8) Move hosted out of PHX and on to new server beach systems. This will > >> likely include creating a new "hosted" group. > > > > Somewhat related is that it'd be good to build up new machines per-SCM > > as opposed to the current fun with chroots on cvs.fedoraproject.org. I > > should get back to the work I started for replacing > > git.fedoraproject.org as a first one to tackle > > That's something worth talking about. As it is we run hg, mercurial and > git in the same chroot (we're not looking to go with another chroot) but > I think we could just run it on the machine, there's a script + ssh_key > setup in those chroots for security. They could all share the same machine, it just makes the apache config more painful. Running them on separate machines[1] also gives us nice isolation and lets us upgrade each on its own as makes sense for it. It would also make it so that we could do things like enabling the cvspserver frontend to git. I'm not really sold much one way or the other, though. I don't think that we want to run it on the same machine as the hosted frontend, though. > I think they'd be just as > affective out of a chroot on the hosted boxes. I also wouldn't mind > moving the locations from: > > git://git.fedoraproject.org/git/hosted/fedora-infrastructure.git/ > to > git://git.fedoraproject.org/fedora-infrastructure.git/ You should be able to at least do 'git clone git://git.fedoraproject.org/hosted/fedora-infrastructure.git' now. But if you're pulling over ssh, the additional /git is necessary due to the fact that you're ssh'ing and looking at the paths on the filesystem Jeremy [1] Separate virtual machines From laroche at redhat.com Wed Nov 21 22:11:45 2007 From: laroche at redhat.com (Florian La Roche) Date: Wed, 21 Nov 2007 23:11:45 +0100 Subject: Fedora 9 - A release preview In-Reply-To: <1195682151.4552.127.camel@localhost.localdomain> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> <1195682151.4552.127.camel@localhost.localdomain> Message-ID: <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> > You should be able to at least do 'git clone > git://git.fedoraproject.org/hosted/fedora-infrastructure.git' now. But > if you're pulling over ssh, the additional /git is necessary due to the > fact that you're ssh'ing and looking at the paths on the filesystem Why not add an additional symlink from the root dir to make the path consistent? regards, Florian La Roche From a.badger at gmail.com Wed Nov 21 22:23:06 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 21 Nov 2007 14:23:06 -0800 Subject: Fedora 9 - A release preview In-Reply-To: <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> <1195682151.4552.127.camel@localhost.localdomain> <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> Message-ID: <4744AFCA.5020300@gmail.com> Florian La Roche wrote: >> You should be able to at least do 'git clone >> git://git.fedoraproject.org/hosted/fedora-infrastructure.git' now. But >> if you're pulling over ssh, the additional /git is necessary due to the >> fact that you're ssh'ing and looking at the paths on the filesystem > > > Why not add an additional symlink from the root dir to make the > path consistent? > Could be done with separate machines but can't be done now due to all the hosted repositories being on the same machine. ie, we have: /git/hosted /bzr/hosted /hg/hosted /svn/hosted Which of these should a /hosted symlink point to? -Toshio From mmcgrath at redhat.com Wed Nov 21 22:24:37 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 21 Nov 2007 16:24:37 -0600 Subject: Fedora 9 - A release preview In-Reply-To: <4744AFCA.5020300@gmail.com> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> <1195682151.4552.127.camel@localhost.localdomain> <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> <4744AFCA.5020300@gmail.com> Message-ID: <4744B025.1060100@redhat.com> Toshio Kuratomi wrote: > Florian La Roche wrote: >>> You should be able to at least do 'git clone >>> git://git.fedoraproject.org/hosted/fedora-infrastructure.git' now. But >>> if you're pulling over ssh, the additional /git is necessary due to the >>> fact that you're ssh'ing and looking at the paths on the filesystem >> >> >> Why not add an additional symlink from the root dir to make the >> path consistent? >> > Could be done with separate machines but can't be done now due to all > the hosted repositories being on the same machine. > > ie, we have: > /git/hosted > /bzr/hosted > /hg/hosted > /svn/hosted > > Which of these should a /hosted symlink point to? I think he's talking about fedora-infrastructure.git -> /git/hosted/fedora-infrastructure This does raise a good point though, surely there's a git approved way of dealing with this. -Mike From a.badger at gmail.com Wed Nov 21 22:31:20 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 21 Nov 2007 14:31:20 -0800 Subject: Fedora 9 - A release preview In-Reply-To: <47449D14.20108@redhat.com> References: <47449D14.20108@redhat.com> Message-ID: <4744B1B8.5070601@gmail.com> Mike McGrath wrote: > So what do we want to see by the Fedora 9 release? Here's a list I'd > like to see: > > 1) Remove all of our FC6 boxes (either by upgrade or move to RHEL) > For the TurboGears apps, this should be pretty easy to do (FC6 and RHEL5 are very close). > 2) Separate Test infrastructure - Right now we have people using test > boxes that connect to production databases and information. This needs > to stop. (I understand luke and dgilmore have had some discussions about > this already) > We're going to need to get FAS2 up and running and expose information either directly via LDAP or via JSON-RPC for this to work. That's definitely a good thing; I just want to mention that we have to get that working first. > 8) Move hosted out of PHX and on to new server beach systems. This will > likely include creating a new "hosted" group. > Are we going to consider anything in the git/bzr/hg/svn repositories to be hosted for this purpose? A few things in there pre-date tthe existence of hosted. > 9) FAS2 - This will be a big project and is one I'm hoping to accomplish > prior to F9 test1. Ricky has done some great work with it, we'll see > what it takes to finish it off. > Thanks Ricky! This is an important piece as there's potentially a lot of porting that needs to be done once it is finished. -Toshio From mmcgrath at redhat.com Wed Nov 21 22:33:21 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Wed, 21 Nov 2007 16:33:21 -0600 Subject: Fedora 9 - A release preview In-Reply-To: <4744B1B8.5070601@gmail.com> References: <47449D14.20108@redhat.com> <4744B1B8.5070601@gmail.com> Message-ID: <4744B231.2030804@redhat.com> Toshio Kuratomi wrote: >> 8) Move hosted out of PHX and on to new server beach systems. This >> will likely include creating a new "hosted" group. >> > Are we going to consider anything in the git/bzr/hg/svn repositories > to be hosted for this purpose? A few things in there pre-date tthe > existence of hosted. Not sure, I'd like to build a clearly defined list of what we provide as part of "hosted". We can grandfather in whats there I suppose if we can't come to an agreement. What projects specifically are you wondering about? -Mike From a.badger at gmail.com Wed Nov 21 22:59:45 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 21 Nov 2007 14:59:45 -0800 Subject: Fedora 9 - A release preview In-Reply-To: <4744B231.2030804@redhat.com> References: <47449D14.20108@redhat.com> <4744B1B8.5070601@gmail.com> <4744B231.2030804@redhat.com> Message-ID: <4744B861.4030409@gmail.com> Mike McGrath wrote: > Toshio Kuratomi wrote: >>> 8) Move hosted out of PHX and on to new server beach systems. This >>> will likely include creating a new "hosted" group. >>> >> Are we going to consider anything in the git/bzr/hg/svn repositories >> to be hosted for this purpose? A few things in there pre-date tthe >> existence of hosted. > > > Not sure, I'd like to build a clearly defined list of what we provide as > part of "hosted". We can grandfather in whats there I suppose if we > can't come to an agreement. What projects specifically are you > wondering about? > I wonder mostly because I want to figure out if we'll be hosting one git/hg repo after the hosted split-off or two. The things in git and hg but not in a hosted repo are: git: gitreleng: releng gitkernel: dhoward.git/ext3-devel linville/netmerge-2.6.git roland.git steved.git gitzaitcev: zaitcev/linux-2.6-volk.git gitpilgrim: pilgrim gitthemes: bluecurve-classic-metacity-theme fedorabubbles-gdm-theme bluecurve-gdm-theme fedoradna-gdm-theme bluecurve-gnome-theme fedoradna-kdm-theme bluecurve-gtk-themes fedoraflyinghigh-gdm-theme bluecurve-icon-theme fedoraflyinghigh-kdm-theme bluecurve-kde-theme fedora-gnome-theme bluecurve-kdm-theme fedora-icon-theme bluecurve-kwin-theme fedorainfinity-gdm-theme bluecurve-metacity-theme fedorainfinity-screensaver-theme bluecurve-qt-engine fedora-screensaver-theme bluecurve-xmms-skin hg: hgfedora: livecd-devel hgolpc: olpc-artwork--devel qemu-admin--devel sugar--devel system-config-qemu--devel qemu-manager--devel capture-pixmap--devel image--devel mem-monitor--devel qemu-network--devel sdk--devel -Toshio From katzj at redhat.com Wed Nov 21 23:46:00 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 21 Nov 2007 18:46:00 -0500 Subject: Fedora 9 - A release preview In-Reply-To: <4744B025.1060100@redhat.com> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> <1195682151.4552.127.camel@localhost.localdomain> <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> <4744AFCA.5020300@gmail.com> <4744B025.1060100@redhat.com> Message-ID: <1195688760.4552.133.camel@localhost.localdomain> On Wed, 2007-11-21 at 16:24 -0600, Mike McGrath wrote: > Toshio Kuratomi wrote: > > Florian La Roche wrote: > >>> You should be able to at least do 'git clone > >>> git://git.fedoraproject.org/hosted/fedora-infrastructure.git' now. But > >>> if you're pulling over ssh, the additional /git is necessary due to the > >>> fact that you're ssh'ing and looking at the paths on the filesystem > >> > >> Why not add an additional symlink from the root dir to make the > >> path consistent? > >> > > Could be done with separate machines but can't be done now due to all > > the hosted repositories being on the same machine. > > > > ie, we have: > > /git/hosted > > /bzr/hosted > > /hg/hosted > > /svn/hosted > > > > Which of these should a /hosted symlink point to? > > I think he's talking about fedora-infrastructure.git -> > /git/hosted/fedora-infrastructure > > This does raise a good point though, surely there's a git approved way > of dealing with this. The normal way is that you just have the /git or whatnot. The addition of hosted/ in there is purely our own doing :) Jeremy From katzj at redhat.com Wed Nov 21 23:47:19 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 21 Nov 2007 18:47:19 -0500 Subject: Fedora 9 - A release preview In-Reply-To: <4744B861.4030409@gmail.com> References: <47449D14.20108@redhat.com> <4744B1B8.5070601@gmail.com> <4744B231.2030804@redhat.com> <4744B861.4030409@gmail.com> Message-ID: <1195688839.4552.134.camel@localhost.localdomain> On Wed, 2007-11-21 at 14:59 -0800, Toshio Kuratomi wrote: > Mike McGrath wrote: > > Toshio Kuratomi wrote: > >>> 8) Move hosted out of PHX and on to new server beach systems. This > >>> will likely include creating a new "hosted" group. > >>> > >> Are we going to consider anything in the git/bzr/hg/svn repositories > >> to be hosted for this purpose? A few things in there pre-date tthe > >> existence of hosted. > > > > > > Not sure, I'd like to build a clearly defined list of what we provide as > > part of "hosted". We can grandfather in whats there I suppose if we > > can't come to an agreement. What projects specifically are you > > wondering about? > > > I wonder mostly because I want to figure out if we'll be hosting one > git/hg repo after the hosted split-off or two. The things in git and hg > but not in a hosted repo are: I'd really prefer to only have to host one machine running services per SCM if at all possible. And there's no reason it shouldn't be. Jeremy From jkeating at redhat.com Thu Nov 22 00:10:19 2007 From: jkeating at redhat.com (Jesse Keating) Date: Wed, 21 Nov 2007 19:10:19 -0500 Subject: Fedora 9 - A release preview In-Reply-To: <1195688760.4552.133.camel@localhost.localdomain> References: <47449D14.20108@redhat.com> <1195680743.4552.121.camel@localhost.localdomain> <4744A4BA.2040502@redhat.com> <1195682151.4552.127.camel@localhost.localdomain> <20071121221145.GA8443@dudweiler.stuttgart.redhat.com> <4744AFCA.5020300@gmail.com> <4744B025.1060100@redhat.com> <1195688760.4552.133.camel@localhost.localdomain> Message-ID: <20071121191019.5ba8e389@redhat.com> On Wed, 21 Nov 2007 18:46:00 -0500 Jeremy Katz wrote: > The normal way is that you just have the /git or whatnot. The > addition of hosted/ in there is purely our own doing :) It was our doing to separate out the non hosted stuff from the hosted stuff, as I had no idea once we split things off if the hosted stuff would live out on it's own, vs the other git stuff. -- Jesse Keating Fedora -- All my bits are free, are yours? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From wtogami at redhat.com Sat Nov 24 03:55:20 2007 From: wtogami at redhat.com (Warren Togami) Date: Fri, 23 Nov 2007 22:55:20 -0500 Subject: Permanent static-repos Mirror Message-ID: <4747A0A8.5000800@redhat.com> warthog9 asked me to write him this mail regarding setup of permanent mirror of static-repos. Why a static-repos mirror? ========================== It is helpful during the rawhide development cycle for hard-core rawhide developers/testers to update their rawhide boxes directly from the buildroot repos used by koji. For example, during the freeze periods of F8 development I updated my laptop from static-repos several times per day. Sometimes a new build broke something in a very obvious fashion. Due to this advance warning, I prevented a few utterly broken packages (like kernel) from hitting the rawhide nightly tree. Traditional rawhide compose + syncing to mirrors means the build/install/test/report turnaround cycle for rawhide is 1-2 days minimum. If some testers update from static-repos, the turnaround time could be as short as 30 minutes. If this quick turnaround allows more nightly rawhide trees to be installable, this tremendously aids our overall testing ability and ultimate release quality. Of course, it would be a huge mistake to point people directly to koji/static-repos, because we can't afford to bog it down. Thus a caching mirror of static-repos would allow us to more widely advertise the option of updating rawhide directly from static-repos. What do we need? ================ - 1 dedicated IP address. - squid configured as reverse proxy server. - ~15GB disk for squid cache. - Metadata refresh monitoring daemon, can run as non-root anywhere on the Internet. We could run this on a fedoraproject.org server. squid needs to be configured to allow the IP address where the refresh daemon is running to do the PURGE command. - Bandwidth... although not very much, since this is a non-default repo, few people will be using it. What will Warren Provide? ========================= - Sample squid configuration file. - Time and attention to set this up. - Continued maintenance of the refresh monitoring daemon. Since this runs at fedoraproject.org, this requires no attention from warthog9. How does this sound warthog9? Thanks, Warren Togami wtogami at redhat.com From wtogami at redhat.com Sat Nov 24 04:30:19 2007 From: wtogami at redhat.com (Warren Togami) Date: Fri, 23 Nov 2007 23:30:19 -0500 Subject: Hosted creation of new project, i'm stuck Message-ID: <4747A8DB.9090003@redhat.com> http://fedoraproject.org/wiki/Infrastructure/ProjectHosting/CreateNewProject I am trying to follow the directions here to create a new project in hosted. I am stuck at step 6: Edit sudo -u apache vim /srv/web/trac/projects/${PROJECTNAME}/conf/trac.ini add base_url = https://hosted.fedoraproject.org/projects/${PROJECTNAME}/ under the [trac] section /srv/web/trac/projects/InstantMirror/conf/trac.ini This is read-only from app1. Where do I need to modify it? Is this source controlled? Warren Togami wtogami at redhat.com From wtogami at redhat.com Sat Nov 24 10:08:29 2007 From: wtogami at redhat.com (Warren Togami) Date: Sat, 24 Nov 2007 05:08:29 -0500 Subject: Hosted creation of new project, i'm stuck In-Reply-To: <4747A8DB.9090003@redhat.com> References: <4747A8DB.9090003@redhat.com> Message-ID: <4747F81D.9000403@redhat.com> Warren Togami wrote: > http://fedoraproject.org/wiki/Infrastructure/ProjectHosting/CreateNewProject > > > I am trying to follow the directions here to create a new project in > hosted. I am stuck at step 6: > Edit sudo -u apache vim > /srv/web/trac/projects/${PROJECTNAME}/conf/trac.ini add base_url = > https://hosted.fedoraproject.org/projects/${PROJECTNAME}/ under the > [trac] section > > /srv/web/trac/projects/InstantMirror/conf/trac.ini > This is read-only from app1. Where do I need to modify it? Is this > source controlled? > Nevermind, ricky and other folks helped me figure this out. Warren From wtogami at redhat.com Sat Nov 24 10:13:32 2007 From: wtogami at redhat.com (Warren Togami) Date: Sat, 24 Nov 2007 05:13:32 -0500 Subject: MirrorManager: Always_up2date broken? Message-ID: <4747F94C.40609@redhat.com> https://admin.fedoraproject.org/mirrormanager/site/225 Warren's Home Mirror https://admin.fedoraproject.org/mirrormanager/host/584 My Home Mirror https://admin.fedoraproject.org/mirrormanager/host_category/1624 "Fedora Linux" corresponds with http://mirror.example.com/pub/fedora/linux Site-local Netblock set to XXX.XXX.XXX.XXX/32 Always_up2date is checked. http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=i386 After 8+ hours it still shows me a list of random US mirrors. Has anyone managed to get this to work with Always_up2date instead of the cron script? Warren Togami wtogami at redhat.com From michael.yingbull at gmail.com Sat Nov 24 18:54:16 2007 From: michael.yingbull at gmail.com (Michael Yingbull) Date: Sat, 24 Nov 2007 10:54:16 -0800 Subject: Log analyzer improvements, ticket #226 Message-ID: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> Hi all, I'm following up from ticket #226, which is tracking improvements to the log analyzer system. This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines: https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 I wanted to capture what we wanted the new analyzer to do. Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise: the current 'analyzed' logs were too verbose and had too much cruft. Did I capture that requirement? Are there other requirements besides improving the presentation? Anything else that people feel they need from the log analyzer that they aren't getting? Currently Epylog is used - I did some looking around, and I'm not seeing something that looks like its any better. If someone knows another open source log analyzer they think would be much better, I'd like to hear. Else, my plan is to continue with Epylog, reconfigure it... and if really needed to get what we need, patch it and contribute upstream. Thanks all, hope everyone is having a good weekend. Cheers, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From buytenh at wantstofly.org Sat Nov 24 21:51:21 2007 From: buytenh at wantstofly.org (Lennert Buytenhek) Date: Sat, 24 Nov 2007 22:51:21 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <20071119110121.GA24667@xi.wantstofly.org> References: <20071119110121.GA24667@xi.wantstofly.org> Message-ID: <20071124215121.GA31470@xi.wantstofly.org> On Mon, Nov 19, 2007 at 12:01:21PM +0100, Lennert Buytenhek wrote: > I think most of the issues with the conversion have been worked > out, and I'd like to make this available to the World in some way. > I was wondering whether it makes sense to host something like this > on Fedora infrastructure. There seemed to be a couple of folks against this, and a couple in favor, but no decision either way. Anyone else have opinions/ideas about this? From wtogami at redhat.com Sun Nov 25 00:43:11 2007 From: wtogami at redhat.com (Warren Togami) Date: Sat, 24 Nov 2007 19:43:11 -0500 Subject: MirrorManager: Always_up2date broken? In-Reply-To: <4747F94C.40609@redhat.com> References: <4747F94C.40609@redhat.com> Message-ID: <4748C51F.9000901@redhat.com> Warren Togami wrote: > https://admin.fedoraproject.org/mirrormanager/site/225 > Warren's Home Mirror > https://admin.fedoraproject.org/mirrormanager/host/584 > My Home Mirror > https://admin.fedoraproject.org/mirrormanager/host_category/1624 > "Fedora Linux" corresponds with http://mirror.example.com/pub/fedora/linux > > Site-local Netblock set to XXX.XXX.XXX.XXX/32 > > Always_up2date is checked. > > http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=i386 > After 8+ hours it still shows me a list of random US mirrors. > > Has anyone managed to get this to work with Always_up2date instead of > the cron script? https://hosted.fedoraproject.org/projects/mirrormanager/ticket/1#comment:1 Ah... apparently always_up2date is not yet finished. I am trying report_mirror now in an attempt to make it work. Warren Togami wtogami at redhat.com From a.badger at gmail.com Sun Nov 25 21:00:53 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Sun, 25 Nov 2007 13:00:53 -0800 Subject: Restart TG apps for high mem-usage Message-ID: <4749E285.3000400@gmail.com> Here's a short script to test our TG apps run via supervisor for excessive memory usage and restart them if necessary. We could run this via cron in alternate hours on each app server. Does this seem like a good or bad idea to people? -Tosjio -------------- next part -------------- A non-text attachment was scrubbed... Name: restart-memhogs.sh Type: application/x-shellscript Size: 835 bytes Desc: not available URL: From paulo.banon at googlemail.com Sun Nov 25 21:58:00 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Sun, 25 Nov 2007 21:58:00 +0000 Subject: Restart TG apps for high mem-usage In-Reply-To: <4749E285.3000400@gmail.com> References: <4749E285.3000400@gmail.com> Message-ID: <7a41c4bc0711251358u3d665ed3y286b30cab5f7b256@mail.gmail.com> sounds sane to me On Nov 25, 2007 9:00 PM, Toshio Kuratomi wrote: > Here's a short script to test our TG apps run via supervisor for > excessive memory usage and restart them if necessary. We could run this > via cron in alternate hours on each app server. Does this seem like a > good or bad idea to people? > > -Tosjio > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > > From cweyl at alumni.drew.edu Mon Nov 26 00:33:33 2007 From: cweyl at alumni.drew.edu (Chris Weyl) Date: Sun, 25 Nov 2007 16:33:33 -0800 Subject: correct CSS urls? Message-ID: <7dd7ab490711251633nb7ce6cej486a87b6dbd76c7e@mail.gmail.com> Hey all -- I use a number of Fedora CSS sheets in a project of mine[1], specifically: http://admin.fedoraproject.org/css/layout.css http://admin.fedoraproject.org/css/content.css http://admin.fedoraproject.org/css/docbook.css I forget where I originally found them, but I suspect it was from something off of the old cvs.fedora.redhat.com pages. They've been sporadically unavailable lately -- sometimes one or more fails to load; on a page refresh the previous failures tend to succeed and the successes, well, fail. It's irksome :) Am I linking to these incorrectly? Should I be using a different location for them? Thanks- -Chris [1] http://fedora.biggerontheinside.net/perl/ -- Chris Weyl Ex astris, scientia From loupgaroublond at gmail.com Mon Nov 26 01:20:15 2007 From: loupgaroublond at gmail.com (Yaakov Nemoy) Date: Sun, 25 Nov 2007 20:20:15 -0500 Subject: Restart TG apps for high mem-usage In-Reply-To: <4749E285.3000400@gmail.com> References: <4749E285.3000400@gmail.com> Message-ID: <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> On Nov 25, 2007 4:00 PM, Toshio Kuratomi wrote: > Here's a short script to test our TG apps run via supervisor for > excessive memory usage and restart them if necessary. We could run this > via cron in alternate hours on each app server. Does this seem like a > good or bad idea to people? > > -Tosjio +1, but does it make sure all transactions are finished? I know smolt does not have good transaction protection. If a transaction fails halfway through, we might have a mess. -Yaakov From a.badger at gmail.com Mon Nov 26 05:08:30 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Sun, 25 Nov 2007 21:08:30 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> References: <4749E285.3000400@gmail.com> <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> Message-ID: <474A54CE.9060401@gmail.com> Yaakov Nemoy wrote: > On Nov 25, 2007 4:00 PM, Toshio Kuratomi wrote: >> Here's a short script to test our TG apps run via supervisor for >> excessive memory usage and restart them if necessary. We could run this >> via cron in alternate hours on each app server. Does this seem like a >> good or bad idea to people? >> >> -Tosjio > > +1, but does it make sure all transactions are finished? I know smolt > does not have good transaction protection. If a transaction fails > halfway through, we might have a mess. > Not if the app doesn't. From a brief test, TG apps do not do this. The script is asking supervisor to shutdown the application. supervisor sends a TERM to the TG app (we can configure it to send something other than TERM if we want but I don't see any documentation that leads me to believe it will be different with a HUP or QUIT). At that point it looks like a TG app will immediately shutdown and rollback any current transactions. smolt is on shaky ground if it's not using transactions correctly... At the beginning of the month when smolt was getting hit hard we did pretty much this same thing except manually instead of via a script when we noticed that smolt was giving timeouts and taking up 1G+ of RAM. I think the current smolt code is using SQLAlchemy, correct? It's pretty easy to use transactions so that you don't leave the db in an inconsistent state with that configuration. Using the session's implicit transaction flushed just before the return should do the safe thing. You can look through the code later and find additional places where you can safely flush the transaction if there's a need. -Toshio From loupgaroublond at gmail.com Mon Nov 26 05:59:37 2007 From: loupgaroublond at gmail.com (Yaakov Nemoy) Date: Mon, 26 Nov 2007 00:59:37 -0500 Subject: Restart TG apps for high mem-usage In-Reply-To: <474A54CE.9060401@gmail.com> References: <4749E285.3000400@gmail.com> <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> <474A54CE.9060401@gmail.com> Message-ID: <7f692fec0711252159o6d134f89t4da8f83082d80679@mail.gmail.com> On Nov 26, 2007 12:08 AM, Toshio Kuratomi wrote: > Yaakov Nemoy wrote: > > On Nov 25, 2007 4:00 PM, Toshio Kuratomi wrote: > Not if the app doesn't. From a brief test, TG apps do not do this. > > The script is asking supervisor to shutdown the application. supervisor > sends a TERM to the TG app (we can configure it to send something other > than TERM if we want but I don't see any documentation that leads me to > believe it will be different with a HUP or QUIT). At that point it > looks like a TG app will immediately shutdown and rollback any current > transactions. It's got my vote then > smolt is on shaky ground if it's not using transactions correctly... At > the beginning of the month when smolt was getting hit hard we did pretty > much this same thing except manually instead of via a script when we > noticed that smolt was giving timeouts and taking up 1G+ of RAM. I > think the current smolt code is using SQLAlchemy, correct? It's pretty > easy to use transactions so that you don't leave the db in an > inconsistent state with that configuration. Using the session's > implicit transaction flushed just before the return should do the safe > thing. You can look through the code later and find additional places > where you can safely flush the transaction if there's a need. We do use transactions where we can, but since most of the code is not tested at all, let alone stress tested, i can't vouch for it doing The Right Thing. (Winter break is only a few weeks away) -Yaakov From ricky at fedoraproject.org Mon Nov 26 12:56:08 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Mon, 26 Nov 2007 07:56:08 -0500 Subject: correct CSS urls? In-Reply-To: <7dd7ab490711251633nb7ce6cej486a87b6dbd76c7e@mail.gmail.com> References: <7dd7ab490711251633nb7ce6cej486a87b6dbd76c7e@mail.gmail.com> Message-ID: <20071126125608.GF5243@Max.nyc1.dsl.speakeasy.net> On 2007-11-25 04:33:33 PM, Chris Weyl wrote: > They've been sporadically unavailable lately -- sometimes one or more > fails to load; on a page refresh the previous failures tend to succeed > and the successes, well, fail. It's irksome :) Yeah- I had been noticing that as well. One of the proxies was missing a directory, which should be copied over now. Thanks, Ricky -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From Roberto.Quagliozzi at edftrading.com Mon Nov 26 14:32:35 2007 From: Roberto.Quagliozzi at edftrading.com (Roberto.Quagliozzi at edftrading.com) Date: Mon, 26 Nov 2007 14:32:35 +0000 Subject: Roberto Quagliozzi is out of the office. Message-ID: I will be out of the office starting 26/11/2007 and will not return until 30/11/2007. If you have an urgent query please call the helpdesk on x4999 or email enterprise. ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ From mmcgrath at redhat.com Mon Nov 26 14:33:27 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 26 Nov 2007 08:33:27 -0600 Subject: Log analyzer improvements, ticket #226 In-Reply-To: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> References: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> Message-ID: <474AD937.5050001@redhat.com> Michael Yingbull wrote: > Hi all, > > I'm following up from ticket #226, which is tracking improvements to the log > analyzer system. > This would be what analyzers the logs on lockbox, which is the syslog host > for infrastructure machines: > https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 > > I wanted to capture what we wanted the new analyzer to do. > Main feedback I had from discussion in #fedora-admin was a need for more > signal, less noise: > the current 'analyzed' logs were too verbose and had too much cruft. > > Did I capture that requirement? > I think this is the biggest thing. Obviously we don't want to /dev/null log lines but at the same time the current format is pretty useless to us. I guess it might be best to do as much cleanup as possible and then see where things are. -Mike From skvidal at fedoraproject.org Mon Nov 26 14:35:52 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 26 Nov 2007 09:35:52 -0500 Subject: Log analyzer improvements, ticket #226 In-Reply-To: <474AD937.5050001@redhat.com> References: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> <474AD937.5050001@redhat.com> Message-ID: <1196087752.15420.3.camel@cutter> On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote: > Michael Yingbull wrote: > > Hi all, > > > > I'm following up from ticket #226, which is tracking improvements to the log > > analyzer system. > > This would be what analyzers the logs on lockbox, which is the syslog host > > for infrastructure machines: > > https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 > > > > I wanted to capture what we wanted the new analyzer to do. > > Main feedback I had from discussion in #fedora-admin was a need for more > > signal, less noise: > > the current 'analyzed' logs were too verbose and had too much cruft. > > > > Did I capture that requirement? > > > > I think this is the biggest thing. Obviously we don't want to /dev/null > log lines but at the same time the current format is pretty useless to > us. I guess it might be best to do as much cleanup as possible and then > see where things are. Actually, there's a huge portion of what is in the current logs that needs to either: 1. be dumped out by epylog's weeder 2. be stopped from occurring on the system generating the message. Michael, if you need any assistance with this, let me know, I have a fair bit of experience adding weedlists to epylog. -sv From mmcgrath at redhat.com Mon Nov 26 14:42:50 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 26 Nov 2007 08:42:50 -0600 Subject: Log analyzer improvements, ticket #226 In-Reply-To: <1196087752.15420.3.camel@cutter> References: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> <474AD937.5050001@redhat.com> <1196087752.15420.3.camel@cutter> Message-ID: <474ADB6A.80205@redhat.com> seth vidal wrote: > On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote: > >> Michael Yingbull wrote: >> >>> Hi all, >>> >>> I'm following up from ticket #226, which is tracking improvements to the log >>> analyzer system. >>> This would be what analyzers the logs on lockbox, which is the syslog host >>> for infrastructure machines: >>> https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226 >>> >>> I wanted to capture what we wanted the new analyzer to do. >>> Main feedback I had from discussion in #fedora-admin was a need for more >>> signal, less noise: >>> the current 'analyzed' logs were too verbose and had too much cruft. >>> >>> Did I capture that requirement? >>> >>> >> I think this is the biggest thing. Obviously we don't want to /dev/null >> log lines but at the same time the current format is pretty useless to >> us. I guess it might be best to do as much cleanup as possible and then >> see where things are. >> > > Actually, there's a huge portion of what is in the current logs that > needs to either: > 1. be dumped out by epylog's weeder > 2. be stopped from occurring on the system generating the message. > > Michael, if you need any assistance with this, let me know, I have a > fair bit of experience adding weedlists to epylog. > 2) would be more favored by me where possible. -Mike From skvidal at fedoraproject.org Mon Nov 26 15:40:46 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Mon, 26 Nov 2007 10:40:46 -0500 Subject: Log analyzer improvements, ticket #226 In-Reply-To: <474ADB6A.80205@redhat.com> References: <8c0317740711241054k47bbbaffv4b6dcf65548e1024@mail.gmail.com> <474AD937.5050001@redhat.com> <1196087752.15420.3.camel@cutter> <474ADB6A.80205@redhat.com> Message-ID: <1196091646.15420.31.camel@cutter> On Mon, 2007-11-26 at 08:42 -0600, Mike McGrath wrote: > 2) would be more favored by me where possible. > No problem. From todays' report a couple of things we can do: 1. remove all user failure reports. They don't do us any good and they're always ssh bruteforce attacks. Denyhosts will do its thing, or not, but we can't be told about them all the time. 2. weed out pretty much everything beginning with: rsyncd - informational messages about rsync processes - not useful puppetd - notices on what is or is not done - not useful, either - if we can turn off the syslog component of this and only have this in the local puppet logs that'd be fine ntpd - garbage noise - not useful for a log report git-daemon - do I really need to explain why we can nuke this? 3. all of these lines: crond[19403]: pam_unix(crond:session): session closed for user root iirc, there is a new login module which handles these 4. puppetmasterd* - these appear to be errors/warnings from puppetmasterd - these need to be fixed. pruning out the items in 2 alone will nuke the better part of this logreport. -sv From Matt_Domsch at dell.com Mon Nov 26 17:16:21 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Mon, 26 Nov 2007 11:16:21 -0600 Subject: Restart TG apps for high mem-usage In-Reply-To: <474A54CE.9060401@gmail.com> References: <4749E285.3000400@gmail.com> <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> <474A54CE.9060401@gmail.com> Message-ID: <20071126171621.GA12670@auslistsprd01.us.dell.com> On Sun, Nov 25, 2007 at 09:08:30PM -0800, Toshio Kuratomi wrote: > >+1, but does it make sure all transactions are finished? I know smolt > >does not have good transaction protection. If a transaction fails > >halfway through, we might have a mess. > > > Not if the app doesn't. From a brief test, TG apps do not do this. MirrorManager doesn't use transactions, I never figured out how to get them to work right. Advice welcome. -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From a.badger at gmail.com Mon Nov 26 17:59:44 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 26 Nov 2007 09:59:44 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <20071126171621.GA12670@auslistsprd01.us.dell.com> References: <4749E285.3000400@gmail.com> <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> <474A54CE.9060401@gmail.com> <20071126171621.GA12670@auslistsprd01.us.dell.com> Message-ID: <474B0990.9020400@gmail.com> Matt Domsch wrote: > On Sun, Nov 25, 2007 at 09:08:30PM -0800, Toshio Kuratomi wrote: >>> +1, but does it make sure all transactions are finished? I know smolt >>> does not have good transaction protection. If a transaction fails >>> halfway through, we might have a mess. >>> >> Not if the app doesn't. From a brief test, TG apps do not do this. > > MirrorManager doesn't use transactions, I never figured out how to get > them to work right. Advice welcome. > By not being able to get transactions working, do you mean explicit transactions or implicit transactions? I see that mirrormanager, bodhi, and noc (not running currently) are using a dburi that disables implicit transactions:: mirrormanager-prod.cfg.erb: sqlobject.dburi="notrans_postgres://mirroradmin: <%= mirrorPassword %>@db2.fedora.phx.redhat.com/mirrormanager" If that was changed to:: sqlobject.dburi="postgres://mirroradmin:[...] TurboGears would at least attempt to use an implicit transaction per http request which should protect the database from shutting down the application in the middle of processing a multi-table update. I don't know if that's the problem you're referring to, though. -Toshio From notting at redhat.com Mon Nov 26 18:17:45 2007 From: notting at redhat.com (Bill Nottingham) Date: Mon, 26 Nov 2007 13:17:45 -0500 Subject: Restart TG apps for high mem-usage In-Reply-To: <4749E285.3000400@gmail.com> References: <4749E285.3000400@gmail.com> Message-ID: <20071126181745.GB15244@nostromo.devel.redhat.com> Toshio Kuratomi (a.badger at gmail.com) said: > Here's a short script to test our TG apps run via supervisor for excessive > memory usage and restart them if necessary. We could run this via cron in > alternate hours on each app server. Does this seem like a good or bad idea > to people? It's a good idea if it's needed, but it's a bad idea that it is needed. What's wrong with TG that it leads to this situation? Bill From mmcgrath at redhat.com Mon Nov 26 19:23:48 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 26 Nov 2007 13:23:48 -0600 Subject: Restart TG apps for high mem-usage In-Reply-To: <20071126181745.GB15244@nostromo.devel.redhat.com> References: <4749E285.3000400@gmail.com> <20071126181745.GB15244@nostromo.devel.redhat.com> Message-ID: <474B1D44.9050204@redhat.com> Bill Nottingham wrote: > Toshio Kuratomi (a.badger at gmail.com) said: > >> Here's a short script to test our TG apps run via supervisor for excessive >> memory usage and restart them if necessary. We could run this via cron in >> alternate hours on each app server. Does this seem like a good or bad idea >> to people? >> > > It's a good idea if it's needed, but it's a bad idea that it is needed. What's > wrong with TG that it leads to this situation? > I was wondering this myself, I know smolt recently had some major changes to keep memory usage down. Which TG apps are having this issue and how often? I know MM uses a lot of memory but, AFAIK, it was determined that there's not much of a leak if there is one and that all of that memory is actually used. -Mike From a.badger at gmail.com Mon Nov 26 19:48:19 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 26 Nov 2007 11:48:19 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <20071126181745.GB15244@nostromo.devel.redhat.com> References: <4749E285.3000400@gmail.com> <20071126181745.GB15244@nostromo.devel.redhat.com> Message-ID: <474B2303.3010501@gmail.com> Bill Nottingham wrote: > Toshio Kuratomi (a.badger at gmail.com) said: >> Here's a short script to test our TG apps run via supervisor for excessive >> memory usage and restart them if necessary. We could run this via cron in >> alternate hours on each app server. Does this seem like a good or bad idea >> to people? > > It's a good idea if it's needed, but it's a bad idea that it is needed. What's > wrong with TG that it leads to this situation? > We have small memory leaks with all our TG apps. This has only been a problem for mirrormanager in the past and smolt this past month. At first I thought that the leaks were directly related to how many requests were served (mirrormanager had troubles when it was serving every request for a mirror and smolt had trouble when it was serving the updating clients at the beginning of this month.) I thought that it was caused purely by the number of requests served, however, the packagedb has been serving a large number of requests lately and hasn't climbed at nearly the same rate. So something in the design of smolt and mirrormanager is leaking beyond the baseline that we're seeing with all the TG apps. -Toshio From a.badger at gmail.com Mon Nov 26 20:38:42 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 26 Nov 2007 12:38:42 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <474B1D44.9050204@redhat.com> References: <4749E285.3000400@gmail.com> <20071126181745.GB15244@nostromo.devel.redhat.com> <474B1D44.9050204@redhat.com> Message-ID: <474B2ED2.1070805@gmail.com> Mike McGrath wrote: > Bill Nottingham wrote: >> Toshio Kuratomi (a.badger at gmail.com) said: >>> Here's a short script to test our TG apps run via supervisor for >>> excessive memory usage and restart them if necessary. We could run >>> this via cron in alternate hours on each app server. Does this seem >>> like a good or bad idea to people? >>> >> >> It's a good idea if it's needed, but it's a bad idea that it is >> needed. What's >> wrong with TG that it leads to this situation? >> > > I was wondering this myself, I know smolt recently had some major > changes to keep memory usage down. Which TG apps are having this issue > and how often? I know MM uses a lot of memory but, AFAIK, it was > determined that there's not much of a leak if there is one and that all > of that memory is actually used. > Looks like smolt was upgraded just before Thanksgiving so it could be that we've plugged the leaks we had to deal with that inspired me to write this. Would it be a good idea to have this in place anyways? With it periodically checking, we would find out that we had problems when cron emails us a notice that the script had to restart a process. Without it, we'll be notified when nagios or a user tells us they're getting timeouts. I noticed that mirrormanager is currently at 761MB of RSS. If that's steady-state for mm we'd want to bump the value the script checks for a bit higher before deploying it or set different values per app. -Toshio From mmcgrath at redhat.com Mon Nov 26 20:42:50 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Mon, 26 Nov 2007 14:42:50 -0600 Subject: Restart TG apps for high mem-usage In-Reply-To: <474B2ED2.1070805@gmail.com> References: <4749E285.3000400@gmail.com> <20071126181745.GB15244@nostromo.devel.redhat.com> <474B1D44.9050204@redhat.com> <474B2ED2.1070805@gmail.com> Message-ID: <474B2FCA.3090405@redhat.com> Toshio Kuratomi wrote: > Mike McGrath wrote: >> Bill Nottingham wrote: >>> Toshio Kuratomi (a.badger at gmail.com) said: >>>> Here's a short script to test our TG apps run via supervisor for >>>> excessive memory usage and restart them if necessary. We could run >>>> this via cron in alternate hours on each app server. Does this >>>> seem like a good or bad idea to people? >>>> >>> >>> It's a good idea if it's needed, but it's a bad idea that it is >>> needed. What's >>> wrong with TG that it leads to this situation? >>> >> >> I was wondering this myself, I know smolt recently had some major >> changes to keep memory usage down. Which TG apps are having this >> issue and how often? I know MM uses a lot of memory but, AFAIK, it >> was determined that there's not much of a leak if there is one and >> that all of that memory is actually used. >> > Looks like smolt was upgraded just before Thanksgiving so it could be > that we've plugged the leaks we had to deal with that inspired me to > write this. Would it be a good idea to have this in place anyways? > With it periodically checking, we would find out that we had problems > when cron emails us a notice that the script had to restart a process. > Without it, we'll be notified when nagios or a user tells us they're > getting timeouts. I think its a good idea if for no other reason then allows us to more actively monitor this stuff, we'll get notified when the app restarts. +1 from me with the intention that, over time, we get fewer and fewer restarts. -Mike From a.badger at gmail.com Mon Nov 26 21:07:09 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 26 Nov 2007 13:07:09 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <474B2FCA.3090405@redhat.com> References: <4749E285.3000400@gmail.com> <20071126181745.GB15244@nostromo.devel.redhat.com> <474B1D44.9050204@redhat.com> <474B2ED2.1070805@gmail.com> <474B2FCA.3090405@redhat.com> Message-ID: <474B357D.2030806@gmail.com> Mike McGrath wrote: > Toshio Kuratomi wrote: >> Mike McGrath wrote: >>> Bill Nottingham wrote: >>>> Toshio Kuratomi (a.badger at gmail.com) said: >>>>> Here's a short script to test our TG apps run via supervisor for >>>>> excessive memory usage and restart them if necessary. We could run >>>>> this via cron in alternate hours on each app server. Does this >>>>> seem like a good or bad idea to people? >>>>> >>>> >>>> It's a good idea if it's needed, but it's a bad idea that it is >>>> needed. What's >>>> wrong with TG that it leads to this situation? >>>> >>> >>> I was wondering this myself, I know smolt recently had some major >>> changes to keep memory usage down. Which TG apps are having this >>> issue and how often? I know MM uses a lot of memory but, AFAIK, it >>> was determined that there's not much of a leak if there is one and >>> that all of that memory is actually used. >>> >> Looks like smolt was upgraded just before Thanksgiving so it could be >> that we've plugged the leaks we had to deal with that inspired me to >> write this. Would it be a good idea to have this in place anyways? >> With it periodically checking, we would find out that we had problems >> when cron emails us a notice that the script had to restart a process. >> Without it, we'll be notified when nagios or a user tells us they're >> getting timeouts. > > I think its a good idea if for no other reason then allows us to more > actively monitor this stuff, we'll get notified when the app restarts. > +1 from me with the intention that, over time, we get fewer and fewer > restarts. > Cool. I'll check it in and set up a cron job. One further piece of information since I have output from testing this on app3 yesterday: AppName Uptime RSS 11/25 RSS 11/26 mirrormanager 2d4h 714336 962268 packagedb --- restarted 13h ago -- smolt 5d3h 299556 299556 transifex 5d3h 42744 42768 -Toshio From skvidal at fedoraproject.org Tue Nov 27 05:59:23 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Tue, 27 Nov 2007 00:59:23 -0500 Subject: [Fwd: mirrors.fedoraproject.org/mirrorlist: HTTP Error 500: Internal Server Error] Message-ID: <1196143163.15420.72.camel@cutter> Forward regarding mirrorlist failures and maybe we should fix that contact email address from sysadmin-devel at rh.c to admin at fp.o -sv -------- Forwarded Message -------- From: Ralf Corsepius To: sysadmin-devel at redhat.com Cc: seth vidal Subject: mirrors.fedoraproject.org/mirrorlist: HTTP Error 500: Internal Server Error Date: Tue, 27 Nov 2007 05:51:38 +0100 Accessing http://mirrors.fedoraproject.org/mirrorlist returns this: Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator, sysadmin-devel at redhat.com and inform them of the time the error occurred, and anything you might have done that may have caused the error. More information about this error may be available in the server error log. ________________________________________________________________________ Apache/2.2.6 (Fedora) Server at app3.fedora.phx.redhat.com Port 80 With http://mirrors.fedoraproject.org/mirrorlist being the default and central bottleneck in fedora configuations, this probably affects all fedora users world-wide, who are trying to use yum: # yum update ... Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=i386 error was [Errno 14] HTTP Error 500: Internal Server Error Error: Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again # repoquery -q --whatrequires 'perl(File::NCopy)' Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=i386 error was [Errno 14] HTTP Error 500: Internal Server Error Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again # date -u Tue Nov 27 04:51:27 UTC 2007 Ralf From skvidal at fedoraproject.org Tue Nov 27 06:48:53 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Tue, 27 Nov 2007 01:48:53 -0500 Subject: manifests/nodes xen6.fedora.phx.redhat.com.pp,1.3,1.4 In-Reply-To: <200711270617.lAR6H1nf022827@puppet.fedoraproject.org> References: <200711270617.lAR6H1nf022827@puppet.fedoraproject.org> Message-ID: <1196146133.15420.83.camel@cutter> On Mon, 2007-11-26 at 23:17 -0700, Seth Vidal wrote: > Author: skvidal > > Update of /cvs/puppet/manifests/nodes > In directory puppet1.fedora.phx.redhat.com:/home/fedora/skvidal/manifests/nodes > > Modified Files: > xen6.fedora.phx.redhat.com.pp > Log Message: > make xen6 stop eating the iptable allow to make it's /u01/bacula export work > this change was to keep bacula from getting fatal errors b/c of the /u01/bacula export off of xen6 was not being allowed to be mounted from xen5. -sv From a.badger at gmail.com Tue Nov 27 17:14:21 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 27 Nov 2007 09:14:21 -0800 Subject: Our Web Apps and SSL Message-ID: <474C506D.7080402@gmail.com> I've had this in the back of my mind for a while but only looked at it yesterday. I think we have a potential problem with the way kojiweb is using SSL. To a lesser extent it affects our TurboGears apps as well. = Koji = Kojiweb uses SSL to authenticate the client. This is fine. Kojiweb then stores a session cookie on the client's machine so the client doesn't have to go through the auth mechanism on every transaction. This is also fine. However, kojiweb does not require that this cookie be sent back to the server via SSL and when you initially hit koji via a non-SSL connection only the authentication itself uses SSL. koji sends the session cookie over an unencrypted connection. This leaves koji open to packet sniffing and man-in-the-middle attacks. To prevent this we should be doing two things: 1) Set the session cookie's secure flag to True 2) Once logged in, return the user to an https URL rather than http. = TurboGears = Our TurboGears apps are all running behind https://admin.fedoraproject.org so they have to use an SSL link in order to pull up content. However, the plain http link is active; it just redirects to the SSL page. This means that if you log in and then explicitly request a plain http URL the session cookie will be returned to the server over an unencrypted connection. This is not too bad as the TG servers should be setup to return https links (so someone would have to actually change the URL to http after logging in) but it is a hole. I sent an email last month to say that we'd be upgrading to TG-1.0.3 to close this hole but dropped the ball on actually doing the upgrade. I'll be doing that today; please let me know if you experience any strange problems with your web application and we'll try to work out if it's TG-1.0.3 related. -Toshio From a.badger at gmail.com Tue Nov 27 19:13:19 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 27 Nov 2007 11:13:19 -0800 Subject: Our Web Apps and SSL In-Reply-To: <474C506D.7080402@gmail.com> References: <474C506D.7080402@gmail.com> Message-ID: <474C6C4F.8010504@gmail.com> Toshio Kuratomi wrote: > I've had this in the back of my mind for a while but only looked at it > yesterday. I think we have a potential problem with the way kojiweb is > using SSL. To a lesser extent it affects our TurboGears apps as well. > > = Koji = > Ticket opened with a patch for one of the two portions of the fix: https://hosted.fedoraproject.org/projects/koji/attachment/ticket/64 > = TurboGears = TurboGears upgraded on the app servers and all apps have been restarted with a config file to set the secure flag in the cookie. Let me know if this breaks anything. -Toshio From mmcgrath at redhat.com Tue Nov 27 19:23:15 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 27 Nov 2007 13:23:15 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <20071119110121.GA24667@xi.wantstofly.org> References: <20071119110121.GA24667@xi.wantstofly.org> Message-ID: <474C6EA3.1000107@redhat.com> Lennert Buytenhek wrote: > Hi, > > For a while now, I've been maintaining a git conversion of the > Fedora CVS tree, pulling in a copy of the CVS tree via rsync, and > running some local scripts to convert that to git, incrementally > updating the git tree as commits are made to the CVS tree. > > (For more background info, see here:) > > https://www.redhat.com/archives/fedora-devel-list/2007-November/msg00561.html > > I think most of the issues with the conversion have been worked > out, and I'd like to make this available to the World in some way. > I was wondering whether it makes sense to host something like this > on Fedora infrastructure. > > Note that this is _not_ a proposal to replace CVS by git. > > The git tree is currently a read-only (slave) version of the CVS > tree, and I expect it to stay that way for some time. But even though > Fedora isn't switching VCSes at this point, I think it would still > make sense to have git/hg/random-other-VCS conversions of the Fedora > CVS tree publically available, for a number of reasons: > - Give package maintainers the option of working with their favorite > VCS for local development (while continuing to use CVS when > committing things upstream.) > - All the advantages of other version control systems over CVS, e.g.: > - Give people the opportunity to pull a local copy of the entire > tree or parts of the tree for local browsing of packages and > their history without having to go through the server (CVS > doesn't support this, although you _could_ just rsync the > entire CVS tree to your local machine...) > - Allow stacking commits, reverting commits, merging commits, > splitting commits, reordering commits, etc., before the changes > are pushed into the CVS tree and become final. > - Allow easy maintaining of local branches of packages. > > What would be needed to host this on Fedora infrastructure: > - Some disk space. The size of the converted git tree is about 725 > megabytes after packing, but for experimenting it would be good to > have a bit more space available, say, 10G or so. > - Open ports. For browsing the git tree via the web, port 80 access > would be needed, and for allowing people to clone the tree over > git://, port 9418 access would be useful. > - Read-only access to the ,v files in the CVS tree, say, over NFS. > > Ideas? > > This sounds like something we could do, but I don't think we should do it. For one, after we move the hosted stuff away from cvs-int, that box will get a complete makover and one that, as far as I can tell, won't include git. I'm not sure what problem this solves really. If there is a real need in Fedora to use git, why not just put together a proposal for migrating cvs to git? Lots of people have tried this, its not an easy task. But adding an additional SCM for GIT which is JUST a copy of what's in CVS sounds like a waste of our resources. Why not also do SVN, BZR and Mercurial? -Mike From jim at meyering.net Tue Nov 27 19:38:16 2007 From: jim at meyering.net (Jim Meyering) Date: Tue, 27 Nov 2007 20:38:16 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474C6EA3.1000107@redhat.com> (Mike McGrath's message of "Tue, 27 Nov 2007 13:23:15 -0600") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> Message-ID: <87r6ibbhsn.fsf@rho.meyering.net> Mike McGrath wrote: > This sounds like something we could do, but I don't think we should do > it. For one, after we move the hosted stuff away from cvs-int, that > box will get a complete makover and one that, as far as I can tell, > won't include git. I'm not sure what problem this solves really. > > If there is a real need in Fedora to use git, why not just put > together a proposal for migrating cvs to git? Lots of people have Hi Mike, That's a much bigger shift, and will probably take a long time to realize. On the other hand, providing a service like this is easy, and might help nay-sayers realize the value. In the mean time, people who find it useful get the benefit right away. > tried this, its not an easy task. But adding an additional SCM for > GIT which is JUST a copy of what's in CVS sounds like a waste of our > resources. Why not also do SVN, BZR and Mercurial? IMHO, they're not as useful. If Fedora doesn't want to do this, I can probably set up something independent and provide public git:// access. From mmcgrath at redhat.com Tue Nov 27 19:39:48 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 27 Nov 2007 13:39:48 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87r6ibbhsn.fsf@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> Message-ID: <474C7284.8060501@redhat.com> Jim Meyering wrote: > Mike McGrath wrote: > >> tried this, its not an easy task. But adding an additional SCM for >> GIT which is JUST a copy of what's in CVS sounds like a waste of our >> resources. Why not also do SVN, BZR and Mercurial? >> > > IMHO, they're not as useful. > > And thats the real trick, I'd imagine the mercurial, svn and bzr guys would disagree with you. > If Fedora doesn't want to do this, I can probably set up > something independent and provide public git:// access. > If someone else wants to host it I'm all for it, we can certainly make it easier to get at the raw CVS repo. If the other officers disagree please let it be known, but this sounds more like a distraction/one off then something that adds value to our infrastructure. -Mike From skvidal at fedoraproject.org Tue Nov 27 19:40:23 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Tue, 27 Nov 2007 14:40:23 -0500 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474C7284.8060501@redhat.com> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> Message-ID: <1196192423.15420.127.camel@cutter> On Tue, 2007-11-27 at 13:39 -0600, Mike McGrath wrote: > Jim Meyering wrote: > > Mike McGrath wrote: > > > >> tried this, its not an easy task. But adding an additional SCM for > >> GIT which is JUST a copy of what's in CVS sounds like a waste of our > >> resources. Why not also do SVN, BZR and Mercurial? > >> > > > > IMHO, they're not as useful. > > > > > > And thats the real trick, I'd imagine the mercurial, svn and bzr guys > would disagree with you. > > > If Fedora doesn't want to do this, I can probably set up > > something independent and provide public git:// access. > > > > If someone else wants to host it I'm all for it, we can certainly make > it easier to get at the raw CVS repo. If the other officers disagree > please let it be known, but this sounds more like a distraction/one off > then something that adds value to our infrastructure. > I concur. -sv From skvidal at fedoraproject.org Tue Nov 27 20:42:17 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Tue, 27 Nov 2007 15:42:17 -0500 Subject: fedorapeople.org documentation Message-ID: <1196196137.15420.139.camel@cutter> Spot asked me to write this up for something else and I realized I'd never given a good full documentation of what we'd done on fedorapeople.org to secure it and make it manageable, so here it is. http://fedoraproject.org/wiki/Infrastructure/FedoraPeopleConfig please let me know about things I've missed or underdocumented. thanks, -sv From a.badger at gmail.com Tue Nov 27 21:21:06 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 27 Nov 2007 13:21:06 -0800 Subject: Updated hostedreposync proposal Message-ID: <474C8A42.3050000@gmail.com> Since opening up creation of hosted sites by people other than Jesse we've had to straighten out a few steps here and there that the people we want to manage the site have not been able to perform on their own. One of those is the script that rsyncs the source repositories from cvs-int to lockbox (where it is stored on the netapp.) This script, hostedreposync, needs to be updated with the names of the new hosted repositories in order to sync them. I'm attaching a new hostedreposync that would rsync the whole $SCM/hosted tree from cvs-int to the netapp instead of cherry picking the individual repositories. This allows us to stop editing the hostedreposync script every time a new hosted repository is added but it does have some differences with the current script: 1) More repositories are pulled over. I count 39 repositories that are present in the hosted trees on cvs-int that aren't listed in the current hostedreposync script that will now be pulled over. 2) Old repositories will be deleted from the netapp. These repositories are present on the netapp but not on cvs-int and would be erased with the new script: pungi.bak (there is a pungi repo) mock (probably replaced by mock.git) func (probably replaced by func.git) surfr Comments? -Toshio -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: hostedreposync URL: From mmcgrath at redhat.com Tue Nov 27 21:35:40 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 27 Nov 2007 15:35:40 -0600 Subject: Updated hostedreposync proposal In-Reply-To: <474C8A42.3050000@gmail.com> References: <474C8A42.3050000@gmail.com> Message-ID: <474C8DAC.8030207@redhat.com> Toshio Kuratomi wrote: > Since opening up creation of hosted sites by people other than Jesse > we've had to straighten out a few steps here and there that the people > we want to manage the site have not been able to perform on their own. > One of those is the script that rsyncs the source repositories from > cvs-int to lockbox (where it is stored on the netapp.) This script, > hostedreposync, needs to be updated with the names of the new hosted > repositories in order to sync them. > > I'm attaching a new hostedreposync that would rsync the whole > $SCM/hosted tree from cvs-int to the netapp instead of cherry picking > the individual repositories. This allows us to stop editing the > hostedreposync script every time a new hosted repository is added but > it does have some differences with the current script: > > 1) More repositories are pulled over. I count 39 repositories that > are present in the hosted trees on cvs-int that aren't listed in the > current hostedreposync script that will now be pulled over. > > 2) Old repositories will be deleted from the netapp. These > repositories are present on the netapp but not on cvs-int and would be > erased with the new script: > pungi.bak (there is a pungi repo) > mock (probably replaced by mock.git) > func (probably replaced by func.git) > surfr +1 from me. -Mike From katzj at redhat.com Tue Nov 27 22:45:28 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 27 Nov 2007 17:45:28 -0500 Subject: Updated hostedreposync proposal In-Reply-To: <474C8A42.3050000@gmail.com> References: <474C8A42.3050000@gmail.com> Message-ID: <1196203528.22911.7.camel@localhost.localdomain> On Tue, 2007-11-27 at 13:21 -0800, Toshio Kuratomi wrote: > Comments? +alot Jeremy From paulo.banon at googlemail.com Tue Nov 27 23:09:20 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Tue, 27 Nov 2007 23:09:20 +0000 Subject: Updated hostedreposync proposal In-Reply-To: <1196203528.22911.7.camel@localhost.localdomain> References: <474C8A42.3050000@gmail.com> <1196203528.22911.7.camel@localhost.localdomain> Message-ID: <7a41c4bc0711271509g194bec6fh31dfdf0916e546b0@mail.gmail.com> +1 from me also, but i still have an issue/doubt. One of the steps while creating a new hosted project, is the repo creation, in cvs-int to which I (and dont know about Ricky) still can't do. And without it there isn't much i can help with. If i'm not mistaken, after the CVS box is rebuilt this problem will be taken care of correct ? Paulo On Nov 27, 2007 10:45 PM, Jeremy Katz wrote: > On Tue, 2007-11-27 at 13:21 -0800, Toshio Kuratomi wrote: > > Comments? > > +alot > > Jeremy > > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From a.badger at gmail.com Tue Nov 27 23:23:04 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 27 Nov 2007 15:23:04 -0800 Subject: Updated hostedreposync proposal In-Reply-To: <7a41c4bc0711271509g194bec6fh31dfdf0916e546b0@mail.gmail.com> References: <474C8A42.3050000@gmail.com> <1196203528.22911.7.camel@localhost.localdomain> <7a41c4bc0711271509g194bec6fh31dfdf0916e546b0@mail.gmail.com> Message-ID: <474CA6D8.1050109@gmail.com> Paulo Santos wrote: > +1 from me also, but i still have an issue/doubt. > > One of the steps while creating a new hosted project, is the repo > creation, in cvs-int to which I (and dont know about Ricky) still > can't do. And without it there isn't much i can help with. If i'm not > mistaken, after the CVS box is rebuilt this problem will be taken care > of correct ? > We should be able to create a new group to shepherd hosted at that time, yes. Currently, with hosted and the packaging repository on the same server we have to regulate access to it more tightly. Ricky happened to be on IRC at the right time so he got the ability to log into cvs-int and filesystem acls were setup to allow him to access the hosted repos but not the package repo. Since the new hosted box is imminent I'd like to wait for that and set you up via a sysadmin-hosted FAS group that gives you the needed permissions but if something delays it we'll need to do the same thing for you. -Toshio From a.badger at gmail.com Wed Nov 28 01:34:39 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 27 Nov 2007 17:34:39 -0800 Subject: Updated hostedreposync proposal In-Reply-To: <1196203528.22911.7.camel@localhost.localdomain> References: <474C8A42.3050000@gmail.com> <1196203528.22911.7.camel@localhost.localdomain> Message-ID: <474CC5AF.9060002@gmail.com> hostedreposync updated. Let me know if there's problems with hosted repositories not showing up in the trac browser. -Toshio From mmcgrath at redhat.com Wed Nov 28 02:56:03 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Tue, 27 Nov 2007 20:56:03 -0600 Subject: Updated hostedreposync proposal In-Reply-To: <474CA6D8.1050109@gmail.com> References: <474C8A42.3050000@gmail.com> <1196203528.22911.7.camel@localhost.localdomain> <7a41c4bc0711271509g194bec6fh31dfdf0916e546b0@mail.gmail.com> <474CA6D8.1050109@gmail.com> Message-ID: <474CD8C3.9020807@redhat.com> Toshio Kuratomi wrote: > Paulo Santos wrote: >> +1 from me also, but i still have an issue/doubt. >> >> One of the steps while creating a new hosted project, is the repo >> creation, in cvs-int to which I (and dont know about Ricky) still >> can't do. And without it there isn't much i can help with. If i'm not >> mistaken, after the CVS box is rebuilt this problem will be taken care >> of correct ? >> > We should be able to create a new group to shepherd hosted at that > time, yes. Currently, with hosted and the packaging repository on the > same server we have to regulate access to it more tightly. > > Ricky happened to be on IRC at the right time so he got the ability to > log into cvs-int and filesystem acls were setup to allow him to access > the hosted repos but not the package repo. Since the new hosted box > is imminent I'd like to wait for that and set you up via a > sysadmin-hosted FAS group that gives you the needed permissions but if > something delays it we'll need to do the same thing for you. Yep, once hosted moves onto its own infrastructure there will be a separate hosted group and you two will certainly be in it (If you want to :) -Mike From lmacken at redhat.com Wed Nov 28 14:40:42 2007 From: lmacken at redhat.com (Luke Macken) Date: Wed, 28 Nov 2007 09:40:42 -0500 Subject: Restart TG apps for high mem-usage In-Reply-To: <4749E285.3000400@gmail.com> References: <4749E285.3000400@gmail.com> Message-ID: <20071128144042.GG9734@crow> On Sun, Nov 25, 2007 at 01:00:53PM -0800, Toshio Kuratomi wrote: > Here's a short script to test our TG apps run via supervisor for excessive > memory usage and restart them if necessary. We could run this via cron in > alternate hours on each app server. Does this seem like a good or bad idea > to people? Probably not a bad idea; I think koji does something similar with apache. However, I don't think we need this for bodhi, at least for the moment. The only time bodhi's memory usage jumps is when it's pushing updates -- so if we were to use this script for bodhi, it would have to check if it is currently running mash. But for now, I'm not sure that it is necessary seeing as how most of the time puppetd eats more memory than bodhi. luke From lmacken at redhat.com Wed Nov 28 15:21:36 2007 From: lmacken at redhat.com (Luke Macken) Date: Wed, 28 Nov 2007 10:21:36 -0500 Subject: Restart TG apps for high mem-usage In-Reply-To: <474B0990.9020400@gmail.com> References: <4749E285.3000400@gmail.com> <7f692fec0711251720r3a00e215o1338461e91d33374@mail.gmail.com> <474A54CE.9060401@gmail.com> <20071126171621.GA12670@auslistsprd01.us.dell.com> <474B0990.9020400@gmail.com> Message-ID: <20071128152136.GH9734@crow> On Mon, Nov 26, 2007 at 09:59:44AM -0800, Toshio Kuratomi wrote: > Matt Domsch wrote: >> On Sun, Nov 25, 2007 at 09:08:30PM -0800, Toshio Kuratomi wrote: >>>> +1, but does it make sure all transactions are finished? I know smolt >>>> does not have good transaction protection. If a transaction fails >>>> halfway through, we might have a mess. >>>> >>> Not if the app doesn't. From a brief test, TG apps do not do this. >> >> MirrorManager doesn't use transactions, I never figured out how to get >> them to work right. Advice welcome. >> > By not being able to get transactions working, do you mean explicit > transactions or implicit transactions? I see that mirrormanager, bodhi, > and noc (not running currently) are using a dburi that disables implicit > transactions:: > mirrormanager-prod.cfg.erb: > sqlobject.dburi="notrans_postgres://mirroradmin: > <%= mirrorPassword %>@db2.fedora.phx.redhat.com/mirrormanager" > > If that was changed to:: > sqlobject.dburi="postgres://mirroradmin:[...] > > TurboGears would at least attempt to use an implicit transaction per http > request which should protect the database from shutting down the > application in the middle of processing a multi-table update. I don't know > if that's the problem you're referring to, though. Removing the notrans_postgres:// from bodhi's sqlobject.dburi causes problems. Modifications don't seem to go through; I'm not sure if they hit the DB or not. I remember encountering this issue early on in bodhi development, and it was mitigated by calling hub.sync() all over the place. I have since removed them, and use notrans_postgres, which has been working fine since day 1 of our production instance. I'm not a db guru, so I'm not sure which is better or worse. I'll have to investigate this further. luke From a.badger at gmail.com Wed Nov 28 17:50:58 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 28 Nov 2007 09:50:58 -0800 Subject: Restart TG apps for high mem-usage In-Reply-To: <20071128144042.GG9734@crow> References: <4749E285.3000400@gmail.com> <20071128144042.GG9734@crow> Message-ID: <474DAA82.6000303@gmail.com> Luke Macken wrote: > On Sun, Nov 25, 2007 at 01:00:53PM -0800, Toshio Kuratomi wrote: >> Here's a short script to test our TG apps run via supervisor for excessive >> memory usage and restart them if necessary. We could run this via cron in >> alternate hours on each app server. Does this seem like a good or bad idea >> to people? > > Probably not a bad idea; I think koji does something similar with > apache. However, I don't think we need this for bodhi, at least for the > moment. The only time bodhi's memory usage jumps is when it's pushing > updates -- so if we were to use this script for bodhi, it would have to check > if it is currently running mash. But for now, I'm not sure that it is necessary > seeing as how most of the time puppetd eats more memory than bodhi. > Sounds good. I've taken both Bodhi and transifex out of the script for now as neither one is load balanced (the idea being that the apps which are load balanced should continue to serve requests off the other instance while one is restarting). I've also changed it to take a different memory limit for each app. It currently has some generous guesses as to what the memory limit should be. I'm running a cron that logs the rss of the apps on app3&4 and will refine the limits after we have more data. -Toshio From a.badger at gmail.com Wed Nov 28 23:12:06 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 28 Nov 2007 15:12:06 -0800 Subject: TG Apps and caching Message-ID: <474DF5C6.3020700@gmail.com> Hey all, good news! We've finally got caching of static data on our TurboGears applications running. This builds on the mod_cache work that paulobanon did earlier to have Apache cache the images, stylesheets, javascript, and other non-dynamic files served by our TurboGears apps. So far, we have the following programs/directories cached: bodhi: admin.fp.o/updates/static admin.fp.o/updates/tg_widgets/turbogears.widgets packagedb: admin.fp.p/pkgdb/static admin.fp.o/pkgdb/tg_js mirrormanager: admin.fp.o/mirrormanager/static If there's other directories of purely static data in your application that you'd like us to set up caching of (glezos, I'm looking at you ;-), please get in touch with me or file a ticket: https://hosted.fedoraproject.org/projects/fedora-infrastructure/newticket -Toshio From paulo.banon at googlemail.com Wed Nov 28 23:24:23 2007 From: paulo.banon at googlemail.com (Paulo Santos) Date: Wed, 28 Nov 2007 23:24:23 +0000 Subject: TG Apps and caching In-Reply-To: <474DF5C6.3020700@gmail.com> References: <474DF5C6.3020700@gmail.com> Message-ID: <7a41c4bc0711281524q13886e4am60369a89a450a177@mail.gmail.com> Toshio, Does this means that we dont have "involuntary impersonation" anymore?! :) Was there any changes, since i remember that on our earlier tests, Bodhi would cache some unwanted files. Thanks, Paulo On Nov 28, 2007 11:12 PM, Toshio Kuratomi wrote: > Hey all, good news! We've finally got caching of static data on our > TurboGears applications running. This builds on the mod_cache work that > paulobanon did earlier to have Apache cache the images, stylesheets, > javascript, and other non-dynamic files served by our TurboGears apps. > So far, we have the following programs/directories cached: > > bodhi: > admin.fp.o/updates/static > admin.fp.o/updates/tg_widgets/turbogears.widgets > > packagedb: > admin.fp.p/pkgdb/static > admin.fp.o/pkgdb/tg_js > > mirrormanager: > admin.fp.o/mirrormanager/static > > If there's other directories of purely static data in your application > that you'd like us to set up caching of (glezos, I'm looking at you ;-), > please get in touch with me or file a ticket: > https://hosted.fedoraproject.org/projects/fedora-infrastructure/newticket > > -Toshio > > _______________________________________________ > Fedora-infrastructure-list mailing list > Fedora-infrastructure-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list > From a.badger at gmail.com Thu Nov 29 00:58:59 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 28 Nov 2007 16:58:59 -0800 Subject: TG Apps and caching In-Reply-To: <7a41c4bc0711281524q13886e4am60369a89a450a177@mail.gmail.com> References: <474DF5C6.3020700@gmail.com> <7a41c4bc0711281524q13886e4am60369a89a450a177@mail.gmail.com> Message-ID: <474E0ED3.2030602@gmail.com> Paulo Santos wrote: > Toshio, > > Does this means that we dont have "involuntary impersonation" anymore?! :) > Was there any changes, since i remember that on our earlier tests, > Bodhi would cache some unwanted files. > That's the idea :-) We made the following additional changes that should fix the underlying cause of that: # Makes it safe to cache /myapp/static and /myapp/tg_js directories Header unset Set-Cookie Please test, and try to break it though as it is a very serious problem if this isn't foolproof. -Toshio From jim at meyering.net Thu Nov 29 16:25:11 2007 From: jim at meyering.net (Jim Meyering) Date: Thu, 29 Nov 2007 17:25:11 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474C7284.8060501@redhat.com> (Mike McGrath's message of "Tue, 27 Nov 2007 13:39:48 -0600") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> Message-ID: <87y7ch2f4o.fsf_-_@rho.meyering.net> Mike McGrath wrote: > Jim Meyering wrote: >> Mike McGrath wrote: >> >>> tried this, its not an easy task. But adding an additional SCM for >>> GIT which is JUST a copy of what's in CVS sounds like a waste of our >>> resources. Why not also do SVN, BZR and Mercurial? >> >> IMHO, they're not as useful. > > And thats the real trick, I'd imagine the mercurial, svn and bzr guys > would disagree with you. > >> If Fedora doesn't want to do this, I can probably set up >> something independent and provide public git:// access. > > If someone else wants to host it I'm all for it, we can certainly make > it easier to get at the raw CVS repo. If the other officers disagree > please let it be known, but this sounds more like a distraction/one > off then something that adds value to our infrastructure. At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too heavy for me. And besides, it'd really be better under the Fedora umbrella. Seeing as how much more efficient the git protocol is, if a few people switch to it from cvs, it'd actually decrease network bandwidth requirements. Is there anything I can do to revive this idea? For example, I'd be happy to own and set up the tools/infrastructure required to make it all work (I've already done this on three public servers). All I'd need is an open git port and access to the config files. Jim From skvidal at fedoraproject.org Thu Nov 29 16:22:55 2007 From: skvidal at fedoraproject.org (seth vidal) Date: Thu, 29 Nov 2007 11:22:55 -0500 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87y7ch2f4o.fsf_-_@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> Message-ID: <1196353375.15390.0.camel@cutter> On Thu, 2007-11-29 at 17:25 +0100, Jim Meyering wrote: > At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too > heavy for me. And besides, it'd really be better under the Fedora > umbrella. Seeing as how much more efficient the git protocol is, > if a few people switch to it from cvs, it'd actually decrease network > bandwidth requirements. The problem is we're not running out of network bandwidth most of the time. We're running out of disk space. Pretty badly, too. -sv From mmcgrath at redhat.com Thu Nov 29 16:26:50 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 29 Nov 2007 10:26:50 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87y7ch2f4o.fsf_-_@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> Message-ID: <474EE84A.8050007@redhat.com> Jim Meyering wrote: > Mike McGrath wrote: > >> Jim Meyering wrote: >> >>> Mike McGrath wrote: >>> >>> >>>> tried this, its not an easy task. But adding an additional SCM for >>>> GIT which is JUST a copy of what's in CVS sounds like a waste of our >>>> resources. Why not also do SVN, BZR and Mercurial? >>>> >>> IMHO, they're not as useful. >>> >> And thats the real trick, I'd imagine the mercurial, svn and bzr guys >> would disagree with you. >> >> >>> If Fedora doesn't want to do this, I can probably set up >>> something independent and provide public git:// access. >>> >> If someone else wants to host it I'm all for it, we can certainly make >> it easier to get at the raw CVS repo. If the other officers disagree >> please let it be known, but this sounds more like a distraction/one >> off then something that adds value to our infrastructure. >> > > At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too > heavy for me. And besides, it'd really be better under the Fedora > umbrella. Seeing as how much more efficient the git protocol is, > if a few people switch to it from cvs, it'd actually decrease network > bandwidth requirements. > > Is there anything I can do to revive this idea? > For example, I'd be happy to own and set up the tools/infrastructure > required to make it all work (I've already done this on three public servers). > All I'd need is an open git port and access to the config files. > If you think git is so much better than CVS (many would agree with you) come up with a proposal on how we can migrate to it, propose it, then convince people its the right thing to do. -Mike From jim at meyering.net Thu Nov 29 17:22:57 2007 From: jim at meyering.net (Jim Meyering) Date: Thu, 29 Nov 2007 18:22:57 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474EE84A.8050007@redhat.com> (Mike McGrath's message of "Thu, 29 Nov 2007 10:26:50 -0600") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <474EE84A.8050007@redhat.com> Message-ID: <87oddd2cge.fsf@rho.meyering.net> Mike McGrath wrote: > Jim Meyering wrote: ... >> At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too >> heavy for me. And besides, it'd really be better under the Fedora >> umbrella. Seeing as how much more efficient the git protocol is, >> if a few people switch to it from cvs, it'd actually decrease network >> bandwidth requirements. >> >> Is there anything I can do to revive this idea? >> For example, I'd be happy to own and set up the tools/infrastructure >> required to make it all work (I've already done this on three public servers). >> All I'd need is an open git port and access to the config files. > > If you think git is so much better than CVS (many would agree with > you) come up with a proposal on how we can migrate to it, propose it, > then convince people its the right thing to do. I consider the automated cvs-to-git mirroring to be the first step in any conversion proposal: First, give people an idea of what they can expect in a git-based dVCS, without requiring any change. It lets people continue to use the tools they're familiar with, and allows the better parts of a dVCS to begin to show up the radar of those who haven't yet had time to explore them. It also begins to highlight the parts (if any) of the existing process that might end up being adjusted with a dVCS. For example, in CVS, there's no reason to put a summary on the first line of the commit log message. But with any dVCS, it's encouraged. This shows up when you compare views of commit summaries in pure-dVCS projects and those where people have not yet adopted the first-line-is-summary standard. Helping a big project transition is a big job, so IMHO, the only way to do it is incrementally. If you try to come up with an all-encompassing proposal, you might never get buy-in from enough people and you'll wait forever. With something like this, a dVCS gets a foot in the door. And if/when people conclude it's worthless or want to try another, it's easy to revert or shift gears. Here's a feature of git that'd be handy if there ever is a cut-over: one can set up a read-only[1] cvs-pserver mirror to the master git repository. Then, while commits/pushes all go directly through git, people can still use their cvs clients for read-only operations on the very same git repository. Offering that service helped to convert a few projects on savannah.gnu.org from CVS to git: automake, autoconf, m4. The git-cvsserver emulation is 'ok', but for example doesn't handle the "-D date-string" part of the protocol. Not a big deal, of course. [1] you can even use git-cvsserver emulation to *write* into the git repository, but I don't think it's mature enough for that. From jim at meyering.net Thu Nov 29 17:25:20 2007 From: jim at meyering.net (Jim Meyering) Date: Thu, 29 Nov 2007 18:25:20 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <1196353375.15390.0.camel@cutter> (seth vidal's message of "Thu, 29 Nov 2007 11:22:55 -0500") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <1196353375.15390.0.camel@cutter> Message-ID: <87hcj52ccf.fsf@rho.meyering.net> seth vidal wrote: > On Thu, 2007-11-29 at 17:25 +0100, Jim Meyering wrote: > >> At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too >> heavy for me. And besides, it'd really be better under the Fedora >> umbrella. Seeing as how much more efficient the git protocol is, >> if a few people switch to it from cvs, it'd actually decrease network >> bandwidth requirements. > > The problem is we're not running out of network bandwidth most of the > time. We're running out of disk space. Pretty badly, too. One big advantage to switching from CVS to git is the savings in disk space. With the example above it's pretty obvious: you can save exactly the same information using git in 1/6 to 1/4th the space. From mmcgrath at redhat.com Thu Nov 29 17:26:54 2007 From: mmcgrath at redhat.com (Mike McGrath) Date: Thu, 29 Nov 2007 11:26:54 -0600 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87hcj52ccf.fsf@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <1196353375.15390.0.camel@cutter> <87hcj52ccf.fsf@rho.meyering.net> Message-ID: <474EF65E.9040909@redhat.com> Jim Meyering wrote: > seth vidal wrote: > >> On Thu, 2007-11-29 at 17:25 +0100, Jim Meyering wrote: >> >> >>> At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too >>> heavy for me. And besides, it'd really be better under the Fedora >>> umbrella. Seeing as how much more efficient the git protocol is, >>> if a few people switch to it from cvs, it'd actually decrease network >>> bandwidth requirements. >>> >> The problem is we're not running out of network bandwidth most of the >> time. We're running out of disk space. Pretty badly, too. >> > > One big advantage to switching from CVS to git is the savings in > disk space. With the example above it's pretty obvious: you can > save exactly the same information using git in 1/6 to 1/4th the space. > Clearly you have tenacity and a desire to see change. If I were you I'd join the SCMSig: http://fedoraproject.org/wiki/Infrastructure/SCMSig (some of us still hang out in #fedora-scm) Schedule and request a meeting, host that meeting and have your voice be heard. I, for one, would attend whatever meeting you schedule and many others in the sig would as well. Take a deep breath, send the email and dive right in. (I'm being totally serious here, I'll be disappointed if I don't get a meeting request soon :-P -Mike From a.badger at gmail.com Thu Nov 29 18:48:02 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 29 Nov 2007 10:48:02 -0800 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87oddd2cge.fsf@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <474EE84A.8050007@redhat.com> <87oddd2cge.fsf@rho.meyering.net> Message-ID: <474F0962.7040506@gmail.com> Jim Meyering wrote: > Mike McGrath wrote: >> Jim Meyering wrote: > ... >>> At 5GB+, (4.5GB for a copy of the cvs repo + 700MB for git) that's too >>> heavy for me. And besides, it'd really be better under the Fedora >>> umbrella. Seeing as how much more efficient the git protocol is, >>> if a few people switch to it from cvs, it'd actually decrease network >>> bandwidth requirements. >>> >>> Is there anything I can do to revive this idea? >>> For example, I'd be happy to own and set up the tools/infrastructure >>> required to make it all work (I've already done this on three public servers). >>> All I'd need is an open git port and access to the config files. >> If you think git is so much better than CVS (many would agree with >> you) come up with a proposal on how we can migrate to it, propose it, >> then convince people its the right thing to do. > > I consider the automated cvs-to-git mirroring to be the first step > in any conversion proposal: > > First, give people an idea of what they can expect in a git-based dVCS, > without requiring any change. It lets people continue to use the tools > they're familiar with, and allows the better parts of a dVCS to begin > to show up the radar of those who haven't yet had time to explore them. I don't really buy this because it's a one-way transaction. The people that need to be convinced that there's value in switching to git vs bzr vs hg vs svn also have commit rights to the main repository. For a demo to reach this audience you need to get them the ability to work from this tree. Which means they need to be able to checkout, checkin, tag, and request builds from it. [snip] > Helping a big project transition is a big job, so IMHO, the only way to > do it is incrementally. If you try to come up with an all-encompassing > proposal, you might never get buy-in from enough people and you'll wait > forever. I'm in agreement with this part. From trying to work up a trial before I have to say that the hardest part is figuring out how we can implement changes incrementally *and* non-disruptively. (Note that the changeover will be disruptive. But we want to make it a one-time event, not an on-going, this time we're switching to bzr, next time we're switching to exploded trees, etc.) However, I don't see a read-only mirror as an incremental step towards moving to a new VCS. It's more of a value-add for downstream distributions. IMHO we'd be much better figuring out a real interim step to make it possible for package maintainers to do actual work within a new VCS. -Toshio From jim at meyering.net Thu Nov 29 19:23:51 2007 From: jim at meyering.net (Jim Meyering) Date: Thu, 29 Nov 2007 20:23:51 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474F0962.7040506@gmail.com> (Toshio Kuratomi's message of "Thu, 29 Nov 2007 10:48:02 -0800") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <474EE84A.8050007@redhat.com> <87oddd2cge.fsf@rho.meyering.net> <474F0962.7040506@gmail.com> Message-ID: <87sl2o26uw.fsf@rho.meyering.net> Toshio Kuratomi wrote: > Jim Meyering wrote: ... >> I consider the automated cvs-to-git mirroring to be the first step >> in any conversion proposal: >> >> First, give people an idea of what they can expect in a git-based dVCS, >> without requiring any change. It lets people continue to use the tools >> they're familiar with, and allows the better parts of a dVCS to begin >> to show up the radar of those who haven't yet had time to explore them. > > I don't really buy this because it's a one-way transaction. The > people that need to be convinced that there's value in switching to > git vs bzr vs hg vs svn also have commit rights to the main > repository. For a demo to reach this audience you need to get them > the ability to work from this tree. Which means they need to be able > to checkout, checkin, tag, and request builds from it. Hi Toshio, [didn't we talk at a Mexican place after the fudcon in Boston?] Using such a mirror need not be a one-way transaction. Obviously, it'd be far less useful if there were such a limitation. When I do serious work against an upstream CVS repository, I arrange to mirror the CVS repo to git, and do all of my work in git, committing changes on private git branches. Then, I can easily rebase each of those branches (sort of like cvs update), to synchronize with newer upstream changes on the parent branch.[*] When I want to commit to cvs, it's easy to automate using git-cvsexportcommit. While this MO is not as comfortable as working in a git-only environment, it does help give you a feel for what it'd be like, and *I* certainly appreciate it. Of course, this can't help for tag/release-related operations, but it's a good start for the rest. Even with this slightly-contorted routine, I've appreciated using git: for example, while using conventional diff, patch, and cvs, it's easy to forget to "cvs add" a new file that was added by a patch. It's also easy to forget to apply "chmod a+x..." to a script just added by a patch. But in git, that doesn't happen as much, because the tools do more of the work for you. And git-cvsexportcommit takes care of the details of making sure everything in a git change set makes it back into a cvs "commit". Jim [*] In case you haven't seen it yet, "git rebase --interactive" is very useful, if you care about the "perfect patch" sort of process. With it, there's no need for quilt/stgit/etc. From ricky at fedoraproject.org Thu Nov 29 22:23:29 2007 From: ricky at fedoraproject.org (Ricky Zhou) Date: Thu, 29 Nov 2007 17:23:29 -0500 Subject: Meeting Log - 2007-11-29 Message-ID: <20071129222329.GA14133@Max.nyc1.dsl.speakeasy.net> 15:00 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Time for a hootinanny 15:00 < mmcgrath> Who's here? 15:00 < abadger1999> I'm at the hootinanny of course! 15:00 * iWolf is here 15:00 * loupgaroublond is bored in class, so he's kibitzing the hootinanny 15:00 < mmcgrath> iWolf: welcome 15:00 -!- MrBawb [i=abob at guppy.drown.org] has joined #fedora-meeting 15:00 < mmcgrath> loupgaroublond: :) 15:00 < iWolf> mmcgrath: Thanks! 15:01 -!- Sopwith [n=elliot at little-black-box.vmware.com] has joined #fedora-meeting 15:01 -!- giarc_w [i=hidden-u at gnat.asiscan.com] has joined #fedora-meeting 15:01 < abadger1999> Hey Sopwith! 15:01 * lmacken is here 15:01 * nirik is in the rabble seats. 15:01 * skvidal is 15:01 < mmcgrath> ricky: ping 15:01 < mmcgrath> paulobanon: ping 15:01 < mmcgrath> dgilmore: ping 15:01 < mmcgrath> jima: ping 15:01 * dgilmore is here 15:01 < mmcgrath> warren: ping 15:01 < warren> pong 15:02 < f13> mmcgrath: pong. 15:02 < mmcgrath> anyone else I forgot: ping 15:02 < mmcgrath> Welcome everyone! 15:02 < mmcgrath> This is the first meeting we've really had since the F8 launch, yippee. 15:03 < mmcgrath> I'd like to take some time and solidify our goals for the F9 launch. 15:03 < f13> beer harder. 15:03 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Fedora 9 - an infrastructure preview. 15:03 < warren> f13, Stacy took all the beer with him. 15:03 < mmcgrath> For those unfamiliar with the thread its: 15:03 < mmcgrath> .tiny https://www.redhat.com/archives/fedora-infrastructure-list/2007-November/msg00161.html 15:03 < mmcgrathbot> mmcgrath: http://tinyurl.com/yu2jju 15:04 < mmcgrath> I saw some people add some things to that list that they'd like to see, here's a preview as it stands right now... (this might take a bit) 15:04 -!- JSchmitt [n=s4504kr at p54B11157.dip0.t-ipconnect.de] has joined #fedora-meeting 15:04 < Sopwith> /query abadger1999 15:04 < Sopwith> (sry) 15:05 < abadger1999> heh 15:05 < loupgaroublond> i can add a bit of advice to #2 15:05 < mmcgrath> Remove all FC6 boxes, separate test infrastructure, finalize backup solution, system hardening and cleanup, further system replication, new torrent server, new collaboration servers, move hosted out of PHX... 15:05 * warren wants to work on hosted. 15:05 < mmcgrath> FAS2 implementation, better ystems integration (bodhi, FAS, pkgdb, koji, etc), less focus this time around on new systems... 15:05 * jima gets out of a meeting and stumbles into another one 15:06 < mmcgrath> SSL auth against apps, better integration with projects like OLPC and CC. 15:06 < mmcgrath> Ok, thats what I've got. 15:06 < mmcgrath> Anyone see anything obvious that I've missed? 15:06 < dgilmore> nope 15:06 < mmcgrath> we can always add stuff but these changes alone will keep us might busy. 15:07 < dgilmore> thats alot of work :) 15:07 < mmcgrath> dgilmore: no doubt. 15:08 < mmcgrath> If there's no objections I'll get these bits added to the F9 milestone. 15:08 -!- JSchmitt [n=s4504kr at fedora/JSchmitt] has quit Client Quit 15:08 -!- jmtaylor [n=jason at c-76-112-119-170.hsd1.mi.comcast.net] has joined #fedora-meeting 15:08 < mmcgrath> All in all F8 launch went very well except for the switch failing which took PHX offline. 15:08 < mmcgrath> For F9 we should be able to completely mitigate any issues in PHX related to distribution. 15:08 < mmcgrath> Even for the F8 launch we did not rely on download.fedora.redhat.com as it was not in the mirror list or public list. 15:09 < jima> are we going to relocate the services, or mirror them? 15:09 < dgilmore> do we have a list of phx based things that are ok if they go down 15:09 < mmcgrath> In fact the only thing that stopped us after the switch went down was mirrormanager. 15:09 < dgilmore> and then mirror everything else out 15:09 < mmcgrath> dgilmore: we don't but we need that, I'm actually working on getting our services divided up into 4 sections. 15:09 < mmcgrath> 1) Buildsystem, 2) distribution 3) support and 4) value-added. 15:10 < mmcgrath> 3 would be like, the website for example and 4) would be hosted.fedoraproject.org or fedorapeople.org 15:10 < skvidal> what's stuff ike FAS? 15:10 < mmcgrath> jima: we're actually just going to create duplicates in other colo's. 15:10 < mmcgrath> skvidal: support. 15:10 < dgilmore> skvidal: critical 15:10 < warren> FAS is needed by multiple parts... 15:10 < skvidal> mmcgrath: okay 15:10 < skvidal> that's what I figured 15:11 < mmcgrath> but each of those sections will also have subsections we can consider critical. 15:11 < jima> mirrors/mm are distribution? 15:11 < mmcgrath> jima: well, the mirrorlist app in particular was the part that failed, had we just had it installed in tummy.com our users wouldn't have even noticed the outage. 15:11 < mmcgrath> its designed PERFECTLY for that purpose actually. 15:11 < warren> in two parts 15:12 < warren> management and serving 15:12 -!- clarkbw [i=clarkbw at nat/redhat/x-fe2f50bfb8c9efc5] has joined #fedora-meeting 15:12 < mmcgrath> warren: yeah, and we can handle the management portion being down for a while, the serving piece needs to be HA, and since it keeps its own cache on each app server, if we lose access to the primary DB it'll continue working. 15:13 < mmcgrath> One of the bigger projects this time around is going to be FAS2. 15:13 * paulobanon is here but going home now 15:13 < mmcgrath> its actually been very close for a long time, but we started looking at migration too late in the F8 process where the dev's and releng really need to have a solid environment. 15:13 < jima> yeah 15:14 < f13> we appreciated that (: 15:14 < mmcgrath> :) 15:14 < dgilmore> mmcgrath: are we going to replicate ldap across a few datacantres 15:14 < dgilmore> centres 15:14 < mmcgrath> dgilmore: we will to at least two places, though for the initial rollout we're going to duplicate what we have now. 15:14 < mmcgrath> with each box downloading their own copy of the stuff. 15:15 < mmcgrath> once we're solid with the ldap infrastructure we'll start looking to migrating actual shell acounts to use ldap directly. 15:15 * skvidal cringes 15:15 < skvidal> really? 15:15 < skvidal> it's been my experience that the reliability of nss_ldap is not-so-fantastic 15:15 < dgilmore> mmcgrath: ok the FDS guys just announced the first beta of FDS 1.1 and RDS 8 15:15 < warren> mmcgrath, might it be more reliable to keep our replicated-like setup? 15:16 < dgilmore> skvidal: ive used it for 3 or 4 years now 15:16 < mmcgrath> skvidal: yeah, if the infrastructure is solid enough we really should. We have a lot of sub issues that ocme about from our current setup (people trying to ssh in and not having accounts right away then getting locked out by deny hosts, etc) 15:16 < skvidal> dgilmore: so have I and it full of hurk - esp if you turn on nscd 15:16 < abadger1999> replicated is a pain though... (one-two hour sync time, etc) 15:16 < dgilmore> skvidal: yeah i never use ncsd 15:16 < mmcgrath> If we find that our ldap setup is not reliable enough to use then we'll continue using what we have, but in the meantime this "syncs at the top of the hour stuff" is not good form. 15:17 < skvidal> so do it more often? 15:17 < mmcgrath> its confusing to people and once we actually have LDAP up and running, there's a viable solution to it. 15:17 < skvidal> it's not like the data is heavy 15:17 < dgilmore> abadger1999: the fds replication is quick 15:17 < mmcgrath> skvidal: thats an option as well. I'd prefer not to have a sync if we don't have to, we'll have to discuss this again once FAS2 ships so we can look at it. 15:17 < skvidal> well, we're syncing _something_ no matter what 15:18 < abadger1999> dgilmore: Sorry. I was talking about our current "replicated-like setup" 15:18 < skvidal> the issue is how transparent fixing it is b/t the two situations 15:18 < dgilmore> abadger1999: :) ok 15:18 -!- kital [n=Joerg_Si at fedora/kital] has joined #fedora-meeting 15:18 < mmcgrath> Yep. 15:18 < mmcgrath> anywho, that will be something we need to talk about when the time comes. 15:18 < skvidal> nod 15:18 < f13> how much of the freeipa stuff would apply? 15:19 < mmcgrath> In the meantime ricky is point man on FAS2. 15:19 < dgilmore> f13: not sure 15:19 < f13> and could we get their help in using us as a real world usage case? 15:19 < mmcgrath> f13: on initial rollout none, but prior to other changes we can certainly look at it. 15:19 < skvidal> f13: freeipa is NOT ready yet 15:19 < mmcgrath> IIRC FAS2 is very close to having OpenID support ready as well. 15:19 < f13> skvidal: yet. 15:19 < skvidal> at least from what Iv'e seen so far 15:19 < mmcgrath> f13: can you give us a quick rundown of this release? It seems that F9 alpha is very very close, when would be your prefered time frame for the switchover to FAS2? 15:20 < f13> F9 alpha should be pretty light in infrastructure. It's not a full freeze, just a nonblocking releng freeze 15:20 < f13> it'll be mostly business as usual. 15:20 < f13> We could do FAS2 between alpha and beta if you need that much time. 15:20 * skvidal has to jet - back after a while 15:20 < mmcgrath> skvidal: solid 15:21 < mmcgrath> f13: thats what I was thinking as well. Probably hugging just after the alpha. 15:21 < jwb> FAS2? 15:21 < mmcgrath> jwb: Fedora Account System 2. 15:21 < jwb> apparently i'm totally missing where this is being discussed 15:21 -!- stick [n=stick at cpe-069-134-113-166.nc.res.rr.com] has joined #fedora-meeting 15:22 -!- loupgaroublond [n=loupgaro at dijk249.athome232.wau.nl] has quit "class is over early" 15:22 < mmcgrath> we're discussing it here now, and in #fedora-admin or https://hosted.fedoraproject.org/projects/fas2/ 15:22 < mmcgrath> really though there's not much more to say on it for the meeting though :) 15:22 < mmcgrath> Anyone have anything they'd like to discuss for the F9 release (either in process or in new features they'd like to have) ? 15:23 < mmcgrath> Ok, we'll move on then. 15:23 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Serverbeach 15:23 * jwb once again confuses -meeting with -devel because of xchat tab reordering 15:24 < mmcgrath> As many of you know we've had 5 serverbeach servers given to us and we're working out some specifics but they should be ready soon. 15:24 < dgilmore> mmcgrath: how have we done with xen bridges 15:24 < mmcgrath> We've got it setup so that we can do xen guests on them though its in a fairly complex way. It will be included in the kickstart SOP I'm writing. 15:24 < dgilmore> mmcgrath: warren had said they dont allow it 15:24 < warren> they wont allow the normal one 15:24 < warren> normal way 15:24 < mmcgrath> dgilmore: its not that they do or don't allow it, its that in their network setup it doesn't work. 15:25 < warren> mmcgrath, you did it with private bridging + NAT? 15:25 < dgilmore> mmcgrath: fun 15:25 < warren> mmcgrath, so you already found a solution? 15:25 < warren> don't need me? 15:25 * warren is sad. 15:26 < mmcgrath> For example, serverbeach2 has an IP address of 64.34.163.95 and when we requested additional IP addresses we were given 64.34.195.12. 15:26 < mmcgrath> warren: sorry :) I do think we have a good solution though. 15:26 < warren> mmcgrath, it is standard procedure for them to give IP's on a different subnet. 15:26 < mmcgrath> the problem with the 64.34.195.0/24 network is that it does not have a default gateway. Each ip given to is is given a static route through 64.34.163.95. 15:26 * ricky is here (darn you, DST!) 15:26 < warren> mmcgrath, I don't mind that you figured out your own solution, but I knew how to do it way back and I feel like I've been ignored. 15:26 -!- sankarshan [n=sankarsh at fedora/sankarshan] has quit Read error: 110 (Connection timed out) 15:27 < mmcgrath> So in order to give 64.34.195.12 access to the internet we have to give our xen dom0 an IP in that range 15:27 < mmcgrath> warren: we just didn't want to have to maintain a natpool for each host. this way is a two cli method and then there's no further upkeep required, even when we add additional machines. 15:28 < warren> mmcgrath, I had to do all of this months ago so I was fully aware of what was needed. I could have given you sample configurations and had it up and running in less than an hour with reproducible documentation. 15:28 < mmcgrath> sorry, didn't mean to ignore you, we were just looking at the nat solution as last resort. 15:28 < warren> mmcgrath, uh, that's exactly what I did. 15:28 < mmcgrath> warren: you used nat? 15:28 < MrBawb> why not use proxy arp and local routing? 15:28 < warren> mmcgrath, well, didn't hear your full explanation yet. 15:29 < warren> mmcgrath, you didn't even give me a chance to setup a demo on one unused box. 15:29 -!- sankarshan [i=sankarsh at fedora/sankarshan] has joined #fedora-meeting 15:29 < mmcgrath> warren: sorry, we were just busy with stuff. The solution we came up with is: 15:29 < mmcgrath> ifconfig eth0:1 64.34.195.13 netmask 255.255.255.0; route add -net 64.34.195.0/24 gw 64.34.195.13 eth0 15:29 < warren> mmcgrath, essentially my solution has each IP on dom0 but each guest behaves as if it owns that IP, without their network seeing the fake IP's. 15:29 < mmcgrath> is that the same as yours? 15:30 < warren> err... fake MAC's 15:30 < warren> mmcgrath, wait, how do the xen guests use that interfacE? 15:31 < warren> you're only so far describing how it works on dom0 15:31 < mmcgrath> don't need to, the dom0 does it. The xen guests are still bridged, they're just contacting that IP as their gateway, the dom0 OS itself handles the routing and the static routing in place on the routers at SB handle the rest. Since its transparent to the xen guests, we can just add more without having to make any changes to the dom0 15:32 < mmcgrath> which makes it handy because the documentation is exactly the same for creating a domU inside serverbeach or inside PHX. 15:32 < dgilmore> mmcgrath: sounds good 15:32 < warren> mmcgrath, is there a xen host I can ssh into to see how it is setup? 15:32 < warren> or is this documented? 15:32 < warren> OK, this setup is different 15:32 < mmcgrath> serverbeach2. I'm still writing the documentation. 15:33 < mmcgrath> serverbeach2.fedoraproject.org that is. 15:33 < warren> mmcgrath, still, I don't appreciate being ignored when I had a similar solution to this. 15:33 -!- mdomsch [n=Matt_Dom at 70.124.62.55] has quit "Leaving" 15:34 < mmcgrath> warren: sorry, I had just assumed you were using nat which was something we knew would work but not something we wanted to have to maintain, it'd make documentation and troubleshooting more difficult. 15:34 < mmcgrath> warren: serverbeach2.fedoraproject.org is up as is its domU at 64.34.195.12 15:35 < mmcgrath> though its domU guest isn't in puppet yet :-/ 15:35 < mmcgrath> ok, lets move on for now. 15:35 < warren> I was even loudly suggesting serverbeach a year ago for this same purpose, knowing the limitations we would face, and when we finally do it I'm kept out of the design. 15:35 < warren> ok, move on 15:35 < dgilmore> warren: lets move on 15:36 < mmcgrath> The other server we're still working on getting is the new Dell in our Frankfurt colo. 15:36 < mmcgrath> Its ordered (has been for over 2 months now) 15:36 < mmcgrath> but its been lost in Dell for a little bit, I think thats been figured out just as of yesterday though I haven't heard an ETA on delivery. 15:37 < dgilmore> mmcgrath: we have half a rack there? 15:37 < mmcgrath> I'm hoping to find funding to fill the frankfurt colo for our expected growth. 15:37 < mmcgrath> dgilmore: yeah, half rack. 15:37 < mmcgrath> and donated remote hands. 15:37 < mmcgrath> so thats going to be a solid location for us I believe. 15:37 < mmcgrath> Anywho, thats where we are expanding at present. 15:37 < mmcgrath> Speaking of expansion....... 15:38 < jima> be a shame if a bladecenter fell into that half rack ;) 15:38 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Koji share 15:38 < mmcgrath> f13: ping 15:39 < f13> yep 15:39 < mmcgrath> f13: did the last GC ever run? 15:39 < mmcgrath> we're at 91% 15:39 < f13> mmcgrath: I haven't heard from mikem sadly, silly paternaty leave :/ 15:39 < f13> I'll ping him via email to see if we can get a response outof him 15:40 < mmcgrath> f13: lets just say that the next gc takes us from 90% to 91%. What steps are we going to take? 15:40 < f13> (I had asked for the information on how to do it myself but I didn't get anything) 15:40 < mmcgrath> We can hope that people won't be coding and building as much over the holidays but that sounds foolish to me ;) 15:40 * warren gets more done during holidays 15:41 < f13> mmcgrath: We can start taking durastic measures like removing all the fe7-merge stuff that isn't latest in an active tag 15:41 < f13> er wait 15:41 < f13> hrm. 15:41 < f13> mmcgrath: did you get the whatever submitted to oracle yet? 15:42 < mmcgrath> f13: I still don't have my access back, spevack is working on it. 15:42 < mmcgrath> spevack: ^^^ BTW :) 15:42 < mmcgrath> f13: I seriously doubt that we'll have a solution ready by the end of the year. 15:42 < warren> spevack, unless you enjoy the silence of the buildsystem. =) 15:42 < mmcgrath> Its going to have to be done on or two the koji share at some point. 15:44 < f13> wha? 15:44 < mmcgrath> f13: if it gets to the point where the koji share fills up and we can't build anymore do we have options to further purge stuff, even if it results in the loss of tagged packages? 15:44 < f13> We might be able to prune more signed packages, things that have already been shipped and could be re-created in koji, the signed copy doesn't need to continue to exist. 15:44 < f13> yes, we have some more pruning options. 15:44 < mmcgrath> k. 15:45 < mmcgrath> I'm going to hope for the best but it might come to that, the koji share grows, it grows! 15:45 < f13> yeah 15:45 < warren> mmcgrath, setup big red buttons 15:45 * mmcgrath could use many big red buttons. 15:45 < mmcgrath> ok, we'll move to the next thing. 15:46 < jima> which is...? :) 15:46 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets 15:46 < mmcgrath> .tiny https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?status=new&status=assigned&status=reopened&group=milestone&keywords=%7EMeeting&order=priority 15:46 < mmcgrathbot> mmcgrath: http://tinyurl.com/yth34b 15:46 < mmcgrath> .ticket 154 15:46 < mmcgrathbot> mmcgrath: #154 (DNS) - Fedora Infrastructure - Trac 15:46 < mmcgrath> The DNS stuff is coming along (this is the vpn.fedoraproject.org stuff) we need to add a few more hosts, its basically blocking on a rebuild of cvs-int. 15:46 < mmcgrath> .ticket 192 15:46 < mmcgrathbot> mmcgrath: #192 (Netapp low on free space) - Fedora Infrastructure - Trac 15:47 < mmcgrath> We already talked about that. 15:47 < mmcgrath> .ticket 222 15:47 < mmcgrathbot> mmcgrath: #222 (sysctl on the proxy servers) - Fedora Infrastructure - Trac 15:47 -!- JSchmitt [n=s4504kr at p54B11157.dip0.t-ipconnect.de] has joined #fedora-meeting 15:47 -!- kwizart [n=kwizart at fedora/kwizart] has joined #fedora-meeting 15:47 < mmcgrath> It seems the sysctl setup we have now works well, I'm going to commit those changes to puppet soon and make it part of our default setup for externally facing hosts. 15:47 < mmcgrath> So thats it on those. 15:47 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor 15:47 < f13> mmcgrath: I just found some more space to clear off the koji store 15:48 < mmcgrath> f13: clear it! 15:48 < f13> I just did 15:48 < mmcgrath> Anyone have anything else they'd like to discuss for this meeting? 15:48 < f13> what is the cacti link again? 15:48 -!- GeroldKa [n=GeroldKa at fedora/geroldka] has joined #fedora-meeting 15:48 * warren throws cacti at f13 15:48 < mmcgrath> .tiny https://admin.fedoraproject.org/cacti/graph.php?action=view&rra_id=all&local_graph_id=280 15:48 < mmcgrathbot> mmcgrath: http://tinyurl.com/25pnep 15:49 < jwb> what is the point of .tiny?? 15:49 < jima> makes it easier to copy/paste, i imagine 15:49 < ricky> It'd be cool to integrate with .ticket :) 15:49 < mmcgrath> f13: am I correct in assuming that the GC stuff that has been implemented is going to be an on-going ad-hoc thing? There's not a monthly job or anything? 15:50 < f13> mmcgrath: it's supposed to be an automated job that runs daily or weekly 15:50 < mmcgrath> jwb: what jima said, some people are in terminals (like me) big url's get annoying. 15:50 -!- knurd is now known as knurd_afk 15:50 < f13> throwing stuff in trash as it goes, emptying out stuff that's been in trash for a timeout period 15:50 -!- GeroldKa [n=GeroldKa at fedora/geroldka] has quit Client Quit 15:50 < mmcgrath> ahh, I'm mistaken then. 15:50 < mmcgrath> keep me in the loop if you get ahold of Mike. 15:50 < f13> so we'd just see continual small amounts cleaned, and an overall slower growth 15:50 < f13> nod 15:51 < J5> hey guys, abadger1999 suggested I let you guys know what I am working on. Let me know if there is a break where I can start babbling 15:51 < f13> we've got 235G free now 15:51 < jwb> mmcgrath, gnome terminal can open links 15:51 < jwb> anyway, i'll shut up 15:51 * mmcgrath doesn't always have these meetings in X. 15:51 < mmcgrath> J5: you can start babbling now. Open floor. 15:51 -!- G__ [n=njones at wikipedia/NigelJ] has joined #fedora-meeting 15:52 -!- JSchmitt [n=s4504kr at fedora/JSchmitt] has quit "Konversation terminated!" 15:52 < J5> ok, so I just put up a wiki page for reference http://wiki.fedoraproject.org/MyFedora 15:52 < mmcgrath> http://fedoraproject.org/wiki/MyFedora 15:52 * mmcgrath had troubles with wiki.fp.o 15:53 -!- sankarshan [i=sankarsh at fedora/sankarshan] has quit Read error: 110 (Connection timed out) 15:53 < J5> I am working on integration of all of fedora's resources to make our developer community more efficent 15:53 < dgilmore> J5: :) 15:53 < mmcgrath> J5: you're talking about number 10) in our F9 target - https://www.redhat.com/archives/fedora-infrastructure-list/2007-November/msg00161.html 15:53 < warren> J5, sounds like beginning to tie all the pieces together like Ubuntu has done with launchpad? 15:54 < J5> this includes talking to all of you guys as well as throwing ideas out and implementing some of theose ideas 15:54 < J5> warren: I'm thinking way beyond launchpad but that is a start 15:54 < jwb> J5, we should call your effort futopia 15:54 < abadger1999> mmcgrath: #10 definitely has a place in this but J5's ideas go beyond it too. 15:54 < mmcgrath> J5: for developers or for users as well. 15:54 < J5> mmcgrath: developers first then users 15:55 < J5> I want to integrate upstream also 15:55 * f13 spots a big glaring problem (: 15:55 < mmcgrath> f13: ? 15:55 < f13> J5: just an fyi and something to think about, THe packages that are 'released' with say F9, don't show up in the bodhi list 15:56 < J5> http://fedorapeople.org/~johnp/fedora_package_maint.pdf is the large overview piece 15:56 < mmcgrath> bah, thats implementation. 15:56 -!- sankarshan [n=sankarsh at fedora/sankarshan] has joined #fedora-meeting 15:56 < f13> mmcgrath: sure, that's why its an fyi 15:56 < J5> f13: that is taken into account - just show the last build :) 15:56 < J5> the idea is that information comes from a number of sources but is consolidated into common views 15:57 < mmcgrath> J5: well I, for one, am generally for this idea, If you're interested in moving forward with it, please create a ticket https://hosted.fedoraproject.org/projects/fedora-infrastructure/ with the links and information you just sent us. That way we can keep the discussion in one spot. 15:57 < ricky> Sounds very cool :) 15:57 -!- sankarshan [n=sankarsh at fedora/sankarshan] has quit Read error: 104 (Connection reset by peer) 15:57 < J5> will do 15:58 < mmcgrath> Ok, anyone have anything else they'd like to discuss? 15:58 < lmacken> J5: this also encompasses the 'amber' project that rnorwood has been working on 15:58 < mmcgrath> J5: FYI, you can also do http://johnp.fedorapeople.org/fedora_package_maint.pdf 15:59 < mmcgrath> Ok, anyone else have anything to bring up, we're almost out of time. 15:59 < J5> so I see myself as a facilitator here. There are a lot of projects with a lot of steam and I don't want to slow them down. So we have this one integration point where we can also discuss common goals. 16:00 < abadger1999> mmcgrath: FYI: I think that scratch builds are going to start being used in reviews and such. 16:00 < mmcgrath> J5: lets chat about this after the meeting in #fedora-admin if you have a moment. 16:00 < mmcgrath> abadger1999: Ugh, can we *not* do that until January? 16:00 < J5> sounds good 16:00 < f13> mmcgrath: 238G free now 16:01 < warren> Let's not do that until we have more storage 16:01 < warren> we're near critical now 16:01 < f13> abadger1999: "start"? scratch buidls have been used in revies for a while, it' sjust not well advertised 16:01 < f13> scratch buidls do automatically get pruned though 16:01 < mmcgrath> f13: you freed up 44G earlier. 16:01 < abadger1999> f13: yeah -- I mean "much more widely publicized" 16:01 < f13> mmcgrath: yes, I found some trees in rel-eng/ that didn't need to be there anymore. 16:01 < f13> currently scratch/ is only taking up 5 g 16:01 < f13> 5.7 to be precise. 16:02 < mmcgrath> f13: weren't a bunch of them recently purged though? 16:02 < f13> mmcgrath: scratch builds are autopruned by internal koji processes 16:02 < mmcgrath> Though I suppose we can purge them again. 16:02 < mmcgrath> f13: ahh, k. that I didn't know. Whats the algorithm? 16:02 < f13> not entirely sure 16:03 < f13> I think they're kept for a week 16:03 < mbonnet> mmcgrath: anything older than 30 days 16:03 < mbonnet> but we can crank that down if we want 16:03 -!- G [n=njones at wikipedia/NigelJ] has quit Connection timed out 16:03 < mmcgrath> abadger1999: I don't know if there's anything we can do about it but at this point I'm generally against encouraging anyone to build anything on our buildsystem. 16:03 < f13> ah 30 16:03 < abadger1999> heh :-) 16:04 < mmcgrath> we'll leave it at 30 for now if its only 5G, but its good to know we can crank that down. 16:04 < abadger1999> mbonnet: Maybe we should watch the size of scratch and if it grows, turn it down? I think a week is reasonable for scratch builds (people should be copying the files out to other storage if they want it longer.) 16:04 < spevack> mmcgrath, warren: i'm on it 16:05 < mmcgrath> spevack: rock, thanks. 16:05 < mmcgrath> ok, we can discuss the rest of this stuff in #fedora-admin, we're running over :) 16:05 < mmcgrath> anyone else have anything to discuss? 16:05 < mmcgrath> if not I'll close in 30 16:05 < mmcgrath> 15 16:05 < mbonnet> oh, no, I take it back, it is 7 days 16:06 < mbonnet> it's in /etc/cron.d/koji-scratch-cleanup 16:06 -!- MrBawb [i=abob at guppy.drown.org] has left #fedora-meeting [] 16:06 < mbonnet> on koji.fp.o 16:06 < mmcgrath> mbonnet: good to know, thanks :) 16:06 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting Closed 16:06 < f13> mbonnet: makes sense, I only see builds as far back as the 21st 16:06 < spevack> mmcgrath: i've given my "approval" to helpdesk. Why they needed it is beyond me :) 16:06 -!- bklemm [n=3745xyz at c-71-201-246-251.hsd1.il.comcast.net] has quit "ircII EPIC4-2.6 -- Are we there yet?" 16:06 < mmcgrath> spevack: I want to know why it was removed in the first place, it came at a terrible time :( 16:06 * spevack wishes he could just sign something that says "if mmcgrath asks for something, give it to him" :) 16:06 < mmcgrath> Anywho, thank everyone for coming! 16:06 < ricky> Thanks a lot! 16:07 * mmcgrath requires a Faberg? egg. 16:07 < spevack> mmcgrath: let's get it reinstated, and then we'll find out why it disappeared 16:07 < ricky> By the way, with the DST change, I won't be able to make the first 10 minutes or so for most meetings :( 16:07 < ricky> Whoops, I mean 20 minutes :( 16:08 -!- mmcgrath changed the topic of #fedora-meeting to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See http://fedoraproject.org/wiki/Communicate/FedoraMeetingChannel for meeting schedule -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From a.badger at gmail.com Thu Nov 29 23:58:54 2007 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 29 Nov 2007 15:58:54 -0800 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <87sl2o26uw.fsf@rho.meyering.net> References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <474EE84A.8050007@redhat.com> <87oddd2cge.fsf@rho.meyering.net> <474F0962.7040506@gmail.com> <87sl2o26uw.fsf@rho.meyering.net> Message-ID: <474F523E.1040907@gmail.com> Jim Meyering wrote: > Toshio Kuratomi wrote: >> Jim Meyering wrote: > ... >>> I consider the automated cvs-to-git mirroring to be the first step >>> in any conversion proposal: >>> >>> First, give people an idea of what they can expect in a git-based dVCS, >>> without requiring any change. It lets people continue to use the tools >>> they're familiar with, and allows the better parts of a dVCS to begin >>> to show up the radar of those who haven't yet had time to explore them. >> I don't really buy this because it's a one-way transaction. The >> people that need to be convinced that there's value in switching to >> git vs bzr vs hg vs svn also have commit rights to the main >> repository. For a demo to reach this audience you need to get them >> the ability to work from this tree. Which means they need to be able >> to checkout, checkin, tag, and request builds from it. > > Hi Toshio, > > [didn't we talk at a Mexican place after the fudcon in Boston?] > [Yeah, I think we did :-) ] > Using such a mirror need not be a one-way transaction. > Obviously, it'd be far less useful if there were such a limitation. > > When I do serious work against an upstream CVS repository, I arrange to > mirror the CVS repo to git, and do all of my work in git, committing > changes on private git branches. Then, I can easily rebase each of > those branches (sort of like cvs update), to synchronize with newer > upstream changes on the parent branch.[*] When I want to commit to > cvs, it's easy to automate using git-cvsexportcommit. While this MO is > not as comfortable as working in a git-only environment, it does help > give you a feel for what it'd be like, and *I* certainly appreciate it. > Of course, this can't help for tag/release-related operations, but it's > a good start for the rest. > That sounds better. Where does all this get setup? On the user's machine or the server? I still don't know if you'll get many developers to try it since you still have to keep both the git and cvs tree so they can make tag and make build. git can't push tags back to cvs? > Even with this slightly-contorted routine, I've appreciated using > git: for example, while using conventional diff, patch, and cvs, > it's easy to forget to "cvs add" a new file that was added by a patch. > It's also easy to forget to apply "chmod a+x..." to a script just added > by a patch. But in git, that doesn't happen as much, because the tools > do more of the work for you. And git-cvsexportcommit takes care of the > details of making sure everything in a git change set makes it back > into a cvs "commit". > Git does that when you apply a patch? Or only when you apply a patch that was generated via git? (I tried git apply foo.patch but that didn't seem to have the behaviour you mention.) -Toshio From jim at meyering.net Fri Nov 30 10:44:29 2007 From: jim at meyering.net (Jim Meyering) Date: Fri, 30 Nov 2007 11:44:29 +0100 Subject: hosting git conversion of Fedora CVS tree on fedora infrastructure? In-Reply-To: <474F523E.1040907@gmail.com> (Toshio Kuratomi's message of "Thu, 29 Nov 2007 15:58:54 -0800") References: <20071119110121.GA24667@xi.wantstofly.org> <474C6EA3.1000107@redhat.com> <87r6ibbhsn.fsf@rho.meyering.net> <474C7284.8060501@redhat.com> <87y7ch2f4o.fsf_-_@rho.meyering.net> <474EE84A.8050007@redhat.com> <87oddd2cge.fsf@rho.meyering.net> <474F0962.7040506@gmail.com> <87sl2o26uw.fsf@rho.meyering.net> <474F523E.1040907@gmail.com> Message-ID: <87abowypv6.fsf@rho.meyering.net> Toshio Kuratomi wrote: ... >> When I do serious work against an upstream CVS repository, I arrange to >> mirror the CVS repo to git, and do all of my work in git, committing >> changes on private git branches. Then, I can easily rebase each of >> those branches (sort of like cvs update), to synchronize with newer >> upstream changes on the parent branch.[*] When I want to commit to >> cvs, it's easy to automate using git-cvsexportcommit. While this MO is >> not as comfortable as working in a git-only environment, it does help >> give you a feel for what it'd be like, and *I* certainly appreciate it. >> Of course, this can't help for tag/release-related operations, but it's >> a good start for the rest. >> > That sounds better. Where does all this get setup? On the user's > machine or the server? The only part that's done on the server is to maintain the git repo in sync with the master CVS repository. For my usage, I keep two checked-out repos for each such project: the cvs one and the git one. I never change anything in the CVS one manually. That's only for the automated commit-from-git-to-cvs process. Other than that, there's almost no set-up required. I use a tiny bash function to encapsulate the details of "apply this git change-set to the corresponding CVS working directory". That just runs git-cvsexportcommit with options that apply the patch and ensure that there was no fuzz. git-cvsexportcommit has an option to perform the commit, but I don't use that -- prefer to verify, first. git-cvsexportcommit's output includes the "cvs commit -F .msg file1 file2 ..." command that'd be required to commit the affected files, using the same message you used in the git commit. > I still don't know if you'll get many > developers to try it since you still have to keep both the git and cvs > tree so they can make tag and make build. We won't know until we try, will we? :-) > git can't push tags back to cvs? That's not part of git-cvsexportcommit's charter. However, the cvs-to-git mirroring operation does preserve tags. >> Even with this slightly-contorted routine, I've appreciated using >> git: for example, while using conventional diff, patch, and cvs, >> it's easy to forget to "cvs add" a new file that was added by a patch. >> It's also easy to forget to apply "chmod a+x..." to a script just added >> by a patch. But in git, that doesn't happen as much, because the tools >> do more of the work for you. And git-cvsexportcommit takes care of the >> details of making sure everything in a git change set makes it back >> into a cvs "commit". >> > Git does that when you apply a patch? Yes. In the scenario I described above, it does, since git generates the patch and applies it, too. The format of classical "diff" output does not (and cannot) contain the information required for git tools to perform the extra checks. > Or only when you apply a patch > that was generated via git? (I tried git apply foo.patch but that > didn't seem to have the behaviour you mention.) Right. That's an inherent limitation in the standard format. FYI, once you've committed a change in git, you can get the latest change set in patch format like this: git-format-patch --stdout --signoff HEAD~1 > patch or the latest N change sets: git-format-patch --stdout --signoff HEAD~N > patch Then, you can later apply that series of change sets via git-am, e.g. git-am patch Applying patches that way preserves file permissions, author name and email, dates, etc.