Meeting Log - 2008-04-09

Fri Apr 10 19:36:55 UTC 2009

15:00 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Who's here?
15:01  * yingbull is.
15:01 < mmcgrath> Alrighty everyone, who's around?
15:01  * warren here
15:01  * skvidal  is here
15:01  * notting is here
15:01 < mmcgrath> jcollie: you around?
15:02 < jcollie> yup
15:02 < ivazquez> Pong.
15:02 < mmcgrath> cool, lets get started then
15:02 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets
15:02 < mmcgrath> .tiny https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assigned&status=reopened&group=milestone&keywords=%7EMeeting&order=priority
15:02 < zodbot> mmcgrath: http://tinyurl.com/2hyyz6
15:02  * MostafaDaneshvar is there
15:02 < mmcgrath> First ticket:
15:02 < mmcgrath> .ticket 395
15:02 < zodbot> mmcgrath: #395 (Audio Streaming of Fedora Board Conference Calls) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/395
15:02 < mmcgrath> jcollie: whats the latest?
15:02 < warren> mmcgrath: is agenda set in stone?  I want to add s/cvsextras/packager/ after F9 release and next convenient scheduled outage.
15:03 < mmcgrath> warren: not at all, we pretty much go over tickets now, then a few other items then open the floor.
15:03 < mmcgrath> warren: Remind me if I forget at the end.
15:03 < jcollie> not much has happened... someone horked up a bunch of my virtual hosts at work so i've been busy rebuilding them
15:03 < mmcgrath> jcollie: virtual hosts at $DAYJOB or something related to fedora?
15:03 -!- Wakko666 [n=blentz at office.cardomain.com] has joined #fedora-meeting
15:03 < jcollie> $DAYJOB
15:03 -!- wolfy [n=lonewolf at fedora/wolfy] has left #fedora-meeting ["When you breathe, you inspire. When you do not breathe, you expire."]
15:04 < mmcgrath> just checking ;-)
15:04 < mmcgrath> jcollie: k, so nothing new there.  We'll move on.
15:04 < mmcgrath> .ticket 398
15:04 < zodbot> mmcgrath: #398 (elfutils `monotone' (mtn) error) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/398
15:04  * mmcgrath sees if rmcgrath is around.
15:04 < jcollie> i think that those of us that are interested in working on the streaming should set up a separate meeting to divvy up some tasks
15:05 < mmcgrath> jcollie: not a bad idea.  It sounds like you've done most of the work already.  Its just a matter of getting that last 10% done (and getting the fas integration for the non streamins stuff)
15:05 < jcollie> yep
15:06 < mmcgrath> alrighty, we'll talk about 398 later if rmcgrath becomes available.
15:06 < mmcgrath> My understanding is that we can do it though.
15:06 < mmcgrath> .tickety 446
15:06 < mmcgrath> .ticket 446
15:06 < zodbot> mmcgrath: #446 (Possibility to add external links on spins page) - Fedora Infrastructure - Trac - https://fedorahosted.org/projects/fedora-infrastructure/ticket/446
15:06 < mmcgrath> doath
15:06  * dgilmore is here
15:07 < mmcgrath> dgilmore: I think last time we talked about you getting this up on the wiki, any news on that?
15:07 < dgilmore> mmcgrath: i started and got sidetracked
15:07 < mmcgrath> dgilmore: ahh, you're still on it though?
15:07 < dgilmore> mmcgrath: will be done by the end of the weekend
15:07 < dgilmore> yep
15:08 < mmcgrath> AFAIK we just have the one spin so far.
15:08 < mmcgrath> dgilmore: sweet, thanks!
15:08 < mmcgrath> Ok, so thats really the end of it on the tickets side.
15:08 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- New Wiki
15:08 < mmcgrath> Just a bit of a roundup on the wiki.  I've been working on th emigration script and docbook exporter script.
15:08 < mmcgrath> The people listed on http://fedoraproject.org/wiki/Infrastructure/WikiMigration all met yesterday.
15:09 < mmcgrath> I'm going to schedule a few more meetings but a few bumps aside it sounds like we are a GO.
15:09 < mmcgrath> dgilmore: are you on the FESCo ?
15:09 < dgilmore> mmcgrath: i am
15:09 < mmcgrath> dgilmore: at the next meeting would you find a delegate to sign up to be the wiki liason for those sections of the wiki?
15:10 < mmcgrath> unless you want to do it :)
15:10 < dgilmore> mmcgrath: sure
15:10 < mmcgrath> Thanks.
15:10 < dgilmore> mmcgrath: i think jwb volunteered
15:10  * jwb wakes up
15:10 < jwb> huh?
15:10  * dgilmore notes he will largely be away next week
15:10 < mmcgrath> so https://publictest1.fedoraproject.org/wiki/FedoraMain is still up and ready
15:10 < dgilmore> jwb: wiki migration?
15:10 < mmcgrath> dgilmore: ohhh yeah.  you're right.  he's on there.
15:11 < jwb> dgilmore, mmcgrath: oh, yeah.  i put my name there
15:11 < mmcgrath> dgilmore: I'll be away next week as well.
15:11 < mmcgrath> jwb: sorry I missed it there.
15:11 < jwb> i have no idea what that means
15:11 < jwb> do i have to do something?
15:11 < jwb> it just said "contact point"
15:11 < dgilmore> jwb: you get to be whiped
15:11 < mmcgrath> jwb: not yet, we'll be scheduling a meeing again the week after next.  In the meantime just go through https://publictest1.fedoraproject.org/wiki/ and look for... doom.
15:11 < dgilmore> jwb: no  Just be a point person.
15:12 < jwb> ok great
15:12 < jwb> either way, count me in
15:12 < mmcgrath> cool.
15:12 < mmcgrath> So thats really the latest on the wiki.
15:12 < mmcgrath> Anyone have any questions or concerns?
15:13 < mmcgrath> allrighty, moving on
15:13 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Next week
15:13 < mmcgrath> I'll be gone again next week for training.  Its the last traing session I actually have scheduled.
15:13 < mmcgrath> So during the days I'll be unavailable but I'll likely be around most evenings.
15:14 < mmcgrath> Next topic
15:14 < dgilmore> mmcgrath: ill be in Brazil
15:14  * skvidal will be here
15:14 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- nfs1
15:14 < skvidal> people can call/page me
15:14 < skvidal> ugh
15:14 < skvidal> nfs1
15:14 < mmcgrath> NFS1 got built yesterday in a hurry.
15:14 < mmcgrath> while working on xen2 the nfslock issues showed up again then the whole box took a panic.
15:15 < mmcgrath> we'd been serving nfs directly from xen2 so when it goes down the buildsystem goes down.
15:15 < mmcgrath> Anyway, we brought nfs1 up as a RHEL4 box to try to fix the nfs issues, they might have fixed nfs issues but brought on ext3 issues since we're using a 10T ext3 filesystem...
15:15 < mmcgrath> RHEl4 couldn't handle it and started to ioerror.
15:15 < mmcgrath> So I kicked a RHEL5 box and its now nfs1.
15:16 < mmcgrath> Its up, its running.
15:16 < mmcgrath> so far no nfslock issues, if we do get one though we're going to contact steved while its in the bad state and give him a shell to look around.
15:16 < dgilmore> mmcgrath: lets get steved a shell on xen2  and let him monitor nfs1
15:16 < mmcgrath> Really nothing new here except that /mnt/koji is no longer available on xen2, its now on nfs1.
15:17 < mmcgrath> dgilmore: you never know, perhaps magically moving to nfs1 fixed our problem :)
15:17 < mmcgrath> we'll see.
15:17  * skvidal likes magic
15:17 < dgilmore> mmcgrath: we can hope
15:17 < dgilmore> skvidal: voodoo magic is best
15:17 < mmcgrath> heheh
15:17  * skvidal makes a doll of dgilmore
15:17 < mmcgrath> Ok, next item.
15:17 < dgilmore> skvidal: need some hair and fingernails?
15:18 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- s/cvsextras/packagers/
15:18 < mmcgrath> warren: ping?
15:18 < warren> hi
15:18 < mmcgrath> warren: so we're just going to rename cvsextras to packagers.
15:18 < mmcgrath> which involves
15:18 < warren> So f13 suggested we rename the group after F9 release, at the first scheduled outage.
15:18 < warren> mmcgrath: packager, singular
15:18 < mmcgrath> 1) updating the db.
15:19 < mmcgrath> 2) updating the scripts (and upload script)
15:19 < mmcgrath> 3) updating the documentation
15:19 < mmcgrath> 4) testing?
15:19 < mmcgrath> I really don't think this will be that huge of a deal.
15:19 < mmcgrath> warren: I'll try to get it done sometime the week after F9 ships.  Would you mind opening a ticket and bugging me about it that week?
15:19 < warren> mmcgrath: ok
15:20 < warren> mmcgrath: are there any other group names we should rename as well?
15:20 < f13> warren: lets keep the name that will make sense in the future given my newmaintianercontainment proposal
15:20 < warren> as long as we have an outage
15:20 < f13> warren: since I hope to get an intern to work on that this summer
15:20 < warren> f13: is this the one from 2 months ago or so?
15:20 < mmcgrath> warren: yeah we'll schedule an outage.
15:20 -!- Milanito [n=Matt at 32.12.70-86.rev.gaoland.net] has quit Remote closed the connection
15:20 < f13> warren: http://fedoraproject.org/wiki/JesseKeating/NewMaintainerContainment
15:21 < mmcgrath> I'm not sure about other group names.
15:21 < warren> Let's ask around
15:21 < warren> there might be other names people want to do
15:21 < warren> f13: ok seems we need to figure out a bigger picture plan then
15:22 < warren> mmcgrath: will get back to you
15:22 < mmcgrath> warren: works for me, lets know well in advance of the outage know because renaming cvsextras I have a pretty good idea of what needs to be done.
15:22 < mmcgrath> but for another group I might not know without more research.
15:22 < mmcgrath> k
15:22 < mmcgrath> warren: when its all ready just create a ticket and bug me.  I think we could have it done in an hour or two some night after F9 ships.
15:22 < f13> warren: well, we don't necessarly have to wait to rename 'cvsextras' to /somethihng/
15:23 < f13> warren: 'fedorapackager' or whatever seems fine, we can pick a name that implies /greater/ rights later
15:23 < warren> f13: I do agree to rename it within the larger plan
15:23 < f13> or less rights.
15:23 < warren> f13: I'm still in favor of numbers instead of group names for that
15:23 < warren> but we're getting off topic
15:23 < mmcgrath> <nod> you guys figure that out :)
15:24 < mmcgrath> anyone have any questions about that?  We've got plenty of time to discuss it.
15:24 < mmcgrath> if not I'll move on.
15:24 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- app server upgrades.
15:24 < mmcgrath> abadger1999: ping?
15:25 < mmcgrath> I believe toshio wants to upgrade sqlalchemy and python-fedora before the freeze.  I think we're on schedule to do that but I'll have to get ahold of him to find out for sure.
15:25 < mmcgrath> which brings me to the next topic
15:25 < abadger1999> here.
15:25 < mmcgrath> oh
15:25 < mmcgrath> there you are :)
15:25 < abadger1999> Today.
15:26 < mmcgrath> abadger1999: want to talk about all of that right quick?
15:26 < abadger1999> Want to do it today/tonight.
15:26 < abadger1999> It would be to both app servers.
15:26 < mmcgrath> abadger1999: "both" ?
15:26 < mmcgrath> which?
15:26 < abadger1999> app4 and app5
15:26 < mmcgrath> app2 is also running tg apps
15:26 < abadger1999> Oh.  I was not aware.
15:27 < dgilmore> mmcgrath: i want the new python-fedora
15:27 < abadger1999> then we wantto upgrade that as well.
15:27 < abadger1999> We can issue a new python-fedora for everything if we want.
15:27 < mmcgrath> abadger1999: your call I'm not sure what we'd be comitting to with the new package.
15:27 < abadger1999> Or we can update that when we do the other updates from RHEL, etc.
15:27 < mmcgrath> what are the risks / benefits?
15:27 < mmcgrath> and will this break transifex?
15:28 < abadger1999> All the FAS1 stuff has been removed.  Only FAS2 stuff is in it.
15:28 < abadger1999> there's been some rearrangement of modules but the old  imports should work with a deprecation warning.
15:28 < mmcgrath> k
15:29 < mmcgrath> sounds pretty low risk, and we can always revert if required.
15:29 < abadger1999> <nod>
15:29 < mmcgrath> abadger1999: I'm fine for any time this week on that, if you want me around just ping me or call me.
15:29 < mmcgrath> abadger1999: anything else on that?
15:30 < abadger1999> Nope that's it.
15:30 < mmcgrath> cool.
15:30 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- app5
15:30 < mmcgrath> So yeah, I don't know WTF is going on but app5 is rebooting... a LOT
15:30 < mmcgrath> this comes after the move from x86_64 to i686
15:30 -!- Milanito [n=Matt at 32.12.70-86.rev.gaoland.net] has joined #fedora-meeting
15:30 < mmcgrath> we're talking between 20-40 times a DAY.
15:30 < mmcgrath> very very strange.
15:30 < mmcgrath> I've got a stack trace, I'm going to try to take it to the virt guys soon.
15:31 < mmcgrath> side note though, aside from that, the i686 move (also done on app2) has proved quite successful memory usage (and therefor swap) is way down.
15:31 < mmcgrath> its been a big win for us.
15:31 < mmcgrath> anyone have any questions / comments?
15:31 -!- viking-ice [n=johannbg at dsl-149-97-225.hive.is] has joined #fedora-meeting
15:31 < mmcgrath> Alrighty, then I'll move on to the last topic of the day before opening the floor.
15:32 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Change Freeze!
15:32 < mmcgrath> This one's for everyone
15:32 < mmcgrath> REMEMBER
15:32 < mmcgrath> DO NOT MAKE ANY CHANGES WITHOUT GETTING APPROVAL ON THE LIST FIRST!
15:32 < mmcgrath> and remember, you'll be asked the question "Why can't this wait until after F9 gets released"
15:33 < mmcgrath> I'll be gone much of next week which makes the freeze perfect.
15:33 < warren> under pain of
15:33  * mmcgrath looks up exact date again
15:33 < mmcgrath> warren: under pain of CURSE!
15:33 < skvidal> under pain of pain
15:33 < warren> I can advise against pain.
15:34 < mmcgrath> There we go, so the freeze starts the 15th.  The release is on the 29th.  The freeze is lifted on the 30th.
15:34 < yingbull> pain's scary.
15:34 < mmcgrath> Obvious exceptions include outages and things already scheduled for the release (mm type stuff, the website, etc)
15:34 < mmcgrath> other then that.  you must get approval on the list before making changes.  If you're working on tickets and people get cranky, just let them know we're under a freeze.
15:34 < skvidal> mmcgrath: it's okay to bring up new boxes unrelated to the release?
15:34 < ivazquez> Should we hold it until the Monday following?
15:35 < skvidal> mmcgrath: ie: if I get all the statics, etc from BU
15:35 < mmcgrath> skvidal: take it to the list!  But yeah, that shouldn't be a problem.
15:35 < skvidal> mmcgrath: will do
15:35 < mmcgrath> ivazquez: I thought about that, in the past though, for the most part, things settled down from the release the day after.
15:36 < ivazquez> Okay.
15:36 < mmcgrath> anyone have any questions / comments about that?
15:36 < mmcgrath> alrighty, then we'll open the floor
15:36 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor
15:36  * dgilmore has nothing right now
15:36 < mmcgrath> Anyone have anything they'd like to discuss?  A release is coming, lots of stuff has been going on.
15:37 < notting> how are we ensuring the ability to get the bits to our tier0 mirrors?
15:37 < mmcgrath> notting: same as we had been, AFAIK our mirror system (and I2) is functional again.
15:37 < notting> that's not what mdomsch's mail implied
15:37 < mmcgrath> notting: which email and when?
15:38 < notting> to mirror-list
15:38 < dgilmore> notting: no,  it implied a work around was setup
15:38 < dgilmore> mmcgrath: a couple of hours ago
15:38 < dgilmore> mmcgrath: 2 boxes were setup for Tier 0 I2 masters
15:39 < mmcgrath> <nod> that was my understanding.  We don't have the permanent solution in place but a working one.
15:39 < mmcgrath> we'll probably want to make sure the working one stays as is until after the 30th.
15:39 < dgilmore> mmcgrath: they have static I2 routes to dl.fedora.redhat.com boxes
15:39 < mmcgrath> notting: I'll follow up with mdomsch to make sure whats going on there.
15:39 < mmcgrath> .any mdomsch
15:39 < zodbot> mmcgrath: mdomsch was last seen in #fedora-meeting 1 hour, 9 minutes, and 24 seconds ago: *** mdomsch has quit IRC (Remote closed the connection)
15:39 -!- Milanito [n=Matt at 32.12.70-86.rev.gaoland.net] has quit Remote closed the connection
15:40 < mmcgrath> Alrighty, anyone have anything else to discuss?
15:40 < dgilmore> nope
15:40 < mmcgrath> k, we'll close the meeting in 30
15:41 < mmcgrath> 15
15:41 < mmcgrath> 5
15:41 < notting> dgilmore: so, the static route & download1 are bandwidth-unclogged enough to sync OK?
15:41 < dgilmore> Thanks mmcgrath :)
15:41 < notting> dgilmore: with no one using it, are we sure that download1 is synced? :)
15:41 < dgilmore> notting: i believe so.
15:42 < mmcgrath> notting: we'll have to talk to them to find out for sure.
15:42  * mmcgrath just sent an email to mdomsch to verify.
15:42 < notting> maybe i'm paranoid
15:42 < mmcgrath> notting: you're once bitten.
15:42 < mmcgrath> nothing wrong with that.  The fact is the whole I2 connection thing showed a major problem we have.  and the fact that galgoci is the _only_ person at Red hat with access and no how to fix it is much less comforting.
15:43 < dgilmore> notting: i believe download1   is down
15:43 < mmcgrath> even his backup told us that he was really the only guy we'd want doing that.
15:44 -!- k0k [n=k0k at fedora/k0k] has quit Connection timed out
15:44 < mmcgrath> notting: we'll just have to wait and see if what we think mdomsch said is actually the way things are.  That mirror has been down since before Feb 20th so its hard to say the state of things (considering they only just recently got back up and running)
15:44 < mmcgrath> Anywho, not much we can do but check with the mirrors list and mdomsch and make sure everyone's happy with where things are for the release.
15:44 < mmcgrath> Anyone have anything else?  If not we'll close the meeting in 30
15:45 -!- Milanito [n=Matt at 32.12.70-86.rev.gaoland.net] has joined #fedora-meeting
15:45 < mmcgrath> 10
15:45 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting Closed
15:45 < mmcgrath> Thanks for coming everone!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-infrastructure-list/attachments/20090410/1a74dd06/attachment.sig>