Architectural Changes

[Mike McGrath]

As we talked about in the meeting yesterday we have a new sponsor 
(http://www.teliasonera.com/).  There are a couple of others in the 
works (I don't want to officially announce until its finalized) but one 
thing is clear.  Pretty soon we're going to have multiple proxy servers 
outside of PHX.  The end goal here would be to use  mod_geoip to 
re-direct people to their nearest location but we're going to take baby 
steps to get there.  Here are the steps as I see them.

1) Finalize the caching stuff paulobanon has been working on.
2) VPN
3) Setup 1 remote proxy server and test
4) Get DNS setup properly to direct people to the proxy servers in a RR 
format
5) mod_geoip.


4) is still a little fuzzy in my mind.  Right now we're using Bind for 
DNS and, AFAIK, the version we're using does not have support for 
geoip.  So my thought is using mod_geoip to direct people to (for 
example) de1.fedoraproject.org or us2.fedoraproject.org.  I'm still a 
little unclear on the best way to do this in our environment.  Those 
keeping an eye on the commit logs will have noticed the odd commit for 
t.fedoraproject.org.  So, for example:

ping -c1 t.fedoraproject.org

For me seems to do the right thing.  I get basically a RR balanced IP 
between 3 addresses (fp.o, yahoo and google)  I just picked two ip's 
that weren't ours to balance around.  The thing, for me at least, is I 
get fp.o every time if I use FireFox.  This is over many days on 
different computers.  I've seen FF bring up the google ip once.  So I 
ask those on the list to go to http://t.fedoraproject.org/ and just tell 
me what you get.  Or, even better, explain to me what the heck is going 
on there, I have one theory about first requests to DNS vs named caching 
in FF and name caching elsewhere.  But we've had different people get 
many different results (some get wget to RR, some with wget always get 
the same thing, same with curl, lynx, w3m, and HEAD)  More investigation 
is needed.

2) is something I'm working on now.  VPN will only be for external 
servers (not users).  We've actually already had a few issues we've had 
to overcome in strange ways from external servers that could have been 
fixed by a VPN.  (puppet and bacula backups immediately come to mind)  
We'll tightly control (iptables) what these boxes have access to on the 
vpn server (bastion).  We'll keep the ttl on our load balanced products 
lower so that if something does go wrong with one of them, we can easily 
take it out of the mix.

The reason for 2) is so we don't have to maintain multiple different 
proxy server types.  If we use VPN we can treat each server the same, 
just like the ones we have now which keeps it maintainable.

Questions / Comments / Suggestions?

    -Mike