extras-buildsys/server User.py, NONE, 1.1 CONFIG.py, 1.6, 1.7 buildmaster.py, 1.4, 1.5 buildserver.py, 1.4, 1.5 client_manager.py, 1.6, 1.7
Daniel Williams (dcbw)
fedora-extras-commits at redhat.com
Tue Jun 14 01:16:32 UTC 2005
- Previous message (by thread): extras-buildsys/common FileDownloader.py, 1.3, 1.4 HTTPSURLopener.py, 1.2, 1.3 SSLCommon.py, 1.1, 1.2 SSLXMLRPCServerProxy.py, 1.2, 1.3 SimpleHTTPSServer.py, 1.3, 1.4 SimpleSSLXMLRPCServer.py, 1.2, 1.3
- Next message (by thread): extras-buildsys/utils package-builder, NONE, 1.1 package-builder.py, NONE, 1.1 user-manager.py, NONE, 1.1
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
Author: dcbw
Update of /cvs/fedora/extras-buildsys/server
In directory cvs-int.fedora.redhat.com:/tmp/cvs-serv21732/server
Modified Files:
CONFIG.py buildmaster.py buildserver.py client_manager.py
Added Files:
User.py
Log Message:
2005-06-13 Dan Williams <dcbw at redhat.com>
* client/buildclient.py
common/FileDownloader.py
common/HTTPSURLopener.py
common/SSLCommon.py
common/SSLXMLRPCServerProxy.py
common/SimpleHTTPSServer.py
common/SimpleSSLXMLRPCServer.py
server/client_manager.py
- Clean up cert handling by stuffing cert file paths into a dict
* common/SSLCommon.py
- QuietSSLServer: new class to quiet certain SSL errors we don't care about
* common/SimpleSSLXMLRPCServer.py
- Add an authorization framework. The actual request gets authorized
in SimpleSSLXMLRPCServer, but since we need to pass the authorization
object back up to the Instance class, we need to override a bunch
of functions in superclasses just to pass the object through. Subclasses
of SimpleSSLXMLRPCServer don't need to use authorization, they just don't
set an auth callback. If authorization is in use, the subclass is asked
to authorize the connection, and passes back an arbitrary authorization
object, which gets passed to each Handler method.
* server/buildmaster.py
server/buildserver.py
- Change buildmaster_db -> jobdb
* server/buildserver.py
- XMLRPCBuildMaster -> UserInterface
- Attach authorization to all public XMLRPC methods
- VerifiableSSLXMLRPCServer: new class to handle request authorization
- Add "guest" user support. If the user's cert checks out but they are not
in the user database, they have read-only permission to the server
* server/User.py
- New User object: encapsulates all permissions that a user might
have.
- New UserAuthentication object: looks up user authentication info in
a user database and creates User objects
* utils/user-manager.py
- Local user administration tool
* utils/package-builder.py
- New CLI front-end to the build system that support XMLRPC over SSL
* utils/certs/fedora-upload-ca.cert
- Fedora CVS Upload CA certificate, required to validate users
--- NEW FILE User.py ---
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Library General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
# Copyright 2005 Dan Williams <dcbw at redhat.com> and Red Hat, Inc.
import CONFIG
import sqlite
class User:
def __init__(self, email):
if not email:
raise Exception
self.email = email
self.own_jobs = False
self.kill_any_job = False
self.modify_users = False
self.server_admin = False
class Authenticator:
"""
Talks to a database of users & capabilities
"""
def __init__(self):
self.dbcx = sqlite.connect("userdb", encoding="utf-8", timeout=2)
self.curs = self.dbcx.cursor()
# Ensure the table exists in the database
create = False
try:
self.curs.execute('SELECT * FROM users')
self.dbcx.commit()
except sqlite._sqlite.DatabaseError, e:
create = True
if create:
self.curs.execute('CREATE TABLE users (email VARCHAR(50), ' \
'own_jobs BOOLEAN, kill_any_job BOOLEAN, ' \
'modify_users BOOLEAN, server_admin BOOLEAN)')
self.dbcx.commit()
def new_authed_user(self, email, client_address):
if not email:
return None
user = None
self.curs.execute('SELECT email, own_jobs, kill_any_job, modify_users, ' \
'server_admin FROM users WHERE email="%s"' % email)
self.dbcx.commit()
item = self.curs.fetchone()
if item:
user = User(email)
user.own_jobs = item['own_jobs']
user.kill_any_job = item['kill_any_job']
user.modify_users = item['modify_users']
user.server_admin = item['server_admin']
else:
if CONFIG.get('guest_allowed'):
user = User('guest at guest')
return user
Index: CONFIG.py
===================================================================
RCS file: /cvs/fedora/extras-buildsys/server/CONFIG.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- CONFIG.py 12 Jun 2005 08:23:43 -0000 1.6
+++ CONFIG.py 14 Jun 2005 01:16:30 -0000 1.7
@@ -9,6 +9,7 @@
config_opts['make_cmd'] = "/usr/bin/make"
config_opts['tmpdir'] = "/tmp"
config_opts['log_url'] = "http://foo.foo.org/logs/"
+config_opts['guest_allowed'] = True
SERVER_BASE_DIR = "/work/fedora-cvs/extras-buildsys/server"
@@ -17,6 +18,7 @@
config_opts['server_cert'] = SERVER_BASE_DIR + "/certs/server_cert.pem"
config_opts['server_key'] = SERVER_BASE_DIR + "/certs/server_key.pem"
config_opts['ca_cert'] = SERVER_BASE_DIR + "/certs/ca_cert.pem"
+config_opts['ui_ca_cert'] = SERVER_BASE_DIR + "/certs/fedora-upload-ca.pem"
# server_work_dir
# - Where logs and finished RPMs are stored
Index: buildmaster.py
===================================================================
RCS file: /cvs/fedora/extras-buildsys/server/buildmaster.py,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- buildmaster.py 12 Jun 2005 08:23:43 -0000 1.4
+++ buildmaster.py 14 Jun 2005 01:16:30 -0000 1.5
@@ -46,8 +46,7 @@
self.hostname = hostname
self.building_jobs = []
self.should_stop = False
- self.dbcx = sqlite.connect("buildmaster_db", encoding="utf-8",
- timeout=2)
+ self.dbcx = sqlite.connect("jobdb", encoding="utf-8", timeout=2)
self.curs = self.dbcx.cursor()
ensure_build_db_tables(self.dbcx)
threading.Thread.__init__(self)
@@ -65,7 +64,7 @@
% (status, job_uid))
def run(self):
- while True:
+ while self.should_stop == True:
# Update all build clients and known jobs
self.bcm.process()
@@ -90,6 +89,4 @@
job.start()
time.sleep(5)
- if self.should_stop == True:
- break
Index: buildserver.py
===================================================================
RCS file: /cvs/fedora/extras-buildsys/server/buildserver.py,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- buildserver.py 12 Jun 2005 08:23:43 -0000 1.4
+++ buildserver.py 14 Jun 2005 01:16:30 -0000 1.5
@@ -21,8 +21,6 @@
import socket
import sys
import os
-import SimpleXMLRPCServer
-import xmlrpclib
import sqlite
import smtplib
from email.MIMEText import MIMEText
@@ -31,6 +29,9 @@
from buildmaster import ensure_build_db_tables
from client_manager import BuildClientManager
import SimpleHTTPSServer
+import SimpleSSLXMLRPCServer
+import User
+
def email_result(username, cvs_tag, resultstring, subject=None):
"""send 'resultstring' to username"""
@@ -48,11 +49,10 @@
s.close()
-class XMLRPCBuildMaster:
+class UserInterface:
def __init__(self, client_manager):
self.bcm = client_manager
- self.dbcx = sqlite.connect("buildmaster_db", encoding="utf-8",
- timeout=2)
+ self.dbcx = sqlite.connect("jobdb", encoding="utf-8", timeout=2)
self.curs = self.dbcx.cursor()
ensure_build_db_tables(self.dbcx)
@@ -76,9 +76,12 @@
item = self.curs.fetchone()
return item['uid']
- def enqueue(self, username, package, cvs_tag, target, buildreq=None):
+ def enqueue(self, user, username, package, cvs_tag, target, buildreq=None):
""" Accept a job to build and stuff it into the job database """
+ if not user.own_jobs:
+ return (-1, "Insufficient privileges.")
+
if CONFIG.get('use_srpm_not_cvs') == True:
email_result(username, cvs_tag, "Error setting up build for %s on "\
"%s: this server builds SRPMs, not CVS checkouts." % (cvs_tag, target))
@@ -98,9 +101,12 @@
jobid = self._insert_job(username, package, cvs_tag, target, buildreq, time.time())
return (0, "Success: package has been queued.", jobid)
- def enqueue_srpm(self, username, package, srpm_file, target, buildreq=None):
+ def enqueue_srpm(self, user, username, package, srpm_file, target, buildreq=None):
""" Accept a job to build from SRPM file and stuff it into the job database """
+ if not user.own_jobs:
+ return (-1, "Insufficient privileges.")
+
if CONFIG.get('use_srpm_not_cvs') == False:
email_result(username, srpm_file, "Error setting up build for %s on "\
"%s: this server builds CVS checkouts, not SRPMS." % (srpm_file, target))
@@ -126,7 +132,8 @@
jobid = self._insert_job(username, package, cvs_tag, target, buildreq, time.time())
return (0, "Success: package has been queued.", jobid)
- def list_waiting_jobs(self):
+
+ def list_waiting_jobs(self, user):
self.curs.execute('SELECT uid, username, package, cvs_tag, target' \
' FROM jobs WHERE status="waiting"')
self.dbcx.commit()
@@ -137,7 +144,8 @@
job_list.append(tempX)
return job_list
- def list_building_jobs(self):
+
+ def list_building_jobs(self, user):
self.curs.execute('SELECT uid, username, package, cvs_tag, target' \
' FROM jobs WHERE status="building"')
self.dbcx.commit()
@@ -148,24 +156,46 @@
job_list.append(tempX)
return job_list
- def update_clients(self):
+
+ def update_clients(self, user):
+ if not user.server_admin:
+ return (-1, "Insufficient privileges.")
+
reload(CONFIG)
print "-----------------------------------------------------"
print " Looking for Build Clients..."
self.bcm.update_clients()
print "-----------------------------------------------------\n"
- return 0
+ return (0, "Success.")
-class MyXMLRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):
- """ XMLRPC server subclass that turns on SO_REUSEADDR """
-
- def __init__(self, address):
- SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, addr=address, logRequests=False)
-
- def server_bind(self):
- self.socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
- self.socket.bind(self.server_address)
+# Unused for SSL connections, but keep around just in case
+# we give users the option of not using SSL at some later date
+#
+#class MyXMLRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):
+# def __init__(self, address):
+# self.allow_reuse_address = 1
+# SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, address, logRequests=0)
+
+
+class VerifiableSSLXMLRPCServer(SimpleSSLXMLRPCServer.SimpleSSLXMLRPCServer):
+ def __init__(self, certs, address):
+ SimpleSSLXMLRPCServer.SimpleSSLXMLRPCServer.__init__(self, certs, address, self.auth_cb)
+ self.authenticator = User.Authenticator()
+
+ def auth_cb(self, request, client_address):
+ """
+ Authenticate the user and determine user's privs.
+ TODO: pull user privs from a DB
+ """
+ peer_cert = request.get_peer_cert()
+ email = peer_cert.get_subject().emailAddress
+ user = None
+ try:
+ user = self.authenticator.new_authed_user(email, client_address)
+ except Exception:
+ pass
+ return user
if __name__ == '__main__':
@@ -182,25 +212,33 @@
bm = BuildMaster(hostname, bcm)
bm.start()
- # Create the BuildMaster XMLRPC server
- xmlrpc_bm = XMLRPCBuildMaster(bcm)
- bm_server = MyXMLRPCServer((hostname, 8887))
- bm_server.register_instance(xmlrpc_bm)
-
# SSL certificate and key filenames
- server_cert = CONFIG.get('server_cert')
- server_key = CONFIG.get('server_key')
- ca_cert = CONFIG.get('ca_cert')
+ srpm_server_certs = {}
+ srpm_server_certs['cert'] = CONFIG.get('server_cert')
+ srpm_server_certs['key'] = CONFIG.get('server_key')
+ srpm_server_certs['ca_cert'] = CONFIG.get('ca_cert')
+ srpm_server_certs['peer_ca_cert'] = CONFIG.get('ca_cert')
+
+ ui_certs = {}
+ ui_certs['cert'] = CONFIG.get('server_cert')
+ ui_certs['key'] = CONFIG.get('server_key')
+ ui_certs['ca_cert'] = CONFIG.get('ca_cert')
+ ui_certs['peer_ca_cert'] = CONFIG.get('ui_ca_cert')
+
+ # Create the BuildMaster XMLRPC server
+ ui = UserInterface(bcm)
+ bm_server = VerifiableSSLXMLRPCServer(ui_certs, (hostname, 8887))
+ bm_server.register_instance(ui)
# SRPM fileserver
http_dir = os.path.join(CONFIG.get('server_work_dir'), "srpm_http_dir")
- srpm_server = SimpleHTTPSServer.SimpleHTTPSServer(server_cert, server_key, ca_cert, (hostname, 8886), http_dir)
+ srpm_server = SimpleHTTPSServer.SimpleHTTPSServer(srpm_server_certs, (hostname, 8886), http_dir)
srpm_server.start()
print "BuildMaster accepting requests on %s:8887.\n" % hostname
try:
bm_server.serve_forever()
- except Exception:
+ except KeyboardInterrupt:
# Make sure the BuildMaster thread shuts down
print "Shutting down..."
bm.stop()
Index: client_manager.py
===================================================================
RCS file: /cvs/fedora/extras-buildsys/server/client_manager.py,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- client_manager.py 12 Jun 2005 08:23:43 -0000 1.6
+++ client_manager.py 14 Jun 2005 01:16:30 -0000 1.7
@@ -30,9 +30,11 @@
# SSL certificate and key filenames
-server_cert = CONFIG.get('server_cert')
-server_key = CONFIG.get('server_key')
-ca_cert = CONFIG.get('ca_cert')
+certs = {}
+certs['cert'] = CONFIG.get('server_cert')
+certs['key'] = CONFIG.get('server_key')
+certs['ca_cert'] = CONFIG.get('ca_cert')
+certs['peer_ca_cert'] = CONFIG.get('ca_cert')
def result_is_finished(result):
@@ -191,7 +193,7 @@
self._manager = manager
self._jobs = []
self._address = address
- self._server = SSLXMLRPCServerProxy.SSLXMLRPCServerProxy(server_cert, server_key, ca_cert, self._address)
+ self._server = SSLXMLRPCServerProxy.SSLXMLRPCServerProxy(certs, self._address)
self._unavail_count = 0
self._arches = []
try:
- Previous message (by thread): extras-buildsys/common FileDownloader.py, 1.3, 1.4 HTTPSURLopener.py, 1.2, 1.3 SSLCommon.py, 1.1, 1.2 SSLXMLRPCServerProxy.py, 1.2, 1.3 SimpleHTTPSServer.py, 1.3, 1.4 SimpleSSLXMLRPCServer.py, 1.2, 1.3
- Next message (by thread): extras-buildsys/utils package-builder, NONE, 1.1 package-builder.py, NONE, 1.1 user-manager.py, NONE, 1.1
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
More information about the fedora-extras-commits
mailing list