Suitability of Python for daemon processes

[Ben Boeckel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Mike McLean wrote:

> On Mon, Oct 26, 2009 at 12:41 AM, Dennis Gilmore <dennis-
OMEdqnBlyiw at public.gmane.org> wrote:
>> On Sunday 25 October 2009 06:26:49 pm Jeroen van Meeuwen wrote:
>>> And koji.fedoraproject.org, no?
>> koji is a mod_python app.  it doesnt run as a daemon at all.
>> but it it all python.
> 
> There are python daemons in the system though. The builders run kojid,
> which is a daemon, and the process that triggers repo regeneration
> (kojira) is a daemon. Of course, these are not public facing -- they
> really only talk to the hub.
> 
> The daemon distinction might be wrong thing to fixate on here. There
> is nothing in that distinction that should exclude python (or most any
> language). I think the real factors of concern are: size, complexity,
> speed, system load, robustness, and security.
True. I think the speed at which we can deliver with Python makes it much more 
attractive, even if not just at first. We could something similar to what git 
did with bash and replace the Python with C incrementally if necessary, but I 
think that if we go about it right, it shouldn't be much of an issue.

> It's entirely possible to create large and complex systems with
> Python. It's expressiveness and interpreted nature are a big help
> here. Robustness is probably more a factor of design and good coding
> than the language itself. That being said, the relative ease of
> programming in Python, coupled with its exception handling, likely
> give it an advantage over C in this department (over the same
> development time at least). That's my opinion; others may disagree
> (and of course Python is not the only rapid development language with
> good exception handling).
I agree here. This thread is a fact-finding mission and research. We wanted to 
make sure we weren't headed down the wrong road before we headed out.

> Security threats are everywhere. With Python at least you have to
> worry much less about buffer overflows, but of course you can
> introduce security flaws in any language.
Of course.

> Obviously, as an interpreted language, system load and speed are where
> Python suffers. There are optimization tricks, but you'll never get
> close to execution speed of C. You can rewrite crucial portions in C,
> though I rarely see that actually done.
One place where we had concerns were when dealing with the "shopping list" I 
referred to elsewhere in the thread. This list is 60MB on one of the Ubuntu 
mirror directories we have here (our Fedora mirror has been decommissioned while 
we await more disk space). I imagine Fedora's will be much larger due to the 
drpm files and secondary architectures put even more into it. We have code that 
is able to extract information without loading the entire file into our process 
space. It's very close to what it would be if it were C except for a .split() 
call on a string (but it's simple enough that a sscanf could do it as well if 
needed) and the memory/strcmp stuff Python has baked in. Hopefully we will have 
unit tests and timing on our code this weekend.

- --Ben
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQIcBAEBCAAGBQJK6RZ8AAoJEKaxavVX4C1XjsMP/0g54tOwnu0MhsH+uhZDx23N
uTHMs0CCOZaG+e4VJR3xocOU/Xul5IfZ2fDodKai9YwsA9S3ORuvLR0riKh0IbLW
Js8RJW7e2oBWDHUK9nCXIGbuJoPhQrwD+3x88LctxXIy6I34Sw+d8GvrT7vTxgdp
S2keUtAgOOm1JGw+JEAyutTa28yRGnFRLZmopgGAXNtUuFf2nvl77Dm1/R1T/9OE
YcFykF9eEt8LeZ/9M895eGswmASH3LFPTeYOhmz1g5xPHauu1y7wHoHBkfeFieVa
mhtMWLaA8ZfhLKt8FUBUALc0KcDlv42yqyCq7FeKipaaPpyWoHGf6Ce4ZOKgLGPs
5KcDDfVg7xxYH9xoTJ0kiYfy/7ExDr70WUwFg4oRVAl6sEZhVl6+Iz+64UXGVU0z
3o5QH7OKOkyKI9iQheg0b3sXLuhNGyrGz+b0vaEYj7TtoQWHk9qV5UEnoawsTXw+
f51GiQdeZmRNZS1N4m1reX2tUtw0ol1cHcpfpvdqOZazLaSVDwu7CbTyIfalmy79
fDc/yPWWPdcS+mU9LbgqBcEng4lzo/aQSMWHDDzk+r5uIuTibGzslP+hB/fnuuws
ibZvpaDZhLElmqcZbAqMZdHVvS5DCXUjeLcPcx5qJBZ3xrcLnO3/GHapXVec3mIr
d9UZxP4emQ7cOl8DMkbJ
=Nhy3
-----END PGP SIGNATURE-----