[Pulp-list] Multiple Threads causes Log File Confusion

[Brian Bouterse]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Ben,

Pulp uses many processes which can cause concurrent writes to the log,
but it doesn't use very much multithreading. One thing you can do is
work with your syslog daemon to include the process ids with your log
files. This will help identify which process is producing any given
log statement.

There is one related logging feature of Pulp that helps disambiguate
multi-logs that are broken up as they are logged. In those cases the
Pulp logger adds a (xxxx-xxxx) to the front of each line. The first
xxxx is the PID, the second xxxx is the truncated thread ID.
Unfortunately, this only occurs if the log line is split into multiple
lines so you can't see the thread ID with all log lines.

- -Brian

On 04/30/2015 04:04 AM, ben.stanley at exemail.com.au wrote:
> I have spent the last few days watching /var/log/pulp.log as I
> execute various pulp-admin commands, trying to figure out why the
> pulp-admin commands don't seem to do much. I have complained about
> those problems separately.
> 
> Regarding the structure of the log file:
> 
> I find reading the logs to be very confusing because of the
> multi-threaded nature of pulp. Unfortunately, the messages from
> multiple threads are all combined, and when there is an error, pulp
> often fails to identify what it was doing or what the error was in
> relation to, so it is necessary to scroll back through the log to
> find the context. However, I have begun to realise that the URL
> mentioned previously in the log may be due to a separate thread
> from the one causing the error. This leads to endless confusion.
> Pulp's error messages need a lot of improvement, particularly on
> the pulp-admin front end (where useful information almost doesn't 
> exist), but *also* in the logs. I see that there is a ticket for
> some of this [0], but I think that another ticket is required to
> improve the error reporting in the logs as well.
> 
> Regards, Ben Stanley.
> 
> [0] https://pulp.plan.io/issues/353
> 
> 
> _______________________________________________ Pulp-list mailing
> list Pulp-list at redhat.com 
> https://www.redhat.com/mailman/listinfo/pulp-list
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJVQk/HAAoJEK48cdELyEfyWqoH/1UgimJH/TaEhDAf+ydYdIkl
5kSOpEyFG+FZ704MTFjSbByOmi1NW6PhtDXCCHwuUFt3rtPLnQJh3Lm5ut/bB5zz
J6QBOwb8E55W0lBxJ1oavtR5Pyah4aBvq65/d2QE+v8xGVQzY2JI8j+B3RWp7jxf
Xe3zk8pncDGRiCKKDWfsbMumAWXo9wEabWZ1owkHIPc4ABOExisxJPdTeHa6mgTP
BzEfZX5gSbyeER96MYdAwTqxXYIStkgBnL9pDaQe4lYsU4DzPoLzjqpN4t48hmGq
3foOi9+fVledWZvVU/60R3Xbf7YPCJwgfsiKFMbRLwdeZ/Yz+rgNHiwthkgZX8M=
=Nm1R
-----END PGP SIGNATURE-----