[Pulp-dev] Tasking System Changes in 3.0
bbouters at redhat.com
Thu Sep 15 21:46:29 UTC 2016
In porting the tasking system to work with the new models I have made
some adjustments that I think are improvements. Please send feedback or
leave it directly on the PR. I plan to turn this into documentation at
some point as a separate effort. This applies to methods we will mark in
the plugin API such as sync(), publish() which are run inside of a pulp
task dedicated to calling into the plugin code.
Here is a recap so everyone is aware and can give feedback:
== Improvements over Fatal vs Non Fatal Exceptions in Tasking System ==
- The tasking system allows non_fatal_errors to be recorded as the task
runs. This is different than before which required any error recording
to be done using the task result. @dkliban pointed out that recording as
you go is better in case you later experience an unexpected, fatal
exception. See the new plugin controller here.
- Fatal exceptions are any exception raised during Task execution. This
is roughly what we did before but we did a poor job of documenting it.
- Fatal exceptions are recorded with the exception traceback in the
'result' attribute. Previously this was recorded in the 'errors'
attribute which caused mixing/overwriting problems between fatal and
non-fatal exceptions. In this new code, if you experience a fatal error
the result *is* your exception. See the implementation here 
- The errors field is renamed to non_fatal_errors to clearly indicate
its semantic purpose to collect non_fatal_errors. It is still a JSON
field but it is now structured due to the use of the plugin controller
. It probably could be more structured, or less. Input would be great
== Less work for plugin writers ==
- spawned_tasks are now populated as you dispatch tasks from within a
task. Previously it was the plugin writer's job to name any spawned
tasks. This is one more responsibility moved to platform. See the
implementation wherever this method is used . If you want to dispatch
tasks and not have them in the spawned_tasks list making it inherit from
PulpTask directly will do that.
== Less code to Maintain ==
- We no longer support sigterm handlers and are going to assume that a
task can be killed at any moment. We deprecated this part of the old
Plugin API on the 2.y line even. On 2.y, the tasking system used to have
code which would "install" a sigterm handler callback for that specific
task. That installation code is now deleted .
- The Task.results field is a JSON field and it stores anything returned
by the task . We used to maintain an object called TaskResult which
no longer has value with all the changes ^. Plugin writers can just
return a JSON serializable python object as a task result.
More information about the Pulp-dev