[Pulp-dev] Tasking System Changes in 3.0
bbouters at redhat.com
Fri Sep 16 14:45:55 UTC 2016
I'll incorporate the structured saving of a PulpCodedException. That is
a good idea. I'll move all the language to "failed" which sounds natural
instead of "Errored".
Other suggestions are welcome.
On 09/15/2016 09:04 PM, Michael Hrivnak wrote:
> Those sound like great improvements!
> For both fatal and non-fatal errors, in the case where the error raised
> is a PulpCodedException, it would be valuable to capture the extra
> user-facing data in a standard structure so it could be rendered in a
> user-friendly way.
> In the diff, I see a mix of using the words "failed" and "errored".
> Assuming there's not an intentional difference, perhaps we should take
> this opportunity to standardize on one. "Errored" is a bit awkward to
> read and more awkward to say, so I'd lean toward going back to the pulp
> 2 state of just "error", or go with "failed". But my reasoning is only
> aesthetic, so really any of the choices would be fine.
> On Thu, Sep 15, 2016 at 5:46 PM, Brian Bouterse <bbouters at redhat.com
> <mailto:bbouters at redhat.com>> wrote:
> In porting the tasking system to work with the new models I have
> made some adjustments that I think are improvements. Please send
> feedback or leave it directly on the PR. I plan to turn this into
> documentation at some point as a separate effort. This applies to
> methods we will mark in the plugin API such as sync(), publish()
> which are run inside of a pulp task dedicated to calling into the
> plugin code.
> Here is a recap so everyone is aware and can give feedback:
> == Improvements over Fatal vs Non Fatal Exceptions in Tasking System ==
> - The tasking system allows non_fatal_errors to be recorded as the
> task runs. This is different than before which required any error
> recording to be done using the task result. @dkliban pointed out
> that recording as you go is better in case you later experience an
> unexpected, fatal exception. See the new plugin controller here.
> - Fatal exceptions are any exception raised during Task execution.
> This is roughly what we did before but we did a poor job of
> documenting it.
> - Fatal exceptions are recorded with the exception traceback in the
> 'result' attribute. Previously this was recorded in the 'errors'
> attribute which caused mixing/overwriting problems between fatal and
> non-fatal exceptions. In this new code, if you experience a fatal
> error the result *is* your exception. See the implementation here 
> - The errors field is renamed to non_fatal_errors to clearly
> indicate its semantic purpose to collect non_fatal_errors. It is
> still a JSON field but it is now structured due to the use of the
> plugin controller . It probably could be more structured, or
> less. Input would be great here.
> == Less work for plugin writers ==
> - spawned_tasks are now populated as you dispatch tasks from within
> a task. Previously it was the plugin writer's job to name any
> spawned tasks. This is one more responsibility moved to platform.
> See the implementation wherever this method is used . If you want
> to dispatch tasks and not have them in the spawned_tasks list making
> it inherit from PulpTask directly will do that.
> == Less code to Maintain ==
> - We no longer support sigterm handlers and are going to assume that
> a task can be killed at any moment. We deprecated this part of the
> old Plugin API on the 2.y line even. On 2.y, the tasking system used
> to have code which would "install" a sigterm handler callback for
> that specific task. That installation code is now deleted .
> - The Task.results field is a JSON field and it stores anything
> returned by the task . We used to maintain an object called
> TaskResult which no longer has value with all the changes ^. Plugin
> writers can just return a JSON serializable python object as a task
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
More information about the Pulp-dev