[Fwd: Tosca widgets, only half the battle]

[Toshio Kuratomi]

> 
> After hacking away at MyFedora and producing a lot of ugly code in the
> process I finally sat down the last two weeks to organize everything
> into a framework make it much more extensible and have patterns for
> people to easily create content.  Most of the technologies are
> solidifying into my head and I have been working on hashing out an API
> design behind the user interaction design I had started with.  The issue
> I am running into now is the fact that Turbo Gears and related
> technology come from a monolithic design and adhere too stringently to
> the Model/View/Controller design pattern.  This is really an issue when
> your models, views and controllers can come from different applications
> or even different servers.  MyFedora is of course a mashup of different
> tools and does not fit the, I'm grabbing data from a single database and
> displaying it via a self contained template, mold.  What I need is a
> complete plugin system where a person can write their own self contained
> controllers, templates and static files which then drop in and are
> loaded on the fly, while integrating with the global project.
> 
FAS2 is at a similar stage.  We want other people to be able to use 
FAS2.  So we want to pull Fedora specific portions into a plugin 
architecture.  We've started using setuptools entry-points since that's 
the plugin architecture that TurboGears already uses but that only 
solves how to load the modules, I think we got templates to work after a 
bit of hacking but I'd have to revisit the code.  Getting the model to 
work is the big sticking point as the model means either:
1) A new db and a new set of db connections
2) Somehow merging the plugin's model with the main apps model.

In myfedora, you might be able to use setuptools entry-points to better 
effect since you're operating largely without a model.  If the plugins 
don't have to save state or can save very minimal state into a generic 
configs table you should be able to use entry-points to load up the code.

> Before I go further let me describe my design.
> 
> Vocabulary:
> 
> Resource - This is the starting point for MyFedora plugins.  A resource
> is any abstract grouping such as "packages", "people" and "projects"
> which contain tools for viewing and manipulating data within the
> resource's context.
> 
> Tools - A tool is a web app for viewing or manipulating data.  For
> example Builds would be a tool for the package resource.
> 
> Data Id - The data id is a pointer to a specific dataset the tools work
> on.  For example the package resource considers each fedora package name
> to be a data_id.
> 
> The way things work are Resources are placed in the resources/ directory
> and contain the logic for routing requests to a specific tool.  They
> also contain the master template which is a cause of path problems with
> the current TG setup (include paths are relative to the including
> template)
> 
This is how we fixed that in fas::

   <?python
   import genshi.template.plugin
   master = 
genshi.template.plugin.MarkupTemplateEnginePlugin().load_template('fas.templates.master')
   ?>
   <xi:include href="${master.filename}" />


> Tools are placed in the tools/ directory and are controllers just like
> any other TG controller.  The exception is there is a standard for
> including the master template and the tool pulls templates and static
> files from its own directory.  Tools can register with more than one
> resource and must modify its behavior based on the resource calling it.
> For instance the Build tool would be able to register with the package
> and people resource and depending which resource is being used it would
> display either a specific person's builds or the build history of a
> package.  Based on the resource being used the master template is pulled
> in by the tool's templates.
>
That's pretty nifty.

> Data id's are simply what the resource passes to the tool and the tool
> needs to be able to accept when dealing with a particular resource.  For
> instance the Packages resource would send a package name as a data id
> and the Peoples resource would send a person's FAS username.
> 
An aside: It sounds like we'd want to implement some sort of global 
cache at some point.  If we always start from the data_ids there's a lot 
of data to be pulled in each request.  And if tools are using similar 
pieces of information but displaying it differently (for instance, when 
using a tool in a different resource) we'd definitely see reuse of the data.

> The issue here is I need the tools to be self contained but still
> integrate correctly with the global assests such as master templates and
> graphics.  Tosca widgets seemed to be the answer until I looked further
> and found out they are just a higher level display layer than a self
> contained controller/template system.  It seems to be confusing because
> it breaks the connection between the controller, data and the display
> when I want that all to be encapsulated.  Basically I don't want the
> master page dolling out the data because the master page is just a
> container to display the tool and links to other tools.  The tools
> should know where to get their data from.
> 
<nod>  You need plugins, not widgets for what you want to do.

> One solution is to use ToscaWidgets as a replacement for templates (or
> more apt another layer between the controller and the template).  That
> makes things more complicated and throws away a lot of the concepts of
> TG controllers.  I guess I am probably just hung up on how I first
> learned TG and we can just document around those issues.  But another
> thing to think about is stuff like WSGI.
> 
I think we think similarly here that ToscaWidgets isn't the ideal 
solution for this problem.

> What do you guys think? Given my design and goals such as the ability to
> display tools on the portal page, what is our plan of attack?  How do we
> concoct a plugin system to make it easy for others to create integrated
> content while really just concentrating on their bits and not the wider
> integration infrastructure?  Are there systems/libraries out there that
> already do this? Tosca is only part of the solution because it only
> deals with encapsulating display and is mostly geared to generic widgets
> like lists and not complete pages.  I would like to have a framework
> that is simple, focused and easy to use.
> 

I suppose part of the question is who do we want to be building these 
plugins for MyFedora?  If it's the author of the app providing the data 
then it should be something that their app can also use.  If it's 
MyFedora developers, then it should be easy for them to create a new 
controller and setup a plugin.  We might want to write a small script 
that makes a sample plugin directory like tg-admin quickstart does for 
the main app.

Take a look at how fas does things:  fas/plugins/dummy_plugin.
That should pretty much be a self-contained plugin using setuptools 
entrypoints to do its dirty work.

-Toshio