[Freeipa-devel] Dojo and Web UI in 3.2

Tue Oct 30 14:41:08 UTC 2012

On 10/29/2012 08:19 PM, Dmitri Pal wrote:
> On 10/29/2012 05:27 AM, Petr Vobornik wrote:
>> Hi,
>>
>> I would like to make a bigger change in Web UI. Basically I think Web
>> UI would benefit from using a Dojo toolkit, a JS framework. I would
>> like to know if I can proceed with incorporating it. I think this is
>> the best time (end of 3.0/3.1 and a beginning of 3.2) to do such change.
>>
>> Why Dojo?
>> ---------
>> It may solve several existing issues in Web UI architecture/code
>> structure.
>>
>> TLDR version:
>> * incorporates a lot of functionality we don't have to write
>> * is considered to be a framework for enterprise class web applications
>> * reuse an establish library which may be known to more developers
>> then custom build in-house solution
>> * makes code more maintainable
>> * opportunity to separate Web UI framework and IPA specific parts
>>
>> Some functionality of Dojo overlaps with jquery which is not good, but
>> the reason to use Dojo are the features jquery lacks.
>> I also look on other frameworks or independent libraries: AngularJS,
>> backbone.js, closure js framework and several single-purpose libraries
>> (crossroads, hasher, underscore, amd-utils,. IMO Dojo is the best fit
>> for us.
>>
>>  From Dojo I would like to use:
>>
>> ### 1) Build system and AMD modules
>> Web UI doesn't use any JavaScript optimizer therefore we need to keep
>> low file count and minimum size of files. We share one file for
>> multiple components to keep the file count low. It leads to very long
>> files. It's harder to maintain than separate files for each component.
>> AMD modules and a builder can solve it.
>>
>> AMD modules [1] encourages to use one file per component. Then a dojo
>> builder can be used to modify the declaration of a module for file
>> concatenation then it can be sent to JavaScript minifier/compiler.
>>
>> Dojo builder [2] is quite powerfull, it can define 'layers' which
>> serves as a part of application - basically a sets of component so an
>> application with ie. 150 files can be compile to one or more (if
>> needed) .js. Dojo loader takes care of the loading.
>>
>> AFAIU build system can be run at rpm build time using rhino (version
>> 1.7R3-4 needed). Uglify.js can be used as a compiler, it's JS library
>> and can be run in rhino too.  The builder and uglify.js would be
>> needed to budle in our source tarball but they won't be in output rpms
>> so they shouldn't break Fedora packaging rules. Both are BSD licensed.
>> Builder would need a patch to support Uglify (it's commited in dojo
>> upstream trac but I want to incorporated it in latest stable version).
>>
>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/modules/
>> [2] http://dojotoolkit.org/documentation/tutorials/1.8/build/
>>
>> ### 2) Object store/model
>> At the beginning Web UI was quite simple - a command filled widgets,
>> widgets got edited, update command was constructed from widgets, it
>> was sent to  server and at the end widgets were updated with new values.
>>
>> Now various parts of displayed page (action lists, panels, status
>> widgets, facet header) needs to use loaded values too or influence
>> others. This issue was often discussed in patch reviews. As a
>> solution, some model object with bidirectional binding capabilities
>> would be greatly beneficial.
>>
>> Dojo provides a extensible object store [1]. I would like to use
>> 'memory store' and 'observable' and extend it with 'dirty-check'
>> capabilities.
>>
>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/intro_dojo_store/
>>
>> ### 3) Class model
>> Yes, in IPA Web UI we are using some class model, but it has flaws,
>> mainly in initialization of a object (some initialization code of
>> parent class may be called before inheritance is finished).
>>
>> Dojo has a class system [1] which is easy to use and support
>> inheritance, mixins and overrided function calls.
>>
>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/declare/
>>
>> ### 4) Localization
>> Web UI and pages in config, migrate and error folder are not localized
>> until successful login. It's because Web UI gets localized strings by
>> API call which needs prior authentication. We should use other method
>> to provide localized config, error and login pages.
>>
>> Dojo has a easy to use localization solution [1]. Basically strings
>> are defined in .js files as modules, translations in subfolders. Dojo
>> loader and i18n framework take care of loading of proper string
>> depending on user's local. It doesn't need communication with JSON-RPC
>> so can be used in config and login pages.
>>
>> We would  have to make a script to translate the format to a format
>> usable by transifex.
>>
>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/i18n/
>>
>> ### 5) Navigation
>> Our navigation code is limited to IPA entities. Introducing non-entity
>> page like log (#3040) or help is problematic. Routing code should be
>> improved. Separate entity initialization and menu definition. As the
>> re-factoring will be at some point done, we might consider to use
>> dojo.router and dojo.hash instead of $.bbq. This is really optional
>> and not strictly related to Dojo, but I wanted to mention it.
>>
>> Better handling of pkeys may fix `#2741 [ipa webui] Intermittent
>> errors - 'cn' is required; limits exceeded for this query`
>>
>> ### 6) Easier unit tests
>> By utilizing AMD modules, Localization, Model binding we might be able
>> to write unit tests of widgets easier. Now we have to simulate entity,
>> facets, fields, strings loaded from JSON-RPC...
>>
>> ### 7) Separation of framework and IPA
>> We often talk about separating Web UI framework and IPA related stuff.
>> We can do it along with this refactoring.
>> https://fedorahosted.org/freeipa/ticket/3030
>>
>> What I don't want to do
>> -----------------------
>> * rewrite whole UI, changes should be gradual
>> * replace jQuery. Most of our widgets are using jquery, I don't want
>> to change this code, so I don't want to replace it with Dojo widget
>> system - too big effort. So in the result the HTML output should not
>> be changed much - not to affect automated tests.
>>
>> ### Steps how could be Dojo incorporated
>> It should be done gradually.
>> 1) make the build system working. Without it a lot of .js files would
>> be transmitted to browser which is really bad.
>> 2) separate components to individual files and transform then into AMD
>> modules. We might change namespaces and dependencies to separate Web
>> UI framework and IPA related stuff.
>> 3) change class system of entities and facets
>> 4) implement a 'model'
>> —
>> At this point it depends on priorities and new tickets with higher
>> priority may appear.
>>
>> Existing Web UI tickets
>> -----------------------
>> Most of the tickets in 3.2 and 3.3 backlog are nice-to-haves.
>> Therefore this effort should not negatively affect fixing of Web UI
>> bugs or developing features.
>> The only bigger and recommended ticket is:
>> * #2792     [RFE] Allow quering user lock status and unlocking user in UI
>> * and probably a DHCP effort.
>> I already have prepared patches (just waiting for a start of 3.2
>> development to post them) for:
>> * #3200     [RFE] Confirmation by 'Enter' is not supported in all
>> confirmation dialogs
>> * #2884     Improve notification and validation in password reset dialogs
>> * #2910     [RFE] [Web UI] Use "Enter" to confirm add dialogs
>> I think, we can close, as the functionality is already there:
>> * #2682     [RFE]: enable/disable users/HBAC/Sudo in search list
>>
>>
>> Comments are welcome
>
> What is the packaging situation? Is Dojo packaged for Fedora and other
> distros?

There is a build for latest version (1.8.1) in koji and AFAIK in RHEL is 
some older version (1.5?) which lacks many of the features.

I would rather pack it with FreeIPA. I would bundle the Dojo builder 
either way because is not part of the Fedora package. Other thing is 
that the build in the fedora package won't be probably suited for 
FreeIPA - it's not build into single file. IMO Fedora JavaScript 
packaging rules are not good. They don't address web applications needs 
-join multiple libraries to one file depending on application needs. Two 
years ago there was some discussion about improving them (to address 
this issue) on fedora-devel list but there is no conclusion. Anyway 
bundling a JS library is not against the rules.

> How healthy is the community?

 From ohloh.net ( https://www.ohloh.net/p/dojo ):

In a Nutshell, Dojo Toolkit...

     ...
     has had 18,548 commits made by 62 contributors
     representing 637,975 lines of code
     ...
     is mostly written in JavaScript
     with an average number of source code comments
     ...
     has a well established, mature codebase
     maintained by a very large development team
     with decreasing year-over-year commits
     ...
     took an estimated 172 years of effort (COCOMO model)
     starting with its first commit in February, 2007
     ending with its most recent commit 3 days ago

It has quite alive 'interest' mailing list. In this October 702 
messages: 
http://mail.dojotoolkit.org/pipermail/dojo-interest/2012-October/thread.html

This summer they released version 1.8 and are working on version 2. 
Version 1.7, released in December 2011, undertook a major refactoring.

The project is maintained by a Dojo foundation. The main corporate 
contributor is probably SitePen (not sure).

> What is the chance other projects in the distros we work with make the
> decision to use Dojo? Do any of them already do? What is the experience?
>

I don't know about any. IMHO we can only guess. There are big 
differences in developers preferences. Some rather use more server 
technology like some jboss-based-framework or django, some a single page 
application (our web UI).

The only package which requires dojo in Fedora is php-ZendFramework. IDK 
if we can find what Fedora web apps are actually using it.
-- 
Petr Vobornik