[Freeipa-devel] Dojo and Web UI in 3.2

[Petr Vobornik]

Hi,

I would like to make a bigger change in Web UI. Basically I think Web UI 
would benefit from using a Dojo toolkit, a JS framework. I would like to 
know if I can proceed with incorporating it. I think this is the best 
time (end of 3.0/3.1 and a beginning of 3.2) to do such change.

Why Dojo?
---------
It may solve several existing issues in Web UI architecture/code structure.

TLDR version:
* incorporates a lot of functionality we don't have to write
* is considered to be a framework for enterprise class web applications
* reuse an establish library which may be known to more developers then 
custom build in-house solution
* makes code more maintainable
* opportunity to separate Web UI framework and IPA specific parts

Some functionality of Dojo overlaps with jquery which is not good, but 
the reason to use Dojo are the features jquery lacks.
I also look on other frameworks or independent libraries: AngularJS, 
backbone.js, closure js framework and several single-purpose libraries 
(crossroads, hasher, underscore, amd-utils,. IMO Dojo is the best fit 
for us.

 From Dojo I would like to use:

### 1) Build system and AMD modules
Web UI doesn't use any JavaScript optimizer therefore we need to keep 
low file count and minimum size of files. We share one file for multiple 
components to keep the file count low. It leads to very long files. It's 
harder to maintain than separate files for each component. AMD modules 
and a builder can solve it.

AMD modules [1] encourages to use one file per component. Then a dojo 
builder can be used to modify the declaration of a module for file 
concatenation then it can be sent to JavaScript minifier/compiler.

Dojo builder [2] is quite powerfull, it can define 'layers' which serves 
as a part of application - basically a sets of component so an 
application with ie. 150 files can be compile to one or more (if needed) 
.js. Dojo loader takes care of the loading.

AFAIU build system can be run at rpm build time using rhino (version 
1.7R3-4 needed). Uglify.js can be used as a compiler, it's JS library 
and can be run in rhino too.  The builder and uglify.js would be needed 
to budle in our source tarball but they won't be in output rpms so they 
shouldn't break Fedora packaging rules. Both are BSD licensed. Builder 
would need a patch to support Uglify (it's commited in dojo upstream 
trac but I want to incorporated it in latest stable version).

[1] http://dojotoolkit.org/documentation/tutorials/1.8/modules/
[2] http://dojotoolkit.org/documentation/tutorials/1.8/build/

### 2) Object store/model
At the beginning Web UI was quite simple - a command filled widgets, 
widgets got edited, update command was constructed from widgets, it was 
sent to  server and at the end widgets were updated with new values.

Now various parts of displayed page (action lists, panels, status 
widgets, facet header) needs to use loaded values too or influence 
others. This issue was often discussed in patch reviews. As a solution, 
some model object with bidirectional binding capabilities would be 
greatly beneficial.

Dojo provides a extensible object store [1]. I would like to use 'memory 
store' and 'observable' and extend it with 'dirty-check' capabilities.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/intro_dojo_store/

### 3) Class model
Yes, in IPA Web UI we are using some class model, but it has flaws, 
mainly in initialization of a object (some initialization code of parent 
class may be called before inheritance is finished).

Dojo has a class system [1] which is easy to use and support 
inheritance, mixins and overrided function calls.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/declare/

### 4) Localization
Web UI and pages in config, migrate and error folder are not localized 
until successful login. It's because Web UI gets localized strings by 
API call which needs prior authentication. We should use other method to 
provide localized config, error and login pages.

Dojo has a easy to use localization solution [1]. Basically strings are 
defined in .js files as modules, translations in subfolders. Dojo loader 
and i18n framework take care of loading of proper string depending on 
user's local. It doesn't need communication with JSON-RPC so can be used 
in config and login pages.

We would  have to make a script to translate the format to a format 
usable by transifex.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/i18n/

### 5) Navigation
Our navigation code is limited to IPA entities. Introducing non-entity 
page like log (#3040) or help is problematic. Routing code should be 
improved. Separate entity initialization and menu definition. As the 
re-factoring will be at some point done, we might consider to use 
dojo.router and dojo.hash instead of $.bbq. This is really optional and 
not strictly related to Dojo, but I wanted to mention it.

Better handling of pkeys may fix `#2741 [ipa webui] Intermittent errors 
- 'cn' is required; limits exceeded for this query`

### 6) Easier unit tests
By utilizing AMD modules, Localization, Model binding we might be able 
to write unit tests of widgets easier. Now we have to simulate entity, 
facets, fields, strings loaded from JSON-RPC...

### 7) Separation of framework and IPA
We often talk about separating Web UI framework and IPA related stuff. 
We can do it along with this refactoring. 
https://fedorahosted.org/freeipa/ticket/3030

What I don't want to do
-----------------------
* rewrite whole UI, changes should be gradual
* replace jQuery. Most of our widgets are using jquery, I don't want to 
change this code, so I don't want to replace it with Dojo widget system 
- too big effort. So in the result the HTML output should not be changed 
much - not to affect automated tests.

### Steps how could be Dojo incorporated
It should be done gradually.
1) make the build system working. Without it a lot of .js files would be 
transmitted to browser which is really bad.
2) separate components to individual files and transform then into AMD 
modules. We might change namespaces and dependencies to separate Web UI 
framework and IPA related stuff.
3) change class system of entities and facets
4) implement a 'model'
—
At this point it depends on priorities and new tickets with higher 
priority may appear.

Existing Web UI tickets
-----------------------
Most of the tickets in 3.2 and 3.3 backlog are nice-to-haves. Therefore 
this effort should not negatively affect fixing of Web UI bugs or 
developing features.
The only bigger and recommended ticket is:
* #2792 	[RFE] Allow quering user lock status and unlocking user in UI
* and probably a DHCP effort.
I already have prepared patches (just waiting for a start of 3.2 
development to post them) for:
* #3200 	[RFE] Confirmation by 'Enter' is not supported in all 
confirmation dialogs
* #2884 	Improve notification and validation in password reset dialogs
* #2910 	[RFE] [Web UI] Use "Enter" to confirm add dialogs
I think, we can close, as the functionality is already there:
* #2682 	[RFE]: enable/disable users/HBAC/Sudo in search list


Comments are welcome
-- 
Petr Vobornik