[Freeipa-devel] Dojo and Web UI in 3.2

[Dmitri Pal]

On 10/30/2012 01:20 PM, Petr Vobornik wrote:
> On 10/30/2012 06:48 AM, Endi Sukma Dewata wrote:
>> On 10/29/2012 4:27 AM, Petr Vobornik wrote:
>>> Hi,
>>>
>>> I would like to make a bigger change in Web UI. Basically I think
>>> Web UI
>>> would benefit from using a Dojo toolkit, a JS framework. I would
>>> like to
>>> know if I can proceed with incorporating it. I think this is the best
>>> time (end of 3.0/3.1 and a beginning of 3.2) to do such change.
>>
>> I don't know about the timing, but I agree we need a better framework.
>> This was also discussed some time ago with Adam.
>>
>>> Why Dojo?
>>> ---------
>>> It may solve several existing issues in Web UI architecture/code
>>> structure.
>>>
>>> TLDR version:
>>> * incorporates a lot of functionality we don't have to write
>>> * is considered to be a framework for enterprise class web applications
>>> * reuse an establish library which may be known to more developers then
>>> custom build in-house solution
>>> * makes code more maintainable
>>> * opportunity to separate Web UI framework and IPA specific parts
>>>
>>> Some functionality of Dojo overlaps with jquery which is not good, but
>>> the reason to use Dojo are the features jquery lacks.
>>> I also look on other frameworks or independent libraries: AngularJS,
>>> backbone.js, closure js framework and several single-purpose libraries
>>> (crossroads, hasher, underscore, amd-utils,. IMO Dojo is the best fit
>>> for us.
>>
>> I have not used Dojo or any of the above frameworks so I can't really
>> say anything good/bad about them, but backbone.js also seems to be a
>> popular lightweight alternative if we aren't planning to use the Dijit.
>> We should consider whether a framework will work nicely with others or
>> whether it will require us to use it for everything.
>
> I fear that with backbone.js we would have to rewrite the whole UI to
> not bend the framework much. From looking at examples IDK how would we
> incorporate it. I think it would be a nice candidate if we would
> rewrite UI from scratch.
>
> I like Dojo because we can make the changes gradual. In theory we can
> just utilize the things from dojo which we want and leave the rest
> untouched.
>
>>
>> If possible, we should try to address the issues below independently
>> from each other (even if we end up using Dojo for everything) while
>> we're still learning about it. Some of the issues are actually general
>> design issues, so it won't be solved simply by converting the framework.
>> Try to avoid refactoring the code and converting the framework at the
>> same time because the patch could become very big and hard to review.
>
> I agree. Originally I was just looking if there is some small library
> which has a reasonable implementation for bindable object store. To
> keep with saying: "best code is the one you don't have to write". I
> ended with Dojo because it can do a lot of stuff we might need in
> time. Of course all of dojo functionality can be replaced with other
> libraries but I think it's better to use one (and because of AMD
> modules, we can create small custom build) and not to worry about
> making them all work together.
>
>>
>>>  From Dojo I would like to use:
>>>
>>> ### 1) Build system and AMD modules
>>> Web UI doesn't use any JavaScript optimizer therefore we need to keep
>>> low file count and minimum size of files. We share one file for
>>> multiple
>>> components to keep the file count low. It leads to very long files.
>>> It's
>>> harder to maintain than separate files for each component. AMD modules
>>> and a builder can solve it.
>>>
>>> AMD modules [1] encourages to use one file per component. Then a dojo
>>> builder can be used to modify the declaration of a module for file
>>> concatenation then it can be sent to JavaScript minifier/compiler.
>>>
>>> Dojo builder [2] is quite powerfull, it can define 'layers' which
>>> serves
>>> as a part of application - basically a sets of component so an
>>> application with ie. 150 files can be compile to one or more (if
>>> needed)
>>> .js. Dojo loader takes care of the loading.
>>>
>>> AFAIU build system can be run at rpm build time using rhino (version
>>> 1.7R3-4 needed). Uglify.js can be used as a compiler, it's JS library
>>> and can be run in rhino too.  The builder and uglify.js would be needed
>>> to budle in our source tarball but they won't be in output rpms so they
>>> shouldn't break Fedora packaging rules. Both are BSD licensed. Builder
>>> would need a patch to support Uglify (it's commited in dojo upstream
>>> trac but I want to incorporated it in latest stable version).
>>>
>>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/modules/
>>> [2] http://dojotoolkit.org/documentation/tutorials/1.8/build/
>>
>> I suppose now if we want to test the static page without web server we
>> will need to build the pages first?
>
> It's not necessary. For developing purposes it is better to use
> uncompiled files. Loader can load each separate file (if it's AMD
> module) if it isn't in bundle.
>
>>
>> If we use Dojo do we need to include Dojo js files into IPA source, or
>> can we use the ones from Dojo rpm and only include it at build time?
>
> I'm for including it because we can create a build with the parts we
> actually use. Dojo builder isn't in the rpm.
>
> AFAIK the  dojo rpm contains already built files. It has a core
> dojo.js file which contains basic modules to work but the rest of
> modules are separated. IDK if it can be used in a build process. I
> have to test it.
>
>>
>> There's also RequireJS (http://requirejs.org/).
>
> require.js is just AMD loader but it uses r.js as optimizer. r.js
> seems more lightweight than dojo builder. I would stick with dojo
> loader and builder if we use other dojo features.
>
>>
>>> ### 2) Object store/model
>>> At the beginning Web UI was quite simple - a command filled widgets,
>>> widgets got edited, update command was constructed from widgets, it was
>>> sent to  server and at the end widgets were updated with new values.
>>>
>>> Now various parts of displayed page (action lists, panels, status
>>> widgets, facet header) needs to use loaded values too or influence
>>> others. This issue was often discussed in patch reviews. As a solution,
>>> some model object with bidirectional binding capabilities would be
>>> greatly beneficial.
>>>
>>> Dojo provides a extensible object store [1]. I would like to use
>>> 'memory
>>> store' and 'observable' and extend it with 'dirty-check' capabilities.
>>>
>>> [1]
>>> http://dojotoolkit.org/documentation/tutorials/1.8/intro_dojo_store/
>>
>> Backbone also has similar mechanisms for events and sync.
>>
>>> ### 3) Class model
>>> Yes, in IPA Web UI we are using some class model, but it has flaws,
>>> mainly in initialization of a object (some initialization code of
>>> parent
>>> class may be called before inheritance is finished).
>>>
>>> Dojo has a class system [1] which is easy to use and support
>>> inheritance, mixins and overrided function calls.
>>>
>>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/declare/
>>
>> At first glance Backbone seems to be better in this area, but Dojo might
>> also be sufficient for our purposes.
>
> Backbone seems to have advantage in calling super class' constructor.
> It is not recommended in Dojo.
>
>>
>> There's going to be some challenges converting the current class
>> framework. Like you said, the current initialization isn't well
>> structured, so we might need to fix that first before converting the
>> class framework. And then we use 'that' instead of 'this', and sometimes
>> both, and they get confusing, so we'd have to be very careful when
>> converting them. Also, the current way to call a method in super class
>> needs to be fixed.
>
> We can come with some strategy to keep patch reviews simple.
> IE:
> * move initialization stuff to init method (or more fine-grained) and
> use builder to create and instance (in some parts its already done),
> this way we eliminate statements in class declaration (except method
> declaration and overriding)
> * change to the classes: change that to this/self(to keep context in
> closures)
> * eliminate method overrides with inherited() calls.
> * put init calls to constructor
> * change builder again
>
>>
>>> ### 4) Localization
>>> Web UI and pages in config, migrate and error folder are not localized
>>> until successful login. It's because Web UI gets localized strings by
>>> API call which needs prior authentication. We should use other
>>> method to
>>> provide localized config, error and login pages.
>>>
>>> Dojo has a easy to use localization solution [1]. Basically strings are
>>> defined in .js files as modules, translations in subfolders. Dojo
>>> loader
>>> and i18n framework take care of loading of proper string depending on
>>> user's local. It doesn't need communication with JSON-RPC so can be
>>> used
>>> in config and login pages.
>>>
>>> We would  have to make a script to translate the format to a format
>>> usable by transifex.
>>>
>>> [1] http://dojotoolkit.org/documentation/tutorials/1.8/i18n/
>>
>> A possible alternative is jquery-i18n-properties
>> (http://code.google.com/p/jquery-i18n-properties/).
>>
>>> ### 5) Navigation
>>> Our navigation code is limited to IPA entities. Introducing non-entity
>>> page like log (#3040) or help is problematic. Routing code should be
>>> improved. Separate entity initialization and menu definition. As the
>>> re-factoring will be at some point done, we might consider to use
>>> dojo.router and dojo.hash instead of $.bbq. This is really optional and
>>> not strictly related to Dojo, but I wanted to mention it.
>>>
>>> Better handling of pkeys may fix `#2741 [ipa webui] Intermittent errors
>>> - 'cn' is required; limits exceeded for this query`
>>
>> Backbone also has router and history.
>>
>>> ### 6) Easier unit tests
>>> By utilizing AMD modules, Localization, Model binding we might be able
>>> to write unit tests of widgets easier. Now we have to simulate entity,
>>> facets, fields, strings loaded from JSON-RPC...
>>
>> Yes, that would be great.
>>
>>> ### 7) Separation of framework and IPA
>>> We often talk about separating Web UI framework and IPA related stuff.
>>> We can do it along with this refactoring.
>>> https://fedorahosted.org/freeipa/ticket/3030
>>
>> Yes, but it might need to be fixed separately.
>
> I don't want to this all stuff in one patch :).
>>
>>> What I don't want to do
>>> -----------------------
>>> * rewrite whole UI, changes should be gradual
>>> * replace jQuery. Most of our widgets are using jquery, I don't want to
>>> change this code, so I don't want to replace it with Dojo widget system
>>> - too big effort. So in the result the HTML output should not be
>>> changed
>>> much - not to affect automated tests.
>>>
>>> ### Steps how could be Dojo incorporated
>>> It should be done gradually.
>>> 1) make the build system working. Without it a lot of .js files
>>> would be
>>> transmitted to browser which is really bad.
>>
>> This is probably the easiest part. The next steps could be difficult to
>> untangle. So be ready for a long haul. :)
>
> The harder part is not to populate git and rpm with a lot of
> third-party stuff. We need to convince dojo builder to use uglify.js
> under rhino to avoid using of closure compiler or shrink-safe which
> are not and probably will not be bundled in fedora and RHEL and
> definitely we don't want to bundle them.
>
> I'm thinking about making a script which would create WebUI-devoloper
> enviroment:
>  * checkout dojo/dojo and dojo/util git repository to 1.8.1 tag
>  * create a symbolic links to the tag
>  * apply custom patches on dojo/util (there will definitely be one)
>  * include in FreeIPA git only builded dojo.js (in the case we have
> separate files for dojo.js and freeipa.js (just a suggestion of name))
>  * will investigate if we can build the builder and include it in git
> for rpm builds.
>
> We can of course include into our git a whole 1.8.1 dojo/dojo and
> dojo/util but I would rather avoid it because it's a LOT of files.
>
>>
>>> 2) separate components to individual files and transform then into AMD
>>> modules. We might change namespaces and dependencies to separate Web UI
>>> framework and IPA related stuff.
>>> 3) change class system of entities and facets
>>> 4) implement a 'model'
>>> —
>>> At this point it depends on priorities and new tickets with higher
>>> priority may appear.
>>
>
>
Still sounds scary... What is the general recommendation? Are the
arguments for or against Dojo convincing? 

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/