[almighty] Architectural Considerations: User based setup (Configs, Wizards, Code)

Thu Jul 28 13:50:51 UTC 2016

On 28 July 2016 at 13:40, Aslak Knutsen <aslak at redhat.com> wrote:
> oops, did not intend to skip list.. see below, comments inline
>
> On Thu, Jul 28, 2016 at 2:22 PM, Max Rydahl Andersen <manderse at redhat.com>
> wrote:
>>
>> was it intentional your skipped the list ? (comments inline)
>>
>> On 28 Jul 2016, at 13:54, Aslak Knutsen wrote:
>>
>> This brings us back to the original federated/distributed idea of using
>> Git
>> as a 'write-through' storage.

Yes, I think this has some distinct advantages, as it allows you to
use a CD approach to configure your own system - you can branch the
ALM with another instance, run up a staging instance, test it, rebase
those changes on other changes etc. This fits very well with a
continuous improvement methodology IMO.

>> For things like the WorkItem Type definitions,
>> * Store changes from UI in the git repo
>> * Expose the repo on some URL
>> That would allow anyone to clone that repo to anywhere and push back
>> changes that are made.

Yes, I referred in a previous email that there are potentially
workflows for a UI that manages a CD-backed configuration. You can
imagine it a bit like:

* user enters admin mode, starts making changes
* hits save, is asked if they want to commit to "master" (the current
instance) or create a branch
* if they create a branch, then they get shown a please wait screen,
whilst the branch is created and the ALM spun up
* they edit away in the branch, then when they are done they have a
screen to "push their changes"
* this creates a PR against their ALM config repo, and runs it through
CI, generating a staging instance
* On merge, the ALM either updates on the fly or restarts, depending
on the scope of the change

This is not a MVP scenario(!), but I think we should consider
architecting for this. I can imagine that we might offer a non-UI
version of this sooner as well.

Related - the customer portal team do something similar with Drupal,
where the continuously deliver certain configuration backed by SCM,
but have other parts of the app backed by a live database.

>> The workflow of updating the definition of the WorkItems that rely on that
>> definition is a different story tho.
>> * We can't assume the user will take on the responsibility of properly
>> versioning the WorkItemTypes so that Core can recreate it.
>> The repo would need to be the latest v of them(?). A change to the repo
>> would trigger core to create a new WorkItemType version to be created.
>>
>> But isn't the above just "another" storage - that it is git is great, but
>> not really solving the issue here right ?
>
> The issue here as far as I can tell is the abillity to confgure certain
> things in the ALM outside of a UI.
>
>>
>> The issue of committing lets say your .jenkinsfile into your projects git
>> repo means that files lifecycle/scope is
>> that project, in that branch.
>>
>> For ALM where a project or lets call it a product might map to multiple
>> projects do you expect to store issues, configuration, etc. into
>> the specific git repo storage for source code ? I've always seen it as a
>> separate git repo.

I think there are probably two levels of configuration - ALM wide, and
project specific. The ALM wide would live in a separate repo, the
project specific in the project repo (default to source code repo, but
overridable). The more we can put in the project repo, the more
powerful our tool IMO. You can also imagine a scenario where we start
applying "ACLs" to certain configs.

If you think of putting ALM config in the project's SCM, then we start
to get some very interesting methods to configuring the ALM itself.

>
>
> Yes, I mean ALM will expose a 'Project/Product' repository for ALM
> configuration. Completly unrelated to any other source code. Think more in
> the lines of how gh-pages work (except the specific branch thing), but for
> ALM configuration. A push to master will Load the data/configuration into
> the ALM, but it might need be activated (might require some additional API
> call to update existing workitems etc etc). But it's a Reposiotry on the
> same line as a Source Repo, so you can 'Plan' issues in the configuration,
> reference changes etc etc
>
> The content would be JSON objects. You can write what ever tool chain your
> heart desires to manipulate them.
> (Not that you couldn't just have a git repo with the REST API payloads and
> build what ever you want around that as well and run a script to invoke it,
> but..)

:-) This matches my rough mental model.

>
>>
>> Which means you do get the benefit of easy sharing and being able to do
>> updates outside of a graphical UI (assuming the format is actually also
>> somewhat humanly readable).
>>
>> But you don't get the benefit of having the config inside your source
>> code.
>
>
> I think the reference to Source Control repo was just tyring to say 'in
> git/svn/hq', not litterally inside the repository where the source code is.
> You'll have 100 source repos with possible conflicting definitions so yea..
>
> (tho I guess we could technically take it one step further and allow 'any'
> repo to contain this type of information and have Build Pipelines for
> updating the ALM it self. It would allow the user to control it, but also
> shoot him self in the foot multiple times.  :)
>>
>> /max
>>
>> -aslak-
>> On Thu, Jul 28, 2016 at 1:30 PM, Max Rydahl Andersen <manderse at redhat.com>
>> wrote:
>>
>> On 28 Jul 2016, at 13:07, Pete Muir wrote:
>> The other thing to think about here is "infrastructure as code",
>>
>> particularly in relation to continuous delivery.
>> How would someone be able to store the ALM configuration in a source
>> control repo, and make changes to the running instance using
>> continuous delivery and at the same time support non-technical users
>> in making changes (such as the UIs that Miroslav discussed).
>>
>> Just to make sure we are talking about the same things here.
>> What ALM configuration are you thinking about specifically ?
>> I'm currently of the mindset that if you are going for the SCM based
>> approach then you are on your own and thus non-technical users cannot make
>> users via
>> a controlled UI. What the ALM will then do is more about being a
>> destination
>> for the build to publish results to rather than be automagically
>> configured.
>> This of course does not mean another project in the same ALM can't support
>> a more UI driven approach.
>> I think this is definitely a scenario we should support, given our
>>
>> focus on supporting companies wanting to use CD/pipelines.
>>
>> +1
>> /max
>>
>> On 27 July 2016 at 08:48, Konrad Kleine <kkleine at redhat.com> wrote:
>>
>> Hi Miroslav,
>> thank you for sharing your thoughts on these three ways to customize an
>> application:
>> 1. Via dedicated UI forms, wizards and DSL
>>
>> 2. Using configuration files
>> 3. Customizing it in application code
>>
>> I believe that 1. and 2. have a somewhat symbiotic effect. Let me try to
>> explain what I mean by this:
>> I think there's the act of customizing and then there is storing of this
>> customization. The later can be a configuration file format (e.g. JSON)
>> with
>> DSL inside. We use PostgreSQL as our database and this can store JSON as
>> native objects. The pure act of customization on the other side can be
>> either writing these JSON files by hand or manipulating them using a UI.
>> I may be naive but I think the benefit of having a configuration file
>> format
>> allows for customization right from the start. A UI can be added later
>> (that
>> is probably the naive part).
>> Regards,
>> Konrad Kleine
>> Software Engineer
>> Red Hat GmbH
>> Werner-von-Siemens-Ring 15
>> 85630 Grasbrunn, Germany
>> On Tue, Jul 26, 2016 at 7:00 PM, Miroslav Hradilek <mhradile at redhat.com>
>> wrote:
>>
>> Hello,
>> this is the first of my emails where I will be sharing my thoughts on
>> some
>> design decisions that, in my opinion, are worth considering when
>> designing
>> ALM architecture.
>> # User Based Setup
>> So in most applications, and especially ones like ALM, there is a big
>> need
>> for customizing everything like objects, behavior etc.
>> It is mostly done in three ways:
>> 1. Via dedicated UI forms, wizards and DSL
>> 2. Using configuration files
>> 3. Customizing it in application code
>> I've personally had a chance to use all three ways throughout my life
>> and
>> I'm sure the readers are the same. All the methods have pros and cons.
>> Some
>> depend on licensing and their application depends on the skill set of
>> the
>> user.
>> # Approaches
>> 1) UI etc
>> This is by far the most consumable method for not very technically
>> skilled
>> people or people not willing to invest in learning the application. The
>> sole
>> configuration form is like reference documentation and configuration
>> files
>> combined. Even higher level of contextual configuration can be achieved
>> by
>> employing wizards which interact with the user until the configuration
>> is
>> finished.
>> The downside to this is that it requires the application developers to
>> burn time to code often quite complex forms. Sometimes the expression of
>> such form is quite limited and Domain Specific Language must be defined
>> to
>> overcome this lack of expression. Example being the languages describing
>> mail filters, query languages etc.
>> After some time the user learns a lot about the application and these
>> interfaces start to limit his/her efficiency.
>> 2) Configuration files
>> Somewhere in between. You do not have to be a coder to configure
>> application this way.
>> What you have to do though, is to read quite a bit about the application
>> and have a reference documentation at hand. This can be minimized by
>> including the documentation inside template configuration files in form
>> of
>> comments. The expression is quite limited. Difficult data structures and
>> Domain Specific Languages must be used to overcome this problem.
>> 3) Hacking the code
>> You need to be involved a lot in the application and you have to
>> understand it's code in order to modify it.
>> The expression is virtually unlimited though with little effort on
>> developers side. No config interpretation code and DSLs are necessary.
>> This
>> can be improved a lot by splitting the configuration part of the code
>> from
>> the system implementation structuring the code as a configuration file.
>> If
>> you include documentation in comments you have a nearly commented config
>> file experience but you still require the user to be quite involved in
>> the
>> app and know at least basic data structures used in the language.
>> The biggest downside is that the app often (not necessarily) needs to be
>> rebuilt to perform a change. Second most severe limitation is that
>> closed
>> source applications would be difficult if not impossible to configure
>> this
>> way.
>> # How to choose
>> There is one important consideration, sometimes overlooked when deciding
>> which style of configuration to employ. Apart from time money and
>> developers
>> personal engagement. It is the TARGET AUDIENCE.
>> Who will be configuring which part in reality? How involved are they
>> going
>> to be in the application when in production? What are their use cases?
>> Are
>> there any security implications if I choose this over that?
>> These are some of the questions you should consider. And please do
>> expand
>> on this. I'm sure you will have some more options to consider.
>> --
>> Miroslav Hradilek
>> Quality Assurance Engineer
>> Base OS Quality Engineering
>> Red Hat Czech, s. r. o.
>> Purkynova 99
>> 612 45 Brno, Czech Republic
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>> /max
>> http://about.me/maxandersen
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>> /max
>> http://about.me/maxandersen
>
>
>
> _______________________________________________
> almighty-public mailing list
> almighty-public at redhat.com
> https://www.redhat.com/mailman/listinfo/almighty-public
>