<div dir="ltr">Thanks Tomas for the clarification ! <div><div class="gmail_extra"> <div class="gmail_quote">On Mon, Oct 24, 2016 at 2:41 PM, Tomas Nozicka <<a href="mailto:tnozicka@redhat.com" target="_blank">tnozicka@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks for the feedback, Shoubhik. Small clarification inline... <div><div class="h5"> On Fri, 2016-10-21 at 23:18 +0530, Shoubhik Bose wrote: > Thanks Tomas, > The overview is detailed and inline with what I was thinking about , > and generally implemented. > Responses inline ---- > On Oct 21, 2016 9:18 PM, "Tomas Nozicka" <<a href="mailto:tnozicka@redhat.com">tnozicka@redhat.com</a>> wrote: > > > > Hi Shoubhik, > > > > you have a good point that I haven't described OAuth specifics in > my > > post, just mentioned it several times. That's mostly because I > haven't > > really decided yet on which approach to take and that mail was > already > > a long one to try to attack this issue. > > > > Continues inline... > > > > On Thu, 2016-10-20 at 19:02 +0530, Shoubhik Bose wrote: > > > Hi Tomas, > > > Very nice overview! > > > I had a few thoughts on the auth tokens. > > > I see it mentioned in a number of sections and wanted to share a > generalized understanding: > > > 1. ALM would need to be pre configured with OAuth application id > + application secret for every external system we integrate with for > builds. These could be read by alm-core as environment variables and > passed on to the SPIs when needed. The environment variables can be > set as Openshift Secrets. > > > This allows ALM to identify itself when it talks to the external > build system. > > Well, in the approach you are correct. Although I don't > > think "application id + application" is actually a requirement for > > OAuth. It is just a way how some services provide you with OAuth > > credentials to get token, but there are other ways. For example > > OpenShift does not have those. (It has a concept of serviceaccounts > > which by themselves provide you with a special long-term token to > use > > for authentication. But they need a namespace to be created in, > setup > > permissions for other namespaces; generally some non-trivial > > management). > > > This sounds good. I agree, the applicationid and application secret > are very external system specific and different systems have > different ways of identifying who is accessing the specific external > system. > > The way I see it: > > - User will enter a URL of OAuth resource (like build environment) > to > > UI (e.g. <a href="http://openshift.com" rel="noreferrer" target="_blank">openshift.com</a>) > > - He will be redirected to <a href="http://openshift.com" rel="noreferrer" target="_blank">openshift.com</a> and asked for his regular > > credentials > > - Almighty will get back a token that will be stored in > configuration > > with the URL > > > > The only issue with this approach is that the token might have > limited > > lifetime so when running build, Almighty shall detect expiry and > ask > > user to refresh token by redirecting him back to <a href="http://openshift.com" rel="noreferrer" target="_blank">openshift.com</a>. But > we > > should be prepared to do the same even if it would be long-term > tokens > > because they can still expire someday. > > > Yes this is the conventional procedure/approach. > > There also a "backup" and more complicated way that we let user to > > login by username and password and provider will setup long-term > > service account on his behalf, but I would rather not go this way; > at > > least for now. (It may be needed in case of cross cluster > deployments > > but in a limited way.) > Do you mean that this is not the usual OAuth workflow but that a > username/password would eventually generate a server token? </div></div>Well, not really. It is a regular OAuth, the difference is with how you setup the OAuth server. Let's be more concrete and use an example with OpenShift here: 1. You can login to OpenShift's OAuth by using your regular credentials (user: developer, password: developer); you will receive short-term token and can do anything that user can 2. You can create service account with associated (long-term) token (Alternative to appid+secret, which provide you with such token after login in) and have some scoped (limited) access (it doesn't have to be limited; scope can be "full") 3. You can use 1. to create 2. on users behalf and use 2. for ongoing operations Option 3. was the one I was referring to as "backup" and more complicated solution. We'll go with option 1 as our starting point. </blockquote><div> </div><div>Understood, this is basically like a server token available from the OAuth provider's dashboard and can be used directly for making API calls from ALM. </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class="HOEnZb"><div class="h5"> > > > > And there will be a default build environment, to the same > OpenShift > > instance as Almighty is running on, for which the above mentioned > > authentication steps will result in "noop". (When user is already > > logged in Almighty.) > > > Sounds good. > > Yes, environment variables and secrets will be pasted to SPI > provider > > using CRUD operations. In case of OSPB this will be mapped to > OpenShift > > secrets and env variables for pipelines. > > > > > 2. With the help of of ALM UI , the user gives ALM authorization > to manage builds on the external system. This step gives ALM an OAuth > access token with which it is able to manage builds on the external > system on the user's behalf. > > > This access token allows ALM to do privileged stuff with the > external system. > > True. > > > > > Expiry : > > > The expiry of access tokens should be set to fairly large as > possible because the alm gets to specify the expiry during OAuth > token generation procedure. > > I don't think client is the one to choose and we can't influence > > default values in target platform (OAuth server) like OpenShift > Online. > > We can do it for the default build environment which will be pre- > > configured, but that's about it. > > > In some cases the client can ask for expiry times but in any case , > as you said there are always default values which could be banked > upon. > > > > > We would need to have ways to notify the user in the UI when the > token expires, gracefully. > > Correct. > > > > > > Regards, > > Tomas > > > > > > > > Let me know if this is what you intended to explain. > > > Thanks > > > Shoubhik > > > > > > On Oct 13, 2016 8:54 PM, "Tomas Nozicka" <<a href="mailto:tnozicka@redhat.com">tnozicka@redhat.com</a>> > wrote: > > > > Hi all, > > > > > > > > this is a followup on Andy's initial mail[1] about this topic > and > > > > presents my view on what the Build Service (and providers) > should look > > > > like. > > > > > > > > Feel free to challenge any design here as you see fit; we are > still in > > > > early stages and figuring it out ourself. > > > > > > > > Andy published the generic design in his last email[1] at the > highest > > > > abstraction level and that stays the same. [2] > > > > > > > > Let's start by defining what Build Service is, because the name > might > > > > seem a bit confusing; at least it was for me when we joined > this > > > > project. Build Service should provide user with the ability to > > > > transform repository code (e.g. git branch) into artifacts > (like docker > > > > images) and the ability to *deploy* them to target environment > (e.g. > > > > OpenShift). Build Service should be able to cover the whole > CI/*CD* > > > > story, not just builds. > > > > > > > > There is already a similar concept implemented in OpenShift > called > > > > *Pipelines* [3]. But, in ALM, we want to be more generic and > provide > > > > users with Build Service with plugable providers. That's why we > are > > > > creating Build SPI; to abstract the provider specifics away. > Generally > > > > speaking regular build (non-pipeline) job looks like a pipeline > of 0 > > > > stages, which is a valid pipeline. > > > > > > > > Although there will be most probably different providers in the > future, > > > > we will start by implementing "OpenShift Build Provider" (OSBP) > that > > > > will use OpenShift's pipelines. > > > > > > > > There are 3 parts connected by Build SPI and Almighty public > API: > > > > - ALM Core - Build Service > > > > - OSBP > > > > - ALM UI > > > > > > > > Good thing is that after we define the interfaces (mainly Build > SPI) > > > > those can be worked on in parallel. > > > > > > > > = ALM UI = > > > > There is a lot of work done for visualizing pipelines in > OpenShift's > > > > Console. This is OSS and we could try to reuse those blocks. > They will > > > > most definitely need modifications because e.g. they read > status > > > > directly from OpenShift's API and that will be abstracted away > by Build > > > > SPI and ALM API in our case. Also I am not sure how well they > visualize > > > > pipelines without any stages. Also I remember Michael > mentioning > > > > Console uses Angular v1 and ALM UI v2. > > > > > > > > UI will also need to be able to indirectly ask user for > credentials to > > > > build and target/prod environment's (more generally to 1-N > > > > environments) using OAuth2. ALM's core and build providers will > > > > authenticate only by token produced by this (OAuth) step. And > the rest > > > > of those tokens will be given to build provider as secrets for > > > > authentication to other environments (e.g. in case of cross- > cluster > > > > deployments). > > > > > > > > > > > > = ALM Build Service = > > > > This will be the service in ALM core that will issue calls > against > > > > Build SPI provider. How it will be represented, mainly what > information > > > > it needs to hold, will be strict superset of provider > configuration. > > > > > > > > > > > > = Build Providers = > > > > Build provider will comunicate with Build Service through Build > SPI > > > > which abstracts provider specifics away. It will have several > functions > > > > like: > > > > - CRUD operations > > > > - StartBuild > > > > - CancelBuild > > > > - GetStages > > > > - GetLogs > > > > - and many more (this will be in Build SPI spec) > > > > which can all be fit to work with both pipelines and regular > build jobs > > > > with the assumption that regular build job is pipeline of 0 > stages. > > > > > > > > > > > > = Full picture = > > > > - ALM admin will add providers (register them) to ALM; or it > will be > > > > registered the way that ALM SPIs do... I haven looked into it > yet:( > > > > - User will choose to add a Build Service to his project > choosing > > > > provider in UI > > > > - User will configure OAuth to his build environment [(URL)- > >(token)] > > > > and optionally other OAuth resources [(URL, name)->(name, > token)] which > > > > will be passed to build provider as secrets that can be used > e.g. for > > > > cross-cluster deployments in that build > > > > Since this will be done through OAuth, ALM shall never see > user's > > > > credentials. > > > > - Build Service instance is created in ALM core > > > > - Build Service uses provider to create BuildConfig instance > passing > > > > it necessary configuration like: token, secrets[(name, token)], > > > > repository reference, ... > > > > - followed by many Build SPI calls... > > > > > > > > ALM core shall verify that all OAuth tokens are valid (not > expired) > > > > before calling Build SPI and refresh them with cooperation with > ALM UI > > > > if needed. (For the main token Build SPI will return > unathorized, but > > > > for the OAuth tokens in secrets the underlying technology does > not > > > > support that and build will be marked as failed otherwise. This > might > > > > be a common scenario since some tokens may last for about a day > or so.) > > > > > > > > Also in case ALM will run on the same OpenShift instance as you > setup > > > > for your builds, or there will be a dedicated OpenShift > instance for > > > > ALM build set up with ALM as OpenShift's identity provider, you > can be > > > > authenticated to your build environment using your ALM token > without > > > > the need to be asked for credentials. If allowed, this should > be the > > > > default. > > > > > > > > = Next steps = > > > > I will work on putting Build SPI proposal on paper and also on > how we > > > > should represent Build Service in core which is connected with > it. > > > > > > > > There is also a userstory for next sprint on GitHub[3]. > > > > > > > > I will appreciate your feedback! > > > > > > > > Regards, > > > > Tomas > > > > > > > > [1] - <a href="https://www.redhat.com/archives/almighty-public/2016-" rel="noreferrer" target="_blank">https://www.redhat.com/archives/almighty-public/2016-</a> > September/ms > > > > g00150.html > > > > [2] > - <a href="https://drive.google.com/file/d/0B10zSvDl_cuwMHZtTmR1RVIteGM/view" rel="noreferrer" target="_blank">https://drive.google.com/file/d/0B10zSvDl_cuwMHZtTmR1RVIteGM/view</a> > > > > (RH Only, but there is a screenshot in attached in [1]) > > > > [3] - <a href="https://github.com/almighty/almighty-core/issues/352" rel="noreferrer" target="_blank">https://github.com/almighty/almighty-core/issues/352</a> > > > > > > > > _______________________________________________ > > > > almighty-public mailing list > > > > <a href="mailto:almighty-public@redhat.com">almighty-public@redhat.com</a> > > > > <a href="https://www.redhat.com/mailman/listinfo/almighty-public" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/almighty-public</a> > > > > </div></div></blockquote></div> </div></div></div>