[almighty] Approach for writing unit tests for Github OAuth2.0 auth

[Shoubhik Bose]

Thank you Thomas , Todd and Pavol for for your suggestions -

The takeaways I have from the above discussion is that both are equally
important !

In this sprint we have contributed tests which interact with Github. I
think that's a good start given the fact that our first objective is to get
things working, and after that we should optimize our test case design to
have it run in 'firewalled' environments as well ( at least have a fallback
).

We have already done this for most tests in other areas where external
resources are involved. What makes it difficult to do in case of OAuth2.0
is that we also need to mock a workflow where a user accepts a set of
permissions. There is a write operation involved unlike the other cases (
like remote work item integration ) which are mostly about reading.

I've opened an issue here [1] to track design discussions for this.


[1] https://github.com/almighty/almighty-core/issues/368

Thanks,
Shoubhik

On Thu, Oct 13, 2016 at 8:06 PM, Pavol Pitonak <ppitonak at redhat.com> wrote:

> I agree that both types of tests make sense.
>
> For unit tests, if I understand it correctly, you are using library
> go-github. In this case, we can expect that the library is working
> correctly and therefore I wouldn't test/mock Github API directly (but I
> advise to do audit of go-github test coverage). In Almighty core unit
> tests, you only need to make sure that correct function of go-github
> library is called with correct parameters and you fake the response if
> necessary (e.g. if you want to simulate errors).
>
> Regards,
> Pavol
>
> On Thu, Oct 13, 2016 at 1:54 PM, Todd Mancini <tmancini at redhat.com> wrote:
>
>> I agree. I'd go one step further -- a mock should be able to produce the
>> same kinds of errors as the real object, and there should be unit tests for
>> those conditions so we know we handle failures gracefully.
>>
>> Sent from my phone, so anticipate hilarious autocorrects
>> ------------------------------
>> From: Thomas Mäder <tmader at redhat.com>
>> Sent: ‎10/‎13/‎2016 4:00 AM
>> To: Shoubhik Bose <shbose at redhat.com>; ALMighty-public
>> <almighty-public at redhat.com>
>> Subject: Re: [almighty] Approach for writing unit tests for Github
>> OAuth2.0 auth
>>
>> Hi Shoubhik,
>>
>> I think both kinds of test have their place. If we want to verify that
>> our code does what we want it to do (tests as specification), it makes a
>> lot of sense to test very specific interactions. Since OAuth is basically a
>> sequence of requests to our server, there really isn't a need to "mock"
>> anything: it's just a sequence of http requests where we need to check the
>> proper responses. I would want this test to work without needing anything
>> outside my local machine, so that I can work when I'm offline.
>>
>> However, in the wild, many things may go wrong: it might be as stupid as
>> a corporate firewall filtering out some headers from requests. Also, our
>> test might not accurately represent the behaviour of Github OAuth in every
>> detail. Having that safety net seems valuable to me. It may be hard to
>> cover every use case in these tests. We may want to start with covering 1-2
>> basic flows and add more as regression tests as we go along.
>>
>> What I think is important for productivity is that I can run all the
>> local tests while I am developing in a simple way.
>>
>> just my 0.02€
>>
>> /Thomas
>>
>>
>> On 10/12/2016 05:55 PM, Shoubhik Bose wrote:
>>
>> Hi folks,
>>
>> This is an extension to a discussion we were having about what would be
>> the apt approach to write tests for Github authentication on the alm core (
>> backend ).
>>
>> We've written a bunch of tests which checks if our code does the Github
>> OAuth flow correctly.
>>
>> However the alternate school of thought is:
>>
>> Should we have a 'mock' Github OAuth provider to also test our code,
>> since conventionally tests should not be depending on external systems.
>>
>> On the flip-side, if test failures are not caught with a real Github
>> OAuth provider we might ending up chasing bugs in production.
>>
>> Let me know your thoughts.
>>
>>
>> -
>> Shoubhik
>>
>>
>> _______________________________________________
>> almighty-public mailing listalmighty-public at redhat.comhttps://www.redhat.com/mailman/listinfo/almighty-public
>>
>>
>>
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/almighty-public/attachments/20161017/32ac8924/attachment.htm>