[Container-tools] Fwd: Containers Testing Framework - Proof of Concept

[Václav Pavlín]

Hey guys,

I'd like to ask all of you interested in QA of what we are doing, to 
take a look at Container Testing Framework. There were some changes 
recently, but the major idea is still the same - to be able to simply 
combine multiple tests - features, steps.. living at multiple locations 
and run them against your project locally or remotely.

There is also an ongoing effort to merge this (CTF) project and 
UATFramework [2] and then potentially rename the result to something 
like Common/Composite Testing Framework, add it to Container Best 
Practices [6] and move it under Project Atomic (if possible).

Cheers,
Vašek


-------- Forwarded Message --------
Subject: 	Containers Testing Framework - Proof of Concept
Date: 	Mon, 27 Apr 2015 10:06:21 +0200
From: 	Tomas Hozza <thozza at redhat.com>
To: 	appinfra-list at redhat.com
CC: 	Václav Pavlín <vpavlin at redhat.com>, Vadim Rutkovsky 
<vrutkovs at redhat.com>, lzachar at redhat.com, langdon at redhat.com



Hi all.

As part of "Red Hack week" we had in Platform BU in Brno, we created
a proof of concept of Containers Testing Framework.

The main motivation was:
- We will be building a lot of images, some of them are similar and
thus the testing will be partially similar.
- It would be handy to have a simple framework (and workflow) for
writing and running tests for containers.
- It would be good to reuse the code as much as possible and share
most of the test steps among the tests

During the last week we designed a simple workflow and PoC tool
for using the framework.

The framework is based on Behave [1] and inspired by UATFramework [2].
The main idea is to be able to combine already implemented Behave
Features and Steps into one working directory, and then execute
Behave in it. Behave then performs the tests on Dockerfile and image.
We want it to be able to perform the tests locally or in a remote
machine. Currently we use ansible for that, but it is not yet working
well. As I already mentioned, the whole framework and tool is PoC,
so this is expected. However we think the design is quite useful
and could be really helpful for testing containers.

Everything is available on GitHub [3] and you can find project
documentation and presentation with some nice pictures in the
internal google drive [4]

The current CLI tool [5] enables you to:
- run local project-specific Behave Features and Steps
- run just remote Behave Features and Steps on the local Dockerfile
- run Features and Steps as combination of the above two
- it enables one to use Steps implemented in remote repo in locally
defined Features

As an example we rewritten couple of existing tests for OpenShift
images into the Behave style for our framework.

Our next plan is to discuss the future of the containers testing
with AppInfra teams OpenShift and any other interested parties
to somehow standardize the containers testing in the community
and within Red Hat.

[1] http://pythonhosted.org/behave/
[2] https://github.com/aweiteka/UATFramework
[3] https://github.com/Containers-Testing-Framework
[4]
https://drive.google.com/a/redhat.com/folderview?id=0B0CTJuCyJVsSfkJuZWZMZG03QmhaeHFJaklhSFUzaWFFQV84ZHFOUWJWMkg4bWZOaXc5MDg&usp=sharing&usp=sharing#
[5] https://github.com/Containers-Testing-Framework/ctf-cli

Regards,
-- 
Tomas Hozza
Software Engineer - EMEA ENG Developer Experience

PGP: 1D9F3C2D
Red Hat Inc.                               http://cz.redhat.com


[6] https://github.com/projectatomic/container-best-practices


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/container-tools/attachments/20150610/83527b6d/attachment.htm>