[Pulp-dev] Pulp smash test docs and issues

Mon Aug 27 15:33:40 UTC 2018

.To kind of sum things up: issues with pulp-smash itself will still be
filled in the pulp-smash Github repo. Also, the documentation for
pulp-smash will continue to reside on readthedocs (RTD).

For Pulp 2, issues with the tests will be filed at the Pulp-2-tests repo
and the docs for tests will reside on RTD.

For Pulp 3, the actual pulp-smash tests for both core and plugins, the
issues will be filed in redmine with the new “Functional Test” tag. Also,
documentation for these tests will live alongside the core/plugin docs.

It seems like there’s consensus around this plan. With that, I’d like to
give it until Wednesday (August 29, 2018) and then we can proceed.

Thanks.

David

On Wed, Aug 22, 2018 at 9:56 AM Brian Bouterse <bbouters at redhat.com> wrote:

> Sounds good Kersom. I think we're on the same page, but want to write a
> description to help be sure. Here is what we can offer in terms of the docs:
>
> Any docstring that is within the same repo as the sphinx project can be
> included. This is true for a plugin repo and core. I'll give an example
> with core because that is the only one with all the pieces right now that I
> need to show. For example with pulp/pulp [0] the sphinx project is here [1]
> and any docstring like this one [2] can be easily included in the docs with
> syntax like this [3] which product output like that [4].
>
> What we aren't able to do is include docstrings from other repos. For
> instance we can't include docstrings from
> https://github.com/PulpQE/pulp-smash/ on a page in the core docs or any
> plugin docs because the docstrings aren't in the same repo as the sphinx
> project they would be included in.
>
> Hopefully this is how everyone understands it.
>
> [0]: https://github.com/pulp/pulp/
> [1]: https://github.com/pulp/pulp/tree/master/docs
> [2]:
> https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L13-L33
> [3]:
> https://github.com/pulp/pulp/blob/master/docs/plugins/plugin-api/stages.rst#artifact-related-stages
> [4]:
> https://docs.pulpproject.org/en/3.0/nightly/plugins/plugin-api/stages.html#pulpcore.plugin.stages.QueryExistingArtifacts
>
> All the best,
> Brian
>
>
>
> On Mon, Aug 20, 2018 at 5:57 PM, Kersom <kersom at redhat.com> wrote:
>
>> I am ok with your suggestions Brian.
>>
>> I like the idea to the documents for the test related to a certain plugin
>> live at the same website as the plugin itself. I am not sure how much
>> adjustments will be necessary to the doctrings for the tests to be added to
>> the current plugin docs. Maybe just add another section related to the
>> tests.
>>
>> > To demo an idea, I just added a 'Functional Test' tag which should be
>> available on all Redmine projects. If we want to rename it, or delete it,
>> we can. How would that work?
>>
>> I thins this a good start. We can try to adjust if necessary.
>>
>>
>>
>>
>> On Mon, Aug 13, 2018 at 4:24 PM, Brian Bouterse <bbouters at redhat.com>
>> wrote:
>>
>>> Thanks for the reply Kersom. I responded inline.
>>>
>>> On Mon, Aug 13, 2018 at 3:49 PM, Kersom <kersom at redhat.com> wrote:
>>>
>>>> Thank you for this thread.
>>>>
>>>> Currently there are dedicated documents for Pulp-Smash on readthedocs.
>>>> [0]
>>>> Issues related to Pulp-Smash itself should be filed here [1]
>>>>
>>> +1
>>>
>>>>
>>>> Documentation for Pulp 2 tests were created on readthedocs [2] after
>>>> these tests were moved from Pulp-Smash.
>>>> Issues related to Pulp 2 should be filed here [3]
>>>>
>>> +1
>>>
>>>>
>>>> Pulp 3 tests were migrated to Pulp repositories, but there are no
>>>> documentation for them right now. In my opinion, documentation for tests,
>>>> code standards for tests, and examples will drive more contributions. I am
>>>> not sure what the best option is to generate and host these docs.
>>>>
>>>
>>> For topics specific to a plugin, I agree it should go in the repo w/ the
>>> functional tests themselves. Some plugins host those docs via github.com
>>> browseable readme's, (e.g.
>>> https://github.com/pulp/pulp_ansible/#pulp-ansible) or via a read the
>>> docs website (e.g. http://pulp-python.readthedocs.io/en/3.0-dev/). The
>>> docs.pulpproject.org site can't host plugin docs for Pulp3, only core
>>> code.
>>>
>>>>
>>>> I think we can use the plugin issue tracker to track tests related to a
>>>> certain plugin. Perhaps a certain field, or anchor can be used to allow
>>>> filters for issues that require tests. Maybe the same approach can be used
>>>> for the pulp core as well.
>>>>
>>>
>>> To demo an idea, I just added a 'Functional Test' tag which should be
>>> available on all Redmine projects. If we want to rename it, or delete it,
>>> we can. How would that work?
>>>
>>>
>>>> [0] https://pulp-smash.readthedocs.io/en/latest/
>>>> [1] https://github.com/PulpQE/pulp-smash
>>>> [2] https://pulp-2-tests.readthedocs.io/en/latest/
>>>> [3] https://github.com/PulpQE/Pulp-2-Tests
>>>>
>>>>
>>>>
>>>> On Fri, Aug 10, 2018 at 10:16 AM, Brian Bouterse <bbouters at redhat.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 9, 2018 at 5:42 PM, Daniel Alley <dalley at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> I think this is fine where it is. pulp2 is going into maintenance
>>>>>>> mode at some point here soon.
>>>>>>
>>>>>>
>>>>>> That makes sense for the Pulp 2 smash test docs, but it's still a
>>>>>> problem if we want to have smash test docs for Pulp 3 (which, we do)
>>>>>>
>>>>> That's true, we do need it to live somewhere.
>>>>>
>>>>> The main thing I want to avoid is trying to include content from one
>>>>> repo, i.e. PulpQE/pulp-smash to be published through the docs of the sphinx
>>>>> project in pulp/pulp. In terms of what pulp-smash offers and how to use it,
>>>>> I think that should be its own site, separate from Pulp's docs.
>>>>> Additionally, I could imagine a section in our docs either recommending
>>>>> pulp-smash and linking to the pulp-smash docs, and maybe expanding on its
>>>>> examples some. Is this kind of what you imagined? How is it
>>>>> similar/different?
>>>>>
>>>>>
>>>>>> On Thu, Aug 9, 2018 at 5:12 PM, Brian Bouterse <bbouters at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for bringing this up. I put some responses inline.
>>>>>>>
>>>>>>> On Thu, Aug 9, 2018 at 4:22 PM, David Davis <daviddavis at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> One of the things that may not make sense anymore is how we
>>>>>>>> document and track issues for pulp-smash tests given that these tests no
>>>>>>>> longer reside in the pulp-smash repo. Currently, all the test-related
>>>>>>>> issues are tracked here[0].
>>>>>>>>
>>>>>>>> With the tests no longer in the pulp-smash repo, I wonder if it
>>>>>>>> makes sense to maybe move them somewhere else like into redmine for Pulp 3
>>>>>>>> (or the pulp-2-tests[1] repo for Pulp 2).
>>>>>>>>
>>>>>>> +1 to moving issues about the testing of a plugin to that plugin's
>>>>>>> tracker
>>>>>>>
>>>>>>>
>>>>>>>> The other question is about documentation. Currently the pulp-smash
>>>>>>>> test documentation is hosted on RTD (e.g.
>>>>>>>> https://pulp-2-tests.readthedocs.io/en/latest/). Should this
>>>>>>>> documentation live alongside the core/plugin docs?
>>>>>>>>
>>>>>>> I think this is fine where it is. pulp2 is going into maintenance
>>>>>>> mode at some point here soon.
>>>>>>>
>>>>>>>
>>>>>>>> QE would like to hear feedback as to how to proceed by August 20,
>>>>>>>> 2018 so please respond by then.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> [0] https://github.com/pulpqe/pulp-smash/issues
>>>>>>>> [1] https://github.com/PulpQE/Pulp-2-Tests
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pulp-dev mailing list
>>>>>>>> Pulp-dev at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pulp-dev mailing list
>>>>>>> Pulp-dev at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>>
>>>>
>>>
>>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180827/3ddc4d35/attachment.htm>