[Pulp-list] What do you want from an API documentation solution?

Chris St. Pierre chris.a.st.pierre at gmail.com
Mon Nov 14 19:55:28 UTC 2011


Community member weighing in here.

AFAICT, outside of you guys I've done as much coding with the Pulp API
as any other human on the planet.  (Some of that is already on github;
hopefully the rest will be coming soon.)  It's frustrating.  There's a
lot of time spent digging through the Sponge code, playing around your
python interpreter of choice divining return values, praying, breaking
stress balls, and so on.  So:

1.  I'll be very interested in helping to proofread and test API docs
that you produce.  I've got a few thousand lines of Django code that
leverages the Pulp API that I can use to test it, and it'll be swell
to see all the things I did wrong. :)

2.  IME, writing docs is a magnificent way to find inconsistencies.
Hopefully little things like return values, keyword argument naming,
and so on will get sorted out through the process.  This has the
potential to improve Pulp in a lot of tangential ways.

I know this isn't much more than a "me too!", but it's an emphatic "me too!" :)

On Mon, Nov 14, 2011 at 2:42 PM, Jay Dobies <jason.dobies at redhat.com> wrote:
> I'd warn you guys this is a long read, but if you haven't realized by now
> that I'm annoyingly long-winded, you haven't been paying attention.
>
> I have a sprint story related to an API docs solution and have been trying
> to find something to get in place before everyone runs around doing their
> tasks to document the currently undocumented APIs. I'm running into problems
> defining what it is we're trying to solve and wanted to think/talk it out
> before throwing in another attempt at a solution. I also hope you'll suffer
> through reading it all and we can discuss at a deep dive this week; bring
> your ideas.
>
> = Problem =
> The way I see it, there are three problems with our API docs:
> * Missing
> * Incomplete
> * Incorrect
>
> == Missing ==
> Not much to say to describe this. This also isn't a blame thing; we knew
> they'd be a priority one day and did what we could. Now, they are a much
> bigger priority and we need to get them done and keep them maintained.
>
> == Incomplete ==
> By this I mean inconsistencies between API docs. Some have examples, some
> don't. Very few appropriately describe error codes. Things like that.
>
> == Incorrect ==
> This isn't so much meant to refer to those APIs that are done wrong
> themselves, more it's meant to address the idea that things change and the
> docs need to be updated to reflect those changes.
>
> = Solutions =
>
> == Automated ==
> We attempted to do some sort of generation of API docs from docstrings and
> there's been another increased interest in this area with the proposal of
> sphinx. I'm not sure either are the answer.
>
> The perfect solution would be to write code and have the API docs generated
> automagically from the code itself. That's never going to happen since we're
> talking about REST APIs, not simply python APIs. The REST APIs need to cover
> things like HTTP status codes and example URLs and response bodies. Those
> things just aren't possible by code introspection.
>
> The last attempt was to use a specific format in the docstrings of the
> methods and have the docs generated from there. I'd argue it failed for a
> few reasons:
>
> 1. No clearly defined template (could be that I just never knew where to
> look, but there is clearly a format I had to follow and I never knew how to
> get started).
>
> 2. Manual process of generate .wiki file and then manually copy/paste into
> the wiki. It's actually _more_ work than editing the wiki directly.
>
> Assuming we fix it so that there's no intermediate manual copy/paste step
> into the wiki, I'm still not convinced it's going to work for a few reasons:
>
> 1. Most developers don't bother reviewing docstrings when they change code.
> It's very unfortunate, but also very true. In Pulp alone there are a number
> of places where the docstrings reflect an older version of the method
> (PyCharm can highlight this for you, but I digress).
>
> 2. It's a new syntax. That means you still have to dig up the template to
> drop in the docstring and figure out how to use it.
>
> So given those two issues, I don't think the question is simply if we edit
> them in the wiki or in code. Either way you still have to hunt around for
> the template. You still have to remember to actually do it. You still have
> to remember to review it after you've changed something.
>
> I'll talk more about technical solutions later. I'm curious what people
> think about this point though. I haven't dug too much into sphinx yet, but I
> have no reason to think it won't be subject to all of the above issues that
> wouldn't render the wiki (almost*) as easy.
>
> * The biggest benefit to sphinx is co-locating the docs in the codebase
> which arguably isn't necessarily a benefit; you can put a browser editing a
> wiki side-by-side with code and not have to scroll up and down to hop
> between impl and docstring. Plus you have more flexibility on a wiki; sphinx
> calls itself WYSIWYG but it's exactly the opposite of that, whereas with a
> wiki it's much simpler to hit "preview".
>
> == Discipline ==
> Since we won't be able to derive the API docs purely from code, at the end
> of the day we all just need to be better about stopping for a second to
> think about if we need to edit them (I'd like to see us use a boyscout rule*
> mentality). In the past, it was easier to blow these off since people
> weren't using them. Now, we have both a community and Katello, so we need to
> be aware of the increased priority (and if you still don't buy in, hop to
> the Katello team for a sprint and use an area of Pulp you didn't write;
> you'll hate us).
>
> * http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule
>
> == Peer Review ==
> Let me get this out of the way now: I'm not suggesting everyone have to take
> part in API documentation review.
>
> John has the skill and patience for debugging really obtuse issues. I hate
> that shit, so I never volunteer for it. I leave it for John (who I _think_
> enjoys those sorts of things).
>
> Same goes for reviewing docs, both user guide and API docs. Some people have
> no interest in it, so there's no reason to force them to have to review
> other people's docs. Personally, I get an odd OCD satisfaction knowing we
> have really clean, readable API docs. So I'm happy to volunteer to do a
> quick proof read as a fresh pair of eyes who doesn't know the code itself
> and who is compulsive enough to make sure the whole docs package fits
> together.
>
> Different skills from different people. Let's take advantage of that and
> request people who are willing to to take a quick once over when you finish
> docs.
>
> == Template ==
> Sayli sent out a proposed template a few weeks ago. I don't know where it is
> on the wiki, but I should. Bookmark the template so it feels like less of a
> chore to dig it up when you have to add new APIs.
>
> I can also do another quick review on what pages on the wiki are slurped
> into the look & feel process when the userguide is generated.
>
> We should also add in a dirt simple way of getting the CLI to dump URLs,
> requests, and responses. I already know where it'd go, I just haven't done
> it yet. Then we all have a consistent way of getting that example data for
> the docs.
>
> == Technical Approach ==
> It's probably clear by now that I'm a fan of just using the wiki like we did
> in the beginning. I'm not convinced an in-code solution gets us anything
> that simple discipline wouldn't; arguably, it's yet another syntax we have
> to learn and frankly, I have a hard enough time just remembering one wiki
> format from another.
>
> I'm still willing to look into sphinx, but my early impressions are that
> we're going to have to invest a reasonably large amount of work getting it
> to look at the places in code we want it to look and format it for the web
> site. Ultimately, even if we do, you still have to find the template for
> what our API docs should look like and remember to write/update them, so I'm
> not convinced it's any better than the wiki -> user guide process we already
> have in place.
>
> = Conclusion =
> Actually, there is no conclusion, I just needed an ending. Please give it
> some thought though. Why are we in this position in the first place? Is an
> in-code solution really going to make that much of a difference to you when
> you edit something or write a new API?
>
> Let me know in some way how you want to proceed. If you don't want to get
> into a giant reply-to-list war on this thread and think it'd be worth a deep
> dive, ping me separately and I'll set one up. If you don't really care and
> just want to be told "here's the new process", that's fine too, just let me
> know. Otherwise, I'm at a bit of a pause on how to proceed with this story,
> so I'm waiting to see what others think.
>
> --
> Jay Dobies
> Freenode: jdob @ #pulp
> http://pulpproject.org | http://blog.pulpproject.org
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>



-- 
Chris St. Pierre




More information about the Pulp-list mailing list