[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Pulp-list] What do you want from an API documentation solution?

I'd warn you guys this is a long read, but if you haven't realized by now that I'm annoyingly long-winded, you haven't been paying attention.

I have a sprint story related to an API docs solution and have been trying to find something to get in place before everyone runs around doing their tasks to document the currently undocumented APIs. I'm running into problems defining what it is we're trying to solve and wanted to think/talk it out before throwing in another attempt at a solution. I also hope you'll suffer through reading it all and we can discuss at a deep dive this week; bring your ideas.

= Problem =
The way I see it, there are three problems with our API docs:
* Missing
* Incomplete
* Incorrect

== Missing ==
Not much to say to describe this. This also isn't a blame thing; we knew they'd be a priority one day and did what we could. Now, they are a much bigger priority and we need to get them done and keep them maintained.

== Incomplete ==
By this I mean inconsistencies between API docs. Some have examples, some don't. Very few appropriately describe error codes. Things like that.

== Incorrect ==
This isn't so much meant to refer to those APIs that are done wrong themselves, more it's meant to address the idea that things change and the docs need to be updated to reflect those changes.

= Solutions =

== Automated ==
We attempted to do some sort of generation of API docs from docstrings and there's been another increased interest in this area with the proposal of sphinx. I'm not sure either are the answer.

The perfect solution would be to write code and have the API docs generated automagically from the code itself. That's never going to happen since we're talking about REST APIs, not simply python APIs. The REST APIs need to cover things like HTTP status codes and example URLs and response bodies. Those things just aren't possible by code introspection.

The last attempt was to use a specific format in the docstrings of the methods and have the docs generated from there. I'd argue it failed for a few reasons:

1. No clearly defined template (could be that I just never knew where to look, but there is clearly a format I had to follow and I never knew how to get started).

2. Manual process of generate .wiki file and then manually copy/paste into the wiki. It's actually _more_ work than editing the wiki directly.

Assuming we fix it so that there's no intermediate manual copy/paste step into the wiki, I'm still not convinced it's going to work for a few reasons:

1. Most developers don't bother reviewing docstrings when they change code. It's very unfortunate, but also very true. In Pulp alone there are a number of places where the docstrings reflect an older version of the method (PyCharm can highlight this for you, but I digress).

2. It's a new syntax. That means you still have to dig up the template to drop in the docstring and figure out how to use it.

So given those two issues, I don't think the question is simply if we edit them in the wiki or in code. Either way you still have to hunt around for the template. You still have to remember to actually do it. You still have to remember to review it after you've changed something.

I'll talk more about technical solutions later. I'm curious what people think about this point though. I haven't dug too much into sphinx yet, but I have no reason to think it won't be subject to all of the above issues that wouldn't render the wiki (almost*) as easy.

* The biggest benefit to sphinx is co-locating the docs in the codebase which arguably isn't necessarily a benefit; you can put a browser editing a wiki side-by-side with code and not have to scroll up and down to hop between impl and docstring. Plus you have more flexibility on a wiki; sphinx calls itself WYSIWYG but it's exactly the opposite of that, whereas with a wiki it's much simpler to hit "preview".

== Discipline ==
Since we won't be able to derive the API docs purely from code, at the end of the day we all just need to be better about stopping for a second to think about if we need to edit them (I'd like to see us use a boyscout rule* mentality). In the past, it was easier to blow these off since people weren't using them. Now, we have both a community and Katello, so we need to be aware of the increased priority (and if you still don't buy in, hop to the Katello team for a sprint and use an area of Pulp you didn't write; you'll hate us).

* http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule

== Peer Review ==
Let me get this out of the way now: I'm not suggesting everyone have to take part in API documentation review.

John has the skill and patience for debugging really obtuse issues. I hate that shit, so I never volunteer for it. I leave it for John (who I _think_ enjoys those sorts of things).

Same goes for reviewing docs, both user guide and API docs. Some people have no interest in it, so there's no reason to force them to have to review other people's docs. Personally, I get an odd OCD satisfaction knowing we have really clean, readable API docs. So I'm happy to volunteer to do a quick proof read as a fresh pair of eyes who doesn't know the code itself and who is compulsive enough to make sure the whole docs package fits together.

Different skills from different people. Let's take advantage of that and request people who are willing to to take a quick once over when you finish docs.

== Template ==
Sayli sent out a proposed template a few weeks ago. I don't know where it is on the wiki, but I should. Bookmark the template so it feels like less of a chore to dig it up when you have to add new APIs.

I can also do another quick review on what pages on the wiki are slurped into the look & feel process when the userguide is generated.

We should also add in a dirt simple way of getting the CLI to dump URLs, requests, and responses. I already know where it'd go, I just haven't done it yet. Then we all have a consistent way of getting that example data for the docs.

== Technical Approach ==
It's probably clear by now that I'm a fan of just using the wiki like we did in the beginning. I'm not convinced an in-code solution gets us anything that simple discipline wouldn't; arguably, it's yet another syntax we have to learn and frankly, I have a hard enough time just remembering one wiki format from another.

I'm still willing to look into sphinx, but my early impressions are that we're going to have to invest a reasonably large amount of work getting it to look at the places in code we want it to look and format it for the web site. Ultimately, even if we do, you still have to find the template for what our API docs should look like and remember to write/update them, so I'm not convinced it's any better than the wiki -> user guide process we already have in place.

= Conclusion =
Actually, there is no conclusion, I just needed an ending. Please give it some thought though. Why are we in this position in the first place? Is an in-code solution really going to make that much of a difference to you when you edit something or write a new API?

Let me know in some way how you want to proceed. If you don't want to get into a giant reply-to-list war on this thread and think it'd be worth a deep dive, ping me separately and I'll set one up. If you don't really care and just want to be told "here's the new process", that's fine too, just let me know. Otherwise, I'm at a bit of a pause on how to proceed with this story, so I'm waiting to see what others think.

Jay Dobies
Freenode: jdob @ #pulp
http://pulpproject.org | http://blog.pulpproject.org

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]