Addressing Yelp Concerns

[John J. McDonough]

Shaun,

Thanks for the excellent summary ...

(lotsa clips below)

On Thu, 2009-12-10 at 22:01 -0600, Shaun McCance wrote:

> I do want to make sure that multiple-page HTML documents are
> handled well in Yelp.  So even if you find that letting Yelp
> render DocBook on the fly doesn't work for you, you may still
> be able to use Yelp with pre-built HTML.

This might help us sort out the right blend of features

> The front page of Yelp will be the Desktop Help ("User Guide"),
> which will be a Mallard document.  Mallard is a new format for

Oh goodie, yet another format

> the primary selling point is that
> it's designed around the idea that downstream distributors will
> extend and modify upstream documents.

Now this sounds like a good idea

> * Fedora documents are found in Help, alongside Gnome documents,
> even though these are stand-alone documents generally not coupled
> to any particular application and are therefore a different kind
> of documentation from what's generally provided in help and what
> users therefore expect to find there.

There's a lot of "that's how we've always done it" here, that perhaps
should be questioned.  But for a dozen releases now, users find the
release notes under help.  Prior to F11 we also installed a half dozen
other documents that I bet 90% of users never knew were there.

> Yelp is moving away from the focus on all installed documents,

I guess I don't totally understand what this means

> There are no real size limitations, except possibly the recursion
> limits in libxml2, although I doubt you're hitting those.  This
> is probably just a symptom of the speed issue below.

Yelp has some very strange behaviors.  It takes a while for the window
to appear, and another while to open a document.  However, after a
document appears to open, the CPU stays pinned for quite some time.  In
the case of the Release Notes, Yelp would originally crash during this
time.  "Quite some time" is several MINUTES, so it took me a while
before I realized it wasn't what I was doing that was causing the crash,
but whatever was going on behind the scenes.

In the case of Release Notes, the only obvious workaround was to drop
the "All Changes" chapter and put a link in to that chapter on the web.
That chapter happens to have several thousand index entries.  It did
appear that making the chapter smaller helped, but a partial "All
Changes" chapter didn't make much sense, so for release, we eliminated
the entire chapter.

It turns out that the index is pretty critical for that chapter, and
since I saw no workaround for the lack of an index, that also indicated
that the solution was to move it to the web.

> * Since yelp handles rendering, there is no control over document
> style.
> 
> I generally view this as an advantage.  Fiddling with design is
> not something authors should spend their time on.  When they do,
> they often create something blingier, but decidedly less readable.

Yes, I see both sides of this, too.  One of the nice things about
Publican, as opposed to naked DocBook, is that somebody else worried
about what it looks like.

> True, although that doesn't mean it couldn't.  That's clearly an
> important feature for Fedora, so I can bump it up on the priority
> list.

Although we didn't use this in the past, as I mentioned earlier, the All
Changes chapter makes this a necessity for now.  It wouldn't be so
critical if the yum groups made some sense, but they don't, and the Docs
Project has no control over this, so the only way a person might find
their favorite app is through the index.

> * Yelp can be slow. For larger documents, very slow. Breaking the
> document into multiple files does not help.
> 
> DocBook is an inherently single-file format.  I assume this is
> referring to using SYSTEM entities or XInclude to slurp other
> files in.  All of that is handled by libxml2.  As far as Yelp
> is concerned, it's reading one big file.

Actually, the XInclude's seem to help the apparent slowness.  As best I
can tell, Yelp reads the entire document, whether or not the user
navigates to a particular chapter.  However, the first file is displayed
before all files are read, so the user's perception of speed is better
with multiple files, at least, as far as I could tell with my limited
experimentation.

I did try breaking down some of the larger chapters into multiple
smaller files but this didn't seem to affect the crashing.  It was only
whacking down the total document size that helped.

> * Yelp does not display the Publican-produced revision history.
> 
> This I can address, even in the 2.30 timeframe if you want.
> Basically, Yelp's interpretation of DocBook revision history
> is a product of the way Sun was using it to satisfy, to the
> letter, the requirements of the GFDL.  It's a quirky legacy
> behavior, and we can't throw it away entirely, but we can
> adapt to what Publican is doing.

In F12, we worked around this by changing the XML used for the revision
history.  Not ideal, and it looks awful, but at least the history is
there (we had bugs filed about the missing history).

> Packaging a document for Yelp 3.0 is easy.  You just have to
> put it in a location Yelp understands.  No ScrollKeeper or
> any registration procedures.  If Publican can't change its
> installation directory, Yelp may be able to adapt to it.

This is really not a Yelp issue.  It is much more a question of how the
Docs Project wants to make the tradeoffs.  Now that we have worked out
how to package a multiple language HTML, we have more variants to
concern ourselves with.

Within Fedora, a user selects the preferred language at logon time, and
thereafter, everything (that has been translated) is presented in his
native language.  For most applications, while this may make the
application's disk footprint larger, is is generally not a great deal
larger.

With a document, however, the size is multiplied by the number of
languages.  So there is an argument to be made about only installing the
needed languages, which in the majority of cases, is probably only one.
There are (now) only two documents we install by default.  There has
been some discussion about whether it makes sense to have Anaconda
somehow make the selection, but any limitation would cause the documents
to behave differently than the rest of Fedora.

Responding to the idea that a single language is probably the most
common use case, Publican will automatically generate an RPM package for
a single HTML.  We have had no success convincing the Publican
developers that packaging for Yelp or packaging multiple languages is a
good idea, although a Yelp that works across desktops might help that
argument.

Unfortunately, installing multiple, Publican produced RPMs for various
languages really isn't an acceptable solution.  The user is presented
with a menu selection for each language, which quickly becomes a very
long list.  Often, the title is the same in several languages, so the
user may have to open several documents to find the one he wants.

Release Notes and About Fedora are currently packaged more or less
manually.  This isn't rocket science, but having Publican do it
automatically is appealing, especially since most of the writers aren't
developers, and since developers aren't writers, getting a new document
package approved borders on impossible.  We have been able to make the
argument that an automatically produced RPM shouldn't need approval, but
a new, manually produced RPM takes more effort to get approval than it
takes to write the document in the first place.

> Of course, I understand that KDE is important.  I've been
> talking to the KDE team, and I hope we can finally come to
> a consensus on a common help system.  That way you can just
> install your documents to a common location, and Gnome, KDE,
> and XFCE would all pick it up.

This would help, although at least for Gnome and KDE, we are close to an
HTML based solution.  I expect XFCE and Sugar are simply a matter of
research.

> * All languages are installed, whether or not needed. The installed
> size in all languages for our longer documents is hundreds of megabytes.
> 
> I don't see how this has anything to do with Yelp.  This
> is a matter of how you package your files.  Are those pesky
> OMF files keeping you from doing language packs properly?
> I'm fairly certain there's nothing in the 3.0 designs that
> will force this.  The Ubuntu team is also interested in
> doing language packs for documentation, so I want to get
> this right.

Clearly it doesn't have anything to do with Yelp.  This has to do with
decisions about how we want to trade off disk footprint for user
flexibility.  There are limitations to RPM that constrain us here, and I
don't see APT as being any different.  Indeed, this is a place where
Yelp might have a leg up over HTML.

> * Package maintenance has a high overhead; the package maintainer
> must update the package any time any translation team needs to make
> a correction or update.
> 
> Again, I'm not sure what this has to do with Yelp.  Please
> let me know what Yelp can do better to help you here.

This is even further removed from Yelp.  I think this is a question of
how much development (gasp) docs wants to do to make the writer's job
easier.

> * Packages for larger documents may be hundreds of megabytes. Each
> language may (should) have localised images. Images can take up
> substantial space. Limiting images is not an option as images are
> useful for various aspects of learning and documentation.
> 
> Right, so this is the language pack thing again.  I will say
> that one of the nice things in Yelp 3.0 is that it looks up
> everything according to a path, and can fall back to content
> elsewhere in the path.  This includes images.  So if you have
> both a "C" version and a localized version of a document, and
> you have non-localized images (common for including icons in
> documentation), you don't need to install those images with
> the localized document.

Isn't this pretty close to where it is now?  Cant we just add a doc,
then add an OMF, and away we go?  Am I forgetting something?

> Thanks for your time; I know this email was long.  And
> thanks to everybody who's been bringing these concerns
> to my attention.  I'm trying to be more involved with
> downstream, but there's a lot to keep up with.

Shaun, we have some difficult head-scratching to do, and I appreciate
your shining some light on some of these issues.  of course many of the
hard ones have little to do with Yelp, but it is encouraging that
perhaps some of those issues that are Yelp-related may get some
attention.

--McD