Planning a future L10N infrastructure (including Fedora)

[Asgeir Frimannsson]

On Tuesday 16 September 2008 13:16:06 Mike McGrath wrote:
> On Mon, 15 Sep 2008, Asgeir Frimannsson wrote:
> > Hi infrastructure wranglers,
> >
> > (cc transifex-devel)
> >
> > Over the last few months, a few of us involved in Red Hat L10N
> > engineering have discussed how to best ensure we have Localisation
> > Infrastructure and Tools that can serve the needs of Red Hat, JBoss,
> > Fedora and 'upstream' communities  in years to come. Let me first
> > describe some of the background and requirements behind this project:
> >
> > Up until now, we have managed translations through version control
> > systems such as CVS, Svn and Git. This has ensured that all contributions
> > are pushed upstream, as we always store translations within the upstream
> > repositories and projects. 'Damned Lies' further gave us a tool to view
> > language-specific translation statistics for modules, branches and
> > releases, as well as convenient information about people, teams and
> > projects. This has been a great help for translators in their work.
> > Dimitris' (and others) work on Transifex has in addition given the
> > translation community a way to submit translations upstream without ever
> > touching a developer-centric version control system, which has been of
> > great help to translators.
> >
> > Some of the immediate needs that could be addressed within the existing
> > framework (some of which are on the Transifex roadmap) are:
> > - Consolidation of Damned Lies and Transifex, allowing retrieving and
> > submitting translations through the same interface
> > - Allowing retrieving and submitting multiple-files at once (e.g. for
> > translating a publican document with many PO files)
> > - Simple workflow on top of Transifex (porting features from Vertimus)
> > - Better usability and easier user registration process (Fedora specific)
> >
> > Transifex is gaining some traction upstream (e.g. within Gnome), and we
> > hope development will continue strong, serving Fedora and potentially
> > other upstream communities.
> >
> > Looking at the bigger picture, some of the core requirements we have
> > identified for Red Hat and community L10N going forward are:
> > - Customizable Translation Workflows and integration with e.g. Content
> > Authoring Workflows
> > - Infrastructure easily adaptable to support new File formats and project
> > types (e.g. OpenOffice formats, CMS formats, DTP formats, Wiki, Dita,
> > Java formats), rather than relying on 'upstream' projects to fit a
> > certain L10N infrastructure.
> > - Managing the life-cycle of a translation project across releases and
> > iterations
> > - Translation Reuse and Terminology Management across projects and
> > iterations - Job management, scoping, tracking and resourcing
> > - Managing and/or Tracking upstream translation projects, pushing changes
> > back upstream.
> >
> > These requirements require a system where the translation lifecycle would
> > be managed within 'Translation Repositories' (similar to e.g. Pootle or
> > Launchpad Translations), rather than directly through e.g. upstream
> > version control systems. With a repository-based approach, we would be
> > able to track and manage changes to a project on a translation unit
> > level, and manage e.g. translation reuse and terminology within and
> > across projects. We could still retain a link with upstream repositories
> > (like with Transifex/Damned Lies). However, this would not be the 'core
> > datamodel', but on a different layer through plug-ins. This link to
> > external repositories could also go beyond traditional version control
> > systems, communicating with external sources like wikis and CMSs.
>
> I'd think much of what you're looking to do could be done in transifex
> farily easily.  I think some of it is already done and being done.
>
> > We have evaluated a number of existing open source L10N frameworks and
> > systems, but haven't found any (yet) that stands out or satisfies our
> > needs or requirements as a development platform. Technology-wise, we are
> > aiming to develop a Java-based(!) system, using technology such as JBoss
> > Seam, Hibernate, jBPM and RichFaces. A java based platform will enable us
> > to make best use of internal expertise in these technologies, as well as
> > making use of technology we are developing (as open source) through
> > collaboration with partners in the L10N industry.
> >
> > We hope some of these requirements and ideas will excite some of you, and
> > ultimately lead to something that can be of use to open source
> > communities. While we have certain requirements and goals for this
> > internally within the company, there is no need for this to be an
> > 'internal' Red Hat project, and most of the requirements and needs
> > overlap with those of community projects like Fedora. In other words, by
> > developing this in collaboration with the community from a very early
> > stage, we are more likely to develop something that may be of use to the
> > greater community.
> >
> > Thoughts and comments, all sorts of comments, are very welcome.
>
> Please correct me if I'm reading this wrong but I see "transifex is great
> or close to it" and "here's how we're going to build our own solution
> anyway" ?

Yes, "Transifex is great and will continue to serve us". 

BUT:

If you look at the state of the art in L10N outside the typical Linux projects 
where PO and Gettext rule, you'll notice we are very short on areas like:
- Translation Reuse
- Terminology Management
- Translation Workflow and Project Management
- Integration with CMSs. 
- Richer Translation Tools

This is an effort in narrowing that gap, and I can't see that effort work by 
evolving an existing tool from this 'cultural background'. Yes, we can get 
some of the way by developing custom solutions for e.g. linking wikis to 
Transifex for CMS integration, or using e.g. Pootle for web-based translation. 
But we would still be limited to the core architecture of the intent of the 
original developers, which is something that would radically slow the project 
down. 

That said, I am not talking down Transifex, and the fact that someone in the 
community has sacrificed a lot and done a great job in getting us this far 
within Fedora. 

cheers,
asgeir