[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Avocado-devel] [RFC]: Avocado supportability and integration with avocado-vt



Hi Folks.

This is a proposal for the supportability of Avocado. We
understand there are now multiple teams depending on the
stability of Avocado and we don't want your work to be disrupted
by our changes.

TL;DR:

  We plan to keep the current approach of sprint releases every
  3-4 weeks, but we're introducing "Long Term Support" releases
  which should be adopted in production environments where users
  can't keep up with frequent upgrades.


Introduction
--------------

We make new releases of Avocado every 3-4 weeks on average.  In
theory at least, we're very careful with backwards compatibility.
We test Avocado for regressions and we try to document any
issues, so upgrading to a new version should be (again, in
theory) safe.

But we know in practice things have to change during development
and regressions are introduced by accident, thus frustrating more
conservative users. We also understand it's not feasible for
users to upgrade Avocado very frequently in a production
environment.

The objective of this RFC is to clarify our supportability
practices and introduce Long Term Support (LTS) releases, which
are intended to solve these issues.


What we support (maintain)
--------------------------

Avocado and its sub-projects are provided 'AS IS' and WITHOUT ANY
WARRANTY, as described in the LICENSE file. The supportability
described here is not a commitment, but a set of best practices
and recommendations.

When something is listed as "supported" it means the development
community makes a conscious effort to keep it working and
consider reports of bugs and issues as high priorities.  Fixes
submitted for these issues are also considered with high
priority, although they will be accepted only if they pass the
general acceptance criteria for new contributions (design,
quality, documentation, testing, etc), at the development team
discretion.

Supported Projects:
  - The Avocado test runner, including its core plugins and
    APIs (the main avocado git repository)
  - Packaged versions of Avocado for:
    - RHEL 7.x (latest)
    - RHEL 6.x (latest)
    - Fedora (stable, supported releases)
  - Hardware platforms: x86, ppc64, ppc64le

Contributions from the community to maintain other platforms and
operating systems are welcome. The list above may change without
prior notice (will be announced in the avocado-devel mailing
list).

Other projects kept under the "Avocado Umbrella" in github may be
maintained by different teams (e.g.: avocado-vt) or be considered
experimental (e.g.: avocado-server and avocado-virt).

More about avocado-vt in a section further down.


Avocado Releases
----------------

The proposal is to have two different types of Avocado releases:

1. Sprint Releases:
  (This is the model we currently adopt in Avocado)
  
  They happen every 3-4 weeks (the schedule is not fixed) and
  their versions are numbered serially, with decimal digits.
  Example: 33.0, 34.0, 35.0.  Minor releases are rare, but
  necessary to correct some major issue with the original release
  (33.1, 33.2, etc).

  Only the latest Sprint Release is supported.

  In Sprint Releases we make a conscious effort to keep backwards
  compatibility in Avocado (APIs and behavior) and as a general
  rule and best practice, incompatible changes in Sprint Releases
  should be documented in the release notes and if possible
  slowly deprecated to give users time to adapt their
  environments.

  But we understand changes are inevitable as the software
  evolves and therefore there's no absolute promise for API and
  behavioral stability.

2. Long Term Support (LTS) Releases:

  LTS releases should happen whenever the team feels the code is
  stable enough to be supported for a longer period of time,
  ideally once or twice per year (no fixed schedule). They should
  be versioned just like Sprint Releases, but have a suffix 'lts'
  added to them (examples: 35.0lts, 36.1lts, etc).

  They should be supported for 18 months, receiving fixes for
  major bugs in the form of minor (sub-)releases. With the
  exception of these fixes, no API or behavior should change in a
  minor LTS release.  In practice each LTS release will be
  tracked through a git branch where only serious issues
  affecting users are fixed. The code in a LTS branch is frozen
  and therefore stable.

  Notice that although within a LTS release there's a promise of
  stability because the code is frozen, different (major) LTS
  releases may include changes in behavior, API incompatibilities
  and new features. The development team will make a considerable
  effort to minimize and properly document these changes (changes
  when comparing it to the last major LTS release).

  Sprint Releases are replaced by LTS releases. I.e., in the
  cycle when 35.0lts is released, that's also the version used as
  a Sprint Release (there's no 35.0 -- non lts -- in this case).

  New LTS releases should be done carefully, with ample time for
  announcements, testing and documentation.  It's recommended
  that at least two sprints are dedicated as preparations for a
  LTS release, with the first Sprint Release serving as a "LTS
  beta" release.
 

Misc details
------------

Sprint and LTS releases, when packaged, should be preferably
distributed through different package channels (repositories).
Users can opt to follow whatever channel they prefer. The actual
layout of the packages repositories has not been specified yet.

Via pip, Avocado Sprint Releases will be made available.

The existence of LTS releases should never be used as an excuse
to break a Sprint Release or to introduce gratuitous
incompatibilities there. In other words, Sprint Releases should
still be taken seriously, just as they are today.


Timeline example
----------------

For simplicity, assume each sprint is taking 1 month. The number
of LTS releases is exaggerated to show how they would co-exist
before EOL.

    sprint release 33.0
    sprint release 34.0
       --> start preparing a LTS release, so 35.0 is a beta LTS
    sprint release 35.0
    RTL release 36.0lts (36lts branch is created)
       --> major bug is found, fix gets added to master and to
           the 36lts branch
    sprint release 37.0 + 36.1lts
    sprint release 38.0
       --> major bug is found, fix gets added to master and
           36lts branches
    sprint release 39.0 + LTS 36.2lts
    sprint release 40.0
    sprint release 41.0
       --> start preparing a LTS release, so 42.0 is a beta LTS
    sprint release 42.0
       --> review and document all compatibility changes
           and features introduced since 36.2lts
    RTL release 43.0lts (43lts branch is created)
    sprint release 44.0
    sprint release 45.0
      --> major bug is found, fix gets added to master and RTL
          branches 36lts and 43lts (if the bug affects users
          there)
    sprint release 46.0 + LTS 36.3lts + LTS 43.1lts
    sprint release 47.0
    sprint release 48.0
       --> start preparing a LTS release, so 49.0 is a beta LTS
    sprint release 49.0
       --> review and document all compatibility changes and
           features introduced since 43.1lts
    sprint release 50.0lts (50lts branch is created)
    sprint release 51.0
    sprint release 52.0
    sprint release 53.0
    sprint release 54.0
       --> EOL for 36lts (18 months since the release of 36.0lts)
    sprint release 55.0
    ...


avocado-vt
----------

avocado-vt is an Avocado plugin that allows "VT tests" to be run
inside Avocado.  It's a third-party project maintained mostly by
Engineers from Red Hat QE with assistance from the Avocado team.

It's a general consensus that QE teams use avocado-vt directly
from git, usually following the master branch, which they
control. 

There's no support statement for avocado-vt. Even though the
upstream community is usually quite friendly and open to both
contributions and bug reports, avocado-vt is made available
without any promises for compatibility of supportability.

When packaged and versioned, avocado-vt rpms should be considered
just snapshots, available in packaged form as a convenience to
users outside of the avocado-vt development community.  Again,
they are made available without any promises of compatibility or
stability.

- Which Avocado version should be used by avocado-vt?

  This is up to the avocado-vt community to decide, but the
  current consensus is that to guarantee stability in production
  environments, avocado-vt should stick to a specific LTS release
  of Avocado. In other words, production users of avocado-vt
  should not install Avocado from its master branch or upgrade it
  from Sprint Releases.
  
  Given each LTS release will be supported for 18 months, it
  should be reasonable to expect avocado-vt to upgrade to a new
  LTS release once a year or so. This process will be done with
  support from the Avocado team to avoid disruptions, with proper
  coordination via the avocado mailing lists.

  In practice the Avocado development team will keep watching
  avocado-vt to detect and document incompatibilities, so when
  the time comes to do an upgrade in production, it's expected
  that it should happen smoothly.

- Will it be possible to use the latest Avocado and avocado-vt
  together?

  Users are welcome to *try* this combination.  The Avocado
  development team itself will probably do it internally as a way
  to monitor incompatibilities and regressions.
  
  Given the open source nature of both projects, we expect
  volunteers to step up and maintain an upstream branch of
  avocado-vt that works with the most recent Avocado Sprint
  Release.

  If no volunteers show up, we might release snapshots of
  avocado-vt in the Avocado LTS channel, for convenience only,
  just as we do today with our Sprint Releases.

Thanks.
   - Ademar

-- 
Ademar Reis
Red Hat

^[:wq!


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]