[Avocado-devel] [RFC v2]: Avocado maintainability and integration with avocado-vt (was: "Avocado supportability and integration with avocado-vt)
Lukáš Doktor
ldoktor at redhat.com
Tue Apr 19 07:52:26 UTC 2016
Dne 14.4.2016 v 00:22 Ademar Reis napsal(a):
>
> Hi Folks.
>
> This RFC contains proposals and clarifications regarding the
> maintenance and release processes of Avocado.
>
> We understand there are multiple teams currently depending on the
> stability of Avocado and we don't want their work to be disrupted
> by incompatibilities or instability in new releases.
>
> TL;DR:
>
> We plan to keep the current approach of sprint releases every
> 3-4 weeks, but we're introducing "Long Term Stability" releases
> which should be adopted in production environments where users
> can't keep up with frequent upgrades.
>
>
> Changes from v1:
> - Changed "Support" to "Stability" and "supported" to
> "maintained" [Jeff Nelson]
I prefer "Support" because we do support it according to terms mentioned
here, but both are fine by me (important is the widely spread "LTS"
suffix, which is the same for both).
> - Misc improvements and clarifications in the
> supportability/stability statements [Jeff Nelson, me]
> - Fixed a few typos [Jeff Nelson, me]
>
>
> Introduction
> --------------
>
> We make new releases of Avocado every 3-4 weeks on average. In
> theory at least, we're very careful with backwards compatibility.
> We test Avocado for regressions and we try to document any
> issues, so upgrading to a new version should be (again, in
> theory) safe.
>
> But we know in practice things have to change during development
> and regressions are introduced by accident, thus frustrating more
> conservative users. We also understand it's not feasible for
> users to upgrade Avocado very frequently in a production
> environment.
>
> The objective of this RFC is to clarify our maintenance practices
> and introduce Long Term Stability (LTS) releases, which are
> intended to solve, or at least mitigate, these problems.
>
>
> Our definition of maintained, or stable
> ---------------------------------------
>
> First of all, Avocado and its sub-projects are provided 'AS IS'
> and WITHOUT ANY WARRANTY, as described in the LICENSE file.
>
> The process described here doesn't imply any commitments or
> promises. It's just a set of best practices and recommendations.
>
> When something is identified as "stable" or "maintained", it
> means the development community makes a conscious effort to keep
> it working and consider reports of bugs and issues as high
> priorities. Fixes submitted for these issues will also be
> considered high priorities, although they will be accepted only
> if they pass the general acceptance criteria for new
> contributions (design, quality, documentation, testing, etc),
> at the development team discretion.
>
>
> Maintained projects and platforms
> ---------------------------------
>
> The only maintained project as of today is the Avocado Test
> Runner, including its APIs and core plugins (the contents of
> the main avocado git repository).
>
> Other projects kept under the "Avocado Umbrella" in github may be
> maintained by different teams (e.g.: avocado-vt) or be considered
> experimental (e.g.: avocado-server and avocado-virt).
>
> More about avocado-vt in its own section further down.
>
> As a general rule, fixes and bug reports for Avocado when running
> in any modern Linux distribution are welcome.
>
> But given the limited capacity of the development team, packaged
> versions of Avocado will be tested and maintained only for the
> following Linux distributions:
>
> * RHEL 7.x (latest)
> * RHEL 6.x (latest)
> * Fedora (stable releases from the Fedora projects)
>
> Ditto for hardware platforms. The development team will maintain:
> * x86
> * ppc64
> * ppc64le
>
> Contributions from the community to maintain other platforms and
> operating systems are very welcome.
>
> The lists above may change without prior notice.
>
>
> Avocado Releases
> ----------------
>
> The proposal is to have two different types of Avocado releases:
>
> 1. Sprint Releases:
> (This is the model we currently adopt in Avocado)
>
> They happen every 3-4 weeks (the schedule is not fixed) and
> their versions are numbered serially, with decimal digits in
> the format <major>.<minor>. Examples: 33.0, 34.0, 35.0. Minor
> releases are rare, but necessary to correct some major issue
> with the original release (33.1, 33.2, etc).
>
> Only the latest Sprint Release is maintained.
>
> In Sprint Releases we make a conscious effort to keep backwards
> compatibility with the previous version (APIs and behavior) and
> as a general rule and best practice, incompatible changes in
> Sprint Releases should be documented in the release notes and
> if possible deprecated slowly, to give users time to adapt
> their environments.
>
> But we understand changes are inevitable as the software
> evolves and therefore there's no absolute promise for API and
> behavioral stability.
>
> 2. Long Term Stability (LTS) Releases:
>
> LTS releases should happen whenever the team feels the code is
> stable enough to be maintained for a longer period of time,
> ideally once or twice per year (no fixed schedule).
>
> They should be versioned just like Sprint Releases, but have a
> suffix 'lts' added to them. Examples: 35.0lts, 36.1lts, etc.
>
> They should be maintained for 18 months, receiving fixes for
> major bugs in the form of minor (sub-)releases. With the
> exception of these fixes, no API or behavior should change in a
> minor LTS release. In practice each major LTS release will
> imply in the creation of a git branch where only serious issues
> affecting users will be fixed. The code in a LTS branch is
> stable, frozen for new features.
>
> Notice that although within a LTS release there's a expectation
> of stability because the code is frozen, different (major) LTS
> releases may include changes in behavior, API incompatibilities
> and new features. The development team will make a considerable
> effort to minimize and properly document these changes (changes
> when comparing it to the last major LTS release).
>
> Sprint Releases are replaced by LTS releases. I.e., in the
> cycle when 35.0lts is released, that's also the version used as
> a Sprint Release (there's no 35.0 -- non lts -- in this case).
>
> New LTS releases should be done carefully, with ample time for
> announcements, testing and documentation. It's recommended
> that one or two sprints are dedicated as preparations for a LTS
> release, with a Sprint Release serving as a "LTS beta" release.
>
> Similarly, there should be announcements about the end-of-life
> (EOL) of a LTS release once it approaches its 18 months of
> life.
>
>
> Misc details
> ------------
>
> Sprint and LTS releases, when packaged, should be preferably
> distributed through different package channels (repositories).
> Users can opt to follow whatever channel they prefer. The actual
> layout of the packages repositories has not been specified yet.
>
> Via pip, Avocado Sprint Releases will be made available.
>
> The existence of LTS releases should never be used as an excuse
> to break a Sprint Release or to introduce gratuitous
> incompatibilities there. In other words, Sprint Releases should
> still be taken seriously, just as they are today.
>
>
> Timeline example
> ----------------
>
> For simplicity, assume each sprint is taking 1 month. The number
> of LTS releases is exaggerated to show how they would co-exist
> before EOL.
>
> sprint release 33.0
> sprint release 34.0
> --> start preparing a LTS release, so 35.0 is a beta LTS
> sprint release 35.0
> LTS release 36.0lts (36lts branch is created)
> --> major bug is found, fix gets added to master and to
> the 36lts branch
> --> minor releases are made whenever the development team
> feels it's convenient to do so.
> sprint release 37.0 + 36.1lts # not necessarily in sync
> sprint release 38.0
> --> major bug is found, fix gets added to master and
> 36lts branches
> --> minor releases are made whenever the development team
> feels it's convenient to do so.
> sprint release 39.0 + LTS 36.2lts # not necessarily in sync
> sprint release 40.0
> sprint release 41.0
> --> start preparing a LTS release, so 42.0 is a beta LTS
> sprint release 42.0
> --> review and document all compatibility changes
> and features introduced since 36.2lts
> LTS release 43.0lts (43lts branch is created)
> sprint release 44.0
> sprint release 45.0
> --> major bug is found, fix gets added to master and LTS
> branches 36lts and 43lts (if the bug affects users
> there)
> --> minor releases are made whenever the development team
> feels it's convenient to do so.
> sprint release 46.0 + LTS 36.3lts + LTS 43.1lts ...
> sprint release 47.0
> sprint release 48.0
> --> start preparing a LTS release, so 49.0 is a beta LTS
> sprint release 49.0
> --> review and document all compatibility changes and
> features introduced since 43.1lts
> sprint release 50.0lts (50lts branch is created)
> --> EOL for 36lts is approaching, there should be
> some sort of announcement.
> sprint release 51.0
> sprint release 52.0
> sprint release 53.0
> sprint release 54.0
> --> EOL for 36lts (18 months since the release of 36.0lts)
> sprint release 55.0
> ...
>
>
> avocado-vt
> ----------
>
> avocado-vt is an Avocado plugin that allows "VT tests" to be run
> inside Avocado. It's a third-party project maintained mostly by
> Engineers from Red Hat QE with assistance from the Avocado team
> and other community members.
>
> It's a general consensus that QE teams use avocado-vt directly
> from git, usually following the master branch, which they
> control.
>
> There's no official maintenance or stability statement for
> avocado-vt. Even though the upstream community is quite
> friendly and open to both contributions and bug reports,
> avocado-vt is made available without any promises for
> compatibility or supportability.
>
> When packaged and versioned, avocado-vt rpms should be considered
> just snapshots, available in packaged form as a convenience to
> users outside of the avocado-vt development community. Again,
> they are made available without any promises of compatibility or
> stability.
>
> * Which Avocado version should be used by avocado-vt?
>
> This is up to the avocado-vt community to decide, but the
> current consensus is that to guarantee some stability in
> production environments, avocado-vt should stick to a specific
> LTS release of Avocado. In other words, the Avocado team
> recommends production users of avocado-vt not to install Avocado
> from its master branch or upgrade it from Sprint Releases.
>
> Given each LTS release will be maintained for 18 months, it
> should be reasonable to expect avocado-vt to upgrade to a new
> LTS release once a year or so. This process will be done with
> support from the Avocado team to avoid disruptions, with proper
> coordination via the avocado mailing lists.
>
> In practice the Avocado development team will keep watching
> avocado-vt to detect and document incompatibilities, so when
> the time comes to do an upgrade in production, it's expected
> that it should happen smoothly.
>
> * Will it be possible to use the latest Avocado and avocado-vt
> together?
>
> Users are welcome to *try* this combination. The Avocado
> development team itself will probably do it internally as a way
> to monitor incompatibilities and regressions.
>
> Given the open source nature of both projects, we expect
> volunteers to step up and maintain an upstream branch of
> avocado-vt that works with the most recent Avocado Sprint
> Release.
>
> If no volunteers show up, we might release snapshots of
> avocado-vt in the Avocado LTS channel, for convenience only,
> just as we do today with our Sprint Releases.
>
> Thanks.
> - Ademar
>
Acked-by: Lukáš Doktor <ldoktor at redhat.com>
More information about the Avocado-devel
mailing list