[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [rdo-list] Multiple tools for deploying and testing TripleO



Please hear me out.
TL;DR, Let's work upstream and make it awesome so that downstream can
be awesome.

I've said this before but I'm going to re-iterate that I do not
understand why there is so much effort spent around testing TripleO
downstream.
By downstream, I mean anything that isn't in TripleO or TripleO-CI proper.

All this work should be done upstream to make TripleO and it's CI
super awesome and this would trickle down for free downstream.

The RDO Trunk testing pipeline is composed of two tools, today.
The TripleO-Quickstart project [1] is a good example of an initiative
that started downstream but always had the intention of being proposed
upstream [2] after being "incubated" and fleshed out.
The WeIRDO project [3] is nothing less than a simple and dumb package
and repository installer and runs upstream gate jobs as is.

I'd like to share an example of the benefits (for everyone involved)
that shifting focus from downstream and upstream can accomplish.

About a year ago, Packstack was installed by Khaleesi (now InfraRed as
I understand it?) to test RDO packages.
Khaleesi is a tool that has knowledge of how to deploy multiple
installers, where to deploy them, how to configure them and how to
test them.
This results in, objectively, a complex and complicated tool because
it does not serve a single purpose - not the UNIX philosophy of "do
one thing and do it well".

So, in the context of Packstack, Khaleesi could install it in a variety of ways.
The important part of the story is that Khaleesi would also install,
configure and run Tempest itself after the Packstack installation was
complete.

But Packstack already supported installing Tempest -- except it was
honestly largely broken and missing features.
So I went out and massively refactored the support of Tempest in
Packstack. It was a lot of work, it took several months of on and off
effort.
In the end, this Tempest support refactor enabled us to add
integration gate jobs that test Packstack in various configurations
[5] -- upstream.

Before we added those integration jobs, we were mostly blind when
merging commits to Packstack: trunk and users were (unfortunately)
often broken.
Today, these jobs are critical and relied on when merging commits,
whether small or incredibly large refactors [6].

And the best thing about that is that we're pulling in those three
integration tests for free in RDO CI with WeIRDO just like the
puppet-openstack-integration tests and their great coverage [7].

Back to my original point, I hope this example shows that by shifting
our focus upstream, we can make upstream better and this will not only
improve the project but it will also directly benefit downstream.
I'm convinced we can all agree that if upstream is awesome, so will
downstream be.

Now, I'm not crazy. I realize that, in the end, Red Hat has a product
and that product may have higher quality standards than the upstream
equivalent.
However, I still strongly believe that we should focus on tooling that
is first adopted - and used - upstream, ensure that tooling is
extendable and pluggable so that downstream can hook in their
additional testing if they wish.

[1]: https://github.com/openstack/tripleo-quickstart
[2]: http://specs.openstack.org/openstack/tripleo-specs/specs/mitaka/tripleo-quickstart.html
[3]: https://github.com/rdo-infra/weirdo
[4]: https://github.com/openstack/packstack#packstack-integration-tests
[5]: https://www.redhat.com/archives/rdo-list/2016-March/msg00007.html
[6]: https://review.openstack.org/#/c/329505/
[7]: https://github.com/openstack/puppet-openstack-integration#description

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Wed, Aug 3, 2016 at 9:00 AM, Wesley Hayutin <whayutin redhat com> wrote:
>
>
> On Wed, Aug 3, 2016 at 5:23 AM, Frank Zdarsky <fzdarsky redhat com> wrote:
>>
>> On Wed, Aug 3, 2016 at 9:28 AM, Tal Kammer <tkammer redhat com> wrote:
>>>
>>> Thanks Arie for starting this discussion! (and sorry for joining in late)
>>>
>>> Some detailed explanations for those not familiar with InfraRed and would
>>> like to get the highlights:  (Feel free to skip to my inline comments if you
>>> are familiar / too long of a post :) ).
>>>
>>> The InfraRed project is an Ansible based project comprised from three
>>> distinct tools: (currently under the InfraRed[1] project, and being split
>>> into their own standalone project soon).
>>>
>>> 1. ir-provisioner - responsible for building up a custom environment -
>>> you can affect the memory, CPU,  HDD size, number of HDD each node has +
>>> number of networks (with/without DHCP) so for example one can deploy the
>>> following topology: (only an example to show the versatile options)
>>> 1 undercloud on RHEL 7 with 16GB of ram + 60GB of HDD with 3 network
>>> interfaces.
>>> 3 controllers on Centos 7 with 8GB of ram  + 40GB of HDD
>>> 2 compute on Centos 7 with 6GB of ram + 60GB HDD
>>> 3 ceph nodes on RHEL 7 with 4GB of ram + 2 HDD one with 20GB + one with
>>> 40GB
>>>
>>> Example usage: (setting up the above VMs with four different HW specs)
>>> ir-provisioner virsh --host-address=<host.address>
>>> --host-key=<host.private.key>
>>> --topology-nodes=undercloud:1,controller:3,compute:2,ceph:3
>>>
>>> *Note: while it is written "controller/compute/ceph" this is just setting
>>> up VMs, the names act more as a reference to the user of what is the role of
>>> each node.
>>> The installation of Openstack is done with a dedicated tool called
>>> `ir-installer` (next)
>>>
>>> 2. ir-installer - responsible for installing the product - supports
>>> "quickstart" mode (setting up a working environment in ~30 minutes) or E2E
>>> mode which does a full installation in ~1h.
>>> The installation process is completely customized. You can supply your
>>> own heat templates / overcloud invocation / undercloud.conf file to use /
>>> etc. You can also just run a specific task (using ansible --tags), so if you
>>> have a deployment ready and just need to run say, the introspection phase,
>>> you can fully choose what to run and what to skip even.
>>>
>>> 3. ir-tester - responsible for installing / configuring / running the
>>> tests - this project is meant to hold all testing tools we use so a user
>>> will be able to run any testing utility he would like without the need to
>>> "dive in". we supply a simple interface requesting simple to choose the
>>> testing tool and the set of tests one wishes to run and we'll do the work
>>> for him :)
>>>
>>> More info about InfraRed can be found here[1] (though I must admit that
>>> we still need some "love" around our docs)
>>>
>>> [1] - http://infrared.readthedocs.io/en/latest/
>>>
>>
>> I agree cleanly separating and modularizing infrastructure provisioining
>> (for some use cases like NFV ideally with mixed VM and baremetal
>> environments), OS installation and testing is a good approach. In that
>> context, what happened to Khaleesi [0]? Haven't seen that mentioned on the
>> original list. Nor Apex [1], the installer from OPNFV that does not only
>> address OpenStack itself, but also integration with complementary projects
>> like ODL?
>
>
> I consider provisioning baremetal for either libvirt hosts or baremetal
> installs is a separate project and conversation from the CI tools used to
> deploy tripleo.
> To be a little more clear, provisioning imho should be completely uncoupled
> from the CI code that deploys tripleo.  Any team should be able to use
> provisioner tools that best fit their needs.
> If there is interest in collaborating on provisioners maybe we can do that
> in another thread.  Why in another thread? Again my thought is they should
> be uncoupled completely from our CI source code.
>
> The rdo infra team is using two provisioners, both completely uncoupled from
> tripleo-quickstart.
>
> 1. Ansible extra heat provisioner:  for ovb, written by Mathieu Bultel
> https://github.com/ansible/ansible-modules-extras/commit/c6a45234e094922786de78a83db3028804cc9141
>
> 2. cico, the official client for ci.centos written by David Simard
> http://python-cicoclient.readthedocs.io/en/latest/cli_usage.html
>
> Our baremetal deployments do not require any provisioning outside of what is
> done w/ ironic and tripleo.
>
> Upstream tripleo ci provisioning is done via node pool and ovb.
>
> FYI.. khaleesi has been deprecated and is no longer used by RDO infra.
>
> Thanks
>
>>
>>
>> [0] https://github.com/redhat-openstack/khaleesi
>> [1] https://wiki.opnfv.org/display/apex/Apex
>>
>>>
>>>
>>> On Tue, Aug 2, 2016 at 9:52 PM, Wesley Hayutin <whayutin redhat com>
>>> wrote:
>>>>
>>>>
>>>>
>>>> On Tue, Aug 2, 2016 at 1:51 PM, Arie Bregman <abregman redhat com>
>>>> wrote:
>>>>>
>>>>> On Tue, Aug 2, 2016 at 3:53 PM, Wesley Hayutin <whayutin redhat com>
>>>>> wrote:
>>>>> >
>>>>> >
>>>>> > On Tue, Aug 2, 2016 at 4:58 AM, Arie Bregman <abregman redhat com>
>>>>> > wrote:
>>>>> >>
>>>>> >> It became a discussion around the official installer and how to
>>>>> >> improve it. While it's an important discussion, no doubt, I actually
>>>>> >> want to focus on our automation and CI tools.
>>>>> >>
>>>>> >> Since I see there is an agreement that collaboration does make sense
>>>>> >> here, let's move to the hard questions :)
>>>>> >>
>>>>> >> Wes, Tal - there is huge difference right now between infrared and
>>>>> >> tripleo-quickstart in their structure. One is all-in-one project and
>>>>> >> the other one is multiple micro projects managed by one project. Do
>>>>> >> you think there is a way to consolidate or move to a different model
>>>>> >> which will make sense for both RDO and RHOSP? something that both
>>>>> >> groups can work on.
>>>>> >
>>>>> >
>>>>> > I am happy to be part of the discussion, and I am also very willing
>>>>> > to help
>>>>> > and try to drive suggestions to the tripleo-quickstart community.
>>>>> > I need to make a point clear though, just to make sure we're on the
>>>>> > same
>>>>> > page..  I do not own oooq, I am not a core on oooq.
>>>>> > I can help facilitate a discussion but oooq is an upstream tripleo
>>>>> > tool that
>>>>> > replaces instack-virt-setup [1].
>>>>> > It also happens to be a great tool for easily deploying TripleO end
>>>>> > to end
>>>>> > [3]
>>>>> >
>>>>> > What I *can* do is show everyone how to manipulate tripleo-quickstart
>>>>> > and
>>>>> > customize it with composable ansible roles, templates, settings etc..
>>>>> > This would allow any upstream or downstream project to override the
>>>>> > native
>>>>> > oooq roles and *any* step that does not work for another group w/ 3rd
>>>>> > party
>>>>> > roles [2].
>>>>> > These 3rd party roles can be free and opensource or internal only, it
>>>>> > works
>>>>> > either way.
>>>>> > This was discussed in depth as part of the production chain meetings,
>>>>> > the
>>>>> > message may have been lost unfortunately.
>>>>> >
>>>>> > I hope this resets your expectations of what I can and can not do as
>>>>> > part of
>>>>> > these discussions.
>>>>> > Let me know where and when and I'm happy to be part of the
>>>>> > discussion.
>>>>>
>>>>> Thanks for clarifying :)
>>>>>
>>>>> Next reasonable step would probably be to propose some sort of
>>>>> blueprint for tripleo-quickstart to include some of InfraRed features
>>>>> and by that have one tool driven by upstream development that can be
>>>>> either cloned downstream or used as it is with an internal data
>>>>> project.
>>>>
>>>>
>>>> Sure.. a blueprint would help everyone understand the feature and the
>>>> motivation.
>>>> You could also just plug in the feature you are looking for to oooq and
>>>> see if it meets
>>>> your requirements.  See below.
>>>
>>>
>>> While I think a blueprint is a good starting point, I'm afraid that our
>>> approach for provisioning machines is completely different so I'm not sure
>>> how to propose such a blueprint as it will probably require quite the design
>>> change from today's approach.
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> OR
>>>>>
>>>>> have InfraRed pushed into tripleo/openstack namespace and expose it to
>>>>> the RDO community (without internal data of course). Personally, I
>>>>> really like the pluggable[1] structure (which allows it to actually
>>>>> consume tripleo-quickstart) so I'm not sure if it can be really merged
>>>>> with tripleo-quickstart as proposed in the first option.
>>>
>>>
>>> I must admit that I like this option better as it introduces a tool to
>>> upstream and let the community drive it further / get more feedback on how
>>> to improve.
>>> A second benefit might be that by introducing a new concept / design we
>>> can take what is best from two worlds and improve.
>>> I would love to see an open discussion on the tool upstream and how we
>>> can improve the overall process.
>>>
>>>>
>>>> The way oooq is built one can plugin or override any part at run time
>>>> with custom playbooks, roles, and config.  There isn't anything that
>>>> needs to be
>>>> checked in directly to oooq to use it.
>>>>
>>>> It's designed such that third parties can make their own decisions to
>>>> use something
>>>> native to quickstart, something from our role library, or something
>>>> completely independent.
>>>> This allows teams, individuals or whom ever to do what they need to with
>>>> out having to fork or re-roll the entire framework.
>>>>
>>>> The important step is to note that these 3rd party roles or
>>>> (oooq-extras) incubate, mature and then graduate to github/openstack.
>>>> The upstream openstack community should lead, evaluate, and via
>>>> blueprints vote on the canonical CI tool set.
>>>>
>>>> We can record a demonstration if required, but there is nothing stopping
>>>> anyone right now from
>>>> doing this today.  I'm just browsing the role library for an example, I
>>>> had no idea [1] existed.
>>>> Looks like Raoul had a requirement and just made it work.
>>>>
>>>> Justin, from the browbeat project has graciously created some
>>>> documentation regarding 3rd party roles.
>>>> It has yet to merge, but it should help illustrate how these roles are
>>>> used. [2]
>>>>
>>>> Thanks Arie for leading the discussion.
>>>>
>>>> [1]
>>>> https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-validate-ha
>>>> [2] https://review.openstack.org/#/c/346733/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> I like the second option, although it still forces us to have two
>>>>> tools, but after a period of time, I believe it will be clear what the
>>>>> community prefers, which will allow us to remove one of the projects
>>>>> eventually.
>>>>>
>>>>> So, unless there are other ideas, I think the next move should be made
>>>>> by Tal.
>>>>>
>>>>> Tal, I'm willing to help with whatever is needed.
>>>
>>>
>>> Thanks Arie for starting this discussion again, I believe we have still
>>> much work ahead of us but this is definitely a step in the right direction.
>>>
>>>>>
>>>>>
>>>>> [1] http://infrared.readthedocs.io/en/latest
>>>>>
>>>>> >
>>>>> > Thanks
>>>>> >
>>>>> > [1] https://blueprints.launchpad.net/tripleo/+spec/tripleo-quickstart
>>>>> > [2]
>>>>> >
>>>>> > https://github.com/redhat-openstack/?utf8=%E2%9C%93&query=ansible-role-tripleo
>>>>> > [3[ https://www.rdoproject.org/tripleo/
>>>>> >
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> Raoul - I totally agree with you, especially with "difficult for
>>>>> >> anyone to start contributing and collaborate". This is exactly why
>>>>> >> this discussion started. If we can agree on one set of tools, it
>>>>> >> will
>>>>> >> make everyone's life easier - current groups, new contributors,
>>>>> >> folks
>>>>> >> that just want to deploy TripleO quickly. But I'm afraid some
>>>>> >> sacrifices need to be made by both groups.
>>>>> >>
>>>>> >> David - I thought WeiRDO is used only for packstack, so I apologize
>>>>> >> I
>>>>> >> didn't include it. It does sound like an anther testing project, is
>>>>> >> there a place to merge it with another existing testing project?
>>>>> >> like
>>>>> >> Octario for example or one of TripleO testing projects. Or does it
>>>>> >> make sense to keep it a standalone project?
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Tue, Aug 2, 2016 at 11:12 AM, Christopher Brown
>>>>> >> <cbrown2 ocf co uk>
>>>>> >> wrote:
>>>>> >> > Hello RDOistas (I think that is the expression?),
>>>>> >> >
>>>>> >> > Another year, another OpenStack deployment tool. :)
>>>>> >> >
>>>>> >> > On Mon, 2016-08-01 at 18:59 +0100, Ignacio Bravo wrote:
>>>>> >> >> If we are talking about tools, I would also want to add something
>>>>> >> >> with regards to user interface of these tools. This is based on
>>>>> >> >> my
>>>>> >> >> own experience:
>>>>> >> >>
>>>>> >> >> I started trying to deploy Openstack with Staypuft and The
>>>>> >> >> Foreman.
>>>>> >> >> The UI of The Foreman was intuitive enough for the discovery and
>>>>> >> >> provisioning of the servers. The OpenStack portion, not so much.
>>>>> >> >
>>>>> >> > This is exactly mine also. I think this works really well in very
>>>>> >> > large
>>>>> >> > enterprise environments where you need to split out services over
>>>>> >> > more
>>>>> >> > than three controllers. You do need good in-house puppet skills
>>>>> >> > though
>>>>> >> > so better for enterprise with a good sysadmin team.
>>>>> >> >
>>>>> >> >> Forward a couple of releases and we had a TripleO GUI (Tuskar, I
>>>>> >> >> believe) that allowed you to graphically build your Openstack
>>>>> >> >> cloud.
>>>>> >> >> That was a reasonable good GUI for Openstack.
>>>>> >> >
>>>>> >> > Well, I found it barely usable. It was only ever good as a
>>>>> >> > graphical
>>>>> >> > representiation of what the build was doing. Interacting with it
>>>>> >> > was
>>>>> >> > not great.
>>>>> >> >
>>>>> >> >> Following that, TripleO become a script based installer, that
>>>>> >> >> required experience in Heat templates. I know I didn’t have it
>>>>> >> >> and
>>>>> >> >> had to ask in the mailing list about how to present this or
>>>>> >> >> change
>>>>> >> >> that. I got a couple of installs working with this setup.
>>>>> >> >
>>>>> >> > Works well now that I understand all the foibles and have invested
>>>>> >> > time
>>>>> >> > into understanding heat templates and puppet modules. Its good in
>>>>> >> > that
>>>>> >> > it forces you to learn about orchestration which is such an
>>>>> >> > important
>>>>> >> > end-goal of cloud environments.
>>>>> >> >
>>>>> >> >> In the last session in Austin, my goal was to obtain information
>>>>> >> >> on
>>>>> >> >> how others were installing Openstack. I was pointed to Fuel as an
>>>>> >> >> alternative. I tried it up, and it just worked. It had the
>>>>> >> >> discovering capability from The Foreman, and the configuration
>>>>> >> >> options from TripleO. I understand that is based in Ansible and
>>>>> >> >> because of that, it is not fully CentOS ready for all the nodes
>>>>> >> >> (at
>>>>> >> >> least not in version 9 that I tried). In any case, as a deployer
>>>>> >> >> and
>>>>> >> >> installer, it is the most well rounded tool that I found.
>>>>> >> >
>>>>> >> > This is interesting to know. I've heard of Fuel of course but
>>>>> >> > there are
>>>>> >> > some politics involved - it still has the team:single-vendor tag
>>>>> >> > but
>>>>> >> > from what I see Mirantis are very keen for it to become the
>>>>> >> > default
>>>>> >> > OpenStack installer. I don't think being Ansible-based should be a
>>>>> >> > problem - we are deploying OpenShift on OpenStack which uses
>>>>> >> > Openshift-
>>>>> >> > ansible - this recently moved to Ansible 2.1 without too much
>>>>> >> > disruption.
>>>>> >> >
>>>>> >> >> I’d love to see RDO moving into that direction, and having an
>>>>> >> >> easy to
>>>>> >> >> use, end user ready deployer tool.
>>>>> >> >
>>>>> >> > If its as good as you say its definitely worth evaluating. From
>>>>> >> > our
>>>>> >> > point of view, we want to be able to add services to the pacemaker
>>>>> >> > cluster with some ease - for example Magnum and Sahara - and it
>>>>> >> > looks
>>>>> >> > like there are steps being taken with regards to composable roles
>>>>> >> > and
>>>>> >> > simplification of the pacemaker cluster to just core services.
>>>>> >> >
>>>>> >> > But if someone can explain that better I would appreciate it.
>>>>> >> >
>>>>> >> > Regards
>>>>> >> >
>>>>> >> >> IB
>>>>> >> >>
>>>>> >> >>
>>>>> >> >> __
>>>>> >> >> Ignacio Bravo
>>>>> >> >> LTG Federal, Inc
>>>>> >> >> www.ltgfederal.com
>>>>> >> >>
>>>>> >> >>
>>>>> >> >> > On Aug 1, 2016, at 1:07 PM, David Moreau Simard
>>>>> >> >> > <dms redhat com>
>>>>> >> >> > wrote:
>>>>> >> >> >
>>>>> >> >> > The vast majority of RDO's CI relies on using upstream
>>>>> >> >> > installation/deployment projects in order to test installation
>>>>> >> >> > of
>>>>> >> >> > RDO
>>>>> >> >> > packages in different ways and configurations.
>>>>> >> >> >
>>>>> >> >> > Unless I'm mistaken, TripleO Quickstart was originally created
>>>>> >> >> > as a
>>>>> >> >> > mean to "easily" install TripleO in different topologies
>>>>> >> >> > without
>>>>> >> >> > requiring a massive amount of hardware.
>>>>> >> >> > This project allows us to test TripleO in virtual deployments
>>>>> >> >> > on
>>>>> >> >> > just
>>>>> >> >> > one server instead of, say, 6.
>>>>> >> >> >
>>>>> >> >> > There's also WeIRDO [1] which was left out of your list.
>>>>> >> >> > WeIRDO is super simple and simply aims to run upstream gate
>>>>> >> >> > jobs
>>>>> >> >> > (such
>>>>> >> >> > as puppet-openstack-integration [2][3] and packstack [4][5])
>>>>> >> >> > outside
>>>>> >> >> > of the gate.
>>>>> >> >> > It'll install dependencies that are expected to be there (i.e,
>>>>> >> >> > usually
>>>>> >> >> > set up by the openstack-infra gate preparation jobs), set up
>>>>> >> >> > the
>>>>> >> >> > trunk
>>>>> >> >> > repositories we're interested in testing and the rest is
>>>>> >> >> > handled by
>>>>> >> >> > the upstream project testing framework.
>>>>> >> >> >
>>>>> >> >> > The WeIRDO project is /very/ low maintenance and brings an
>>>>> >> >> > exceptional
>>>>> >> >> > amount of coverage and value.
>>>>> >> >> > This coverage is important because RDO provides OpenStack
>>>>> >> >> > packages
>>>>> >> >> > or
>>>>> >> >> > projects that are not necessarily used by TripleO and the
>>>>> >> >> > reality
>>>>> >> >> > is
>>>>> >> >> > that not everyone deploying OpenStack on CentOS with RDO will
>>>>> >> >> > be
>>>>> >> >> > using
>>>>> >> >> > TripleO.
>>>>> >> >> >
>>>>> >> >> > Anyway, sorry for sidetracking but back to the topic, thanks
>>>>> >> >> > for
>>>>> >> >> > opening the discussion.
>>>>> >> >> >
>>>>> >> >> > What honestly perplexes me is the situation of CI in RDO and
>>>>> >> >> > OSP,
>>>>> >> >> > especially around TripleO/Director, is the amount of work that
>>>>> >> >> > is
>>>>> >> >> > spent downstream.
>>>>> >> >> > And by downstream, here, I mean anything that isn't in TripleO
>>>>> >> >> > proper.
>>>>> >> >> >
>>>>> >> >> > I keep dreaming about how awesome upstream TripleO CI would be
>>>>> >> >> > if
>>>>> >> >> > all
>>>>> >> >> > that effort was spent directly there instead -- and then that
>>>>> >> >> > all
>>>>> >> >> > work
>>>>> >> >> > could bear fruit and trickle down downstream for free.
>>>>> >> >> > Exactly like how we keep improving the testing coverage in
>>>>> >> >> > puppet-openstack-integration, it's automatically pulled in RDO
>>>>> >> >> > CI
>>>>> >> >> > through WeIRDO for free.
>>>>> >> >> > We make the upstream better and we benefit from it
>>>>> >> >> > simultaneously:
>>>>> >> >> > everyone wins.
>>>>> >> >> >
>>>>> >> >> > [1]: https://github.com/rdo-infra/weirdo
>>>>> >> >> > [2]:
>>>>> >> >> > https://github.com/rdo-infra/ansible-role-weirdo-puppet-openst
>>>>> >> >> > ack
>>>>> >> >> > [3]:
>>>>> >> >> > https://github.com/openstack/puppet-openstack-integration#desc
>>>>> >> >> > ription
>>>>> >> >> > [4]: https://github.com/rdo-infra/ansible-role-weirdo-packstack
>>>>> >> >> > [5]:
>>>>> >> >> > https://github.com/openstack/packstack#packstack-integration-t
>>>>> >> >> > ests
>>>>> >> >> >
>>>>> >> >> > David Moreau Simard
>>>>> >> >> > Senior Software Engineer | Openstack RDO
>>>>> >> >> >
>>>>> >> >> > dmsimard = [irc, github, twitter]
>>>>> >> >> >
>>>>> >> >> > David Moreau Simard
>>>>> >> >> > Senior Software Engineer | Openstack RDO
>>>>> >> >> >
>>>>> >> >> > dmsimard = [irc, github, twitter]
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > On Mon, Aug 1, 2016 at 11:21 AM, Arie Bregman
>>>>> >> >> > <abregman redhat com>
>>>>> >> >> > wrote:
>>>>> >> >> > > Hi,
>>>>> >> >> > >
>>>>> >> >> > > I would like to start a discussion on the overlap between
>>>>> >> >> > > tools
>>>>> >> >> > > we
>>>>> >> >> > > have for deploying and testing TripleO (RDO & RHOSP) in CI.
>>>>> >> >> > >
>>>>> >> >> > > Several months ago, we worked on one common framework for
>>>>> >> >> > > deploying
>>>>> >> >> > > and testing OpenStack (RDO & RHOSP) in CI. I think you can
>>>>> >> >> > > say it
>>>>> >> >> > > didn't work out well, which eventually led each group to
>>>>> >> >> > > focus on
>>>>> >> >> > > developing other existing/new tools.
>>>>> >> >> > >
>>>>> >> >> > > What we have right now for deploying and testing
>>>>> >> >> > > --------------------------------------------------------
>>>>> >> >> > > === Component CI, Gating ===
>>>>> >> >> > > I'll start with the projects we created, I think that's only
>>>>> >> >> > > fair
>>>>> >> >> > > :)
>>>>> >> >> > >
>>>>> >> >> > > * Ansible-OVB[1] - Provisioning Tripleo heat stack, using the
>>>>> >> >> > > OVB
>>>>> >> >> > > project.
>>>>> >> >> > >
>>>>> >> >> > > * Ansible-RHOSP[2] - Product installation (RHOSP). Branch per
>>>>> >> >> > > release.
>>>>> >> >> > >
>>>>> >> >> > > * Octario[3] - Testing using RPMs (pep8, unit, functional,
>>>>> >> >> > > tempest,
>>>>> >> >> > > csit) + Patching RPMs with submitted code.
>>>>> >> >> > >
>>>>> >> >> > > === Automation, QE ===
>>>>> >> >> > > * InfraRed[4] - provision install and test. Pluggable and
>>>>> >> >> > > modular,
>>>>> >> >> > > allows you to create your own provisioner, installer and
>>>>> >> >> > > tester.
>>>>> >> >> > >
>>>>> >> >> > > As far as I know, the groups is working now on different
>>>>> >> >> > > structure of
>>>>> >> >> > > one main project and three sub projects (provision, install
>>>>> >> >> > > and
>>>>> >> >> > > test).
>>>>> >> >> > >
>>>>> >> >> > > === RDO ===
>>>>> >> >> > > I didn't use RDO tools, so I apologize if I got something
>>>>> >> >> > > wrong:
>>>>> >> >> > >
>>>>> >> >> > > * About ~25 micro independent Ansible roles[5]. You can
>>>>> >> >> > > either
>>>>> >> >> > > choose
>>>>> >> >> > > to use one of them or several together. They are used for
>>>>> >> >> > > provisioning, installing and testing Tripleo.
>>>>> >> >> > >
>>>>> >> >> > > * Tripleo-quickstart[6] - uses the micro roles for deploying
>>>>> >> >> > > tripleo
>>>>> >> >> > > and test it.
>>>>> >> >> > >
>>>>> >> >> > > As I said, I didn't use the tools, so feel free to add more
>>>>> >> >> > > information you think is relevant.
>>>>> >> >> > >
>>>>> >> >> > > === More? ===
>>>>> >> >> > > I hope not. Let us know if are familiar with more tools.
>>>>> >> >> > >
>>>>> >> >> > > Conclusion
>>>>> >> >> > > --------------
>>>>> >> >> > > So as you can see, there are several projects that eventually
>>>>> >> >> > > overlap
>>>>> >> >> > > in many areas. Each group is basically using the same tasks
>>>>> >> >> > > (provision
>>>>> >> >> > > resources, build/import overcloud images, run tempest,
>>>>> >> >> > > collect
>>>>> >> >> > > logs,
>>>>> >> >> > > etc.)
>>>>> >> >> > >
>>>>> >> >> > > Personally, I think it's a waste of resources. For each task
>>>>> >> >> > > there is
>>>>> >> >> > > at least two people from different groups who work on exactly
>>>>> >> >> > > the
>>>>> >> >> > > same
>>>>> >> >> > > task. The most recent example I can give is OVB. As far as I
>>>>> >> >> > > know,
>>>>> >> >> > > both groups are working on implementing it in their set of
>>>>> >> >> > > tools
>>>>> >> >> > > right
>>>>> >> >> > > now.
>>>>> >> >> > >
>>>>> >> >> > > On the other hand, you can always claim: "we already tried to
>>>>> >> >> > > work on
>>>>> >> >> > > the same framework, we failed to do it successfully" - right,
>>>>> >> >> > > but
>>>>> >> >> > > maybe with better ground rules we can manage it. We would
>>>>> >> >> > > defiantly
>>>>> >> >> > > benefit a lot from doing that.
>>>>> >> >> > >
>>>>> >> >> > > What's next?
>>>>> >> >> > > ----------------
>>>>> >> >> > > So first of all, I would like to hear from you if you think
>>>>> >> >> > > that
>>>>> >> >> > > we
>>>>> >> >> > > can collaborate once again or is it actually better to keep
>>>>> >> >> > > it as
>>>>> >> >> > > it
>>>>> >> >> > > is now.
>>>>> >> >> > >
>>>>> >> >> > > If you agree that collaboration here makes sense, maybe you
>>>>> >> >> > > have
>>>>> >> >> > > ideas
>>>>> >> >> > > on how we can do it better this time.
>>>>> >> >> > >
>>>>> >> >> > > I think that setting up a meeting to discuss the right
>>>>> >> >> > > architecture
>>>>> >> >> > > for the project(s) and decide on good review/gating process,
>>>>> >> >> > > would be
>>>>> >> >> > > a good start.
>>>>> >> >> > >
>>>>> >> >> > > Please let me know what do you think and keep in mind that
>>>>> >> >> > > this
>>>>> >> >> > > is not
>>>>> >> >> > > about which tool is better!. As you can see I didn't mention
>>>>> >> >> > > the
>>>>> >> >> > > time
>>>>> >> >> > > it takes for each tool to deploy and test, and also not the
>>>>> >> >> > > full
>>>>> >> >> > > feature list it supports.
>>>>> >> >> > > If possible, we should keep it about collaborating and not
>>>>> >> >> > > choosing
>>>>> >> >> > > the best tool. Our solution could be the combination of two
>>>>> >> >> > > or
>>>>> >> >> > > more
>>>>> >> >> > > tools eventually (tripleo-red, infra-quickstart? :D )
>>>>> >> >> > >
>>>>> >> >> > > "You may say I'm a dreamer, but I'm not the only one. I hope
>>>>> >> >> > > some
>>>>> >> >> > > day
>>>>> >> >> > > you'll join us and the infra will be as one" :)
>>>>> >> >> > >
>>>>> >> >> > > [1] https://github.com/redhat-openstack/ansible-ovb
>>>>> >> >> > > [2] https://github.com/redhat-openstack/ansible-rhosp
>>>>> >> >> > > [3] https://github.com/redhat-openstack/octario
>>>>> >> >> > > [4] https://github.com/rhosqeauto/InfraRed
>>>>> >> >> > > [5]
>>>>> >> >> > > https://github.com/redhat-openstack?utf8=%E2%9C%93&query=ansi
>>>>> >> >> > > ble-role
>>>>> >> >> > > [6] https://github.com/openstack/tripleo-quickstart
>>>>> >> >> > >
>>>>> >> >> > > _______________________________________________
>>>>> >> >> > > rdo-list mailing list
>>>>> >> >> > > rdo-list redhat com
>>>>> >> >> > > https://www.redhat.com/mailman/listinfo/rdo-list
>>>>> >> >> > >
>>>>> >> >> > > To unsubscribe: rdo-list-unsubscribe redhat com
>>>>> >> >> > _______________________________________________
>>>>> >> >> > rdo-list mailing list
>>>>> >> >> > rdo-list redhat com
>>>>> >> >> > https://www.redhat.com/mailman/listinfo/rdo-list
>>>>> >> >> >
>>>>> >> >> > To unsubscribe: rdo-list-unsubscribe redhat com
>>>>> >> >>
>>>>> >> > --
>>>>> >> > Regards,
>>>>> >> >
>>>>> >> > Christopher Brown
>>>>> >> > OpenStack Engineer
>>>>> >> > OCF plc
>>>>> >> >
>>>>> >> > Tel: +44 (0)114 257 2200
>>>>> >> > Web: www.ocf.co.uk
>>>>> >> > Blog: blog.ocf.co.uk
>>>>> >> > Twitter: @ocfplc
>>>>> >> >
>>>>> >> > Please note, any emails relating to an OCF Support request must
>>>>> >> > always
>>>>> >> > be sent to support ocf co uk for a ticket number to be generated
>>>>> >> > or
>>>>> >> > existing support ticket to be updated. Should this not be done
>>>>> >> > then OCF
>>>>> >> > cannot be held responsible for requests not dealt with in a timely
>>>>> >> > manner.
>>>>> >> >
>>>>> >> > OCF plc is a company registered in England and Wales. Registered
>>>>> >> > number
>>>>> >> > 4132533, VAT number GB 780 6803 14. Registered office address: OCF
>>>>> >> > plc,
>>>>> >> > 5 Rotunda Business Centre, Thorncliffe Park, Chapeltown, Sheffield
>>>>> >> > S35
>>>>> >> > 2PG.
>>>>> >> >
>>>>> >> > This message is private and confidential. If you have received
>>>>> >> > this
>>>>> >> > message in error, please notify us immediately and remove it from
>>>>> >> > your
>>>>> >> > system.
>>>>> >> >
>>>>> >> > _______________________________________________
>>>>> >> > rdo-list mailing list
>>>>> >> > rdo-list redhat com
>>>>> >> > https://www.redhat.com/mailman/listinfo/rdo-list
>>>>> >> >
>>>>> >> > To unsubscribe: rdo-list-unsubscribe redhat com
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Arie Bregman
>>>>> >> Red Hat Israel
>>>>> >> Component CI: https://mojo.redhat.com/groups/rhos-core-ci/overview
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Arie Bregman
>>>>> Red Hat Israel
>>>>> Component CI: https://mojo.redhat.com/groups/rhos-core-ci/overview
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Tal Kammer
>>> Associate manager, automation and infrastracture, Openstack platform.
>>> Red Hat Israel
>>> Automation group mojo: https://mojo.redhat.com/docs/DOC-1011659
>>>
>>> _______________________________________________
>>> rdo-list mailing list
>>> rdo-list redhat com
>>> https://www.redhat.com/mailman/listinfo/rdo-list
>>>
>>> To unsubscribe: rdo-list-unsubscribe redhat com
>>
>>
>>
>>
>> --
>> {Kind regards | Mit besten Grüßen},
>>
>> Frank
>>
>> ________________________________
>> Frank Zdarsky | NFV Partner Engineering | Office of Technology | Red Hat
>> e: fzdarsky redhat com | irc: fzdarsky @freenode | m: +49 175 82 11 64 4 |
>> t: +49 711 96 43 70 02
>
>
>
> _______________________________________________
> rdo-list mailing list
> rdo-list redhat com
> https://www.redhat.com/mailman/listinfo/rdo-list
>
> To unsubscribe: rdo-list-unsubscribe redhat com


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]