[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [rdo-list] Building overcloud images with TripleO and RDO



Hi!

First, thanks for digging into the recent issue detailed below.

I agree that:

---

We can always do more to document what processes and tooling we have, and
clearly we need to!

Open mechanisms that allow anyone to build images, see how it all works,
customize, override, debug, etc are indeed both ideal and IMHO are a
requirement.

Declarative mechanisms are an ideal base for tools that can adapt, that are a joy to use, can evolve rationally, and are robust enough in the long term to be used broadly. We have a variety of use-cases, scenarios, and users to consider.
It’s a big tent!

---

Regarding ansible-role-tripleo-image-build (artib) [1], I wanted to chime in
briefly with a few thoughts and references to help advance the conversation.
Earlier this year this was all new to me, so I thought others might be in the
same place.

---

tripleo-quickstart (oooq) [2] has the primary goal of providing an “easy button” for new users/contributors. It is designed to deploy current stable bits [3] to libvirt quickly and reliably, using declarative configuration for topology and
inputs [4]. It’s does so using the documented steps to deploy tripleo, and
encourages learning and onboarding. It can run the the generated scripts (on the undercloud) or the user can. It manages to do so in a way that that is a boon to CI and iterative development workflows. A set of building blocks [5] exist and
is growing, and is being used in our CI today.

artib packages images for use with oooq, but does not attempt to define a new build mechanism. It uses a shared upstream library (tripleo-common [6]) which invokes DIB [7] to create the images. The input to tripleo-common is declarative
YAML, and there’s a CLI interface provided as well.

Regarding the pre-built images, we’re using them not because it’s “too hard” but rather “saves time in CI.” Using the same tools anyone can create their own images, tweak them, put custom test or debug tools in them, etc. One can also simply (optionally) use one to quickly verify a bug, or experiment, or ____. We are also building images today to enable stuff like the oooq-usbkey [8] (cool
right ?!?!)


---

+1 to talking more about this, and submitting blueprints on how we can advance tooling around image building. I’m an interested party that’s new to OpenStack,
and look forward to collaborating with (all of) you.

Let’s brainstorm how we might improve both the discoverability and utility of the various image building tools, or what they might look like moving forward.
What are we missing?  What can we do/make better?

In my view this begins with listening - to understand the requirements,
constraints, and expectations that folks have for these tools. Let’s recognize up front that while converging/consolidating the toolchains might be a possible
outcome, this is OpenStack. Carrots > Sticks, and it might just happen
naturally.

I think these tools have immense potential to help the OpenStack development
process. Given the recent advances in both containers and composable roles, as well as the plethora of tools we have already, I look forward to seeing how we
can improve our collective union-set “utility belt” [9]

Matt


[1] https://github.com/redhat-openstack/ansible-role-tripleo-image-build
[2] https://github.com/openstack/tripleo-quickstart
[3] http://artifacts.ci.centos.org/rdo/images/master/delorean/stable/
[4] https://github.com/openstack/tripleo-quickstart/blob/master/doc/configuring.md [5] https://github.com/redhat-openstack?utf8=%E2%9C%93&query=ansible-role-tripleo
[6] https://github.com/openstack/tripleo-common/tree/master/tripleo_common
[7] http://docs.openstack.org/developer/diskimage-builder/
[8] https://www.rdoproject.org/tripleo/oooq-usbkey/
[9] yes...batman. He is/was self-created!


On 08/11/2016 11:36 PM, Graeme Gillies wrote:
Hi,

I spent the last day or two trying to get to the bottom of the issue
described at [1], which turned out to be because the version of galera
that is in EPEL is higher than what we have in RDO mitaka stable, and
when it attempts to get used, mariadb-galera-server fails to start.

In order to understand why epel was being pulled in, how to stop it, and
how this seemed to have slipped through CI/testing, I've been trying to
look through and understand the whole state of the image building
process across TripleO, RDO, and our CI.

Unfortunately what I've discovered hasn't been great. It looks like
there is at least 3 different paths being used to build images.
Apologies if anything below is incorrect, it's incredibly convoluted and
difficult to follow for someone who isn't intimately familiar with it
all (like myself).

1) Using "openstack overcloud image build --all", which is I assume the
method end users are supposed to be using, or at least it's the method
documented in the docs. This uses diskimagebuilder under the hood, but
the logic behind it is in python (under python-tripleoclient), with a
lot of stuff hardcoded in

2) Using tripleo.sh, which, while it looks like calls "openstack
overcloud image build", also has some of it's own logic and messes with
things like the ~/.cache/image-create/source-repositories file, which I
believe is how the issue at [1] passed CI in the first place

3) Using the ansible role ansible-role-tripleo-image-build [2] which
looks like it also uses diskimagebuilder, but through a slightly
different approach, by using an ansible library that can take an image
definition via yaml (neat!) and then all diskimagebuilder using
python-tripleo-common as an intermediary. Which is a different code path
(though the code itself looks similar) to python-tripleoclient

I feel this issue is hugely important as I believe it is one of the
biggest barriers to having more people adopt RDO/TripleO. Too often
people encounter issues with deploys that are hard to nail down because
we have no real understanding exactly how they built the images, nor as
an Operator I don't feel like I have a clear understanding of what I get
when I use different options. The bug at [1] is a classic example of
something I should never have hit.

We do have stable images available at [3] (built using method 3) however
there are a number of problems with just using them

1) I think it's perfectly reasonable for people to want to build their
own images. It's part of the Open Source philosophy, we want things to
be Open and we want to understand how things work, so we can customise,
extend, and troubleshoot ourselves. If your image building process is so
convoluted that you have to say "just use our prebuilt ones", then you
have done something wrong.

2) The images don't get updated (they current mitaka ones were built in
April)

3) There is actually nowhere on the RDO website, nor the tripleo
website, that actually references their location. So as a new user, you
have exactly zero chance of finding these images and using them.

I'm not sure what the best process is to start improving this, but it
looks like it's complicated enough and involves enough moving pieces
that a spec against tripleo might be the way to go? I am thinking the
goal would be to move towards everyone having one way, one code path,
for building images with TripleO, that could be utilised by all use
cases out there.

My thinking is the method would take image definitions in a yaml format
similar to how ansible-role-tripleo-image-build works, and we can just
ship a bunch of different yaml files for all the different image
scenarios people might want. e.g.

/usr/share/tripleo-images/centos-7-x86_64-mitaka-cbs.yaml
/usr/share/tripleo-images/centos-7-x86_64-mitaka-trunk.yaml
/usr/share/tripleo-images/centos-7-x86_64-trunk.yaml

Etc etc. you could then have a symlink called default.yaml which points
to whatever scenario you wish people to use by default, and the scenario
could be overwritten by a command line argument. Basically this is
exactly how mock [4] works, and has been proven to be a nice, clean,
easy to use workflow for people to understand. The added bonus is if
people wanted to do their own images, they could copy one of the
existing files as a template to start with.

If people feel this is worthwhile (and I hope they do), I'm interested
in understanding what the next steps would be to get this to happen.

Regards,

Graeme

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1365884
[2] https://github.com/redhat-openstack/ansible-role-tripleo-image-build
[3]
http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/mitaka/cbs/
[4] https://fedoraproject.org/wiki/Mock


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]