[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [rdo-list] Multiple tools for deploying and testing TripleO



On 02/08/16 09:59, Pedro Sousa wrote:
> I was pretty sure I that was using cbs pre-built images and they had
> delorean repos but I'll check again tomorrow.

Please do, if they do have delorean repos we need to get that checked/fixed.

> 
> Also, is epel still needed? Because in undercloud I'm not using it.

My understanding is EPEL is still needed on the under and overcloud, but
I would appreciate someone else from the RDO/TripleO team confirming.

Regards,

Graeme

> 
> Thanks
> 
> On Tue, Aug 2, 2016 at 12:48 AM, Graeme Gillies <ggillies redhat com
> <mailto:ggillies redhat com>> wrote:
> 
>     On 02/08/16 09:40, Pedro Sousa wrote:
>     > HI Graeme,
>     >
>     > I'm more interested in following RHEL OSP approach, so  I install both
>     > Director or TripleO depending on my customers, but I'll take a close
>     > look on Kolla.
> 
>     Sure thing. To be clear here, we are very interested in containers and
>     looking at how they can currently fit into TripleO/Director. Containers
>     are an amazing technology, but come with a lots of pros and cons for
>     Operators, so we want to make sure we are considering them in a fashion
>     that makes the most sense. Kolla is a great way to use them now if you
>     are just purely interested in looking at a container based deployment.
> 
>     >
>     > I have a question (I know it's out of scope) but if you can answer me, I
>     > appreciate it. For overcloud nodes images do we still need delorean
>     > repos? Or can we just install a clean Centos image with SIG
>     > repositories? I want the images to be as stable as possible.
> 
>     For my stable RDO deployments I make sure not to use any delorean repos,
>     and I would expect that would be the same for most people.
> 
>     You have two ways of doing this. Either using the pre-built stable rdo
>     images at
> 
>     http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/mitaka/cbs/
> 
>     Which is preferred as they are what have been tested and validated
>     by us.
> 
>     If you wish to build your own overcloud images using stable pacakges
>     only, you need to do the following
> 
>     Manually patch the undercloud to build overcloud images using
>     rhos-release rpm only (which utilises the stable Mitaka repo from
>     CentOS, and nothing from RDO Trunk [delorean]). I do this by modifying
>     the file
> 
>     /usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_image.py
> 
>     At around line 467 you will see a reference to epel, I add a new line
>     after that to include the rdo_release DIB element to the build as well.
>     This typically makes the file look something like
> 
>     http://paste.openstack.org/show/508196/
> 
>     (note like 468). Then I create a directory to store my images and build
>     them specifying the mitaka version of rdo_release. I then upload these
>     images
> 
>     # mkdir ~/images
>     # cd ~/images
>     # export RDO_RELEASE=mitaka
>     # openstack overcloud image build --all
>     # openstack overcloud image upload --update-existing
> 
>     I'm not sure if someone else can shed some light on an easier way to
>     build overcloud images with rdo-release and not delorean.
> 
>     Hope this helps,
> 
>     Regards,
> 
>     Graeme
> 
>     >
>     > Thanks
>     >
>     >
>     >
>     >
>     >
>     >
>     >
>     > On Tue, Aug 2, 2016 at 12:27 AM, Graeme Gillies <ggillies redhat com <mailto:ggillies redhat com>
>     > <mailto:ggillies redhat com <mailto:ggillies redhat com>>> wrote:
>     >
>     >     On 02/08/16 08:01, Pedro Sousa wrote:
>     >     > My 2 cents here as an operator/integrator, since I've been
>     using the
>     >     > CentOS SIG repositories (mitaka) and following the RHEL Oficial
>     >     > Documentation, I've managed to install several baremetal
>     tripleo based
>     >     > clouds with success. I've not tried tripleo quickstart,
>     >     >
>     >     > I've also tried Fuel in the past and it works pretty well
>     with the
>     >     > plugin architecture and the network validation among other
>     things, but
>     >     > still I prefer tripleo, it gives me more flexibility to
>     setup the
>     >     > network the way I want it, and using ironic to provision the
>     baremetal
>     >     > hosts is pretty cool too. Also personally I prefer to use
>     Centos than
>     >     > Ubuntu as O.S base system, I find it more stable.
>     >     >
>     >     > Still tripleo lacks the ease of installation that Fuel has,
>     and an UI
>     >     > would be great. Also, I'm not sure that using heat templates
>     is the best
>     >     > approach, specially when someone makes a mistake editing the
>     yaml files
>     >     > and stack returns an error. This could happen when you try
>     to update the
>     >     > overcloud nodes, scaling the compute nodes for example. It's
>     not easy to
>     >     > revert the heat stack when you make a mistake.
>     >     >
>     >     > There's a lot of room to improve, specially in terms of
>     complexity of
>     >     > installation and update. Maybe containers (kolla) could be a
>     good
>     >     > approach in the future?
>     >
>     >     Hi Pedro,
>     >
>     >     As an Operator and long time user of TripleO, I sympathise
>     with you that
>     >     the combination of heat templates and puppet are difficult to
>     learn and
>     >     don't have the mature tooling to help you understand and test how
>     >     changes to the code will reflect in the real environment.
>     >
>     >     One thing I will point out is that if you do a stack update
>     that fails,
>     >     more often than not it's not the end of the world. If you go
>     on your
>     >     controller plane and make sure pacemaker and all services are
>     up and
>     >     running, the state of the stack in heat on the undercloud
>     doesn't really
>     >     matter as much.
>     >
>     >     This way we always try to "fail forward". If we do a bad stack
>     update,
>     >     we make sure the environment is stable again, and then push a
>     new stack
>     >     update with the fixes.
>     >
>     >     Having a staging or test environment you can utilise to
>     perform changes
>     >     on first in order to identify these problems is also
>     beneficial. We try
>     >     and get all our Operators to have a virtual tripleo setup on a
>     desktop
>     >     for them to "develop" changes on, as well as a shared staging
>     >     environments to do final testing of any proposed change before
>     rolling
>     >     into production.
>     >
>     >     If you are interested in understanding our full
>     development/rollout
>     >     process I can go into more detail.
>     >
>     >     Also kolla already supports centos/RDO, so you can head over
>     to the
>     >     kolla project and follow their documentation if you are
>     interested in
>     >     giving it a go. You are able to use Centos and RDO with
>     containers right
>     >     now, no need to wait for anything in the future.
>     >
>     >     Regards,
>     >
>     >     Graeme
>     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > On Mon, Aug 1, 2016 at 9:45 PM, Mohammed Arafa
>     <mohammed arafa gmail com <mailto:mohammed arafa gmail com>
>     <mailto:mohammed arafa gmail com <mailto:mohammed arafa gmail com>>
>     >     > <mailto:mohammed arafa gmail com <mailto:mohammed arafa gmail com>
>     <mailto:mohammed arafa gmail com
>     <mailto:mohammed arafa gmail com>>>> wrote:
>     >     >
>     >     >     I too am an end user and have a similar story. I had tried packstack
>     >     >     all in one but when it was time to deploy to actual servers I looked
>     >     >     to Ubuntu Maas. It was buggy so after a month or so of several
>     >     >     attempts I went to RDO. And was happy when I had my environment up.
>     >     >     But it was not reproducible. I spent months trying. And finally I
>     >     >     looked elsewhere and was told fuel.
>     >     >     With fuel I have ha and ceph and live migration with in 2 hours. And
>     >     >     repeatable too
>     >     >
>     >     >     And yes. When tripleo quick start showed up. I did not even look at
>     >     >     it. Information overload? Too much time spent evaluating and too
>     >     >     little building something productive? And now I hear of even more.
>     >     >
>     >     >     In honesty with the rename of RDO to triple o is there any need for
>     >     >     an installer?
>     >     >
>     >     >     /outburst over
>     >     >
>     >     >
>     >     >     On Aug 1, 2016 2:01 PM, "Ignacio Bravo" <ibravo ltgfederal com <mailto:ibravo ltgfederal com>
>     <mailto:ibravo ltgfederal com <mailto:ibravo ltgfederal com>>
>     >     >     <mailto:ibravo ltgfederal com
>     <mailto:ibravo ltgfederal com> <mailto:ibravo ltgfederal com
>     <mailto:ibravo ltgfederal com>>>>
>     >     wrote:
>     >     >
>     >     >         If we are talking about tools, I would also want to add
>     >     >         something with regards to user interface of these tools.
>     >     This is
>     >     >         based on my own experience:
>     >     >
>     >     >         I started trying to deploy Openstack with Staypuft
>     and The
>     >     >         Foreman. The UI of The Foreman was intuitive enough
>     for the
>     >     >         discovery and provisioning of the servers. The OpenStack
>     >     >         portion, not so much.
>     >     >
>     >     >         Forward a couple of releases and we had a TripleO GUI
>     >     (Tuskar, I
>     >     >         believe) that allowed you to graphically build your
>     Openstack
>     >     >         cloud. That was a reasonable good GUI for Openstack.
>     >     >
>     >     >         Following that, TripleO become a script based
>     installer, that
>     >     >         required experience in Heat templates. I know I
>     didn’t have it
>     >     >         and had to ask in the mailing list about how to present
>     >     this or
>     >     >         change that. I got a couple of installs working with
>     this
>     >     setup.
>     >     >
>     >     >         In the last session in Austin, my goal was to obtain
>     >     information
>     >     >         on how others were installing Openstack. I was
>     pointed to Fuel
>     >     >         as an alternative. I tried it up, and it just worked. It
>     >     had the
>     >     >         discovering capability from The Foreman, and the
>     configuration
>     >     >         options from TripleO. I understand that is based in
>     >     Ansible and
>     >     >         because of that, it is not fully CentOS ready for
>     all the
>     >     nodes
>     >     >         (at least not in version 9 that I tried). In any
>     case, as a
>     >     >         deployer and installer, it is the most well rounded tool
>     >     that I
>     >     >         found.
>     >     >
>     >     >         I’d love to see RDO moving into that direction, and
>     having an
>     >     >         easy to use, end user ready deployer tool.
>     >     >
>     >     >         IB
>     >     >
>     >     >
>     >     >         __
>     >     >         Ignacio Bravo
>     >     >         LTG Federal, Inc
>     >     >         www.ltgfederal.com <http://www.ltgfederal.com>
>     <http://www.ltgfederal.com>
>     >     <http://www.ltgfederal.com>
>     >     >
>     >     >
>     >     >>         On Aug 1, 2016, at 1:07 PM, David Moreau Simard
>     >     >>         <dms redhat com <mailto:dms redhat com>
>     <mailto:dms redhat com <mailto:dms redhat com>>
>     >     <mailto:dms redhat com <mailto:dms redhat com>
>     <mailto:dms redhat com <mailto:dms redhat com>>>> wrote:
>     >     >>
>     >     >>         The vast majority of RDO's CI relies on using upstream
>     >     >>         installation/deployment projects in order to test
>     >     installation
>     >     >>         of RDO
>     >     >>         packages in different ways and configurations.
>     >     >>
>     >     >>         Unless I'm mistaken, TripleO Quickstart was originally
>     >     created
>     >     >>         as a
>     >     >>         mean to "easily" install TripleO in different
>     topologies
>     >     without
>     >     >>         requiring a massive amount of hardware.
>     >     >>         This project allows us to test TripleO in virtual
>     deployments
>     >     >>         on just
>     >     >>         one server instead of, say, 6.
>     >     >>
>     >     >>         There's also WeIRDO [1] which was left out of your
>     list.
>     >     >>         WeIRDO is super simple and simply aims to run
>     upstream gate
>     >     >>         jobs (such
>     >     >>         as puppet-openstack-integration [2][3] and
>     packstack [4][5])
>     >     >>         outside
>     >     >>         of the gate.
>     >     >>         It'll install dependencies that are expected to be
>     there
>     >     (i.e,
>     >     >>         usually
>     >     >>         set up by the openstack-infra gate preparation
>     jobs), set up
>     >     >>         the trunk
>     >     >>         repositories we're interested in testing and the
>     rest is
>     >     >>         handled by
>     >     >>         the upstream project testing framework.
>     >     >>
>     >     >>         The WeIRDO project is /very/ low maintenance and
>     brings an
>     >     >>         exceptional
>     >     >>         amount of coverage and value.
>     >     >>         This coverage is important because RDO provides
>     OpenStack
>     >     >>         packages or
>     >     >>         projects that are not necessarily used by TripleO
>     and the
>     >     >>         reality is
>     >     >>         that not everyone deploying OpenStack on CentOS
>     with RDO will
>     >     >>         be using
>     >     >>         TripleO.
>     >     >>
>     >     >>         Anyway, sorry for sidetracking but back to the topic,
>     >     thanks for
>     >     >>         opening the discussion.
>     >     >>
>     >     >>         What honestly perplexes me is the situation of CI
>     in RDO
>     >     and OSP,
>     >     >>         especially around TripleO/Director, is the amount
>     of work
>     >     that is
>     >     >>         spent downstream.
>     >     >>         And by downstream, here, I mean anything that isn't in
>     >     TripleO
>     >     >>         proper.
>     >     >>
>     >     >>         I keep dreaming about how awesome upstream TripleO CI
>     >     would be
>     >     >>         if all
>     >     >>         that effort was spent directly there instead -- and
>     then that
>     >     >>         all work
>     >     >>         could bear fruit and trickle down downstream for free.
>     >     >>         Exactly like how we keep improving the testing
>     coverage in
>     >     >>         puppet-openstack-integration, it's automatically pulled
>     >     in RDO CI
>     >     >>         through WeIRDO for free.
>     >     >>         We make the upstream better and we benefit from it
>     >     simultaneously:
>     >     >>         everyone wins.
>     >     >>
>     >     >>         [1]: https://github.com/rdo-infra/weirdo
>     >     >>         [2]:
>     >     >>
>     >      https://github.com/rdo-infra/ansible-role-weirdo-puppet-openstack
>     >     >>         [3]:
>     >     >>
>     >     
>     https://github.com/openstack/puppet-openstack-integration#description
>     >     >>         [4]:
>     >     https://github.com/rdo-infra/ansible-role-weirdo-packstack
>     >     >>         [5]:
>     >     >>
>     >     
>     https://github.com/openstack/packstack#packstack-integration-tests
>     >     >>
>     >     >>         David Moreau Simard
>     >     >>         Senior Software Engineer | Openstack RDO
>     >     >>
>     >     >>         dmsimard = [irc, github, twitter]
>     >     >>
>     >     >>         David Moreau Simard
>     >     >>         Senior Software Engineer | Openstack RDO
>     >     >>
>     >     >>         dmsimard = [irc, github, twitter]
>     >     >>
>     >     >>
>     >     >>         On Mon, Aug 1, 2016 at 11:21 AM, Arie Bregman
>     >     >>         <abregman redhat com <mailto:abregman redhat com>
>     <mailto:abregman redhat com <mailto:abregman redhat com>>
>     >     <mailto:abregman redhat com <mailto:abregman redhat com>
>     <mailto:abregman redhat com <mailto:abregman redhat com>>>> wrote:
>     >     >>>         Hi,
>     >     >>>
>     >     >>>         I would like to start a discussion on the overlap
>     between
>     >     >>>         tools we
>     >     >>>         have for deploying and testing TripleO (RDO &
>     RHOSP) in CI.
>     >     >>>
>     >     >>>         Several months ago, we worked on one common
>     framework for
>     >     >>>         deploying
>     >     >>>         and testing OpenStack (RDO & RHOSP) in CI. I think you
>     >     can say it
>     >     >>>         didn't work out well, which eventually led each
>     group to
>     >     focus on
>     >     >>>         developing other existing/new tools.
>     >     >>>
>     >     >>>         What we have right now for deploying and testing
>     >     >>>       
>      --------------------------------------------------------
>     >     >>>         === Component CI, Gating ===
>     >     >>>         I'll start with the projects we created, I think
>     that's only
>     >     >>>         fair :)
>     >     >>>
>     >     >>>         * Ansible-OVB[1] - Provisioning Tripleo heat stack,
>     >     using the
>     >     >>>         OVB project.
>     >     >>>
>     >     >>>         * Ansible-RHOSP[2] - Product installation (RHOSP).
>     >     Branch per
>     >     >>>         release.
>     >     >>>
>     >     >>>         * Octario[3] - Testing using RPMs (pep8, unit,
>     functional,
>     >     >>>         tempest,
>     >     >>>         csit) + Patching RPMs with submitted code.
>     >     >>>
>     >     >>>         === Automation, QE ===
>     >     >>>         * InfraRed[4] - provision install and test.
>     Pluggable and
>     >     >>>         modular,
>     >     >>>         allows you to create your own provisioner,
>     installer and
>     >     tester.
>     >     >>>
>     >     >>>         As far as I know, the groups is working now on
>     different
>     >     >>>         structure of
>     >     >>>         one main project and three sub projects
>     (provision, install
>     >     >>>         and test).
>     >     >>>
>     >     >>>         === RDO ===
>     >     >>>         I didn't use RDO tools, so I apologize if I got
>     >     something wrong:
>     >     >>>
>     >     >>>         * About ~25 micro independent Ansible roles[5].
>     You can
>     >     >>>         either choose
>     >     >>>         to use one of them or several together. They are
>     used for
>     >     >>>         provisioning, installing and testing Tripleo.
>     >     >>>
>     >     >>>         * Tripleo-quickstart[6] - uses the micro roles for
>     deploying
>     >     >>>         tripleo
>     >     >>>         and test it.
>     >     >>>
>     >     >>>         As I said, I didn't use the tools, so feel free to
>     add more
>     >     >>>         information you think is relevant.
>     >     >>>
>     >     >>>         === More? ===
>     >     >>>         I hope not. Let us know if are familiar with more
>     tools.
>     >     >>>
>     >     >>>         Conclusion
>     >     >>>         --------------
>     >     >>>         So as you can see, there are several projects that
>     >     eventually
>     >     >>>         overlap
>     >     >>>         in many areas. Each group is basically using the
>     same tasks
>     >     >>>         (provision
>     >     >>>         resources, build/import overcloud images, run tempest,
>     >     >>>         collect logs,
>     >     >>>         etc.)
>     >     >>>
>     >     >>>         Personally, I think it's a waste of resources. For
>     each task
>     >     >>>         there is
>     >     >>>         at least two people from different groups who work on
>     >     exactly
>     >     >>>         the same
>     >     >>>         task. The most recent example I can give is OVB.
>     As far as I
>     >     >>>         know,
>     >     >>>         both groups are working on implementing it in
>     their set of
>     >     >>>         tools right
>     >     >>>         now.
>     >     >>>
>     >     >>>         On the other hand, you can always claim: "we already
>     >     tried to
>     >     >>>         work on
>     >     >>>         the same framework, we failed to do it successfully" -
>     >     right, but
>     >     >>>         maybe with better ground rules we can manage it.
>     We would
>     >     >>>         defiantly
>     >     >>>         benefit a lot from doing that.
>     >     >>>
>     >     >>>         What's next?
>     >     >>>         ----------------
>     >     >>>         So first of all, I would like to hear from you if
>     you think
>     >     >>>         that we
>     >     >>>         can collaborate once again or is it actually
>     better to keep
>     >     >>>         it as it
>     >     >>>         is now.
>     >     >>>
>     >     >>>         If you agree that collaboration here makes sense,
>     maybe you
>     >     >>>         have ideas
>     >     >>>         on how we can do it better this time.
>     >     >>>
>     >     >>>         I think that setting up a meeting to discuss the right
>     >     >>>         architecture
>     >     >>>         for the project(s) and decide on good
>     review/gating process,
>     >     >>>         would be
>     >     >>>         a good start.
>     >     >>>
>     >     >>>         Please let me know what do you think and keep in
>     mind that
>     >     >>>         this is not
>     >     >>>         about which tool is better!. As you can see I
>     didn't mention
>     >     >>>         the time
>     >     >>>         it takes for each tool to deploy and test, and
>     also not
>     >     the full
>     >     >>>         feature list it supports.
>     >     >>>         If possible, we should keep it about collaborating
>     and not
>     >     >>>         choosing
>     >     >>>         the best tool. Our solution could be the
>     combination of two
>     >     >>>         or more
>     >     >>>         tools eventually (tripleo-red, infra-quickstart? :D )
>     >     >>>
>     >     >>>         "You may say I'm a dreamer, but I'm not the only
>     one. I hope
>     >     >>>         some day
>     >     >>>         you'll join us and the infra will be as one" :)
>     >     >>>
>     >     >>>         [1] https://github.com/redhat-openstack/ansible-ovb
>     >     >>>         [2] https://github.com/redhat-openstack/ansible-rhosp
>     >     >>>         [3] https://github.com/redhat-openstack/octario
>     >     >>>         [4] https://github.com/rhosqeauto/InfraRed
>     >     >>>         [5]
>     >     >>>
>     >     
>     https://github.com/redhat-openstack?utf8=%E2%9C%93&query=ansible-role
>     >     >>>         [6] https://github.com/openstack/tripleo-quickstart
>     >     >>>
>     >     >>>         _______________________________________________
>     >     >>>         rdo-list mailing list
>     >     >>>         rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>
>     >     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>>
>     >     >>>         https://www.redhat.com/mailman/listinfo/rdo-list
>     >     >>>
>     >     >>>         To unsubscribe: rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>
>     >     >>>         <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     >     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>>
>     >     >>
>     >     >>         _______________________________________________
>     >     >>         rdo-list mailing list
>     >     >>         rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>
>     >     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>>
>     >     >>         https://www.redhat.com/mailman/listinfo/rdo-list
>     >     >>
>     >     >>         To unsubscribe: rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>
>     >     >>         <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     >     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>>
>     >     >
>     >     >
>     >     >         _______________________________________________
>     >     >         rdo-list mailing list
>     >     >         rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>
>     >     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>>
>     >     >         https://www.redhat.com/mailman/listinfo/rdo-list
>     >     >
>     >     >         To unsubscribe: rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>
>     >     >         <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     >     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>>
>     >     >
>     >     >
>     >     >     _______________________________________________
>     >     >     rdo-list mailing list
>     >     >     rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>
>     >     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>>
>     >     >     https://www.redhat.com/mailman/listinfo/rdo-list
>     >     >
>     >     >     To unsubscribe: rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>
>     >     >     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     >     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>>
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > _______________________________________________
>     >     > rdo-list mailing list
>     >     > rdo-list redhat com <mailto:rdo-list redhat com>
>     <mailto:rdo-list redhat com <mailto:rdo-list redhat com>>
>     >     > https://www.redhat.com/mailman/listinfo/rdo-list
>     >     >
>     >     > To unsubscribe: rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>
>     <mailto:rdo-list-unsubscribe redhat com
>     <mailto:rdo-list-unsubscribe redhat com>>
>     >     >
>     >
>     >
>     >     --
>     >     Graeme Gillies
>     >     Principal Systems Administrator
>     >     Openstack Infrastructure
>     >     Red Hat Australia
>     >
>     >
> 
> 
>     --
>     Graeme Gillies
>     Principal Systems Administrator
>     Openstack Infrastructure
>     Red Hat Australia
> 
> 


-- 
Graeme Gillies
Principal Systems Administrator
Openstack Infrastructure
Red Hat Australia


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]