[libvirt PATCH 0/5] ci: Use GitLab container registry

[Daniel P. Berrangé]

On Mon, Jun 01, 2020 at 05:31:45PM +0200, Andrea Bolognani wrote:
> On Mon, 2020-06-01 at 16:51 +0200, Pavel Hrdina wrote:
> > On Fri, May 29, 2020 at 03:00:39PM +0200, Andrea Bolognani wrote:
> > > Branch: https://gitlab.com/abologna/libvirt/-/tree/ci-full-gitlab-registry
> > > Pipeline: https://gitlab.com/abologna/libvirt/pipelines/150891361
> > > 
> > > This is what we're already doing with the subprojects we've migrated
> > > to GitLab CI and, as of earlier today, all projects under the
> > > libosinfo umbrella.
> > > 
> > > Once this is merged, we can stop publishing container images on Quay
> > > and archive the libvirt-dockerfiles repository.
> > > 
> > > Patch 3/5 has been trimmed in order to comply with the size limits
> > > of the mailing list. You can grab the unabridged version with
> > > 
> > >   $ git fetch https://gitlab.com/abologna/libvirt ci-full-gitlab-registry
> > 
> > This is a lot of files and lines of text/code. I was wondering about
> > building the dockerfiles as part of the container_job_definition.
> > 
> > To me it seems like a lot of duplication and a lot of noise in the
> > future if we decide to change the dockerfiles generation. The main
> > difference that I can think of is that with files in repository we
> > need to regenerate all the dockerfiles to apply changes made in
> > libvirt-ci but with the automatic generation we would have that for
> > free.
> > 
> > Both approaches have some benefits and drawbacks and I guess we could
> > have some variable to prevent automatic generation of dockerfiles to
> > make sure that unwanted changes in libvirt-ci doesn't affect CI for
> > all libvirt repositories, on the other hand it would automatically
> > check that changes to libvirt-ci doesn't break anything.
> > 
> > I personally don't like the need to introduce 5000+ lines just for
> > compilation testing.
> 
> To prevent unwanted changes to slip in, we could make libvirt-ci
> a submodule and only bump the hash when we specifically want to
> update something.
> 
> Overall I'd be perfectly okay with the approach you suggest, though
> I reserve the right to change my mind about this after having tried
> to implement it :) Adding Dan to the conversation so that he
> can weigh in.

Submodules introduce a extra layer of pain for people whose changes
involve modifications to the dockerfile. Per my other reply, the goal
here is that someone changinging libvirt.git should be able to modify
the dockerfile and have CI jobs "just work" for their merge request.
They should only need to touch libvirt-ci.git once they have got
everything working for their libvirt.git changes & CI.

Involving libvirt-ci.git makes the update workflow something like
this:

 - Fork libvirt.git
 - Fork libvirt-ci.git
 - Make code changes that need dockerfile update in libvirt.git
 - Update forked libvirt.git to point to fork of libvirt-ci.git
 - Make lcitool related changes in libvirt-ci.git
 - Commit lcitool related changes in libvirt-ci.git
 - Update forked libvirt.git submodule hash to point to libvirt-ci.git update
 - If CI fails, repeat last three steps
 - Submit merge request for libvirt-ci.git
 - Submit merge request for libvirt.git
 - If libvirt.git change is approved, then merge libvirt-ci.git change
 - Update forked libvirt.git to point back to main libvirt-ci.git
 - Refresh merge request for libvirt.git
 - Approve and merge libvirt.git merge request

By committing dockerfiles we have a simpler life

 - Fork libvirt.git
 - Fork libvirt-ci.git
 - Make code changes that need dockerfile update in libvirt.git
 - Make lcitool related changes  in libvirt-ci.git
 - Re-generate dockerfiles with lcitool from your fork
 - If CI fails, repeat last two steps
 - Commit lcitool related changes in libvirt-ci.git
 - Submit merge request for libvirt.git
 - Submit merge request for libvirt-ci.git
 - Approve and merge libvirt.git merge request
 - Approve and merge libvirt-ci.git merge request

In the second case, the preson updating libvirt-ci.git doesn't even
have to be the same as the person who submits the libvirt.git updates,
as it can be done out of band to some extent. eg we can do this:

 - Fork libvirt.git
 - Make code changes that need dockerfile update in libvirt.git
 - Edit dockerfiles with needed changes
 - If CI fails, repeat last step
 - Submit merge request for libvirt.git
 - Approve and merge libvirt.git merge request

and

 - Fork libvirt-ci.git
 - Make lcitool related changes  in libvirt-ci.git
 - Commit lcitool related changes in libvirt-ci.git
 - Submit merge request for libvirt-ci.git
 - Approve and merge libvirt-ci.git merge request

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|