Changing the layout of repodata to simplify making local yum repo.

[David Timms]

I have searched the archives in relation to this but only found Jesse's:
"Changing the way that Development lands", so if this has been done
before kindly point me to some better google terms.

What I would like to see changed is a common structure between online
and CD/DVD layouts and the downloaded yum cache eg currently:

online: {effectively the top level of the dvd iso}
<http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os>
Fedora/
Fedora/RPMS
repodata/
repodata/other.xml.gz
repodata/filelists.xml.gz
repodata/primary.xml.gz
repodata/comps.xml

yum cache:
eg /var/cache/yum/core
repomd.xml
primary.xml.gz
comps.xml
{cachecookie}
{primary.xml.gz.sqlite}
headers/
packages/ {RPMS}

Now, if the layouts were consistent:
machine1:
- yum.conf: keepcache=1
- yum update
- ftp/http serv the yum cache folder.
machine2:
- set yum.repos.d/fedora-core.repo baseurl to machine1/core etc.
- yum update
would just work. No createrepo etc would be required.

So if machine1 has a superset of packages required by all other internal
machines, the fact that machine 1 is up2date means it becomes very easy
and external bandwidth efficient to get the rest of machines up2date.

This is a hindrance for updates repo, because it's repodata is stored
within the <basearch>/repodata on the repo server:
<http://download.fedora.redhat.com/pub/fedora/linux/core/updates/5/i386/>
{all the RPMS at this level}
repodata/
repodata/other.xml.gz
repodata/filelists.xml.gz
repodata/primary.xml.gz
repodata/comps.xml

So if you point fedora-updates.repo @ another machines
cache/yum/updates, it wont find repodata/repomd.xml
This can be worked around by creating appropriate sym linking; it could
however just work with some tweaking to the structure.

Also, while you can point fedora-core.repo at another machine with a
mounted dvd iso, the structure yum creates on disk differs from the dvd.
This structure is then not immediately sharable to further machines.

core dvd primary.xml:
...
   <location href="Fedora/RPMS/binutils-2.16.91.0.6-4.i386.rpm"/>
...
-----
It would seem that a preferred layout could be created- eg {i386/}
repodata/
packages/
headers/

This could be advantageous in other ways eg:
1. the repodata folder is not inside the packages/RPMS dir, hence
machines trying to ftp repodata/* wont need to cause a complete folder
contents traversal of say thousands of rpms.
2. rsync or other mirroring of the structure could be done, and the
result is immediately usable by yum.
3. mock building easier since a local {partial} mirror easier to setup
  (just copy the downloads that mock does first time, ht/f tp serve it,
config mock-yum to retrieve from above. and keep it up2date with what
further mock runs retrieve).

Disadvantages:
1. other repos would need to catch up for the yum changes.
2. Is it of limited usefulness ?

I can see that at least various bits would need to change:
1. yum to store it's cached repodata in a folder one level down called
repodata, ie changes to yum itself
2. layout of each repo changed and rebuild the filelist.xml to suit
3. mirroring {as Jesse worked out for the other layout changes} needs
some tricks.

What problems/etc can people see with this proposal ? Is changing yum in 
this way a bad idea ?

DaveT.