[Pulp-list] reset -- here are your requirements as compared to cobbler today

[Michael DeHaan]

    *   *Replication*: Replicate software repositories from any location
      (http, file system, ISO, RHN) to a local on-site repository.
    *
          o Rationale: upstreams differ in their delivery of software,
            and sometimes you may aggregate multiple upstreams into one
            stream requiring the usage of multiple delivery mechanisms.
            You should be able to manage software streams based how
            you'd like the software grouped, and not based on the
            technology chosen for delivery.



Cobbler currently does http, file system, ftp, ISO, rhn (one arch at a 
time), and rsync.  
As per James Anthill's blog post, yum is more efficient with lots of 
smaller repos so I do not believe aggregation of repos into a common 
repo is prudent.

    * *Addition*: Add additional content into a repository.



Create a drop directory.   Cobbler repo add to this directory and new 
software will come in every time you sync.  To be clear this is a level 
above yum's reposync, in that it also supports rsync whether over ssh or 
rsync protocol, and also works locally.  It also takes care of running 
createrepo for you and can support excludes files.  It also supports 
whitelists and dependency resolution through yumdownloader (--rpm-list)

    * *Removal*: Remove content from a repository.


Remove a file from the directory and reposync again.   Reposync will 
take care of it.

    * *Automation*: Policy-based automated repository mirroring/cloning.
          o Rationale: Keep your repositories in sync with upstream
            based on policy, whether it's an every-week type of sync or
            if you are only interested in syncing whitelisted packages,
            and/or not syncing blacklisted packages without manual
            intervention/approval.


This is implemented by putting cobbler reposync on cron.   If we want to 
add something to manage the crontabs that seems trivial.

    * *Access*: Provide an easy mechanism for systems to get access to
      these repositories and install software from them.


Cobbler generates a URL that presents the yum.repos.d configuration for 
the system, containing all subscribed repos.   We could make an RPM to 
do this to be pushed out with config management though it makes as much 
sense to just tell config management to wget the file.   The RPM would 
be doable, but is not as efficient.

    *


          o Automatic generation of a package clients can use to install
            the repo's .repo file and import GPG keys if necessary
    * *Tracking*: Track which systems are using which repositories.
          o Track how many systems are registered to a repository so you
            know how widely-used it is.
    *

          o Track which systems are registered to an old repository so
            you know to bug the owners to upgrade their systems :)
          o Track what repositories a particular system has been
            registered to when for auditing and debugging purposes.


It is better for systems management technology to be declarative rather 
than diagnostic.  In this case I would use puppet to push out the 
yum.repos.d configuration and manage yum.repos.d, trusting puppet to 
keep it in in shape.   This is the more commonly used method in the 
field.   Rather than "bug the owners", you just request that the package 
be installed when needed and control the update policy there.


    * *Diff*: Diff one repository against another.
          o Right now it is not easy to note the differences in content
            between two repositories. This can help debugging issues
            with systems - e.g., system A is subscribed to repo A and it
            works, system B is subscribed to repo B and it's broken.
            What are the differences in content that may be causing the
            breakage?


For this I would just run diff against the two mirrored repo trees on 
the filesystem.   If something more advanced is needed it does not seem 
difficult to add.

    * *Lock*: Lockdown of a repository so it is only available to
      authorized users.
          o This will help users stay in compliance with
            subscription/license agreements.


For this I would bring back yumgate.   This has the advantage of working 
outside other solutions though we can provide commands to make this more 
consumable if needed.



    * *Resign*: Resigning of packages in a repository.
          o RPM doesn't support multiple signatures, but you may want to
            resign packages built elsewhere with your organization's own
            key.

This is not complicated, essentially it just requires storing the key 
location and walking the directory tree at most.  I can see this easily 
being added.


Cobbler also supports lots of things not in these requirements.   The 
Wiki documentation here might be missing a few options, though see 
https://fedorahosted.org/cobbler/wiki/ManageYumRepos

I would also look at "cobbler repo add --help" and "man cobbler" and 
explore CobblerWeb.

This is all Python and is very easy to extend.

I should also add that the repo cloning/hardlinking, which is probably 
the most requested feature not in cobbler yet, is not on the 
requirements list.  I also think this would be easy to add and is one of 
the main
things you'd want to do with a repo management tool -- manage QA and 
stage repos.     This is basically just a cp and a hardlink script, 
though more complex implementations are also possible.

--Michael