[Spacewalk-list] Need for a lightweight mirroring solution

Thomas von Steiger thomas.vonsteiger at bluewin.ch
Wed Jul 23 00:18:24 UTC 2008


What i think about this:

There are to many very nice repo tools.
For manage os content the best way is one way for all the different  
software framworks.

Currently for http://www.ovirt.org/ and other projects there is  
cobbler the solution.
For pulp there is a own repo tool with very nice ideas about content  
handling.
For Satelllite there is the knowlege about the os content implemented  
in:
	- Filesystem Tree with rpm name/version/arch information. There you  
can have easy more then 100Gb of Data's with one distribution over the  
years.
and 	- Meta Data's about all the rpm's and logical defined software  
channels for os/cluster/virtualisation/storage/tools....what you want.
And for Spacewalk there is currently the same concept "like the  
satellite" implemented.
To manage differend distros with the spacewalk it need's now a  
filesystem tree extention.

Better woul'd be one concept for all that differend tools.
Because Spacewalk(Satellite also...) can not manage virtual servers  
and storage like ovirt.
And ovirt can not manage systems, patching and os content like  
spacewalk and satellite.

With the satellite we have now some years production experience and  
activationkeys are very practicale to attach a system to the right  
channels.
With spacewalk and to other important new projects we shoul'd now be  
able to create the opensource tools/framework for next generation it.
And it need's more than one tool. This tools should be live hand in  
hand because there are many new synergys between available.
Then we can share system information between all the tools or import/ 
move services/servers from one framwork to a other framwork.

And for this it need's one way for repo managment for all?
What you think about this?

Thomas

On Jul 22, 2008, at 8:54 PM, Mike McCune wrote:

> Michael DeHaan wrote:
>> Covil, K (Kim) wrote:
>>>> From: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-
>>>> bounces at redhat.com] On Behalf Of Michael DeHaan
>>>> Sent: 21 July 2008 13:38
>>>> To: Jon Stanley
>>>> Cc: spacewalk-list at redhat.com
>>>> Subject: Re: [Spacewalk-list] Need for a lightweight mirroring  
>>>> solution
>>>>
>>>> Jon Stanley wrote:
>>>>
>>>>> On Mon, Jul 21, 2008 at 8:05 AM, Michael DeHaan <mdehaan at redhat.com 
>>>>> >
>>>>>
>>>> wrote:
>>>>
>>>>>> Many questions I get on cobbler's mailing list deal with using
>>>>>> yum_rhn_plugin to mirror RHEL packages.   These are typically  
>>>>>> from
>>>>>>
>>>> users who
>>>>
>>>>>> have small-mid size environments that cannot use Satellite for
>>>>>>
>>>> various
>>>>
>>>>>> reasons.
>>>>>>
>>>>>>
>>>>> That;s why we have spacewalk now :)
>>>>>
>>>>>
>>>>>
>>>> Well, no, the problem is still the same.    The small  
>>>> disconnected labs
>>>> are looking to a solution, and they need to buy Satellite in  
>>>> order to
>>>> do
>>>> mirroring, even if they only have 10 systems.
>>>>
>>>>
>>> This seems like a perfect situation for the suggestion made  
>>> earlier about allowing Spacewalk to refer to external  
>>> repositories. Use squid to only cache the packages required by the  
>>> 10 systems rather than downloading the entire RHEL/CentOS  
>>> repository, and just handle package lists in Spacewalk rather than  
>>> the packages themselves.
>>>
>
> I like the above ideas as well.  I've wanted to add on to Pulp's CLI  
> and make a stand alone tool for content synchronization.  A while  
> back I threw up some updated info in the Pulp wiki with my thoughts  
> on what I thought we might need in a pulp CLI tool:
>
> https://fedorahosted.org/pulp
>
> ""PULP ¶
>
> Pulp is an an application for managing software content. Its goals  
> are:
>
>    * The replication software repositories from any location (http,  
> file system, ISO, RHN) to a local onsite repository.
>    * Provide and easy mechanism for systems to get access to these  
> repositories and install software from them.
>    * An accounting mechanism that keeps track of what systems are  
> using what repositories
>
> At its core we envision pulp being able to provide a series of input  
> and output formats to synchronize content to and from various  
> locations.
>
> Examples would be:
>
> Mirror F9 i386 everything repository:
>
>  pulp mirror --input=http://download.fedora.redhat.com/pub/fedora/linux/releases/9/Everything/i386/os/Packages/ 
> \
>    --output=file:///var/www/repo/f9-i386
>
> Mirror Red Hat Enterprise Linux version 5 to an ISO:
>
> pulp mirror --input=rhn://rhel-i386-server-5 --output=iso://root/ 
> RHEL5-PACKAGES.ISO
>
> Mirror Fedora 9 into a Spacewalk Channel:
>
>  pulp mirror --input=http://download.fedora.redhat.com/pub/fedora/linux/releases/9/Everything/i386/os/Packages/ 
> \
>    --output=spacewalk://somespacewalk.example.com  --username=admin  
> --password=redhat
>
> Pulp will provide an easy web, web-services, and command line  
> interface for managing all of this.""
>
> That said, my personal bandwidth for implementing such a project is  
> somewhat low, although as others have said, the desire for things  
> like above is definitely there.
>
> The central problem we still have is that the only way to  
> synchronize all the current, old and various arches of RHEL to a  
> machine is by the use of satellite-sync to an RHN Satellite.
>
> If we are looking to synchronize RHEL, we will need to use the  
> satellite-sync API or work with the Red Hat Network team to provide  
> new APIs for any new tools we develop.
>
> If we want a tool to work with things *other* than RHEL then we are  
> in better shape because the content is easier to get too.
>
> With Spacewalk users are currently tasked with fetching their own  
> content and using rhnpush to get the content into a Spacewalk  
> channel. While this is workable, it isn't the most user friendly of  
> methods.   I think there still is a need for a tool for fetching  
> remote content and keeping it in sync with some form of local  
> storage (disk or spacewalk channel) which is what Dehaan and the  
> Cobbler users are clamoring for.
>
> I think we should develop a new tool like described above for  
> content retrieval and use Spacewalk channels to track usage.
>
> As for the feature to make Spacewalk Channels be proxies for remote  
> content, I think that is a good thing but doesn't help the users who  
> want all the content available locally.
>
>> That doesn't solve the disconnected lab case, but probably does  
>> solve the problem where you need to route through a system in a DMZ  
>> for updates -- probably 90% of the cases we need to solve.   I like  
>> this.
>
> Disconnected labs will need to sync all the content to some local  
> store for sure.  A CLI tool like proposed above really does help  
> with this case.
>
> The thing about using Spacewalk Proxy is that it still requires  
> either a Spacewalk server, a RHN Satellite or a connection to  
> rhn.redhat.com for it to function.  It has no notion of really what  
> a channel is or any local 'management' of the content.  It is a  
> glorified squid cache.
>
>> If we solve it generically, this could also possibly allow using  
>> Spacewalk Proxy to be a Proxy for Red Hat Network as well as just a  
>> Proxy for another Spacewalk/Satellite?
>
> Not quite following this statement ...
>
> Mike
> -- 
> Mike McCune
> mmccune AT redhat.com
> Engineering               | Portland, OR
> RHN Satellite             | 650.567.9039x79248
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list





More information about the Spacewalk-list mailing list