[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] Questions about "distributions" and "yum repo metadata files"



Hi Christina,  Pulp 2.2 & earlier used a fixed name "Packages" for the package directory.  In Pulp 2.3 we added support for the "packagedir" field in the treeinfo file.  The files are linked in both locations specifically because of the needs of the treeinfo file/kickstart.  The original bug can be found at https://bugzilla.redhat.com/show_bug.cgi?id=1021672

-Barnaby


----- Original Message -----
From: "Christina Plummer" <cplummer gmail com>
To: "Dennis Gregorovic" <dgregor redhat com>
Cc: pulp-list redhat com
Sent: Tuesday, December 17, 2013 4:40:29 PM
Subject: Re: [Pulp-list] Questions about "distributions" and "yum repo metadata files"

Thanks, Dennis, for the clarification. I've now setup repos in Pulp pointing to the 2 new kickstart feeds, with --relative-url set to "ks/rhel5/x86_64" and "ks/rhel6/x86_64". The repos synced just fine and I'm now showing both the 5.10 and 6.5 distributions. 

However, when I go to kickstart a 5.10 server, it is failing with errors about not being able to download the packages from http://my-pulp-server/repos/ks/rhel5/x86_64//Server/ <specific-rpm> 

This makes sense, because there are no RPMs under Server. The published Pulp repo tree looks like this: 
ks/rhel5/x86_64/ 
images/ 
pxeboot/ 
Packages -> /var/lib/pulp/working/repos/live-rhel-5-x86_64-ks/distributors/yum_distributor 
repodata/ 
Server/ 
repodata/ 
<links to files under /var/lib/pulp/content/distribution/ks-Red Hat Enterprise Linux Server-5.10-x86_64/Server/repodata/> 
<some-rpm-1> 
<some-rpm-2> 
<...> 


All the RPMs are accessible either under the ks/rhel5/x86_64/ top level, or under ks/rhel5/x86_64/Packages (why are they in 2 locations, anyway?) - but not under Server. So the distribution Server/repodata files are wrong. 

Not sure if this is an upstream problem or a Pulp problem. FWIW, the primary.xml.gz under the 5.9 distribution seems to point to location href="../Packages/<specific-rpm>" for all the packages, while the 5.10 distribution just points to location href="<specific-rpm>". 

Any thoughts? 
Christina 


On Mon, Dec 16, 2013 at 2:00 PM, Dennis Gregorovic < dgregor redhat com > wrote: 


Hi, 

This is actually intentional. As of 6.5 we have moved the kickstart 
content into a separate repo. Having the installation files intermixed 
with errata and update RPMs all under os/ was not a clean model. 
Apologies for not announcing this to pulp-list before; I didn't realize 
that folks were actually syncing the "distributions" from 
cdn.redhat.com . 

So, the path that you now want to use is: 

https://cdn.redhat.com/content/dist/rhel/server/6/6.5/x86_64/kickstart/ 
https://cdn.redhat.com/content/dist/rhel/server/5/5.10/x86_64/kickstart/ 

The kickstart repos are there for all variants, arches, and minor 
releases of RHEL back to 5.7 and 6.1. 

Cheers 
-- Dennis 

On Mon, 2013-12-16 at 18:27 +0000, Baird, Josh wrote: 
> Arnold, 
> 
> Do you have BZ id for this issue? I would like to follow this case as well. There may be a chance some of the RH guys here can get that escalated. :) 
> 
> Thanks, 
> 
> Josh 
> 
> -----Original Message----- 
> From: pulp-list-bounces redhat com [mailto: pulp-list-bounces redhat com ] On Behalf Of Arnold Bechtoldt 
> Sent: Monday, December 16, 2013 1:26 PM 
> To: pulp-list redhat com 
> Subject: Re: [Pulp-list] Questions about "distributions" and "yum repo metadata files" 
> 
> Hi Christina, 
> 
> We do have a similar issue on RHEL 6.5 which RHEL 6/ 6Server should also point to. 
> 
> The file https://cdn.redhat.com/.../x86_64/treeinfo is present on RHEL 
> 6.4 but on RHEL 6.5 (aka 6) it's missing. As you may already know this file contains some special data like the path and metadata to the images/ folder which itself is necessary to kickstart. 
> 
> We already have reported it to Red Hat but they're still debugging the problem for days... 
> 
> > The RHEL6 repo didn't have any errors on the sync but still no 
> > distribution - is showed 0/0 items for Tree Files. 
> 
> 
> Yours sincerely, 
> 
> Arnold 
> 
> -- 
> Arnold Bechtoldt 
> Magdeburger Str. 29 
> 76139 Karlsruhe 
> Germany 
> 
> Am 16.12.13 17:12, schrieb Christina Plummer: 
> > Hi Barnaby, 
> > 
> > Thanks a lot for your reply - I got swamped last week and didn't get 
> > an opportunity to circle back on this until today, so I apologize for 
> > the delay. 
> > 1. I am still trying to understand what a "distribution" is. It 
> > sounds like what defines a distribution is the existence of an 
> > upstream treeinfo file - is that right? Why would a repository 
> > created from install media not also be a distribution? Is there just 
> > no method for importing boot images at this point? 
> > 2. When you say that distributions are required for kickstart - is 
> > that just if you need to use PXE to boot? If you were booting in some 
> > other way, I can't think why the treeinfo, etc. would be required for install. 
> > 3. Regarding the "Yum Metadata Files" - so those would only exist if 
> > the upstream repositority had a repomd/ directory - not a repodata/ ? 
> > When I create a new repo and publish it, there is (of course) a 
> > repodata/ created - what is the difference between this and "Yum Metadata Files" 
> > as listed by Pulp? 
> > 4. I don't see any treeinfo in the Oracle upstream repository, so that 
> > makes sense that I don't have a distribution there. 
> > 5. I can't easily browse the Red Hat repositories, due to the 
> > certificates required - I would expect that the 5.10 and 6.5 
> > distributions would be there now, but when I sync I am not getting 
> > them. It also looks like I had 5.9 and 6.4 at one time (based on the 
> > files under /var/lib/pulp/content/distributions), but as you can see 
> > from my repolist output below, the 6.4 distribution is not being 
> > listed any more. 
> > 
> > I made a copy of everything (including the distribution) from my RHEL6 
> > repo before I synced it on 12/2 (the last time it was synced before 
> > that was probably September). The copied repo still shows the 6.4 
> > distribution present. But the one that is fed from RH no longer has 
> > that one - and 6.5 is not there either. 
> > 
> > Slightly differently, my RHEL5 repo seems to contain the 5.9 
> > distribution, but 5.10 hasn't shown up. My feed URLs are: 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os 
> > https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os 
> > 
> > I did get some 404 errors when I tried to sync RHEL5 again this 
> > morning on these files, though it showed "Tree Files: 6/6 items": 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Se 
> > rver/repodata/908500781155e417f50c845157ee922d22008961-primary.sqlite. 
> > bz2 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Se 
> > rver/repodata/4ec225f636e39c137f8330c4ac2618f0020e85d6-other.xml.gz 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Se 
> > rver/repodata/ 53939e47b5c6e5e960f18fc7c209d60ce37ccc01-other.sqlite.bz 
> > 2 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Se 
> > rver/repodata/d984304da8889ba9e906d8a0455418f4b7edc206-primary.xml.gz 
> > https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/Se 
> > rver/repodata/101ecad62a146f3ab4e06ca16104cdb90b225a48-filelists.xml.g 
> > z 
> > 
> > The RHEL6 repo didn't have any errors on the sync but still no 
> > distribution - is showed 0/0 items for Tree Files. 
> > 
> > Please let me know if I can provide more information that would be helpful. 
> > 
> > Thanks, 
> > Christina 
> > 
> > On Wed, Dec 11, 2013 at 2:04 PM, Barnaby Court < bcourt redhat com 
> > <mailto: bcourt redhat com >> wrote: 
> > 
> > Hi Chris, 
> > 
> > You have a few questions in here so I'll try to hit all of them. V2 
> > definitely supports distributions. Distributions are required for 
> > kickstart. I'm not sure why you would be seeing the listing for a 
> > distribution type if there is no treeinfo file in the upstream 
> > repository. If you have a treeinfo file in the upstream repository 
> > you should have a distribution created in your repository in pulp. 
> > Copying the distribution from one repo to another should work. 
> > Creating a distribution by hand and uploading to a repository in 
> > pulp is not currently supported. However, if the upstream 
> > repository has a distribution it should be synced automatically and 
> > can then be copied from one pulp-repo to another. 
> > 
> > API information about distributions can be found at: 
> > 
> > http://pulp-rpm-dev-guide.readthedocs.org/en/latest/yum-plugins.html#d 
> > istribution 
> > 
> > The yum repo metadata files (generally) are the files in the 
> > /repomd/ subdirectory of a repository. They contain information 
> > about the repository ant its contents. 
> > 
> > My primary concern would be if you are syncing a repo that has a 
> > treeinfo or .treeinfo file upstream and you are not getting a 
> > distribution in your pulp repo. RHEL 6.4 absolutely does have a 
> > treeinfo included in the CDN and we use it for testing regularly. 
> > If you are running into trouble there I definitely want to know 
> > about it. 
> > 
> > Regards, 
> > 
> > Barnaby 
> > 
> > ----- Original Message ----- 
> > From: "Christina Plummer" < cplummer gmail com 
> > <mailto: cplummer gmail com >> 
> > To: pulp-list redhat com <mailto: pulp-list redhat com > 
> > Sent: Wednesday, December 11, 2013 10:47:07 AM 
> > Subject: [Pulp-list] Questions about "distributions" and "yum repo 
> > metadata files" 
> > 
> > Hello all, 
> > 
> > I am using Pulp 2.1.3. I've created a few feeds synced from vendor 
> > repos (Oracle and Red Hat), and then have been creating clones 
> > (copies) of those repos to use as a stable "patch set" for each 
> > release by copying each of the content types. I've noticed that the 
> > repos synced from Red Hat display content types of "Distribution" 
> > and "Yum Repo Metadata File", while the ones from Oracle do not. 
> > Also, I've noticed that the RHEL5 repo actually contains a 
> > Distribution, while the RHEL6 repo shows the content type but with a 
> > count of 0: 
> > 
> > Id: live-rhel-6-x86_64-os 
> > Display Name: live-rhel-6-x86_64-os 
> > Description: None 
> > Content Unit Counts: 
> > Distribution: 0 
> > Erratum: 2228 
> > Package Category: 10 
> > Package Group: 201 
> > Rpm: 3788 
> > Yum Repo Metadata File: 1 
> > 
> > I found these files: 
> > /var/lib/pulp/content/distribution/ks-Red Hat Enterprise Linux 
> > Server-5.9-x86_64/treeinfo 
> > /var/lib/pulp/content/distribution/ks-Red Hat Enterprise 
> > Linux-Server-6.4-x86_64/treeinfo 
> > 
> > I found this documentation from Pulp v1 discussing Distributions: 
> > http://www.pulpproject.org/ug/UGDistributions.html 
> > https://fedorahosted.org/pulp/wiki/UGRepo 
> > 
> > But I couldn't find any reference to Distributions or Yum Repo 
> > Metadata Files in the Pulp v2 documentation at all. 
> > 
> > Is there a good explanation of these somewhere? Do I need the 
> > distribution in order to kickstart? If I wanted to create one in 
> > each of my cloned repos for the distros that were missing from the 
> > vendor feeds, how would I do that? 
> > 
> > Also, what is the yum repo metadata file? 
> > 
> > Thanks for any assistance in understanding. 
> > 
> > Christina 
> > 
> > _______________________________________________ 
> > Pulp-list mailing list 
> > Pulp-list redhat com <mailto: Pulp-list redhat com > 
> > https://www.redhat.com/mailman/listinfo/pulp-list 
> > 
> > 
> > 
> > 
> > _______________________________________________ 
> > Pulp-list mailing list 
> > Pulp-list redhat com 
> > https://www.redhat.com/mailman/listinfo/pulp-list 
> > 
> 
> 
> _______________________________________________ 
> Pulp-list mailing list 
> Pulp-list redhat com 
> https://www.redhat.com/mailman/listinfo/pulp-list 


_______________________________________________ 
Pulp-list mailing list 
Pulp-list redhat com 
https://www.redhat.com/mailman/listinfo/pulp-list 


_______________________________________________
Pulp-list mailing list
Pulp-list redhat com
https://www.redhat.com/mailman/listinfo/pulp-list


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]