[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Pulp-list] Grinder orphanes packages after download



Hi all,

I'm trying to sync a HP hosted repository (downloads.linux.hp.com).
Most of the packages are processed fine but a few are constantly
orphaned by Grinder after being downloaded. A few of them do show up
in /var/lib/pulp/packages/ but are not linked to a repo… I tried
creating the repo with '--feed=http://...' and also with an added
'--preserve_metada'.

Then I downloaded the packages with reposync, created metadata with
createrepo and placed this directory on a internal web server. With
the same result as mentioned above.

Finally, I tried '--feed=file:///...' (essentially bypassing Grinder)
and all packages were assigned correctly.

See below for detailed actions.

Any suggestions?

Regards,
Willem.

## FIRST ATTEMPT: ##

# pulp-admin repo create \
  --id=hp-test \
  --preserve_metadata \
  --feed=http://downloads.linux.hp.com/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64/
Successfully created repository [ hp-test ]

# pulp-admin repo sync --id=hp-test --threads=1 --foreground
Sync for repository hp-test started
Sync: Finished
46/46 new items downloaded
0/46 existing items processed
Item Details:
RPMs: 46/46

# pulp-admin package search -f "^hp-.*"
+------------------------------------------+
             Package Information
+------------------------------------------+
Name             EVRA                       Filename
            Repositories
hp-tg3           0:3.116j-3.src             hp-tg3-3.116j-3.src.rpm
             hp-test
hp-smh-templates 0:8.7.0.10-10.noarch
hp-smh-templates-8.7.0.10-10.noarch.rpm hp-test
hp-smh-templates 0:8.6.0.14-16.noarch
hp-smh-templates-8.6.0.14-16.noarch.rpm hp-test
...

# pulp-admin package search -f "^hp-snmp.*"
error:  No packages found.

# ls -1d /var/lib/pulp/packages/hp-snmp*
/var/lib/pulp/packages/hp-snmp-agents

# cat /var/log/pulp/grinder.log
...
2011-11-20 21:13:35,155 13269:139726048700160: grinder.YumInfo:INFO:
activeobject:160 46 packages have been marked to be fetched
...
2011-11-20 21:13:37,708 13269:139726048700160:
grinder.ParallelFetch:INFO: ParallelFetch:177 46 items are marked to
be fetched
...
2011-11-20 21:14:53,340 13269:139725956445952: grinder.BaseFetch:INFO:
activeobject:160 Symlink missing in repo directory. Creating link
/var/lib/pulp//repos/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64//hp-snmp-agents-8.6.2.3-9.x86_64.rpm
to ../../../../../../../../packages/hp-snmp-agents/8.6.2.3/9/x86_64/e96/hp-snmp-agents-8.6.2.3-9.x86_64.rpm
...
2011-11-20 21:16:07,138 13269:139726048700160:
grinder.ParallelFetch:INFO: ParallelFetch:238 ParallelFetch: 46 items
successfully processed, 33 downloaded, 0 items had errors
...
2011-11-20 21:16:07,140 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:154 Cleaning any orphaned packages..
...
2011-11-20 21:16:07,143 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:190 Removing orphan package:
/var/lib/pulp//repos/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64/hp-snmp-agents-8.6.2.3-9.x86_64.rpm
...
2011-11-20 21:16:07,144 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:159 Processed <46> items in [154] seconds. Report: 46
successes, 33 downloads, 0 errors
2011-11-20 21:16:
...

# pulp-admin repo delete --id=hp-test
Successful deleted repository [ hp-test ]

# rm -r /var/lib/pulp/packages/{cpqacuxe,hpacucli,hpvca,hpdiags,hp-health,hp-snmp-agents}

## SECOND ATTEMPT: ##

# cat > /etc/yum.repos.d/hp-test.repo << EOF
[hp-test]
name=hp-test
baseurl=http://downloads.linux.hp.com/SDR/downloads/proliantsupportpack/redhat/6/packages/i686/
enabled=1
gpgcheck=0
EOF

# cd /var/www/html

# reposync --repoid=hp-test .
...
[hp-test: 11    of 31    ] Downloading hp-smh-templates-8.6.0.14-16.noarch.rpm
hp-smh-templates-8.6.0.14-16.noarch.rpm                         | 3.4
MB     00:02
[hp-test: 12    of 31    ] Downloading hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm
hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm                         | 1.7
MB     00:00
[hp-test: 13    of 31    ] Downloading hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm
hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm                       | 1.7
MB     00:00
...

# cd hp-test

# createrepo .
Spawning worker 0 with 31 pkgs
...

# zcat repodata/*-primary.xml.gz | grep "<name>hp-snmp"
  <name>hp-snmp-agents</name>
  <name>hp-snmp-agents</name>

# pulp-admin repo create \
  --id=hp-test \
  --feed=http://n004.localdomain/hp-test/

# pulp-admin repo sync --id=hp-test --threads=1 --foreground

# pulp-admin package search -f "^hp-snmp.*"
<output identical to first attempt>

# cat /var/log/pulp/grinder.log
<output identical to first attempt>

# ls -1d /var/lib/pulp/packages/hp-*
<output identical to first attempt>

# pulp-admin repo delete --id=hp-test
Successful deleted repository [ hp-test ]

# rm -r /var/lib/pulp/packages/{cpqacuxe,hpacucli,hpdiags,hp-health,hp-snmp-agents}

## THIRD ATTEMPT: ##

# pulp-admin repo create \
  --id=hp-test \
  --feed=file:///var/www/html/hp-test/

# pulp-admin repo sync --id=hp-test --threads=1 --foreground
pulp-admin repo sync --id=hp-test --threads=1 --foreground
Sync for repository hp-test started
Sync: Finished
31/31 new items downloaded
0/31 existing items processed
Item Details:
RPMs: 31/31

# pulp-admin package search -f "^hp-.*"
+------------------------------------------+
             Package Information
+------------------------------------------+
Name             EVRA                       Filename
              Repositories
hp-snmp-agents   0:8.7.0.23-16.i386
hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm hp-test
hp-snmp-agents   0:8.6.2.3-8.i386
hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm   hp-test
...


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]