[Spacewalk-list] Invalid cross-device link error

[Pierre Casenove]

Please find attached a partial patch (using diff -u and not git, as I
don't have my development up and running), using shutil.move instead
of os.rename.
Why partial? because in the case of a local repository, the move is
done from the real repository to Spacewalk mount point. So the
original repository gets corrupted after this move (the package
imported in spacewalk is missing). The behavior is correct for remote
repository.
As it allows me to get back a working spacewalk, I go with it. But any
improvement would be really appreciated.


[root at tu-spa-d15 importlib]# diff -u /root/importlib.py.orig importLib.py
--- /root/importlib.py.orig     2012-01-23 15:52:03.649825333 +0000
+++ importLib.py        2012-01-23 15:55:59.625754383 +0000
@@ -17,6 +17,7 @@
 #

 import os
+import shutil
 from types import IntType, StringType, InstanceType
 from UserDict import UserDict
 from UserList import UserList
@@ -808,7 +809,7 @@
     # Create the directory where the file will reside
     if not os.path.exists(dir):
         createPath(dir)
-    os.rename(filename, packagePath)
+   shutil.move(filename, packagePath)
     # set the path perms readable by all users
     os.chmod(packagePath, 0644)


Pierre

2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
> I keep working on my issue,
> Here is a working strace, when downloading package osad from spacewalk
> client web repository:
> stat("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm",
> 0x7fff36060790) = -1 ENOENT (No such file or directory)
> stat("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1",
> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
> rename("/data/satellite/redhat/1/stage/osad-5.10.34-1.el5.noarch.rpm",
> "/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm")
> = 0
> chmod("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm",
> 0644) = 0
>
> It seems that spacewalk-repo-sync behaves differently when a local
> repo is used (ie file:///var/...) or a web repo.
> With a local repo, the rename is called on the source name to the dest name.
> With a web repo, the repo is created in /MOUNT_POINT/redhat/1/stage
> and then renamed, which is much better.
>
> Could you please help me with this one? I'm really stuck and have no
> idea where to look at in the code.
>
> Pierre
>
>
>
> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>> Hello,
>> I've straced spacewalk-repo-sync call.
>> Here is the error:
>> stat("/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/mrepo-0.8.7-2.el5.noarch.rpm",
>> 0x7fff4f62f540) = -1 ENOENT (No such file or directory)
>> stat("/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73",
>> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>> rename("/var/www/html/pub/epel/5/epel5-x86_64/mrepo-0.8.7-2.el5.noarch.rpm",
>> "/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/mrepo-0.8.7-2.el5.noarch.rpm")
>> = -1 EXDEV (Invalid cross-device link)
>>
>> /data/satellite is Spacewalk mount_point
>> /var/www/html/pub/epel/5/epel5-x86_64 is where the rpm is stored
>> locally on the server. It gets here using reposync.
>> Folder /data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/
>> is correctly created.
>>
>> I've also dug a bit in bugzilla and found BZ 676369 being corrected
>> for spacewalk 1.6. And this commit in particular:
>> 2dbe4deade884d410f93bfcd30947bb77dd18e88
>> -    pkgfd = os.open(packagePath, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
>> -    os.lseek(fd, 0, 0)
>> -    while 1:
>> -        buffer = os.read(fd, 65536)
>> -        if not buffer:
>> -            break
>> -        n = os.write(pkgfd, buffer)
>> -        if n != len(buffer):
>> -            # Error writing to the file
>> -            raise IOError, "Wrote %s out of %s bytes in file %s" % (
>> -                n, len(buffer), packagePath)
>> -    os.close(pkgfd)
>> +    os.rename(filename, packagePath)
>>     # set the path perms readable by all users
>> -    setPermsPath(packagePath, chmod=0644)
>> +    os.chmod(packagePath, 0644)
>>
>>
>> This patch introduces the use of os.rename command... which seems to
>> be a problem with local repositories.
>>
>> Thanks in advance for your help,
>>
>> Pierre
>>
>>
>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>> Sorry to spam: it works well with public repositories, it only fails
>>> with repositories built on the server, in my case:
>>> - mrepo
>>> - a subset of EPEL : I use reposync to download and built locally a
>>> repo with only some rpms of EPEL.
>>>
>>> Pierre
>>>
>>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>>> Some more information:
>>>> - reposync also fails with public repositories from internet (like
>>>> spacewalk client)
>>>> - on my system /usr partition is mounted readonly
>>>> - the rpms are stored on a NFS volume
>>>>
>>>> Thanks,
>>>>
>>>> Pierre
>>>>
>>>>
>>>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>>>> Hello,
>>>>> I have spacewalk 1.6 on postgresql and I have the following error log
>>>>> when trying to reposync packages dowloaded by mrepo:
>>>>> 57/57 : busybox-1.2.0-10.el5_7.1-1.x86_64
>>>>> (50, 'Package upload failed: [Errno 18] Invalid cross-device link',
>>>>> 'Invalid information uploaded to the server')
>>>>>
>>>>> Could you please tell me waht is the meaning of such error?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Pierre