[Spacewalk-list] Invalid cross-device link error

Tue Jan 24 09:23:34 UTC 2012

Hello,
Please find attached a complete patch.
Here is what I've done: If the filename of the package to push starts
with CFG.MOUNT_POINT, it means that it was downloaded from a remote
repo, and so we should move it.
Instead, we should just copy it.

Hope this helps,

Pierre

2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
> Please find attached a partial patch (using diff -u and not git, as I
> don't have my development up and running), using shutil.move instead
> of os.rename.
> Why partial? because in the case of a local repository, the move is
> done from the real repository to Spacewalk mount point. So the
> original repository gets corrupted after this move (the package
> imported in spacewalk is missing). The behavior is correct for remote
> repository.
> As it allows me to get back a working spacewalk, I go with it. But any
> improvement would be really appreciated.
>
>
> [root at tu-spa-d15 importlib]# diff -u /root/importlib.py.orig importLib.py
> --- /root/importlib.py.orig     2012-01-23 15:52:03.649825333 +0000
> +++ importLib.py        2012-01-23 15:55:59.625754383 +0000
> @@ -17,6 +17,7 @@
>  #
>
>  import os
> +import shutil
>  from types import IntType, StringType, InstanceType
>  from UserDict import UserDict
>  from UserList import UserList
> @@ -808,7 +809,7 @@
>     # Create the directory where the file will reside
>     if not os.path.exists(dir):
>         createPath(dir)
> -    os.rename(filename, packagePath)
> +   shutil.move(filename, packagePath)
>     # set the path perms readable by all users
>     os.chmod(packagePath, 0644)
>
>
> Pierre
>
> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>> I keep working on my issue,
>> Here is a working strace, when downloading package osad from spacewalk
>> client web repository:
>> stat("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm",
>> 0x7fff36060790) = -1 ENOENT (No such file or directory)
>> stat("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1",
>> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>> rename("/data/satellite/redhat/1/stage/osad-5.10.34-1.el5.noarch.rpm",
>> "/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm")
>> = 0
>> chmod("/data/satellite/redhat/1/b4f/osad/5.10.34-1.el5/noarch/b4f797216885818ee51d9a01813422a1/osad-5.10.34-1.el5.noarch.rpm",
>> 0644) = 0
>>
>> It seems that spacewalk-repo-sync behaves differently when a local
>> repo is used (ie file:///var/...) or a web repo.
>> With a local repo, the rename is called on the source name to the dest name.
>> With a web repo, the repo is created in /MOUNT_POINT/redhat/1/stage
>> and then renamed, which is much better.
>>
>> Could you please help me with this one? I'm really stuck and have no
>> idea where to look at in the code.
>>
>> Pierre
>>
>>
>>
>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>> Hello,
>>> I've straced spacewalk-repo-sync call.
>>> Here is the error:
>>> stat("/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/mrepo-0.8.7-2.el5.noarch.rpm",
>>> 0x7fff4f62f540) = -1 ENOENT (No such file or directory)
>>> stat("/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73",
>>> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
>>> rename("/var/www/html/pub/epel/5/epel5-x86_64/mrepo-0.8.7-2.el5.noarch.rpm",
>>> "/data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/mrepo-0.8.7-2.el5.noarch.rpm")
>>> = -1 EXDEV (Invalid cross-device link)
>>>
>>> /data/satellite is Spacewalk mount_point
>>> /var/www/html/pub/epel/5/epel5-x86_64 is where the rpm is stored
>>> locally on the server. It gets here using reposync.
>>> Folder /data/satellite/redhat/1/bbe/mrepo/0.8.7-2.el5/noarch/bbe01b9ea61f85f4892dedc0fbb89b73/
>>> is correctly created.
>>>
>>> I've also dug a bit in bugzilla and found BZ 676369 being corrected
>>> for spacewalk 1.6. And this commit in particular:
>>> 2dbe4deade884d410f93bfcd30947bb77dd18e88
>>> -    pkgfd = os.open(packagePath, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
>>> -    os.lseek(fd, 0, 0)
>>> -    while 1:
>>> -        buffer = os.read(fd, 65536)
>>> -        if not buffer:
>>> -            break
>>> -        n = os.write(pkgfd, buffer)
>>> -        if n != len(buffer):
>>> -            # Error writing to the file
>>> -            raise IOError, "Wrote %s out of %s bytes in file %s" % (
>>> -                n, len(buffer), packagePath)
>>> -    os.close(pkgfd)
>>> +    os.rename(filename, packagePath)
>>>     # set the path perms readable by all users
>>> -    setPermsPath(packagePath, chmod=0644)
>>> +    os.chmod(packagePath, 0644)
>>>
>>>
>>> This patch introduces the use of os.rename command... which seems to
>>> be a problem with local repositories.
>>>
>>> Thanks in advance for your help,
>>>
>>> Pierre
>>>
>>>
>>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>>> Sorry to spam: it works well with public repositories, it only fails
>>>> with repositories built on the server, in my case:
>>>> - mrepo
>>>> - a subset of EPEL : I use reposync to download and built locally a
>>>> repo with only some rpms of EPEL.
>>>>
>>>> Pierre
>>>>
>>>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>>>> Some more information:
>>>>> - reposync also fails with public repositories from internet (like
>>>>> spacewalk client)
>>>>> - on my system /usr partition is mounted readonly
>>>>> - the rpms are stored on a NFS volume
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Pierre
>>>>>
>>>>>
>>>>> 2012/1/23 Pierre Casenove <pcasenove at gmail.com>:
>>>>>> Hello,
>>>>>> I have spacewalk 1.6 on postgresql and I have the following error log
>>>>>> when trying to reposync packages dowloaded by mrepo:
>>>>>> 57/57 : busybox-1.2.0-10.el5_7.1-1.x86_64
>>>>>> (50, 'Package upload failed: [Errno 18] Invalid cross-device link',
>>>>>> 'Invalid information uploaded to the server')
>>>>>>
>>>>>> Could you please tell me waht is the meaning of such error?
>>>>>>
>>>>>> Thanks in advance,
>>>>>>
>>>>>> Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Use-shutil-instead-of-os-to-push-packages.patch
Type: application/octet-stream
Size: 1688 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20120124/c84cbefa/attachment.obj>