[Pulp-dev] Proposal: Changing in 3.1 that Artifact.save() will hard-link/copy, not move

[Simon Baatz]

On Tue, Jan 07, 2020 at 04:45:54PM -0500, Brian Bouterse wrote:
>    We had two bugs filed recently [0][1] which suggest that when using the
>    default backend for Pulp, i.e. pulpcore.app.models.storage.FileSystem
>    Pulp should not be "moving" files. This is the default behavior Django
>    gives us, and it destroys data when sync'ed from file:/// for example
>    [1].

I am surprised that file:/// should be supported at all. This means
that a worker process must be able to access basically any file on the
system. Is this covered by the (upcoming?) SELinux policy? I would
expect workers to be more constrained than that. Or do we expect
the user to label files before importing?

>    I propose that with 3.1 we fix this bug by switching
>    pulpcore.app.models.storage.FileSystem to leave files in place and
>    either hard-link (same filesystem) or copy (different filesystem).

We should not use hard links:

- On systems with sysctl "fs.protected_hardlinks" enabled (Ubuntu, Fedora),
  the files would have to be owned by the pulp user to be able to
  create hard-links at all.
- On systems with SELinux enabled, these hard links will share
  the labels.
- Imported hard links are not protected against modification of the
  original file.

We could try to reflink the file and fall-back to copy (like "cp
--reflink=auto" does)