[Pulp-dev] To integrate Fus or not to....

Milan Kovacik mkovacik at redhat.com
Fri Oct 12 16:37:29 UTC 2018


On Fri, Oct 12, 2018 at 5:17 PM Jeff Ortel <jortel at redhat.com> wrote:

>
>
> On 10/12/2018 09:53 AM, Milan Kovacik wrote:
>
>
>
> On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel <jortel at redhat.com> wrote:
>
>>
>>
>> On 10/10/2018 08:59 AM, Milan Kovacik wrote:
>>
>> ...that might be the question we should ask ourselves once again when it
>> comes to recursive copying of units between repositories.
>>
>> I'd like to poll folks opinions about the possibilities that we may have
>> when it comes to integrating third party solvers in Pulp. My yesterday's
>> chat with the #fedora-modularity folks about us integrating the Fus[1]
>> solver in order to reuse the Fus algorithm ran into a couple of bumps:
>>
>> * it would be laborous to create a programmatic Python API between Fus
>> and Pulp because we can't directly use the libsolv thingies (pools,
>> solvables and friends) in such an API because Fus is written utilizing
>> GObject, which is incompatible with Swig, which in turn is used in libsolv
>> to expose the python bindings. One would have to either re-wrap libsolv
>> code in Fus to work with pygobject or submit PRs against libsolv to support
>> GObject introspection. I dunno the details of either approach (yet) but
>> from the sad faces on the IRC and the Fus PR[1] it seemed like a lot of
>> work but it's still an option
>>
>> * we still should be able to integrate thru a pipe into Fus, that would
>> make it possible to dump modular and ursine metadata into Fus to perform
>> the dependency solving in a separate subprocess. We should probably
>> re-check the reasons behind our previous decision not to do the same with
>> DNF[2].
>>
>>
>> How is Integration with Fus via pipe (CLI) easier than with gobject?
>> Either way, you "can't directly use the libsolv thingies (pools, solvables
>> and friends)".  Right?  What am I missing?
>>
>>
> Right, a publish-like operation would be required every time, for all
> repositories involved in the copy to dump the metadata to the pipe(s);
> sample of this interface is can be found in Pungi:
> https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the "query"
> is passed thru command line.
> I just learnt Fedora will keep modules and their ursine deps in separate
> repos, so the source repo won't necessarily be closed on dependencies thus
> multiple source repos would be needed.
>
>
> This be done using the Fus gobject interface as well?
>

we'd just dump the XML (and YAML) metadata and run: fus --repo
source1,1,/path/to/pipe1 --repo source2,2,/path/to/pipe2 --repo
target,system,/path/to/target_pipe  "module(walrus)" "penguin:1-2.3" etc
then parse the textual output of fus such as:

# ---%>---------
  - nothing provides policycoreutils-python-utils needed by
container-selinux-2:2.69-3.git452b90d.module_2040+0e96cf1b.noarch

Problem 1 / 1:
  - conflicting requests
  - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libc.so.6(GLIBC_2.2.5)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /bin/bash needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides /usr/bin/python3 needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides python3-dateutil needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
  - nothing provides dbus needed by
atomic-1.22.1-2.module_1637+1872e86a.x86_64
# ---->%----------
(fus:8524): fus-WARNING **: 15:13:09.350: Can't resolve all solvables
module:docker:2017.0:20180816194539:3ff668f0.x86_64 at f29
module:container-tools:2017.0:20180816194450:80bd9113.x86_64 at f29
*docker-devel-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.noarch at f29
*containers-common-0.1.31-14.dev.gitb0b750d.module_2040+0e96cf1b.x86_64 at f29




>
>>
>> * we should be able to extend current libsolv solver in Pulp,
>> reimplementing the algorithm from Fus. This might be as laborous as the
>> first option. It would probably give us more flexibility as well as more
>> room for screwing things up but the responsibility would be ours alone.
>>
>> Please let me know what option seems more appealing to you; other option
>> suggestion are welcome  too.
>>
>> Cheers,
>> milan
>>
>> [1] https://github.com/fedora-modularity/fus/pull/46
>> [2] https://pulp.plan.io/issues/3528#note-7
>>
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181012/72561a90/attachment.htm>


More information about the Pulp-dev mailing list