[Pulp-dev] To integrate Fus or not to....

Fri Oct 12 17:11:28 UTC 2018

On 10/12/2018 11:37 AM, Milan Kovacik wrote:
>
>
> On Fri, Oct 12, 2018 at 5:17 PM Jeff Ortel <jortel at redhat.com 
> <mailto:jortel at redhat.com>> wrote:
>
>
>
>     On 10/12/2018 09:53 AM, Milan Kovacik wrote:
>>
>>
>>     On Fri, Oct 12, 2018 at 3:59 PM Jeff Ortel <jortel at redhat.com
>>     <mailto:jortel at redhat.com>> wrote:
>>
>>
>>
>>         On 10/10/2018 08:59 AM, Milan Kovacik wrote:
>>>         ...that might be the question we should ask ourselves once
>>>         again when it comes to recursive copying of units between
>>>         repositories.
>>>
>>>         I'd like to poll folks opinions about the possibilities that
>>>         we may have when it comes to integrating third party solvers
>>>         in Pulp. My yesterday's chat with the #fedora-modularity
>>>         folks about us integrating the Fus[1] solver in order to
>>>         reuse the Fus algorithm ran into a couple of bumps:
>>>
>>>         * it would be laborous to create a programmatic Python API
>>>         between Fus and Pulp because we can't directly use the
>>>         libsolv thingies (pools, solvables and friends) in such an
>>>         API because Fus is written utilizing GObject, which is
>>>         incompatible with Swig, which in turn is used in libsolv to
>>>         expose the python bindings. One would have to either re-wrap
>>>         libsolv code in Fus to work with pygobject or submit PRs
>>>         against libsolv to support GObject introspection. I dunno
>>>         the details of either approach (yet) but from the sad faces
>>>         on the IRC and the Fus PR[1] it seemed like a lot of work
>>>         but it's still an option
>>>
>>>         * we still should be able to integrate thru a pipe into Fus,
>>>         that would make it possible to dump modular and ursine
>>>         metadata into Fus to perform the dependency solving in a
>>>         separate subprocess. We should probably re-check the reasons
>>>         behind our previous decision not to do the same with DNF[2].
>>
>>         How is Integration with Fus via pipe (CLI) easier than with
>>         gobject?  Either way, you "can't directly use the libsolv
>>         thingies (pools, solvables and friends)".  Right?  What am I
>>         missing?
>>
>>
>>     Right, a publish-like operation would be required every time, for
>>     all repositories involved in the copy to dump the metadata to the
>>     pipe(s); sample of this interface is can be found in Pungi:
>>     https://pagure.io/pungi/blob/master/f/pungi/wrappers/fus.py the
>>     "query" is passed thru command line.
>>     I just learnt Fedora will keep modules and their ursine deps in
>>     separate repos, so the source repo won't necessarily be closed on
>>     dependencies thus multiple source repos would be needed.
>
>     This be done using the Fus gobject interface as well?
>
>
> we'd just dump the XML (and YAML) metadata and run: fus --repo 
> source1,1,/path/to/pipe1 --repo source2,2,/path/to/pipe2 --repo 
> target,system,/path/to/target_pipe  "module(walrus)" "penguin:1-2.3" etc
> then parse the textual output of fus such as:

Can't this ^ be done with Fus through gobject as well and instead of 
parsing textual output, inspect the objects returned?

>
> # ---%>---------
>   - nothing provides policycoreutils-python-utils needed by 
> container-selinux-2:2.69-3.git452b90d.module_2040+0e96cf1b.noarch
> Problem 1 / 1:
>   - conflicting requests
>   - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides libc.so.6(GLIBC_2.2.5)(64bit) needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides /bin/bash needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides /usr/bin/python3 needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides python3-dateutil needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
>   - nothing provides dbus needed by 
> atomic-1.22.1-2.module_1637+1872e86a.x86_64
> # ---->%----------
> (fus:8524): fus-WARNING **: 15:13:09.350: Can't resolve all solvables
> module:docker:2017.0:20180816194539:3ff668f0.x86_64 at f29
> module:container-tools:2017.0:20180816194450:80bd9113.x86_64 at f29
> *docker-devel-2:1.13.1-61.git9cb56fd.module_2109+7c83ead1.noarch at f29
> *containers-common-0.1.31-14.dev.gitb0b750d.module_2040+0e96cf1b.x86_64 at f29
>
>
>
>>>
>>>         * we should be able to extend current libsolv solver in
>>>         Pulp, reimplementing the algorithm from Fus. This might be
>>>         as laborous as the first option. It would probably give us
>>>         more flexibility as well as more room for screwing things up
>>>         but the responsibility would be ours alone.
>>>
>>>         Please let me know what option seems more appealing to you;
>>>         other option suggestion are welcome  too.
>>>
>>>         Cheers,
>>>         milan
>>>
>>>         [1] https://github.com/fedora-modularity/fus/pull/46
>>>         [2] https://pulp.plan.io/issues/3528#note-7
>>>
>>>
>>>         _______________________________________________
>>>         Pulp-dev mailing list
>>>         Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>>>         https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>         _______________________________________________
>>         Pulp-dev mailing list
>>         Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>>         https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181012/3045b1ac/attachment.htm>