<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <br>
    <div class="moz-cite-prefix">On 05/15/2018 05:58 AM, Austin
      Macdonald wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
      <div dir="auto">
        <div>Here's another complexity, how do 2 plugins create a single
          publication? </div>
      </div>
    </blockquote>
    <br>
    The plugin API could make this seamless.  <br>
    <br>
    <blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
      <div dir="auto">
        <div>We basically have the same problem of 2 parallel operations
          creating content from a single source. <br>
        </div>
      </div>
    </blockquote>
    <br>
    I don't think so.  plugins should not manipulate content outside of
    their domain (other plugins content) so either serial or parallel
    should be safe.<br>
    <br>
    <blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
      <div dir="auto">
        <div><br>
          <div class="gmail_quote">
            <div dir="ltr">On Tue, May 15, 2018, 06:27 Ina Panova <<a
                href="mailto:ipanova@redhat.com" target="_blank"
                rel="noreferrer" moz-do-not-send="true">ipanova@redhat.com</a>>
              wrote:<br>
            </div>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div dir="ltr">
                <div>
                  <div>
                    <div>
                      <div>
                        <div>+1 on not introducing dependencies between
                          plugins.<br>
                          <br>
                        </div>
                        What will be the behavior in case there is a
                        composed repo of rpm and ks trees but just the
                        rpm plugin is installed?</div>
                    </div>
                  </div>
                </div>
              </div>
            </blockquote>
          </div>
        </div>
        <div dir="auto">
          <div class="gmail_quote">
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div dir="ltr">
                <div>
                  <div>
                    <div>Do we fail and say we cannot sync this repo at
                      all or we just sync the rpm part?<br>
                    </div>
                  </div>
                </div>
              </div>
            </blockquote>
          </div>
        </div>
        <div dir="auto"><br>
        </div>
        <div dir="auto">
          <div dir="auto" style="font-family:sans-serif">Assuming
            plugins do not depend on each other, I think that when each
            plugin looks at the upstream repo, they will only "see" the
            content of that type. Conceptually, we will have 2 remotes,
            so it will feel like we are syncing from 2 totally distinct
            repositories.</div>
          <div dir="auto" style="font-family:sans-serif"><br>
          </div>
          <div dir="auto" style="font-family:sans-serif">The solution
            I've been imagining is a lot like 2. Each plugin would sync
            to a *separate repository.* These separate repositories are
            then published creating *separate publications*. This
            approach allows the plugins to live completely in ignorance
            of each other.</div>
          <div dir="auto" style="font-family:sans-serif"><br>
          </div>
          <div dir="auto" style="font-family:sans-serif">The final step
            is to associate *both publications to one distribution*,
            which composes the publications as they are served.</div>
          <div dir="auto" style="font-family:sans-serif"><br>
          </div>
          <div dir="auto" style="font-family:sans-serif">The downside is
            that we have to sync and publish twice, and that the
            resulting versions and publications aren't locked together.
            But I think this is better than leaving versions and
            publications unfinished with the assumption that another
            plugin will finish the job. Maybe linking them together
            could be a good use of the notes field.</div>
        </div>
      </div>
    </blockquote>
    <br>
    Pulp should support repositories with composed (mixed) content for
    the same reason RH does.  The repository is a collection of content
    that users want to manage together.  Consider the promotion cases:
    dev, test, prod.<br>
    <br>
    <blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
      <div dir="auto">
        <div dir="auto">
          <div dir="auto" style="font-family:sans-serif"><br>
          </div>
        </div>
        <div dir="auto">
          <div class="gmail_quote">
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div dir="ltr">
                <div>
                  <div>
                    <div><br>
                    </div>
                    Depends how we plan this ^ i guess we'll decide
                    which option 1 or 2 fits better.<br>
                    <br>
                  </div>
                  Don't want to go wild, but what if notion of composed
                  repos will be so popular in the future that's its
                  amount will increase? I think we do want to at least
                  partially being able to sync it and not take the
                  approach all or nothing?<br>
                  <br>
                </div>
                #2 speaks to me more for now.<br>
                <div><br>
                </div>
              </div>
              <div class="gmail_extra"><br clear="all">
                <div>
                  <div
                    class="m_5241349277208944567m_-8498754615663569299gmail_signature"
                    data-smartmail="gmail_signature">
                    <div dir="ltr"><br>
                      <br>
                      --------<br>
                      Regards,<br>
                      <br>
                      Ina Panova<br>
                      Software Engineer| Pulp| Red Hat Inc.<br>
                      <br>
                      "Do not go where the path may lead,<br>
                       go instead where there is no path and leave a
                      trail."<br>
                    </div>
                  </div>
                </div>
                <br>
                <div class="gmail_quote">On Mon, May 14, 2018 at 9:44
                  PM, Jeff Ortel <span dir="ltr"><<a
                      href="mailto:jortel@redhat.com" rel="noreferrer
                      noreferrer" target="_blank" moz-do-not-send="true">jortel@redhat.com</a>></span>
                  wrote:<br>
                  <blockquote class="gmail_quote" style="margin:0 0 0
                    .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    <div text="#000000" bgcolor="#FFFFFF"> <font
                        face="DejaVu Sans">Let's brainstorm on
                        something.<br>
                        <br>
                        Pulp needs to deal with remote repositories that
                        are composed of multiple content types which may
                        span the domain of a single plugin.  Here are a
                        few examples.  Some Red Hat RPM repositories are
                        composed of: RPMs, DRPMs, , ISOs and Kickstart
                        Trees.  Some OSTree repositories are composed of
                        OSTrees & Kickstart Trees. This raises a
                        question:  <br>
                        <br>
                        How can pulp3 best support syncing with remote
                        repositories that are composed of multiple
                        (unrelated) content types in a way that doesn't
                        result in plugins duplicating support for
                        content types?<br>
                        <br>
                        Few approaches come to mind:<br>
                        <br>
                        1. Multiple plugins (Remotes) participate in the
                        sync flow to produce a new repository version.</font><font
                        face="DejaVu Sans"><font face="DejaVu Sans"><br>
                          2. Multiple plugins (Remotes) are sync'd
                          successively each producing a new version of a
                          repository.  Only the last version contains
                          the fully sync'd composition.<br>
                          3. Plugins share code.</font><br>
                        4. Other?<br>
                        <br>
                        <br>
                        Option #1: Sync would be orchestrated by core or
                        the user so that multiple plugins (Remotes)
                        participate in populating a new repository
                        version.  For example: the RPM plugin (Remote)
                        and the Kickstart Tree plugin (Remote) would
                        both be sync'd against the same remote
                        repository that is composed of both types.  The
                        new repository version would be composed of the
                        result of both plugin (Remote) syncs.  To
                        support this, we'd need to provide a way for
                        each plugin to operate seamlessly on the same
                        (new) repository version.  Perhaps something
                        internal to the RepositoryVersion.  The
                        repository version would not be marked
                        "complete" until the last plugin (Remote) sync
                        has succeeded.  More complicated than #2 but
                        results in only creating truly complete versions
                        or nothing.  No idea how this would work with
                        current REST API whereby plugins provide sync
                        endpoints.<br>
                      </font><br>
                      <font face="DejaVu Sans"><font face="DejaVu Sans">Option
                          #2: Sync would be orchestrated by core or the
                          user so that multiple plugins (Remotes) create
                          successive repository versions.  For example:
                          the RPM plugin (Remote) and the Kickstart Tree
                          plugin (Remote) would both be sync'd against
                          the same remote repository that is a
                          composition including both types.  The
                          intermediate versions would be incomplete.  </font></font><font
                        face="DejaVu Sans"><font face="DejaVu Sans"><font
                            face="DejaVu Sans"><font face="DejaVu Sans">Only
                              the last version contains the fully sync'd
                              composition.  This approach can be
                              supported by core today :) but will
                              produce incomplete repository versions
                              that are marked complete=True.  This
                              /seems/ undesirable, right?  This may not
                              be a problem for distribution since I
                              would imaging that only the last (fully
                              composed) version would be published.  But
                              what about other usages of the
                              repository's "latest" version?<br>
                              <br>
                            </font></font></font></font><font
                        face="DejaVu Sans"><font face="DejaVu Sans">Option
                          #3: requires a plugin to be aware of specific
                          repository composition(s); other plugins and
                          creates a code dependency between plugins. 
                          For example, the RPM plugin could delegate
                          ISOs to the File plugin and Kickstart Trees to
                          the KickStart Tree plugin.</font><br>
                        <br>
                        For all options, plugins (Remotes) need to limit
                        sync to affect only those content types within
                        their domain.  For example, the RPM (Remote)
                        sync cannot add/remove ISO or KS Trees.<br>
                        <br>
                        I am an advocate of some from of options #1 or
                        #2.  Combining plugins (Remotes) as needed to
                        deal with arbitrary combinations within remote
                        repositories seems very powerful; does not
                        impose complexity on plugin writers; and does
                        not introduce code dependencies between plugins.<br>
                        <br>
                        Thoughts?<br>
                      </font> </div>
                    <br>
                    _______________________________________________<br>
                    Pulp-dev mailing list<br>
                    <a href="mailto:Pulp-dev@redhat.com" rel="noreferrer
                      noreferrer" target="_blank" moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
                    <a
                      href="https://www.redhat.com/mailman/listinfo/pulp-dev"
                      rel="noreferrer noreferrer noreferrer"
                      target="_blank" moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
                    <br>
                  </blockquote>
                </div>
                <br>
              </div>
              _______________________________________________<br>
              Pulp-dev mailing list<br>
              <a href="mailto:Pulp-dev@redhat.com" rel="noreferrer
                noreferrer" target="_blank" moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
              <a href="https://www.redhat.com/mailman/listinfo/pulp-dev"
                rel="noreferrer noreferrer noreferrer" target="_blank"
                moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
            </blockquote>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>