[Pulp-dev] How to list orphan content in pulp3 before remocing them ?
Daniel Alley
dalley at redhat.com
Thu Apr 14 15:53:55 UTC 2022
Oh, ignore me, I didn't realize you don't need this to be via public HTTP
API.
On Thu, Apr 14, 2022 at 11:52 AM Daniel Alley <dalley at redhat.com> wrote:
> There is not a way to do this, figuring out which content are orphans is
> pretty expensive so I'm not certain it would be suitable for an endpoint
> unless we changed how it works.
>
> On Thu, Apr 14, 2022 at 11:30 AM Sayan Das <saydas at redhat.com> wrote:
>
>> Hello Grant,
>>
>> I hope you are doing great.
>>
>> Few other support folks had reached out to me to understand if we can
>> list the Orphaned data anyway in Pulp3 (Sat 6.10\RHUI 4).
>>
>> By anychance, Were you able to find out a way to do so in the version of
>> pulpcore being shipped for Sat 6.10 or RHUI 4 (3.14 i believe)?
>>
>>
>> Thanks & Regards,
>>
>> Sayan das
>>
>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>
>> Red Hat India
>> <https://www.redhat.com/>
>>
>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>
>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>
>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>> <https://red.ht/sig>
>>
>>
>> On Mon, Jan 31, 2022 at 9:31 PM Grant Gainey <ggainey at redhat.com> wrote:
>>
>>> On Mon, Jan 31, 2022 at 9:09 AM Sayan Das <saydas at redhat.com> wrote:
>>>
>>>> Hi Grant,
>>>>
>>>> I was trying your Django method but it fails in Satellite 6.10.2 .
>>>>
>>>
>>> Oh...drat. This is what comes of answering emails w/out enough caffeine.
>>> orphan-cleanup being asynchronous, and therefore that call, isn't available
>>> for 6.10, it's in 3.15 and beyond (and so slated for Sat7). See
>>> https://docs.pulpproject.org/pulpcore/changes.html#id140
>>>
>>> Give me a little, I'll see what we can do in the 3.14 timeframe.
>>>
>>> Sorry about that,
>>> G
>>>
>>>
>>>>
>>>> # sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' pulpcore-manager
>>>> shell << EOF
>>>> from pulpcore.app.models.content import Content
>>>> minutes_since_touched = 1
>>>> print(Content.objects.orphaned(minutes_since_touched).all())
>>>> EOF
>>>>
>>>> Traceback (most recent call last):
>>>> File "/bin/pulpcore-manager", line 11, in <module>
>>>> load_entry_point('pulpcore==3.14.9', 'console_scripts',
>>>> 'pulpcore-manager')()
>>>> File "/usr/lib/python3.6/site-packages/pulpcore/app/manage.py", line
>>>> 11, in manage
>>>> execute_from_command_line(sys.argv)
>>>> File
>>>> "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line
>>>> 381, in execute_from_command_line
>>>> utility.execute()
>>>> File
>>>> "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line
>>>> 375, in execute
>>>> self.fetch_command(subcommand).run_from_argv(self.argv)
>>>> File
>>>> "/usr/lib/python3.6/site-packages/django/core/management/base.py", line
>>>> 323, in run_from_argv
>>>> self.execute(*args, **cmd_options)
>>>> File
>>>> "/usr/lib/python3.6/site-packages/django/core/management/base.py", line
>>>> 364, in execute
>>>> output = self.handle(*args, **options)
>>>> File
>>>> "/usr/lib/python3.6/site-packages/django/core/management/commands/shell.py",
>>>> line 92, in handle
>>>> exec(sys.stdin.read())
>>>> File "<string>", line 3, in <module>
>>>> AttributeError: 'ContentManagerFromBulkTouchQuerySet' object has no
>>>> attribute 'orphaned'
>>>>
>>>> I definitely don't see *orphaned *inside *Content.objects*
>>>>
>>>> >>> print(Content.objects.
>>>> Content.objects.aggregate( Content.objects.datetimes(
>>>> Content.objects.from_queryset( Content.objects.prefetch_related(
>>>> Content.objects.all( Content.objects.db
>>>> Content.objects.get( Content.objects.raw(
>>>> Content.objects.annotate( Content.objects.db_manager(
>>>> Content.objects.get_or_create( Content.objects.reverse(
>>>> Content.objects.auto_created Content.objects.deconstruct(
>>>> Content.objects.get_queryset(
>>>> Content.objects.select_for_update(
>>>> Content.objects.bulk_create( Content.objects.defer(
>>>> Content.objects.in_bulk( Content.objects.select_related(
>>>> Content.objects.bulk_get_or_create( Content.objects.difference(
>>>> Content.objects.intersection( Content.objects.touch(
>>>> Content.objects.bulk_update( Content.objects.distinct(
>>>> Content.objects.iterator( Content.objects.union(
>>>> Content.objects.check( Content.objects.earliest(
>>>> Content.objects.last( Content.objects.update(
>>>> Content.objects.complex_filter( Content.objects.exclude(
>>>> Content.objects.latest( Content.objects.update_or_create(
>>>> Content.objects.contribute_to_class( Content.objects.exists(
>>>> Content.objects.model( Content.objects.use_in_migrations
>>>> Content.objects.count( Content.objects.explain(
>>>> Content.objects.name Content.objects.using(
>>>> Content.objects.create( Content.objects.extra(
>>>> Content.objects.none( Content.objects.values(
>>>> Content.objects.creation_counter Content.objects.filter(
>>>> Content.objects.only( Content.objects.values_list(
>>>> Content.objects.dates( Content.objects.first(
>>>> Content.objects.order_by(
>>>>
>>>>
>>>> My pulpcore version is 3.14.9. So Do I need to be on a higher version ?
>>>>
>>>>
>>>>
>>>> Thanks & Regards,
>>>>
>>>> Sayan das
>>>>
>>>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>>>
>>>> Red Hat India
>>>> <https://www.redhat.com/>
>>>>
>>>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>>>
>>>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>>>
>>>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>>>> <https://red.ht/sig>
>>>>
>>>>
>>>> On Mon, Jan 31, 2022 at 7:19 PM Sayan Das <saydas at redhat.com> wrote:
>>>>
>>>>> Hello Grant,
>>>>>
>>>>> Thanks for your quick response. I guess, mostly what would matter is
>>>>> the Type of Orphaned Content and the count of the same for each type.
>>>>>
>>>>> Pulp2 API and pulp-admin used to provide this nice summary and then
>>>>> for individual content types, There were endpoints available as well to see
>>>>> what are those contents.
>>>>>
>>>>>
>>>>> +----------------------------------------------------------------------+
>>>>> Summary
>>>>>
>>>>> +----------------------------------------------------------------------+
>>>>>
>>>>> Distribution: 0
>>>>> Docker Blob: 0
>>>>> Docker Image: 0
>>>>> Docker Manifest: 0
>>>>> Docker Manifest List: 0
>>>>> Docker Tag: 0
>>>>> Drpm: 0
>>>>> Erratum: 105
>>>>> Iso: 0
>>>>> Modulemd: 2
>>>>> Modulemd Defaults: 0
>>>>> Ostree: 0
>>>>> Package Category: 0
>>>>> Package Environment: 0
>>>>> Package Group: 0
>>>>> Package Langpacks: 0
>>>>> Puppet Module: 0
>>>>> Rpm: 700
>>>>> Srpm: 0
>>>>> Yum Repo Metadata File: 0
>>>>> Total: 807
>>>>>
>>>>>
>>>>> So, I guess, It makes sense to have something similar in Pulp3 as
>>>>> well.
>>>>>
>>>>> I will try out the method you have mentioned and see what else can be
>>>>> done with it.
>>>>>
>>>>>
>>>>> Thanks & Regards,
>>>>>
>>>>> Sayan das
>>>>>
>>>>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>>>>
>>>>> Red Hat India
>>>>> <https://www.redhat.com/>
>>>>>
>>>>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>>>>
>>>>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>>>>
>>>>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>>>>> <https://red.ht/sig>
>>>>>
>>>>>
>>>>> On Mon, Jan 31, 2022 at 5:59 PM Grant Gainey <ggainey at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Hey Sayan,
>>>>>>
>>>>>> The following dumps all the Content-objects that are orphaned (ie,
>>>>>> "not assigned to a repository and not touched in the last
>>>>>> "minutes_since_touched" minutes). Needs some expansion, but that depends on
>>>>>> what you want to get out of it. Artifact checksums? Content "name"
>>>>>> (whatever that means)? UUIDs?
>>>>>>
>>>>>> $ pulpcore-manager shell
>>>>>> from pulpcore.app.models.content import Content
>>>>>> minutes_since_touched = 1
>>>>>> print(Content.objects.orphaned(minutes_since_touched).all())
>>>>>> <BulkTouchQuerySet [<Content (pulp_type=rpm.packagelangpacks):
>>>>>> pk=2aee273b-3c1d-493a-9849-5ea717076795>, <Content
>>>>>> (pulp_type=rpm.distribution_tree):
>>>>>> pk=7aa4d756-fb5b-4630-bfe6-9e9c4f7461ee>, <Content
>>>>>> (pulp_type=rpm.packagegroup): shark>, <Content
>>>>>> (pulp_type=rpm.packagecategory): all>, <Content
>>>>>> (pulp_type=rpm.packageenvironment): SharkEnvironment>]>
>>>>>>
>>>>>> G
>>>>>>
>>>>>> On Mon, Jan 31, 2022 at 3:33 AM Sayan Das <saydas at redhat.com> wrote:
>>>>>>
>>>>>>> Hello All,
>>>>>>>
>>>>>>> I hope everyone is doing great.
>>>>>>>
>>>>>>> I am reaching out to the developers here as I wanted to find out *How
>>>>>>> I can list all orphan contents in Pulp 3* before deleting them?
>>>>>>>
>>>>>>> I have checked in Pulpcore API as well as Pulp cli but The only
>>>>>>> option I see is to remove the orphan contents directly.
>>>>>>>
>>>>>>> There are many situations where due to customer demand or our own
>>>>>>> understanding we may need to understand what all orphan data is present
>>>>>>> before we clear them up.
>>>>>>>
>>>>>>> Keeping that in mind, can anyone please suggest or confirm if we
>>>>>>> have any way to simply list the orphan content details in Pulp 3 similar to
>>>>>>> how we were able to do with Pulp 2?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>>>
>>>>>>> Sayan das
>>>>>>>
>>>>>>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>>>>>>
>>>>>>> Red Hat India
>>>>>>> <https://www.redhat.com/>
>>>>>>>
>>>>>>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>>>>>>
>>>>>>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>>>>>>
>>>>>>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>>>>>>> <https://red.ht/sig>
>>>>>>> _______________________________________________
>>>>>>> Pulp-dev mailing list
>>>>>>> Pulp-dev at redhat.com
>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Grant Gainey
>>>>>> Principal Software Engineer, Red Hat System Management Engineering
>>>>>>
>>>>>
>>>
>>> --
>>> Grant Gainey
>>> Principal Software Engineer, Red Hat System Management Engineering
>>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20220414/928e5ab5/attachment-0001.htm>
More information about the Pulp-dev
mailing list