[Pulp-dev] How to list orphan content in pulp3 before remocing them ?
Sayan Das
saydas at redhat.com
Mon Jan 31 16:07:46 UTC 2022
Sure, No hurry and thank you again for sparing some time on this.
-- Sayan
On Mon, Jan 31, 2022, 9:31 PM Grant Gainey <ggainey at redhat.com> wrote:
> On Mon, Jan 31, 2022 at 9:09 AM Sayan Das <saydas at redhat.com> wrote:
>
>> Hi Grant,
>>
>> I was trying your Django method but it fails in Satellite 6.10.2 .
>>
>
> Oh...drat. This is what comes of answering emails w/out enough caffeine.
> orphan-cleanup being asynchronous, and therefore that call, isn't available
> for 6.10, it's in 3.15 and beyond (and so slated for Sat7). See
> https://docs.pulpproject.org/pulpcore/changes.html#id140
>
> Give me a little, I'll see what we can do in the 3.14 timeframe.
>
> Sorry about that,
> G
>
>
>>
>> # sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' pulpcore-manager
>> shell << EOF
>> from pulpcore.app.models.content import Content
>> minutes_since_touched = 1
>> print(Content.objects.orphaned(minutes_since_touched).all())
>> EOF
>>
>> Traceback (most recent call last):
>> File "/bin/pulpcore-manager", line 11, in <module>
>> load_entry_point('pulpcore==3.14.9', 'console_scripts',
>> 'pulpcore-manager')()
>> File "/usr/lib/python3.6/site-packages/pulpcore/app/manage.py", line
>> 11, in manage
>> execute_from_command_line(sys.argv)
>> File
>> "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line
>> 381, in execute_from_command_line
>> utility.execute()
>> File
>> "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line
>> 375, in execute
>> self.fetch_command(subcommand).run_from_argv(self.argv)
>> File "/usr/lib/python3.6/site-packages/django/core/management/base.py",
>> line 323, in run_from_argv
>> self.execute(*args, **cmd_options)
>> File "/usr/lib/python3.6/site-packages/django/core/management/base.py",
>> line 364, in execute
>> output = self.handle(*args, **options)
>> File
>> "/usr/lib/python3.6/site-packages/django/core/management/commands/shell.py",
>> line 92, in handle
>> exec(sys.stdin.read())
>> File "<string>", line 3, in <module>
>> AttributeError: 'ContentManagerFromBulkTouchQuerySet' object has no
>> attribute 'orphaned'
>>
>> I definitely don't see *orphaned *inside *Content.objects*
>>
>> >>> print(Content.objects.
>> Content.objects.aggregate( Content.objects.datetimes(
>> Content.objects.from_queryset( Content.objects.prefetch_related(
>> Content.objects.all( Content.objects.db
>> Content.objects.get( Content.objects.raw(
>> Content.objects.annotate( Content.objects.db_manager(
>> Content.objects.get_or_create( Content.objects.reverse(
>> Content.objects.auto_created Content.objects.deconstruct(
>> Content.objects.get_queryset( Content.objects.select_for_update(
>> Content.objects.bulk_create( Content.objects.defer(
>> Content.objects.in_bulk( Content.objects.select_related(
>> Content.objects.bulk_get_or_create( Content.objects.difference(
>> Content.objects.intersection( Content.objects.touch(
>> Content.objects.bulk_update( Content.objects.distinct(
>> Content.objects.iterator( Content.objects.union(
>> Content.objects.check( Content.objects.earliest(
>> Content.objects.last( Content.objects.update(
>> Content.objects.complex_filter( Content.objects.exclude(
>> Content.objects.latest( Content.objects.update_or_create(
>> Content.objects.contribute_to_class( Content.objects.exists(
>> Content.objects.model( Content.objects.use_in_migrations
>> Content.objects.count( Content.objects.explain(
>> Content.objects.name Content.objects.using(
>> Content.objects.create( Content.objects.extra(
>> Content.objects.none( Content.objects.values(
>> Content.objects.creation_counter Content.objects.filter(
>> Content.objects.only( Content.objects.values_list(
>> Content.objects.dates( Content.objects.first(
>> Content.objects.order_by(
>>
>>
>> My pulpcore version is 3.14.9. So Do I need to be on a higher version ?
>>
>>
>>
>> Thanks & Regards,
>>
>> Sayan das
>>
>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>
>> Red Hat India
>> <https://www.redhat.com/>
>>
>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>
>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>
>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>> <https://red.ht/sig>
>>
>>
>> On Mon, Jan 31, 2022 at 7:19 PM Sayan Das <saydas at redhat.com> wrote:
>>
>>> Hello Grant,
>>>
>>> Thanks for your quick response. I guess, mostly what would matter is the
>>> Type of Orphaned Content and the count of the same for each type.
>>>
>>> Pulp2 API and pulp-admin used to provide this nice summary and then for
>>> individual content types, There were endpoints available as well to see
>>> what are those contents.
>>>
>>> +----------------------------------------------------------------------+
>>> Summary
>>> +----------------------------------------------------------------------+
>>>
>>> Distribution: 0
>>> Docker Blob: 0
>>> Docker Image: 0
>>> Docker Manifest: 0
>>> Docker Manifest List: 0
>>> Docker Tag: 0
>>> Drpm: 0
>>> Erratum: 105
>>> Iso: 0
>>> Modulemd: 2
>>> Modulemd Defaults: 0
>>> Ostree: 0
>>> Package Category: 0
>>> Package Environment: 0
>>> Package Group: 0
>>> Package Langpacks: 0
>>> Puppet Module: 0
>>> Rpm: 700
>>> Srpm: 0
>>> Yum Repo Metadata File: 0
>>> Total: 807
>>>
>>>
>>> So, I guess, It makes sense to have something similar in Pulp3 as well.
>>>
>>> I will try out the method you have mentioned and see what else can be
>>> done with it.
>>>
>>>
>>> Thanks & Regards,
>>>
>>> Sayan das
>>>
>>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>>
>>> Red Hat India
>>> <https://www.redhat.com/>
>>>
>>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>>
>>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>>
>>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>>> <https://red.ht/sig>
>>>
>>>
>>> On Mon, Jan 31, 2022 at 5:59 PM Grant Gainey <ggainey at redhat.com> wrote:
>>>
>>>> Hey Sayan,
>>>>
>>>> The following dumps all the Content-objects that are orphaned (ie, "not
>>>> assigned to a repository and not touched in the last
>>>> "minutes_since_touched" minutes). Needs some expansion, but that depends on
>>>> what you want to get out of it. Artifact checksums? Content "name"
>>>> (whatever that means)? UUIDs?
>>>>
>>>> $ pulpcore-manager shell
>>>> from pulpcore.app.models.content import Content
>>>> minutes_since_touched = 1
>>>> print(Content.objects.orphaned(minutes_since_touched).all())
>>>> <BulkTouchQuerySet [<Content (pulp_type=rpm.packagelangpacks):
>>>> pk=2aee273b-3c1d-493a-9849-5ea717076795>, <Content
>>>> (pulp_type=rpm.distribution_tree):
>>>> pk=7aa4d756-fb5b-4630-bfe6-9e9c4f7461ee>, <Content
>>>> (pulp_type=rpm.packagegroup): shark>, <Content
>>>> (pulp_type=rpm.packagecategory): all>, <Content
>>>> (pulp_type=rpm.packageenvironment): SharkEnvironment>]>
>>>>
>>>> G
>>>>
>>>> On Mon, Jan 31, 2022 at 3:33 AM Sayan Das <saydas at redhat.com> wrote:
>>>>
>>>>> Hello All,
>>>>>
>>>>> I hope everyone is doing great.
>>>>>
>>>>> I am reaching out to the developers here as I wanted to find out *How
>>>>> I can list all orphan contents in Pulp 3* before deleting them?
>>>>>
>>>>> I have checked in Pulpcore API as well as Pulp cli but The only option
>>>>> I see is to remove the orphan contents directly.
>>>>>
>>>>> There are many situations where due to customer demand or our own
>>>>> understanding we may need to understand what all orphan data is present
>>>>> before we clear them up.
>>>>>
>>>>> Keeping that in mind, can anyone please suggest or confirm if we have
>>>>> any way to simply list the orphan content details in Pulp 3 similar to how
>>>>> we were able to do with Pulp 2?
>>>>>
>>>>>
>>>>>
>>>>> Thanks & Regards,
>>>>>
>>>>> Sayan das
>>>>>
>>>>> *S*enior* T*echnical *S*upport *E*ngineer, RHCE
>>>>>
>>>>> Red Hat India
>>>>> <https://www.redhat.com/>
>>>>>
>>>>> Red Hat India Pvt. Ltd, Level-5, Tower-10, Cyber City
>>>>>
>>>>> Magarpatta City Hadapsar, Pune-411013, Maharashtra, India.
>>>>>
>>>>> saydas at redhat.com M: +91-7890892756 IRC: Sayan
>>>>> <https://red.ht/sig>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> Grant Gainey
>>>> Principal Software Engineer, Red Hat System Management Engineering
>>>>
>>>
>
> --
> Grant Gainey
> Principal Software Engineer, Red Hat System Management Engineering
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20220131/3401dd7f/attachment.htm>
More information about the Pulp-dev
mailing list