[Devtools] CDK 2.4 and Minishift CDK uses lots of CPU

[Burr Sutter]

So, I left it running just to see what it would do...eventually the "du"
processes seem to quiet down and virtualbox went quiet....and it seems that
openshift crashed.  The VM is still up but I can't find an openshift
process.
oc login

error: dial tcp 192.168.99.104:8443: getsockopt: connection refused


and after a "minishift ssh"

ps -ef | grep openshift

docker   10876 10855  0 16:05 pts/0    00:00:00 grep --color=auto
*openshift*


On Wed, Apr 5, 2017 at 10:53 AM, Clayton Coleman <ccoleman at redhat.com>
wrote:

> Could be clock skew after sleep, if something is calculating a schedule
> for runs and sees it has fallen behind (all the time the box was asleep?)
> and then tries to run all the missing du executions.
>
> One way to test - set the clock forward a few hours and see if you get a
> spike in du
>
> On Apr 5, 2017, at 3:52 PM, Lalatendu Mohanty <lmohanty at redhat.com> wrote:
>
>
>
> On Wed, Apr 5, 2017 at 7:15 PM, Jimmi Dyson <jdyson at redhat.com> wrote:
>
>> Ah I see that minishift isn't using a centos VM so the du processes are
>> almost definitely cadvisor checking aufs writable layer usage. Why there's
>> so many du processes I don't know though...
>>
>
> I do not think we have added any du process. I am not aware of this.
>
> -Lala
>
>
>>
>> On Wed, Apr 5, 2017 at 2:41 PM, Burr Sutter <bsutter at redhat.com> wrote:
>>
>>> Aha...left it running overnight (machine sleeping), woke it up this
>>> morning and
>>> Virtualbox is using 197% of CPU
>>> https://www.screencast.com/t/u2TuTzut
>>> Lots of du processes, working on the CPU
>>> https://www.screencast.com/t/JhG3mGMdRkYO
>>>
>>> now, this with Helloworld MSA and metrics
>>> ./minishift --metrics true start
>>>
>>>
>>>
>>> On Tue, Apr 4, 2017 at 11:42 PM, Burr Sutter <bsutter at redhat.com> wrote:
>>>
>>>> After lots of testing on the CDK variant, is no longer demonstrating
>>>> the crazy du processes eating all the CPU.  I see one or two du processes
>>>> spike up but not to the point where it is overly problematic.  I did have
>>>> to turn on metrics (which doesn't work anyway) to see the du process via
>>>> top.
>>>>
>>>> Minishift version: 1.0.0-beta.5
>>>>
>>>> CDK Version: 3.0.0-beta.3
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Apr 4, 2017 at 10:37 AM, Gerard Braad <gbraad at redhat.com>
>>>> wrote:
>>>>
>>>>> Hi Burr,
>>>>>
>>>>> On Tue, Apr 4, 2017 at 8:34 PM, Burr Sutter <bsutter at redhat.com>
>>>>> wrote:
>>>>> > Adding devtools back to the thread.
>>>>> > On Tue, Apr 4, 2017 at 2:13 AM, Gerard Braad <gbraad at redhat.com>
>>>>> wrote:
>>>>> >> On Thu, Mar 30, 2017 at 10:47 PM, Burr Sutter <bsutter at redhat.com>
>>>>> wrote:
>>>>> >> > Still eats all the CPU :-)
>>>>> >> > du is the culprit according to top
>>>>> >> @burr when did you notice `du` was the culprit of consuming the CPU
>>>>> as
>>>>> >> a resource? during the `oc cluster up`, or after?
>>>>> > I tested with the upstream version overnight and Helloworld MSA, no
>>>>> problems
>>>>> > Minishift version: 1.0.0-rc.1
>>>>> > it had no problems.
>>>>>
>>>>> great to hear... mostly, Minishift tries to stay out of the way of the
>>>>> actual OpenShift deployment. In that sense, we prepare the environment
>>>>> and allow configuration, and dealing with OpenShift when the
>>>>> deployment happened. From this perspective, I am interested in how 'it
>>>>> seems to use more memory/cpu'. Minishift itself should not cause this
>>>>> directly. But of course, our foundation (the Operating System/ISO)
>>>>> could be a problem. So, if you have any quantifiable metrics, please
>>>>> ;-). It might be helpful to also test different OpenShift versions
>>>>> with a release of the Minishift (CDK) binary+ISOs.
>>>>>
>>>>> Sorry, to make you have to consider a lot of testing... but this
>>>>> information can be valuable to track down the actual root cause. And
>>>>> yes, we have several in recent time which should improve the quality
>>>>> and performance of deploying using Minishift.
>>>>>
>>>>> Looking forward to more feedback...
>>>>>
>>>>>
>>>>> Gerard
>>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Devtools mailing list
>>> Devtools at redhat.com
>>> https://www.redhat.com/mailman/listinfo/devtools
>>>
>>>
>>
>> _______________________________________________
>> Devtools mailing list
>> Devtools at redhat.com
>> https://www.redhat.com/mailman/listinfo/devtools
>>
>>
> _______________________________________________
> Devtools mailing list
> Devtools at redhat.com
> https://www.redhat.com/mailman/listinfo/devtools
>
>
> _______________________________________________
> Devtools mailing list
> Devtools at redhat.com
> https://www.redhat.com/mailman/listinfo/devtools
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/devtools/attachments/20170405/25b5962a/attachment.htm>