[EnMasse] Disconnect from time to time

Wed Mar 27 09:27:05 UTC 2019

Hi,

Yeah I was hoping to have more of the router log to see if it had the same
view of the world :) If you haven't got any logging facility capturing all
logs on the cluster, then I think it is gone.

How often would you say the disconnection happens? Is there any regular
pattern?

Best regards,

Ulf

On Wed, Mar 27, 2019 at 10:07 AM Bob Claerhout <bob.claerhout at aloxy.io>
wrote:

> Hi Ulf,
>
> These are the logs from the admin pod:
>
>     2019-03-21T00:10:05.458Z INFO  [AddressController] total: 435848727
> ns, resolvedPlan: 7329 ns, calculatedUsage: 66505 ns, checkedQuota: 9831
> ns, listClusters: 16603774 ns, provisionResources: 5975 ns, checkStatuses:
> 419078275 ns, deprovisionUnused: 11260 ns, upgradeClusters: 59877 ns,
> replaceAddresses: 4197 ns, gcTerminating: 1704 ns
>     2019-03-21T00:10:35.458Z INFO  [AddressController] Total: 6, Active:
> 6, Configuring: 0, Pending: 0, Terminating: 0, Failed: 0
>
> The logs from the qdrouterd is the one I already sent you (other logs from
> the qdrouterd are gone since I restarted the pod to enable PN_TRACE_FRM, if
> I can collect the logs from a stopped stateful set, please advise me:) ).
>
>     2019-03-21 10:30:20.777139 +0000 ROUTER_CORE (info) Link Route
> Deactivated 'ragent-gw-gw-out' on container gw-out
>
> Kind regards,
> Bob
>
> On 3/27/19 9:54 AM, Ulf Lilleengen wrote:
>
> Hi Bob,
>
> Do you happen to have the logs of the qdrouterd and admin pods as well?
> This seems to indicate one of the ends of router and broker is closing the
> TCP connection.
>
> Best regards,
>
> Ulf
>
> On Wed, Mar 27, 2019 at 9:26 AM Bob Claerhout <bob.claerhout at aloxy.io>
> wrote:
>
>> Hi all,
>>
>> Hereby the logging of the broker of this event.
>>
>> 2019-03-21T10:30:20.778Z INFO  [ProtonClientConnectionManager] Connection
>> messaging-aloxy/172.30.9.50:56671 destroyed
>> 2019-03-21T10:30:20.778Z WARN  [server] AMQ222061: Client connection
>> failed, clearing up resources for session
>> a4ebe8f7-4676-11e9-bac7-0a580a800051
>> 2019-03-21T10:30:20.988Z WARN  [server] AMQ222107: Cleared up resources
>> for session a4ebe8f7-4676-11e9-bac7-0a580a800051
>> 2019-03-21T10:30:20.988Z INFO  [amqp] connectionDestroyed for connector
>> address-connector-out
>> 2019-03-21T10:30:25.988Z INFO  [amqp] Starting connector
>> address-connector-out
>> 2019-03-21T10:30:25.999Z INFO  [amqp] connectionCreated for connector
>> address-connector-out
>> 2019-03-21T10:30:25.999Z INFO  [ProtonClientConnectionManager] Connection
>> messaging-aloxy/172.30.9.50:56671 created
>>
>> The broker has not restarted.
>>
>> Kind regards,
>> Bob
>>
>>
>>
>> On 3/22/19 6:02 PM, Ulf Lilleengen wrote:
>>
>> Hi Bob,
>>
>> Regarding Gordon and Teds reply, do you have the logs from the broker?
>> Alternatively could you also check if it got restarted for some reason?
>>
>> Best regards,
>>
>> Ulf
>>
>> fre. 22. mar. 2019 kl. 12:19 skrev Bob Claerhout <bob.claerhout at aloxy.io
>> >:
>>
>>> Hi Ulf,
>>>
>>> Thanks for the quick response.
>>> I've enabled this tracing. I'll come back to this when it occurs again.
>>>
>>> Enjoy the weekend
>>> Kind regards,
>>> Bob
>>>
>>>
>>> On 3/22/19 11:10 AM, Ulf Lilleengen wrote:
>>>
>>> Hi Bob,
>>>
>>> If your client is based on qpid-proton, turning on frame tracing might
>>> give some more info on why it is disconnected. You can do that by setting
>>> the environment PN_TRACE_FRM=1 for your client (I think this works with
>>> most qpid clients).
>>>
>>> Another alternative is to manually edit the qdrouterd statefulset on
>>> your cluster and add this env variable to the router to get more info in
>>> the router logs:
>>>
>>> kubectl set env statefulset/qdrouterd-$uuid PN_TRACE_FRM=1
>>>
>>> Best regards,
>>>
>>> Ulf
>>>
>>> On Fri, Mar 22, 2019 at 10:55 AM Bob Claerhout <bob.claerhout at aloxy.io>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> From time to time we experience a disconnect from enmasse. I'm trying
>>>> to figure out what's actually happening. In the logs I'm getting following
>>>> line:
>>>>     `2019-03-21 10:30:20.777139 +0000 ROUTER_CORE (info) Link Route
>>>> Deactivated 'ragent-gw-gw-out' on container gw-out`.
>>>>
>>>> The address I'm using is called "gw". Can you point me towards
>>>> somehwere I can find more information about why this link route is
>>>> deactivated?
>>>> Can I enable debug logging somewhere? Is it logged somewhere why a link
>>>> is deactivated?
>>>>
>>>> Kind regards,
>>>> Bob
>>>> --
>>>>
>>>> *Bob Claerhout*
>>>> Software developer
>>>> M +32 479 34 84 92
>>>> The Beacon, Sint-Pietersvliet 7
>>>> <https://maps.google.com/?q=Sint-Pietersvliet+7&entry=gmail&source=g>,
>>>> 2000
>>>> Antwerp
>>>> bob.claerhout at aloxy.io | www.aloxy.io
>>>> _______________________________________________
>>>> enmasse mailing list
>>>> enmasse at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/enmasse
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/enmasse/attachments/20190327/4ca28b23/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cmfaafbecndjnoom.png
Type: image/png
Size: 6330 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/enmasse/attachments/20190327/4ca28b23/attachment.png>