<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<br>
<div class="moz-cite-prefix">On 06/24/2015 11:47 AM, Ludwig Krispenz
wrote:<br>
</div>
<blockquote cite="mid:558A7CA5.1090301@redhat.com" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<div class="moz-cite-prefix">On 06/24/2015 11:36 AM, Oleg Fayans
wrote:<br>
</div>
<blockquote cite="mid:558A7A09.9060408@redhat.com" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 06/24/2015 11:25 AM, Ludwig
Krispenz wrote:<br>
</div>
<blockquote cite="mid:558A7798.8020000@redhat.com" type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
Oleg,<br>
<br>
the topology plugin relies on existing connection between
servers which remain in a topolgy. If you remove a central
node in your topology you are asking for trouble.<br>
With Petr's patch it warns you that your topology will be
disconnected, and if you insist we cannot guarantee anything.<br>
</blockquote>
Agree. I just wanted to try edge cases to see how one can break
the system :)
<blockquote cite="mid:558A7798.8020000@redhat.com" type="cite">
should we completely prohibit this ? I don't know, I think you
could also enforce an uninstall of vm175 with probably the
same result.<br>
what you mean be calculating the remaining topology and send
it to the remaining servers does not work, it would require to
send a removal of a segment, which would be rejected.<br>
<br>
The topology is broken, and I don't know how much we should
invest in making this info consistent on all servers. <br>
<br>
More interesting would be if we can heal this later by adding
new segments.<br>
</blockquote>
Yes, here comes the biggest question raised from this case:
obviously, when none of the nodes possess the correct topology
information (including the one which deleted the central node),
there is no way to fix it by adding segments connecting the
nodes that became disconnected. </blockquote>
It shoul not need the full information, but it has to be able to
reach one of the nodes to be connected. when the topology is
broken, you loose to feature to be ably to apply a change on any
node, eg in your case if you want to connect vm036 and vm056 an
have removed vm175, you have to do it on vm056, vm036 or vm244.
This should work, if not we have to fix it - unless we completely
prevent disconnecting a topology<br>
</blockquote>
Well, this is exactly the problem here: all replicas should contain
precise copies of all the info: accounts, hosts, sudorules, etc,
including topology information. However, if in this case I manually
connect disconnected node at vm127 (or vm056, does not matter) it
results in topology information inconsistency across the
infrastructure:<br>
This would be the topology from the point of view of vm127:<br>
<br>
vm056 vm036<br>
\ / |<br>
vm175 |<br>
\ |<br>
vm127 vm244<br>
<br>
And this - from the point of view of vm244 and vm036<br>
<br>
vm056 vm036<br>
\ |<br>
vm175 |<br>
|<br>
vm127 ----- vm244<br>
<blockquote cite="mid:558A7CA5.1090301@redhat.com" type="cite">
<blockquote cite="mid:558A7A09.9060408@redhat.com" type="cite">I
still think that the recalculation of the resulting tree should
be done at least on the node that performs the removal action.
And when later some other node gets connected, it should
understand somehow that it's topology information is outdated<br>
<blockquote cite="mid:558A7798.8020000@redhat.com" type="cite">
<br>
Ludwig<br>
<div class="moz-cite-prefix">On 06/24/2015 11:04 AM, Oleg
Fayans wrote:<br>
</div>
<blockquote cite="mid:558A729F.1030307@redhat.com" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
Hi everybody,<br>
<br>
Current implementation of topology plugin (including patch
878 from Petr) allows the deletion of the central node in
the star topology.<br>
I had the following topology:<br>
<br>
vm056 vm036<br>
\ / |<br>
vm175 |<br>
/ \ |<br>
vm127 vm244<br>
<br>
I was able to remove node vm175 from node vm244:<br>
<br>
[17:54:48]ofayans@vm-244:~]$ ipa-replica-manage del
vm-175.idm.lab.eng.brq.redhat.com <br>
Topology after removal of vm-175.idm.lab.eng.brq.redhat.com
will be disconnected:<br>
Server vm-036.idm.lab.eng.brq.redhat.com can't contact
servers: vm-056.idm.lab.eng.brq.redhat.com,
vm-127.idm.lab.eng.brq.redhat.com<br>
Server vm-056.idm.lab.eng.brq.redhat.com can't contact
servers: vm-244.idm.lab.eng.brq.redhat.com,
vm-036.idm.lab.eng.brq.redhat.com,
vm-127.idm.lab.eng.brq.redhat.com<br>
Server vm-127.idm.lab.eng.brq.redhat.com can't contact
servers: vm-244.idm.lab.eng.brq.redhat.com,
vm-056.idm.lab.eng.brq.redhat.com,
vm-036.idm.lab.eng.brq.redhat.com<br>
Server vm-244.idm.lab.eng.brq.redhat.com can't contact
servers: vm-056.idm.lab.eng.brq.redhat.com,
vm-127.idm.lab.eng.brq.redhat.com<br>
Continue to delete? [no]: yes<br>
Waiting for removal of replication agreements<br>
unexpected error: limits exceeded for this query<br>
<br>
I would expect this operation to delete 4 replication
agreements on all nodes:<br>
vm056 - vm175<br>
vm127 - vm175<br>
vm244 - vm175<br>
vm036 - vm175<br>
<br>
However an arbitrary set of replication agreements was
deleted on each node leading to total infrastructure
inconsistency:<br>
===============================================================<br>
vm056<b> </b>thought the topology was as follows:<br>
vm056 vm036<br>
/ |<br>
vm175 |<br>
/ \ |<br>
vm127 vm244<br>
[10:28:55]ofayans@vm-056:~]$ ipa topologysegment-find realm<br>
------------------<br>
4 segments matched<br>
------------------<br>
Segment name: 036-to-244<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-127.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-127.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com<br>
Left node: vm-175.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
----------------------------<br>
Number of entries returned 4<br>
----------------------------<br>
===============================================================<br>
both vm036<b> </b>vm244 thought the topology was as
follows:<br>
vm056 vm036<br>
\ |<br>
vm175 |<br>
/ |<br>
vm127 vm244<br>
<br>
[10:26:23]ofayans@vm-036:~]$ ipa topologysegment-find<br>
Suffix name: realm<br>
------------------<br>
3 segments matched<br>
------------------<br>
Segment name: 036-to-244<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-056.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-127.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-127.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
----------------------------<br>
Number of entries returned 3<br>
----------------------------<br>
<br>
===============================================================<br>
<b> </b>vm127 thought the topology was as follows:<br>
vm056 vm036<br>
\ / |<br>
vm175 |<br>
\ |<br>
vm127 vm244<br>
<br>
[10:31:08]ofayans@vm-127:~]$ ipa topologysegment-find realm<br>
------------------<br>
4 segments matched<br>
------------------<br>
Segment name: 036-to-244<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-056.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com<br>
Left node: vm-175.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
----------------------------<br>
Number of entries returned 4<br>
----------------------------<br>
<br>
If I, for example, add a segment connecting vm127 and vm244,
these two nodes will not synchronize the topology info:<br>
<br>
[10:51:03]ofayans@vm-127:~]$ ipa topologysegment-add realm
127-to-244 --leftnode=vm-127.idm.lab.eng.brq.redhat.com
--rightnode=vm-244.idm.lab.eng.brq.redhat.com
--direction=both<br>
--------------------------<br>
Added segment "127-to-244"<br>
--------------------------<br>
Segment name: 127-to-244<br>
Left node: vm-127.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
[10:53:33]ofayans@vm-127:~]$ ipa topologysegment-find realm<br>
------------------<br>
5 segments matched<br>
------------------<br>
Segment name: 036-to-244<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name: 127-to-244<br>
Left node: vm-127.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-036.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-056.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-175.idm.lab.eng.brq.redhat.com-to-vm-244.idm.lab.eng.brq.redhat.com<br>
Left node: vm-175.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
----------------------------<br>
Number of entries returned 5<br>
----------------------------<br>
[10:54:02]ofayans@vm-127:~]$ <br>
<br>
=============================================================<br>
<br>
[10:49:38]ofayans@vm-244:~]$ ipa topologysegment-find realm<br>
------------------<br>
3 segments matched<br>
------------------<br>
Segment name: 036-to-244<br>
Left node: vm-036.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name: 127-to-244<br>
Left node: vm-127.idm.lab.eng.brq.redhat.com<br>
Right node: vm-244.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
<br>
Segment name:
vm-056.idm.lab.eng.brq.redhat.com-to-vm-175.idm.lab.eng.brq.redhat.com<br>
Left node: vm-056.idm.lab.eng.brq.redhat.com<br>
Right node: vm-175.idm.lab.eng.brq.redhat.com<br>
Connectivity: both<br>
----------------------------<br>
Number of entries returned 3<br>
----------------------------<br>
[10:56:34]ofayans@vm-244:~]$ <br>
<br>
<big>Conclusion:</big><br>
We either should completely prohibit the removal of the
middle nodes (I mean, nodes that hide another active nodes),<br>
or at the removal stage first recalculate the resulting
topology and send it to all nodes before actual removal.<br>
<pre class="moz-signature" cols="72">--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.</pre>
</body>
</html>