<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div>Please feel free to do it. Thanks.</div><div><br></div><div>--David</div><div><br></div>  <div style="font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; "> <div style="font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; "> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1">  <b><span style="font-weight:bold;">From:</span></b> Dmitri Pal <dpal@redhat.com><br> <b><span style="font-weight: bold;">To:</span></b> Rich Megginson <rmeggins@redhat.com> <br><b><span style="font-weight: bold;">Cc:</span></b> David Copperfield <cao2dan@yahoo.com>; Rob Crittenden <rcritten@redhat.com>; E Deon Lackey <dlackey@redhat.com>; "freeipa-users@redhat.com" <freeipa-users@redhat.com> <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, May 11,
 2012 2:53 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???<br> </font> </div> <br>
<div id="yiv1007810804">

  

    
  
  <div>
    On 05/10/2012 10:54 PM, Rich Megginson wrote:
    <blockquote type="cite">

      
      On 05/10/2012 07:54 PM, David Copperfield wrote:
      <blockquote type="cite">
        <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; ">
          <div><span>OK,</span></div>
          <div><span><br>
            </span></div>
          <div><span> that means the steps below:</span></div>
          <div><span><br>
            </span></div>
          <div>1) on IPA replica, lets create 4 IPA users: A,B,C and D.
             Now make a backup with '<a target="_blank" href="http://db2ldif.pl">db2ldif.pl</a> -r ...'</div>
          <div><br>
          </div>
          <div>2) on IPA replica, delete the user D. 'ipa user-del D'.</div>
          <div><br>
          </div>
          <div>3, on IPA master, delete the user C. 'ipa user-del C'.</div>
          <div><br>
          </div>
          <div>4, now check on other IPA master and IPA replica, both
            shows only two users 'A' and 'B'. this is expected.</div>
          <div><br>
          </div>
          <div>5, now on IPA replica, restore the backup with
            '<a target="_blank" href="http://ldif2db.pl">ldif2db.pl</a>'</div>
          <div><br>
          </div>
          <div>6, check on IPA replica immediately, 'ipa user-find'
            shows 4 users 'A, B, C, D' at the beginning.</div>
          <div><br>
          </div>
          <div>7, check IPA Master, 'ipa user-find' shows still only two
            users 'A, B'.</div>
          <div><br>
          </div>
          <div>8, wait 3 minutes or so, check on IPA replica, and found
            that there are only THREE users 'A, B, D'. The users 'C' is
            deleted now -- change propagated from IPA Master.</div>
          <div><br>
          </div>
          <div>9, check on IPA Master again and again, there are still
            only two users 'A, B'.</div>
          <div><br>
          </div>
          <div>10, check on IPA Replica again and again, there are still
            three users 'A, B,D'. --- this status is different from IPA
            Master's 'A,B', or backup's  'A, B, C, D'.</div>
          <div><br>
          </div>
          <div><br>
          </div>
          <div>If backup was created without '-r' option, then the step
            8 above will always show 'A,B,C,D', the same as backup.
             with '-r' option make the final result between.</div>
          <div><br>
          </div>
          <div><br>
          </div>
          <div>Hope I have explained it clearly. Please advice something
            like <a target="_blank" href="http://ipa2ldif.pl">ipa2ldif.pl</a> and <a target="_blank" href="http://ldif2ipa.pl">ldif2ipa.pl</a> tools. There are really the
            key useful feature for serious production IPA deployment,
            which is definitely of much higher priority than dogtag.</div>
        </div>
      </blockquote>
      <br>
      Sounds like a bug.  What should happen is that the deletion of C
      and D should be propagated to replica.<br>
    </blockquote>
    <br>
    Was a bug or a ticket filed?<br>
    <br>
    <blockquote type="cite"> <br>
      <blockquote type="cite">
        <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: 'times new roman', 'new york', times, serif; ">
          <div><br>
          </div>
          <div>Thanks a lot.</div>
          <div><br>
          </div>
          <div>--David</div>
          <div><br>
          </div>
          <div><span><br>
            </span></div>
          <div><br>
          </div>
          <div style="font-size: 12pt; font-family: times, serif; ">
            <div style="font-size: 12pt; font-family: times, serif; ">
              <div dir="ltr"> <font size="2" face="Arial">
                  <hr size="1"> <b><span style="font-weight:bold;">From:</span></b>
                  Rich Megginson <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rmeggins@redhat.com" target="_blank" href="mailto:rmeggins@redhat.com"><rmeggins@redhat.com></a><br>
                  <b><span style="font-weight:bold;">To:</span></b>
                  David Copperfield <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:cao2dan@yahoo.com" target="_blank" href="mailto:cao2dan@yahoo.com"><cao2dan@yahoo.com></a>
                  <br>
                  <b><span style="font-weight:bold;">Cc:</span></b> E
                  Deon Lackey <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:dlackey@redhat.com" target="_blank" href="mailto:dlackey@redhat.com"><dlackey@redhat.com></a>;
                  Petr Spacek <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:pspacek@redhat.com" target="_blank" href="mailto:pspacek@redhat.com"><pspacek@redhat.com></a>;
                  Rob Crittenden <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rcritten@redhat.com" target="_blank" href="mailto:rcritten@redhat.com"><rcritten@redhat.com></a>;
                  <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com">"freeipa-users@redhat.com"</a>
                  <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com"><freeipa-users@redhat.com></a>
                  <br>
                  <b><span style="font-weight:bold;">Sent:</span></b>
                  Thursday, May 10, 2012 6:37 PM<br>
                  <b><span style="font-weight:bold;">Subject:</span></b>
                  Re: [Freeipa-users] backup/restore IPA servers with
                  <a target="_blank" href="http://db2ldap.pl">db2ldap.pl</a>, <a target="_blank" href="http://ldap2db.pl">ldap2db.pl</a> ???<br>
                </font> </div>
              <br>
              <div id="yiv1007810804">
                <div> On 05/10/2012 07:32 PM, David Copperfield wrote:
                  <blockquote type="cite">
                    <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                      <div><span>Hi Rich and all,</span></div>
                      <div><span><br>
                        </span></div>
                      <div><span>the '-r' option to <a rel="nofollow" target="_blank" href="http://db2ldif.pl">db2ldif.pl</a>
                          doesn't work neither, it make few difference. </span></div>
                      <div><span><br>
                        </span></div>
                      <div><span>My command, backup and restore commands
                          on the IPA replica are:</span></div>
                      <div><span><br>
                        </span></div>
                      <div>db2ldif.pl -D 'cn=Directory Manager' -w - -r
                        -s 'dc=example,dc=com'</div>
                      <div><br>
                      </div>
                      <div><a rel="nofollow" target="_blank" href="http://ldif2db.pl">ldif2db.pl</a> -D
                        'cn=Directory Manager' -w - -i
                        <the_backup_file_in_LDIF_format></div>
                      <div><br>
                      </div>
                      <div>The only difference is: after IPA master
                        restart (restart happens after IPA replica's
                        restore operation), the changes -- which applied
                        on IPA master before backup -- are propagated to
                        IPA replica. Which is in fact, make the
                        restoration test end up with a result completely
                        unusable on IPA replica, an result that is
                        different from backup, and different from IPA
                        master. <br>
                      </div>
                    </div>
                  </blockquote>
                  <br>
                  I don't quite understand what you mean.<br>
                  <br>
                  <blockquote type="cite">
                    <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                      <div><br>
                      </div>
                      <div>Please let me know if there are any other
                        options/steps to follow. Thanks.</div>
                    </div>
                  </blockquote>
                  <br>
                  Not sure what else to try.<br>
                  <br>
                  <blockquote type="cite">
                    <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                      <div><br>
                      </div>
                      <div>--David</div>
                      <div><br>
                      </div>
                      <div><span><br>
                        </span></div>
                      <div><span><br>
                        </span></div>
                      <div><br>
                      </div>
                      <div style="font-size: 12pt; font-family: times, serif; ">
                        <div style="font-size: 12pt; font-family: times, serif; ">
                          <div dir="ltr"> <font size="2" face="Arial">
                              <hr size="1"> <b><span style="font-weight:bold;">From:</span></b>
                              Rich Megginson <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rmeggins@redhat.com" target="_blank" href="mailto:rmeggins@redhat.com"><rmeggins@redhat.com></a><br>
                              <b><span style="font-weight:bold;">To:</span></b>
                              David Copperfield <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:cao2dan@yahoo.com" target="_blank" href="mailto:cao2dan@yahoo.com"><cao2dan@yahoo.com></a>
                              <br>
                              <b><span style="font-weight:bold;">Cc:</span></b>
                              <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com">"freeipa-users@redhat.com"</a>
                              <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com"><freeipa-users@redhat.com></a>;
                              Rob Crittenden <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rcritten@redhat.com" target="_blank" href="mailto:rcritten@redhat.com"><rcritten@redhat.com></a>;
                              Petr Spacek <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:pspacek@redhat.com" target="_blank" href="mailto:pspacek@redhat.com"><pspacek@redhat.com></a>
                              <br>
                              <b><span style="font-weight:bold;">Sent:</span></b>
                              Thursday, May 10, 2012 5:28 PM<br>
                              <b><span style="font-weight:bold;">Subject:</span></b>
                              Re: [Freeipa-users] backup/restore IPA
                              servers with <a rel="nofollow" target="_blank" href="http://db2ldap.pl">db2ldap.pl</a>,
                              <a rel="nofollow" target="_blank" href="http://ldap2db.pl">ldap2db.pl</a>
                              ???<br>
                            </font> </div>
                          <br>
                          <div id="yiv1007810804">
                            <div> On 05/10/2012 04:37 PM, David
                              Copperfield wrote:
                              <blockquote type="cite">
                                <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                                  <div><span>Hi Rich and all,</span></div>
                                  <div><span><br>
                                    </span></div>
                                  <div><span>Thanks for correction. They
                                      are <a rel="nofollow" target="_blank" href="http://db2ldif.pl">db2ldif.pl</a>
                                      and <a rel="nofollow" target="_blank" href="http://ldif2db.pl">ldif2db.pl</a>
                                      scripts, which are originally for
                                      389 Directory Servers' backup and
                                      restore purposes. </span></div>
                                  <div><span><br>
                                    </span></div>
                                  <div><span>There are no IPA tools for
                                      IPA system backup and restore. </span>Is

                                    there a plan to develop tools like <a rel="nofollow" target="_blank" href="http://ipa2ldif.pl">ipa2ldif.pl</a>
                                    and <a rel="nofollow" target="_blank" href="http://ldif2ipa.pl">ldif2ipa.pl</a>
                                    soon? or, at least, whether it is in
                                    IPA roadmap?</div>
                                  <div><br>
                                  </div>
                                  <div>For the second question: I use
                                    the simple way: ipa
                                    user-add/user-delete/user-find to
                                    see whether data is propagated. My
                                    testing steps are like this:</div>
                                  <div><br>
                                  </div>
                                  <div> 1, run 'ipa user-add testuser'
                                    on IPA replica, check it on IPA
                                    master with 'ipa user-find testuser'
                                    and it is found in a few seconds --
                                    not 5 minutes.</div>
                                  <div><br>
                                  </div>
                                  <div> 2, run 'db2ldif.pl on IPA
                                    replica to save a backup.</div>
                                  <div><br>
                                  </div>
                                  <div> 3, run 'ipa user-del testuser'
                                    on IPA replica, then 'ipa user-find'
                                    on IPA replica, and  it shows that
                                    the user is deleted.</div>
                                  <div><br>
                                  </div>
                                  <div> 4, double check 'ipa user-find
                                    test user' on IPA master, and it is
                                    found deleted, which is as expected
                                    and it is propagated in just a few
                                    seconds.</div>
                                  <div><br>
                                  </div>
                                  <div> 5, run 'ldif2db.pl' on the same
                                    IPA replica where the backup was
                                    created.</div>
                                  <div><br>
                                  </div>
                                  <div> 6, run 'ipa user-find testuser'
                                    on IPA replica and it is found that
                                    the user testuser is alive again.</div>
                                  <div><br>
                                     7, run 'ipa user-find testuser' on
                                    IPA master. 1/3 times we can find it
                                    -- and in just a few seconds. other
                                    2/3 times it could not be found even
                                    after HALF HOUR.</div>
                                  <div><br>
                                  </div>
                                  <div>Please have a quick duplicate
                                    tests at your side and advice what
                                    normal users should do, because a
                                    reliable backup/restore solution is
                                    definitely one of the key criteria.
                                    Thanks a lot.</div>
                                  <div><br>
                                  </div>
                                </div>
                              </blockquote>
                              <br>
                              Ok, I see.  The problem is that a regular
                              db2ldif[.pl] does not save the replication
                              meta-data.  You must use the -r option to
                              generate an ldif file with the replication
                              meta-data.  ldif2db[.pl] is destructive -
                              it wipes out your database completely and
                              replaces it, wiping out any replication
                              meta-data in the process.  If you
                              ldif2db[.pl] a file exported with
                              db2ldif[.pl] -r, it will replace the
                              replication meta-data too.<br>
                              <br>
                              See
                              <a rel="nofollow" class="yiv1007810804moz-txt-link-freetext" target="_blank" href="http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line">http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line</a><br>
                              <br>
                              <blockquote type="cite">
                                <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                                  <div>--David</div>
                                  <div> </div>
                                  <div><br>
                                  </div>
                                  <div><br>
                                  </div>
                                  <div> </div>
                                  <div><span><br>
                                    </span></div>
                                  <div><span><br>
                                    </span></div>
                                  <div><br>
                                  </div>
                                  <div style="font-size: 12pt; font-family: times, serif; ">
                                    <div style="font-size: 12pt; font-family: times, serif; ">
                                      <div dir="ltr"> <font size="2" face="Arial">
                                          <hr size="1"> <b><span style="font-weight:bold;">From:</span></b>
                                          Rich Megginson <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rmeggins@redhat.com" target="_blank" href="mailto:rmeggins@redhat.com"><rmeggins@redhat.com></a><br>
                                          <b><span style="
font-weight:bold;">To:</span></b>
                                          David Copperfield <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:cao2dan@yahoo.com" target="_blank" href="mailto:cao2dan@yahoo.com"><cao2dan@yahoo.com></a>
                                          <br>
                                          <b><span style="
font-weight:bold;">Cc:</span></b> <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com">"freeipa-users@redhat.com"</a>
                                          <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:freeipa-users@redhat.com" target="_blank" href="mailto:freeipa-users@redhat.com"><freeipa-users@redhat.com></a>;
                                          Rob Crittenden <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:rcritten@redhat.com" target="_blank" href="mailto:rcritten@redhat.com"><rcritten@redhat.com></a>;
                                          Petr Spacek <a rel="nofollow" class="yiv1007810804moz-txt-link-rfc2396E" ymailto="mailto:pspacek@redhat.com" target="_blank" href="mailto:pspacek@redhat.com"><pspacek@redhat.com></a>
                                          <br>
                                          <b><span style="
font-weight:bold;">Sent:</span></b>
                                          Thursday, May 10, 2012 3:19 PM<br>
                                          <b><span style="
font-weight:bold;">Subject:</span></b>
                                          Re: [Freeipa-users]
                                          backup/restore IPA servers
                                          with <a rel="nofollow" target="_blank" href="http://db2ldap.pl">db2ldap.pl</a>,
                                          <a rel="nofollow" target="_blank" href="http://ldap2db.pl">ldap2db.pl</a>
                                          ???<br>
                                        </font> </div>
                                      <br>
                                      <div id="yiv1007810804">
                                        <div> On 05/10/2012 03:57 PM,
                                          David Copperfield wrote:
                                          <blockquote type="cite">
                                            <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                                              <div>Hi Rob, Petr and all,</div>
                                              <div><br>
                                              </div>
                                              <div>Because recently
                                                crashes of my IPA master
                                                and IPA replicas
                                                servers, I'm thinking of
                                                methods of
                                                backup/restore IPA user
                                                data: users, groups,
                                                host and server
                                                certificates etc.  </div>
                                              <div><br>
                                              </div>
                                              <div>It's said that the
                                                only official way is to
                                                create an extra IPA
                                                replica and
                                                backup/snapshot that
                                                replica all the way. But
                                                there still has a big
                                                chance that some
                                                mistakes propagate for a
                                                to whole IPA
                                                domain/realm before the
                                                IAP administrator find
                                                it and data got lost
                                                forever and some may not
                                                even be recovered.</div>
                                              <div><br>
                                              </div>
                                              <div>What I think is
                                                because both Dogtag and
                                                IPA store data in
                                                backend 389 directory
                                                servers separately, then
                                                if I freeze the change
                                                on one IPA replica for a
                                                few minutes first, then
                                                run <a rel="nofollow" target="_blank" href="http://db2ldap.pl">db2ldap.pl</a>
                                                for both 389 ldap
                                                backends, then un-freeze
                                                the IPA replica to get
                                                sync from master.</div>
                                              <div><br>
                                              </div>
                                              <div> When data needs to
                                                be restored because of
                                                disasters, the backup
                                                files(in LDIF format --
                                                for easy to read) can be
                                                restored to the two 389
                                                LDAP backends on IPA
                                                replica with command <a rel="nofollow" target="_blank" href="http://ldap2db.pl">ldap2db.pl</a>
                                                during the freezing
                                                period.</div>
                                            </div>
                                          </blockquote>
                                          <br>
                                          It's <a rel="nofollow" target="_blank" href="http://ldif2db.pl">ldif2db.pl</a>
                                          <a rel="nofollow" target="_blank" href="http://db2ldif.pl">db2ldif.pl</a>
                                          not ldap<br>
                                          <br>
                                          <blockquote type="cite">
                                            <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                                              <div><br>
                                              </div>
                                              <div> Have anyone tried
                                                this solution yet? Is
                                                there any limitations?</div>
                                              <div><br>
                                              </div>
                                              <div>My experiences showed
                                                that the IPA replica did
                                                get data restored
                                                successfully (no dogtag
                                                is involved so only one
                                                LDAP backend is
                                                saved/restored). But the
                                                IPA master some times
                                                didn't get the data
                                                synced from IPA replica
                                                ( 1/3 times it is
                                                synced, 2/3 times needs
                                                manual command
                                                'ipa-replica-manage
                                                force-sync  --from
                                                <ipaReplicaServer>'
                                                ).</div>
                                            </div>
                                          </blockquote>
                                          <br>
                                          How did you verify that the
                                          data was synced?  Note that if
                                          a server has been down for a
                                          while, it will take the
                                          supplier up to 5 minutes to
                                          recognize that the consumer is
                                          up again, without force sync.<br>
                                          <br>
                                          <blockquote type="cite">
                                            <div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-size: 12pt; font-family: times, serif; ">
                                              <div><br>
                                              </div>
                                              <div>Please shed a light
                                                in this area, as
                                                backup/restore of IPA
                                                master/replica is even
                                                not mentioned on the IPA
                                                document at all. </div>
                                              <div><br>
                                              </div>
                                              <div>Thanks a lot.</div>
                                              <div><br>
                                              </div>
                                              <div>--David</div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                            </div>
                                            <br>
                                            <fieldset class="yiv1007810804mimeAttachmentHeader"></fieldset>
                                            <br>
                                            <pre>_______________________________________________
Freeipa-users mailing list
<a rel="nofollow" class="yiv1007810804moz-txt-link-abbreviated" ymailto="mailto:Freeipa-users@redhat.com" target="_blank" href="mailto:Freeipa-users@redhat.com">Freeipa-users@redhat.com</a>
<a rel="nofollow" class="yiv1007810804moz-txt-link-freetext" target="_blank" href="https://www.redhat.com/mailman/listinfo/freeipa-users">https://www.redhat.com/mailman/listinfo/freeipa-users</a></pre>
                                          </blockquote>
                                          <br>
                                        </div>
                                      </div>
                                      <br>
                                      <br>
                                    </div>
                                  </div>
                                </div>
                              </blockquote>
                              <br>
                            </div>
                          </div>
                          <br>
                          <br>
                        </div>
                      </div>
                    </div>
                  </blockquote>
                  <br>
                </div>
              </div>
              <br>
              <br>
            </div>
          </div>
        </div>
      </blockquote>
      <br>
    </blockquote>
    <br>
    <br>
    <pre class="yiv1007810804moz-signature">-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
<a rel="nofollow" class="yiv1007810804moz-txt-link-abbreviated" target="_blank" href="http://www.redhat.com/carveoutcosts/">www.redhat.com/carveoutcosts/</a>


</pre>
  </div>

</div><br><br> </div> </div>  </div></body></html>