<div dir="ltr"><div><div>Hi <span style="color:rgb(0,0,0)"><span class="gmail-HOEnZb">Štěpán</span></span>,<br><br></div>Just to round this out, I believe the changes that cause this speedup went into Augeas 1.5.0, and they are the commits from <a href="https://github.com/hercules-team/augeas/pull/303">this PR</a>.<br><br></div>David<br><div> <div><div><div class="gmail-gE gmail-iv gmail-gt"><table class="gmail-cf gmail-gJ" cellpadding="0"><tbody><tr class="gmail-acZ"><td class="gmail-gF gmail-gK"><table class="gmail-cf gmail-ix" width="231" height="24" cellpadding="0"><tbody><tr><td></td></tr></tbody></table></td></tr></tbody></table></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 14, 2017 at 7:27 AM, Štěpán Balážik <span dir="ltr"><<a href="mailto:stepan.balazik@nic.cz" target="_blank">stepan.balazik@nic.cz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi again,<br>
<br>
so I was using Augeas version 1.4.0 from Ubuntu repositories. Installing latest stable version 1.8.0 made the process (literally) 100x faster.<br>
<br>
I am sorry I was bothering you all.<span class="gmail-HOEnZb"><font color="#888888"><br>
<br>
Štěpán</font></span><div class="gmail-HOEnZb"><div class="gmail-h5"><br>
<br>
<br>
On 14.6.2017 14:40, Josef Reidinger wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
On Wed, 14 Jun 2017 14:13:31 +0200<br>
"Štěpán Balážik" <<a href="mailto:stepan.balazik@nic.cz" target="_blank">stepan.balazik@nic.cz</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Hi,<br>
<br>
we want to use Augeas to programmatically parse and generate tests for<br>
Deckard <<a href="https://gitlab.labs.nic.cz/knot/deckard" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kn<wbr>ot/deckard</a>>[1] (our test harness<br>
for DNS resolvers).<br>
<br>
I wrote a lens<br>
<<a href="https://gitlab.labs.nic.cz/knot/deckard/blob/augeas_wip/pydnstest/deckard.aug" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kn<wbr>ot/deckard/blob/augeas_wip/pyd<wbr>nstest/deckard.aug</a>>[2]<br>
which works fine but on longer config files such as this one<br>
<<a href="https://gitlab.labs.nic.cz/knot/deckard/blob/augeas_wip/sets/resolver/iter_badraw.rpl" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kn<wbr>ot/deckard/blob/augeas_wip/set<wbr>s/resolver/iter_badraw.rpl</a>>[3]<br>
the match operation on /scenario/step nodes takes about 3 ms per call on<br>
my machine using both augtool and the python binding. This sums up to<br>
over 10 s for the whole file which too slow for our purposes.<br>
<br>
We have no idea what is the cause of this slowness. Are Augeas matches<br>
always so slow on "shallow and wide" trees?<br>
<br>
Thanks in advance for any help.<br>
Štěpán Balážik @ CZ.NIC<br>
<br>
[1] <a href="https://gitlab.labs.nic.cz/knot/deckard" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kno<wbr>t/deckard</a><br>
[2]<br>
<a href="https://gitlab.labs.nic.cz/knot/deckard/blob/augeas_wip/pydnstest/deckard.aug" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kno<wbr>t/deckard/blob/augeas_wip/pydn<wbr>stest/deckard.aug</a><br>
[3]<br>
<a href="https://gitlab.labs.nic.cz/knot/deckard/blob/augeas_wip/sets/resolver/iter_badraw.rpl" rel="noreferrer" target="_blank">https://gitlab.labs.nic.cz/kno<wbr>t/deckard/blob/augeas_wip/sets<wbr>/resolver/iter_badraw.rpl</a><br>
<br>
</blockquote>
Hi Stepan,<br>
we face similar issue when parsing /etc/hosts that are auto-filled by some blocking software ( so around 10k entries there ). In the end solution for us is to introduce caching of match results and creating own map. We use ruby, but I think it is so similar to python, that you get idea.[1]<br>
If you need any explanation, do not hesitate to ask. Speed up in our case is approximately half of time spend at parsing.<br>
<br>
Josef<br>
<br>
[1] <a href="https://github.com/config-files-api/config_files_api/blob/master/lib/cfa/augeas_parser/keys_cache.rb" rel="noreferrer" target="_blank">https://github.com/config-file<wbr>s-api/config_files_api/blob/<wbr>master/lib/cfa/augeas_parser/<wbr>keys_cache.rb</a><br>
</blockquote>
<br>
______________________________<wbr>_________________<br>
augeas-devel mailing list<br>
<a href="mailto:augeas-devel@redhat.com" target="_blank">augeas-devel@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/augeas-devel" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/augeas-devel</a></div></div></blockquote></div><br></div></div></div>