[augeas-devel] Lens Help: disambiguating 1] regex difference 2] overlapping lenses in union.put
Mol, Xavier (SCC)
xavier.mol at kit.edu
Mon Jan 11 09:48:40 UTC 2016
Hi Raphael,
thank you for your kind words. I took the liberty to start a new page on the Augeas GitHub wiki: https://github.com/hercules-team/augeas/wiki/List-of-frequent-errors.
Ciao,
Xavier.
From: Raphaël Pinson [mailto:raphael.pinson at camptocamp.com]
Sent: Monday, January 11, 2016 9:18 AM
To: Mol, Xavier (SCC)
Cc: Yclept Nemo; augeas-devel
Subject: Re: [augeas-devel] Lens Help: disambiguating 1] regex difference 2] overlapping lenses in union.put
Yes Xavier, this is a good (and useful) explanation.
It would be great if that could end up on a wiki page actually.
Raphaël
On Fri, Jan 8, 2016 at 2:24 PM, Mol, Xavier (SCC) <xavier.mol at kit.edu> wrote:
Hi Yclept,
this is a late reply and you possibly don't need it anymore, but I think I have finally understood the issue regarding "overlapping lenses in union.put", so I want to share my experience with you.
As you and David explained, Augeas has trouble when putting a tree into a file that (potentially) has nodes, which can be matched by more than one lens. This problem can be avoided in get direction, because parent nodes might be distinguishable in some way. We can make use of that key difference, by giving alternate node options to Augeas (as opposed to alternate expressions).
Say we have a tree
/a
/a/color = "blue"
/b
/b/color = "red"
Depending on the lens definitions for nodes "a" and "b", the "color" subnode might have to be read/written differently. Yet just from looking at the tree, Augeas cannot distinguish between either "color" node definition. That is what David said in his mail: "when Augeas sees a lens construct like 'l1|l2' it only looks at the labels of the tree node it's currently at to decide whether to use l1 or l2". In this situation, the label is identical for both nodes, hence the conflict.
How can we avoid this problem with actually applying ugly hacks? We change the construct such that Augeas has to consider the depth of the tree. Without concrete examples, this will be difficult to understand, so lets expand on the above example with these lens definitions:
let a = key "a" . [ label "color" . store /red|blue|green|yellow/ ]
let b = key "b" . [ label "color" . del /dark-/ "dark-" . store /red|blue|green|yellow/ ]
let lns = [ a | b ]
So for "b"-nodes color strings are prefixed with "dark-". When parsing a source file, Augeas can clearly distinguish between when to apply lenses "a" or "b". But because in either case the resulting "color"-node is identical, saving the tree to a file fails. We make Augeas aware of the difference on node-level, by alternating between nodes:
let lns = [ a ]|[ b ]
I hope this was somewhat understandable and helpful to you (or others finding this mail with Google J ).
Ciao,
Xavier.
From: augeas-devel-bounces at redhat.com [mailto:augeas-devel-bounces at redhat.com] On Behalf Of Yclept Nemo
Sent: Friday, October 23, 2015 7:07 AM
To: augeas-devel
Subject: [augeas-devel] Lens Help: disambiguating 1] regex difference 2] overlapping lenses in union.put
Hi,
I'm working on a lens that has two errors:
>> 1 (*
>> 2 # example configuration follows:
>> 3 # default = value
>> 4 key = value
>> 5
>> 6 # default = value # inline comment
>> 7 key = value # inline comment
>> 8 *)
>> 9
>> 10 (*
>> 11 # key/value formats:
>> 12
>> 13 keys
>> 14 [a-z_]
>> 15
>> 16 values
>> 17 boolean: true/false
>> 18 int 0, 100
>> 19 real 3.0
>> 20 hex int ffeedd
>> 21 string "..."
>> 22 unquoted string sdl, f, f2
>> 23 *)
>> 24
>> 25 (*
>> 26 # quoted string regex scratchpad
>> 27 quoted string regex =
>> 28 good:
>> 29 "asdf\"qwer"
>> 30 "asdf\\qwer"
>> 31 "asdf\\\\\"qwer"
>> 32 bad:
>> 33 "asdf\\"qwer"
>> 34 "asdf\\\\\\"qwer"
>> 35
>> 36 /.*/ - /.*(\\)*[^\] <file:///\\)*[%5e\%5d%22.*\> ".*/
>> 37
>> 38 /([^"](\")?)*/
>> 39 *)
>> 40
>> 41 module ConfigObj =
>> 42 let _ = print_regexp /"/
>> 43 let _ = print_string "\n"
>> 44 let var0 = "\""
>> 45 let _ = print_string (var0 . "\n")
>> 46 let _ = print_string "\n"
>> 47
>> 48 let var1 = "\"asdf\\\\\\\\"qwer\""
>> 49 let _ = print_string (var1 . "\n")
>> 50 let _ = print_string ((regexp_match (/\".*\"/ - /\"(.*[^\])?([\]{2})*".*\"/) var1) . "\n")
>> 51
>> 52 let option = key /[a-z]([a-z_]*[a-z])?/
>> 53 let value_bool = store /(true|false)/ . [ label "type" . value "bool" ]
>> 54 let value_int = store /(0|[1-9][0-9]*)/ . [ label "type" . value "int" ]
>> 55 let value_decimal = store ((lens_ctype value_int) . /\.[0-9]+/) . [ label "type" . value "decimal" ]
>> 56 let value_string = (del "\"" "\"") . (store (/.*/ - /(.*[^\]|)([\]{2})*".*/)) . (del "\"" "\"") . [ label "type" . value "string" ]
>> 57 let value_all = value_bool | value_int | value_decimal
>> 58 let sep_assign = del /[ \t]*=[ \t]*/ " = "
>> 59 let sep_comment = del /[ \t]*#[ \t]*/ " # "
>> 60 let record_comment = [ label "comment" . sep_comment . store /([^ \t\n].*)?/ ]
>> 61 let record = option . sep_assign . value_all . record_comment?
>> 62 let record_not = label "comment" . store (/([^ \t\n].*)?/ - lens_ctype record)
>> 63 let comment = del /#[ \t]*/ "# " . ( [ record . [ label "commented" ] ] | [ record_not ] )
>> 64 let line = del /[ \t]*/ "" . ( comment | [ record ] ) . del "\n" "\n"
>> 65 let lns = line *
The first is a regex problem: I want to match double-quoted c-style strings, ie strings with escape sequences such as "\"", "\\\"" and so on. The regex I've crafted (line#50) evidently fits the bill - try modifying the input test values (line#48) to verify - so I don't understand why the derived lens (line#56) is failing, and thereby causing this "ambiguous concatenation" error:
>> Syntax error in lens definition
>> configobj.aug:61.4-.75:Failed to compile record
>> configobj.aug:61.26-.75:exception: ambiguous concatenation
>> First regexp: /([a-z]([a-z_]*[a-z])?)([ \t]*=[ \t]*)((((true|false)))|(((0|[1-9][0-9]*)))|(((((0|[1-9][0-9]*)))(\\.[0-9]+)))|((")(([^\n"\\][^\n"\\]*\\\\|\\\\)([^\n\\][^\n"\\]*\\\\)*\\\\(([^\n"\\][^\n"\\]*\\\\|\\\\)([^\n\\][^\n"\\]*\\\\)*\\\\)*(([^\n"\\][^\n"\\]*\\\\|\\\\)([^\n\\][^\n"\\]*\\\\)*([^\n\\][^\n"\\]*|)|[^\n"\\][^\n"\\]*|)|([^\n"\\][^\n"\\]*\\\\|\\\\)([^\n\\][^\n"\\]*\\\\)*([^\n\\][^\n"\\]*|)|[^\n"\\][^\n"\\]*|)(")))/
>> Second regexp: /(([ \t]*#[ \t]*)(([^ \t\n].*)?))?/
>> 'a="\\"#"' can be split into
>> 'a="\\"|=|#"'
>>
>> and
>> 'a="\\"#"|=|' <----- not possible per the regex
>>
>> First lens: configobj.aug:61.26-.57:
>> Second lens: configobj.aug:61.60-.75:
The second problem - you'll see if the first is fixed - is a limitation of augeas' tree-text conversion, the inability to differentiate trees on depth. This generates the error "exception: overlapping lenses in tree union.put" and crops up in two locations:
'[ record . [ label "commented" ] ]' (line#63) ----vs---- '[ record_not ]' (line#63)
::: The "record_not" label of "comment" is also a possible valid label of the "record" lens [line#52]. Hacky workaround is to change the label to an invalid key, ie "comment2", etc.
'[ record . [ label "commented" ] ]' (line#63) ----vs---- '[ record ]' (line#64)
::: Unlike previous, no hacky workaround.
Obviously this can be fixed by scanning the *depth* of the tree. In both cases, one branch is guaranteed to have a subnode with the label "commented" unlike the other. While augeas doesn't implement this, there was a suggestion on the mailing list regarding a lens-based workaround[1] that I didn't understand. I'd appreciate if someone could provide a concrete example for my case.
Sincerely,
[1] https://www.redhat.com/archives/augeas-devel/2009-June/msg00088.html
_______________________________________________
augeas-devel mailing list
augeas-devel at redhat.com
https://www.redhat.com/mailman/listinfo/augeas-devel
--
Raphaël Pinson
Infrastructure Developer & Training Leader
+33 458 482 013
Camptocamp France
Savoie Technolac
BP 352
48, avenue du Lac du Bourget
73372 Le Bourget du Lac, Cedex
www.camptocamp.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/augeas-devel/attachments/20160111/4c51e07f/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6401 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/augeas-devel/attachments/20160111/4c51e07f/attachment.p7s>
More information about the augeas-devel
mailing list