[augeas-devel] Creating a custom lense
David Lutterkort
dlutter at redhat.com
Wed Apr 23 20:14:28 UTC 2008
On Wed, 2008-04-23 at 19:05 +0100, Dean Wilson wrote:
> After David fixed my configure on Debian issues I thought I should
> make myself useful. Instead I've got more questions :)
Seeing how welltested Augeas is at this point, questions are very
useful.
> First - when I change one of the literal values in the
> test_aptsource.aug and run augparse aptsources.aug
> tests/test_aptsource.aug I don't get failures. Should I? Or am I not
> invoking the tests correctly?
The invocation should be
augparse -I SOME_DIR tests/test_aptsource.aug
where SOME_DIR contains your aptsources.aug.
When Augeas interprets test_aptsource.aug and sees an identifier like
'Aptsources.lns' it knows that that is defined in a module Aptsources,
which in turn must be in a file 'aptsources.aug'. It then searches its
include path for 'aptsources.aug' and loads that file. With the -I
switch, you add directories to the search path (similar to -I for gcc)
When I ran augparse on the files you sent, there were a few small
problems:
* The 'simple_source' and 'multi_components' string was missing an
'\n' at the end
* The regexps used some constructs that are not supported; in
particular, you can't use '^' to anchor to the beginning of the
line (it's never needed in lenses, anyway, since they need to
match the entire string) and you can't use '\s' as an
abbreviation for '[ \t]' - for better or worse, the regexps use
extended POSIX syntax (if you want to see them in all their
glory, have a look at regex(7))
* The definition of sep_ws was enclosed in '[ .. ]' which created
a bunch of spurious nodes with NULL labels (the reason I use
that construct for toplevel comments in hosts.aug and similar is
a little subtle, and has to do with how comments are preserved
when entries are deleted; it's probably the most arcane aspect
of writing lenses. I need to describe that better ;)
* the test on line 14 was expecting that the 'components' node has
children '1', '2', '3', but the lens just creates three
'components' nodes. I just changed the expected value for that
test
*
> Second - I tried to add a hander for blank lines but when I try to use
> it in "let lns" in the aptsources.aug file I get -
>
> aptsources.aug:21.1-.41:Failed to compile lns
> aptsources.aug:21.13-.28:exception: useless tree union: the first lens
> completely shadows the second lens
> First lens: aptsources.aug:10.16-.37
> Second lens: aptsources.aug:11.16-.48
>
> I've seen lenses built of more than one element in the examples so I'm
> guessing I've got something else wrong.
What that means is that you have a construct 'L1|L2' in there, where in
the transformation from tree -> file, L2 will never get used, because L1
already matches everything that L2 could match. The error is about the
'put' direction from tree -> file because it says 'useless _tree_
union'; if it were in the 'get' direction from file -> tree it would
just say 'useless union' (should probably sprinkle the words 'get' and
'put' into those error messages) [1]
Specifically, in aptsources.aug, where you had
let blank = [ del /^\s*\n/ "\n" ]
let comment = [ del /(^\s*\n)|(^#.*\n)/ "# " ]
let lns = ( blank | comment | record )
both the blank and comment lens produce tree nodes with a NULL label
(which is fine) When Augeas then goes to transform the tree back to a
file with 'lns', and encounters a node with a NULL label, it can't
decide whether it should use the 'blank' or the 'comment' lens to do
that, because they both match such a node. And that's what augparse is
whining about.
The way you fixed it is the right one: just have one lens 'comment' that
matches actual comments and blank lines.
> Third -
>
> I can't write newly added apt-sources back out to disk -
>
> set /files/etc/apt/sources.list/4/type deb
> set /files/etc/apt/sources.list/4/uri ftp://mirror.bytemark.co.uk/debian/
> set /files/etc/apt/sources.list/4/distribution etch/updates
> set /files/etc/apt/sources.list/4/component[1] main
> set /files/etc/apt/sources.list/4/component[2] notmain
>
> augtool> save
> /usr/share/augeas/lenses/aptsources.aug:15.17-19.22:Short split for concat
With that, I am going for most arcane error message, ever. It happens
when Augeas tries to write a tree back to a file and the tree is missing
some nodes it is expecting. I definitely need to fix the wording of that
error message.
I am not sure though why it is complaining in your example above. It
looks right (may be a bug)
> Any pointers or help for any of these three is much appreciated.
I attach your files with changes that at least make the tests with
augparse pass; I'll have a look at the 'short split' issue after lunch.
David
[1] Nate Foster, the main author of Boomerang, has pointed out to me
that the check for union is actually too weak; I'll have to fix that
fairly soon.
-------------- next part --------------
(* Parsing /etc/apt/sources.list *)
module Aptsources =
autoload xfm
let sep_ws = del /[ \t]+/ " "
let eol = Util.del_str "\n"
let blank = [ del /[ \t]*\n/ "\n" ] (* not used *)
let comment = [ del /([ \t]*\n)|(#.*\n)/ "# " ]
let word = /[^# \n\t]+/
let record = [ seq "source" . [ label "type" . store word ] . sep_ws .
[ label "uri" . store word ] . sep_ws .
[ label "distribution" . store word ] .
[ label "component" . sep_ws . store word ]* .
(del /[ \t]*#.*/ "") ?
. eol ]
(* let lns = ( blank | comment | record ) * didn't work *)
let lns = ( comment | record ) *
let filter = (incl "/etc/apt/sources.list") . (incl "/etc/apt/sources.list.d/*")
. Util.stdexcl
let xfm = transform lns filter
(* Local Variables: *)
(* mode: caml *)
(* End: *)
-------------- next part --------------
module Test_aptsource =
let simple_source = "deb ftp://mirror.bytemark.co.uk/debian/ etch main\n"
let multi_components = "deb http://security.debian.org/ etch/updates main contrib non-free\n"
test Aptsources.lns get simple_source =
{ "1"
{ "type" = "deb" }
{ "uri" = "ftp://mirror.bytemark.co.uk/debian/" }
{ "distribution" = "etch" }
{ "component" = "main" }
}
test Aptsources.lns get multi_components =
{ "1"
{ "type" = "deb" }
{ "uri" = "http://security.debian.org/" }
{ "distribution" = "etch/updates" }
{ "component" = "main" }
{ "component" = "contrib" }
{ "component" = "non-free" }
}
let multi_line = "#deb http://www.backports.org/debian/ sarge postfix
# deb http://people.debian.org/~adconrad sarge subversion
deb ftp://mirror.bytemark.co.uk/debian/ etch main non-free contrib
deb http://security.debian.org/ etch/updates main contrib non-free # security line
deb-src http://mirror.bytemark.co.uk/debian etch main contrib non-free\n"
test Aptsources.lns get multi_line =
{} {} {}
{ "1"
{ "type" = "deb" }
{ "uri" = "ftp://mirror.bytemark.co.uk/debian/" }
{ "distribution" = "etch" }
{ "component" = "main" }
{ "component" = "non-free" }
{ "component" = "contrib" }
}
{ "2"
{ "type" = "deb" }
{ "uri" = "http://security.debian.org/" }
{ "distribution" = "etch/updates" }
{ "component" = "main" }
{ "component" = "contrib" }
{ "component" = "non-free" }
}
{ "3"
{ "type" = "deb-src" }
{ "uri" = "http://mirror.bytemark.co.uk/debian" }
{ "distribution" = "etch" }
{ "component" = "main" }
{ "component" = "contrib" }
{ "component" = "non-free" }
}
(* Local Variables: *)
(* mode: caml *)
(* End: *)
More information about the augeas-devel
mailing list