[augeas-devel] augeas in non linux environment and generic configuration files

[David Lutterkort]

On Thu, 2010-08-19 at 00:08 +1000, David Timms wrote:
> Hi, I was considering using augeas to either:
> a) create conforming config files for an embedded system:
> 
> that is: create the files on a linux machine and then transfer to the
> device where they will get used.
> 
> b) actually port augeas to the embedded system to handle config file
> creation, editing, writing, reading:
> 
> How complex/ difficult would it be to implement on a coldfire CPU, where
> code I run is a lowly second class citizen, and has many gotchas and an
> overly simplified c like development language ?

I don't know the development environment at all, so it's hard for me to
say how far you'll get. It's a safe bet though that unless you have a
C99 compiler and support from gnulib, you'll have to do a lot of work.

> How small/large might the code/compiled code be ?

That's hard to say without knowing the env. On my x86_64 machine,
libaugeas takes up 300k, and libfa another 130k.

Your biggest worry though won't be the size of the executable, but the
memory demands during parsing. The regexp matcher that Augeas uses (from
glibc/gnulib) can be a bit of a memory pig, though you can get a handle
on that by being careful what lenses you ship, and how they are written
- that's mostly about avoiding certain constructs like regexp
subtraction.

> The config data I want to store/read represents memory structures (of a
> one-way linked list):
> ArrayA[number of elements] has a number of toplevel fields, then an item
> to indicate count of ArrayB[items] are associated to ArrayA[elements],
> and field to indicate which ArrayB item is associated for count items. A
> similar arrangement is stored for ArrayC[items], but these are
> essentially unrelated to count or index of ArrayB items there are.
> ArrayA[1].BCount = 4
> ArrayA[1].BIndex[1] = 2
> ArrayA[1].BIndex[2] = 3
> ArrayA[1].BIndex[3] = 4
> ArrayA[1].BIndex[4] = 6
> ArrayA[2].BCount = 3
> ArrayA[2].BIndex[1] = 1
> ArrayA[2].BIndex[2] = 5
> ArrayA[2].BIndex[3] = 7
> 
> ArrayB[3].field1=value1
> ArrayB[3].field2=value2
> ArrayB[3].field3=value3
> ArrayB[3].field4=value4
> 
> ArrayA[1].CCount = 2
> ArrayA[1].CIndex[1] = 1
> ArrayA[1].CIndex[2] = 2
> 
> ArrayC[1].field1=value1
> ArrayC[2].field1=value2
> and so on.
> 
> Generic questions:
> 1. what would be the most appropriate config file format for storing
> non-square (non matrix) data like this ?

It sounds to me that the datastructure you're representing here is an
ordered tree - which is exactly what Augeas uses to represent parsed
files.

> 2. having spent some time reviewing various formats, yaml, and to some
> extent json, seem promising. If the first method is used, I'll need to
> be able to suport having the user manually edit the config files with a
> text editor, hopefully in a format that is clear and obvious to adjust,
> and not too picky on actual formatting within the file. Does this
> requirement make a case for using some other file format than the
> previous answer ?

Config files are really user interface - if you anticipate that users
will need to modify the files manually, you should make sure that users
can understand the format and semantics of the entries in the file.

Since it sounds to me that what you are describing is a tree with 2
levels, an inifile would seem the simplest choice; if the tree really
has more than 2 levels, you could go with something generic like YAML or
json, though they aren't terribly easy to modify manually. You could
also mimick the krb5.conf format, which represents a deeper tree - I
wouldn't follow that format outright, but it's a good example of how you
can structure your config into a tree with ley/value pairs at the
leaves.

There's also always XML.

> 3. Are there already augeas lens' for the formats you suggest in 1 and 2 ?

There are lenses for many of the inifile variants, krb5.conf and for
json. Francis is also working on an XML lens.

David