[augeas-devel] Some ideas about how to use Augeas with IPA

[David Lutterkort]

On Tue, 2008-05-13 at 11:39 -0400, Dmitri Pal wrote:
> The internal representation is the tree. Tree is abstracted from lenses. 
> Tree has nodes, labels and values.
> I can write a lenses that will save this tree into the configuration 
> file. Using the same lenses I would be able to restore the tree from the 
> file.
> Now I can create another lenses that will store the same tree in the XML 
> file and restore the tree from this file.
> Now with these two lenses I can convert the data from XML to config file 
> and back.
> Library can't do this at the moment since the tree is associated with 
> only one lenses. To achieve what I need I would have to manually go 
> through the tree and copy nodes from one tree to another using set 
> command. I view it as an overhead. It seems it is much easier to have a 
> "copy tree command" in the library, and/or allow to associate tree with 
> a different lenses. This is only possible and would work if the 
> developer knows that two lenses create same tree.

I am not against supporting this, of course; but I would like to see
some actual working code that demonstrates that before starting to make
big changes, especially since it's not clear how to best support that:
by adding tree move/copy to the API or by associating multiple lenses
with the same tree.

> Another point is that the library works only with file. For our purposes 
> we would need to feed library with a buffer. Library will behave the 
> same way it will parse the data using specified lenses but not from 
> file. This would allow us to accomplish format conversion that we have 
> to do.

Again, this would be much easier to judge with some working code; it
shouldn't be too hard to write some python/ruby/whatever code that
simulates what you do by writing strings to files and working on those
files.

> I do not understand how the regular expressions can change when nodes 
> are inserted or deleted.
> So I do not understand the explanation on why the validation can't be 
> performed.

  let lns =  ([ key /u/ ] . [ key /b/ . store /[0-9]+/ ])
           | ([ key /v/ ] . [ key /b/ . store /[a-z]+/ ])

You can get two kinds of trees from this, one where the value for "b"
matches a number, and one where it matches [a-z]+, all depending on
wherher there is a node labelled "u" or labelled "v" preceding the "b"
node. (The full file to play with that is attached, just run 'augparse
pass_switch_value_regexp.aug')

> I am trying to  optimize things. I do not see a reason why an 
> application would have to create a parallel in memory representation of 
> the tree outside Augeas library. The library already creates a tree. It 
> would be much more valuable to be able to manipulate it directly (which 
> to some extent already exists with the commands) by copying tree, 
> sorting tree, changing order and so on. There are configuration files 
> where order matters but in many others it does not. It is up to the 
> developer to understand the ordering requirements of the data and file 
> and act with it accordingly. Library shall not restrict or enforce the 
> ordering.

I am all for making Augeas easier to use, and meeting your needs better;
but I am loathe to do that without actual code that demonstrates the
issues. Everything you mentioned so far can be achieved with the current
API, even if the API forces you to do it in a suboptimal way. Once there
is such suboptimal code, it becomes much easier to understand what
changes will make that code faster/easier to write etc.

> Out client application that will do a lot of policy merging and conversion can't 
> spend time doing n-square lookups. It will not fly.

Show me the code that's slow, and we'll figure out how to speed it up.
As an example: for libfa, I've spent considerable time improving its
performance; in some cases it meant changing code to reduce its
asymptotic complexity, in a lot of others it was a matter of fiddling
with constants, e.g. reducing number of allocations. Often, what my
intuition of the bottleneck was and what the actual bottleneck was were
very different.

> I understand that you try to keep the library very file focused. Without 
> extending the library into the direction I am talking about we would 
> have to duplicate a lot of what library already does in our code. I 
> think it is not efficient.
> I am not saying that we would not help. I think we need to come to 
> agreement of what can/would be done and depending upon that what can be 
> our involvement.

I am open to any changes that make Augeas more usable, with the caveat
that I'd like to see the problems supported by actual code. The reason I
am being so conservative is that I do not want to end up having to
support a big API where most of it will not be used.

David