[augeas-devel] multiline inifile values

[Kent Tenney]

Raphaël,

I copied php.aug to buildout.aug and changed
let entry = IniFile.entry
to
let entry = IniFile.entry_multiline

Now I can write buildout.cfg files which I can manage with with Augeas.
Great!

FYI, there are some idioms allowed by Python's configparser
which break the lens.

[buildout]
parts =
    a
    b

#    c
    d
my var with spaces = legal

results in:
{'buildout': { 'parts': '\na\nb\nd',  'my var with spaces': 'legal'}}

1. blank lines are allowed in the multiline value
2. a comment on the first column hides the line
3. spaces are allowed in keys

Thanks,
Kent

2011/6/8 Raphaël Pinson <raphink at gmail.com>:
> I've actually committed the patch already since it doesn't break
> current functionalities, so you can grab the files from the git
> repository:
> inifile.aug is at [0] and tests/test_inifile.aug is at [1].
> Now to benefit from this patch, you'll have to actually make use of
> the new IniFile.entry_multiline definition, which can be used the same
> way as IniFile.entry. test_inifile.aug contains a test which can serve
> as an example:
>
>  (* TEST multiline values *)
>  let multiline_test = "test_ace = val1\n  val2\n   val3\n"
>  let multiline_ace = IniFile.entry_multiline IniFile.entry_re sep_ace
> comment_ace
>  test multiline_ace get multiline_test =
>      { "test_ace" = "val1\n  val2\n   val3" }
>
>
> It shows that the string ""test_ace = val1\n  val2\n   val3\n" is now
> parsed as  { "test_ace" = "val1\n  val2\n   val3" }.
>
>
> Raphaël
>
>
> [0] http://git.fedorahosted.org/git/?p=augeas.git;a=blob_plain;f=lenses/inifile.aug;h=b10652aba00e7725df4d1108da5fb8ac8842c3d3;hb=289ef13aaf9b0f0556413c2da8b809d2be966802
> [1] http://git.fedorahosted.org/git/?p=augeas.git;a=blob_plain;f=lenses/tests/test_inifile.aug;hb=289ef13aaf9b0f0556413c2da8b809d2be966802
>
> 2011/6/8 Kent Tenney <ktenney at gmail.com>
>>
>> Raphaël,
>>
>> I appreciate the quick response.
>> Sadly my understanding of Augeas at any level below
>> simple augtool commands is next to nil.
>>
>> I applied the patch and get undefined variable for
>> sep_ace, comment_ace, and multiline_ace
>>
>> I commented out that test section, and those messages went away.
>>
>> I'm running the following script
>>
>> #! /usr/bin/augtool -Asif
>> set /augeas/load/IniFile/lens IniFile.lns
>> set /augeas/load/IniFile/incl /.../buildout.cfg
>> load
>>
>> and getting
>> "Can not find lens IniFile.lns"
>>
>> Clearly, I don't know what I'm doing, or what I should do.
>> I've been studying doc, but it's slow going.
>>
>> Thanks,
>> Kent
>>
>> 2011/6/8 Raphaël Pinson <raphink at gmail.com>:
>> > Alright, here is a patch to implement this:
>> >
>> >
>> > diff --git a/lenses/inifile.aug b/lenses/inifile.aug
>> > index aadb3c3..cda79b6 100644
>> > --- a/lenses/inifile.aug
>> > +++ b/lenses/inifile.aug
>> > @@ -79,11 +79,20 @@ Variable: sto_to_eol
>> >  let sto_to_eol         = Sep.opt_space . store Rx.space_in
>> >
>> >  (*
>> > +Variable: to_comment_re
>> > +  Regex until comment
>> > +*)
>> > +let to_comment_re = /[^;# \t\n][^;#\n]*[^;# \t\n]|[^;# \t\n]/
>> > +
>> > +(*
>> >  Variable: sto_to_comment
>> >    Store until comment
>> >  *)
>> > -let sto_to_comment     = Sep.opt_space
>> > -                         . store /[^;# \t\n][^;#\n]*[^;# \t\n]|[^;# \t\n]/
>> > +let sto_to_comment = Sep.opt_space . store to_comment_re
>> > +
>> > +let sto_multiline = Sep.opt_space
>> > +         . store (to_comment_re
>> > +               . (/[ \t]*\n/ . Rx.space . to_comment_re)*)
>> >
>> >
>> >  (* Group: Define comment and defaults *)
>> > @@ -140,6 +149,11 @@ View: entry
>> >  let entry (kw:regexp) (sep:lens) (comment:lens)
>> >                         = [ key kw . sep . sto_to_comment? . (comment|eol) ]
>> > | comment
>> >
>> > +
>> > +let entry_multiline (kw:regexp) (sep:lens) (comment:lens)
>> > +                       = [ key kw . sep . sto_multiline? . (comment|eol) ]
>> > | comment
>> > +
>> > +
>> >  (*
>> >  View: indented_entry
>> >    Generic INI File entry that might be indented with an arbitrary
>> > diff --git a/lenses/tests/test_inifile.aug b/lenses/tests/test_inifile.aug
>> > index 6155e30..a87b504 100644
>> > --- a/lenses/tests/test_inifile.aug
>> > +++ b/lenses/tests/test_inifile.aug
>> > @@ -203,4 +203,12 @@ test_bdf =
>> >           { "test_bdf" } }
>> >
>> >
>> > +  (* TEST multiline values *)
>> > +  let multiline_test = "test_ace = val1\n  val2\n   val3\n"
>> > +  let multiline_ace = IniFile.entry_multiline IniFile.entry_re sep_ace
>> > comment_ace
>> > +  test multiline_ace get multiline_test =
>> > +      { "test_ace" = "val1\n  val2\n   val3" }
>> > +
>> > +
>> > +
>> >
>> >
>> >
>> > Let me know if that fits your need.
>> >
>> >
>> > Raphaël
>> >
>> >
>> >
>> > 2011/6/8 Kent Tenney <ktenney at gmail.com>
>> >>
>> >> A quick test of a buildout shows that a multiline value
>> >> becomes a string with embedded newlines, and degree
>> >> of indentation doesn't matter:
>> >>
>> >> [buildout]
>> >> parts =
>> >>  a
>> >>  b
>> >>   c
>> >>
>> >> results in
>> >> ... 'parts': '\na\nb\nc'
>> >>
>> >> Thanks,
>> >> Kent
>> >>
>> >> 2011/6/8 Raphaël Pinson <raphink at gmail.com>:
>> >> > Hi Kent,
>> >> >
>> >> > On Tue, Jun 7, 2011 at 9:58 PM, Kent Tenney <ktenney at gmail.com> wrote:
>> >> >>
>> >> >> Howdy,
>> >> >>
>> >> >> from inifile.aug
>> >> >>
>> >> >> "About: TODO
>> >> >>  Things to add in the future
>> >> >>  - Support double quotes in value
>> >> >>  - Support multiline values (is it standard?)"
>> >> >>
>> >> >> I don't know if multiline values are standard or not,
>> >> >> but they are part of the Python configparser dialect of ini.
>> >> >> http://docs.python.org/library/configparser.html
>> >> >>
>> >> >> zc.buildout is a very widely used tool for managing
>> >> >> Python software installations, it is based on ini files
>> >> >> with multiline values. It would be great if Augeas
>> >> >> understood them. If I could provide a patch I would ...
>> >> >>
>> >> >
>> >> > The hard part about multiline values in IniFile is their format:
>> >> > indented
>> >> > values with a constant indentation level.
>> >> > I've scratched my head in the past to see how Augeas could parser and
>> >> > represent that.
>> >> > The first part is how to represent them. Should multiline values be
>> >> > stored
>> >> > in one node with "\n" characters (and indentations), or should they be
>> >> > split
>> >> > into several nodes?
>> >> > The second question is how to modify a multiline value, namely how do we
>> >> > keep the same indentation level for new lines in Augeas, since Augeas
>> >> > cannot
>> >> > remember (afaik) the indentation level of previous lines?
>> >> >
>> >> > Maybe David would have a clue on this.
>> >> >
>> >> > Regards
>> >> > Raphaël
>> >
>> >
>