[Freeipa-users] CSV support in IPA administration tools - to be, or not to be?

[Martin Kosek]

On 01/14/2013 09:09 AM, Petr Viktorin wrote:
> On 01/11/2013 09:57 PM, John Dennis wrote:
>> On 01/11/2013 03:52 PM, Dmitri Pal wrote:
>>> On 01/11/2013 03:27 PM, John Dennis wrote:
>>>> On 01/11/2013 03:10 PM, Dmitri Pal wrote:
>>>>> On 01/10/2013 11:00 AM, John Dennis wrote:
>>>>>> On 01/10/2013 08:15 AM, Petr Spacek wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> is there any user of CSV support built-in to IPA administration tools
>>>>>>> ("ipa"
>>>>>>> command)? Do you consider it sane or even useful? Please reply.
>>>>>>
>>>>>> I've always disliked our use of CSV values on both the command line
>>>>>> and internally. They're just weird, nothing else in Unix works like
>>>>>> this and as you point out below there are easier better alternatives.
>>>>>> Plus with the use of CSV's there is a lot of awkward quoting in a
>>>>>> variety of places.
>>>>>>
>>>>>> On the command line I always thought multiple values should be
>>>>>> specified multiple times and internally they should be encapsulated in
>>>>>> lists rather than parsing a CSV string (if it's logically a list then
>>>>>> why isn't it a list?)
>>>>>>
>>>>>> However at this juncture I'm not sure we can make such a change, we
>>>>>> have a published API that we would be violating. But perhaps we're not
>>>>>> so far down the road we can't make such a change and we're better off
>>>>>> doing it now while there is even a chance. It's not clear to me how
>>>>>> much the command line is being used and specifically with CSV values.
>>>>>>
>>>>>> Do I think CSV's are sane and useful? No. Can we change that? That's a
>>>>>> whole other story.
>>>>>>
>>>>>>
>>>>>>> I wanted to add single TXT record with double quotation marks (")
>>>>>>> inside the
>>>>>>> TXT data.
>>>>>>>
>>>>>>> I spent some time figuring out how it is supposed to work ... and
>>>>>>> with help of
>>>>>>> Petr^3 I managed to write the command.
>>>>>>>
>>>>>>> The resulting command (for BASH) is absolutely crazy:
>>>>>>> ipa dnsrecord-add example.test. newrec --txt-rec='"""created on
>>>>>>> 13:01:23"""'
>>>>>>>
>>>>>>> Do we really need support for this piece of insanity? Shells can do
>>>>>>> the same
>>>>>>> thing with much less pain :-)
>>>>>>>
>>>>>>> IPA with CSV support can add multiple attributes at once, e.g.
>>>>>>> ipa dnsrecord-add example.test. newrec --txt-rec=1,2,3,4,5,6,7,8,9
>>>>>>> will add TXT records with value 1, 2, 3 etc.
>>>>>>>
>>>>>>> BASH can do the same thing (without the escaping hell):
>>>>>>> ipa dnsrecord-add example.test. newrec --txt-rec={1,2,3,4,5,6,7,8,9}
>>>>>>> and
>>>>>>> ipa dnsrecord-add example.test. newrec --txt-rec={1..9}
>>>>>>> BASH would expand to
>>>>>>> ipa dnsrecord-add example.test. newrec --txt-rec=1 --txt-rec=2
>>>>>>> --txt-rec=3
>>>>>>> --txt-rec=4 --txt-rec=5 --txt-rec=6 --txt-rec=7 --txt-rec=8
>>>>>>> --txt-rec=9
>>>>>>>
>>>>>>
>>>>>>
>>>>> Do we already have CSV support?
>>>>> Where is it used?
>>>>> It is not clear to me if BASH example above requires the CSV support or
>>>>> it does expansion on its own. Please explain.
>>>>>
>>>>
>>>> We already have CSV support. It's a mechanism that allows multiple
>>>> values to be passed for one command line argument. The alternate
>>>> approach is rather than having one command line arg that takes
>>>> multiple values is to allow multiple command line args, each one
>>>> taking a single value. This is the UNIX methodology. I believe the
>>>> original thinking was who would want to type out multiple command line
>>>> args, it's too verbose. However the shell expansion illustrated above
>>>> shows how with simple shell syntax one can have succicent args and
>>>> allow the shell to expand them into the preferred verbose form.
>>>>
>>>
>>> So both are already supported and we want to stop using CSV and
>>> deprecate it over time?
>>> This makes sense if there are good examples of how to use bash expansion.
>>> I suggest we create a page and describe preferred method of dealing with
>>> the lists and document it.
>>> Also do the same with the manual, i.e. review it to make sure we do not
>>> show CSV syntax in the docs, same with the man pages.
>>> On the project page we will say that CSV will not be added to the new
>>> and existing commands and will be deprecated over time (probably by IPA
>>> version 4).
>>> Do I get it right?
>>>
>>
>> I'm not sure both are currently supported. I'm not sure we permit
>> multiple args with the same name and aggregate them, I thought that was
>> part of the proposal.
>>
> 
> We do support multiple arguments.
> 
> The trouble with CSV is that to put commas (or in some cases double quotes) in
> the data, they need to be escaped in a way that's not intuitive. The proposal
> is to change this, so that instead of
>     --txt-rec='"""created on 13:01:23"""'
> you can use just
>     --txt-rec='"created on 13:01:23"'
> and instead of
>     --txt-rec='Record one, record two'
> you need to say
>     --txt-rec={'Record one','record two'}
> The assumption is that this makes more sense to a sysadmin, who is much more
> likely to know bash scripting than our flavor of CSV escaping.
> 

Right. I also think this would a good step, there were too many issues related
to CSV parsing in our commands.

As Dmitri said, we would just need to carefully go through our Documentation
and fix all uses of CSV-style. In some cases, people may be confused at the
beginning (I know I will) until they get used to the new {} style, e.g. when
adding self-service permissions:

# ipa selfservice-mod --attrs=street,postalCode,l,c,st,telephoneNumber "Users
manage their own address"

and now they would need to carefully add {...} everywhere to avoid unexpected
issues:

# ipa selfservice-add --permissions={read,write}
--attrs={street,postalCode,l,c,st} "Users manage their own address"

Martin