[Freeipa-devel] [PATCH] convert SAFE_STRING_PATTERN from string to integer sequence to prevent xgettext from aborting when it sees a non-ascii string

[Rob Crittenden]

John Dennis wrote:
> ---
>  ipapython/ipautil.py |   16 +++++++++++++++-
>  ipaserver/ipautil.py |   16 +++++++++++++++-
>  2 files changed, 30 insertions(+), 2 deletions(-)
> 
> diff --git a/ipapython/ipautil.py b/ipapython/ipautil.py
> index 1399c70..3768fa0 100644
> --- a/ipapython/ipautil.py
> +++ b/ipapython/ipautil.py
> @@ -293,7 +293,21 @@ class CIDict(dict):
>  # It was extracted because ipaldap.py is naughtily reaching into the ldif
>  # module and squashing this regexp.
>  #
> -SAFE_STRING_PATTERN = '(^(\000|\n|\r| |:|<)|[\000\n\r\200-\377]+|[ ]+$)'
> +# This is the original definition of SAFE_STRING_PATTERN
> +# SAFE_STRING_PATTERN = '(^(\000|\n|\r| |:|<)|[\000\n\r\200-\377]+|[ ]+$)'
> +#
> +# But the regular expression string is neither ASCII nor UTF-8
> +# which causes xgettext to fail because xgettext expects all strings it 
> +# finds in a file to be either ASCII or UTF-8. To prevent xgettext from
> +# aborting when it sees this string we build it from integers using this code:
> +#
> +# print "SAFE_STRING_PATTERN = \\\n'%s' %% \\\n(%s)" % \
> +# ("%c" * len(SAFE_STRING_PATTERN), 
> +# ",".join(["%d" % ord(c) for c in SAFE_STRING_PATTERN]))
> +
> +SAFE_STRING_PATTERN = \
> +'%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c' % \
> +(40,94,40,0,124,10,124,13,124,32,124,58,124,60,41,124,91,0,10,13,128,45,255,93,43,124,91,32,93,43,36,41)
>  safe_string_re = re.compile(SAFE_STRING_PATTERN)
>  
>  def needs_base64(s):
> diff --git a/ipaserver/ipautil.py b/ipaserver/ipautil.py
> index 7042c1c..968d392 100644
> --- a/ipaserver/ipautil.py
> +++ b/ipaserver/ipautil.py
> @@ -125,7 +125,21 @@ class CIDict(dict):
>  # It was extracted because ipaldap.py is naughtily reaching into the ldif
>  # module and squashing this regexp.
>  #
> -SAFE_STRING_PATTERN = '(^(\000|\n|\r| |:|<)|[\000\n\r\200-\377]+|[ ]+$)'
> +# This is the original definition of SAFE_STRING_PATTERN
> +# SAFE_STRING_PATTERN = '(^(\000|\n|\r| |:|<)|[\000\n\r\200-\377]+|[ ]+$)'
> +#
> +# But the regular expression string is neither ASCII nor UTF-8
> +# which causes xgettext to fail because xgettext expects all strings it 
> +# finds in a file to be either ASCII or UTF-8. To prevent xgettext from
> +# aborting when it sees this string we build it from integers using this code:
> +#
> +# print "SAFE_STRING_PATTERN = \\\n'%s' %% \\\n(%s)" % \
> +# ("%c" * len(SAFE_STRING_PATTERN), 
> +# ",".join(["%d" % ord(c) for c in SAFE_STRING_PATTERN]))
> +
> +SAFE_STRING_PATTERN = \
> +'%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c%c' % \
> +(40,94,40,0,124,10,124,13,124,32,124,58,124,60,41,124,91,0,10,13,128,45,255,93,43,124,91,32,93,43,36,41)
>  safe_string_re = re.compile(SAFE_STRING_PATTERN)
>  
>  def needs_base64(s):

Is this code used by anything any more? This was part of the old XML-RPC 
server. It was used to determine whether a data type needed to be the 
XML-RPC Binary type or not. In v2 Jason wrote a similar function that 
bases the output type based on the python type (strings are unicode, 
binary data is str).

So I think this can just be dropped (after testing to confirm, of course).

rob