[Libosinfo] [PATCH] force a UTF-8 locale for python3 to avoid broken ascii codec

Daniel P. Berrangé berrange at redhat.com
Wed Mar 27 09:57:06 UTC 2019


The python3 ascii codec violates POSIX C locale requirements by not being
8-bit clean in its text handling. It raises an error for any byte with
top bit set

  >       return codecs.ascii_decode(input, self.errors)[0]
  E       UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 419: ordinal not in range(128)

To avoid this python bug we must force use of a UTF-8 locale. Ideally we
would use the C.UTF-8 locale, however, that is not portable across OS,
only existing on certain Linux distros. Instead we use the en_us.UTF-8
locale, but only for the character set data.

Signed-off-by: Daniel P. Berrangé <berrange at redhat.com>
---

Pushed as a CI build fix for FreeBSD distros

 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index c63cb6e..9d7f109 100644
--- a/Makefile
+++ b/Makefile
@@ -123,4 +123,4 @@ update-po:
         done
 
 check: $(DATA_FILES) $(SCHEMA_FILES)
-	$(PYTHON) -m pytest $(PYTEST_LOG_LEVEL)
+	LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) -m pytest $(PYTEST_LOG_LEVEL)
-- 
2.20.1




More information about the Libosinfo mailing list