Page MenuHomePhorge

[pykolab][wallace][Debian 12] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 3620: invalid continuation byte
Open, Needs TriagePublic

Description

python 3.11.2

With patch D4625 from T7472 installed, I had:

2023-12-25 10:38:38,149 pykolab.wallace/modules ERROR [1236] - Module 'invitationpolicy' - Unknown error occurred; UnicodeDecodeError('utf-8', b'X-Kolab-From: ecard+xxxx.xxxxatexample.org@coolphotos.de\r\nX-Kolab-To: recipent@kolab.localserver
---snip---
Traceback (most recent call last):

File "/usr/lib/python3/dist-packages/wallace/modules.py", line 127, in execute
  return modules[name]['function'](*args, **kw)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/wallace/module_invitationpolicy.py", line 270, in execute
  message = Parser().parse(open(filepath, 'r'))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/email/parser.py", line 53, in parse
  data = fp.read(8192)
         ^^^^^^^^^^^^^
File "<frozen codecs>", line 322, in decode

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 3620: invalid continuation byte
2023-12-25 10:38:38,152 pykolab.wallace/modules DEBUG [1236] Akzeptiere Nachricht in: '/var/spool/pykolab/wallace/invitationpolicy/incoming/tmpx_owdlk9'
2023-12-25 10:38:38,154 pykolab.wallace DEBUG [1236] Exception in thread
2023-12-25 10:38:38,154 pykolab.wallace DEBUG [1236] Thread-4
2023-12-25 10:38:38,154 pykolab.wallace DEBUG [1236] :
2023-12-25 10:38:38,155 pykolab.wallace DEBUG [1236] Traceback (most recent call last):
2023-12-25 10:38:38,155 pykolab.wallace DEBUG [1236] File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
2023-12-25 10:38:38,172 pykolab.wallace DEBUG [1236] self.run()
2023-12-25 10:38:38,172 pykolab.wallace DEBUG [1236] File "/usr/lib/python3/dist-packages/wallace/__init.py", line 146, in run
2023-12-25 10:38:38,173 pykolab.wallace DEBUG [1236] self.function(*self.args, **self.kwargs)
2023-12-25 10:38:38,174 pykolab.wallace DEBUG [1236] File "/usr/lib/python3/dist-packages/wallace/
init.py", line 487, in pickup_spool_messages
2023-12-25 10:38:38,175 pykolab.wallace DEBUG [1236] pickup_message(filepath, self.modules, module=module, stage=stage)
2023-12-25 10:38:38,175 pykolab.wallace DEBUG [1236] File "/usr/lib/python3/dist-packages/wallace/
init__.py", line 103, in pickup_message
2023-12-25 10:38:38,176 pykolab.wallace DEBUG [1236] cb_action_ACCEPT('wallace', filepath)
2023-12-25 10:38:38,176 pykolab.wallace DEBUG [1236] File "/usr/lib/python3/dist-packages/wallace/modules.py", line 372, in cb_action_ACCEPT
2023-12-25 10:38:38,177 pykolab.wallace DEBUG [1236] message = Parser().parse(open(filepath, 'r'), True)
.....

This caused a looping wallace spool file.
I applied the following patch, which resolved my issue.

diff --git a/wallace/modules.py b/wallace/modules.py
--- a/wallace/modules.py	
+++ b/wallace/modules.py
@@ -365,7 +365,7 @@
     log.debug(_("Accepting message in: %r") %(filepath), level=8)
 
     # parse message headers
-    message = Parser().parse(open(filepath, 'r'), True)
+    message = Parser().parse(open(filepath, 'r', encoding="utf8", errors="ignore"), True)
 
     messageid = message['message-id'] if 'message-id' in message else None
    sender = [formataddr(x) for x in getaddresses(message.get_all('X-Kolab-From', []))]

this is only quick and dirty, within a freezed wallace queue on python 3 only systems with unicode decode byte error

Details

Ticket Type
Task

Event Timeline

Same file opening problem exists in /usr/lib/python3.9/site-packages/wallace/module_invitationpolicy.py - line 270 :-

2024-04-20 08:29:05,594 pykolab.wallace/modules ERROR [1298] tmpl260kkxt Module 'invitationpolicy' - Unknown error occurred; UnicodeDecodeError('utf-8', b'X-Kolab-From: <<--->>', 3311, 3312, 'invalid start byte')
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/wallace/modules.py", line 127, in execute
    return modules[name]['function'](*args, **kw)
  File "/usr/lib/python3.9/site-packages/wallace/module_invitationpolicy.py", line 270, in execute
    message = Parser().parse(open(filepath, 'r'))
  File "/usr/lib64/python3.9/email/parser.py", line 53, in parse
    data = fp.read(8192)
  File "/usr/lib64/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 3311: invalid start byte

Installed package: wallace-0.9.0.6-1.4.el9.kolab_16.noarch