Page MenuHomePhorge
Create Task
Maniphest T2582

Possibly a bug with wallace finding user personal folders, that probably makes emails stuck in wallace queue
Closed, ResolvedPublic
Actions

Description

While investigating why some messages are stuck in Wallace queue I noticed:

2017-07-20 05:35:11,275 pykolab.wallace ERROR Unknown error occurred; readonly('Other Users/smith/Calendar is not writable',)
2017-07-20 05:35:11,276 pykolab.wallace ERROR Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/wallace/modules.py", line 118, in execute
    return modules[name]['function'](*args, **kw)
  File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", line 367, in execute
    done = processor_func(itip_event, policy, recipient_email, sender_email, receiving_user)
  File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", line 599, in process_itip_reply
    propagate_changes_to_attendees_accounts(existing, updated_attendees)
  File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", line 1377, in propagate_changes_to_attendees_accounts
    (attendee_object, master_object) = find_existing_object(object.uid, object.type, recurrence_id, attendee_user, True)  # does IMAP authenticate
  File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", line 847, in find_existing_object
    imap.imap.m.select(imap.folder_utf7(folder))
  File "/usr/lib64/python2.7/imaplib.py", line 671, in select
    raise self.readonly('%s is not writable' % mailbox)
readonly: Other Users/smith/Calendar is not writable

Going along with code excution path:
File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", function find_existing_object has the following comment:

def find_existing_object(uid, type, recurrence_id, user_rec, lock=False):
    """
        Search user's private folders for the given object (by UID+type)
    """
    .....

but the error clearly indicates that the folder is not users personal / private folder, but shared to the user by the other user.
Moving further, function list_user_folders is defined as:

def list_user_folders(user_rec, type):
    """
        Get a list of the given user's private calendar/tasks folders
    """

and part of this function looks like:

for folder in folders:
    # exclude shared and other user's namespace
    if ns_other is not None and folder.startswith(ns_other) and '_delegated_mailboxes' in user_rec:
        # allow shared folders from delegators
        if len([_mailbox for _mailbox in user_rec['_delegated_mailboxes'] if folder.startswith(ns_other + _mailbox + '/')]) == 0:
            continue
    # TODO: list shared folders the user has write privileges ?
    if ns_shared is not None and len([_ns for _ns in ns_shared if folder.startswith(_ns)]) > 0:
        continue

I'm not sure yet what _delegated_mailboxes is in this case, but first of all it looks to me that function list_user_folders gets not just private folders, but also other users folders and that leads to wallace to stop on error and email never leaves wallace queue.

Other information:

# kolab lam user/smith/Calendar@domain.tld
Folder user/smith/Calendar@domain.tld
  lr            anyone
  lrswipkxtecdan smith@domain.tld
  lr            anonymous
wallace-0.8.7-1.1.el7.kolab_16.noarch

Details

Ticket Type
Task

Revisions and Commits

rP pykolab
D485rP7ee0bb402496 Address inefficiencies and possible misfunction over delegated mailboxes in…

Event Timeline

adomaitis created this task.Jul 20 2017, 6:21 AM
adomaitis added a project: PyKolab.
adomaitis added a subscriber: PyKolab Developers.
machniak subscribed.Jul 20 2017, 6:35 AM

Is there delegation setup between these users? For me it looks like there is, but delagtor's calendar folder is non writable and the code does not check that trying to write to it.

machniak added a comment.Jul 20 2017, 6:38 AM

Or to be more precise (according to the traceback). It does not even tries to write to the folder, but just tries to select it (and then search for an object). I'm not sure why select fails while the folder is 'lr'.

adomaitis added a comment.Jul 20 2017, 6:38 AM

No, user smith@domain.tld doesn't have any delegates in LDAP.

machniak added a subscriber: vanmeeuwen.Jul 20 2017, 7:21 AM

Now I see where's the problem:

  1. @vanmeeuwen, a side of the main reason of the issue (see below). There must be some bug somewhere because imap SELECT on a folder that has 'lr' for anyone should not fail.
  2. If you consider this code
# exclude shared and other user's namespace
if ns_other is not None and folder.startswith(ns_other) and '_delegated_mailboxes' in user_rec:
    # allow shared folders from delegators
    if len([_mailbox for _mailbox in user_rec['_delegated_mailboxes'] if folder.startswith(ns_other + _mailbox + '/')]) =
        continue
# TODO: list shared folders the user has write privileges ?
if ns_shared is not None and len([_ns for _ns in ns_shared if folder.startswith(_ns)]) > 0:
    continue

You will see that if user_rec['_delegated_mailboxes'] is not set (which is the case here) no other user folders will be excluded. So, my proposed fix is:

--- a/wallace/module_invitationpolicy.py
+++ b/wallace/module_invitationpolicy.py
@@ -796,7 +796,9 @@ def list_user_folders(user_rec, type):
 
     for folder in folders:
         # exclude shared and other user's namespace
-        if ns_other is not None and folder.startswith(ns_other) and '_delegated_mailboxes' in user_rec:
+        if ns_other is not None and folder.startswith(ns_other):
+            if '_delegated_mailboxes' not in user_rec:
+                continue
             # allow shared folders from delegators
             if len([_mailbox for _mailbox in user_rec['_delegated_mailboxes'] if folder.startswith(ns_other + _mailbox + '/')]) == 0:
                 continue
machniak assigned this task to vanmeeuwen.Jul 20 2017, 7:32 AM
machniak added a comment.EditedJul 20 2017, 7:35 AM

ps. there's possible performance optimization in find_user_folders: don't use imap.list_folders('*'), instead depend on imap.get_metadata('*'), which we do anyway. But of course we should ask for the metadata we need, not all of them.

adomaitis added a comment.Jul 20 2017, 8:13 AM

I have changed the acls for anyone to lrs and that did the trick - don't see errors anymore. Wallace messages were dispatched.

vanmeeuwen added a revision: D485: Address inefficiencies and possible misfunction over delegated mailboxes in other user namespace.Jul 21 2017, 10:16 AM
vanmeeuwen closed this task as Resolved by committing rP7ee0bb402496: Address inefficiencies and possible misfunction over delegated mailboxes in….Jul 22 2017, 11:07 AM
vanmeeuwen added a commit: rP7ee0bb402496: Address inefficiencies and possible misfunction over delegated mailboxes in….
pasik subscribed.May 11 2018, 8:23 AM