Comments in T2166: wallace fails to read resource calendar might be related.

I was able to bypass the issue in debug mode 8, by setting self.LOGFD = open('/var/log/kolab/cyruslib.log', 'a') around line 319 in cyruslib/py library. Now wallace successfully parse the message and resource calendar and accepts the invitation.
The same trick gives different results in debug level 9:

2017-01-05 16:23:43,776 pykolab.wallace ERROR Module resources.heartbeat() failed with error: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/wallace/__init__.py", line 89, in modules_heartbeat
    modules.heartbeat(module, lastrun)
  File "/usr/lib/python2.7/site-packages/wallace/modules.py", line 128, in heartbeat
    return modules[name]['heartbeat'](*args, **kw)
  File "/usr/lib/python2.7/site-packages/wallace/module_resources.py", line 442, in heartbeat
    imap.connect()
  File "/usr/lib/python2.7/site-packages/pykolab/imap/__init__.py", line 170, in connect
    self._imap[hostname].login(admin_login, admin_password)
  File "/usr/lib/python2.7/site-packages/pykolab/imap/cyrus.py", line 142, in login
    cyruslib.CYRUS.login(self, *args, **kw)
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 419, in login
    self.__doexception("LOGIN", error)
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 359, in __doexception
    self.__doraise( function.upper(), msg )
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 368, in __doraise
    raise CYRUSError( idError[0], mode, msg )
CYRUSError: (10, 'LOGIN', '[Errno 9] Bad file descriptor')

and

2017-01-05 16:25:10,389 pykolab.imap DEBUG [13257]: Logging on to Cyrus IMAP server localhost
2017-01-05 16:25:10,422 pykolab.wallace ERROR Unknown error occurred; CYRUSError(10, 'LOGIN', '[Errno 32] Broken pipe')
2017-01-05 16:25:10,423 pykolab.wallace ERROR Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/wallace/modules.py", line 118, in execute
    return modules[name]['function'](*args, **kw)
  File "/usr/lib/python2.7/site-packages/wallace/module_invitationpolicy.py", line 363, in execute
    imap.connect()
  File "/usr/lib/python2.7/site-packages/pykolab/imap/__init__.py", line 170, in connect
    self._imap[hostname].login(admin_login, admin_password)
  File "/usr/lib/python2.7/site-packages/pykolab/imap/cyrus.py", line 142, in login
    cyruslib.CYRUS.login(self, *args, **kw)
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 419, in login
    self.__doexception("LOGIN", error)
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 359, in __doexception
    self.__doraise( function.upper(), msg )
  File "/usr/lib/python2.7/site-packages/cyruslib.py", line 368, in __doraise
    raise CYRUSError( idError[0], mode, msg )
CYRUSError: (10, 'LOGIN', '[Errno 32] Broken pipe')

and

2017-01-05 16:25:00,082 pykolab.auth ERROR LDAP server unavailable: SERVER_DOWN({'desc': "Can't contact LDAP server"},)
2017-01-05 16:25:00,084 pykolab.auth ERROR Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pykolab/auth/ldap/__init__.py", line 3065, in _search
    secondary_domains
  File "<string>", line 10, in <module>
  File "/usr/lib/python2.7/site-packages/pykolab/auth/ldap/__init__.py", line 2963, in _regular_search
    (_result_type, _result) = self.ldap.result(_search, False, 0)
  File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 458, in result
    resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
  File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 462, in result2
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
  File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 469, in result3
    resp_ctrl_classes=resp_ctrl_classes
  File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 476, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/lib64/python2.7/site-packages/ldap/ldapobject.py", line 99, in _ldap_call
    result = func(*args,**kwargs)
SERVER_DOWN: {'desc': "Can't contact LDAP server"}

Although there is still a progress, because looks like the invitation is parsed, just these tracebacks are thrown later.

Are there any other related patches? I searched and was unable to locate similar issues.

I'm not able to reproduce the issue anymore.

This was fixed in 66310b2fc46138.

clamav-update package is required dependency of kolab-16.0.1-3 according to https://obs.kolabsys.com/package/rdiff/Kolab:16/kolab?linkrev=base&rev=9 So, this probably is fixed already. @vanmeeuwen ?

With {tls, no} guam will not initiate the STARTTLS to the backend server and present the non-starttls response. With { tls, starttls } guam will already initialize a STARTTLS Session to the backend resulting in the security problem descriped above and guam allows unencrypted login, Even the LOGIN is stripped from the CAPABILITY afail

In T2082#31878, @dhoffend wrote:

Test @ 143

# nc localhost 143
* OK [CAPABILITIES IMAP4rev1 LITERAL+ ID ENABLE STARTTLS LOGINDISABLED] kolab Cyrus IMAP 2.5.10-49-g2e214b4-Debian-2.5.10.49-0~kolab1 server ready

Yes. I've already played around with this setup. Here're the results. The only thing that's strange are the capabilities after STARTTLS but before! AUTH

In T2082#31869, @dhoffend wrote:

A possible workaround (especially for Kolab:16) would be to update the default configurations of Cyrus and Guam.

Cyrus should listen on 1143 (non-ssl) and 9993 (ssl) while guam forwards 143 to 1143 and 993 to 9993 (2 different upstream servers and 2 listeners). This way guam isn't exposing and you wouldn't have to deal with it. But in the long term it would be great of guam could take over this work so you wouldn't need cyrus to listen on 2 different ports and running 2 different sets of worker/threads.

A possible workaround (especially for Kolab:16) would be to update the default configurations of Cyrus and Guam.

The fact is that the clients may respond to the availability of authentication capabilities before TLS is started (notably URLAUTH, SASL-IR, while AUTH= parameters are already stripped):

Known issue.

I spoke too soon, after posting the above comment, the system would not allow me to log in. No calendar entries we made by anyone.

I have implemented T1988 patch, and the problem cannot be replicated.

@vanmeeuwen No, this is not a duplicate. The issue still persists with pykolab code from git-master. See Bifrost T16075 for the same issue.

@vanmeeuwen, The issue is fixed, but two tests fail now:

======================================================================
FAIL: test_002_check_event_conflict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 134, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/lib64/python2.7/site-packages/twisted/internet/utils.py", line 191, in runWithWarningsSuppressed
    result = f(*a, **kw)
  File "/root/pykolab/tests/unit/test-011-itip.py", line 450, in test_002_check_event_conflict
    self.assertTrue(itip.check_event_conflict(event3, itip_event), "Conflict in two recurring events")
  File "/usr/lib64/python2.7/site-packages/twisted/trial/unittest.py", line 224, in failUnless
    raise self.failureException(msg)
FailTest: Conflict in two recurring events

I did a complete reinstall with kolab 16 some days ago and i am also unable to reproduce. I noticed that the ldif export from the ldap looks different now.

I've just attempted to reproduce the symptoms and it does not seem like I can. I'm going to request you wait for PyKolab version 0.8.7 in Kolab 16 and then try again.

Promoted to BifrostT9064

This should be documented here, shouldn't it?
https://docs.kolab.org/upgrade-guide/kolab-16.html

This is likely a duplicate of T1988

Resolved per the aforementioned commits.

D308 isn't the proper solution to these symptoms.

I have the same syptoms on a fully updated kolab16 on centos7.
Rebooting the sever starts things going for a while until ldap is blocked again.
If it happens Roundcube login does not work and sending mails via kontact is blocked.

D308 is a little bit hacky (and may not find all conflicts), but works. And the old method wasn't perfect anyway.

In T1988#30665, @machniak wrote:

It is not infinite. It is just so extremally slow... For two weekly recurring events it does around 10 * 50 * 10 * 50 iterations. And every iteration calls getLastOccurrence(), getOccurenceEndDate(), etc. on EventCal (from kolab.calendaring). Looks like these are slow.

It is not infinite. It is just so extremally slow... For two weekly recurring events it does around 10 * 50 * 10 * 50 iterations. And every iteration calls getLastOccurrence(), getOccurenceEndDate(), etc. on EventCal (from kolab.calendaring). Looks like these are slow.

Hi machniak, the current build 16 that we are using already has the fix:
https://git.kolab.org/rP54cb493d655bb67719572acd934994161c3751af

Aye, looks like so - don't see any TypeErrors with patch you proposed.

I think a date is possible here if the event is full-day recurring event. In such a case to_dt() is not used on the event last occurence date in https://git.kolab.org/diffusion/P/browse/master/wallace/module_resources.py;71df0cc7b4fc8373e6d84db8e8c01e56ca99ad25$506

Translation was added to transifex: https://www.transifex.com/kolab/kolab/translate/#de/pykolab/93629155

was not intended to change supscribers...

This issue was resolved with https://git.kolab.org/rPe75b015fa96721968083a10d205187ffe8e335d3 in respect to T1417 and D209. The implementation is quite close to what the submitted patch was meant for. So we should close this T1389 and T10702

Now, partially translated:

This ticket has been superseded by Bifrost#T9061.

please consider this submit request for Kolab 16: https://obs.kolabsys.com/request/show/1796

we should fix this in Kolab 16 as soon as possible? I guess I will add a patch?

The headers are indeed added by smtp access policy and come from:

[kolab_smtp_access_policy]
delegate_sender_header = True
alias_sender_header = True
sender_header = True
xsender_header = True

Problem is that Roundcube already set X-Sender and Sender headers.

Differential accepted.

Installed Packages
pykolab.noarch
0.7.28-1.1.el6.kolab_14

Anyway, I see this label is indeed not translated in de.po and de_DE.po localization files.

This label comes from pykolab. What version of pykolab are you using?

Still not translated with the latest package:
roundcubemail 1.1.5.21-1.1.el6.kolab_14

Interesting. When can we expect to have the missing feature implemented?

Just to confirm: It was a faulty PHP-Mailer-Version that caused this issue.

Looks like it is not implemented at all, https://git.kolab.org/diffusion/P/browse/master/pykolab/auth/ldap/__init__.py;9d70f9d837c50e0b74b69d7e8b69c93294dd767a$1735

I finally found the culprit. Differential updated.

Differential created. Note that python-tzlocal need to be added as a pykolab dependency.

I applied the fix into Kolab Winterfell installation, and I still see "infinite" processing loop with the same traceback. I restarted wallace and kolabd.

I think I know why we don't use kolab.calendaring to parse iTip data. It's because it does not support VTODO. Grrrr... Lets start from the beginning...

From what I see iTip attachments are parsed using iCalendar. From what I see we could fix this issue by using kolab.calendaring instead. This is quite a big change, but I'm working on this, I've got a few unit tests failing. If there was any reason to not use kolab.calendaring please speak now.

My "it should" should be verified with Jeroen ;)