Page MenuHomePhorge

With this patch I'm trying to introduce a file-type object in logger, which could swallow everything thrown to stderr (and possibly stdout) and redirect to python logger. Python smtplib debug mode prints everything to stderr, but when wallace runs...
ClosedPublic

Authored by adomaitis on Mar 15 2018, 3:18 PM.
Tags
None
Referenced Files
F11835180: D577.id1400.diff
Sun, Apr 21, 5:15 PM
F11835177: D577.id1394.diff
Sun, Apr 21, 5:15 PM
F11835174: D577.id1388.diff
Sun, Apr 21, 5:15 PM
F11835171: D577.id1382.diff
Sun, Apr 21, 5:15 PM
F11835168: D577.id1379.diff
Sun, Apr 21, 5:15 PM
F11835165: D577.id1373.diff
Sun, Apr 21, 5:15 PM
F11835162: D577.id1367.diff
Sun, Apr 21, 5:15 PM
F11835159: D577.id1363.diff
Sun, Apr 21, 5:15 PM

Diff Detail

Repository
rP pykolab
Branch
master
Lint
Lint Skipped
Unit
No Test Coverage
Build Status
Buildable 18110
Build 9434: arc lint + arc unit

Event Timeline

This is using the new file-like object from pykolab.logger. With this patch smtplib debug output gets redirected to logger rather than to stderr, which is not available in fork mode

This amendment tries to avoid cyruslib printing to non-available file descriptor, which causes traceback of wallace.

pykolab/imap/cyrus.py
147–153

Looks like indentation here is broken.

In addition LDAP trace_file needs to be specified. It is a file-like object define in logger.py Now we get ldap debug logs in pykolab.log file

Additionally, making wallace log.exception to log tracebacks

I think we need process id (PID) logged for any level log record, that makes it easy to associate and track wallace and other pykolab modules work flow.

Fixing another wrong intendation

Specifying a bit more detailed log message.

Need to catch imaplib debug output as well

Fixing previous mistake. Need that for messages 9 and lower to be logged.

It turns out smtplib debug logging redirected to logger produces excessive new lines. This is attempt to mitigate the problem, make logs more pretty.

Make invitationpolicy module use smtplib sendmail wrapper _sendmail from modules, which now supports debug level 9 logging.

Make all pykolab log messages appear on level <8. This will isolate pykolab debug messages from 3rd partty libraries debug messages. -d debug -l 8 will print pykolab debug messages only, while -l debug -d 9 will print 3rd party libraries messages in addition to level 8 pykolab messages.

Looking good, except for the corner-case of an errmsg over an un-"captured" exception.

pykolab/imap/cyrus.py
152

I think you may need to;

except Exception, errmsg:

before you can refer to errmsg in the call to log.exception().

In send_update_notification() we send mail in a loop trying up to 5 times if an error occurrs. But this is the only place where we do this. I think it would make sense to do this in all cases, i.e. in _sendmail(). I see also we don't call smtp.quit() consistently. Once we do this on success only, once we do this on error too.

In D577#6587, @machniak wrote:

In send_update_notification() we send mail in a loop trying up to 5 times if an error occurrs. But this is the only place where we do this. I think it would make sense to do this in all cases, i.e. in _sendmail(). I see also we don't call smtp.quit() consistently. Once we do this on success only, once we do this on error too.

The calls to modules._sendmail() are themselves looped, by the caller. Does that address the concern?

Eliminating inconsistensies of sending mail from wallace. Now all the logic is in modules._sendmail() wrapper. Invitationpolicy module is using that wrapper and makes it responsible for re-submistion logic.

Fix conner case exception handling.

A few more remarks.

wallace/modules.py
156

This line would prevent smtp.quit() and other cleanup from being called.

159–163

Note that the existing smtplib.SMTP() used to be rebuilt the connection for here.

163

This does not invoke recreating the smtplib.SMTP() instance, therefore possibly looping?

wallace/module_invitationpolicy.py
1350

I'd put this log.debug() after modules._sendmail().

Harbormaster completed remote builds in B18056: Diff 1373.
Harbormaster completed remote builds in B18062: Diff 1379.
Harbormaster completed remote builds in B18065: Diff 1382.

Log debug message after _sendmail.

yet another attempt to create a better wrapper for smtplib.sendmail(). Now instantiation of smtplib object and connect are separated.

This revision is now accepted and ready to land.May 17 2018, 8:41 AM
This revision was automatically updated to reflect the committed changes.