Page MenuHomekolab.org

Test REPORT on calendar with URI UIDs does not return 404
Closed, InvalidPublic

Description

From https://issues.kolab.org/show_bug.cgi?id=3049:

If a VEVENT object has a full URI as its unique identifier, invalid CalDAV URLs for these objects are created and thus synchronization fails. For example, events from the f1calendar.com feed look like this:

  SUMMARY:Australian Grand Prix
  UID:http://2014.f1calendar.com#GP1_2014

In a multistatus request, iRony produces invalid URIs like this:

<d:href>/calendars/john.die%40mykolab.com/ff228802b9-53dabd4f3356-f270fcdd4/http://2014.f1calendar.com#GP1_2014.ics</d:href>

which is wrongly encoded and when a client wants to fetch such an event a "Object not found" exception is returned.

The original fixes are in rI4561e85 (then master, now 0.3) and rIe916334 (0.2), and should have been resolved as per version 0.2.9.

It cannot be reproduced on current master (0.4), but it seems to occur on 0.3.0 (caldav.kolabsys.com) but not on 0.3 (developer workstation).

Details

Ticket Type
Task

Event Timeline

greve created this task.Jun 15 2015, 9:13 AM
greve updated the task description. (Show Details)
greve raised the priority of this task from to 60.
greve added a project: iRony.
greve changed Ticket Type from Task to Task.
greve added a subscriber: greve.
greve added a comment.Jun 15 2015, 9:19 AM

This comment raises the question of whether 404 is a valid response to a valid URL in case there are no entries to report. This may warrant checking against the specification and other implementations.

petersen added a subscriber: Product Owners.
petersen removed a subscriber: Product Owners.

section 3.6 of RFC 3253 is the relevant RFC section. Wondering whether "(DAV:supported-report)" is what should be issued if REPORT is supported. Also raised that question with the client:

https://github.com/bitfireAT/davdroid/issues/542#issuecomment-111971251

and whether more resilience should not be added in any case:

https://github.com/bitfireAT/davdroid/issues/542#issuecomment-111971417

Update https://github.com/bitfireAT/davdroid/issues/542#issuecomment-112361204 copied here for convenience:

@greve, the REPORT is not a plain WebDAV REPORT, but a CALDAV:calendar-query REPORT which MUST be supported:

  1. Calendaring reports

    This section defines the reports that CalDAV servers MUST support on calendar collections and calendar object resources.

    CalDAV servers MUST advertise support for these reports on all calendar collections and calendar object resources with the DAV: supported-report-set property, defined in Section 3.1.5 of [RFC3253]. CalDAV servers MAY also advertise support for these reports on ordinary collections.

So DAVdroid doesn't check for it.

The REPORT method is essential for task sync because DAVdroid needs a way to filter only files with VEVENT / VTODO. Working around this with a hack that uses PROPFIND and checks the content type of each resource would be possible, but an ugly and error-prone approach. As CalDAV/CardDAV servers must support the REPORT methods, I think this is the way to go – and it seems to work almost everywhere, with a few exceptions. I guess we'll have to track them and find out what's wrong on the server side.

Is there any way how I can assist you in finding the problem? The first task would be to find a reproducible way to generate those 404 errors.

grote moved this task from Incoming to In Triage on the Product Owners board.Jun 18 2015, 10:49 AM

Confirmed with apps.kolabnow.com but not with a direct connection to a Kolab server. The 404 error seems to be a response from the haproxy and the REPORT request never makes it to the iRony backend service. Looking at the payload of the response, you'll see a HTML error page from Kolab Now and not a valid DAV response. Thus, haproxy needs to be configured to forward REPORTrequest types to the iRony backends.

Executing a REPORT request against iRony directly responds with a correct multistatus response. Same for requests to caldav.kolabsys.com

greve added a comment.Jun 19 2015, 9:48 AM

Interesting, thanks a lot for that.

FWIW, I see this against kolabsystems.com as well as kolabnow.com -- but then they will both be behind HAproxy, I assume.

It works for me with caldav.kolabsys.com but I also see the same (HTML) 404 response from caldav.kolabsystems.com. Thus I assume the latter is also behind a haproxy configuration and the former lets me access the iRony access directly.

greve added a comment.Jun 19 2015, 9:59 AM

Ah, interesting. Indeed I tried kolabsystems.com. So I think we narrowed this down enough that it should be thrown over to Ops for looking at HAProxy, and perhaps documenting what has been going on there also for future deployments.

greve added a subscriber: grote.Jun 19 2015, 2:38 PM

Torsten reports this also happens to him with his personal server, no HA Proxy involved: https://github.com/bitfireAT/davdroid/issues/542#issuecomment-113499114

@grote: Can you perhaps help in how to reproduce this?

greve added a comment.Jun 19 2015, 3:27 PM

Torsten: Look for events that have a UID of https://some.some.some/some/some -- these apparently make SabreDAV choke and die badly.

In T474#6337, @greve wrote:

Torsten: Look for events that have a UID of https://some.some.some/some/some -- these apparently make SabreDAV choke and die badly.

This is a known problem fixed loooong time ago: https://issues.kolab.org/show_bug.cgi?id=3049
But unfortunately still in testing...

This is a known problem fixed loooong time ago: https://issues.kolab.org/show_bug.cgi?id=3049

This fix is applied to my Kolab 3.3 installation already and does not fix the issue.

That's anyway a different issue likely coming from a GET request subsequent to a REPORT or PROPFIND request. That should go into a separate ticket, preferably with a sample calendar entry that causes sync failures.

This particular ticket, however, should be forwarded to our sysadmins with high priority as it blocks every Kolab Now user synching with DAVDroid.

grote assigned this task to vanmeeuwen.Jun 24 2015, 11:39 AM
grote added a subscriber: vanmeeuwen.

Hi @vanmeeuwen, since you debugged this, do you have a "sample calendar entry that causes sync failures" that you can attach to this ticket?

grote added a comment.Jun 24 2015, 6:10 PM

A sample calendar entry has been added to https://issues.kolab.org/show_bug.cgi?id=3049

I'm unable to reproduce any false behavior. See https://issues.kolab.org/show_bug.cgi?id=3049#c9

But again, this should not block the actual issue of REPORT requests not getting through to the iRony backend in the first place.

I'm unable to reproduce any false behavior. See https://issues.kolab.org/show_bug.cgi?id=3049#c9

That renders this ticket about testing for this, because the iRony 0.3.0 version (and before) in which this occurs (vendorized and old version of Sabre*) is a long-term commitment.

But again, this should not block the actual issue of REPORT requests not getting through to the iRony backend in the first place.

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

And do we have iRony 0.3.0 already deployed on Kolab Now?

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

OK, that seems to be resolved for apps.kolabnow.com by now. But not for caldav.kolabsystems.com

FWIW, my private account (on apps.kolabnow.com) is still failing in the same way right now. Let me know if/how to provide debugging info.

vanmeeuwen renamed this task from iRony responding to REPORT with 404 to Test REPORT on calendar with URI UIDs does not return 404.Jun 26 2015, 10:04 AM
vanmeeuwen updated the task description. (Show Details)

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

URLs that are not recognized as specifically belonging to "apps" on kolabnow.com (tricky business, so PROPFIND on '/' for example, but not GET on '/') are configured to use the corporate website backend and/or redirect to a landing page on kolabnow.com.

greve added a comment.Sep 21 2015, 4:05 PM

BTW: Still seeing these after the recent Kolab Now service window. Should it have resolved them or is this something new/residual?

machniak closed this task as Invalid.Mar 13 2017, 10:24 AM
machniak added a subscriber: machniak.

Hard to say it's deployment or code issue and it's old. Closing.