Page MenuHomePhorge
Create Task
Maniphest T474

Test REPORT on calendar with URI UIDs does not return 404
Closed, InvalidPublic
Actions

Description

From https://issues.kolab.org/show_bug.cgi?id=3049:

If a VEVENT object has a full URI as its unique identifier, invalid CalDAV URLs for these objects are created and thus synchronization fails. For example, events from the f1calendar.com feed look like this:

  SUMMARY:Australian Grand Prix
  UID:http://2014.f1calendar.com#GP1_2014

In a multistatus request, iRony produces invalid URIs like this:

<d:href>/calendars/john.die%40mykolab.com/ff228802b9-53dabd4f3356-f270fcdd4/http://2014.f1calendar.com#GP1_2014.ics</d:href>

which is wrongly encoded and when a client wants to fetch such an event a "Object not found" exception is returned.

The original fixes are in rI4561e85 (then master, now 0.3) and rIe916334 (0.2), and should have been resolved as per version 0.2.9.

It cannot be reproduced on current master (0.4), but it seems to occur on 0.3.0 (caldav.kolabsys.com) but not on 0.3 (developer workstation).

Details

Ticket Type
Task

Related Objects

Event Timeline

greve created this task.Jun 15 2015, 9:13 AM
greve raised the priority of this task from to 60.
greve updated the task description. (Show Details)
greve added a project: iRony.
greve changed Ticket Type from Task to Task.
greve subscribed.
greve added a comment.Jun 15 2015, 9:19 AM

This comment raises the question of whether 404 is a valid response to a valid URL in case there are no entries to report. This may warrant checking against the specification and other implementations.

petersen added a project: Product Owners.Jun 15 2015, 9:33 AM
petersen added a subscriber: Product Owners.
petersen removed a subscriber: Product Owners.
greve added a comment.Jun 15 2015, 10:05 AM

section 3.6 of RFC 3253 is the relevant RFC section. Wondering whether "(DAV:supported-report)" is what should be issued if REPORT is supported. Also raised that question with the client:

https://github.com/bitfireAT/davdroid/issues/542#issuecomment-111971251

and whether more resilience should not be added in any case:

https://github.com/bitfireAT/davdroid/issues/542#issuecomment-111971417

greve added a comment.Jun 16 2015, 11:32 AM

Update https://github.com/bitfireAT/davdroid/issues/542#issuecomment-112361204 copied here for convenience:

@greve, the REPORT is not a plain WebDAV REPORT, but a CALDAV:calendar-query REPORT which MUST be supported:

  1. Calendaring reports

    This section defines the reports that CalDAV servers MUST support on calendar collections and calendar object resources.

    CalDAV servers MUST advertise support for these reports on all calendar collections and calendar object resources with the DAV: supported-report-set property, defined in Section 3.1.5 of [RFC3253]. CalDAV servers MAY also advertise support for these reports on ordinary collections.

So DAVdroid doesn't check for it.

The REPORT method is essential for task sync because DAVdroid needs a way to filter only files with VEVENT / VTODO. Working around this with a hack that uses PROPFIND and checks the content type of each resource would be possible, but an ugly and error-prone approach. As CalDAV/CardDAV servers must support the REPORT methods, I think this is the way to go – and it seems to work almost everywhere, with a few exceptions. I guess we'll have to track them and find out what's wrong on the server side.

Is there any way how I can assist you in finding the problem? The first task would be to find a reproducible way to generate those 404 errors.

grote moved this task from Incoming to In Triage on the Product Owners board.Jun 18 2015, 10:49 AM
bruederli subscribed.Jun 19 2015, 9:46 AM

Confirmed with apps.kolabnow.com but not with a direct connection to a Kolab server. The 404 error seems to be a response from the haproxy and the REPORT request never makes it to the iRony backend service. Looking at the payload of the response, you'll see a HTML error page from Kolab Now and not a valid DAV response. Thus, haproxy needs to be configured to forward REPORTrequest types to the iRony backends.

Executing a REPORT request against iRony directly responds with a correct multistatus response. Same for requests to caldav.kolabsys.com

greve added a comment.Jun 19 2015, 9:48 AM

Interesting, thanks a lot for that.

FWIW, I see this against kolabsystems.com as well as kolabnow.com -- but then they will both be behind HAproxy, I assume.

bruederli added a comment.Jun 19 2015, 9:52 AM

It works for me with caldav.kolabsys.com but I also see the same (HTML) 404 response from caldav.kolabsystems.com. Thus I assume the latter is also behind a haproxy configuration and the former lets me access the iRony access directly.

greve added a comment.Jun 19 2015, 9:59 AM

Ah, interesting. Indeed I tried kolabsystems.com. So I think we narrowed this down enough that it should be thrown over to Ops for looking at HAProxy, and perhaps documenting what has been going on there also for future deployments.

greve added a subscriber: grote.Jun 19 2015, 2:38 PM

Torsten reports this also happens to him with his personal server, no HA Proxy involved: https://github.com/bitfireAT/davdroid/issues/542#issuecomment-113499114

@grote: Can you perhaps help in how to reproduce this?

greve added a comment.Jun 19 2015, 3:27 PM

Torsten: Look for events that have a UID of https://some.some.some/some/some -- these apparently make SabreDAV choke and die badly.

bruederli added a comment.Jun 21 2015, 5:21 PM
In T474#6337, @greve wrote:

Torsten: Look for events that have a UID of https://some.some.some/some/some -- these apparently make SabreDAV choke and die badly.

This is a known problem fixed loooong time ago: https://issues.kolab.org/show_bug.cgi?id=3049
But unfortunately still in testing...

grote added a comment.Jun 22 2015, 12:35 PM

This is a known problem fixed loooong time ago: https://issues.kolab.org/show_bug.cgi?id=3049

This fix is applied to my Kolab 3.3 installation already and does not fix the issue.

bruederli added a comment.Jun 23 2015, 9:01 AM

That's anyway a different issue likely coming from a GET request subsequent to a REPORT or PROPFIND request. That should go into a separate ticket, preferably with a sample calendar entry that causes sync failures.

This particular ticket, however, should be forwarded to our sysadmins with high priority as it blocks every Kolab Now user synching with DAVDroid.

grote assigned this task to vanmeeuwen.Jun 24 2015, 11:39 AM
grote added a subscriber: vanmeeuwen.

Hi @vanmeeuwen, since you debugged this, do you have a "sample calendar entry that causes sync failures" that you can attach to this ticket?

grote added a comment.Jun 24 2015, 6:10 PM

A sample calendar entry has been added to https://issues.kolab.org/show_bug.cgi?id=3049

bruederli added a comment.Jun 26 2015, 9:38 AM

I'm unable to reproduce any false behavior. See https://issues.kolab.org/show_bug.cgi?id=3049#c9

But again, this should not block the actual issue of REPORT requests not getting through to the iRony backend in the first place.

vanmeeuwen added a comment.Jun 26 2015, 9:47 AM
In T474#6993, @bruederli wrote:

I'm unable to reproduce any false behavior. See https://issues.kolab.org/show_bug.cgi?id=3049#c9

That renders this ticket about testing for this, because the iRony 0.3.0 version (and before) in which this occurs (vendorized and old version of Sabre*) is a long-term commitment.

But again, this should not block the actual issue of REPORT requests not getting through to the iRony backend in the first place.

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

bruederli added a comment.Jun 26 2015, 9:52 AM
In T474#6994, @vanmeeuwen wrote:

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

And do we have iRony 0.3.0 already deployed on Kolab Now?

bruederli added a comment.Jun 26 2015, 9:55 AM
In T474#6995, @bruederli wrote:

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

OK, that seems to be resolved for apps.kolabnow.com by now. But not for caldav.kolabsystems.com

greve added a comment.Jun 26 2015, 10:03 AM

FWIW, my private account (on apps.kolabnow.com) is still failing in the same way right now. Let me know if/how to provide debugging info.

vanmeeuwen renamed this task from iRony responding to REPORT with 404 to Test REPORT on calendar with URI UIDs does not return 404.Jun 26 2015, 10:04 AM
vanmeeuwen updated the task description. (Show Details)
bruederli mentioned this in rIa1b388ae76f9: Correctly encoode event URIs in calendar query results (#3049 and T474).Jun 26 2015, 10:25 AM
vanmeeuwen added a comment.Jun 26 2015, 10:25 AM
In T474#6995, @bruederli wrote:
In T474#6994, @vanmeeuwen wrote:

The entire HAProxy side of the topic is busted, it had nothing to do with anything. The REPORT requests are getting through to iRony.

Really? And why am I seeing the HTML content of the Kolab Now website in the 404 responses I get in those REPORT requests?

URLs that are not recognized as specifically belonging to "apps" on kolabnow.com (tricky business, so PROPFIND on '/' for example, but not GET on '/') are configured to use the corporate website backend and/or redirect to a landing page on kolabnow.com.

bruederli mentioned this in rI0f87b35174d6: Correctly encoode event URIs in calendar query results (#3049 and T474).Jun 26 2015, 10:28 AM
vanmeeuwen added subtasks: T560: Release iRony-0.2.10, T557: Release iRony-0.3.1.Jul 1 2015, 2:24 PM
vanmeeuwen removed subtasks: T557: Release iRony-0.3.1, T560: Release iRony-0.2.10.
greve added a comment.Sep 21 2015, 4:05 PM

BTW: Still seeing these after the recent Kolab Now service window. Should it have resolved them or is this something new/residual?

machniak closed this task as Invalid.Mar 13 2017, 10:24 AM
machniak subscribed.

Hard to say it's deployment or code issue and it's old. Closing.