Android synching problem (again)
Setup: DAViCal server on NAS. CalDAVzap on linux client. Business Calendar Pro + CalDAVsynch on Samsung Galaxy Note 2 10.1. Business Calendar Pro + CalDAVsynch on Motorola Moto G. iPad with standard iPad calendar. The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on. Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on. The change is picked up correctly by the iPad. The Samsung now shows the (new?) appointment in weeks 11, 13, ... but still shows the (old?) appointment in weeks 10, 12, ... The Motorola doesn't show either appointment. Unfortunately I was just a little too late to intercept the log file. I'l try to be more quick if something like this happens again. -- Johan
Can you send me the event (i.e. the ics file)? Marten Am 04.03.2014 20:38, schrieb Johan Vromans:
Setup: DAViCal server on NAS. CalDAVzap on linux client. Business Calendar Pro + CalDAVsynch on Samsung Galaxy Note 2 10.1. Business Calendar Pro + CalDAVsynch on Motorola Moto G. iPad with standard iPad calendar.
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on.
The change is picked up correctly by the iPad.
The Samsung now shows the (new?) appointment in weeks 11, 13, ... but still shows the (old?) appointment in weeks 10, 12, ...
The Motorola doesn't show either appointment.
Unfortunately I was just a little too late to intercept the log file. I'l try to be more quick if something like this happens again.
-- Johan
-- *Marten Gajda* Schandauer Straße 34 01309 Dresden Germany tel: +49 177 4427167 email: marten@dmfs.org twitter: twitter.com/dmfs_org VAT Reg. No.: DE269072391
Hi Johan, the file you sent me is definitely broken. It contains two VEVENTs with different UIDs, which is not allowed in CalDAV: BEGIN:VEVENT ... UID:8xgll0ja-tmwa-uubj-n5fo-cnngv7hgkx5w .... END:VEVENT BEGIN:VEVENT ... UID:jgp2g2m2-yc1t-bgl4-817g-s3ynu2aub2sr ... END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.10.0.2//EN How exactly did you export/download that file? @Ján can you check how that could happen? cheers Marten Am 04.03.2014 20:38, schrieb Johan Vromans:
Setup: DAViCal server on NAS. CalDAVzap on linux client. Business Calendar Pro + CalDAVsynch on Samsung Galaxy Note 2 10.1. Business Calendar Pro + CalDAVsynch on Motorola Moto G. iPad with standard iPad calendar.
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on.
The change is picked up correctly by the iPad.
The Samsung now shows the (new?) appointment in weeks 11, 13, ... but still shows the (old?) appointment in weeks 10, 12, ...
The Motorola doesn't show either appointment.
Unfortunately I was just a little too late to intercept the log file. I'l try to be more quick if something like this happens again.
-- Johan
-- Marten Gajda Schandauer Straße 34 01309 Dresden Germany tel: +49 177 4427167 email: marten@dmfs.org twitter: twitter.com/dmfs_org VAT Reg. No.: DE269072391
Hi Johan and Marten, can you send me that file too? I can check if something goes wrong. I repeated every step for creating such situation in our CalDavZAP, but I get same UIDs. Thanks, Andrej Dňa 5. 3. 2014 9:45 Marten Gajda wrote / napísal(a):
Hi Johan,
the file you sent me is definitely broken. It contains two VEVENTs with different UIDs, which is not allowed in CalDAV:
BEGIN:VEVENT ... UID:8xgll0ja-tmwa-uubj-n5fo-cnngv7hgkx5w .... END:VEVENT BEGIN:VEVENT ... UID:jgp2g2m2-yc1t-bgl4-817g-s3ynu2aub2sr ... END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.10.0.2//EN
How exactly did you export/download that file?
@Ján can you check how that could happen?
cheers
Marten
Am 04.03.2014 20:38, schrieb Johan Vromans:
Setup: DAViCal server on NAS. CalDAVzap on linux client. Business Calendar Pro + CalDAVsynch on Samsung Galaxy Note 2 10.1. Business Calendar Pro + CalDAVsynch on Motorola Moto G. iPad with standard iPad calendar.
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on.
The change is picked up correctly by the iPad.
The Samsung now shows the (new?) appointment in weeks 11, 13, ... but still shows the (old?) appointment in weeks 10, 12, ...
The Motorola doesn't show either appointment.
Unfortunately I was just a little too late to intercept the log file. I'l try to be more quick if something like this happens again.
-- Johan
Marten Gajda <marten@dmfs.org> writes:
the file you sent me is definitely broken. It contains two VEVENTs with different UIDs, which is not allowed in CalDAV:
BEGIN:VEVENT ... UID:8xgll0ja-tmwa-uubj-n5fo-cnngv7hgkx5w .... END:VEVENT BEGIN:VEVENT ... UID:jgp2g2m2-yc1t-bgl4-817g-s3ynu2aub2sr ... END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.10.0.2//EN
How exactly did you export/download that file?
wget --password 'xxxx' http://jv@davical.squirrel.nl/caldav.php/jv/joan/e8a9ff60ab4f01ce92cde275d90... I got the url by using the HTML element inspector of Google-Chrome. -- Johan
Am 05.03.2014 13:41, schrieb Johan Vromans:
Marten Gajda <marten@dmfs.org> writes:
the file you sent me is definitely broken. It contains two VEVENTs with different UIDs, which is not allowed in CalDAV:
BEGIN:VEVENT ... UID:8xgll0ja-tmwa-uubj-n5fo-cnngv7hgkx5w .... END:VEVENT BEGIN:VEVENT ... UID:jgp2g2m2-yc1t-bgl4-817g-s3ynu2aub2sr ... END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.10.0.2//EN
How exactly did you export/download that file? wget --password 'xxxx' http://jv@davical.squirrel.nl/caldav.php/jv/joan/e8a9ff60ab4f01ce92cde275d90...
I got the url by using the HTML element inspector of Google-Chrome.
-- Johan
So it's the same file that has been retrieved by CalDAV-Sync. I'll have to think about what's the best way to handle an event like this. I wonder how iOS was able to determine the right event. Maybe they just ignore additional VEVENTs with UIDs that don't match the original UID. I guess that would be the best way. But I wonder which of those UIDs was the original one. None of them matches the file name. Which client created the event initially? cheers Marten -- Marten Gajda Schandauer Straße 34 01309 Dresden Germany tel: +49 177 4427167 email: marten@dmfs.org twitter: twitter.com/dmfs_org VAT Reg. No.: DE269072391
Marten Gajda <marten@dmfs.org> writes:
Which client created the event initially?
From the OP:
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on.
So the suspect is CalDAVzap. But it could very well be the case that the appointment already was corrupt (in some way) at that time and that the changes I made just made the corruption apparent. Anyone has a tool to verify ics files for correctness? Also, it would be very helpful if there was an (easy) was to show the raw event (ics) data (or copyable url) from CalDAVzap... -- Johan
Hi Johan, On 05 Mar 2014, at 16:25, Johan Vromans <jvromans@squirrel.nl> wrote:
Marten Gajda <marten@dmfs.org> writes:
Which client created the event initially?
From the OP:
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on.
So the suspect is CalDAVzap. But it could very well be the case that the appointment already was corrupt (in some way) at that time and that the changes I made just made the corruption apparent.
Anyone has a tool to verify ics files for correctness?
Also, it would be very helpful if there was an (easy) was to show the raw event (ics) data (or copyable url) from CalDAVzap...
if you click to "inspect element" in the browser you will see the URL of the "inspected" event ... If it is possible please try to reproduce the problem and send us the steps you performed. JM
-- Johan
Hi Ján,
Also, it would be very helpful if there was an (easy) was to show the raw event (ics) data (or copyable url) from CalDAVzap...
if you click to "inspect element" in the browser you will see the URL of the "inspected" event ...
When I click "inspect element" I get a window with an awful lot of information, and somewhere hidden is the URL. To be able to copy the URL I first need to select the appropriate item and the RichtClick > Edit as HTML. So yes, I can get the URL but its not easy.
If it is possible please try to reproduce the problem and send us the steps you performed.
Yes, I'll definitely try. Stay tuned. -- Johan
On 03/05/2014 10:43 AM, Ján Máté wrote:
Hi Johan,
On 05 Mar 2014, at 16:25, Johan Vromans <jvromans@squirrel.nl> wrote:
Marten Gajda <marten@dmfs.org> writes:
Which client created the event initially? From the OP:
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on. So the suspect is CalDAVzap. But it could very well be the case that the appointment already was corrupt (in some way) at that time and that the changes I made just made the corruption apparent.
Anyone has a tool to verify ics files for correctness?
Also, it would be very helpful if there was an (easy) was to show the raw event (ics) data (or copyable url) from CalDAVzap... if you click to "inspect element" in the browser you will see the URL of the "inspected" event ...
If it is possible please try to reproduce the problem and send us the steps you performed.
Its very easy to reproduce: 1. Create a recurring event, say daily for 7 instances 2. Edit the middle instance, selecting "This and all future" and change the time The telemetry below show what it looks like on my server, which rejects the resource as invalid (multiple VEVENT with different UIDs). Note that I stripped the VTIMEZONEs for brevity. If you want to split the instances like this, they need to be put back as separate resources. Or you can use the same UID and have the second VEVENT be linked to the first with RECURRENCE-ID (see RFC 5545, section 3.8.4.4) PUT /dav/calendars/user/ken/339B1B76-FAA8-4792-B829-FF74D6E8F5E1/e5a 9973885faad790705afdc3a84eca74c08f849eef105f61cac4a8af21bc598.ics HTTP/1.1 Accept-encoding: gzip, deflate Content-type: text/calendar; charset=UTF-8 Accept-language: en-US,en;q=0.5 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 Host: localhost Accept: text/plain, */*; q=0.01 X-client: CalDavZAP 0.9.42 (Inf-IT CalDAV Web Client) Connection: keep-alive Content-length: 5494 If-none-match: * X-requested-with: XMLHttpRequest Authorization: Basic ... Referer: http://localhost/caldavzap-0.10.0rc4/ BEGIN:VCALENDAR VERSION:2.0 CALSCALE:GREGORIAN BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224655Z DTSTAMP:20140305T224655Z UID:f0cr1lsr-n01t-ba84-mf6p-ee7brhb3lr2i SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=7 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140305T080000 DTEND;TZID=America/New_York:20140305T090000 END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.9.42//EN END:VCALENDAR HTTP/1.1 201 Created Keep-Alive: timeout=60 Connection: Keep-Alive Date: Wed, 05 Mar 2014 22:46:55 GMT Cache-Control: no-cache Vary: Accept-Encoding Server: Cyrus/git2.4.17-caldav-beta9+78 (Murder) Cyrus-SASL/2.1.26 OpenSSL/1.0.1e zlib/1.2.7 libxml2/2.9.1 SQLite/3.7.13 libical/0.48 Jansson/2.4 ETag: "dbb8c1330bbc553df2c508a2c52b7a725e240820" Last-Modified: Wed, 05 Mar 2014 22:46:55 GMT Content-Length: 0 PUT /dav/calendars/user/ken/339B1B76-FAA8-4792-B829-FF74D6E8F5E1/e5a 9973885faad790705afdc3a84eca74c08f849eef105f61cac4a8af21bc598.ics HTTP/1.1 Accept-encoding: gzip, deflate Content-type: text/calendar; charset=UTF-8 Accept-language: en-US,en;q=0.5 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 Host: localhost Accept: text/plain, */*; q=0.01 X-client: CalDavZAP 0.9.42 (Inf-IT CalDAV Web Client) Connection: keep-alive Content-length: 743 If-match: "dbb8c1330bbc553df2c508a2c52b7a725e240820" X-requested-with: XMLHttpRequest Authorization: Basic ... Referer: http://localhost/caldavzap-0.10.0rc4/ BEGIN:VCALENDAR VERSION:2.0 CALSCALE:GREGORIAN BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224655Z DTSTAMP:20140305T224655Z UID:f0cr1lsr-n01t-ba84-mf6p-ee7brhb3lr2i SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=3 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140305T080000 DTEND;TZID=America/New_York:20140305T090000 END:VEVENT BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224755Z DTSTAMP:20140305T224755Z UID:gcko6hrs-5n1l-t71l-xadx-cj667j4r0rey SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=4 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140308T100000 DTEND;TZID=America/New_York:20140308T110000 END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.9.42//EN END:VCALENDAR HTTP/1.1 403 Forbidden Keep-Alive: timeout=60 Connection: Keep-Alive Date: Wed, 05 Mar 2014 22:47:55 GMT Cache-Control: no-cache Vary: Accept-Encoding Server: Cyrus/git2.4.17-caldav-beta9+78 (Murder) Cyrus-SASL/2.1.26 OpenSSL/1.0.1e zlib/1.2.7 libxml2/2.9.1 SQLite/3.7.13 libical/0.48 Jansson/2.4 Content-Type: application/xml; charset=utf-8 Content-Length: 153 <?xml version="1.0" encoding="utf-8"?> <D:error xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav"> <C:valid-calendar-object-resource/> </D:error> -- Kenneth Murchison Principal Systems Software Engineer Carnegie Mellon University
Hi Ken, you are right, will fix. 0.10.0.3 is coming (very probably) tomorrow JM On 06 Mar 2014, at 00:04, Ken Murchison <murch@andrew.cmu.edu> wrote:
On 03/05/2014 10:43 AM, Ján Máté wrote:
Hi Johan,
On 05 Mar 2014, at 16:25, Johan Vromans <jvromans@squirrel.nl> wrote:
Marten Gajda <marten@dmfs.org> writes:
Which client created the event initially? From the OP:
The appointment is a repeating event, to repeat every two weeks. Starting tomorrow. Ending 2014-12-31. The appointment exists on all clients. It is displayed in weeks 10, 12, and so on.
Using CalDAVzap I edit the first occurrence of the appointment and change all occurrences: change the starting date to one week later. It is now displayed in weeks 11, 13, and so on. So the suspect is CalDAVzap. But it could very well be the case that the appointment already was corrupt (in some way) at that time and that the changes I made just made the corruption apparent.
Anyone has a tool to verify ics files for correctness?
Also, it would be very helpful if there was an (easy) was to show the raw event (ics) data (or copyable url) from CalDAVzap... if you click to "inspect element" in the browser you will see the URL of the "inspected" event ...
If it is possible please try to reproduce the problem and send us the steps you performed.
Its very easy to reproduce:
1. Create a recurring event, say daily for 7 instances 2. Edit the middle instance, selecting "This and all future" and change the time
The telemetry below show what it looks like on my server, which rejects the resource as invalid (multiple VEVENT with different UIDs). Note that I stripped the VTIMEZONEs for brevity.
If you want to split the instances like this, they need to be put back as separate resources. Or you can use the same UID and have the second VEVENT be linked to the first with RECURRENCE-ID (see RFC 5545, section 3.8.4.4)
PUT /dav/calendars/user/ken/339B1B76-FAA8-4792-B829-FF74D6E8F5E1/e5a 9973885faad790705afdc3a84eca74c08f849eef105f61cac4a8af21bc598.ics HTTP/1.1 Accept-encoding: gzip, deflate Content-type: text/calendar; charset=UTF-8 Accept-language: en-US,en;q=0.5 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 Host: localhost Accept: text/plain, */*; q=0.01 X-client: CalDavZAP 0.9.42 (Inf-IT CalDAV Web Client) Connection: keep-alive Content-length: 5494 If-none-match: * X-requested-with: XMLHttpRequest Authorization: Basic ... Referer: http://localhost/caldavzap-0.10.0rc4/
BEGIN:VCALENDAR VERSION:2.0 CALSCALE:GREGORIAN BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224655Z DTSTAMP:20140305T224655Z UID:f0cr1lsr-n01t-ba84-mf6p-ee7brhb3lr2i SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=7 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140305T080000 DTEND;TZID=America/New_York:20140305T090000 END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.9.42//EN END:VCALENDAR
HTTP/1.1 201 Created Keep-Alive: timeout=60 Connection: Keep-Alive Date: Wed, 05 Mar 2014 22:46:55 GMT Cache-Control: no-cache Vary: Accept-Encoding Server: Cyrus/git2.4.17-caldav-beta9+78 (Murder) Cyrus-SASL/2.1.26 OpenSSL/1.0.1e zlib/1.2.7 libxml2/2.9.1 SQLite/3.7.13 libical/0.48 Jansson/2.4 ETag: "dbb8c1330bbc553df2c508a2c52b7a725e240820" Last-Modified: Wed, 05 Mar 2014 22:46:55 GMT Content-Length: 0
PUT /dav/calendars/user/ken/339B1B76-FAA8-4792-B829-FF74D6E8F5E1/e5a 9973885faad790705afdc3a84eca74c08f849eef105f61cac4a8af21bc598.ics HTTP/1.1 Accept-encoding: gzip, deflate Content-type: text/calendar; charset=UTF-8 Accept-language: en-US,en;q=0.5 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 Host: localhost Accept: text/plain, */*; q=0.01 X-client: CalDavZAP 0.9.42 (Inf-IT CalDAV Web Client) Connection: keep-alive Content-length: 743 If-match: "dbb8c1330bbc553df2c508a2c52b7a725e240820" X-requested-with: XMLHttpRequest Authorization: Basic ... Referer: http://localhost/caldavzap-0.10.0rc4/
BEGIN:VCALENDAR VERSION:2.0 CALSCALE:GREGORIAN BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224655Z DTSTAMP:20140305T224655Z UID:f0cr1lsr-n01t-ba84-mf6p-ee7brhb3lr2i SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=3 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140305T080000 DTEND;TZID=America/New_York:20140305T090000 END:VEVENT BEGIN:VEVENT CREATED:20140305T224655Z LAST-MODIFIED:20140305T224755Z DTSTAMP:20140305T224755Z UID:gcko6hrs-5n1l-t71l-xadx-cj667j4r0rey SUMMARY:asdasdasda RRULE:FREQ=DAILY;COUNT=4 TRANSP:OPAQUE CLASS:PUBLIC DTSTART;TZID=America/New_York:20140308T100000 DTEND;TZID=America/New_York:20140308T110000 END:VEVENT PRODID:-//Inf-IT//CalDavZAP 0.9.42//EN END:VCALENDAR
HTTP/1.1 403 Forbidden Keep-Alive: timeout=60 Connection: Keep-Alive Date: Wed, 05 Mar 2014 22:47:55 GMT Cache-Control: no-cache Vary: Accept-Encoding Server: Cyrus/git2.4.17-caldav-beta9+78 (Murder) Cyrus-SASL/2.1.26 OpenSSL/1.0.1e zlib/1.2.7 libxml2/2.9.1 SQLite/3.7.13 libical/0.48 Jansson/2.4 Content-Type: application/xml; charset=utf-8 Content-Length: 153
<?xml version="1.0" encoding="utf-8"?> <D:error xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav"> <C:valid-calendar-object-resource/> </D:error>
-- Kenneth Murchison Principal Systems Software Engineer Carnegie Mellon University
Ján Máté <jan.mate@inf-it.com> writes:
you are right, will fix.
0.10.0.3 is coming (very probably) tomorrow
Great! In the mean time, I wrote a little check program. It takes a downloaded calendar ics as input, and for each VEVENT UID it looks up the UID in the davical database, retrieves the event ics from the davical server, and verifies the returned information to contain only one event UID which must match the original. The program finds a substantial amount of offending entries in my calendars :( What would be the best way to fix these? Download the ics files, manually correct them, and then upload them (overwriting the current database)? -- Johan
Hi Johan, On 06 Mar 2014, at 14:00, Johan Vromans <jvromans@squirrel.nl> wrote:
Ján Máté <jan.mate@inf-it.com> writes:
you are right, will fix.
0.10.0.3 is coming (very probably) tomorrow
Great!
In the mean time, I wrote a little check program. It takes a downloaded calendar ics as input, and for each VEVENT UID it looks up the UID in the davical database, retrieves the event ics from the davical server, and verifies the returned information to contain only one event UID which must match the original.
The program finds a substantial amount of offending entries in my calendars :(
What would be the best way to fix these?
if everything goes well then click to "edit" => "save" in the upcoming version will fix these events for you. JM
Download the ics files, manually correct them, and then upload them (overwriting the current database)?
-- Johan
participants (5)
-
Andrej Lezo -
Johan Vromans -
Ján Máté -
Ken Murchison -
Marten Gajda