Hi,
After migrating DAViCal to another server, I keep getting a 'vCard is not conformant to RFC 2426' error on every entry in my addresses database. New entries initially appear correctly, but after a short while (30 secs?) the entry on the screen is replaced with the same non-conformant error message.
Any ideas what can cause this? I copied (via SQL export / import) the database from the old server to the new server so the contents is the same. A difference in software versions (apache, php, davical, postgresql)?
CalDAVzap is working fine in the same setup.
-- Johan
Johan Vromans jvromans@squirrel.nl writes:
[...] I keep getting a 'vCard is not conformant to RFC 2426' error on every entry in my addresses database. New entries initially appear correctly, but after a short while (30 secs?) the entry on the screen is replaced with the same non-conformant error message.
Some debugging shows that the vCard entry that is processed internally (addressbook.js, line 1206) looks like:
<![CDATA[BEGIN:VCARD ADR;TYPE=HOME:********** CATEGORIES:Unfiled EMAIL;TYPE=INTERNET:*************** END:VCARD FN:********** N:************** REV:2001-07-07T13:00:00+01:00 TEL;TYPE=HOME:*********** UID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl VERSION:3.0 X-ALSO:***]]>
In other words, the attributes are sorted alphabetically, causing parsing to fail (I assume) due to a misplaced END:VCARD.
When I retrieve the entry directly from DAViCal, using wget:
BEGIN:VCARD VERSION:3.0 PRODID:-//Johan Vromans//NONSGML pdb2vcard 0.01//EN UID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl
CATEGORIES:Unfiled FN:******* N:******* ADR;TYPE=HOME:********** TEL;TYPE=HOME:********** EMAIL:******** X-ALSO:****** REV:2001-07-07T13:00:00+01:00 END:VCARD
(The empty line contains a single space, apparently a continuation line from the UID.)
Any idea what is going on here?
-- Johan
Hi Johan,
CardDavMATE really shorts all vCard attributes because then I can find/remove duplicate lines using "uniq()", but the END:VCARD is not sorted if your vCard is correct ...
valid vCard looks like:
BEGIN:VCARD\r\n ... ... END:VCARD\r\n
are you sure that your vCard not looks like this invalid vCard:
BEGIN:VCARD\r\n ... ... END:VCARD\r\n \r\n
?
JM
p.s.: the next version will not sort vCard attributes, there is another algorithm to remove duplicate attribute/value pairs
On 02 Dec 2013, at 15:18, Johan Vromans jvromans@squirrel.nl wrote:
Johan Vromans jvromans@squirrel.nl writes:
[...] I keep getting a 'vCard is not conformant to RFC 2426' error on every entry in my addresses database. New entries initially appear correctly, but after a short while (30 secs?) the entry on the screen is replaced with the same non-conformant error message.
Some debugging shows that the vCard entry that is processed internally (addressbook.js, line 1206) looks like:
<![CDATA[BEGIN:VCARD ADR;TYPE=HOME:********** CATEGORIES:Unfiled EMAIL;TYPE=INTERNET:*************** END:VCARD FN:********** N:************** REV:2001-07-07T13:00:00+01:00 TEL;TYPE=HOME:*********** UID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl VERSION:3.0 X-ALSO:***]]>
In other words, the attributes are sorted alphabetically, causing parsing to fail (I assume) due to a misplaced END:VCARD.
When I retrieve the entry directly from DAViCal, using wget:
BEGIN:VCARD VERSION:3.0 PRODID:-//Johan Vromans//NONSGML pdb2vcard 0.01//EN UID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl
CATEGORIES:Unfiled FN:******* N:******* ADR;TYPE=HOME:********** TEL;TYPE=HOME:********** EMAIL:******** X-ALSO:****** REV:2001-07-07T13:00:00+01:00 END:VCARD
(The empty line contains a single space, apparently a continuation line from the UID.)
Any idea what is going on here?
-- Johan
Ján Máté jan.mate@inf-it.com writes:
are you sure that your vCard not looks like this invalid vCard:
BEGIN:VCARD\r\n ... ... END:VCARD\r\n \r\n
?
This is what I get from wget, with \r and \n replaced for readbility:
BEGIN:VCARD\r\nVERSION:3.0\r\nPRODID:-//Johan Vromans//NONSGML pdb2vcard 0.01//EN\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\n \r\nCATEGORIES:Unfiled\r\nFN:****\r\nN:****\r\nADR;TYPE=HOME:****\r\nTEL;TYPE=HOME:*****\r\nEMAIL:******\r\nX-ALSO:*******\r\nREV:2001-07-07T13:00:00+01:00\r\nEND:VCARD\r\n
This is exactly what is in the DAViCal database (table caldav_data, column caldav_data).
-- Johan
Hi Johan,
I tested your vCard using: http://www.inf-it.com/carddavmate/ (demo) ... after the login I entered the following code into JavaScript console (your vCard with ALL CardDavMATE data processing):
normalizeVcard(additionalRFCFixes(basicRFCFixesAndCleanup('BEGIN:VCARD\r\nVERSION:3.0\r\nPRODID:-//Johan Vromans//NONSGML pdb2vcard 0.01//EN\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\n\r\nCATEGORIES:Unfiled\r\nFN:****\r\nN:****\r\nADR;TYPE=HOME:****\r\nTEL;TYPE=HOME:*****\r\nEMAIL:******\r\nX-ALSO:*******\r\nREV:2001-07-07T13:00:00+01:00\r\nEND:VCARD\r\n').vcard));
the result:
"BEGIN:VCARD ADR;TYPE=HOME:**** CATEGORIES:Unfiled EMAIL;TYPE=INTERNET:****** FN:**** N:**** REV:2001-07-07T13:00:00+01:00 TEL;TYPE=HOME:***** UID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl VERSION:3.0 X-ALSO:******* END:VCARD "
all the vCard processing is based on the 3 functions above:
basicRFCFixesAndCleanup => fixes the biggest problems (missing \n, remove empty lines, missing required RFC attributes such as N, UID, ...) additionalRFCFixes => fixes attribute parameters and values (RFC requirements) normalizeVcard => transform the vCard to editor expected format (e.g. transform X-AIM to IMPP;TYPE=AIM:, ..., and remove duplicate lines /by "| sort | unique"/)
As you can see your vCard is still valid after all CardDavMATE processing ... the result is then displayed in the interface without any problems ... you can try to click to "+" (add new contact) and then enter the code below into the console (show the final vCard after all processing):
vcardToData({vcard: 'BEGIN:VCARD\r\nADR;TYPE=HOME:****\r\nCATEGORIES:Unfiled\r\nEMAIL;TYPE=INTERNET:******\r\nFN:****\r\nN:****\r\nREV:2001-07-07T13:00:00+01:00\r\nTEL;TYPE=HOME:*****\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\nVERSION:3.0\r\nX-ALSO:*******\r\nEND:VCARD\r\n'}, false );
So you really must have some invalid characters in your DB (maybe invisible whitespace characters?) ... try to dump the caldav_data table and then reduce the result to one of the problematic vCards and check it using hex editor ...
JM
On 02 Dec 2013, at 17:17, Johan Vromans jvromans@squirrel.nl wrote:
This is what I get from wget, with \r and \n replaced for readbility:
BEGIN:VCARD\r\nVERSION:3.0\r\nPRODID:-//Johan Vromans//NONSGML pdb2vcard 0.01//EN\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\n \r\nCATEGORIES:Unfiled\r\nFN:****\r\nN:****\r\nADR;TYPE=HOME:****\r\nTEL;TYPE=HOME:*****\r\nEMAIL:******\r\nX-ALSO:*******\r\nREV:2001-07-07T13:00:00+01:00\r\nEND:VCARD\r\n
This is exactly what is in the DAViCal database (table caldav_data, column caldav_data).
-- Johan
Hi Ján,
Ján Máté jan.mate@inf-it.com writes:
vcardToData({vcard: 'BEGIN:VCARD\r\nADR;TYPE=HOME:****\r\nCATEGORIES:Unfiled\r\nEMAIL;TYPE=INTERNET:******\r\nFN:****\r\nN:****\r\nREV:2001-07-07T13:00:00+01:00\r\nTEL;TYPE=HOME:*****\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\nVERSION:3.0\r\nX-ALSO:*******\r\nEND:VCARD\r\n'}, false );
The problem is that in addressbook.js, line 1206
if(vcardToData(loadContact,is_readonly))
loadContact.vcard does not contain a "BEGIN:VCARD...END:VCARD" string, it contains
"<![CDATA[BEGIN:VCARD...END:VCARD...X-ALSO:***]]>"
So it looks as if the input received from the DAViCal server is somehow not correctly understood (at the XML level).
I tested some setups (ws = Intel AMD64 workstation, nas = Synologic DS413):
carddav davical postgres result ws ws ws ok ws ws nas ok ws nas nas fail ws nas ws fail
This seems to indicate a DAViCal problem. Both systems are running the same version (1.1.1).
Any idea where to look?
-- Johan
On 02 Dec 2013, at 23:06, Johan Vromans jvromans@squirrel.nl wrote:
Hi Ján,
Ján Máté jan.mate@inf-it.com writes:
vcardToData({vcard: 'BEGIN:VCARD\r\nADR;TYPE=HOME:****\r\nCATEGORIES:Unfiled\r\nEMAIL;TYPE=INTERNET:******\r\nFN:****\r\nN:****\r\nREV:2001-07-07T13:00:00+01:00\r\nTEL;TYPE=HOME:*****\r\nUID:palm-addressbook-00c57015-00003880-Johan Vromans@phoenix.squirrel.nl\r\nVERSION:3.0\r\nX-ALSO:*******\r\nEND:VCARD\r\n'}, false );
The problem is that in addressbook.js, line 1206
if(vcardToData(loadContact,is_readonly))
loadContact.vcard does not contain a "BEGIN:VCARD...END:VCARD" string, it contains
"<![CDATA[BEGIN:VCARD...END:VCARD...X-ALSO:***]]>"
Hmm, this is interesting, because "<![CDATA[" ... "]]>" must be parsed by the browser XML parser and/or jQuery (NOT the client app) ... if you get the whole "<![CDATA string" as a result then it is very probably a browser bug. The DAViCal "<![CDATA[" patch is my patch for OS X Contacts.app, because vCard groups in this app expects "<![CDATA[" format due to OS X bug ... my server response also looks like "<![CDATA[REAL_DATA]]>" and it works without any problems (tested with Safari/Chrome/Firefox on Mac/Windows).
Another problem is maybe your DAViCal version, because my patch is not included in 1.1.1 ... so are you sure that both your DAViCal installations are exactly the same? What is your client OS and browser+version?
JM
So it looks as if the input received from the DAViCal server is somehow not correctly understood (at the XML level).
I tested some setups (ws = Intel AMD64 workstation, nas = Synologic DS413):
carddav davical postgres result ws ws ws ok ws ws nas ok ws nas nas fail ws nas ws fail
This seems to indicate a DAViCal problem. Both systems are running the same version (1.1.1).
Any idea where to look?
-- Johan
Ján Máté jan.mate@inf-it.com writes:
Hmm, this is interesting, because "<![CDATA[" ... "]]>" must be parsed by the browser XML parser and/or jQuery (NOT the client app) ... if you get the whole "<![CDATA string" as a result then it is very probably a browser bug. [...]
Another problem is maybe your DAViCal version, because my patch is not included in 1.1.1 ... so are you sure that both your DAViCal installations are exactly the same? What is your client OS and browser+version?
The kits for davical (and awl) claim to be the same version but come from different repos. I manually diffed the installed files on both systems and found a small diff in awl/inc/XMLElement.php:
*** ./inc/XMLElement.php Mon May 28 12:42:09 2012 --- ./inc/XMLElement.php Fri Sep 28 23:01:43 2012 *************** *** 259,265 **** * Render the content, with special characters escaped * */ ! $r .= htmlspecialchars($this->content, ENT_NOQUOTES ); } return $r; } --- 259,268 ---- * Render the content, with special characters escaped * */ ! if(strpos($this->content, '<![CDATA[')===0 && strrpos($this->content, ']]>')===strlen($this->content)-3) ! $r .= '<![CDATA[' . str_replace(']]>', ']]]]><![CDATA[>', substr($this->content, 9, -3)) . ']]>'; ! else ! $r .= htmlspecialchars($this->content, ENT_NOQUOTES ); } return $r; }
I applied this change manually to the awl on the nas and lo and behold! it works!
Thanks for your ongoing patience in helping me out.
-- Johan
-
Johan Vromans -
Ján Máté