get_dcr() returns ISO instead of UTF-8

System

Hi,

after upgrade to TS 6.1 (on AIX 5.3) some of our scripts stopped working.
Seems that XML:arser that get_dcr() relies on returns ISO-8859-1 encoding instead of UTF-8.

I have attached test script with output files.

On our preview server (TS 5.5.2 with 5.6 Perl) it runs correctly.

Thanks for any help on it.

Bosse.

Frederik

When you upgrade to TS6.1+, you need to know that the Perl version (iwperl) changes, from 5.005_03 to 5.8.2. Cfr KB article 51304. (So it's weird you claim using Perl 5.6)

The important relevant issue here, is that these 2 versions of Perl treat strings and their encoding TOTALLY DIFFERENT. Any perl script running under 5.8.2 should either treat string values that are in the native charset of the server OR the script should be explicit about the char encoding.

Your test script is flawed, I believe.

For example, Perl 5.8 will convert a UTF-8 string back to Latin1 (assuming latin1 (aka ISO-8859-1) is the server's native charset) if the string value fits entirely within Latin1. I guess that's what's happening when you write out out_2.dcr - since you don't force an output encoding on the filehandle.

as a starter, read
http://www.ahinea.com/en/tech/perl-unicode-struggle.html


We were having tons of unicode related issues when upgrading ts55 to ts65, and I know I spent easily 5-10 days debugging them, and coming to grips with Perl's new charencoding handling. Some experiences I wrote in this thread

--greets, Fred