Special characters in tpl

I have a tpl that reads a field from the dcr and then via perl in that tpl uses DBI to write to a MSSQL database. The problem I am having is that the fields in the DCR can contain french characters (or German or Spanish) . When I try to populate the MSSQL table they end up as garbage. I know the MSSQL tables can accept those characters as I can paste them directly into the database. How can I get those characters to appear correctly? I've been investigating changing character encoding (which I'm fuzzy at). I've read that I might have to actually save the tpl and dct files in utf-8.

Any help would be appreciated. If need be I can include code.

Find more posts tagged with

Comments

Migrateduser

bump bump

Reghunath

Download a UTF-8 editor and open your DCR in it.
Free UTF-8 editor, "First Object XML Editor".

If you DCR content is showing up as you had entered, then its not DCR issue.

Cheers!

Migrateduser

I never said it was a DCR issue. It's what I'm doing with it in the TPL that I'm having trouble with. Anyway, I gave up and have choosen to use the regex from the Tech Library about how to change hex to ascii. I still would like to store the actual characters in the database instead of an ascii code.

nipper

You probably do not have the BOM set in the TPL. That is the byte order mark and is 3 characters in the very beginning of the TPL.

Some editors will automatically set it. Open the TPL in notepad and save it as UTF-8 because notepad does set the BOM.

Search for BOM there have been a few threads on it as well

Valentine

From my experience MS SQL, had to stand out, and have a separate way of dealing with UTF-8.

Are your columns nvarchars? or regular varchars?

Are you inserting the data with the N in front of the value?
ie INSERT INTO SOME(C1, C2) values(N'sssss', N'22222')
?

iwTech

I hope the following will help

I had a situation to publish the regional languages along with English
(say Chinese , Spanish etc…) My situation like the user will add/ copy the text
into the DCR, (mixed test values / English and other languages)

The work flow will update the DCR values into the database ( PostgreSQL)
I faced the same situation , finally I Solved

If it is a special character for a field , convert it into the UTF-8 Format-ASCII then
Insert into DB .I basically parse the DCR and get the value and check for special character

Example :-

熱門王建民姓名學方志友台灣小綠- INTERWOVEN-人伊能靜(圖)張懸(影)毛孔粗大

my $parser = TeamSite: Smiley Very Happy

CRparser->new();
my $rootnode = $parser->parse($dcr_buf);
$debugLog .= "*** Displaying the DCR Values ***\n";
my $Title = $rootnode->value("pressrelease.main.Title");
.
# UTF-8 Format ASCII convertion -Title
$Title =~ s/(.)/asciiize($1)/eg; # Process by char

sub asciiize {
return $_[0] if (ord($_[0]) < 128); # ASCII
return sprintf('&#x%04X;', ord($_[0])); # Non-ASCII
}

$debugLog .= "Title = $Title\n";

# Prepare the DB INSERT query

If you look the DB values for Title , Like
;熱
;門
;王
;建
;民
;姓
(I changed the format ,so you can see the asciii values)

Let me know , it helped
-Vin