Special characters from inline not displaying properly

TeamSite 6.7.1
Windows 2003

I am dynamically writting my DCT elements using an inline. These elements need to be populated with English and French text.

I can't get special characters from the inline into the default tag of the text fields.

I've tried passing the special characters as themselves (À[size=2]) and as html entities (À). I've tried putting cdata tags around the text in the xml tags. Nothing works.[/size]

[size=2]Code from DCT:[/size][HTML]

Site Information Architecture

[/HTML]

code from inline:
[HTML]#!E:\iw-home/iw-perl/bin/iwperl
use strict;
#---------------------------------------------------------------------
# Code body starts now
#---------------------------------------------------------------------
my $DCTCode;
$DCTCode = '';
$DCTCode .= 'À propos de nous';
$DCTCode .= '';
$DCTCode .= 'Page Title French';
$DCTCode .= 'À propos de nous';
$DCTCode .= '';
print '';
print '';
print $DCTCode;
print '';
exit(0);
[/HTML]

Image of results:

Any suggestions how to make this work?

Thanks

Find more posts tagged with

Comments

moonkhanna

Did not try this but you can try..

Write a function to set the value and trigger it onFormInit

GriffTheDog

Did not try this but you can try..

Write a function to set the value and trigger it onFormInit

I can write this function, but the fact is that the characters come back from the inline and display in the label tag fine. (as demonstrated in the image in the original message)

Doesn't that mean that there must be a way to make it display in a text field also?

moonkhanna

Yes, I think, you are right but the fact is that the Label tag is html wheras the content of your text field is raw text.. The very fact that your text field is not recognising the charater means that this is an encoding problem,.. I did not think abt that earlier...

If you do not use an inline and instead use raw XML for your DCT.. Is that working to populate the text field with your intended character?

GriffTheDog

This issue is only with inline. When I use a server callout I can use htm::entities on the characters and then use FormAPI to place them into the text box just fine.

moonkhanna

seems like formspublisher has turned naughty lately... Does not work with Inlines... ;-)...

Can you paste your server callout code to the post..

ISCBorisB

This issue is only with inline. When I use a server callout I can use htm::entities on the characters and then use FormAPI to place them into the text box just fine.

You have to provide true UTF-8 for "A Aggravate" symbol (that's two bytes in UTF-8) or provide it as a numeric Entity.

Try the following AS IS in your Inline to see what I mean and then modify it to your needs:
$DCTCode .= '<text><default>Ã€ OR À OR Àpropos de nous</default></text></item>';

Alternatively, you can use html::entities in the Inline to replace *default* String entities with the corresponding Unicode characters

Johnny

The problem is in the way perl treats file handles for UTF8 content.

By default all strings are managed as UTF8 but when you print something to the screen it gets converted to iso-8859-1. It also happens by default for sending/receiving data to all file handles.

To keep it as UTF8 put this at the top of your script...



use open ':utf8'; # makes sure that any file handles which you open are treated as utf8

binmode (STDOUT, ":utf8"); # these file handles are already open for you - change them to utf8
binmode (STDIN,  ":utf8");
binmode (STDERR, ":utf8");

This will let you read/write real unicode characters. Remember that if your text editor is not set up to read/write as utf8 then you may actually corrupt the data just by editing the script/dcr in your editor.

GriffTheDog

The problem is in the way perl treats file handles for UTF8 content.

I tried your suggestion but it had no effect.

I think the problem is in how the text item in the DCT handles the entities because the character shows up fine in other parts of the DCT using the exact some code, as demostrated in the gif in the original post.

Alternatively, you can use html::entities in the Inline to replace *default* String entities with the corresponding Unicode characters.

This worked. I take the html entity, decode it to it's actual value (html::entities::decode), convert it to a numeric value (html::entities::encode_entities_numeric) and then pass it to the DCT.

Thanks

ISCBorisB

This worked. I take the html entity, decode it to it's actual value (html::entities::decode), convert it to a numeric value (html::entities::encode_entities_numeric) and then pass it to the DCT

Plain Encode may be simpler. Like that

#!E:\iw-home/iw-perl/bin/iwperl
use strict;
#---------------------------------------------------------------------
# Code body starts now
#---------------------------------------------------------------------
use Encode;
my $cnv = Encode::encode_utf8('Àpropos de nous');

my $DCTCode;
$DCTCode = '<container name="pagetype" location="pagetype" min="0" max="unbounded">';
$DCTCode .= '<label>&#192;propos de nous</label>';
$DCTCode .= '<item name="title_f" pathid="title_f">';
$DCTCode .= '<label>Page Title French</label><description/>';
$DCTCode .= qq(<text><default>$cnv</default></text></item>);
$DCTCode .= '</container>';  
print '<?xml version="1.0" encoding="UTF-8"?>';
print '<substitution>';
print $DCTCode;
print '</substitution>';
exit(0);

moonkhanna

great post rating it with 5 stars

Johnny

That suprised me - it's got me over the line before. Sorry for the giving the wrong lead.

ISCBorisB

That suprised me - it's got me over the line before. Sorry for the giving the wrong lead.

Nothing to be sorry about. What you said in your post was correct, 100%.

The reason it did not make any difference to the Dog was that he used "Named Entity", À. It just *represents* symbol À by naming convention.
Formally though it's 8 one-byte symbols in certain order, pure ASCII. As we all know, UTF-8 for symbols 0x00 - 0x7F is well, just that same symbol.

Johnny

Ahh well the original post did mention the use of the real unicode character as well as the entity names (which would not translate in a text field). Maybe when he tried it out he used the entity name instead of the actual unicode characters.

Thanks - I'm not so baffled now Smiley Happy

GriffTheDog

Johnny -

ISCBorisB is correct. When I tried your advice I was still using the html Entity. To make you advice work, I discovered after, I needed to decode the html entity to it's true value.

Thanks!