Clear accents from input file

Hi,

I was trying to remove accents (french,german..etc) from input file by using several modules (Text::Unaccent...etc), but nothing seems to be as effective as following ~s/[^[:ascii:]]//g;.

But, this too has its pitfall. It replaces with space, but does not bring back the English equivalent character for same.

Any thought on this is much appreciated.

Thanks.

Find more posts tagged with

Comments

prithvi

Additionally, I'm trying to write to a XML file with strings containing French accented characters.

open FILE,">:utf8","$DCR/fr-CA/Products.xml";

But, when I open XML file, I get following error

The XML page cannot be displayed

option value="1" label="Complexe r & eacute

é get's written out as & e acute;

Not sure what else to use.

Adam Stoller

Additionally, I'm trying to write to a XML file with strings containing French accented characters.

open FILE,">:utf8","$DCR/fr-CA/Products.xml";

But, when I open XML file, I get following error

The XML page cannot be displayed

option value="1" label="Complexe r & eacute

é get's written out as & e acute;

Not sure what else to use.

XML has a very limited set of predefined character entities.
I suggest you look at these references (and other such references that you can find via Google) to learn how to reference other characters using their hexadecimal unicode specifications:

prithvi

Thanks a ton Ghoti. Your replies are always extensive. I'm not sure why inspite of using UTF-8 in the XML header, character's tend to convert to Hex.

We are currently planning to implement a low complexity solution by writing out the output to text file instead of XML. Will post my update soon.

Thanks for your response Ghoti.

Adam Stoller

I'm not sure why inspite of using UTF-8 in the XML header, character's tend to convert to Hex.

Perhaps you didn't do the reading that was suggested? It's wise to use UTF-8, but that in and of itself does not extend the list of named entities that XML will accept. Once you've gotten past those few named entities everything else must be in hex (or must be declared / defined within the DOCTYPE header section)