DataDeploy Oracle and UTF-8 characters

When deploying TeamSite metadata to Oracle using Datadeploy standalone it appears UTF-8 characters are not being encoded correctly in the database.

Example: Metadata value has the en dash character UTF-8 encoded (â\200\223 when viewed in a text file).

When this value is deployed to an Oracle varchar2 column the character is translated to a non-displayable character.

Versions are Oracle 9.2.0.5 and OpenDeploy v6.0.2.

Any ideas what the problem could be ? I thought JDBC Oracle dirvers were meant to automagically do character conversions.

Find more posts tagged with

Comments

marcussteiner

That should not be a problem. Did you check what it looks like on your website? Are you also sure your database is UTF-8 encoded? run "SELECT * FROM nls_database_parameters" to check. If possible try it once with data and once with metadata and compare the results.

1_original_tree_ok.jpg

2_ecplise_design_view.jpg

3_birt_html_result.jpg

4_birt_ppt_result(wrong).jpg

array_pitree.rptdesign

jed

When deploying TeamSite metadata to Oracle using Datadeploy standalone it appears UTF-8 characters are not being encoded correctly in the database.

Example: Metadata value has the en dash character UTF-8 encoded (â\200\223 when viewed in a text file).

When this value is deployed to an Oracle varchar2 column the character is translated to a non-displayable character.

Versions are Oracle 9.2.0.5 and OpenDeploy v6.0.2.

Any ideas what the problem could be ? I thought JDBC Oracle dirvers were meant to automagically do character conversions.

On a few boxes we had to add the following to the OD startup script to force UTF-8 encodings.

1. Open $odhome/processod
2. Make sure -Dfile.encoding=UTF-8 is one of the java options for Solaris/Linux JAVA_OPTS variable. Example:
JAVA_OPTS="-Dfile.encoding=UTF-8 -server -Xsqnopause -XX:+PrintClassHistogram -XX:+PrintGCDetails -verbose:gc -Xss2m -Xms512m -Xmx1024m"
3. Restart OpenDeploy
4. Deploy test data and verify data is inserted as UTF-8

Generally, I only had to do this on Solaris 10 receivers that did the DD via dbLoader, so my situation is different, but it might be worth a try.

1_original_tree_ok.jpg

2_ecplise_design_view.jpg

3_birt_html_result.jpg

4_birt_ppt_result(wrong).jpg

array_pitree.rptdesign

debus

Thanks for the sugesstions.

The Oracle encoding is
NLS_CHARACTERSET = WE8ISO8859P1
NLS_NCHAR_CHARACTERSET = AL16UTF16

The target columns are vchar2 and not nvarchar or nchar. The default encoding for Oracle is NLS_CHARACTERSET = WE8ISO8859P1, so it appears this should be changed to AL16UTF16 or UTF8.

Changing the startup options for OpenDeploy is also worth a try.

Thanks again.

1_original_tree_ok.jpg

2_ecplise_design_view.jpg

3_birt_html_result.jpg

4_birt_ppt_result(wrong).jpg

array_pitree.rptdesign

mstradling

It's the Oracle encoding that is causing your problems - from memory Oracle will be re-encoding the characters as they are written to the tables.

1_original_tree_ok.jpg

2_ecplise_design_view.jpg

3_birt_html_result.jpg

4_birt_ppt_result(wrong).jpg

array_pitree.rptdesign