deploying international chars

i had a curious problem where deployment failed because one of the fields was over allowed size. When i looked at the saved DCR i couldnt find the problem with any of the elements.

the elements use japanise chars, if i VI the DCR i will see something like this:
BPHã~A«ã~A¤ã~A~Dã~A¦
which makes me wonder if japanise chars take up 2x the space when they are saved, and if the max length was 100 then it would only support 50 japanise characters.

am i on the right track?

also:
the text element has a max-length attribute. is this enough to contstrain length? or should i also be using validation-regex attribute?

snacks,
Mikhail

Find more posts tagged with

Comments

ISCBorisB

i had a curious problem where deployment failed because one of the fields was over allowed size. When i looked at the saved DCR i couldnt find the problem with any of the elements.

the elements use japanise chars, if i VI the DCR i will see something like this:
BPHã~A«ã~A¤ã~A~Dã~A¦
which makes me wonder if japanise chars take up 2x the space when they are saved, and if the max length was 100 then it would only support 50 japanise characters.

am i on the right track?

also:
the text element has a max-length attribute. is this enough to contstrain length? or should i also be using validation-regex attribute?

snacks,
Mikhail

Yes, you are on the right track.

1. "Japanese characters" is a loaded term. It usually (but not always!) means Katakana, Hiragana or Kanji hieroglyphics.
IWOV TS standard encoding is UTF-8, in it Japanese symbols are represented by 2-3 bytes each - normally 3.

2. For 'text' max-length is sufficient, 'textarea' however does not have it, some form of validation regex is an option.

koryakm

if this is the case, should i tell the japanese content creators that they must make all of their content about 1/3 of the size of US content because of a teamsite limitation? that would be bad.

ISCBorisB

if this is the case, should i tell the japanese content creators that they must make all of their content about 1/3 of the size of US content because of a teamsite limitation? that would be bad.

Of course not. You have to make sure that your design satisfies business requirements, be it English, Japanese or Martian encoding.

koryakm

please enlighten this n00b about the following:

- you said that japanese chars may be 2-3x bigger
- my db column is setup as nvarchar(100)

will 100 japanese chars fit into my nvarchar(100) ?

- i have a deployment which which failed because i tried to insert 104 chars into my nvarchar(100), yet on the DCT the textfield in question was set up as max-length="100"

i am trying to understand what part of my process failed to allow 104 chars to come over

Thank you for your time.

ISCBorisB

>>you said that Japanese chars may be 2-3x bigger
I never said that. Please note that as soon as you start talking Unicode 'symbol', 'char', 'character' and 'byte' stop being synonyms!
What I said was that TS uses UTF-8. That in turn means that Japanese hieroglyphs(symbols) are saved as 2-3 bytes (16 - 24 bits).
That's what you see in VI.

>>my db column is setup as nvarchar(100). Will 100 Japanese chars fit into my nvarchar(100)?
I *think* you should be Ok.

As far as I recall, 'nvarchar' uses UCS-2. that's 2-byte encoding which in turn is an obsolete predecessor to UTF-16.
Both Hiragana and Katakana are 2 byte codes so as long as your Content Contributors use only them you are fine. Kanji is
3-bytes in UTF-8 (and there are much more of those than the other two combined) but I *think* that's rarely used in Internet.
Still, you'd be wise to confirm that with your Japanese Contributors.

>>I am trying to understand what part of my process failed to allow 104 chars to come over...
You'd have to debug it, what else? Open up a DCR, paste 101-letters English phrase (that'll be one byte symbols in UTF-8) and follow your procedures.

koryakm

thank you for your response!