nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Breaking encoding from .page -> html

cesarecastro

Hello Everyone,

We just upgraded from TS 6.7.2 to 7.3.1 but we have a significant defect. When a user creates/edits a .page, the HTML that is produced that contains any character other than ascii is replaced with ????? marks or squares.

We have searched for a solution in the documentation to check if the page conversion servlet required a new param but it seem like it doesn't. Has anyone else encouter this issue?

Our DCRs are working correctly. The content shows correctly on the .page.

Find more posts tagged with

Comments

Ranveer

Hello Everyone,

We just upgraded from TS 6.7.2 to 7.3.1 but we have a significant defect. When a user creates/edits a .page, the HTML that is produced that contains any character other than ascii is replaced with ????? marks or squares.

We have searched for a solution in the documentation to check if the page conversion servlet required a new param but it seem like it doesn't. Has anyone else encouter this issue?

Our DCRs are working correctly. The content shows correctly on the .page.

Does this issue appear on preview? I have also other char-set but not facing this issue. I guess this issue is related to OS.

Also check from other system/browser.

Adam Stoller

Hello Everyone,

We just upgraded from TS 6.7.2 to 7.3.1 but we have a significant defect. When a user creates/edits a .page, the HTML that is produced that contains any character other than ascii is replaced with ????? marks or squares.

We have searched for a solution in the documentation to check if the page conversion servlet required a new param but it seem like it doesn't. Has anyone else encouter this issue?

Our DCRs are working correctly. The content shows correctly on the .page.

Weren't you using 7.2.1 before?
Anyway - check the browser's Text (or Character) Encoding settings - chances are it's set for something like Western - and you probably want it set to UTF-8.

If that's the issue, then you probably need to make sure to inject a meta tag in the generated HTML:[html]<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />[/html] to try to force the browsers to render the text with the correct encoding (this is, of course, assuming you're not dealing with something like Chinese characters, which I believe needs a different character encoding setting)

Also - look out for text which originated in Word and was copy/pasted into components - If people are going to use Word as their initial mechanism for generating text - try to teach / coerce them into turning off things like "smart quotes" and "em-dashes" - as those tend to be the biggest source of pain when the text gets stored in XML and then later rendered as HTML.

cesarecastro

This issue does not appear on the preview window or when a user is working on the component.

It only appears in the generated HTML page. This happens with existing content that we have managed for months and works correctly in 6.7.2. We also tested on 7.2.1 and it works correctly BUT on 7.3.1 it does not.

We tested on different computers with the correct browser settings.

Thanks Ghoti and Raveer for your comments

Rick Poulin

the HTML that is produced that contains any character other than ascii is replaced with ????? marks or squares.

And if ghoti's suggestion doesn't fix your issue, you'll have to clarify the above statement. Generating HTML or using Display Services? .NET or Java front-end?

Open up your .page in a decent text editor (e.g. EditPlus or Notepad++) and try to figure out if the character encoding is ok pre-render. There's no reason for it not to be unless your users are pasting MSWord garbage into non-RTE fields (TinyMCE does a fair but not perfect job at converting to entities). If the characters are still ****, what do they look like? (the garbage can indicate the encoding, for those of us that have seen too much of it)

More than likely, your front-end web or app server isn't correctly setting the Content-type header's charset portion, or the browser you're using has an explicit override that isn't set to UTF-8. Use firebug to inspect the content type coming back, and use Firefox's "Page Info" to see what encoding the browser actually used to interpret the page content.

cesarecastro

Hi rpoulin,

The statement generating html from a .page, is just that. When a user creates or edits a .page, a pageconversion servlet generates an html under the html/site folder.

The very first thing I did when we saw the problem on our website, was to go to the generated file in TeamSite and check the encoding. It is UTF-8. This issue exist only on 7.3.1 becuase if you take exactly the same content and generate the same html on versions 6.7.2 and 7.2.1, the generated HTML is good.

Support just provided me with a file to replace on the 7.3.1 file and test. I will post our results.

cesarecastro

TeamSite 7.3.1 has a known issue where character encoding is lost when html is produced from a .page. The support engineers have a patch for this. The effects where most visible with Chinese, replaced all characters with ????marks, but it also affected other languages.

Interesting enough, the patch also seems to have fixed an issue where blank/empty html files where being produced from .pages containing content.

Adam Stoller

TeamSite 7.3.1 has a known issue where character encoding is lost when html is produced from a .page. The support engineers have a patch for this. The effects where most visible with Chinese, replaced all characters with ????marks, but it also affected other languages.

Interesting enough, the patch also seems to have fixed an issue where blank/empty html files where being produced from .pages containing content.

Can you post the case number used for this issue, in case others need to reference it to retrieve the same patch?