Encoding problem

[pca]

Hi everyone

 

 

 

I'm having a problem with some characters. I've searched and I know that I have to add the BOM bytes for UTF-8 encoding.

 

However, I've searched the forum and then googled and I can't seem to find a solution for my problem.

 

 

 

I am using xslt to dynamically create my xml file and I would like to know how can I add the BOM to the XML file.

 

 

 

Thanks in advance

[srividya_sharma]

Hi

Which server-side technology are you using?

Please post part of your code.

Edited April 29, 2009 by Guest

[pca]

I'm using the Apache Cocoon Framework, on a Tomcat server.

 

 

 

As for the sample code I sent it as an attachment.

 

 

 

If you need more, please let me know.

codesample.txt

[srividya_sharma]

Hi

Are you using dataXML or dataURL method to set the xml to the chart?

Can you post the jsp code where the xml is being set to the chart.

Thanks

[pca]

I'm using setDataURL. The link used leads to the sitemap pattern get-chart-xml that was on the attachment in my previous post.

 

 

 

buildFusionChart = function(id, chartXML, isPie, multiAxis, width, heigth, CHARTS_PATH, renderDivId) {

 

var registerWithJS = "1", c = null, scaleMode = null, lang = null, detectFlashVersion = null, autoInstallRedirect = null ;

 

var debug = "0" ;

 

var swfUrl = "" ;

 

 

 

if (isPie == "1") {

 

swfUrl = CHARTS_PATH + "charts/Pie3D.swf" ;

 

} else {

 

if (multiAxis == "1") {

 

swfUrl = CHARTS_PATH + "charts/MSCombiDY2D.swf" ;

 

} else {

 

swfUrl = CHARTS_PATH + "charts/MSCombi2D.swf" ;

 

}

 

}

 

 

 

var chart = new FusionCharts(swfUrl, id, width, heigth, debug, registerWithJS, c, scaleMode, lang, detectFlashVersion, autoInstallRedirect) ;

 

chart.addParam("WMode", "Transparent");

 

chart.setDataURL(CHARTS_PATH + escape(chartXML));

 

chart.render(renderDivId);

 

}

 

 

 

Thank you for your assistance

[srividya_sharma]

Hi

Here is what I have understood about your application:

1. You are using javascript to set the dataURL to the chart.

2. The url is "get-chart-xml" which is mapped in your application to finally generate xml using generate-graphic-document.xsl.

There are several components involved in this process of creating the xml.

My first suggestion would be, to add the following statement:

<xsl:output encoding='utf-8'/>

Include this statement just after

<xsl:stylesheet xmlns:xsl ...>

in your generate-graphic-document.xsl file.

[pca]

Your understanding of the application is correct, and thank you for your suggestion, but unfortunately I have already tried it and the result was the same. I tried using it on the file you suggested, and on every file that helps to create the xml for the charts at the same time. I also tried passing the xml through another xsl with the sole purpose of converting to utf-8 and it still didn't work.

 

 

 

I have also tried to convert the strings to utf-8 using javascript, with and without .

 

 

 

I may be doing something wrong, but I still haven't discovered what it is.

[srividya_sharma]

Suggestions:

1. Try saving the xsl with UTF-8 encoding using notepad or other editors.

2. Also, first try by giving url to a xml file instead of the xsl mapping, get it to work and then try the xslt process.

Can you please attach the screen-shot of the output and the html source that gets generated ( from view source ) ?

[pca]

Hello and thank you again for your help

 

 

 

1. Try saving the xsl with UTF-8 encoding using notepad or other editors.

 

 

 

This didn't work. I've tried it with multiple files and the result was the same.

 

 

 

2. Also, first try by giving url to a xml file instead of the xsl mapping, get it to work and then try the xslt process.

 

 

 

With the static file it worked. Using the xslt code it still doesn't work.

 

 

 

In attachment I send the html for the chart and a screenshot of a chart with the wrong characters.

htmlcode.txt

[srividya_sharma]

Hi

When there are several pieces involved, missing UTF-8 encoding even in one page/component, causes this problem.  I know it is very frustrating to get it all to work, but at the end, it will all look very simple and nice. Just wondering, are you getting the data from database too? ( that would be one more component then )

If you get ??? in your output, then UTF-8 has been completely missed out but from the screen-shot, it seems that the data is getting converted to some other encoding somewhere in between. ( this might also imply that "Absence of BOM in the xml" is not causing much problem )

Please check out this document on UTF-8 in Cocoon and follow all the steps.

http://cocoon.apache.org/2.2/1366_1_1.html

Hope this works

Srividya

[pca]

Hi

 

 

 

I decided to change my strategy, and instead of using setDataURL, I now use setDataXML and the encoding problem appears to be solved. I create the chart without giving it any XML and then use an Ajax request to obtain the XML and send it to the chart using setDataXML.

 

 

 

However, I ended up trading one problem for another.

 

 

 

For some unexplained reason the chart sometimes does not load the XML or it returns a error in the XML when there is none.

 

 

 

My current code is in the attachment.

chartCode.txt

[srividya_sharma]

Hi

Before calling the render function on the chart Object, you have to initialize the xml and ( optional ) give an initial message.

This can be done like this:

swfUrl = CHARTS_PATH + "charts/MSCombi2D.swf?ChartNoDataText=Loading..."

chart.setDataXML("<chart></chart>");

chart.render( ... )

Hope this solves the issue.

[pca]

Hi

 

 

 

Your solution did help in eliminating the errors. But I'm still left with the question of why some times the chart loads, and others it doesn't.

 

 

 

After some debugging I found that whatever it is, it happens during the instruction: chartObj.setDataXML(transport.responseText);

 

 

 

Since the request is asynchronous could the cause be the application is trying to supply the chart with XML when the chart isn't fully loaded?

 

 

 

Thank you for your help so far

[FusionCharts Support]

Hi,

 

 

 

You can write a function FC_Rendered(DOMId) in JavaScript which gets called from chart when chart loads completely. Please update chart only after that.

[pca]

Hello.

 

 

 

Although I still can't say what, exactly, was wrong with my previous code, I was able to solve it by implementing a similar solution.

 

 

 

First I wrote my code to load the chart on the onSuccess method of the Ajax request. This way I could easily get rid of errors caused by the request being asynchronous.

 

 

 

This didn't solve my problem. In fact it made it worst because before the chart did load sometimes and like this it didn't load at all, returning an error in the XML. I made an output of the XML and stored it in a file to use it with xmllint to check for errors. It didn't find any. I then used that same XML file with a chart on one of the Evaluation pages and it worked.

 

 

 

Moving back to the application, I used the javascript escape function on the XML before sending it to the chart, and it worked but text with accented characters was all gibberish.

 

 

 

Long story short, after playing a little with static strings containing a sample XML I found what was causing the XML error. It was the quote marks ("). I replaced them with apostrophes ('), and while I was at it, I also URL encoded the characters & and % that might appear in the XML.

 

 

 

I can now build the charts correctly, with the right characters and without errors (so far...). It still eludes me why in the previous setting the chart load only on some occasions and mostly after trying several times to load a chart. The XML sent to the chart was the same, but sometimes it would return an error.

 

 

 

I send the working code in attachment. Hope it helps someone

 

 

 

Thank you all for your help

workingCode.txt

[saptarshi]

Hello,

 

 

 

Good to see that it is working now

 

 

 

And thank you for sharing the code.