Forum

Thread tagged as: Problem, Error

Chinese content not displaying on a multilingual site

I have an issue where some Chinese content is not displaying on a multilingual site. I have done some research and it seems that it could be something to do with character encoding? I have downloaded the database and installed the site on my local environment (MAMP), republished the content and all the content is displayed as it should be. Republishing on the live site doesn't have the same effect, the content remains missing. Could this be due to the windows server?

When I try and save the page, I get this error in the Debug info:

Invalid query: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xE7' for column 'indexValue' at row 1

Diagnostics report:

Perch: 2.8.32, PHP: 5.4.45, MySQL: mysqlnd 5.0.10 - 20111026 - $Id: c85105d7c6f7d70d609bb4c000257868a40840ab $, with PDO
Server OS: WINNT, cgi-fcgi
Installed apps: content (2.8.32), assets (2.8.32), categories (2.8.32)
App runtimes: <?php $apps_list = array( 'content', 'categories', );
PERCH_LOGINPATH: /admin
PERCH_PATH: D:\home\site\wwwroot\admin
PERCH_CORE: D:\home\site\wwwroot\admin\core
PERCH_RESFILEPATH: D:\home\site\wwwroot\admin\resources
Image manipulation: GD
PHP limits: Max upload 10M, Max POST 10M, Memory: 256M, Total max file upload: 10M
F1: 0243314cd872a1b8ddf8ff917af81c7e
Resource folder writeable: Yes
SCRIPT_NAME: /admin/core/settings/diagnostics/index.php
REQUEST_URI: /admin/core/settings/diagnostics/
DOCUMENT_ROOT: D:\home\site\wwwroot
HTTP_HOST: www.domainname.com
Andy Knight

Andy Knight 1 points

  • 4 years ago
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

Is everything UTF8 throughout?
Andy Knight

Andy Knight 1 points

4 years ago

Drew McLellan said:

Is everything UTF8 throughout?

Hi Drew,

Yes, I think so, I certainly haven't specified anything other than defaults.
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

Are you serving your pages as UTF8?
Andy Knight

Andy Knight 1 points

4 years ago

Yes...

<meta charset="utf-8">
Andy Knight

Andy Knight 1 points

4 years ago

Hi Drew,

I have isolated the problematic characters as being commas ','. If I remove them from the text and save the content, the page loads as expected. Here is an example piece of text that causes an issue.

随后我们对基金进行严格的统计分析，查看各种历史表现和风险指标，然后使用股票及固定收益基金、以及某些消极型指数基金，根据投资组合中不同资产类别之间的目标配置来选择基金。
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

Those don't look like normal commas. What are they?
Andy Knight

Andy Knight 1 points

4 years ago

They seem to be a character in their own right. I have replaced them with a regular comma + space but the page is still not displayed. Strangely, they are used in other areas on the site that is displaying correctly. If i just have a regular comma without the space, the page saves and the content is displayed.
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

Can you show me the template?
Andy Knight

Andy Knight 1 points

4 years ago

Sure, FYI this seems to be an issue with the database on the clients setup, I have it working correctly on my local setup. If I export my local db tables and import them to the live server all works as expected and the page content shows. If you go to the region and press save the content disappears. Full template below. The bit we are having issues with is <perch:content id="product-intro" type="textarea" html="true" editor="ckeditor" label="Product Intro" help="Use H2 for headings" />

<div class="product">

    <section class="">
        <div class="container">
            <div class="row">
                <div class="col-sm-12 dotted-top">
                    <perch:content id="product-intro" type="textarea" html="true" editor="ckeditor" label="Product Intro" help="Use H2 for headings" />
                </div>
            </div>
        </div>
    </section>

    <section class="fund-selection-row">
        <div class="container">
            <div class="row">
                <div class="col-sm-12 dotted-top-single">
                    <h3><perch:content id="row-title" type="text" label="Row Title" size="xl" divider-before="Fund Selection and Monitoring" /></h3>

                    <div class="table-responsive">
                        <table class="table table-striped"> 
                            <thead>
                                <tr>
                                    <th width="33.333%"><perch:content id="column-heading-1" type="text" label="Column Heading 1" size="xl" help="e.g. Performance" /></th>
                                    <th width="33.333%"><perch:content id="column-heading-2" type="text" label="Column Heading 2" size="xl" help="e.g. Volatility" /></th>
                                    <th width="33.333%"><perch:content id="column-heading-3" type="text" label="Column Heading 3" size="xl" help="e.g. Strategy & Assets" /></th>
                                </tr>
                            </thead>
                            <tbody>
                                <perch:repeater id="table-row" label="Table Row">
                                    <tr>
                                        <td><perch:content id="column-1" type="text" label="Column 1" size="xl" /></td>
                                        <td><perch:content id="column-2" type="text" label="Column 2" size="xl" /></td>
                                        <td><perch:content id="column-3" type="text" label="Column 3" size="xl" /></td>
                                    </tr>
                                </perch:repeater>
                            </tbody>
                        </table>
                    </div>
                </div>
            </div>
        </div>
    </section>

    <section class="model-portfolios-row">
        <div class="container">
            <div class="row">
                <div class="col-sm-12 dotted-top-single">
                    <h3><perch:content id="section-heading" type="text" label="Section heading" size="xl" help="e.g. Northstar Global VIP Model Portfolios" divider-before="Model Portfolios Section" /></h3>
                </div>
                <perch:blocks>
                  <perch:block type="model-portfolios-row" label="Model Portfolios Row">
                    <div class="col-sm-12 dotted-bottom-single">
                            <div class="row">
                                <div class="col-sm-6">
                                    <h4><perch:content id="model-portfolios-row-title" type="text" label="Row Title" size="xl" /></h4>
                                    <div class="row">
                                        <div class="col-sm-4">
                                            <img src="<perch:content type="image" id="icon" label="Icon" />" class="img-responsive" />
                                        </div>
                                        <div class="col-sm-8">
                                            <perch:content id="model-portfolios-row-text" type="textarea" html="true" editor="ckeditor" label="Row Text" help="Use H4 for headings" />
                                        </div>
                                    </div>
                                </div>
                                <perch:if exists="model-portfolios-row-title-2">
                                    <div class="col-sm-6">
                                        <h4 class="alternate"><perch:content id="model-portfolios-row-title-2" type="text" label="Row Title" size="xl" /></h4>
                                        <div class="row">
                                            <div class="col-sm-4">
                                                <img src="<perch:content type="image" id="icon-2" label="Icon" />" class="img-responsive" />
                                            </div>
                                            <div class="col-sm-8">
                                                <perch:content id="model-portfolios-row-text-2" type="textarea" html="true" editor="ckeditor" label="Row Text" help="Use H4 for headings" />
                                            </div>
                                        </div>
                                    </div>
                                </perch:if>
                            </div>
                        </div>
                  </perch:block>
                </perch:blocks>
            </div>
        </div>
    </section>

    <section class="notes">
        <div class="container">
            <div class="row">
                <div class="col-sm-12 text-center dotted-top-single">
                    <perch:content id="notes" type="textarea" html="true" editor="ckeditor" label="Notes" divider-before="Notes" help="" />
                </div>
            </div>
        </div>
    </section>

</div>
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

You're using CKEditor? Have you turned that off to eliminate it as a cause?
Andy Knight

Andy Knight 1 points

4 years ago

Just tried it, changed to markitup and saved but still not working.
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

Have you checked the table encoding?
Andy Knight

Andy Knight 1 points

4 years ago

Encoding: UTF-8 Unicode (utf8) Collation: utf8_general_ci
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

What's different between that server and yours?
Andy Knight

Andy Knight 1 points

4 years ago

Local:

Perch: 2.8.32, PHP: 5.6.10, MySQL: mysqlnd 5.0.11-dev - 20120503 - $Id: 3c688b6bbc30d36af3ac34fdd4b7b5b787fe5555 $, with PDO
Server OS: Darwin, apache2handler

Live

Perch: 2.8.32, PHP: 5.4.45, MySQL: mysqlnd 5.0.10 - 20111026 - $Id: c85105d7c6f7d70d609bb4c000257868a40840ab $, with PDO
Server OS: WINNT, cgi-fcgi
Drew McLellan

Drew McLellan 2638 points
Perch Support

4 years ago

You might need to delve a bit deeper with your server admin.