Forum

Thread tagged as: Problem, Error, Twitter

Twitter App - tweets with emojis at the beginning

Hi,

I am having a small issue with the Twitter app, where when it comes across a tweet with emojis at the beginning it displays an empty result. I am calling it as standard like so:

perch_twitter_get_latest([
  'twitter_id'=>'@Casino36',
  'type'=>'mine',
  'count'=>1,
  'exclude_replies'=>1
]);

And using the default template. However when it comes across a tweet with emojis at the beginning it returns this:

<blockquote class="twitter-tweet">
<p dir="ltr" lang="en">
</p></blockquote>

The tweet column within Perch is also empty.

This is an example of one of the tweets that threw it off:

https://twitter.com/Casino36/status/994975036552433665

This may be a limitation with Twitter rather than the Perch implementation but thought I would mention it. My short diagnostics is below.

Thanks

Mike


Perch Runway: 3.1.1, PHP: 5.6.31, MySQL: mysqlnd 5.0.11-dev - 20120503 - $Id: 76b08b24596e12d4553bd41fc93cccd5bac2fe7a $, with PDO Server OS: Darwin, apache2handler Installed apps: content (3.1.1), assets (3.1.1), categories (3.1.1), perch_blog (5.6.1), perch_forms (1.11), perch_twitter (4.0) App runtimes: <?php $apps_list = [ 'perch_forms', 'perch_twitter', 'perch_blog' ]; PERCH_LOGINPATH: /admin PERCH_PATH: /Users/mikeharrison/Google Drive/Client Work/Depict Creative/Casino 36 Website/Site/admin PERCH_CORE: /Users/mikeharrison/Google Drive/Client Work/Depict Creative/Casino 36 Website/Site/admin/core PERCH_RESFILEPATH: /Users/mikeharrison/Google Drive/Client Work/Depict Creative/Casino 36 Website/Site/admin/resources Image manipulation: GD PHP limits: Max upload 32M, Max POST 32M, Memory: 256M, Total max file upload: 32M F1: 3b606135b33e6a102526838f4152a807 Resource folder writeable: Yes HTTP_HOST: casino36.loc DOCUMENT_ROOT: /Users/mikeharrison/Google Drive/Client Work/Depict Creative/Casino 36 Website/Site REQUEST_URI: /admin/core/settings/diagnostics/ SCRIPT_NAME: /admin/core/settings/diagnostics/index.php
Mike Harrison

Mike Harrison 36 points

  • 3 months ago

I have had similar trouble with non standard content in Instagram posts. A shortcodes app we built failed on non standard content. We never solved it but my main suspect was the character-set and php's handling of the values. ( or lack of ). Love to know how you go.

It's fine with ones with emojis inside - shows them really well. For example:

https://twitter.com/Casino36/status/991709530047344640

Shows up exactly as I would expect:

<blockquote class="twitter-tweet"><p dir="ltr" lang="en">ITS ALMOST TIME!<br><br>⚽WATCH CHAMPIONS LEAGUE ON OUR BIG SCREENS !⚽<br><br>TONIGHT!<br>ROMA V LIVERPOOL - 19:45 - LIVE HERE at Casino 36 Wolverhampton in our SPORTS BAR &amp; GRILL<br><br>For further information please visit us at <a href="https://t.co/cxwZgjee3k">https://t.co/cxwZgjee3k</a>  <a href="https://twitter.com/hashtag/Sport?src=hash&amp;ref_src=twsrc%5Etfw">#Sport</a> <a href="https://twitter.com/hashtag/Casino?src=hash&amp;ref_src=twsrc%5Etfw">#Casino</a> <a href="https://twitter.com/hashtag/UefaChampionsLeague?src=hash&amp;ref_src=twsrc%5Etfw">#UefaChampionsLeague</a> <a href="https://t.co/rGDf8kyPui">pic.twitter.com/rGDf8kyPui</a></p>— Casino 36 (@Casino36) <a href="https://twitter.com/Casino36/status/991709530047344640?ref_src=twsrc%5Etfw">May 2, 2018</a></blockquote>

It's just where the tweet starts with one where the issue is

Drew McLellan

Drew McLellan 2578 points
Perch Support

If you have the option, switch the encoding of your tweets table from utf8 to utf8mb4.

Done that, no joy I'm afraid. I can just let the client know that they won't be able to structure tweets that way in the future.

Drew McLellan

Drew McLellan 2578 points
Perch Support

Have you imported the tweets since updating the table?

Yes I clicked "Get Tweets" in the Twitter app, still coming up blank for those starting with an emoji. Those just containing an emoji are fine.

To confirm my perch_3_twitter_tweets table collation is now set to utf8mb4_general_ci. Type is MyISAM if that is of use.

Good tip Drew. I'm going to try that on my Insta shortcode issue.

Drew McLellan

Drew McLellan 2578 points
Perch Support

Clicking Get Tweets only gets new tweets. It won't re-fetch existing tweets that haven't been stored correctly.

The issue is how MySQL interprets the unicode characters, so once it's been stored badly just changing the table properties won't fix it.