Tags

, , , ,

I will be launching CRYPTOSTAR.MONEY soon (“Find Rated Crypto Businesses”) which is built on WordPress.

JSON Requires Straight Quotes

Many integrations are required and at one point I stash some JSON from the database in a DIV and then read it into Javascript (it would be better to use AJAX but I am doing this step-by-step). When I

JSON.parse(document.getElementById('jbMPJSON').innerHTML)

I get the error shown below.

quote-error

Can you see the problem?

The first double-quote is bent – it is a not the straight double-quote required by JSON. The JSON in the database is valid so the process of writing it out of the database into the page has corrupted it.

It’s a WordPress feature not a bug

A bit of Googling shows that this is a feature of WordPress. WordPress changes ‘ugly’ straight quotes to pretty curvy ones. This breaks any website that publishes PHP or Javascript tutorials and it can be changed by removing filters in functions.php:

remove_filter(‘the_content’, ‘wptexturize’);

You can also remove the curvy quotations from titles by adding this code:

remove_filter(‘the_title’, ‘wptexturize’)

 A note on character encoding

Javascript uses very old UCS-2 scheme to encode characters. This a two-byte scheme that covers the first 65,535 used on the web. UCS-2 was later extended to cover more characters and UTF-16 was born. Web pages and URLs are usually encoded using the more compact UTF-8 scheme.

If a JSON string won’t parse you can URI encode it to quickly detect unicode characters.

alert(encodeURI(mpJSON));
Encoding my URI showed “%E2%80%9C” and searching “character e2809c” gave me the result.
char

This is very helpful because by referring to the UTF-16 encoding (coincident with the UCS-2 encoding used by JavaScript) I can now use the Javascript string replace method to replace the illegal single and double quotes.

var goodQuotes = badQuotes
  .replace(/[\u2018\u2019]/g, "'")
  .replace(/[\u201C\u201D]/g, '"');

It is possible to be even more aggressive and remove all non-ascii characters from a string.

str.replace(/[^\x00-\x7F]/g, "");

Summary

Although stashing data for use by JavaScript into a hidden div has some advantages over AJAX because it is available as soon as the pages loads it is fragile because WordPress or other plugins may filter (modify) it. It is possible to disable the WordPress curly quote feature but in general developers may wish to sanitise their strings before using JSON.parse.

Advertisements