01530: emoji could/should display inside editor
Description: Emojis are excellent for display in a page. When editing a page they are converted to ugly looking text.
There could be an option to display emojis in the editor to make it easier for most people to use.
For instance:๐บ๐ธ usa flag
๐ด๓ ต๓ ณ๓ ด๓ ธ๓ ฟ texas flag
๐ฆ dinosaur
๐คก clown
๐ world
in the editor look like:
🇺🇸 usa flag 🏴󠁵󠁳󠁴󠁸󠁿 texas flag 🦕 dinosaur 🤡 clown 🌎 world
-- gnuzoo
This is already available, see the source code of this page: ๐ป ๐ฅ๏ธ โจ๏ธ ๐ฑ๏ธ ๐ง ๐ค ๐งโ๐ป ๐จโ๐ป ๐ฉโ๐ป ๐จ๏ธ ๐พ ๐ ๐ก ๐ ๐ โก ๐งฌ ๐ฌ ๐ญ ๐งช ๐งซ ๐งฏ ๐ฐ๏ธ ๐ ๏ธ โ๏ธ ๐งฎ ๐ ๐ ๐งโ๐ฌ ๐จโ๐ฌ ๐ฉโ๐ฌ ๐งโ๐ ๐จโ๐ ๐ฉโ๐ ๐๏ธ ๐๏ธ ๐๏ธ ๐งพ ๐ถ ๐ ๐ ๐ ๐งฉ ๐ค ๐ ๐ ๐น๏ธ ๐ฎ ๐ช ๐งฒ. Enable UTF-8 to use it. Petko
What am I missing? If emojis display normally in the page 'view', why cant emojis display normally in the page 'edit' in my wiki? --gnuzoo
You seem to have missed reading the last sentence of my previous comment. Petko
It may work on this page, but does not work when editing this PmWiki page โก๏ธ GNUZoo Emojis. -- gnuzoo
If you type the HTML entities like 🐒=๐ in the page, they are not supposed to be converted to characters. Petko
Do you mean if they are in the form '🐒' they are not converted to '๐' after being saved and then edited later ? -- gnuzoo
When UTF-8 is enabled, whatever writers type in the edit form stays in the edit form (unless $ROEPatterns or $ROSPatterns change it). Petko
I do not understand. I can see emoji images in a page 'view'. I can see emoji
images in the editor when I paste them. After saving the page and viewing the
page I can see emoji images in a page 'view'. When I edit the page again the
prior emoji images are converted to text like '🐒'. While editing
again I can paste more imoji images into the page and they look like images.
When I save and edit again the newly pasted emoji images are converted from
images to text like '🐒'.
Why can you see the imoji images in page view, but not in page editor?
Could there be a toggle in the editor to convert or toggle or turn on/off emoji
image vs text in the editor?
If they can be viewed on the page, why not in the PmWiki editor? -- gnuzoo
Is UTF-8 is enabled? Petko
Could changing from emoji image to emoij text be done with $ROEPatterns or $ROSPatterns ?
AFAIK UTF8 is enabled. Do we have to even use Migr8 ? -- gnuzoo
No, and probably no. What you describe happens if the characters you type are not in the encoding of the page. Since the UTF-8 encoding has all possible characters including Emojis, the wiki page is likely using a different encoding. You probably don't need to use MigrateUTF8 -- that page explains in what cases you may need it. If you want me to check, show me the wiki with the problem, ideally with edit access to a wiki sandbox. Petko
Could I/we make a recipe that would run after opening the editor that would do this page:-------------------------------------------------------------- Below is a Greasemonkey script that changes the emoji text entities to emojis.
You may have to change the @match for your URLs.
This does work!
--------------------------------------------------------------
// ==UserScript==
// @name PmWiki Edit Area Emoji Decoder (Final Comprehensive)
// @namespace https://www.greasespot.net/
// @version 2.0
// @description Decodes ALL numeric entities (ensuring 100% emoji coverage), while preserving critical HTML-formatting NAMED entities like & and <. NOTE: Non-emoji numeric symbols (like ©) will also be converted.
// @author GNUZoo
// @match http://localhost/z/*action=edit*
// @grant none
// @run-at document-end
// ==/UserScript==
(function() {
'use strict';
// 1. Create a temporary element for decoding (off-screen)
const decoder = document.createElement('div');
// Mappings for critical NAMED entities we want to protect from decoding.
// The browser must NOT convert these to prevent breaking PmWiki markup.
const PROTECTED_ENTITIES = {
// Protect & -> &
'&': '---AMPSAFE---',
// Protect < -> <
'<': '---LTSAFE---',
// Protect > -> >
'>': '---GTSAFE---',
};
// Reverse mapping for restoration
const RESTORE_ENTITIES = Object.keys(PROTECTED_ENTITIES).reduce((acc, key) => {
acc[PROTECTED_ENTITIES[key]] = key;
return acc;
}, {});
/**
* Decodes all HTML entities (numeric and named) using the browser's parser.
* @param {string} html The string containing HTML entities.
* @returns {string} The decoded string.
*/
function decodeHtmlEntities(html) {
if (!html || typeof html !== 'string') {
return '';
}
// Set the HTML and let the browser decode all entities
decoder.innerHTML = html;
// Extract the decoded text. This includes all emojis, flags, and symbols.
return decoder.textContent;
}
/**
* Main function to find the text area and perform the decoding.
*/
function decodePmWikiEditArea() {
const textarea = document.getElementById('text');
if (textarea && textarea.value) {
const originalText = textarea.value;
// Optimization check
if (originalText.includes('&')) {
let workingText = originalText;
// 1. PROTECTION: Temporarily replace critical NAMED entities.
for (const entity in PROTECTED_ENTITIES) {
workingText = workingText.replaceAll(entity, PROTECTED_ENTITIES[entity]);
}
// 2. FULL DECODE: Decode all numeric entities (emojis, flags, symbols)
// and any unprotected named entities (e.g., -> non-breaking space)
const decodedText = decodeHtmlEntities(workingText);
// 3. RESTORATION: Restore the protected critical NAMED entities.
let finalDecodedText = decodedText;
for (const placeholder in RESTORE_ENTITIES) {
// Use a global regex replacement to handle any browser interpretation issues
// But typically replaceAll works fine here.
finalDecodedText = finalDecodedText.replaceAll(placeholder, RESTORE_ENTITIES[placeholder]);
}
// Update the textarea value only if the content was actually changed
if (originalText !== finalDecodedText) {
textarea.value = finalDecodedText;
console.log("PmWiki Emoji Decoder (Comprehensive): ALL emojis converted. Critical HTML entities preserved. Non-emoji numeric symbols converted.");
}
}
}
}
// Run the decoding function shortly after the page has finished loading
setTimeout(decodePmWikiEditArea, 100);
})();
--------------------------------------------------------------
Could something like this be inserted into the PmWiki code or a recipe instead of using Greasemonkey? -- gnuzoo
A standard PmWiki installation with UTF-8 properly enabled, like this wiki, does not require any additional code to display emojis entered in the edit textarea. If emojis are not working on your site, it usually means that UTF-8 is not fully enabled, or that something else in the configuration or skin is interfering.
To help track this down, it may be useful to follow the steps in the first section of Troubleshooting, which are designed to isolate exactly this kind of issue.
The ChatGPT-generated code you pasted above will not be added to the PmWiki core. That said, of course, it is entirely up to you how you choose to run your wiki. Nothing in the core prevents you from adding a local recipe or customization if that works better for your setup, so if you decide to go that route, that is perfectly fine. Petko
If the page charset is not UTF-8, the browser will entity-encode characters it cannot represent in the declared encoding when submitting a form. Emoji are outside ISO-8859-1 and similar legacy charsets, so the browser substitutes a numeric character reference like 🐒 before the data ever reaches PmWiki.
This means UTF-8 is not actually enabled from the browser's point of view, regardless of what the user believes.
Why this happens
- Browsers do not send raw characters that are not representable in the document's declared charset.
- If the edit page is served as text/html; charset=ISO-8859-1 (or no charset, defaulting to Latin-1), emoji cannot be encoded.
- The browser converts the emoji to a numeric character reference during form submission.
- PmWiki then saves exactly what it receives.
Things to check in PmWiki
1. HTTP headers
Check the edit page headers in the browser developer tools and confirm:
Content-Type: text/html; charset=UTF-8
If it says ISO-8859-1 or the charset is missing, the browser will entity-encode characters.
2. PmWiki configuration must be complete and early
In config.php, UTF-8 must be enabled before output starts:
include_once("scripts/xlpage-utf-8.php");
3. Meta charset tag
The edit page HTML should contain:
<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />
If the skin overrides the header and omits or changes this, the browser will not use UTF-8 even if scripts/xlpage-utf-8.php is included.
4. Old skins or custom headers
Some older skins hardcode:
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
This alone is sufficient to cause the conversion.
The code was from Gemini -- gnuzoo
I do not know how I fixed it. I have been trying things for days.I changed the include to a require because it gives a better error message when an include fails.
I hate when programmers do that. Everything should be error checked IMHO.
In my farmconfig.php:
require_once($FarmD.'/scripts/xlpage-utf-8.php');
$HTTPHeaders['utf-8'] = 'Content-type: text/html; charset=UTF-8';
$DefaultPageCharset = array(''=>'ISO-8859-1');
It is not really clear in the documentation how to "enable UTF8", with
comments and responses all over the page. I would recommend a complete
rewrite of the "How to enable UTF8" page(s). -- gnuzoo