20.04 – Unicode characters do not have correct width in the terminal

This doesn’t happen with all unicode characters, but does happen with scissors: ✂️
It seems to be a problem independent of font as I’ve tried multiple different monospace fonts, patched and unpatched with “Nerd fonts” (I did try system default monospace)

neovim:
scissors emoji in neovim scissors overlap the next character in neovim

vscode:
scissors emoji rendered in vscode uses a different character / emoji set, but it looks like vscode renders differently, as there is no overlap

gedit:
scissors emoji is rendered correctly renders as I’d expect: no overlap

terminal:
scissors emoji in the terminal
has the same problem as in neovim

github: https://github.com/NullVoxPopuli/dotfiles/blob/75ad3987e4eaa550e7dfb70a37ffe2b756594bd9/home/scripts/unme

github rendering of the scissors emoji
GitHub in Firefox looks correct

Is there a way to configure the terminal to have consistent emoji/unicode width/rendering?

parse – Gutenberg Block manipulation: Undo parse_blocks() with serialize_blocks() results in unicode issues

I would like to manipulate Gutenberg Blocks with PHP.
Therefore I need to parse post_content into an array and the same way back after manipulation. But I don’t get that far, because already the parsing back and forth doesn’t work.

function change_post_data_before_save( $data, $postarr ) {
    $post_data = $data('post_content'); //post_content seems to be json encoded 
    //stripping slashes otherwise $blocks(0)('attrs') would be NULL
    $post_data = str_replace('\"', '"', $post_data);
    //trying to replace double slash in unicodes but it seems not to work
    $post_data = str_replace('\u', 'u', $post_data);
    $blocks = parse_blocks($post_data); //there are still some issues with schema and other array parts

    //manipulation of $blocks if needed

    $post_content = serialize_blocks( $blocks ); //seems to work correct
    $data('post_content') = $post_content;

    return $data;
}
add_filter( 'wp_insert_post_data', 'change_post_data_before_save', 10, 2 );

Any ideas how to json decode $data(‘post_data’) without destroying parts in $blocks(0)(‘attrs’) which need to be json encoded? I tried replacement but it seems not to work for unicodes and serialize_blocks() destroyes it in the end.

Example Code:

//expected result - saved in database without wp_insert_post_data (see above)
<!-- wp:uagb/faq {"block_id":"f1ad85bc","enableSchemaSupport":true,"schema":"{u0022@contextu0022:u0022https://schema.orgu0022,u0022@typeu0022:u0022FAQPageu0022,u0022@idu0022:u0022http://localhost/specials/gutenberg/u0022,u0022mainEntityu0022:({u0022@typeu0022:u0022Questionu0022,u0022nameu0022:u0022Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)u0022,u0022acceptedAnsweru0022:{u0022@typeu0022:u0022Answeru0022,u0022textu0022:u0022This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)u0022}})}"} -->
<div class="wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height" data-faqtoggle="true" role="tablist"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}</script><div class="uagb-faq__wrap uagb-buttons-layout-wrap"><!-- wp:uagb/faq-child {"block_id":"61cac9b0","question":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","answer":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"} -->
<div class="wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0"><div class="uagb-faq-child__wrapper"><div class="uagb-faq-item" role="tab" tabindex="0"><div class="uagb-faq-questions-button uagb-faq-questions"><span class="uagb-icon uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-icon-active uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-question">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class="uagb-faq-content"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
<!-- /wp:uagb/faq-child --></div></div>
<!-- /wp:uagb/faq -->

<!-- wp:kadence/iconlist {"items":({"icon":"fe_alertCircle","link":"","target":"_self","size":20,"width":2,"text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","color":"","background":"","border":"","borderRadius":0,"padding":5,"borderWidth":1,"style":"default"}),"uniqueID":"_a79c18-e7"} -->
<div class="wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone"><ul class="kt-svg-icon-list"><li class="kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0"><div style="display:inline-flex;justify-content:center;align-items:center" class="kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle"><svg style="display:inline-block;vertical-align:middle" viewbox="0 0 24 24" height="20" width="20" fill="none" stroke="currentColor" xmlns="http://www.w3.org/2000/svg" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"></circle><line x1="12" y1="8" x2="12" y2="12"></line><line x1="12" y1="16" x2="12" y2="16"></line></svg></div><span class="kt-svg-icon-list-text">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
<!-- /wp:kadence/iconlist -->

Debugging:

//result of $data('post_content')
<!-- wp:uagb/faq {\"block_id\":\"f1ad85bc\",\"enableSchemaSupport\":true,\"schema\":\"{\\u0022@context\\u0022:\\u0022https://schema.org\\u0022,\\u0022@type\\u0022:\\u0022FAQPage\\u0022,\\u0022@id\\u0022:\\u0022http://localhost/specials/gutenberg/\\u0022,\\u0022mainEntity\\u0022:({\\u0022@type\\u0022:\\u0022Question\\u0022,\\u0022name\\u0022:\\u0022Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\\u0022,\\u0022acceptedAnswer\\u0022:{\\u0022@type\\u0022:\\u0022Answer\\u0022,\\u0022text\\u0022:\\u0022This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\\u0022}})}\"} -->
<div class=\"wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height\" data-faqtoggle=\"true\" role=\"tablist\"><script type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"FAQPage\",\"@id\":\"http://localhost/specials/gutenberg/\",\"mainEntity\":({\"@type\":\"Question\",\"name\":\"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\"}})}</script><div class=\"uagb-faq__wrap uagb-buttons-layout-wrap\"><!-- wp:uagb/faq-child {\"block_id\":\"61cac9b0\",\"question\":\"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\",\"answer\":\"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\"} -->
<div class=\"wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0\"><div class=\"uagb-faq-child__wrapper\"><div class=\"uagb-faq-item\" role=\"tab\" tabindex=\"0\"><div class=\"uagb-faq-questions-button uagb-faq-questions\"><span class=\"uagb-icon uagb-faq-icon-wrap\"><svg xmlns=\"http://www.w3.org/2000/svg\" viewbox=\"0 0 448 512\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"></path></svg></span><span class=\"uagb-icon-active uagb-faq-icon-wrap\"><svg xmlns=\"http://www.w3.org/2000/svg\" viewbox=\"0 0 448 512\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"></path></svg></span><span class=\"uagb-question\">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class=\"uagb-faq-content\"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
<!-- /wp:uagb/faq-child --></div></div>
<!-- /wp:uagb/faq -->

<!-- wp:kadence/iconlist {\"items\":({\"icon\":\"fe_alertCircle\",\"link\":\"\",\"target\":\"_self\",\"size\":20,\"width\":2,\"text\":\"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\",\"color\":\"\",\"background\":\"\",\"border\":\"\",\"borderRadius\":0,\"padding\":5,\"borderWidth\":1,\"style\":\"default\"}),\"uniqueID\":\"_a79c18-e7\"} -->
<div class=\"wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone\"><ul class=\"kt-svg-icon-list\"><li class=\"kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0\"><div style=\"display:inline-flex;justify-content:center;align-items:center\" class=\"kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle\"><svg style=\"display:inline-block;vertical-align:middle\" viewbox=\"0 0 24 24\" height=\"20\" width=\"20\" fill=\"none\" stroke=\"currentColor\" xmlns=\"http://www.w3.org/2000/svg\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\"><circle cx=\"12\" cy=\"12\" r=\"10\"></circle><line x1=\"12\" y1=\"8\" x2=\"12\" y2=\"12\"></line><line x1=\"12\" y1=\"16\" x2=\"12\" y2=\"16\"></line></svg></div><span class=\"kt-svg-icon-list-text\">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
<!-- /wp:kadence/iconlist -->
//result of $post_data
<!-- wp:uagb/faq {"block_id":"f1ad85bc","enableSchemaSupport":true,"schema":"{\u0022@context\u0022:\u0022https://schema.org\u0022,\u0022@type\u0022:\u0022FAQPage\u0022,\u0022@id\u0022:\u0022http://localhost/specials/gutenberg/\u0022,\u0022mainEntity\u0022:({\u0022@type\u0022:\u0022Question\u0022,\u0022name\u0022:\u0022Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\u0022,\u0022acceptedAnswer\u0022:{\u0022@type\u0022:\u0022Answer\u0022,\u0022text\u0022:\u0022This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)\u0022}})}"} -->
<div class="wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height" data-faqtoggle="true" role="tablist"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}</script><div class="uagb-faq__wrap uagb-buttons-layout-wrap"><!-- wp:uagb/faq-child {"block_id":"61cac9b0","question":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","answer":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"} -->
<div class="wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0"><div class="uagb-faq-child__wrapper"><div class="uagb-faq-item" role="tab" tabindex="0"><div class="uagb-faq-questions-button uagb-faq-questions"><span class="uagb-icon uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-icon-active uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-question">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class="uagb-faq-content"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
<!-- /wp:uagb/faq-child --></div></div>
<!-- /wp:uagb/faq -->

<!-- wp:kadence/iconlist {"items":({"icon":"fe_alertCircle","link":"","target":"_self","size":20,"width":2,"text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","color":"","background":"","border":"","borderRadius":0,"padding":5,"borderWidth":1,"style":"default"}),"uniqueID":"_a79c18-e7"} -->
<div class="wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone"><ul class="kt-svg-icon-list"><li class="kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0"><div style="display:inline-flex;justify-content:center;align-items:center" class="kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle"><svg style="display:inline-block;vertical-align:middle" viewbox="0 0 24 24" height="20" width="20" fill="none" stroke="currentColor" xmlns="http://www.w3.org/2000/svg" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"></circle><line x1="12" y1="8" x2="12" y2="12"></line><line x1="12" y1="16" x2="12" y2="16"></line></svg></div><span class="kt-svg-icon-list-text">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
<!-- /wp:kadence/iconlist -->
//result of $blocks
array (
  0 => 
  array (
    'blockName' => 'uagb/faq',
    'attrs' => 
    array (
      'block_id' => 'f1ad85bc',
      'enableSchemaSupport' => true,
      'schema' => '{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}',
    ),
    'innerBlocks' => 
    array (
      0 => 
      array (
        'blockName' => 'uagb/faq-child',
        'attrs' => 
        array (
          'block_id' => '61cac9b0',
          'question' => 'Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)',
          'answer' => 'This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)',
        ),
        'innerBlocks' => 
        array (
        ),
        'innerHTML' => '
<div class="wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0"><div class="uagb-faq-child__wrapper"><div class="uagb-faq-item" role="tab" tabindex="0"><div class="uagb-faq-questions-button uagb-faq-questions"><span class="uagb-icon uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-icon-active uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-question">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class="uagb-faq-content"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
',
        'innerContent' => 
        array (
          0 => '
<div class="wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0"><div class="uagb-faq-child__wrapper"><div class="uagb-faq-item" role="tab" tabindex="0"><div class="uagb-faq-questions-button uagb-faq-questions"><span class="uagb-icon uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-icon-active uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-question">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class="uagb-faq-content"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
',
        ),
      ),
    ),
    'innerHTML' => '
<div class="wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height" data-faqtoggle="true" role="tablist"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}</script><div class="uagb-faq__wrap uagb-buttons-layout-wrap"></div></div>
',
    'innerContent' => 
    array (
      0 => '
<div class="wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height" data-faqtoggle="true" role="tablist"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}</script><div class="uagb-faq__wrap uagb-buttons-layout-wrap">',
      1 => NULL,
      2 => '</div></div>
',
    ),
  ),
  1 => 
  array (
    'blockName' => NULL,
    'attrs' => 
    array (
    ),
    'innerBlocks' => 
    array (
    ),
    'innerHTML' => '

',
    'innerContent' => 
    array (
      0 => '

',
    ),
  ),
  2 => 
  array (
    'blockName' => 'kadence/iconlist',
    'attrs' => 
    array (
      'items' => 
      array (
        0 => 
        array (
          'icon' => 'fe_alertCircle',
          'link' => '',
          'target' => '_self',
          'size' => 20,
          'width' => 2,
          'text' => 'This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)',
          'color' => '',
          'background' => '',
          'border' => '',
          'borderRadius' => 0,
          'padding' => 5,
          'borderWidth' => 1,
          'style' => 'default',
        ),
      ),
      'uniqueID' => '_a79c18-e7',
    ),
    'innerBlocks' => 
    array (
    ),
    'innerHTML' => '
<div class="wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone"><ul class="kt-svg-icon-list"><li class="kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0"><div style="display:inline-flex;justify-content:center;align-items:center" class="kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle"><svg style="display:inline-block;vertical-align:middle" viewbox="0 0 24 24" height="20" width="20" fill="none" stroke="currentColor" xmlns="http://www.w3.org/2000/svg" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"></circle><line x1="12" y1="8" x2="12" y2="12"></line><line x1="12" y1="16" x2="12" y2="16"></line></svg></div><span class="kt-svg-icon-list-text">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
',
    'innerContent' => 
    array (
      0 => '
<div class="wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone"><ul class="kt-svg-icon-list"><li class="kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0"><div style="display:inline-flex;justify-content:center;align-items:center" class="kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle"><svg style="display:inline-block;vertical-align:middle" viewbox="0 0 24 24" height="20" width="20" fill="none" stroke="currentColor" xmlns="http://www.w3.org/2000/svg" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"></circle><line x1="12" y1="8" x2="12" y2="12"></line><line x1="12" y1="16" x2="12" y2="16"></line></svg></div><span class="kt-svg-icon-list-text">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
',
    ),
  ),
)

Final result after serialize_blocks():

//result of $post_content (can not be shown in Gutenberg UI "This block contains unexpected or invalid content.")
<!-- wp:uagb/faq {"block_id":"f1ad85bc","enableSchemaSupport":true,"schema":"{\u0022@context\u0022:\u0022https:\/\/schema.org\u0022,\u0022@type\u0022:\u0022FAQPage\u0022,\u0022@id\u0022:\u0022http:\/\/localhost\/specials\/gutenberg\/\u0022,\u0022mainEntity\u0022:({\u0022@type\u0022:\u0022Question\u0022,\u0022name\u0022:\u0022Headline with special characters: (\u00b0C, \u00e4, \u00f6, \u00fc, \u00df, \u00d6, \u00c4, \u00dc)\u0022,\u0022acceptedAnswer\u0022:{\u0022@type\u0022:\u0022Answer\u0022,\u0022text\u0022:\u0022This is a test with special characters: (\u00b0C, \u00e4, \u00f6, \u00fc, \u00df, \u00d6, \u00c4, \u00dc)\u0022}})}"} -->
<div class="wp-block-uagb-faq uagb-faq__outer-wrap uagb-block-f1ad85bc uagb-faq-icon-row uagb-faq-layout-accordion uagb-faq-expand-first-false uagb-faq-inactive-other-true uagb-faq-equal-height" data-faqtoggle="true" role="tablist"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","@id":"http://localhost/specials/gutenberg/","mainEntity":({"@type":"Question","name":"Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)","acceptedAnswer":{"@type":"Answer","text":"This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)"}})}</script><div class="uagb-faq__wrap uagb-buttons-layout-wrap"><!-- wp:uagb/faq-child {"block_id":"61cac9b0","question":"Headline with special characters: (\u00b0C, \u00e4, \u00f6, \u00fc, \u00df, \u00d6, \u00c4, \u00dc)","answer":"This is a test with special characters: (\u00b0C, \u00e4, \u00f6, \u00fc, \u00df, \u00d6, \u00c4, \u00dc)"} -->
<div class="wp-block-uagb-faq-child uagb-faq-child__outer-wrap uagb-block-61cac9b0"><div class="uagb-faq-child__wrapper"><div class="uagb-faq-item" role="tab" tabindex="0"><div class="uagb-faq-questions-button uagb-faq-questions"><span class="uagb-icon uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-icon-active uagb-faq-icon-wrap"><svg xmlns="http://www.w3.org/2000/svg" viewbox="0 0 448 512"><path d="M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z"></path></svg></span><span class="uagb-question">Headline with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></div><div class="uagb-faq-content"><span><p>This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</p></span></div></div></div></div>
<!-- /wp:uagb/faq-child --></div></div>
<!-- /wp:uagb/faq -->

<!-- wp:kadence/iconlist {"items":({"icon":"fe_alertCircle","link":"","target":"_self","size":20,"width":2,"text":"This is a test with special characters: (\u00b0C, \u00e4, \u00f6, \u00fc, \u00df, \u00d6, \u00c4, \u00dc)","color":"","background":"","border":"","borderRadius":0,"padding":5,"borderWidth":1,"style":"default"}),"uniqueID":"_a79c18-e7"} -->
<div class="wp-block-kadence-iconlist kt-svg-icon-list-items kt-svg-icon-list-items_a79c18-e7 kt-svg-icon-list-columns-1 alignnone"><ul class="kt-svg-icon-list"><li class="kt-svg-icon-list-style-default kt-svg-icon-list-item-wrap kt-svg-icon-list-item-0"><div style="display:inline-flex;justify-content:center;align-items:center" class="kt-svg-icon-list-single kt-svg-icon-list-single-fe_alertCircle"><svg style="display:inline-block;vertical-align:middle" viewbox="0 0 24 24" height="20" width="20" fill="none" stroke="currentColor" xmlns="http://www.w3.org/2000/svg" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"></circle><line x1="12" y1="8" x2="12" y2="12"></line><line x1="12" y1="16" x2="12" y2="16"></line></svg></div><span class="kt-svg-icon-list-text">This is a test with special characters: (°C, ä, ö, ü, ß, Ö, Ä, Ü)</span></li></ul></div>
<!-- /wp:kadence/iconlist -->

Related Topic: serialize_blocks breaking html tags in content

mobile – Samsung Android phone rendering chess unicode as Emoji?

I’m developing a [javascript implementation][1] of Singularity Chess and I’m having a really weird issue with rendering on Samsung phones. Samsung phones render unicode chess characters as what appear to be a custom Emoji representation, as opposed to the plain text version. This would not be a problem, except for the fact that the Emoji set has only black pieces.

Is there a nice fix for this, or do I have to use the User Agent string to dynamically switch to a different display method such as SVG?

PostgreSQL upper() and lower() functions don’t understand UNICODE?

I just discovered something very strange! My PostgreSQL database is set up to use UTF-8 internally and locale C and all very straight forward. Yet look at this:

select upper('Lösungsmittel') = upper('LÖSUNGSMITTEL');

 ?column?
----------
 f
(1 row)

select upper('Lösungsmittel') union all
select upper('LÖSUNGSMITTEL');

     upper
---------------
 LöSUNGSMITTEL
 LÖSUNGSMITTEL
(2 rows)

isn’t that strange?! The upper() function was unable to realize upper(‘ö’) = ‘Ö’!

The Java Character class can do that without a problem. I am surprised to see that, and we are on PostgreSQL server 13.1, not something super old.

Is there some configuration parameter to configure this character set stuff beyond selection of UTF-8?

Cannot open text files in Python 3: Unicode error

I am having a problem open a simple text file in Python 3.8. I setup a simple test.

Here is my test code:

import os

file_path = "c:UsersusernameDocumentsfolder1some_file.txt"

with open(file_path, 'r') as f:
    for line in f:
        print(line)

I get the following error: Unicode Error “unicodeescape” codec can’t decode bytes in position 2-3.

I have read other posts about putting an ‘r’ in front of the file path and when I do I get an “No such file or directory: ‘c:UsersusernameDocumentsfolder1some_file.txt’

import os

file_path = r"c:UsersusernameDocumentsfolder1some_file.txt"

with open(file_path, 'r') as f:
    for line in f:
        print(line)

I have also tried using double backslash in the path c:\Users\username\Documents\folder1\some_file.txt and that did not work either.

big sur – Apple’s unicode keyboard layout versions location

Based on the “ISO” standard, say you we’re to buy a keyboard and then plug it in. Note, I have never attempted this, but am assured it will work after changing your keyboard’s input source.

Is there a location on the Mac OSX Big Sur operating system where these different key mappings are stored? Where it would be easier to program based on these files? Where are these mapping located?

Keyboard
Goal: (Input) American

Translated: (Other) Language

Or would it be best to programmatically change this.

encoding scheme – Coding efficiency of a code representing 50,000 characters, using unicode

I need to find the coding efficiency of a code representing 50,000 characters using unicode.
Here is what I did,

the number of characters is 50,000

and 2 raised to the power of 15 is 32,768

so bits used is 32,768 and bits needed is 50,000

so 32,768/50,000 multiplied by 100 is 65.536%.

I am completely new to this topic, sorry if I am wrong. Thank you.