Skip to content

Database: Add utf8mb4 support - #12144

Open
lscharmer wants to merge 1 commit into
ILIAS-eLearning:trunkfrom
lscharmer:utf8mb4-migration
Open

lscharmer wants to merge 1 commit into
ILIAS-eLearning:trunkfrom
lscharmer:utf8mb4-migration

Conversation

@lscharmer

@lscharmer lscharmer commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds utf8mb4 support for ILIAS and is based on PR #5943.

PRs #12020 & #12021 must be merged first, before this PR can be merged.

This PR implements the following features:

  • the database is able to work with UTF8MB4 collations.
  • a migration to convert all tables from UTF8MB3 to UTF8MB4.

UTF8MB* support

The database is still able to work with UTF8MB3 tables, so this doesn't require the migration to be run.
New installations will and must still be installed with UTF8MB3. This, as well as the default collation (see below), will be changed when PRs #11718, #12020 & #12021 are integrated into the SQL dump.
This means when this PR is merged installations still must be installed with utf8mb3 and can be (optionally) converted to UTF8MB4 after all update steps are executed.

Collation

Currently we chose the collation utf8mb4_unicode_520_ci as the preferred one (used in the migration).
The migration will check if the collation utf8mb4_unicode_520_ci is available and will fallback to utf8mb4_unicode_ci.

We chose this collation for the following reasons:

  • Older collations like utf8mb4_general_ci and utf8mb4_unicode_ci have some problems like treating certain emojis as the same (e.g. 🍺️ = 🍣️).
  • Newer collations are not supported on MariaDB and at the same time on MySQL as well and we want to avoid different collations for each system to prevent further problems with migrations or incopatibilities between the two.

For completeness: MariaDB & MySQL both support the newer utf8mb4_0900_ai_ci collation but on MariaDB this is an alias to uca1400_ai_ci which is not exactly the same collation (UCA v14.0.0 vs. v9.0.0).

IMO we could also use utf8mb4_0900_ai_ci as ILIAS preferred & default collation, as it is newer & equivalent to the default collation on MariaDB (uca1400_ai_ci) & MySQL (utf8mb4_0900_ai_ci).
But as they are not exactly the same utf8mb4_unicode_520_ci would be the safer option IMO.

Summary: When will what be possible

When this PR is merged:
ILIAS MUST be installed with UTF8MB3 but can then be optionally migrated to UTF8MB4.

After this:
When #11718, #12020 & #12021 are integrated into the SQL dump:
ILIAS can also be installed with UTF8MB4 but can't create UTF8MB4 schemas (the schema must already exist).

After this:
When PR #12145 is merged:
ILIAS can also create UTF8MB4 schemas and new installation will be created with a UTF8MB4 collation by default.

At any point ILIAS will still be able to work with UTF8MB3 database schemas.
And even when UTF8MB4 is the default collation one can configure a UTF8MB3 collation in the config.json to still be able to create new installations with UTF8MB3.

Best regards
@lscharmer

@matthiaskunkel

Copy link
Copy Markdown
Member

Jour Fixe, 05 OCT 2026: We highly appreciate these PRs (#12144 and #12145) and accept them for trunk. We also declare mb3 support for depraceted with ILIAS 12 and will remove it with ILIAS 13.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement jour fixe php Pull requests that update Php code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants