issue with umlauts

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

issue with umlauts

Marten Greiser

Hello all,

possible someone can help me.

I try to update our SLES 11.3 to SLES 12.2. I finished the update no everything is working.

I have one Problem:

Now, “old” comments on our website look incorrect. There are Problems with umlauts like “ä,ü,ö….”

If I write a new comment, everything Is okay.

I Use:

PHP 5.5.14

mariadb-10.0.30-28.1.x86_64

Server version: Apache/2.4.23 (Linux/SUSE)

Vufind  1.4

-- 
Mit freundlichen Grüßen,
Marten Greiser

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: issue with umlauts

Demian Katz

Marten,

 

There is an issue with the VuFind 1.x line that was fixed in release 2.0 – the MySQL database was configured to store data in Latin1, but VuFind was sending data in UTF-8. This resulted in a system that worked correctly from the user’s perspective, but which was storing data in an inappropriate way. Basically, everything was fine as long as you looked at it through the VuFind interface, but if you examined data directly in the database using a tool like PHPMyAdmin, you would discover that special characters were misrepresented – you would see UTF-8 byte sequences instead of appropriate Latin1 characters.

 

The upgrade script to 2.0 corrects this problem by converting all TEXT fields to BLOB fields, then changing the encoding settings in the database to the correct utf8 (instead of latin1) settings, and then changing the BLOB fields back to TEXT fields. It’s a bit of a hack, but it works. Some code also had to be added to VuFind 2.0 to explicitly connect to MySQL with the correct utf8 encoding instead of the default of latin1.

 

I’m wondering if what has happened here is that MariaDB has changed some default behavior, and it is now using utf8 instead of latin1 as the default. This is certainly more appropriate for a new database, but it is causing problems because it is inconsistent with what was happening in the past with early versions of VuFind.

 

I’m assuming that you’re working on a test server and can start again using a clean dump of your production database. If that is the case, I have two possible suggestions that might solve this problem. For each, you should start by resetting your database to a clean dump.

 

1.)    Try installing a copy of VuFind 2.0 on your test server, and run through the upgrade process from VuFind 1.x. Obviously I understand that you don’t want to go through the work of a real upgrade at this time, but by running the upgrade, VuFind 2.0 will correct the database structure automatically to use UTF-8 encoding. It shouldn’t make any other changes that are incompatible with 1.x. It’s possible that after you have run the upgrade, you can delete VuFind 2.0, and you’ll find that the 1.x encoding problem has been solved.

2.)    Investigate the code that connects to the database in VuFind 1.x. Perhaps you can explicitly adjust it to use latin1 encoding. This is technically incorrect and inappropriate as discussed above, but it’s what VuFind used to do in the past, and so at least it will keep things consistent with the past.

 

I hope this is helpful – please let me know if these suggestions do not help or if you have any other questions. In any case, I’m sorry that this very old bug is still causing people hassles!

 

- Demian

 

From: Marten Greiser [mailto:[hidden email]]
Sent: Wednesday, July 26, 2017 4:33 AM
To: [hidden email]; [hidden email]
Subject: [VuFind-Tech] issue with umlauts

 

Hello all,

possible someone can help me.

I try to update our SLES 11.3 to SLES 12.2. I finished the update no everything is working.

I have one Problem:

Now, “old” comments on our website look incorrect. There are Problems with umlauts like “ä,ü,ö….”

If I write a new comment, everything Is okay.

I Use:

PHP 5.5.14

mariadb-10.0.30-28.1.x86_64

Server version: Apache/2.4.23 (Linux/SUSE)

Vufind  1.4

-- 
Mit freundlichen Grüßen,
Marten Greiser

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Vufind-tech mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/vufind-tech
Loading...