IIS Hacks | Server and System Administration » PHP

Converting MySQL tables from Latin1 to UTF8

Chris Stinson — Fri, 20 Nov 2009 17:33:13 +0000

Many content management systems and forums were originally created using the latin1 character set and the latin1_swedish_ci collation in MySQL. The problem many of these systems are facing today is the growing demand for multi-language content using special characters that cannot be accurately represented in the latin1 character set. This is where utf8 comes in.

The problem with simply converting a database from latin1 to utf8 is in the data itself. When you convert a database or table to a different character set or collation, it does not convert the content held within the tables. What happens is users end up with ‘strange’ characters in their data.

You can prevent this by taking steps to protect the data before you convert the database or tables.

An overview of the steps:

Backup the database. No, really, do it.
Alter the string field types to their binary equivalents in all tables. This will protect the data during conversion.
• VARCHAR to VARBINARY
• CHAR to BINARY
• LONGTEXT to LONGBLOB
• MEDIUMTEXT to MEDIUMBLOB
• TEXT to BLOB
• TINYTEXT to TINYBLOB
Convert the database and tables from latin1 to utf8 character set and latin1_swedish_ci to utf8_general_ci collation.
Convert the binary field types back to their original string field types.
Backup the resulting database.
Finished

Ex. Generic convert column to BLOB from TEXT

ALTER TABLE tbl_name MODIFY column_name BLOB

Ex. Generic convert table to new character set

ALTER TABLE tbl_name CHARACTER SET charset_name COLLATE collation_name

Ex. Generic convert database to new character set utf8

ALTER DATABASE database_name CHARACTER SET utf8;

Wordpress Dynamic Replacement

Chris Stinson — Sat, 13 Sep 2008 03:28:09 +0000

Wordpress has long had a feature that dynamically replaces standard characters with their more visually appealing (and perhaps more accurate) symbols. In some cases the formatting may not be desirable.

Last year I wrote a few articles on Exchange Server errors. The errors typically follow the form of 0xNumber, ex. 0x8000000F. What wordpress was doing was replacing the “x” in the error with a “times” or multiplication symbol. So 0x0 would show up as 0×0.

Dynamic Replace Active

Dynamic Replace Inactive

My concern was that search engines DO distinguish between “x” and the symbol for multiplication. So when people were searching for 0x8004011D my posts did not show up, but when searching for 0×8004011D or 8004011D they did. If you do a google search with the “times” symbol instead of an “x” for exchange errors, you will find many posts otherwise hidden from the world.

The last string of the regular expression below represents a case for when there is a character followed by an x and by another character without spaces. The x will be replaced with the special character for “times” or &# 215 ;

The characters in the image below are the symbols for replacement. Generally the quotes are not misleading to search engines.

The following dynamic string variable is found in the \wp-includes\formatting.php file. If you wish to remove all dynamic characters, simply delete both lines and your wordpress posts will render correctly.

$dynamic_characters = array
('/\'(\d\d(?:’|\')?s)/', '/(\s|\A|")\'/',
'/(\d+)"/', '/(\d+)\'/', '/(\S)\'([^\'\s])/', '/(\s|\A)"(?!\s)/',
'/"(\s|\S|\Z)/', '/\'([\s.]|\Z)/', '/(\d+)x(\d+)/');

 $dynamic_replacements = array
('’$1','$1‘', '$1″',
'$1′', '$1’$2', '$1“$2',
'”$1', '’$1', '$1×$2');

If you only want to remove the “x” replacement symbol ( × ) use the following lines in formatting.php in place of the originals.

$dynamic_characters = array(’/\’(\d\d(?:’|\’)?s)/’, ‘/(\s|\A|”)\’/',’/(\d+)”/’, ‘/(\d+)\’/', ‘/(\S)\’([^\'\s])/’, ‘/(\s|\A)”(?!\s)/’,'/”(\s|\S|\Z)/’, ‘/\’([\s.]|\Z)/’);

$dynamic_replacements = array(’’$1′,’$1‘’, ‘$1″’, ‘$1′’, ‘$1’$2′, ‘$1“$2′, ‘”$1′, ‘’$1′);

How to install PHP ISAPI on Windows 2008 IIS7 x64

Chris Stinson — Thu, 03 Jul 2008 20:43:27 +0000

With the release of Windows Server 2008 and IIS 7, Microsoft has included PHP5 FASTCGI support. ISAPI is still faster in my opinion, and if used correctly, very stable. PHP uses a 32-bit DLL so it will not work with an x64 system. There are several ports of PHP to x64, but all have proved to be unstable. Below I will outline the steps to install PHP 32-bit on Windows 2008 x64 (and have it stable).

Install the PHP4 or PHP5 package (32-bit) in C:\PHP or wherever you like. Only use the Windows installer from php.net if you do not need any extensions. I would recommend downloading the PHP zip package.
Update April 2010: The VC6 x86 Thread Safe package is ideal for ISAPI. If you are going to use FastCGI, download the VC6 x86 Non Thread Safe package.
Open the Internet Information Services (IIS) Manager.
Double-click “Handler Mappings” from the main IIS screen.
Click on “Add Script Map.”
Set up the handler mapping for c:\PHP\php5isapi.dll with extension *.php and check to allow the ISAPI extension and execution of scripts.
Double-click “ISAPI & CGI Restrictions” on the main IIS screen. Right-click on PHP and select “Edit Feature Settings” and check “Allow unspecified ISAPI modules.”
Right-click on the Default Application Pool (or the one you want to use if more than one) and select “Advanced Settings.”
Change the “Enable 32-bit Applications” to True. Click OK. This spawns the App Pool in 32-bit mode, so if you have other modules that need to be run in 64-bit mode, best to separate the website into two App Pools: one 32-bit and one 64-bit.
Restart the server.

Wordpress & PHP on Windows Server IIS 6

Chris Stinson — Tue, 10 Jul 2007 22:38:36 +0000

This is a quick tip to those who are having trouble installing or using Wordpress on Windows Server with PHP.

Make sure in the PHP.ini file that the following is set:

; Magic quotes for runtime-generated data, e.g. data from SQL, from exec(), etc.
magic_quotes_runtime = Off

If this is ON, Wordpress will not allow you to login. It will spit out the following error:

“You do not have sufficient permissions to access this page.”

You CAN have magic_quotes ON though, which is a different setting. The above setting is for data being pulled from a MySQL database for example.

Also make sure the following is set:

; cgi.fix_pathinfo provides *real* PATH_INFO/PATH_TRANSLATED support for CGI. PHP’s
; previous behaviour was to set PATH_TRANSLATED to SCRIPT_FILENAME, and to not grok
; what PATH_INFO is. For more information on PATH_INFO, see the cgi specs. Setting
; this to 1 will cause PHP CGI to fix it’s paths to conform to the spec. A setting
; of zero causes PHP to behave as before. Default is zero. You should fix your scripts
; to use SCRIPT_FILENAME rather than PATH_TRANSLATED.
cgi.fix_pathinfo=1

PHP 5.2.x ISAPI stability on Windows IIS 6.0

Chris Stinson — Sat, 07 Jul 2007 05:26:31 +0000

Having dealt with multiple IIS servers with all kinds of PHP configurations, I’ll let you in on a little secret. PHP 4.4.4 or 4.4.x ISAPI is by far the most stable in any situation. PHP 5.2.x has terrible memory management. If you have a piece of bad code that eats up memory, PHP will not put a stop to it like it should. Here’s an error I received:

The description for Event ID ( 2 ) in Source ( PHP-5.2.2 ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: php[3556], PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 2061628868 bytes) in Unknown on line 0.

And of course no matter how much memory I allocate, it eats it all up. It’s just a matter of time. Then comes this error:

Faulting application w3wp.exe, version 6.0.3790.3959, faulting module php5ts.dll, version 5.2.2.2, fault address 0x#.

The obvious solution is to write better code that doesn’t leak memory. However, this isn’t always possible, especially if you are hosting websites run by other people. Many shared hosting accounts on IIS 6.0 experience total PHP shutdown due to one site’s bad code. I have thoroughly tested every 5.2.x line of PHP and they all suffer from this problem. If you want to or need to use PHP 5.2.x, use CGI or Fast-CGI (there’s a beta from IIS.net) as you will not experience any of these memory issues. However Fast-CGI still isn’t as quick as ISAPI, and in terms of the IIS.net beta, suffers from max uploads of ~1 MB.

Summary for maximum stability/speed combo:

PHP 4.4.x -> use ISAPI

PHP 5.2.x -> CGI (in PHP package) or Fast-CGI (from IIS.net)

Don’t use any other line of PHP since they are no longer supported and have multiple security issues.