What I Was Trying to Do…

I’m currently adding a clearance item section to a website. This will aggregate items from over 35 stores across Canada and allow a user to search for items in a number of ways. As part of this I’ve created a business object to represent a clearance item. This mainly consists of protected properties with accessors and mutators for setting and getting the properties. Since the site is in English and French the accessors allow the properties to be formatted accordingly before being returned. I use this clearance item class when importing information into the database (items are imported from Excel worksheets, but that’s a topic for another post) and when returning results from searches. I use data mappers to handle the CRUD operations in the database. What I wanted to do was to find a quick and easy way to transform a row from a resultset into an instance of a clearance item object without using a full ORM solution. I thought I had found this with PDO.

The ‘Solution’

PDO allows you to set the fetch mode to fetch an object of a user defined class for each row returned from the database. You can either use PDOStatement::fetchObject() or PDOStatement::setFetchMode() to define this behaviour. It will take the names of the columns and assume that these are properties of the object, setting the appropriate values for these properties. In theory this sounds great but unfortunately it’s not as easy as it sounds.

The Problem

The problem in my case is that all properties of my object are declared as protected and I use mutators to set them. I anticipated this and provided a __set() method to provide the needed property overloading. I thought that what would happen is that PDO would attempt to set an undefined public property, triggering the __set() method where there is logic to call the appropriate mutator. Unfortunately it doesn’t work like that at all. According to comments in the PHP manual it seems that PDO uses something called reflection injection to set the properties inside the object regardless of their visibility. This meant that I was getting back an instance of my clearance item class with the protected properties set directly by PDO. This may not seem like too much of a problem, but my mutators include logic to transform the properties being set to the correct data type. For instance, columns containing integers or floating point numbers come back as strings and the mutators make sure they are set as numeric data types. Worst of all for me was the information on stores. I’m using MySQL’s GROUP_CONCAT() function to ‘implode’ information about stores and the inventory in each into a string, meaning that I can fetch all of the information I need in a single query. The logic in the mutator then splits that string apart into an array of information on stores and the inventory. PDO was bypassing this completely and setting the protected stores property as the string that came back from GROUP_CONCAT().

A Partial Solution

Once again the PHP manual was my partial saviour. There is a way to fetch a row from a resultset into an existing instance of a class. In this case the visibility of properties is honoured and the __set() function is called, in turn triggering the correct mutators. The code I came up with looks like this:

//The statement is prepared before this.

$item = new ClearanceItem($this->lang);
 $st->setFetchMode(PDO::FETCH_INTO, $item);
 $st->bindParam(':ItemId', $id, PDO::PARAM_INT);
 $st->execute();
 foreach ($st as $result) {
//The query in this case only returns a single row.

return $result;
 }

While this works it’s a bit of a hack to say the least. It seems that PDOStatement::fetch() doesn’t work here to fetch the info into the object, necessitating the foreach loop and the immediate return. This is fine in this case but in other areas of the code I need to iterate over a resultset, getting a new instance of ClearanceItem each time. This wouldn’t work as the information is populated into the existing object bound to the statement. The best I could do is to clone the object each time through the loop and store the cloned copy.

There is another way out of this: return each row as an associative array and create a factory method to return instances of ClearanceItem from this. I actually coded this solution already when I intitially found the strangeness that PDO was doing in return instances of ClearanceItem. That solution works perfectly well but it seems strange that it’s not possible to loop over a record set, getting a new instance of a class each time.

Conclusion

It seems that it’s quite possible to return instances of a class from a PDOStatement providing that all of the properties are public or if you don’t mind PDO directly setting your private and protected properties. If you want to use mutators to control how your data is set and to make sure it’s of the correct type it seems that the options are far more limited and you may be out of luck. Of course there are other options, such as writing a factory method to create an object from an array or using a full blown ORM like Doctrine. In this instance, as I haven’t used Doctrine before and as time is of the essence I was hoping that PDO would give me what I need.

It seems very strange that PDO is allowed to directly set private and protected properties when all other code accessing instances of ClearanceItem have to use the accessors and mutators. It seems to me to drive a coach and horses through the concept of property visibility. Why should PDO (at least I’m assuming it’s just PDO) have the ability to bypass the visibility protections that OO code provides while everyone else has to work with it? Is there some good reason for doing this that I’m missing? I’d be really interested to hear what the possible advantages of this are.

Development environment

My development environment is Windows 7 with Apache (compiled using VC9), PHP 5.3.2 and MySQL 5.1.47. For coding I use NuSphere’s PHPED. All of my timings were made using the built in profiler that comes with PHPED and the DBG PHP debugging extension. Why is this information important? Hopefully that will become clear later.

My first attempt

My first attempt used a while loop to iterate 50,000 times to create the values. On each iteration a code was generated and a prepared statement executed to insert the code into the database. The column holding the code has a unique index on it, causing an exception to be thrown if PHP generates a duplicate 8 digit code. This is then caught in the inner catch block, which causes the loop to go through another iteration. If the code is inserted successfully the counter is incremented and the loop carries on. Code for this is below:

<?php
 //Up the execution time to 15 minutes as this takes ages to run
 ini_set('max_execution_time', 900);
 try {
 //Create a PDO connection to the database
 $db = new PDO('mysql:dbname=test;host=localhost', 'root', 'PASSWORD');
 //Set the PDO error mode to exceptions
 $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
 //Delete all entries from the test table
 $db->exec('TRUNCATE test');
 //Prepare the statement
 $st = $db->prepare('INSERT INTO test(code) VALUES(?)');
 //Bind the parameter to the $code variable.
 //When the statement is executed whatever value is in $code will be used as the value for the parameter
 $st->bindParam(1, $code);
 //Set the counter for the main loop
 $i = 0;
 //Possible characters for the code string
 $possible = "0123456789bcdfghjkmnpqrstvwxyz";
 //Length of the possible characters (0 based)
 $len = strlen($possible) - 1;
 //Loop 50,000 times!
 while ($i < 50000) {
 try {
 //Reset the $code variable
 $code = "";
 //Create a counter for the inner loop
 $j = 0;
 //add random characters to $code until it is 8 characters long
 do {
 //Get a random character
 $char = $possible[mt_rand(0, $len)];
 //we don't want this character if it's already in the code
 if (! strpos($code, $char)) {
 $code .= $char;
 $j++;
 }
 } while ($j < 8);
 //Execute the statement. The value of $code will be used as the parameter
 $st->execute();
 $i++;
 }
 catch (PDOException $e) {
 //If an exception is caught here it's probably a duplicate value in the code column. Continue to get a new value.
 continue;
 }
 }
 }
 catch (PDOException $e) {
 //Any other exceptions caught here.
 echo $e->getMessage();
 }
?>

The major problem with this was the time taken to execute it: after 15 minutes I still didn’t have 50,000 rows in the database but only something like 38,000! The code worked but was amazingly slow. After running the code through a profiler I found that 14 mins 54 seconds were spent on the line ‘$st->execute()’ to insert the row. I was quite amazed by this as I didn’t expect the database to add quite such an overhead. The other important point to mention here is that I used the mt_rand() function. Why that’s important will become clear in a moment.

At this point I had two main questions:

1. What’s the quickest way to create 50,000 unique 8 digit codes?
2. What’s the quickest way to insert these rows into the database?

I should add that the need to find answers to both of these questions was somewhat academic and more to satisfy my own curiosity. This code would never be run on a live server but would be used on my development box to generate a database table. This could then be dumped into a SQL file and used to create the table on a live server. As a result performance was not the number one consideration in this case. I still wanted to run this in less than 15 minutes though!

The ‘Hamlet’ solution

At this point I turned to the Open University web development forums for help. I learnt web development at the OU and their forums are open to students, tutors and ex-students like myself. Some very clever people hang out there and I knew I would get some good suggestions. Two of the tutors of the course on open source web development tools, Michelle Hoyle and Keith Evetts, which focuses on PHP as a server side scripting language engaged in a discussion with me about how to generate codes quickly. From the beginning I noticed that ideas they were posting were not working for me. They were using PHP functions like str_shuffle(), array_rand() and rand() and producing 50,00 unique values quickly. I had independently tried str_shuffle() and array_rand() too but had given up on them as they generated huge numbers of duplicate codes after a while. At this point I couldn’t understand why Keith and Michelle could produce solutions using these functions that I couldn’t get to work.

Keith Evetts came up with an idea which seemed crazy at the time but which generates random values very well and quickly. His idea was to take a large piece of text, use mcrypt() to encrypt it, discard all non alpha-numeric characters from the encrypted text and use this to generate the 8 digit codes. He used the the third act of Hamlet as the text (hence my calling this the Hamlet solution). This worked extremely well except for the fact that even the third act of Hamlet couldn’t be used to produce 50,000 codes. The solution was to loop over the code generation, encrypting the text using different initialisation vectors and keystrings, until 50,00 uniques values had been produced. I also worked out at this time that the fastest way to insert this many values quickly into a MySQL table was to use the ‘LOAD DATA INFILE’ command. I used Keith’s function to generate the codes, writing these to a text file before using ‘LOAD DATA INFILE’ to insert the records into the database. The PHP code ran in 1.9 seconds while the database insert took around 6 seconds. Here’s the PHP code:

<?php
 /* ---------  array function generate_codes ( int $number, string $plaintext , string $keystring, $length ) -----------
Keith Evetts 3 July 2010 license: LGPL 2.  This notice and author name must remain intact.
Args: number of codes to be generated,
 plaintext string from which to generate them, minimum 100 chars (optimal length is 4000 chars)
 keystring in plain text; minimum 12 chars
 length of code strings to be generated
Returns: enumerated array of  unique alphanumeric codes of length $length
Requires: PHP mcrypt lib with Rijndael 256 (v. 2.4 + of mcrypt)
-------------------------------------------------------------------------------------------------------------------------------------- */
 function generate_codes ( $number, $plaintext, $keystring, $length = 8 ) {
 switch (true) {
 case ( ! is_int($number) )                                                :
 case ( ! is_int($length) )                                                  :
 case ( ! is_string($plaintext)  || strlen($plaintext ) < 100)   :
 case ( ! is_string($keystring) || strlen($keystring) < 12 )    :
 throw new Exception (' function generate_codes called with incorrect params ');
 break;
 // default is proceed
 }
 // use the same text and randomly different keys to reach desired number of codes
 // for e.g. 50000 codes this will take several passes
 $unique_array = array();
 do {
 // get a new key
 $key =  substr ( sha1 ( str_shuffle ( $keystring )  ) , 0 , 32 );
 // get a new initialisation vector
 $iv = substr ( sha1 ( str_shuffle ( "the slings and arrows of outrageous fortune" )  ) , 0, 32 );
 $ciphertext = mcrypt_encrypt (   MCRYPT_RIJNDAEL_256,
 $key,
 $plaintext,
 MCRYPT_MODE_CBC,
 $iv
 );
 /* clean out non-alphanumeric chars at some cost to code redundancy */
 $ciphertext = preg_replace( '/[^2346789abdefhjkmnprtwxyz]/' ,  "" , strtolower($ciphertext) );
 $codearray = str_split ( $ciphertext, $length );
 // dump leftover element at end
 array_pop( $codearray );
 $size = sizeof($codearray);
 for ($i = 0; $i < $size; $i++) $unique_array[] = $codearray[$i];
 /*somewhat amazingly, it is far quicker to enlarge the array by adding elements one at a time in a for loop, than to use array_merge() ! */
 } while ( sizeof ( $unique_array ) <= ( $number  + 1 ) ) ;
 // now remove any duplicates at end of whole process
 $unique_array = array_unique ( $unique_array );
 return array_slice($unique_array, 0, $number);
 }
 try {
 $codes = generate_codes(50000, file_get_contents('plaintext_Hamlet_Act3.txt'), 'This is the keystring');
 }
 catch (Exception $e) {
 echo $e->getMessage();
 }
 $file = fopen('codes.txt', 'w');
 foreach($codes as $code) {
 fwrite($file, "$code\r\n");
 }
 fclose($file);
?>

This was clearly the winner for me on Windows but it still didn’t explain why PHP random functions couldn’t be used for me on Windows.

‘Random’ functions on Windows-a theory

At this point I was left wondering why the various ‘random’ functions on Windows had performed so badly for me. I knew that Michelle Hoyle uses a Mac and that Keith Evetts scripts executed without a problem on a web server. I began to think that the problem was PHP on Windows. To test this out I uploaded one of the scripts that generated huge numbers of duplicates for me to a live server running CentOS Linux and everything worked without any problems. So what’s going on? I have a theory and once again I need to thank Keith Evetts for pointing me on the way to this. It seems that the PHP function rand() is simply a wrapper for the operating systems native random function (see here for more details). The PHP manual states:

Note: On some platforms (such as Windows), getrandmax() is only 32768. If you require a range larger than 32768, specifying min and max will allow you to create a range larger than this, or consider using mt_rand() instead.

My theory is that functions such as str_shuffle() or array_rand() are also using the native operating system random function. It just happens to be that the function on the Linux/Unix platform is far better than the one available under Windows, which would explain why the scripts run so differently under different platforms. Normally this wouldn’t create any problems but when you’re dealing with very large datasets where randomness is a necessity it becomes a problem. This would also explain why my initial attempt had no problems generating 50,000 unique values as I was using the mt_rand() function. This is based on mathematics known as the Mersenne Twister, which generates better random numbers, and does not rely on the operating systems random number generator. I don’t know any C or the PHP source code well enough to confirm this but this is my strong hunch. Is someone able to confirm this?

Conclusion

As I said at the beginning what should have been a fairly simple exercise turned into something a little more involved and ultimately informative. I would suggest that if you need to make a large number of random values that these are the guidelines you might want to follow:

• If you’re going to be running the script on a Linux/Unix box or you’re developing on such a system go ahead and use whichever PHP functions you like. As the underlying OS random number generation is better than on Windows you won’t run into any problems and will almost certainly get better performance this way.
• If you’re running on Windows your options are more limited. Here I would suggest that you use either mt_rand() or the ‘Hamlet’ approach if you require more than 32,768 random values.

I know that all programs rely on the OS they’re executing on but this discrepancy seems quite big to me. Is there any way that PHP’s ‘random’ functions could be re-written to take advantage of the Mersenne Twister? With my limited knowledge of the PHP core code that would seem to me to offer the combination of good random generation and consistency across platforms. Of course, if it were that simple I’m sure someone else would already have done it.

I recently had a situation where I wanted to use an array of values as bound parameters in a SQL IN clause. Easy enough to do except the array was of variable length and I didn’t know how long it would be each time the script was ran. Here’s the solution I came up with.

<?php

try {

$array = array('some value', 'another', 'another');//Variable length array, unknown length before runtime

$db = new PDO(CONNSTR, USERNAME, PASS);

$sql = "SELECT SomeColumn FROM table WHERE SomeOtherColumn IN (" . implode(',', array_fill(0, count($array), '?')) . ') ORDER BY SomeColumn';

$st = $db->prepare($sql);

$st->execute($array);

}

catch (PDOExeception $e) {}

?>

Quick and easy!

In creating the persistent login token I followed the principles outlined by Chris Shiflet in ‘Essential PHP Security’. This book is fantastic and would be my top recommendation for any web developer who is concerned about application security (which should be all of us!) to read. The basic principle is to create an identifier and a token if a user chooses to persist a login. These are then set in a cookie and recorded in the database along with a ‘timeout’ value. Shiflet recommends that the maximum time a persistent login cookie should be valid for is seven days to minimise the risk of loss or exposure of the cookie. Since a new cookie is generated every time the user visits the site as long as they visit at least once every seven days they will never need to login again. Finally, if a user chooses to logout the persistent login cookie is deleted so that they really are logged out.

Creating the Cookie

My login form displays a checkbox to the user which, if selected, triggers the mechanism whereby a cookie is set. If the user can be logged in from the submitted username and password I call a method to create the necessary token and identifier and record the values in the database. The code to do this follows:

$identifier = sha1(SALT . sha1($organisation . SALT));
 $token = sha1(uniqid(mt_rand(), true));
 $user->setPersistentLogin($identifier, $token, $organisation);
 setcookie('auth', "$identifier:$token", time() + 60 * 60 * 24 * 7, '/', '' , true);

The identifier is generated from the user’s username for the application, which is then double hashed along with a salt. By creating a secondary identifier in this way I ensure that no details that an attacker could use to log into the application through the login form are set in the cookie. A random token is then generated and hashed before the information is recorded in the database in the setPersistentLogin method of the user object. Finally the cookie holding the login information is set, with an expiry time of one week. My database table includes columns for the identifier and token and a datetime column for the expiry date. The SQL to record the information looks like this (my database access is done using PDO prepared statements):

UPDATE users SET Identifier = :identifier, Token = :token, Timeout = DATE_ADD(NOW(), INTERVAL 7 DAY) WHERE Organisation = :Organisation

Logging in Using the Cookie

When a user requests the login page my controller for that page executes the following code:

if ((isset($_POST['token']) && $_POST['token'] === $_SESSION['token']) || isset($_COOKIE['auth'])) {

if (isset($_POST['token']) && $_POST['token'] === $_SESSION['token']) {

$captcha = isset($_SESSION['captcha']) ? $_SESSION['captcha'] : false;

$this->model->inputs = $_POST;

$this->model->captcha = $captcha;

$this->model->setContent();

} else if (isset($_COOKIE['auth'])) {

list($identifier, $token) = explode(':', $_COOKIE['auth']);

$this->model->identifier = $identifier;

$this->model->token = $token;

$this->model->setContent();

}

if ($this->model->valid) {

//Log the user in and redirect them.

}

} else {

//Call setContent with no values set.

$this->model->setContent();

}

This code looks for the existence of either a form submission or a persistent login cookie. If either is set appropriate properties are set in the model for the page before the setContent() method is called. The part of the setContent() method handling the login looks like this:

try {
if (($this->identifier && $this->token) || $this->inputs) {
 $user = new userAuth();
 if ($this->identifier && $this->token) {
 //Attempt a login based on the persistent token values
 $result = $user->checkPersistentLogin($this->identifier, $this->token);
//No exceptions thrown, successful login.
$this->setPersistentLogin($result['Organisation'], $user);
} else if (isset($this->inputs)) {
 if (! $this->validate($this->inputs)) {
 throw new InvalidArgumentException('Errors in form submission values');
 }
 //Attempt a login based on the form submitted
 $result = $user->checkLogIn($this->results['username'], $this->results['password']);
 }
 //No exceptions thrown, successful login.
 $this->organisation = $result['Organisation'];
 $this->contactName = $result['ContactName'];
 $this->email = $result['Email'];
 $this->questionsAnswered = $result['Questions Answered'] === 'True' ? true : false;
 if (isset($this->results['remember']) && $this->results['remember'] === 'rememberMe') {
 $this->setPersistentLogin($this->organisation, $user);
 }
 $this->valid = true;
 return;
 }
 }
 catch (InvalidArgumentException $e) {
 /**
 * Invalid login attempt.
 * If inputs is set display an error message as there was an invalid form submission.
 * Invalid login from a token won't display this message but the user will see a login form
 */
 if ($this->inputs) {
 $this->content['error'] = 'Either your username or password is incorrect. Please try again.';
 $this->content['captcha'] = true;
 $this->content['username'] = is_array($this->results['username']) ? $this->results['username']['value'] : $this->results['username'];
 $this->content['password'] = is_array($this->results['password']) ? $this->results['password']['value'] : $this->results['password'];
 }
 }

This code looks for the existence of either form values ($this->inputs) or values from a persistent login cookie ($this->identifier and $this->token). For the code checking the persistent login cookie the checkPersistentLogin() method is then called. This throws an InvalidArgumentException if the users details cannot be verified, which is caught later in the example above. If the user is verified a result array is returned and a new persistent login cookie is set. The SQL to check the persistent login is as follows:

SELECT Organisation, ContactName, Email
FROM users
WHERE Identifier = :identifier
AND Token = :token
AND Timeout > DATE_SUB(NOW(), INTERVAL 7 DAY)

Logging a User Out

When logging a user out it’s necessary to delete a persistent login cookie to make sure that they really are logged out. This is done in the following code:

if (isset($_COOKIE['auth'])) {
 setcookie('auth', false, time() + 60 * 60 * 24 * 7, '/', '', true);
 }

The php manual states that:

If the value argument is an empty string, or FALSE, and all other arguments match a previous call to setcookie, then the cookie with the specified name will be deleted from the remote client. This is internally achieved by setting value to ‘deleted’ and expiration time to one year in past.

By using the same arguments to set cookie but setting the value to false this code makes sure that the persistent login cookie is deleted.

I’m fairly confident that this implementation of a persistent login is as secure as I can make it but it can never be as secure as requiring a manual login from a user on each visit. In deciding that a persistent login mechanism was appropriate for this application I considered the user base and how they would likely be using the application. While a persistent login was appropriate for this application it may well not be for others. I’d appreciate any comments or ideas anyone may have on how I’ve done this or on the benefits and risks of persistent login.

I have a project that I’m working on where I’m storing some images in a database as binary data. PDO allows you to bind a file handle to a parameter in a prepared statement and when the statement is executed the contents of the file are slurped into the database. This works perfectly but the problem comes when getting the image out of the database again to display it. According to the PHP manual the following code should work:

<?php
$db = new PDO('odbc:SAMPLE', 'db2inst1', 'ibmdb2');
$stmt = $db->prepare("select contenttype, imagedata from images where id=?");
$stmt->execute(array($_GET['id']));
$stmt->bindColumn(1, $type, PDO::PARAM_STR, 256);
$stmt->bindColumn(2, $lob, PDO::PARAM_LOB);
$stmt->fetch(PDO::FETCH_BOUND);

header("Content-Type: $type");
fpassthru($lob);
?>

Binding a column from a result set to a variable using PDO::PARAM_LOB is supposed to return a stream resource into the variable when PDO::fetch() is called. This stream can then be operated on using any PHP function that handles files. Unfortunately there’s a bug which means that instead of returning a stream into $lob PDO returns a string containing the binary data. When this is then passed to fpassthru() an error is triggered. Fortunately there’s a simple fix for displaying the image: replace the call to fpassthru() with echo or print. Since the browser is expecting an image after the call to header() writing the binary data using echo or print has the same effect as calling fpassthru(). In my code I’ve added the following just in case this bug is fixed in a future release:

if (is_string($lob)) {

echo $lob;

} else {

fpassthru($lob);

}

This neatly gets around the problem if you just want to send the binary data back to the browser to be displayed. Anything more requiring the use of any file functions or image editing functions would need quite a few contortions in the code. The information from the database would probably need to be written to a temporary file to allow it to be operated on. This bug was first reported almost three years ago in PHP 5.2.6 and it’s still not fixed today in the most recent version, 5.3.1. It would be great if this bug was finally taken care of.

Edit: Joshua Johnston has posted a comment below that explains how to convert a string of data into a stream using the data stream wrapper. I’ve tried it out and it works very well. I think it gives a cleaner solution to the problem and allows the data returned from the database to be manipulated with file functions.

To display the local store information I needed to go through a number of distinct steps in the code.

1. Determine the visitors location from their IP address.
2. If the visitor is in Canada calculate the distance to their closest store.
3. If they’re within 100km of a store display information on that store on the homepage.
4. If the user is not in Canada, is more than 100km away from a store or if there is any kind of error display a generic message about store locations.

Getting the Users Location

Determining a visitors location from their IP address can be done through a number of GeoIp services, some available for free and some paid for. Ideally I wanted to use a PECL PHP extension to do the location lookup but this was not possible as I could not persuade my web host to install this. I fell back to using a free web service provided by ipinfodb for the lookup. I found a class on PHPClasses that used this web service, which I substantially rewrote to serve my purposes. The code that performs the lookup in the class is below:

$strAPIURL = $this->apiUrl . "?ip=" . urlencode ($this->remoteAddress);
 //Make a call to the api and fetch the result.
 $ch        = curl_init ($strAPIURL);
 curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
 $xmlResult = curl_exec ($ch);
 curl_close ($ch);

 if ($xmlResult && strlen ($xmlResult) > 2) {
 //Process the result
 $result = simplexml_load_string ($xmlResult);

 if ((string)$result->Status == 'OK') {
 $this->countryName   = (string)$result->CountryName;
 $this->countryCode   = (string)$result->CountryCode;
 $this->cityName      = (string)$result->City;
 $this->zipPostalCode = (string)$result->ZipPostalCode;
 $this->regionName    = (string)$result->RegionName;
 $this->regionCode    = (string)$result->RegionCode;
 $this->timezone      = (int)$result->Timezone;
 $this->gmtOffset     = (int)$result->Gmtoffset;
 $this->dstOffset     = (int)$result->Dstoffset;
 $this->lat           = (float)$result->Latitude;
 $this->long          = (float)$result->Longitude;
 //Set the ipSniffed flag to true
 $this->ipSniffed     = true;
 }
 }

Once the call to the web service has been completed the information returned is stored in public properties and a flag is set to show that a successful lookup was performed. My class uses filter_var() to make sure a valid IP address that is not in a private range is passed to it, throwing an InvalidArgument Exception if not. Of course, if an exception is thrown I fall back to my default position of displaying a generic message about store locations.

Finding the Closest Store

Once I have the users location from their IP address and I’ve determined that they’re visiting from Canada I proceed to calculate the distance to their closest store. This is done with a database query. Information about all of the stores is stored in a database table along with the latitude and longitude of each store. Using the latitude and longitude returned from the web service I can then calculate the users distance to the nearest store. This is done in the following code:

protected function getClosestStore ($lat, $long) {
 $sql = <<< _SQL_

SELECT StoreId, CONCAT_WS(', ', Address, City, CONCAT(Province, ' ', PostCode)) AS Address, Lat, Lon, ROUND(6371 * 2 * ASIN(SQRT(POWER(SIN((:lat - abs(lat)) * pi()/180 / 2), 2) +
COS(:lat * pi()/180 ) * COS(abs(lat) * pi()/180) *  POWER(SIN((:lon - lon) * pi()/180 / 2), 2) )),2)
AS Distance
FROM Stores
HAVING Distance <= 100
ORDER BY Distance Limit 1
_SQL_;
 try {
 $db = db::getConn();
 $st = $db->prepare($sql);
 $st->execute(array(':lat' => $lat, ':lon' => $long));
 $this->storeInfo = $st->fetch(PDO::FETCH_ASSOC);
 }
 catch (PDOException $e) {
 error_log($e->getMessage());
 }
 }

This PHP in this code simply connects to the database (db::getConn() is a static method that returns a singleton instance of a PDO object), performs the query and stores the result in the storeInfo property. If no result is returned from the database the fetch() method will return false, which then tells me that a user is not within 100 km of a store. The real meat of this code is in the SQL query. It selects some information about the store and then uses some math to calculate the distance to the nearest store. This is done using the Haversine formula, which calculates the distance between two sets of latitude and longitude, taking into account the curvature of the earth. I’m not going to try to explain the math (mostly because I don’t fully understand it myself!) but there is a Wikipedia article on the formula for anyone who is interested. The query then limits the results to stores that are within 100km, orders the results by distance to make sure the closest one is listed first and then returns the first result. If a result is returned I then record the StoreId of the closest store in a cookie, which is then used when the visitor visits the site again. This is to cut down on processing time for subsequent request for the home page and means that I won’t need to hit the ipinfodb web service on every request for the index page of the site.

Limitations of this Solution

There are three major limitations to this solution that I can see: the accuracy of GeoIP services, the database query and saving the local store information in a cookie.

GeoIP services claim about an 80% accuracy when tracking the geographic location of an IP address down to the city level (accuracy is about 95% for finding the country a user is visiting from). For example I am writing this in Guelph, Ontario but the IP address I am connected to the internet with resolves geographically to the nearby city of Kitchener. This is not a major problem for my application as the client only has 35 stores across Canada and I am just trying to find the closest one to a user. Given the small number of stores chances are that I will hit on the closest one, even allowing for only 80% accuracy in GeoIP tracking. For other applications this could be a problem. The W3C has a draft Geolocation API which some browsers (such as Firefox) are starting to implement. Using this (perhaps with some AJAX) could help to get a more accurate fix on some users locations but would open up some privacy concerns. For this application I display links that allow a user to manually select their closest store if the GeoIP tracking is wrong, hopefully alleviating any problems caused by the 80% accuracy.

The SQL query as I’ve written it would have performance issues if a large number of locations were being stored. To calculate the distance the query has to find the distance to every store held in the database, only then narrowing it down to the closest. This is not a problem currently as the client only has 35 locations, but if hundreds or thousands of locations were being stored the query would be extremely innefficient and take a long time to execute. In the course of my research I did read something about limiting the search to a radius (100 km in this case). This would involve several queries and probably a stored procedure to carry them out in. As I was dealing with a small number of locations I decided against this for simplicity. For another solution involving more locations I would probably go with this approach.

Saving the information on the users local store in a cookie makes the application more efficient but it potentially slightly degrades the user experience. In a scenario where a user travels and expects to see information on the closest store to where they currently are this would not work. I made a judgement call here and decided that it was better to go with the more efficient approach for this client, but in other cases this may not be so. Of course, if I could install the PECL GeoIP extension I wouldn’t need to store the cookie. Looking up a users location using this extension would be no more complicated than calling some PHP functions. The GeoIP information would be stored locally as part of the extension. This would be quicker than creating a call to and processing a response from a web service. Unfortunately this was not an option for me on this occasion. This problem can be overcome by the user manually selecting their closest store, overriding the mechanism I programmed.

All in all I am fairly happy with the solution I arrived at for this problem. It enables me to tailor content on the page based on where a user is. My example is a fairly simple one but it would be possible to do far more sophisticated things using GeoIP such as automatically displaying content in different languages depending on which country a visitor is coming from or displaying prices in different currencies. Due to the less than 100% accuracy of GeoIP mechanisms would always need to be provided to allow a user to override the conclusions found by the code though.