Iterating over Objects

PHP natively allows us to iterate over objects in a foreach loop. However, this only works with public properties of the object. If your object has public properties these will be the values provided in each iteration. If you’re following the concept of encapsulation in your objects though you’ll probably be using private or protected properties with getters and setters. Obviously this presents a problem if you want to iterate through the properties of the object. This is where IteratorAggregate and Iterator come in. If you implement one of these interfaces in your class you can define your own logic which governs the values to return when an object is iterated over.

On Being Traversable

Before we have a look at implementing IteratorAggregate or Iterator I should briefly mention the Traversable interface. Traversable is a bit of an oddity. The PHP manual defines it as an ‘abstract base interface that cannot be implemented alone. Instead it must be implemented by either IteratorAggregate or Iterator’. The manual goes on to explain that internal classes can implement Traversable and these can be used in a foreach construct. If Traversable is an interface that you can’t implement yourself then what good is it? You can use it in type hinting or with the instanceof operator to detect if an object can be used in a foreach construct. If you simply need to find out if you can use an object in a foreach loop and are not too worries about the exact type of the object checking if Traversable is implemented is a quick and easy solution.

Implementing IteratorAggregate

Implementing IteratorAggreagate is extremely easy. You only need to implement one method, getIterator. This method has to return a data structure that can be iterated over and the PHP engine automatically calls this method when it detects an object implementing this interface being used in a foreach loop. Of course it’s up to you whether this is an array or an object that implements the Traversable interface. A simple, contrived example of implementing this interface is as follows:

class Foo implements IteratorAggregate

{

protected $values = array();

//various methods here to work with the object and set values in $values

public function getIterator()

{

return $this->values;

}

}

IteratorAggregate is perfect for occasions when your object holds an array or Traversable object of values and all you want to do is to iterate over these. If you need to do anything more complicated, such as altering or examining values before they’re returned, you need to use it’s big brother, Iterator.

Implementing Iterator

The iterator interface is a bit more complicated that IteratorAggregate and requires that you implement five methods: current, key, next, rewind and valid. Current should return whatever the current value should be in the iteration while key should return a key value for it. Next is used to advance the Iterator forwards by one step while rewind should reset whatever pointer is being used to determine the iteration. Finally, valid should return a boolean value to indicate if a value exists for the current iteration. This seems like a lot but the PHP engine takes care of calling these methods for you at the appropriate times in a foreach loop. In most cases you’ll also find that next, rewind and valid can be pretty minimal methods. A (once again contrived) example of using Iterator is below.

class Bar implements Iterator

{

protected $values;

protected $counter = 0;

//Various methods to set data and to work with the object

public function rewind()

{

$this->counter = 0;

}

public function next()

{

$this->counter++;

}

public function valid()

{

return array_key_exists($this->counter, $this->values);

}

public function current()

{

return $this->values[$this->counter];

}

public function key()

{

return $this->counter;

}

}

This example is obviously very simple and holds no real advantages over IteratorAggregate. The true advantage of Iterator over IteratorAggregate is that it allows you to manipulate the data in key and current before it is returned. One place that I’ve done this a lot recently is in working with the SalesNet CRM API. The API returns results as XML which I process using XPath. The result of this is an array of SimpleXML elements that I can iterate over. By using Iterator for this instead of IteratorAggregate I can transform the SimpleXMLElement objects into scalar values in the current method before they’re returned. I can also have the key method return account or deal id’s for the current result. This would not be possible with IteratorAggregate. For me this was an ideal way of working with the results of a web service call in a view layer. I would usually implement the countable interface too which would be used to return the number of results held by the object. A simple call to count() would check this, followed by iteration over the object and display of the results.

I’ll post another article in this series in the next few weeks looking at using Countable, ArrayAccess and Iterator/IteratorAggregate all together in one object. I’ll also take a brief spin through the ArrayObject class and when you might want to use it.

Solving this problem in PHPUnit

This problem isn’t exclusive to unit testing with Doctrine. Mike Lively, the author of PHPUnit’s DbUnit extension, has provided a solution in a Gist. Basically, this extends the PHPUnit_Extensions_Database_Operation_Truncate class to suspend MySQL foreign key checks for the length of the truncate operation. The getSetUpOperation() method then needs to be overridden in the test class to use the custom MySQL truncate class. All in all a neat and elegant solution.

Porting this Solution to Doctrine

Porting this solution turned out to be fairly simple, with the exception of one issue which needed to be addressed. I started with the custom MySQL truncate class, which is almost an exact copy of the one Mike Lively wrote:

<?php

namespace Tests\Models\Db;

use PHPUnit_Extensions_Database_Operation_Truncate,
 PHPUnit_Extensions_Database_DB_IDatabaseConnection,
 PHPUnit_Extensions_Database_DataSet_IDataSet;
/**
 * MySQL truncate class that allows foreign key checks to be suspended for the duration of the truncate operation.
 * @author Mike Lively
 * @author Jeremy Cook
 * @version 1.0
 * @package Veridis
 * @see https://gist.github.com/1319731
 */
class MySQLTruncate extends PHPUnit_Extensions_Database_Operation_Truncate
{
 /**
 * Executes the truncate.
 * Note: the $connection variable here is an instance of DoctrineExtensions\PHPUnit\TestConnection
 * (non-PHPdoc)
 * @see PHPUnit_Extensions_Database_Operation_Truncate::execute()
 */
 public function execute(PHPUnit_Extensions_Database_DB_IDatabaseConnection $connection, PHPUnit_Extensions_Database_DataSet_IDataSet $dataSet)
 {
 $connection->getConnection()->exec("SET @PREVIOUS_foreign_key_checks = @@foreign_key_checks");
 $connection->getConnection()->exec("SET @@foreign_key_checks = 0");
 parent::execute($connection, $dataSet);
 $connection->getConnection()->exec("SET @@foreign_key_checks = @PREVIOUS_foreign_key_checks");
 }

}

This is pretty self explanatory and simply disables MySQL’s foreign key checks while the database tables are truncated and populated with data. I then created an abstract base class which extends the OrmTestCase class to set up the entity manager. This can then be extended by any class that needs to test against the database with an entity manager. To this class I added the code to register the custom MySQL truncate class, and a simplified version of the class looks like this:

<?php

use DoctrineExtensions\PHPUnit\OrmTestCase,
 Doctrine\ORM\Configuration,
 Doctrine\Common\Cache\ArrayCache,
 Doctrine\ORM\EntityManager,
 PHPUnit_Extensions_Database_Operation_Composite,
 PHPUnit_Extensions_Database_Operation_Factory;

/**
 * Base class for test cases that need an entity manager to work with.
 * Defines an entity manager for the tests to work with.
 * @author Jeremy Cook
 * @version 1.0
 */
 abstract class Base extends OrmTestCase
 {
 /**
 * Method to create the entity manager, used by parent methods.
 * @return \Doctrine\ORM\EntityManager
 * @see OrmTestCase::createEntityManager()
 */
 protected function createEntityManager ()
 {
 $config = new Configuration;
 $cache = new ArrayCache();
 //Set up config here.
 $conn = array();
 return EntityManager::create($conn, $config);
 }
 /**
 * Overrides the parent method to add a custom MySQL truncate operation.
 * This suspends foreign key checks for the duration of the truncate command.
 * (non-PHPdoc)
 * @see DoctrineExtensions\PHPUnit\DatabaseTestCase::getSetUpOperation()
 */
 public function getSetUpOperation()
 {
 $truncate = new MySQLTruncate();
 return new PHPUnit_Extensions_Database_Operation_Composite(array(
 $truncate,
 PHPUnit_Extensions_Database_Operation_Factory::INSERT()
 ));
 }
 }

The createEntityManager method is defined as abstract in OrmTestCase and needs to return an instance of Doctrine\ORM\EntityManager. The getSetUpOperation method is defined in PHPUnit_Extension_Database_TestCase and is overriden here to add the custom MySQL truncate class. There’s one final abstract method that needs to be implemented in a test class and that’s getDataSet. This is defined in PHPUnit_Extension_Database_TestCase and needs to return a PHPUnit_Extensions_Database_DataSet_IDataSet instance to populate the database with data. With this I created a test case extending this class, added data to populate the database, ran it and got an exception. What was going on?

The Last Problem

I traced the exception down to DoctrineExtensions\PHPUnit\TestConnection::getTruncateCommand(). This method was hard coded to throw an exception when called. I spoke to Benjamin Beberlei on IRC to ask him about this and he couldn’t remember why this code was added here. I made an edit so that the getTruncateCommand method looks as follows:

<?php

/**

* Returns the command used to truncate a table.
*
* @return string
*/
public function getTruncateCommand()
{
 return 'TRUNCATE';
}

With this everything seems to work. Doctrine’s PHPUnit test extension is able to truncate tables in a MySQL database that have foreign key constraints.

What is the ArrayAccess Interface?

ArrayAccess allows you to treat an object that implements it as if it is an array for the purposes of setting, unsetting and retrieving data from it. Please note the emphasis in the last sentence! ArrayAccess does not make an object behave like an array in any other way. If you pass an object that implements ArrayAccess to a PHP array function such as in_array() you’ll still get an error. This will become a little clearer with some of the examples below.

To implement ArrayAccess you need to implement four methods: offsetExists, offsetGet, offsetSet and offsetUnset. ArrayAccess::offsetExists must return a boolean, offsetGet can return any valid PHP type while offsetSet and offsetUnset should not return any value. Once you implement these methods you can treat an object as if it is an array for the purposes of saving and retrieving properties. Here’s a brief example.

  1: class Foo implements ArrayAccess {

  2:

  3: 	protected $_values = array();

  4:

  5: 	public function offsetExists($offset)

  6: 	{

  7: 		return array_key_exists($offset, $this->_values);

  8: 	}

  9:

 10: 	public function offsetGet($offset)

 11: 	{

 12: 		return $this->offsetExists($offset) ? $this->_values[$offset]:NULL;

 13: 	}

 14:

 15: 	public function offsetSet($offset, $value)

 16: 	{

 17: 		$this->_values[$offset] = $value;

 18: 	}

 19:

 20: 	public function offsetUnset($offset)

 21: 	{

 22: 		if ($this->offsetExists($offset)

 23: 			unset($this->_values[$offset]);

 24: 	}

 25: }

 26:

 27: $bar = new Foo();

 28: $baz = isset($bar['baz']); //Calls offsetExists and returns false

 29: $bar['baz'] = 1; //Calls offsetSet and adds a new member to the _values array with the key of 'baz' and the value of 1

 30: $baz = isset($bar['baz']); //true

 31: echo $bar['baz']; //Calls offsetGet and prints 1.

 32: unset($bar['baz']);//Calls offsetUnset and removes the value from the internal _values array

Hopefully this should all be fairly self-explanatory and you can see how ArrayAccess can allow you to treat an object like an array. Once again, note that this only allows you to use the array notation to set, unset, retrieve and find out if values are set in the internal _values property. If you want to count the number of values set you’d need to implement Countable and if you wanted to be able to iterate over the object you’d need to implement Iterator or IteratorAggregate. Even then the object could not be passed to PHP array functions.

The above example is also incredibly simplistic. There’s nothing to say that the internal storage for the properties being set has to be an array. Likewise, the logic in your implementation of the ArrayAccess methods can be as simple or as complicated as you need.

Hang on a minute…

In reality the only thing that ArrayAccess gives you that can’t be achieved using PHP’s magic methods is the ability to use array notation. Apart from that there’s nothing here that can’t be achieved with the magic __get(), __set(), __isset() and __unset() methods. So, the question here has to be why bother with ArrayAccess at all when it only provides syntactic sugar? This is a hard one to answer and really comes down to a matter of personal preference. I personally quite like the syntax ArrayAccess gives, especially when coupled with the Countable and Iterator interfaces. For me as well it also gives me a different mindset. I’m used to my IDE providing me with code completion, which of course doesn’t work with magic methods. I always find it a little unnerving when the code completion doesn’t appear, even when I know that it’s not working because of PHP’s ‘magic’. This isn’t a problem with the array syntax that ArrayAccess provides. Of course this is a very subjective viewpoint and those of you reading this will I’m sure have or develop your own opinions on this.

A practical example

One place where ArrayAccess is very useful is in an object where you have a large number of arbitrary values that you need to store and retrieve. I’ve recently been doing a lot of work with the SalesNet CRM system for a client through their SOAP API. The SalesNet API exposes a massive amount of functionality and a lot of the methods can accept a large number of fields to set data for or retrieve data from. SalesNet also allows it’s customers to create custom defined fields and these can both be set and retrieved through the API.

Since there are so many possible values that can be set and retrieved this is a perfect case for using ArrayAccess. I simple pass the name of the SalesNet field that I’m setting or retrieving data from as the ‘array key’. If I’m storing the data in the object to be sent to SalesNet I can retrieve the key and value pairs and process them as needed before making the SOAP call.

Things can get a little more interesting when retrieving data from SalesNet. The response payload is a large lump of XML in most cases. I’ve often resorted to processing this with XPath to get to the nodes I interested in and storing these as an internal property of the object. I can then use offsetGet and offsetExists to work with the data, usually transforming it in some way before returning it. I also often implement the Countable interface to return the number of results. Consider the following (simplified) example:

  1: class SalesNetDemo implements ArrayAccess, Countable

  2: {

  3: 	//Array of SimpleXML objects following processing of the response.

  4: 	protected $_data = array();

  5:

  6: 	protected $_soap;

  7:

  8: 	public function __construct($soap)

  9: 	{

 10: 		$this->_soap = $soap;

 11: 	}

 12:

 13: 	public function doSoapCall()

 14: 	{

 15: 		//Code to do a Soap call and process the results here using XPath.

 16: 	}

 17:

 18: 	public function count()

 19: 	{

 20: 		return count($this->_data);

 21: 	}

 22:

 23: 	public function offsetExists($offset)

 24: 	{

 25: 		return (isset($this->_data->$offset) &&

                 '' !== (string) $this->_data->$offset);

 26: 	}

 27:

 28: 	public function offsetGet($offset)

 29: 	{

 30: 		return (isset($this->_data->$offset) &&

                 '' !== (string) $this->_data->$offset) ? (string) $this->_data->$offset : NULL;

 31: 	}

 32:

 33:

 34:     	public function offsetSet ($offset, $value)

 35:     	{

 36:         	throw new BadMethodCallException(sprintf(

 37:             		'Values cannot be changed in this class, thrown in %s',

 38:             		__METHOD__

 39:         	));

 40:     	}

 41:

 42:     	public function offsetUnset ($offset)

 43:     	{

 44:         	throw new BadMethodCallException(sprintf(

 45:             		'Values cannot be changed in this class, thrown in %s',

 46:             		__METHOD__

 47:         	));

 48:     	}

 49: }

 50:

 51: $foo = new SalesNetDemo(new SalesNetSoapObject);

 52: $foo->doSoapCall();

 53: if(count($foo)) {

 54: 	//Results retrieved from the API.

 55: 	//Try to get a valueand echo it.

 56: 	echo $foo['CDF12345'];

 57: }

Again, this is a pretty simple example. The object is only designed to allow data to be retrieved and read, which is why offsetSet and offsetUnset throw exceptions. A real world example might allow the same object to amend the data through the API. The real point of the example though is to show you how the data can be transformed before returning it from the object. Again, the example is deliberately simplistic but you can get the idea of the kinds of things that can be achieved. This is also an example as to how multiple predefined interfaces can be combined to achieve some pretty powerful results.

I hope you’ve enjoyed this brief, and simple, look at ArrayAccess. Let me know in your comments how you you’ve used ArrayAccess in your own code. I’ll add another post as soon as I have time looking at the Iterator and IteratorAggregate interfaces and I’ll also try to add an example that implements Countable, ArrayAccess and Iterator to show how they can work together.

The Countable Interface

The Countable interface at it’s simplest level allows you to pass objects that implement it to PHP’s native count function. The interface is, on the surface, extremely easy to implement. You only need to add the Countable::count() method to implement the interface in your class and this method only needs to return an integer. If you pass an object that implements Countable to the PHP count function the PHP interpreter will automatically call the count method in the object, returning the value from it to you to use.

Why is Countable so Useful?

Like all of the predefined interfaces and SPL functionality the real power of countable comes from the ability to effectively override and re-implement core PHP functionality with your own logic. I’ll give three brief examples here of ways that I’ve personally used Countable recently.

Accessing the State of a Private or Protected Property

One of the simplest uses of countable is to return the count of an array that’s held as a protected or private property of an object. Take the following example:

class SomeClass implements Countable {

 protected $_data = array();

 public function doSomething()
 {
//Code to change $_data here
 }

 public function count()
 {
return count($this->_data);
 }
}

In this example some data is stored in the object as an array and the Countable:Count method simply returns the count of that array. While this is obviously a very simple example it can be very powerful. I’ve been doing a lot of work with a web service in the last few weeks and have used code very much like this. After a web service call the resulting XML is processed using XPath and the resulting array is stored in a protected property. The count method would then return the number of results of the web services call. Using count in this way obviously makes a lot of sense when coupled with implementing Iterator or ArrayAccess to allow access to the data held in the protected property.

Using Countable to Signal the State of an Object

Countable can also be used to signal the state of an object. Imagine the above code, amended slightly to this:

class SomeOtherClass implements Countable {

protected $_data;

public function doSomething()
{
//Code to change $_data here
}

public function count()
{
return is_null($this->_data) ? 0 : 1;
}
}

In this case count will only ever return 0 or 1, depending on whether the protected property $_data has a value set in it or not. Using this code could look something like this:

$obj = new SomeOtherClass();

$obj->doSomething();

if (count($obj)) {

//$_data is not null, proceed accordingly.

}

I’ve also used this recently with a web service call. In this case the web service call was expected to return a single value, which is stored in $_data. Implementing countable like this simply signifies to the calling code if a value has been successfully retrieved or not.

Using Countable with a Database Table

Another interesting use of Countable is with a class that maps to a database table in some way. Imagine a case where you have a database table where values are only considered ‘live’ if some condition is met. Implementing countable on an object that encapsulates the db table can then be used to flag if the table contains any ‘live’ values. The following example uses Zend_Db_Table:

class AnotherClass extends Zend_Db_Table implements Countable {

 public function count()
 {
$select = $this->select(Zend_Db_Table::SELECT_WITH_FROM_PART)
->from('some_table', array('Count' => 'COUNT(*)'))
->where('SomeCondition = SomeValue');
$row = $this->fetchRow($select);
return (int) $row->Count;
 }
}

In your calling code you would then simply do this:

$obj = new AnotherClass();

if (count($obj)) {

//Table has 'live' values. Do something here

}

This can be a very useful way of working with a database table.

A Word of Caution…

The only limit to the way Countable can be used is the imagination of the developer, and this is also a danger. All developers are familiar with using the count() function to get hold of the number of values in an array. When implementing Countable the further you move the count() method away from this functionality the less intuitive the code becomes and the greater the chance that the developer using it might make a mistake. This is true of all of the predefined interfaces however. While they provide great power the further away from core PHP functionality you get while implementing them in your own code the less intuitive they become. However, don’t let this stop you! The next time you go to write a method called something like ‘getItemCount()’ in a class don’t do it! Implement the Countable interface instead and start hooking into the power of the core PHP engine.

API Format

The SalesNet API is SOAP based and consists of 8 separate endpoints, each of which allows you to operate on different parts of a companies data such as deals, accounts, contacts, etc. Before you can use any of these services it’s necessary to authenticate against a separate security endpoint. This returns a token which is valid for between 1-12 hours (depending on the account setup in SalesNet). This token must then be set in a SOAP header for use in requests to any of the main SalesNet endpoints. Each of the API methods accepts many arguments to control precisely the data that is returned and the only thing I can suggest to anyone looking to use the API is to refer to the documentation. One of the quirks of the API is that data is returned as XML, embedded in a stdClass object. For example, a call to the GetDeals method of the deals endpoint returns a stdClass object (say $obj), and to access the response data the following would be needed:

$response = $obj->GetDealsResult->any;

In the above example $response would now include the XML data ‘payload’, which can then be parsed. It’s often not that easy to parse the XML returned as it seems to be an XML representation of a Microsoft ADODB recordset, which in some cases is not even properly formed XML. I’ve had luck using SimpleXML and Xpath to extract the nodes I’m after however.

The Code

To work with the various SalesNet API’s I created two classes, one of which handles authentication and the other of which acts as a proxy to catch and pass SOAP calls through to the API. Both of the classes make use of ‘lazy loading’ and will only connect to SalesNet as and when a method is called that requires a SOAP call.

The Authentication Class

My goal in the authentication class was to simplify authenticating against the SalesNet security endpoint. The object accepts a company name, username and password and will retrieve a security token from SalesNet. This is then set in a PHP SoapHeader object and is returned for use whenever the doLogin() method is called. Internally the class will keep track of when the token expires and will automatically fetch a new token from SalesNet as and when required. My thought is that an object of this class can be serialized and cached to be stored and shared between many requests as a security token can be valid for up to 12 hours.

The SoapProxy Class

The SoapProxy class provides a proxy to trap calls to the many SalesNet API methods. Since there are so many methods cross so many endpoints it would be impossible to map these out to individual methods so I chose to use __call() instead to pass calls through to a PHP SoapClient object. The class stores the URL endpoints of the various services as constants and an API endpoint can be set when an object is instantiated or before any method is called. The object also accepts an instance of the Authentication class and uses this internally to set the security SoapHeader for API calls.

A Brief Example

So, enough talk! As I said earlier full code can be found on GitHub but here’s a brief example of how to use the classes to call the GetDeals method of the Deals endpoint.

<?php
use WebServices\Soap\SalesNet\Authentication, WebServices\Soap\SalesNet\SoapProxy;

try {
    $auth = new Authentication();
    $auth->setCompanyLogin('COMPANY LOGIN HERE')
         ->setUserName('USERNAME HERE')
         ->setPassword('PASSWORD HERE');

    $soap = new SoapProxy(SoapProxy::DEALS, $auth);
    $args = array(
        //Set up method arguments here
    );
    $response = $soap->GetDeals($args);
} catch (BadMethodCallException $e) {
    /**
     * Errors will be caught here.
     * The BadMethodCallException can have a previous exception of a SoapFault if there is one.
     */
}

That’s all there is to it. To use other endpoints simply change the constant passed to the SoapProxy constructor and call another Soap method, with the correct arguments of course. Let me know of any comments on this code or improvements people can see. Enhancements and fixes on GitHub are welcome.

Zend_Validate includes two classes to work with database columns: Zend_Validate_Db_NoRecordExists and  Zend_Validate_Db_RecordExists. As their names suggest these check that a value either does or doesn’t exist in a specified column in a database table. The classes work really well in a form when specified as a validator for a value but things can get a little trickier if you need to check two columns in a table. The problem I had was in creating a simple FAQ page for a site. The site owner gets to add FAQs to the site and each FAQ is contained in a category. As a little sanity check I wanted to make sure that the combination of category and question was unique. The database also has a multiple column unique key on these to enforce the integrity. My first attempt looked something like this in a class that extends Zend_Form:

public function init() {
//Category options are populated elsewhere.
$categories = new Zend_Form_Element_Select('category');
$categories->setAllowEmpty(FALSE)
	   ->setAttrib('required', 'required')
	   ->setLabel('FAQ Category:')
	   ->addErrorMessage('Please choose a valid category from the list.')
	   ->addMultiOption('', '--Please Choose--');
$question = new Zend_Form_Element_Text('question');
$question->setRequired(TRUE)
	 ->setLabel('Question:')
	 ->setAttrib('required', 'required')
	 ->setAttrib('max', '100')
	 ->addValidator('StringLength', FALSE, array(0,100))
	 ->addErrorMessage('Enter a question of up to 100 characters in length.');
$validate = new Zend_Validate_Db_NoRecordExists(array('table' => 'faqs', 'field' => 'title'));
$validate->getSelect()->where('FaqCategoryId = ?', $this->getValue('category'), Zend_Db::PARAM_INT);
$question->addValidator($validate);
}

I thought that this would add the extra where clause to the select statement for the validation. This failed miserably though. The problem is that the value referenced in ‘$this->getValue(‘category’)’  doesn’t exist when the init() method is called. Values are only added to the form when the isValid() method is called. The solution was to extend the isValid() method and to perform the extra check there.

public function isValid($data) {
	$result = parent::isValid($data);
	if ($result) {
		$validate = new Zend_Validate_Db_NoRecordExists(array('table' => 'faqs', 'field' => 'question'));
		//Add a where clause to check the FaqCategoryId
	        $validate->getSelect()->where('FaqCategoryId = ?', $this->getValue('category'), Zend_Db::PARAM_INT);
	       	$result = $validate->isValid($this->getValue('question'));
	       	if (! $result)
	       		$this->getElement('question')->setErrors(array('The selected category already has an FAQ with that question. Please edit the question.'));
		}
		return $result;
	}

By extending the isValid() method I can call the parent method to do the majority of the work. If the form looks valid I then perform the extra check to make sure the combination of question and category is unique. If it isn’t an error message is set and the validation fails. I’ve also used this in another part of the site where the client is adding news articles to the site. In that instance the combination of published date and title needs to be unique as these are used in the url to uniquely reference the article. The same method in form validation checks that when the form is submitted. Using this method I’ve been able to clean up my controller and move all validation checks into the form, where they should be. I’d love to hear in the comments how other people approach this problem though.

We’re hoping to have a discussion at the first meeting about the format people would like the meetings to take and what they would like to get out of it. We’re also hoping to have an ‘open mic’ event at each meeting where people can get up and talk about any programming problems or issues they’re having or to simply ask for feedback on a site they’re working on. There will also be plenty of time for socialising.

A big thank you goes out to our sponsors, Well.ca, for providing the venue and other support for the event. Drop us a line if you have any ideas or questions for the group and if you want to attend let us know by registering on Guestlistapp. Hope to see as many of you as possible on September 28th!

I’m building an app where I need to encrypt files uploaded by a user to add an extra security layer. I was initially thinking of using stream_filter_register() to create my own stream filter as the files are read and written. If you’re not familiar with the concept of stream filters in PHP they’re a very powerful feature. By attaching a filter to a stream you can perform various operations to data as it is being read from or written to a stream. Once the filter is defined and attached to the stream this is done completely transparently. Anyway, coming back to my problem of encrypting files, I did a quick search on Google and there didn’t seem to be an easy way of doing this. I then came across this gem in the PHP manual. PHP has a number of built in stream filters and one of them is an encryption filter. Providing you have the mcrypt extension installed encrypting and decrypting files is as easy as registering a stream filter on a stream! Pasted below is the example code from the PHP manual.

Here’s how to encrypt data as it’s written to a file:

<?php

$passphrase = 'My secret';

/* Turn a human readable passphrase

* into a reproducable iv/key pair

*/

$iv = substr(md5('iv'.$passphrase, true), 0, 8);

$key = substr(md5('pass1'.$passphrase, true) .

md5('pass2'.$passphrase, true), 0, 24);

$opts = array('iv'=>$iv, 'key'=>$key);

$fp = fopen('secret-file.enc', 'wb');

stream_filter_append($fp, 'mcrypt.tripledes', STREAM_FILTER_WRITE, $opts);

fwrite($fp, 'Secret secret secret data');

fclose($fp);
?>

and here’s how to decrypt data read from a file:

<?php

$passphrase = 'My secret';

/* Turn a human readable passphrase

* into a reproducable iv/key pair

*/

$iv = substr(md5('iv'.$passphrase, true), 0, 8);

$key = substr(md5('pass1'.$passphrase, true) .

md5('pass2'.$passphrase, true), 0, 24);

$opts = array('iv'=>$iv, 'key'=>$key);

$fp = fopen('secret-file.enc', 'rb');

stream_filter_append($fp, 'mdecrypt.tripledes', STREAM_FILTER_READ, $opts);

$data = rtrim(stream_get_contents($fp));

fclose($fp);echo $data<code>;</code>

?>

It couldn’t be easier! The example in the manual uses tripledes but you can use any of the cyphers and modes available in mcrypt.

The next step for me is to put together an example for my application. I want to store the uploaded files in a database so I need to open the file for reading from its location in $_FILES, adding an encryption filter to the stream, and bind it as a LOB parameter to a PDO insert prepared statement. To read the information back out I need to bind the decryption stream filter to the stream returned from PDO on a select operation. Bearing in mind the issues that PDO/MySQL has with returning streams from LOB fields I’m hoping this should be fairly easy. Hopefully this post will help someone looking to do this task.

Pre-Conference Party and Day 1

The conference unofficially opened with a pre-conference party on Tuesday evening. I spent much of the evening talking with Frédéric Hardy and Quentin Adam, both of whom had come from France to speak at the conference. Not for the last time at the conference I felt guilty for making French speakers speak in English so that I could join in the conversation! Frédéric has been working on an alternative to PHPUnit called Atoum and we spent some time discussing the benefits he is hoping it will provide over PHPUnit. I’m looking forward to writing some tests using it when I have a chance and seeing how it is.

Day one for me was dominated by the YellowAPI ConFoo Hackfest. The Yellow Pages in Canada is releasing an API that allows developers free access to all 1.5 million business listings they have in Canada. Each business is even geo-located with latitude and longitude to enable building geo-localised applications. As one of the sponsors of ConFoo they put together a hack day with prizes to allow developers to try out the API. While I didn’t win anything I spent the day putting together a very rough proof of concept for an app that I really think could be useful. It worked and there was a lot of interest in my idea from the YellowAPI people. I’m planning on spending some time to write the app properly when I get some time (probably in a few months) and I’ll write some more about it then. The standard was very high and the winning team deserved to win for their pub crawl application. After the hack was over I still had time to hear Andre Zmievski talk about Elastic Search. This seems to be a very interesting technology and one I’ll definitely keep in mind for the future.

Day 2

In many ways Day 2 was my first ‘proper’ day at the conference as I attended talks all day. The day had three real highlights for me: Michelangelo van Dam’s talk on ‘Improving QA on PHP development projects’, Antonio Fontes on ‘Threat Modelling: detecting threats before coding’ and the conference keynote given by Christian Heilmann on ‘HTML5-Moving from hacks to solutions’.

Michelangelo’s talk introduced several new tools to me that help to ensure code quality in a project that I’m very much looking forward to integrating into my workflow. The first of these is PHP Copy Paste Detector by Sebastian Bergmann. As it’s name suggests it will analyse code looking for places where it is duplicated. This can indicate places where the duplication should be abstracted out into a function or method. PHP Code Sniffer can be used to analyse a code base to ensure that a pre-defined coding standard has been adhered to and PHP Depend can be used to help analyse the overall quality of the code. PHP Mess Detector is another tool that can look at a code base and helpt to highlight potential bugs and problems. Michelangelo wrapped up with talking about using Phing as a build tool and Phar to package up code libraries for use in production. A really great and thought provoking talk.

Antonio Fontes introduced an entirely new topic for me, Threat Analysis and Modelling. This approach aims to help look at the security risks in  building a web application before a single line of code is written. It does this by having one or more people describe the application that is to be created along with how critical the system is from a business perspective. It then moves on to model the various risks that could occur from having the application fail or hacked, the likely types of people who might try to hack it and techniques that can mitigate these risks. This talk was a real eye opener for me and outlined a technique that I’m very keen to use in the future.

The keynote, delivered by Christian Heilmann of Mozilla, was one of the most inspiring talks of the whole conference for me. His central theme was that we have lost our excitement for new technologies and the wonders that it can offer, becoming somewhat blase about the opportunities that we have available. He presented a passionate and often funny exhortation to all of us to go and create great things using the new possibilities that HTML5 and its related API’s offer us, allowing these to degrade gracefully for older browsers. A really inspiring session.

Day 3

The highlights of the final day of the conference for me were Sebastian Bergmann’s session on ‘Your tests are lying!’ and Alia Alshanetsky’s talk on ‘Hidden features of PHP’. I also very much enjoyed hearing Arne Blankerts presenting on a new documentation generating tool he is writing which is currently in pre-alpha.

Sebastian Bergmann’s session discussed some of the problems associated with writing good unit tests. It was illustrated with samples he has seen (edited to protect the guilty!) and provided much practical advice about writing good tests. Alia Alshanetsky’s session was a great whistlestop tour through some of the lesser known features of PHP. These ranged from underused parts of the language to new features in PHP 5.3.  I made many notes in this session and took away a great many ideas to try.

I really enjoyed the conference and was very impressed with the whole thing. What really struck me was the way that developers from all different languages and areas came together to attend and share ideas with each other. While it’s so easy to get on our soap boxes and proclaim our chosen language as ‘the best’ there was none of that at ConFoo, which I found very refreshing. The conference was also a great networking opportunity and I made some new friends in the process. I’d really like to thank the organisers for putting on such a great event and my employer for recognising the value in having me attend. I’m already looking forward to ConFoo 2012.

What I Was Trying to Do…

I’m currently adding a clearance item section to a website. This will aggregate items from over 35 stores across Canada and allow a user to search for items in a number of ways. As part of this I’ve created a business object to represent a clearance item. This mainly consists of protected properties with accessors and mutators for setting and getting the properties. Since the site is in English and French the accessors allow the properties to be formatted accordingly before being returned. I use this clearance item class when importing information into the database (items are imported from Excel worksheets, but that’s a topic for another post) and when returning results from searches. I use data mappers to handle the CRUD operations in the database. What I wanted to do was to find a quick and easy way to transform a row from a resultset into an instance of a clearance item object without using a full ORM solution. I thought I had found this with PDO.

The ‘Solution’

PDO allows you to set the fetch mode to fetch an object of a user defined class for each row returned from the database. You can either use PDOStatement::fetchObject() or PDOStatement::setFetchMode() to define this behaviour. It will take the names of the columns and assume that these are properties of the object, setting the appropriate values for these properties. In theory this sounds great but unfortunately it’s not as easy as it sounds.

The Problem

The problem in my case is that all properties of my object are declared as protected and I use mutators to set them. I anticipated this and provided a __set() method to provide the needed property overloading. I thought that what would happen is that PDO would attempt to set an undefined public property, triggering the __set() method where there is logic to call the appropriate mutator. Unfortunately it doesn’t work like that at all. According to comments in the PHP manual it seems that PDO uses something called reflection injection to set the properties inside the object regardless of their visibility. This meant that I was getting back an instance of my clearance item class with the protected properties set directly by PDO. This may not seem like too much of a problem, but my mutators include logic to transform the properties being set to the correct data type. For instance, columns containing integers or floating point numbers come back as strings and the mutators make sure they are set as numeric data types. Worst of all for me was the information on stores. I’m using MySQL’s GROUP_CONCAT() function to ‘implode’ information about stores and the inventory in each into a string, meaning that I can fetch all of the information I need in a single query. The logic in the mutator then splits that string apart into an array of information on stores and the inventory. PDO was bypassing this completely and setting the protected stores property as the string that came back from GROUP_CONCAT().

A Partial Solution

Once again the PHP manual was my partial saviour. There is a way to fetch a row from a resultset into an existing instance of a class. In this case the visibility of properties is honoured and the __set() function is called, in turn triggering the correct mutators. The code I came up with looks like this:

//The statement is prepared before this.

$item = new ClearanceItem($this->lang);
 $st->setFetchMode(PDO::FETCH_INTO, $item);
 $st->bindParam(':ItemId', $id, PDO::PARAM_INT);
 $st->execute();
 foreach ($st as $result) {
//The query in this case only returns a single row.

return $result;
 }

While this works it’s a bit of a hack to say the least. It seems that PDOStatement::fetch() doesn’t work here to fetch the info into the object, necessitating the foreach loop and the immediate return. This is fine in this case but in other areas of the code I need to iterate over a resultset, getting a new instance of ClearanceItem each time. This wouldn’t work as the information is populated into the existing object bound to the statement. The best I could do is to clone the object each time through the loop and store the cloned copy.

There is another way out of this: return each row as an associative array and create a factory method to return instances of ClearanceItem from this. I actually coded this solution already when I intitially found the strangeness that PDO was doing in return instances of ClearanceItem. That solution works perfectly well but it seems strange that it’s not possible to loop over a record set, getting a new instance of a class each time.

Conclusion

It seems that it’s quite possible to return instances of a class from a PDOStatement providing that all of the properties are public or if you don’t mind PDO directly setting your private and protected properties. If you want to use mutators to control how your data is set and to make sure it’s of the correct type it seems that the options are far more limited and you may be out of luck. Of course there are other options, such as writing a factory method to create an object from an array or using a full blown ORM like Doctrine. In this instance, as I haven’t used Doctrine before and as time is of the essence I was hoping that PDO would give me what I need.

It seems very strange that PDO is allowed to directly set private and protected properties when all other code accessing instances of ClearanceItem have to use the accessors and mutators. It seems to me to drive a coach and horses through the concept of property visibility. Why should PDO (at least I’m assuming it’s just PDO) have the ability to bypass the visibility protections that OO code provides while everyone else has to work with it? Is there some good reason for doing this that I’m missing? I’d be really interested to hear what the possible advantages of this are.