Archive for the Category Tutorials

Some Doctrine 2 Best Practices

Doctrine 2 has a very powerful database abstraction layer and features an Object Relational Mapper (ORM) that makes it very easy to manage relationships between the data in your application and provides tools that aid rapid application development (RAD).

With Doctrine also comes DQL, a powerful language for querying your object models,  which is similar to SQL.

Use of Doctrine abstracts away the database concepts of tables and columns, rather you think in terms of objects, entities, properties and relationships. While this is very useful, it is very important for the developer to have an understanding of the underlying SQL that is generated from using the entity manager/DQL and querying your objects/entities.

Without an understanding of the code that Doctrine generates, it is relatively easy to build an inefficient and slow application, here are a few pitfalls that I have found are easy to fall into, and some tips that may be useful for people starting with Doctrine, or if you are trying to get a little more performance out of an existing application.

For the rest of this post I will use the cookie cutter example of a Blog website with article, user and comment object models (entities). In terms of object mapping we will assume all mapping is specified with the default (lazy) fetch method, and MySQL as our DB.

1. Always write a DQL statement for querying your object models

Doctrine repositories provide some ready made methods for fetching your objects such as “->findAll()” and “->findBy()”. These provide some convenience and brevity when initially developing your application but use of these functions is less flexible when further developing your application and can lead to some of the pitfalls below.

By writing the DQL yourself, you will expand your understanding of Doctrine, you have more control of the data you are trying to query and people reading your code will have a better understanding of what you are trying to do (and will find it easier to update and extend your code).

2. Beware of lazy loading when querying entities with associations

If you do not specify an ‘eager’ fetch in your object mapping, or do not explicitly fetch your relationships in your DQL statements (or if you query your objects using the Doctrine methods mentioned in point 1) Doctrine will not fetch those associations in the SQL it generates. This is helpful as you do not want to be loading resources you do not need, however when you call a related object, Doctrine needs a method of fetching these objects and it does this by lazy loading through proxies.

For example if you have an article entity, with associated comments and you want to display a list of all your articles with number of comments on the front page of your blog, you may do something like this (this code is purely for demonstration).

$articles = $article_repository->findAll();

foreach($articles as $article)
{
    echo $article->getTitle();
    echo count($article->getComments());
}

Using this code, Doctrine will initially generate a query for all the rows in the article table. As we have not ‘fetched’ the comment association, Doctrine will lazy load in the comment object for each article via an additional SQL statement. For a table containing 10 rows, this will result in 11 database queries.

//$em is the entity manager
$qb = $em->createQueryBuilder();

$qb->select('Article', 'Comment')
    ->from('Entity\Article', 'Article')
    ->leftJoin('Article.comment', 'Comment');

$query = $qb->getQuery();

$result = $query->getResult();

return $result;

Instead, by using DQL and specifying our relationships, we can use the above code to fetch the same data with 1 query vs the  11 (unnecessary) database queries generated previously. This problem grows with each additional row in a given database.

3. Use array hydration for read only actions

The Doctrine documentation recommends the use of array hydration for read only actions, this is in part due to the extra overhead (increased memory needed from loading proxies and the like) associated with hydrating objects.

If you are just displaying lists of items, or lists of entity properties for a single item (for example a product page, a list of recent blog articles, or indeed most front end operations!) you can use:

$query->getResult(Query::HYDRATE_ARRAY);

This approach can also prevent any accidental lazy-loading problems that may arise as discussed previously (due to the lack of proxies!)

4. By default Doctrine will fetch all of the properties for a given entity

This doesn’t exactly fall under a best practice, indeed making any changes here too early in the development cycle would fall under premature optimisation. Nonetheless consider the following:

$qb->select('Article')
->from('Entity\Article');

It is somewhat wasteful in terms of resources to fetch data from your database that you aren’t going to use. The above code snippet would fetch all the properties for the article entity.

Rather than selecting the whole object, in DQL you can specify specific properties to return in your select statement:

$qb->select('Article.title')
->from('Entity\Article');

However this will always return a flat array, regardless of relationships between your entities. Instead you can select and hydrate partial objects in your DQL, which will give you an object or a nested array representing your object graph (depending on the hydration selected)

$query = $em->createQuery("SELECT partial Article.{title} Entity\Article Article");

As stated in the Doctrine documents it is important to note here that this optimisation can lead to more fragile code, it may not always be clear (especially if working as part of a team, and working code written by others) that object properties have not been fetched.

5. Use prepared statements

Using prepared statements helps to prevent SQL injection attacks that have proven problematic for developers and businesses alike – Sony notably exposed a lot of user data in Summer 2011. See below for a quick example of how to use prepared statements in Doctrine.

$qb = $em->createQueryBuilder();

$qb->select('Article', 'Comment')
    ->from('Entity\Article', 'Article')
    ->leftJoin('Article.comment', 'Comment')
    ->where($qb->expr()->eq('Article.title', ':filter_title')
    ->setParameter('filter_title', $some_user_input);

$query = $qb->getQuery();

$result = $query->getResult();

return $result;

The Doctrine documentation provides a more in depth explanation for achieving this.

6. Be aware of what Doctrine is up to

If you are using Symfony 2, you can use the in-built profiler for monitoring the number of queries generated by doctrine. If you are using Zend Framework you can refer to my previous blog post covering some methods for profiling doctrine 1 or 2.

 

I hope these tips prove helpful if you are just getting started with Doctrine, or if you have been using it for a little while. If you have any tips of your own let us know on twitter @uv_d or leave a comment

Implementing Barclaycard’s ePDQ CPI in the Zend Framework

Barclaycard's ePDQ + ZendWe are working on an e-commerce project at the moment (built in Zend), which at it’s commercial core, involves selling digital products to consumers. Our client is using Barclaycard’s ePDQ Cardholder Payment Interface (CPI) to take secure payments from their customers.

ePDQ is well documented by Barclaycard and we had no trouble designing the transaction process for our client. However, when it came to the actual Zend development I was surprised not to be able to locate any existing Zend or even PHP classes online, over and above the simple CURL examples Barclaycard provide in their documentation. Subsequently, when designing the technical ePDQ integration I decided to put my reusability hat on and looked to write a set of Zend assets which we could add to our ever growing UVd Zend library.

The following article outlines how I developed these assets in the context of the ePDQ process. I will make the code available in uvd Labs as well.

User Flow

The diagram below illustrates the user flow when interacting with ePDQ from a generic application.

ePDQ User Flow

  1. User is presented with a checkout screen in the application, which along with an order summary includes a ‘Continue to payment’ button. When pressed, the user is redirected to the first of two ePDQ CPI screens (CPI 1) to securely input their billing information
  2. User inputs their details and presses ‘Submit’
  3. User sees result of validation on CPI 2 and is presented with a ‘Continue’ button
  4. When pressed, the user is redirected back to the application where they continue with the sales process accordingly

Application Flow

The application logic required to implement the user flow is prescribed by the ePDQ process itself.

ePDQ Application Flow

  1. To access CPI 1, an HTTP POST request must be made which contains a strict set of variables. These are both supplier specific (e.g. the application’s merchant ID) and transaction specific (i.e. order number or total value). These must be sent to the CPI encrypted, so before the checkout screen is outputted to the user, a server side call must be made to ePDQ to encrypt the details. The configuration of the CPI, (including client Logo, whether delivery address is required, currency, pre-populated fields etc) is all based on the data supplied in this POST request; the most straight forward way to implement this being an HTML form.
  2. Once the user enters their details and presses submit, two things happen. The CPI performs its verification on the transaction itself. The CPI performs a callback to the application via a POST request to a specific script. This URL must be the same for every transaction (it can’t contain OID for example) and must be protected behind the HTTP Basic authentication mechanism. It also must NOT require a port number to be specified (as I found out the hard way). The callback body contains the results of the user’s credit card verification and allows the application to perform the necessary processing such as write to a log file or database. The user will not be presented with CPI 2 until ePDQ has received a response from the application, so it is preferable to perform a minimum amount of processing here.
  3. Once the user presses continue on CPI 2, they will be redirected to a URL on the application (this URL can be transaction specific). As far as ePDQ is concerned the transaction is now complete, so the application can handle the user’s return based on the results of the callback function.

Implementing ePDQ in Zend

Based on the application and user flows of the ePDQ process, what areas lend themselves to generating reusable assets? Its clear that anything application specific such as models (i.e. sales orders or products), views (such as the checkout) should be left to the specific application to implement. Similarly, once a user is directed back to the application, any ePDQ involvement is now over so there wouldn’t be much in terms of reusable code at (3) above. However, the initial encryption request and the callback are specifically controlled by the ePDQ process, making them good candidates.

So, in terms of building reusable assets in a Zend application, what options do you have?

  • Controllers
  • Modules
  • Library Classes
  • Services
  • Action Helpers
  • View Helpers

From my experience of building assets in Zend you want something that:

  • is fairly self contained (whereas controllers perhaps are not so good, modules are self contained)
  • suits all types of implementations (our Services provide our Doctrine integration so wouldn’t suit all applications)
  • are easy to make application agnostic (plugins and classes are)
  • can be developed with caching in mind

Having digested the technical documentation from Barclaycard and after thinking long and hard about Zend, I decided the best approach would be to develop:

  1. A library class – UVd_Commerce_Epdq_Service (Epdq_Service)
  2. A plugin – UVd_Commerce_Epdq_Plugin (Epdq_Plugin)

Zend Application Integration

The following diagram outlines how these assets integrate into a Zend implementation of the ePDQ process.
ePDQ Zend Integration

1. Generating the checkout view

In order to render the checkout view, the checkout controller instantiates Epdq_Service. This collects application specific variables (merchant Id) from an epdq .ini file and all transaction specific variables are passed by the controller. Epdq_Service makes the server side HTTP request (I used Zend_Http_Request, using CURL). Once this is received the Checkout controller can call htmlify() on the object and an html form will be generated.

2. Processing the callback

ePDQ needs the URL of the callback function (which you configure in the ePDQ admin interface). This URL is stored in the epdq.ini as a collection of module, controller and action names, however the actual controller does not need to exist in your application. Epdq_Plugin looks at the ini file and intercepts any calls made to this combination; this is checked using the dispatchLoopStartup hook. The plugin provides the HTTP authentication mechanism and then calls Epdq_Service to perform basic processing on the results, which is a simple log to a text file. Each application will want to process this result differently so the advantage of this approach is that all application specific code can be put into an actual controller with the same name.

3. User exits ePDQ

The user will then press continue and will be redirected to a URL on your application that is transaction specific. It makes sense to include the OID in here so the application knows how to continue with the user flow. I am also storing this in a session to prevent URL tampering.
I hope this has proven a useful resource for anyone using ePDQ with Zend. I will also make my final code available in uvd Labs so look out for that!

Importing a swc into Flex Builder for pure actionscript projects

I have previously written a post about the potential bug which seems to occur when importing multiple swcs into Flex Builder 3 for pure actionscript projects. Despite this I cannot express how handy it is to be able to import swcs into Flex Builder; for one reason because it renders the Flash IDE even more redundant (which is a good thing).

Whilst talking to a friend who has recently started using Flex Builder for AS projects I discovered that he hadn’t as yet used the method of importing swcs to gain access to graphics, sounds, animations, etc which have been created in Flash IDE.

So far he has been using the Loader class to import the swf at runtime and then to gain access to the individual items in the fla/swf library. There are obvious advantages to this method, for example loading the swf at runtime avoids the increased file size of your project which you will incur from importing the swc which will add the graphics/sounds at compile time.

However for smaller projects where file size isn’t an issue or for importing the fundamental graphical elements which will be re-used throughout the project a swc is a quick and easy solution.

Firstly when creating you or a designer is creating the graphical elements in the flash library you can also add a class path to each movieclip or asset in the library.

1) Right click on the assets in the library and click Linkage…..

2) Tick the checkbox marked Export for Actionscript

3) Enter a class path which relates to the path structure of your application into the Class text field e.g. uk.co.ultravioletdesign.graphics.ThisAsset - nb If you just put a simply identifier/class name without a full class/package path (e.g. ThisAsset) the item will be available throughout your application which sounds useful but is the same as making classes public which if avoidable is good practice and good for application performance.

This also enables you to organise your assets in the Flash library into convenient packages. For example putting all sound assets in uk.co.ultravioletdesign.assets.sounds and all graphics into uk.co.ultravioletdesign.assets.graphics. You don’t need to worry about creating the equivalent folders and packages within Flex Builder because they will be discreetly generated when the swc is imported.

4) Go to Publish Settings (File > Publish Settings) and then click the Flash tab. Providing the Fla is AS3 the checkbox Export SWC will be available. Check it!

5) Now once the movie is published a .swc file will be snuggling up next to your .swf which you can promptly delete (if you feel the need).

In Flex Builder:

6) Right click on your Actionscript project in the Flex Navigator window and click properties. Click on Actionscript Build Path and then on the Library path tab.

7) Click on the Add SWC button and browse to your newly generated .swc file.

Flex now adds your library assets to your project and these are made available within classes using the import directive:

import uk.co.ultravioletdesign.graphics.ThisAsset;

var thisAsset : ThisAsset = new ThisAsset();

Using this method in my experience has been invaluable and a massive timesaver. Despite this it is very useful to have the knowledge to be able to import assets from swfs at runtime. There is very handy for other strategies and techniques such as the importing fonts at runtime. This article by Chris Rebstock has been very valuable.

Tracking with Google Analytics from Flash

Just a quick update on the Google Analytics work I have been doing in-amongst writing a desktop application to detect proxy settings (maybe more on that later).

I have completed the move from development to live server and have implemented some code for Flash banners to call the Google Analytics JavaScript _trackPage function. Its really simple stuff but as I have done 2 examples (one in Actionscript 2 and one in Actionscript 3) for the client to be able to supply the creative departments of advertisers (as they usually create the banners and supply .gif or in this case .swf files) I thought I might as well post the examples. For those people trying to find there way with Actionscript 3, a button click action is now triggered by an Event (MouseEvent.CLICK) and interestingly a call to some JavaScript in the page that embeds the Flash file is now called using the ExternalInterface class (it won’t work with the old method of calling the function from the getURL method). For example:

ExternalInterface.call("pageTracker._trackPageview('/clks/adv/insert-name-here')");

Replaces:

getURL("javascript:pageTracker._trackPageview('/clks/adv/insert-name-here');");

Anyways, here are the examples for download.

It’s basic but it might help someone :-)

Search