Domain-Driven Design: Data Access Strategies

Part 1: Domain-Driven Design and MVC Architectures

The Domain Model

Here are some of the features a Domain-driven design framework should support:

  • A domain model that is independent and decoupled from the application.
  • A reusable library that can be used in many different domain-specific applications.
  • Dependency Injection in order to inject Repositories and Services into Domain Objects.
  • Integration with unit testing frameworks, such as PHPUnit.
  • Good integration with other frameworks, such as Zend, Symfony, Doctrine, etc.

The Zend Framework, for example, is part of the infrastructure layer and acts as a supporting library for all the other layers. However, the domain layer should be well isolated from the other layers of the application and should not be dependent on the framework you are using.

Data Access Strategies

When accessing data from a data source, you have to decide how your application will communicate with a data source. Each data access strategy has its own advantages and disadvantages. Here are some design patterns that support DDD:

Generic DAO’s

Data Access Objects (DAO’s) and Repositories play an important role in DDD. The goal of a DAO is to abstract and encapsulate all access to the data and provide an interface. The DAO always connects, reads and saves data to a data source. From the applications point of view, it makes no difference when it accesses a database, XML file or Web service:

$user = new UserDbDao();
$row = $user->findUserById(456);
echo $row->name;

$user = new UserXmlDao();
$row = $user->findUserById(456);
echo $row->name;

Table Data Gateway

An object that acts as a Gateway to a database table. One instance handles all the rows in the table. The Zend_Db_Table solution is an implementation of the Table Data Gateway pattern:

$user = new User();
$rows = $user->find(456);
$row = $rows->current();
echo $row->name;

More info: Table Data Gateway

Row Data Gateway

An object that acts as a Gateway to a single record in a data source. There is one instance per row. Zend_Db_Table_Row is an implementation of the Row Data Gateway pattern:

$user = new User();
$row = $user->fetchRow($user->select()->where('user_id = ?', 1));
echo $row->name;

More info: Row Data Gateway

Active Record

An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data. The Mad_Model component is an  ORM layer that follows the Active Record pattern where tables map to classes, rows map to objects, and columns to object attributes:

$user = new User();
$userFinder = new UserFinder();
$row = $userFinder->find(456);

More info: Active Record

Data Mapper

A layer of Mappers that moves data between objects and a database while keeping them independent of each other and the mapper itself:

$user = new User();
$userMapper = new UserMapper();
$row = $userMapper->findByEmail($email);

More info: Data Mapper

Data Transfer Objects

A Data Transfer Object (DTO) in Domain-driven design is not the same as a Value Object. A DTO is often used in combination with a DAO to encapsulate data. It allows you to reduce communication effort when dealing with a lot of small data entities. For example:

class UserData
{
    public function setId($id) {}
    public function getId() {}
    public function setFirstName($firstName) {}
    public function getFirstName() {}
    public function setLastName($lastName) {}
    public function getLastName() {}
}

$user = new UserData();
$user->setFirstName('Jim');
$user->setLastName('Morrison');

$userDao = new UserDbDao();
$userDao->insert($user);

A DTO is a simple container for a set of aggregated data. It should contain no business logic and limit its behaviour to activities such as internal consistency checking and basic validation. Be careful not to make the DTO depend on any new classes as a result of implementing these methods.

Part 3: Domain-Driven Design: The Repository

Domain-Driven Design and MVC Architectures

According to Eric Evans, Domain-driven design (DDD) is not a technology or a methodology. It’s a different way of thinking about how to organize your applications and structure your code. This way of thinking complements very well the popular MVC architecture. The domain model provides a structural view of the system. Most of the time, applications don’t change, what changes is the domain. MVC, however, doesn’t really tell you how your model should be structured. That’s why some frameworks don’t force you to use a specific model structure, instead, they let your model evolve as your knowledge and expertise grows.

Domain-driven design separates the model layer “M” of MVC into an application, domain and infrastructure layer. The infrastructure layer is used to retrieve and store data. The domain layer is where the business knowledge or expertise is. The application layer is responsible for coordinating the infrastructure and domain layers to make a useful application. Typically, it would use the infrastructure to obtain the data, consult the domain to see what should be done, and then use the infrastructure again to achieve the results. Srini Penchikala explains this in more detail here: “Domain Driven Design and Development In Practice“.

Object-oriented programming is the most important element in the domain implementation. Domain objects are designed using classes and interfaces, and take advantage of OOP concepts like inheritance, encapsulation, and polymorphism. Most of the domain elements are true objects with both State (attributes) and Behaviour (methods or operations that act on the state). Entities and Value Objects in DDD are classic examples of OOP concepts since they have both state and behaviour.

Terminology used by Domain-driven design

  • Entity: An object which has a distinct identity. For example, a User entity has a distinct identity with an ID.
  • Value Object: An object which has no distinct identity, like numbers and dates. For example, if two Users have the same date of birth, you can have multiple copies of an object that represents the date 16 Jan 1982.
  • Factory: Used to create Entities. For example, you can use a Factory to create a Profile entity from a User entity.
  • Repository: Used to store, retrieve and delete domain objects from different storage implementations. For example, you can use a Repository to store the Profile your Factory created.
  • Service: When an operation does not conceptually belong to any object.

The purpose of this post was to provide a brief introduction to Domain-driven design.

Part 2: Domain-Driven Design: Data Access Strategies

Links

If you’re interested in learning more about Domain-driven design or Model-driven design, I recommend the following articles:

Zend Framework: The Cost of Flexibility is Complexity

The Zend Framework is a very flexible system, in fact, it can be used to build practically anything. But, the problem with using a flexible system is that flexibility costs. And the cost of flexibility is complexity.

Martin Fowler wrote:

Every time you put extra stuff into your code to make it more flexible, you are usually adding more complexity. If your guess about the flexibility needs of your software is correct, then you are ahead of the game. You’ve gained. But if you get it wrong, you’ve only added complexity that makes it more difficult to change your software.

Don’t assume that just because you’re using an object-oriented framework you are writing reusable and maintainable code. You are just structuring your spaghetti code. As Paul Graham put it: “Object-oriented programming offers a sustainable way to write spaghetti code”. The main problem with flexibility is that most developers give up trying to understand. I don’t blame them, no one likes dealing with complexity. However, flexibility is sometimes required and generally always desirable. But, sometimes we forget that frameworks exist to help us solve or address complex issues, not to fix bad design decisions for us. So if your code is difficult to understand or unit test, then you are probably doing something wrong.

Just because you can, doesn’t mean you should

What’s the best way to render a view script in the Zend Framework? Think, don’t give up yet. Is it enabling or disabling the ViewRenderer action helper? Remember, the framework allows you to shape the Zend_Controller_Action class to your application’s needs.

The manual clearly states that:

By default, the front controller enables the ViewRenderer action helper. This helper takes care of injecting the view object into the controller, as well as automatically rendering views.

However, no one stops you from doing it manually:

Zend_Controller_Action provides a rudimentary and flexible mechanism for view integration. Two methods accomplish this, initView() and render(); the former method lazy-loads the $view public property, and the latter renders a view based on the current requested action, using the directory hierarchy to determine the script path.

Developer A

class IndexController extends Zend_Controller_Action
{
    public function searchAction()
    {
        $query = $this->_getParam('q', null);
        if (null === $query) {
            $this->view->error = 'Invalid query';
            return;
        }

        $this->view->query = $query;
        if (is_numeric($query)) {
            $this->setUserDetailsById($query);
        } else {
            $this->setUserDetailsByName($query);
        }
    }

    public function setUserDetailsByName($name)
    {
        $row = $this->findUserByName($name);
        $this->view->userId = $row['id'];
        $this->view->userName = $row['name'];
    }

    public function setUserDetailsById($id)
    {
        $row = $this->findUserById($id);
        $this->view->userId = $row['id'];
        $this->view->userName = $row['name'];
    }
}

Developer B

class IndexController extends Zend_Controller_Action
{
    protected $_viewParams = array(
        'error'    => null,
        'query'    => null,
        'userId'   => null,
        'userName' => null,
    );

    public function init()
    {
        $this->_helper->removeHelper('viewRenderer');
    }

    public function searchAction()
    {
        $query = $this->_getParam('q', null);
        if (null === $query) {
            $this->_viewParams['error'] = 'Invalid query';
            $this->renderView('search');
            return false;
        }

        return $this->displayUserDetails($query);
    }

    public function displayUserDetails($query)
    {
        if (is_numeric($query)) {
            $row = $this->getUserDetailsById($query);
        } else {
            $row = $this->getUserDetailsByName($query);
        }

        if (!$row) {
            $this->_viewParams['error'] = 'User not found';
            $this->renderView('search');
            return false;
        }

        foreach ($this->_viewParams as $key => $value) {
            if (!array_key_exists($key, $row) {
                continue;
            }
            $this->_viewParams[$key] = $row[$key];
        }

        $this->_viewParams['query'] = $query;
        $this->renderView('search');

        return true;
    }

    public function renderView($template)
    {
        $view = $this->initView();
        foreach ($this->_viewParams as $key => $value) {
            $view->$key = $value;
        }
        $this->render($template);
    }

    public function getUserDetailsByName($name)
    {
        return $this->findUserByName($name);
    }

    public function getUserDetailsById($id)
    {
        return $this->findUserById($id);
    }
}

Same problem, different solutions.

Now, guess who is doing test-driven development, has all his code unit tested, puts commonly used code into libraries, abstracts out common patterns of behaviour, writes his code using the top-down fashion and uses the principle of “least astonishment”.

Developer A or B?

Conclusion

Many requirements lie in the future and are unknowable at the time our application is designed and built. To avoid burdensome maintenance costs developers must therefore rely on a system’s ability to change gracefully its flexibility. The combination of flexibility and smart design can reduce the maintenance burden. It’s up to us, the developers, to change our thinking when using a flexible framework. Like I said before, frameworks exist to help us solve complex issues, not to make decisions for us.

Apache Log Analyzer 2 Feed

I found this project thanks to Raphael’s post Turning a Zend_Log log file into an RSS feed.

Developed by Simone Carletti, ApacheLogAnalyzer2Feed is a really powerful open source PHP 5 class to parse and analyse Apache Web Server log files. Results are converted into a feed to let users subscribe them with a feed reader. You can define custom filters based on logs data — for instance User-Agent, IP, requested page, etc— and combine them to select just a limited resultset. The class can easily be extended with additional filters and custom feed handlers.

On a side note, Simone’s Wiki is powered by Redmine, an awesome Ruby on Rails Wiki and project management system that’s worth checking out.

Zend Framework Automatic Dependency Tracking

When you develop or deploy an application, dependency tracking is one of the problems you must solve. Keeping track of dependencies for every application you develop is not an easy task. To solve this problem I’ve created Zend_Debug_Include, a Zend Framework component that supports automatic dependency tracking.

We all agree that dependencies cannot be maintained by hand, it’s almost impossible, specially if you are using packages from Zend, Solar, Maintainable and/or Zym (if you don’t know any of these frameworks, check them out, they are pretty cool). Every time you add an include/require statement or create an instance of an object, you introduce a new dependency. And that’s why tracking internal project dependencies can become complex when using a framework.

The concept behind Zend_Debug_Include is that the dependencies for each source file are stored in a separate file. If the source file is modified, the file containing that source file’s dependencies is rebuilt. This concept enables you to determine run-time dependencies of files using arbitrary components. This solution is also useful if you are deploying your application using Linux packages. But, dependency tracking isn’t just useful for deploying applications, it can also be used to evaluate packages. Sometimes packages create unnecessary dependencies and this is something that we need to monitor.

Zend_Debug_Include comes with 3 built-in adapters: File, Package and Url.

Tracking File Dependencies

To track file dependencies you need to create an instance of Zend_Debug_Include_Manager and set the Zend_Debug_Include_Adapter_File adapter. This needs to happen inside your bootstrapper file, before the Front Controller dispatches the request.

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_File());
$included->setOutputDir('/var/www/my-app/dependencies');

/* Dispatch request */
$frontController->dispatch();

This creates a zf-files.txt in your output directory containing all the files included or required on that request. Every time the Front Controller dispatches a request, Zend_Debug_Include checks for new dependencies and adds them to the zf-files.txt file:

/var/www/my-app/public/index.php
/var/www/my-app/application/bootstrap.php
/var/www/my-app/library/Zend/Loader.php
/var/www/my-app/library/Zend/Controller/Front.php
/var/www/my-app/library/Zend/Controller/Action/HelperBroker.php
/var/www/my-app/library/Zend/Controller/Action/HelperBroker/PriorityStack.php
/var/www/my-app/library/Zend/Controller/Exception.php
/var/www/my-app/library/Zend/Exception.php
/var/www/my-app/library/Zend/Controller/Plugin/Broker.php
/var/www/my-app/library/Zend/Controller/Plugin/Abstract.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Standard.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Abstract.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Interface.php
/var/www/my-app/library/Zend/Controller/Request/Abstract.php
/var/www/my-app/library/Zend/Controller/Response/Abstract.php
/var/www/my-app/library/Zend/Controller/Router/Route.php
/var/www/my-app/library/Zend/Controller/Router/Route/Abstract.php
/var/www/my-app/library/Zend/Controller/Router/Route/Interface.php
/var/www/my-app/library/Zend/Config.php
... (63 files in total) ...

To change the name of the file:

$included = new Zend_Debug_Include_Manager();
$included->setOutputDir('/var/www/my-app/dependencies');
$included->setFilename('files.dep');
...

Tracking Package Dependencies

Similar to Zend_Debug_Include_Adapter_File, but groups all the files into packages.

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_Package());
$included->setOutputDir('/var/www/my-app/dependencies');

The code above creates a zf-packages.txt file and adds the following data:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php

Now, if you introduce a new dependency, for example, Zend_Mail:

class IndexController extends Zend_Controller_Action
{
    public function indexAction()
    {
        $mail = new Zend_Mail();
        $this->view->message  = 'test';
    }
}

The next time the Front Controller dispatches the request and calls the index action, Zend_Debug_Include will automatically add the Zend_Mail package and all its dependencies to the zf-packages.txt file:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php
Zend/Mail.php
Zend/Mail
Zend/Mime.php
Zend/Mime

You can then use this information to keep track of dependencies and tell your build tool the name of the files and directories you need to copy and package.

External Dependencies

Here is where everything starts to make sense. Zend_Debug_Include allows you to search for external dependencies as well, you just need to tell the Adapter the libraries you are using. For example:

$libraries = array('Zend', 'Solar');
$adapter = new Zend_Debug_Include_Adapter_Package($libraries);

$included = new Zend_Debug_Include_Manager();
$included->setAdapter($adapter);
$included->setOutputDir('/var/www/my-app/dependencies');

The Solar packages will also be added to the zf-packages.txt file:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php
Zend/Mail.php
Zend/Mail
Zend/Mime.php
Zend/Mime
Solar/Base.php
Solar/File.php
Solar/Factory.php
Solar/Sql.php
Solar/Sql
Solar/Cache.php
Solar/Cache

URL Adapter

If you want to create a different file for each request, use the URL adapter instead:

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_Url());
$included->setOutputDir('/var/www/my-app/dependencies');

The URL adapter maps the URL path to a filename. So, if you request the following URI:

http://my-app/blog/2009/02/01

It creates the file blog_2009_02_01.txt.

SVN

$ cd libraries/Zend
$ svn co http://svn.fedecarg.com/repo/Zend/Debug

The Multimaster Replication Problem

Replication has its problems, specially if you have a multimaster replication system. To make matters worse, none of the PHP frameworks support multimaster replication systems nor handle master failover. Symfony uses Propel and only supports master-slave replication systems. When the master fails, it’s true that you have the slaves ready to replace it, but the process of detecting the failure and acting upon it requires human intervention. Zend Framework, on the other hand, doesn’t support replication at all.

I strongly believe that a master failover needs to be handled appropriately on the application side. Of course, you can always use an SQL proxy or any other server-side solution, but they are either limited or unreliable.

From Digg’s Blog:

The Digg database access layer is written in PHP and lives at the level of the application server. Basically, when the application decides it needs to do a read query, it passes off the query with a descriptor to a method that grabs a list of servers for the database pool that can satisfy the query, then picks one at random, submits the query, and returns the results to the calling method.

If the server picked won’t respond in a very small amount of time, the code moves on to the next server in the list. So if MySQL is down on a database machine in one of the pools, the end-user of Digg doesn’t notice. This code is extremely robust and well-tested. We worry neither that shutting down MySQL on a read slave in the Digg cluster, nor a failure in alerting on a DB slave that dies will cause site degradation.

Every few months we consider using a SQL proxy to do this database pooling, failover, and balancing, but it’s a tough sell since the code is simple and works extremely well. Furthermore, the load balancing function itself is spread across all our Apaches. Hence there is no “single point of failure” as there would be if we had a single SQL proxy.

If you are building your own solution and need a sandbox to test it, I recommend using MySQL Sandbox. Also, you might find this script useful: MySQL Master-Master Replication Manager

Designing a CMS Architecture

François Zaninotto wrote:

When faced with the alternative between an off-the-shelf CMS or a custom development, many companies pick solutions like ezPublish or Drupal. In addition to being free, these CMS seem to fulfill all possible requirements. But while choosing an open-source solution is a great idea, going for a full-featured CMS may prove more expensive than designing and developing your own Custom Management System.

Given number of available open-source CMS solutions, building one on your own sounds like a stupid idea. But if your website is 50% content management and 50% something else, you probably need to start with a web application framework like Symfony or Django, rather than a CMS. These frameworks provide plugins that do part of the Content Management job already, so creating a CMS today is like assembling Lego bricks to build something that exactly fits your needs.

Continue reading

Where is the include coming from?

The includes of the system map out the dependencies of the system, which files depend on which, which subsystem depends on which. When working with a system, it’s always useful to map out the dependencies before hand.

Here are some examples:

WordPress 2.2.1
http://wordpress.org

MediaWiki 1.12
http://www.mediawiki.org/

phpBB 3.0
http://www.phpbb.com/

phpMyAdmin 2.9.1.1
http://www.phpmyadmin.net/

Symfony 1.1
http://www.symfony-project.org/

Zend Framework 1.5.2
http://framework.zend.com/

CakePHP
http://www.cakephp.org/

CodeIgniter
http://codeigniter.com/

Seagull Framework
http://seagullproject.org/
(Demian Turner)

Django 1.0 will be released in early September

Django has not been updated for a long time. The most current release, 0.96, was released in March 2007. This is a very long time when you look at the roadmap of other frameworks. But guess what, all that is about to change, because Django 1.0 is only two months away! Over the next six weeks sprints will take place all over the world to help get everything done by the deadline. This document details the schedule and roadmap towards Django 1.0. Although “Release early, release often” is not as easy as it seems, I hope it becomes part of Django’s development model.

The framework is now owned by a new, nonprofit, organization. The foundation, headed by Jacob Kaplan-Moss, will now be able to accept donations to help it grow. Finally we can say that Django is in the hands of the people who care most about its long-term success.

Django 1.0 Roadmap and Schedule

TDD with Symfony: The first test always fails

Symfony is one of the few PHP frameworks that gives you basic tools for starting to write tests. It provides a special object, called sfBrowser, which acts like a browser connected to an application without actually needing a server and without the slowdown of the HTTP transport. It gives access to the core objects of each request (the request, session, context, and response objects). Symfony also provides an extension of this class called sfTestBrowser, designed especially for functional tests, which has all the abilities of the sfBrowser object plus some smart assert methods.

For example, every time you generate a module skeleton Symfony creates a simple functional test for this module. The test makes a request to the default action of the module and checks the response status code, the module and action calculated by the routing system, and the presence of a certain sentence in the response content.

Browser

For a foobar module, the generated foobarActionsTest.php file looks like:

include(dirname(__FILE__).'/../../bootstrap/functional.php');

// Create a new test browser
$browser = new sfTestBrowser();
$browser->initialize();

$browser->
  get('/foobar/index')->
  isStatusCode(200)->
  isRequestParameter('module', 'foobar')->
  isRequestParameter('action', 'index')->
  checkResponseElement('body', '!/This is a temporary page/');

Console

And you can launch the functional test from the command line:

> symfony test-functional frontend foobarActions

# get /comment/index
ok 1 - status code is 200
ok 2 - request parameter module is foobar
ok 3 - request parameter action is index
not ok 4 - response selector body does not match regex /This is a temporary page/
# Looks like you failed 1 tests of 4.
1..4

Test-driven development

The generated functional tests for a new module don’t pass by default. As long as you don’t modify the index action, the tests for this module will fail, and this guarantees that you cannot pass all tests with an unfinished module.

Now, that’s what I call TDD!

In test-driven development, each new feature begins with writing a test. This test must inevitably fail because it is written before the feature has been implemented.