Archive for the ‘Agile Development’ Category
Managing Multiple Build Environments
Last updated: 3 March, 2010
One of the challenges of Web development is managing multiple build environments. Most applications pass through several environments before they are released. These environments include: A local development environment, a shared development environment, a system integration environment, a user acceptance environment and a production environment.
Automated Builds
Automated builds provide a consistent method for building applications and are used to give other developers feedback about whether the code was successfully integrated or not. There are different types of builds: Continuous builds, Integration builds, Release builds and Patch builds.
A source control system is the main point of integration for source code. When your team works on separate parts of the code base, you have to ensure that your checked in code doesn’t break the Integration build. That’s why it is important that you run your unit tests locally before checking in code.
Here is a recommended process for checking code into source control:
- Get the latest code from source control before running your tests
- Verify that your local build is building and passing all the unit tests before checking in code
- Use hooks to run a build after a transaction has been committed
- If the Integration build fails, fix the issue because you are now blocking other developers from integrating their code
Hudson can help you automate these tasks. It’s extremely easy to install and can be configured entirely from a Web UI. Also, it can be extended via plug-ins and can execute Phing, Ant, Gant, NAnt and Maven build scripts.
Build File
We need to create a master build file that contains the actions we want to perform. This script should make it possible to build the entire project with a single command line.
First we need to separate the source from the generated files, so our source files will be in the “src” directory and all the generated files in the “build” directory. By default Ant uses build.xml as the name for a build file, this file is usually located in the project root directory.
Then, you have to define whatever environments you want:
project/
build/
files/
local/
development/
integration/
production/
packages/
development/
project-development-0.1-RC.noarch.rpm
integration/
production/
default.properties
local.properties
development.properties
production.properties
src/
application/
config/
controllers/
domain/
services/
views/
library/
public/
tests/
build.xml
Build files tend to contain the same actions:
- Delete the previous build directory
- Copy files
- Manage dependencies
- Run unit tests
- Generate HTML and XML reports
- Package files
The target element is used as a wrapper for a sequences of actions. A target has a name, so that it can be referenced from elsewhere, either externally from the command line or internally via the “depends” or “antcall” keyword. Here’s a basic build.xml example:
<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">
<target name="init"></target>
<target name="test"></target>
<target name="test-selenium"></target>
<target name="profile"></target>
<target name="clean"></target>
<target name="build" depends="init, test, profile, clean"></target>
<target name="package"></target>
</project>
The property element allows the declaration of properties which are like user-definable variables available for use within an Ant build file. Properties can be defined either inside the buildfile or in a standalone properties file. For example:
<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">
<property file="${basedir}/build/default.properties" />
<property file="${basedir}/build/${build.env}.properties" />
...
</project>
The core idea is using property files which name accords to the environment name. Then simply use the custom build-in property build.env. For better use you should also provide a file with default values. The following example intends to describe a typical Ant build file, of course, it can be easily modified to suit your personal needs.
<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">
<property file="${basedir}/build/default.properties" />
<condition property="build.env" value="${build.env}" else="local">
<isset property="build.env" />
</condition>
<property file="${basedir}/build/${build.env}.properties" />
<property environment="env" />
<condition property="env.BUILD_ID" value="${env.BUILD_ID}" else="">
<isset property="env.BUILD_ID" />
</condition>
<target name="init">
<echo message="Environment: ${build.env}"/>
<echo message="Hudson build ID: ${env.BUILD_ID}"/>
<echo message="Hudson build number: ${env.BUILD_NUMBER}"/>
<echo message="SVN revision: ${env.SVN_REVISION}"/>
<tstamp>
<format property="build.datetime" pattern="dd-MMM-yy HH:mm:ss"/>
</tstamp>
<echo message="Build started at ${build.datetime}"/>
</target>
<target name="test">
...
</target>
<target name="clean">
<delete dir="${build.dir}/files/${build.env}"/>
<delete dir="${build.dir}/packages/${build.env}"/>
<mkdir dir="${build.dir}/files/${build.env}"/>
<mkdir dir="${build.dir}/packages/${build.env}"/>
</target>
<target name="build" depends="init, test, profile, clean">
...
</target>
...
</project>
Using ant -Dname=value lets you define values for properties on the Ant command line. These properties can then be used within your build file as any normal property: ${name} will put in value.
$ ant build -Dbuild.env=development
There are different ways to target multiple environments. I hope I have covered enough of the basic functionality to get you started.
Testing Zend Framework Action Controllers With Mocks
In this post I’ll demonstrate a unit test technique for testing Zend Framework Action Controllers using Mock Objects. Unit testing controllers independently has a number of advantages:
- You can develop controllers test-first (TDD).
- It allows you to develop and test all of your controller code before developing any of the view scripts.
- It helps you quickly identify problems in the controller, rather than problems in one of the combination of Model, View and Controller.
The Action Controller I’m going to test has only one method, profileAction():
tests/application/controllers/UserController.php
class UserController extends Zend_Controller_Action
{
public function profileAction()
{
$this->view->userId = $this->_getParam('user_id');
return $this->render();
}
}
tests/application/ControllerTestCase.php
class ControllerTestCase extends Zend_Test_PHPUnit_ControllerTestCase
{
public $application;
public function setUp()
{
$this->application = new Zend_Application(
APPLICATION_ENV,
APPLICATION_PATH . '/config/application.ini'
);
$this->bootstrap = array($this, 'bootstrap');
parent::setUp();
}
public function tearDown()
{
Zend_Controller_Front::getInstance()->resetInstance();
$this->resetRequest();
$this->resetResponse();
$this->request->setPost(array());
$this->request->setQuery(array());
}
public function bootstrap()
{
$this->application->bootstrap();
}
}
tests/application/controllers/UserControllerTest.php
require_once TESTS_PATH . '/application/ControllerTestCase.php';
require_once APPLICATION_PATH . '/controllers/UserController.php';
class UserControllerTest extends ControllerTestCase
{
public function testStubRenderMethodCall()
{
$request = $this->getRequest()
->setRequestUri('/user/profile/1')
->setParams(array('user_id'=>1))
->setPathInfo(null);
$response = $this->getResponse();
$this->getFrontController()
->setRequest($request)
->setResponse($response)
->throwExceptions(true)
->returnResponse(false);
$controller = $this->getMock(
'UserController',
array('render'),
array($request, $response, $request->getParams())
);
$controller->expects($this->once())
->method('render')
->will($this->returnValue(true));
$this->assertTrue($controller->profileAction());
$this->assertTrue($controller->view->user_id == 1);
}
}
You can go further making both the tests and the implementation more sophisticated. The main point is that you can build and test a controller in a way that doesn’t require a view script to be written to do so.
Zend Framework Known Issues
By default Zend_Test_PHPUnit_ControllerTestCase sets the redirector exit value to false, leading to unexpected behavior when unit testing your code. For that reason, make sure you always add a return statement after calling a utility method:
class UserController extends Zend_Controller_Action
{
public function profileAction()
{
if (null == $this->_getParam('user_id', null) {
return $this->_redirect('/');
}
return $this->render();
}
}
If you want the Front Controller to throw exceptions, you have no other choice than to overwrite the dispatch method and pass a boolean TRUE to the throwExceptions() method:
class ControllerTestCase extends Zend_Test_PHPUnit_ControllerTestCase
{
...
public function dispatch($url = null)
{
// redirector should not exit
$redirector = Zend_Controller_Action_HelperBroker::getStaticHelper('redirector');
$redirector->setExit(false);
// json helper should not exit
$json = Zend_Controller_Action_HelperBroker::getStaticHelper('json');
$json->suppressExit = true;
$request = $this->getRequest();
if (null !== $url) {
$request->setRequestUri($url);
}
$request->setPathInfo(null);
$this->getFrontController()
->setRequest($request)
->setResponse($this->getResponse())
->throwExceptions(true)
->returnResponse(false);
$this->getFrontController()->dispatch();
}
...
}
The Dispatcher not only violates the DRY principle but also suffers from amnesia. The problem is that it doesn’t store the instance of the Action Controller, instead, it destroys it (Zend_Controller_Dispatcher_Standard Line 305). You can easily get around this issue by extending the standard dispatcher and overwriting the dispatch() method:
class Zf_Controller_Dispatcher_Standard extends Zend_Controller_Dispatcher_Standard
{
...
public function dispatch($url = null)
{
...
Zend_Registry::set('Zend_Controller_Action', $controller);
// Destroy the page controller instance and reflection objects
$controller = null;
}
This will allow you to access the view object after dispatching the request:
class ExampleControllerTest extends ControllerTestCase
{
public function testDefaultActionRendersViewObject()
{
$this->dispatch('/');
$controller = Zend_Registry::get('Zend_Controller_Action');
$this->assertEquals('ExampleController', get_class($controller));
$this->assertTrue(isset($controller->view));
}
Links
PHPUnit: Testing Zend Framework Controllers
PHPUnit: Mock Objects
Domain-Driven Design: Sample Application
Last updated: 15 Feb, 2010
Part 1: Domain-Driven Design and MVC Architectures
Part 2: Domain-Driven Design: Data Access Strategies
Part 3: Domain-Driven Design: The Repository
Some of the Domain-driven design concepts explained above are applied in this sample application.
Directory Structure
app/
config/
controllers/
UserController.php
domain/
entities/
User.php
UserProfile.php
repositories/
UserRepository.php
views/
lib/
public/
The domain layer should be well separated from the other layers and it should have few dependencies on the framework you are using.
User Entity
The User and UserProfile objects have a one-to-one relationship and form an Aggregate. An Aggregate is as a collection of related objects that have references between each other. Within an Aggregate there’s always an Aggregate Root (parent Entity), in this case User:
class User
{
private $id;
private $name;
/* @var UserProfile */
private $profile;
public function __construct($id, $name)
{
$this->id = $id;
$this->name = $name;
}
public function setProfile(UserProfile $profile)
{
$this->profile = $profile;
}
public function getProfile()
{
return $this->profile;
}
}
class UserProfile
{
private $id;
public function __construct($id)
{
$this->id = $id;
}
}
Users Collection
A collection is simply an object that groups multiple elements into a single unit.
class Users
{
private $elements = array();
public function __construct(array $users)
{
foreach ($users as $user) {
if (!($user instanceof User)) {
throw new Exception();
}
$this->elements[] = $user;
}
}
public function toArray()
{
return $this->elements;
}
}
User DAO
The UserDAO class allows data access mechanisms to change independently of the code that uses the data:
interface UserDao
{
public function find($id);
public function findAll();
}
class UserDatabaseDao implements UserDao
{
public function find($id)
{
$dataSource = Zf_Orm_Manager::getInstance()->getDataSource()
$db = $dataSource->getConnection('slave');
$query = $db->select()->from('user')->where('id = ?', $id);
return $db->fetchRow($query);
}
public function findAll()
{
...
return $db->fetchAll($query);
}
}
interface UserProfileDao
{
public function find($id);
public function findByUserId($id);
}
class UserProfileDatabaseDao implements UserProfileDao
{
public function find($id)
{
...
}
public function findByUserId($id)
{
$dataSource = Zf_Orm_Manager::getInstance()->getDataSource()
$db = $dataSource->getConnection('slave');
$query = $db->select()->from('user_profile')->where('user_id = ?', $id);
return $db->fetchRow($query);
}
}
User Repository
A Repository is basically a collection of Aggregate Roots. Collections are used to store, retrieve and manipulate Entities and Value objects, however, object management is beyond the scope of this post. The Repository object can inject dependencies on demand, making the instantiation process inexpensive.
class UserRepository
{
/* @var UserDatabaseDao */
private $userDao;
/* @var UserProfileDatabaseDao */
private $userProfileDao;
public function __construct()
{
$this->userDao = new UserDatabaseDao();
$this->userProfileDao = new UserProfileDatabaseDao();
}
public function find($id)
{
$row = $this->userDao->find($id);
$user = new User($row['id'], $row['name']);
$row = $this->userProfileDao->findByUserId($id);
if (isset($row['id'])) {
$profile = new UserProfile($row['id']);
$user->setProfile($profile);
}
return $user;
}
public function findAll()
{
$users = array();
$rows = $this->userDao->findAll();
foreach ($rows as $row) {
$users[] = new User($row['id'], $row['name']);
}
return new Users($users);
}
}
Usage:
$repository = new UserRepository(); $user = $repository->find(1); $profile = $user->getProfile(); $users = $repository->findAll();
Source Code
http://svn.fedecarg.com/repo/Zf/Orm
Links
If you’re interested in learning more about Domain-driven design, I recommend the following articles:
Domain Driven Design and Development In Practice
Domain-Driven Design in an Evolving Architecture
Domain-Driven Design: The Repository
Part 2: Domain-Driven Design: Data Access Strategies
The Ubiquitous Language
The ubiquitous language is the foundation of Domain-driven design. The concept is simple, developers and domain experts share a common language that both understand. This language is set in business terminology, not technical terminology. This ubiquitous language allows the technical team become part of the business.
The Repository
Repositories play an important part in DDD, they speak the language of the domain and act as mediators between the domain and data mapping layers. They provide a common language to all team members by translating technical terminology into business terminology.
In a nutshell, a Repository:
- Is not a data access layer
- Provides a higher level of data manipulation
- Is persistence ignorance
- Is a collection of aggregate roots
- Offers a mechanism to manage entities
Data Access Objects
In the absence of an ORM framework, the Data Access Object (DAO) handles the impedance mismatch that a relational database has with object-oriented techniques. In DDD, you inject Repositories, not DAO’s in domain entities. For example, imagine you have an entity named User that needs to access the database to retrieve the User details:
class User
{
private $id;
private $name;
public function __construct(UserDao $dao, $id)
{
$row = $dao->find($id);
$this->setId($row['id']);
$this->setName($row['name']);
}
public function setId($id) {}
public function getId() {}
public function setName($name) {}
public function getName() {}
}
The User DAO class will look something like this:
interface UserDao
{
public function fetchRow($id);
}
class UserDatabaseDaoImpl implements UserDao
{
public function fetchRow($id)
{
...
$query = $db->select();
$query->from('user');
$query->where('id = ?', $id);
return $db->fetchRow($query);
}
}
$dao = new UserDatabaseDaoImpl();
$user = new User($dao, 1);
$userId = $user->getId();
But, what about separation of concerns? DAO’s are related to persistence, and persistence is infrastructure, not domain. The main problem with the example above is that we have lots of different concerns polluting the domain. According to DDD, an object should be distilled until nothing remains that does not relate to its meaning or support its role in interactions. And that’s exactly the problem the Repository pattern tries to solve.
Injecting Repositories
Lets create a UserRepository class to isolate the domain object from details of the UserDatabaseDaoImpl class:
class User
{
private $id;
private $name;
public function __construct(UserRepository $repository, $id)
{
$row = $repository->find($id);
$this->setId($row['id']);
$this->setName($row['name']);
}
public function setId($id) {}
public function getId() {}
public function setName($name) {}
public function getName() {}
}
It’s the responsibility of the UserRepository to work with all necessary DAO’s and provide all data access services to the domain model in the language which the domain understands.
interface UserRepository
{
public function find($id);
}
class UserRepositoryImpl implements UserRepository
{
private $databaseDao;
public function setDatabaseDao(UserDao $dao) {}
public function getDatabaseDao() {}
public function find($id)
{
return $this->getDatabaseDao()->find($id);
}
}
$userRepository = new UserRepositoryImpl();
$userRepository->setDatabaseDao(new UserDatabaseDaoImpl());
$user = new User($userRepository, 1);
$userId = $user->getId();
$userName = $user->getName();
The main difference between the Repository and the DAO is that the DAO is at a lower level of abstraction and doesn’t speak the ubiquitous language of the domain.
So, what’s next?
The process of creating an entity is complex, because an entity always has relationship with other objects in your domain model. When creating an entity, we have to initialize its relationships as well. Therefore, it’s a good practice to delegate this task to another object.
We’ve seen how to write a persistence-ignorant domain model. Next, I’ll explain how to automate the creation and injection of dependencies.
Domain-Driven Design: Data Access Strategies
Part 1: Domain-Driven Design and MVC Architectures
The Domain Model
Here are some of the features a Domain-driven design framework should support:
- A domain model that is independent and decoupled from the application.
- A reusable library that can be used in many different domain-specific applications.
- Dependency Injection in order to inject Repositories and Services into Domain Objects.
- Integration with unit testing frameworks, such as PHPUnit.
- Good integration with other frameworks, such as Zend, Symfony, Doctrine, etc.
The Zend Framework, for example, is part of the infrastructure layer and acts as a supporting library for all the other layers. However, the domain layer should be well isolated from the other layers of the application and should not be dependent on the framework you are using.
Data Access Strategies
When accessing data from a data source, you have to decide how your application will communicate with a data source. Each data access strategy has its own advantages and disadvantages. Here are some design patterns that support DDD:
Generic DAO’s
Data Access Objects (DAO’s) and Repositories play an important role in DDD. The goal of a DAO is to abstract and encapsulate all access to the data and provide an interface. The DAO always connects, reads and saves data to a data source. From the applications point of view, it makes no difference when it accesses a database, XML file or Web service:
$user = new UserDbDao(); $row = $user->findUserById(456); echo $row->name; $user = new UserXmlDao(); $row = $user->findUserById(456); echo $row->name;
Table Data Gateway
An object that acts as a Gateway to a database table. One instance handles all the rows in the table. The Zend_Db_Table solution is an implementation of the Table Data Gateway pattern:
$user = new User(); $rows = $user->find(456); $row = $rows->current(); echo $row->name;
More info: Table Data Gateway
Row Data Gateway
An object that acts as a Gateway to a single record in a data source. There is one instance per row. Zend_Db_Table_Row is an implementation of the Row Data Gateway pattern:
$user = new User();
$row = $user->fetchRow($user->select()->where('user_id = ?', 1));
echo $row->name;
More info: Row Data Gateway
Active Record
An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data. The Mad_Model component is an ORM layer that follows the Active Record pattern where tables map to classes, rows map to objects, and columns to object attributes:
$user = new User(); $userFinder = new UserFinder(); $row = $userFinder->find(456);
More info: Active Record
Data Mapper
A layer of Mappers that moves data between objects and a database while keeping them independent of each other and the mapper itself:
$user = new User(); $userMapper = new UserMapper(); $row = $userMapper->findByEmail($email);
More info: Data Mapper
Data Transfer Objects
A Data Transfer Object (DTO) in Domain-driven design is not the same as a Value Object. A DTO is often used in combination with a DAO to encapsulate data. It allows you to reduce communication effort when dealing with a lot of small data entities. For example:
class UserData
{
public function setId($id) {}
public function getId() {}
public function setFirstName($firstName) {}
public function getFirstName() {}
public function setLastName($lastName) {}
public function getLastName() {}
}
$user = new UserData();
$user->setFirstName('Jim');
$user->setLastName('Morrison');
$userDao = new UserDbDao();
$userDao->insert($user);
A DTO is a simple container for a set of aggregated data. It should contain no business logic and limit its behaviour to activities such as internal consistency checking and basic validation. Be careful not to make the DTO depend on any new classes as a result of implementing these methods.
Domain-Driven Design and MVC Architectures
According to Eric Evans, Domain-driven design (DDD) is not a technology or a methodology. It’s a different way of thinking about how to organize your applications and structure your code. This way of thinking complements very well the popular MVC architecture. The domain model provides a structural view of the system. Most of the time, applications don’t change, what changes is the domain. MVC, however, doesn’t really tell you how your model should be structured. That’s why some frameworks don’t force you to use a specific model structure, instead, they let your model evolve as your knowledge and expertise grows.
Domain-driven design separates the model layer “M” of MVC into an application, domain and infrastructure layer. The infrastructure layer is used to retrieve and store data. The domain layer is where the business knowledge or expertise is. The application layer is responsible for coordinating the infrastructure and domain layers to make a useful application. Typically, it would use the infrastructure to obtain the data, consult the domain to see what should be done, and then use the infrastructure again to achieve the results. Srini Penchikala explains this in more detail here: “Domain Driven Design and Development In Practice“.
Object-oriented programming is the most important element in the domain implementation. Domain objects are designed using classes and interfaces, and take advantage of OOP concepts like inheritance, encapsulation, and polymorphism. Most of the domain elements are true objects with both State (attributes) and Behaviour (methods or operations that act on the state). Entities and Value Objects in DDD are classic examples of OOP concepts since they have both state and behaviour.
Terminology used by Domain-driven design
- Entity: An object which has a distinct identity. For example, a User entity has a distinct identity with an ID.
- Value Object: An object which has no distinct identity, like numbers and dates. For example, if two Users have the same date of birth, you can have multiple copies of an object that represents the date 16 Jan 1982.
- Factory: Used to create Entities. For example, you can use a Factory to create a Profile entity from a User entity.
- Repository: Used to store, retrieve and delete domain objects from different storage implementations. For example, you can use a Repository to store the Profile your Factory created.
- Service: When an operation does not conceptually belong to any object.
The purpose of this post was to provide a brief introduction to Domain-driven design.
Part 2: Domain-Driven Design: Data Access Strategies
Links
If you’re interested in learning more about Domain-driven design or Model-driven design, I recommend the following articles:
10 reasons to switch from CruiseControl to Hudson
Ten things no one ever told you about Hudson:
- It’s extremely easy to install (unzip the file and that’s it)
- It can be configured entirely from its friendly Web UI (no XML required)
- It has an attractive dashboard with colourful and meaningful icons
- It’s extremely flexible
- It can be extended via plug-ins
- It offers a much better UI than CruiseControl
- It can execute Phing, Ant, Gant, NAnt and Maven build scripts
- It gives you clean readable URLs for most of its pages
- It has RSS, e-mail and IM integration
- It can distribute build/test loads to multiple computers
This tutorial guides you step-by-step through the fundamental concepts of Continuous Integration and Hudson. When you are done with this one-hour tutorial, you will understand the benefits of Continuous Integration as well as how to set up your environment.
Links
Four Great InfoQ Presentations
Hope you like these recommendations and if you know of any other good tech-related video, then please let me know.
1. Developing Expertise: Herding Racehorses, Racing Sheep
One of my favourites. In this presentation Dave Thomas (The Pragmatic Programmer) talks about expanding people’s expertise in their domains of interest by not treating them uniformly as they had the same amount of knowledge and level of experience.
2. Real World Web Services
Another good presentation. In this one Scott Davis provides a pragmatic, down-to-earth introduction to Web services as used in the real world by public sites, including SOAP-based and REST examples.
3. CouchDB and me
This presentation is different, and that’s why I like it so much. Damien Katz shares his experiences and reminds people how difficult but at the same time gratifying is to be an open source developer. He talks about the history of CouchDB development from a very personal point of view. His inspirations for CouchDB and why he decided to move my wife and kids to a cheaper place and live off savings to build this thing.
4. Yahoo! Communities Architectures
In this presentation, Ian Flint tries to explain the infrastructure and architecture employed by Yahoo! to keep going a multitude of servers running of different platforms and offering different services. Very interesting!
The Importance of Branching Models
Among the branching models used in software configuration management, the branch-by-purpose model offers better support for parallel development efforts and improved control of both planned and emergency software releases. If you want to improve software quality, you must first understand your software. What are its pieces? How are they organized and related to one another? If you do not understand your code base, your odds of updating it without breaking something are poor.
The Importance of Branching Models (PDF)
Links
High-level Best Practices in Software Configuration Management
Scrum Backlog Templates
Here are two useful backlog templates provided by Agile Software Development. Both of them are in Excel format (XLS). Check them out: