Federico Cargnelutti

Simple is better than complex. Complex is better than complicated. | @fedecarg

OSCON 2010, The O’Reilly Open Source Convention

leave a comment »

A couple of weeks ago I attended the O’Reilly Open Source Convention (OSCON) in Portland. OSCON has hundreds of sessions and activities focused on all aspects of open source software. I met some great people, the talks were good and I saw some promising ideas and technologies.

Workshops attended

  • Android for Java Developers
    Marko Gargenta (Marakana)
  • Building a NoSQL Data Cloud
    Krishna Sankar (Cisco Systems Inc)
  • Building Native Mobile Apps Using Open Source
    Kevin Whinnery (Appcelerator)

Sessions attended

  • Building Mobile Apps with HTML, CSS, and JavaScript
    Jonathan Stark (Jonathan Stark Consulting)
  • Open Source Tool Chains for Cloud Computing
    Mark Hinkle (Zenoss), John Willis (Opscode, Inc.), Alex Honor
  • Doctor, I Have a Problem with My Innovation.
    Rolf Skyberg (eBay, Inc.)
  • Ingex: Bringing Open Source to the Broadcast Industry
    By Brendan Quinn (BBC R&D)
  • membase.org: The Simple, Fast, Elastic NoSQL Database
    Matt Ingenthron (NorthScale, Inc.)
  • Introducing WebM: High Quality, Royalty-Free, Open Source Video
    John Koleszar (Google, Inc.)
  • Whiskey, Tango, Foxtrot: Understanding API Activity
    Clay Loveless (Mashery)
  • Deploying an Open Source Private Cloud On a Shoe String Budget
    Louis Danuser (AT&T Labs, Inc.)
  • Eucalyptus: The Open Source Infrastructure for Cloud Computing
    Shashi Mysore (Eucalyptus Systems Inc.)
  • Hadoop, Pig, and Twitter
    Kevin Weil (Twitter, Inc.)
  • Mahout: Mammoth Scale Machine Learning
    Robin Anil (Apache Software Foundation)
  • BlackBerry development for Web Application Developers
    Kevin Falcone (Best Practical Solutions)
  • Practical Concurrency
    Tim Bray (Google, Inc.)
  • Scribe – Moving Data at Massive Scale
    Robert Johnson (Facebook)
  • Make Open Easy
    Dan Bentley (Google)

Written by Federico

August 9, 2010 at 10:10 am

Posted in Open-source

Most Visited Posts of 2009

leave a comment »

Written by Federico

April 2, 2010 at 8:50 pm

Posted in Programming

Implementing Dynamic Finders and Parsing Method Expressions

with 3 comments

Most ORMs support the concept of dynamic finders. A dynamic finder looks like a normal method invocation, but the method itself doesn’t exist, instead, it’s generated dynamically and processed via another method at runtime.

A good example of this is Ruby. When you invoke a method that doesn’t exist, it raises a NoMethodError exception, unless you define “method_missing”. Rails ActiveRecord::Base class implements some of its magic thanks to this method. For example, find_by_title(title) and find_by_title_and_date(title, date) are turned into:

find(:first, :conditions => ["title = ?", title])
find(:first, :conditions => ["title = ? AND date = ?", title, date])

What’s nice about Ruby is that the language allows you to define methods dynamically using the “define_method” method. That’s how Rails defines each dynamic finder in the class after it is first invoked, so that future attempts to use it do not run through the “method_missing” method.

Method Expressions

GORM, Grails ORM library, introduces the concept of dynamic method expressions. A method expression is made up of the prefix such as “findBy” followed by an expression that combines one or more properties. Grails takes advantage of Groovy features to provide dynamic methods:

findByTitle("Example")
findByTitleLike("Exa%")

Method expressions can also use a boolean operator to combine two criteria:

findAllByTitleLikeAndDateGreaterThan("Exampl%", '2010-03-23')

In this case we are using AND in the middle of the query to make sure both conditions are satisfied, but you could equally use OR:

findAllByTitleLikeOrDateGreaterThan("Exampl%", '2010-03-23')

Parsing Method Expressions

MethodExpressionParser is a PHP library for parsing method expressions. It’s designed to quickly and easily parse method expressions and construct conditions based on attribute names and arguments.

Description

[finderMethod]([attribute][expression][logicalOperator])?[attribute][expression]

Expressions

  • LessThan: Less than the given value
  • LessThanEquals: Less than or equal a give value
  • GreaterThan: Greater than a given value
  • GreaterThanEquals: Greater than or equal a given value
  • Like: Equivalent to a SQL like expression
  • NotEqual: Negates equality
  • IsNotNull: Not a null value (doesn’t require an argument)
  • IsNull: Is a null value (doesn’t require an argument)

Examples

findByTitleAndDate('Example', date('Y-m-d'));
SELECT * FROM book WHERE title = ? AND date = ?

findByTitleOrDate('Example', date('Y-m-d'))
SELECT * FROM book WHERE title = ? OR date = ?

findByPublisherOrTitleAndDate('Name', 'Example', date('Y-m-d'))
SELECT * FROM book WHERE publisher = ? OR (title = ? AND date = ?)

findByPublisherInAndTitle(array('Name1', 'Name2'), 'Example')
SELECT * FROM book WHERE publisher IN (?, ?) AND date = ?

findByTitleLikeAndDateNotNull('Examp%')
SELECT * FROM book WHERE title LIKE ? AND date NOT NULL

findByIdOrTitleAndDateNotNull(1, 'Example')
SELECT * FROM book WHERE (id = ?) OR (title = ? AND date NOT NULL)

Example 1:

findByTitleLikeAndDateNotNull('Examp%');

Outputs:

array
  0 =>
    array
      0 =>
        array
          'attribute' => string 'title'
          'expression' => string 'Like'
          'format' => string '%s LIKE ?'
          'placeholders' => int 1
          'argument' => string 'Examp%'
      1 =>
        array
          'attribute' => string 'date'
          'expression' => string 'NotNull'
          'format' => string '%s IS NOT NULL'
          'placeholders' => int 0
          'argument' => null

Example 2:

findByTitleAndPublisherNameOrTitleAndPublisherName('Title', 'a', 'Title', 'b');

Outputs:

array
  0 =>
    array
      0 =>
        array
          'attribute' => string 'title'
          'expression' => string 'Equals'
          'format' => string '%s = ?'
          'placeholders' => int 1
          'argument' => string 'Title'
      1 =>
        array
          'attribute' => string 'publisher_name'
          'expression' => string 'Equals'
          'format' => string '%s = ?'
          'placeholders' => int 1
          'argument' => string 'a'
  1 =>
    array
      0 =>
        array
          'attribute' => string 'title'
          'expression' => string 'Equals'
          'format' => string '%s = ?'
          'placeholders' => int 1
          'argument' => string 'Title'
      1 =>
        array
          'attribute' => string 'publisher_name'
          'expression' => string 'Equals'
          'format' => string '%s = ?'
          'placeholders' => int 1
          'argument' => string 'b'

See more examples: Project Wiki

Usage

class EntityRepository
{
    private $methodExpressionParser;

    // Return a single instance of MethodExpressionParser
    public function getMethodExpressionParser() {
    }

    // Finder methods
    public function findBy($conditions) {
        var_dump($conditions);
    }
    public function findAllBy($conditions) {
        var_dump($conditions);
    }

    // Invoke finder methods
    public function __call($method, $args) {
        if ('f' === $method{0}) {
            try {
                $result = $this->getMethodExpressionParser()->parse($method, $args);
                $finderMethod = key($result);
                $conditions = $result[$finderMethod];
            } catch (MethodExpressionParserException $e) {
                $message = sprintf('%s: %s()', $e->getMessage(), $method);
                throw new EntityRepositoryException($message);
            }
            return $this->$finderMethod($conditions);
        }

        $message = 'Invalid method call: ' . __METHOD__;
        throw new BadMethodCallException($message);
    }
}

Performance

PHP doesn’t allow you to define methods dynamically, this means that every time you invoke a finder method the parser has to search, extract and map all the attribute names and expressions. To avoid introducing this performance overhead you can cache the attribute names. For example:

class EntityRepository
{
    private $methodExpressionParser;
    private $classMetadata;

    // Return a single instance of MethodExpressionParser
    public function getMethodExpressionParser() {
    }

    // Return a single instance of ClassMetadata
    public function getClassMetadata() {
    }

    // Invoke finder methods
    public function __call($method, $args) {
        if ('f' === $method{0}) {
            $parser = $this->getMethodExpressionParser();
            $classMetadata = $this->getClassMetadata();
            try {
                $finderMethod = $parser->determineFinderMethod($method);
                if ($classMetadata->hasMissingMethod($method)) {
                    $attributes = $classMetadata->getMethodAttributes($method);
                    $conditions = $parser->map($args, $attributes);
                } else {
                    $expressions = substr($method, strlen($finderMethod));
                    $attributes = $this->extractAttributeNames($expressions);
                    $conditions = $parser->map($args, $attributes);
                    $classMetadata->setMethodAttributes($method, $attributes);
                }
            } catch (MethodExpressionParserException $e) {
                $message = sprintf('%s: %s()', $e->getMessage(), $method);
                throw new EntityRepositoryException($message);
            }
            return $this->$finderMethod($conditions);
        }

        $message = 'Invalid method call: ' . __METHOD__;
        throw new BadMethodCallException($message);
    }
}

The Expression objects are lazy-loaded, depending on the expressions found in the method name.

Extensibility

The MethodExpressionParser class was designed with extensibility in mind, allowing you to add new Expressions to the library.

abstract class Expression {
}
class EqualsExpression extends Expression {
}

Source Code

Browse source code:

http://fedecarg.com/repositories/show/expressionparser

Check out the current development trunk with:

$ svn checkout http://svn.fedecarg.com/repo/Zf/Orm

Written by Federico

March 22, 2010 at 11:51 pm

Review: Zend Framework 1.8 Web Application Development

with 6 comments

Zend Framework is one of the most popular and hottest open-source frameworks being used today. The number of books about Web development using Zend Framework has increased over the last couple of years.

Packt Publishing sent me a copy of the book Zend Framework 1.8 Web Application Development by Keith Pope to review. I found this book to be a good introduction to the topics that a Zend Framework developer will need to know when developing enterprise Web applications. The book is also aimed at advanced users, considering there were a couple of things that I learned about this framework from reading the book. The book not only shows you how the Zend Framework works, but also how to write an application for real-world usage. The book covers access control, performance optimization, testing, debugging and application design. The writing is clear, the code examples are good and Keith does an excellent job of walking you through the life-cycle of a request, explaining how things work and how you can extend the framework to fit your needs.

Here’s an example of the Storefront application:
http://code.google.com/p/zendframeworkstorefront/source/browse

Conclusion

The book is very well-written, nicely structured and full of highly practical advice. Overall, I’m happy to say that Zend Framework 1.8 Web Application Development fulfilled my expectations.

Other Reviews

Raphael Stolt

“The content of the book is delivered in a fluent, very enthusiastic and ‘knowledge-pillowed’ writing tone. By implementing or working through the Storefront application seasoned web developers using older versions of the Framework will get a good blue sheet on new components like Zend_Application and it’s implication in the bootstrapping process; while new developers tending towards picking up the Zend Framework will get a current and well compiled guide, which might first start off with a steep learning-curve but will turn into profund knowledge once hanging in there.”

Fred Wu

“The flow of this book is heavily inspired by the famous Ruby on Rails book, Agile Web Development with Rails, where the author invites you to join the process of building a demo application, which in both cases is a shopping cart system. Judging by the feedback of the Rails book, most people feel quite comfortable learning a framework this way, some don’t. I guess if you are not a fan of following a defined learning structure, this book probably isn’t for you.”

Written by Federico

December 11, 2009 at 11:50 am

Sky Named Britain’s Most Admired Company

leave a comment »

Based on a survey of thousands of managers and investment analysts, Management Today has named BSkyB as Britain’s Most Admired Company for 2009. BSkyB is the youngest company ever to win this Award.

BSkyB beat off the superstore giant Tesco into second place. Johnson Mathey took the third slot, with Cadbury, GlaxoSmithKline and Rolls-Royce trailing at numbers four, five and six.

BSkyB headed off the sector in all of the criteria laid down by the organizers, coming out top in “quality of goods and services”, “quality of marketing” and “capacity to innovate”.

Most Admired Top 20, 2009

(Last year’s position in brackets)

1 (4)     BSkyB 72.25
2 (5)     Tesco 71.38
3 (2)     Johnson Matthey 71.00
4 (18)    Cadbury 70.40
5 (19)    GlaxoSmithKline 70.00
6 (7)     Rolls-Royce 69.96
7 (26)    BP 67.08
8 (11)    BG Group 67.03
9 (1)     Diageo 65.83
10 (47)   Cobham 65.75
11 (3)    Unilever 65.0
12 (52)   BAE Systems 64.9
13 (51)   Ultra Electronics 64.7
14 (154)  Centrica 64.4
14 (24)   Royal Dutch Shell 64.4
16 (81)   Admiral 63.9
16 (17)   Capita Group 63.9
18 (27)   Sainsbury 63.8
19 (55)   Balfour Beatty 63.1
20 (29)   Marks & Spencer 62.9

BSkyB is a great company to work for, filled with talented people. Congratulation for this prestigious award!

Links

Management Today
Sky News

Written by Federico

December 6, 2009 at 8:40 pm

Posted in Awards

Tagged with

Command-line memcached stat reporter

leave a comment »

Nicholas Tang wrote a nice little perl script that shows a basic memcached top display for a list of servers. You can specify thresholds, for instance, and it’ll change color to red if you exceed the thresholds. You can also choose the refresh/sleep time, and whether to show immediate (per second) stats, or lifetime stats.

To install it you only need to download the script and make it executable:

$ curl http://memcache-top.googlecode.com/files/memcache-top-v0.6 > ~/bin/memcache-top
$ chmod +x ~/bin/memcache-top
$ memcache-top --sleep 3 --instances 10.50.11.3,10.50.11.4,10.50.11.5

Here’s some sample output:

memcache-top v0.6       (default port: 11211, color: on, refresh: 3 seconds)

INSTANCE                USAGE   HIT %   CONN    TIME    EVICT/s GETS/s  READ/s  WRITE/s
10.50.11.3:11211        88.9%   69.7%   1661    0.9ms   0.3     47      13.9K   9.8K
10.50.11.4:11211        88.8%   69.9%   2121    0.7ms   1.3     168     17.6K   68.9K
10.50.11.5:11211        88.9%   69.4%   1527    0.7ms   1.7     48      14.4K   13.6K
AVERAGE:                84.7%   72.9%   1704    1.0ms   1.3     69      13.5K   30.3K   

TOTAL:          19.9GB/ 23.4GB          20.0K   11.7ms  15.3    826     162.6K  363.6K
(ctrl-c to quit.)

Project Home
http://code.google.com/p/memcache-top/

Written by Federico

November 29, 2009 at 3:28 pm

Posted in Open-source, Tools

Managing Multiple Build Environments

with 2 comments

Last updated: 3 March, 2010

One of the challenges of Web development is managing multiple build environments. Most applications pass through several environments before they are released. These environments include: A local development environment, a shared development environment, a system integration environment, a user acceptance environment and a production environment.

Automated Builds

Automated builds provide a consistent method for building applications and are used to give other developers feedback about whether the code was successfully integrated or not. There are different types of builds: Continuous builds, Integration builds, Release builds and Patch builds.

A source control system is the main point of integration for source code. When your team works on separate parts of the code base, you have to ensure that your checked in code doesn’t break the Integration build. That’s why it is important that you run your unit tests locally before checking in code.

Here is a recommended process for checking code into source control:

  • Get the latest code from source control before running your tests
  • Verify that your local build is building and passing all the unit tests before checking in code
  • Use hooks to run a build after a transaction has been committed
  • If the Integration build fails, fix the issue because you are now blocking other developers from integrating their code

Hudson can help you automate these tasks. It’s extremely easy to install and can be configured entirely from a Web UI. Also, it can be extended via plug-ins and can execute Phing, Ant, Gant, NAnt and Maven build scripts.

Build File

We need to create a master build file that contains the actions we want to perform. This script should make it possible to build the entire project with a single command line.

First we need to separate the source from the generated files, so our source files will be in the “src” directory and all the generated files in the “build” directory. By default Ant uses build.xml as the name for a build file, this file is usually located in the project root directory.

Then, you have to define whatever environments you want:

project/
    build/
        files/
            local/
            development/
            integration/
            production/
        packages/
            development/
                project-development-0.1-RC.noarch.rpm
            integration/
            production/
        default.properties
        local.properties
        development.properties
        production.properties
    src/
        application/
            config/
            controllers/
            domain/
            services/
            views/
        library/
        public/
    tests/
    build.xml

Build files tend to contain the same actions:

  • Delete the previous build directory
  • Copy files
  • Manage dependencies
  • Run unit tests
  • Generate HTML and XML reports
  • Package files

The target element is used as a wrapper for a sequences of actions. A target has a name, so that it can be referenced from elsewhere, either externally from the command line or internally via the “depends” or “antcall” keyword. Here’s a basic build.xml example:

<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">

    <target name="init"></target>
    <target name="test"></target>
    <target name="test-selenium"></target>
    <target name="profile"></target>
    <target name="clean"></target>
    <target name="build" depends="init, test, profile, clean"></target>
    <target name="package"></target>

</project>

The property element allows the declaration of properties which are like user-definable variables available for use within an Ant build file. Properties can be defined either inside the buildfile or in a standalone properties file. For example:

<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">

    <property file="${basedir}/build/default.properties" />
    <property file="${basedir}/build/${build.env}.properties" />
    ...

</project>

The core idea is using property files which name accords to the environment name. Then simply use the custom build-in property build.env. For better use you should also provide a file with default values. The following example intends to describe a typical Ant build file, of course, it can be easily modified to suit your personal needs.

<?xml version="1.0" encoding="iso-8859-1"?>
<project name="project" basedir="." default="main">

    <property file="${basedir}/build/default.properties" />
    <condition property="build.env" value="${build.env}" else="local">
        <isset property="build.env" />
    </condition>
    <property file="${basedir}/build/${build.env}.properties" />

     <property environment="env" />
     <condition property="env.BUILD_ID" value="${env.BUILD_ID}" else="">
         <isset property="env.BUILD_ID" />
     </condition>

    <target name="init">
        <echo message="Environment: ${build.env}"/>
        <echo message="Hudson build ID: ${env.BUILD_ID}"/>
        <echo message="Hudson build number: ${env.BUILD_NUMBER}"/>
        <echo message="SVN revision: ${env.SVN_REVISION}"/>
        <tstamp>
            <format property="build.datetime" pattern="dd-MMM-yy HH:mm:ss"/>
        </tstamp>
        <echo message="Build started at ${build.datetime}"/>
    </target>

    <target name="test">
        ...
    </target>

    <target name="clean">
        <delete dir="${build.dir}/files/${build.env}"/>
        <delete dir="${build.dir}/packages/${build.env}"/>
        <mkdir dir="${build.dir}/files/${build.env}"/>
        <mkdir dir="${build.dir}/packages/${build.env}"/>
    </target>

    <target name="build" depends="init, test, profile, clean">
        ...
    </target>
    ...

</project>

Using ant -Dname=value lets you define values for properties on the Ant command line. These properties can then be used within your build file as any normal property: ${name} will put in value.

$ ant build -Dbuild.env=development

There are different ways to target multiple environments. I hope I have covered enough of the basic functionality to get you started.

Written by Federico

November 14, 2009 at 6:31 pm

Testing Zend Framework Action Controllers With Mocks

with 10 comments

In this post I’ll demonstrate a unit test technique for testing Zend Framework Action Controllers using Mock Objects. Unit testing controllers independently has a number of advantages:

  1. You can develop controllers test-first (TDD).
  2. It allows you to develop and test all of your controller code before developing any of the view scripts.
  3. It helps you quickly identify problems in the controller, rather than problems in one of the combination of Model, View and Controller.

The Action Controller I’m going to test has only one method, profileAction():

tests/application/controllers/UserController.php

class UserController extends Zend_Controller_Action
{
    public function profileAction()
    {
        $this->view->userId = $this->_getParam('user_id');
        return $this->render();
    }
}

tests/application/ControllerTestCase.php

class ControllerTestCase extends Zend_Test_PHPUnit_ControllerTestCase
{
    public $application;

    public function setUp()
    {
        $this->application = new Zend_Application(
            APPLICATION_ENV,
            APPLICATION_PATH . '/config/application.ini'
        );

        $this->bootstrap = array($this, 'bootstrap');
        parent::setUp();
    }

    public function tearDown()
    {
        Zend_Controller_Front::getInstance()->resetInstance();

        $this->resetRequest();
        $this->resetResponse();

        $this->request->setPost(array());
        $this->request->setQuery(array());
    }

    public function bootstrap()
    {
        $this->application->bootstrap();
    }
}

tests/application/controllers/UserControllerTest.php

require_once TESTS_PATH . '/application/ControllerTestCase.php';
require_once APPLICATION_PATH . '/controllers/UserController.php';

class UserControllerTest extends ControllerTestCase
{
    public function testStubRenderMethodCall()
    {
        $request = $this->getRequest()
            ->setRequestUri('/user/profile/1')
            ->setParams(array('user_id'=>1))
            ->setPathInfo(null);

        $response = $this->getResponse();

        $this->getFrontController()
            ->setRequest($request)
            ->setResponse($response)
            ->throwExceptions(true)
            ->returnResponse(false);

        $controller = $this->getMock(
            'UserController',
            array('render'),
            array($request, $response, $request->getParams())
        );
        $controller->expects($this->once())
                 ->method('render')
                 ->will($this->returnValue(true));

        $this->assertTrue($controller->profileAction());
        $this->assertTrue($controller->view->user_id == 1);
    }
}

You can go further making both the tests and the implementation more sophisticated. The main point is that you can build and test a controller in a way that doesn’t require a view script to be written to do so.

Zend Framework Known Issues

By default Zend_Test_PHPUnit_ControllerTestCase sets the redirector exit value to false, leading to unexpected behavior when unit testing your code. For that reason, make sure you always add a return statement after calling a utility method:

class UserController extends Zend_Controller_Action
{
    public function profileAction()
    {
        if (null == $this->_getParam('user_id', null) {
            return $this->_redirect('/');
        }
        return $this->render();
    }
}

If you want the Front Controller to throw exceptions, you have no other choice than to overwrite the dispatch method and pass a boolean TRUE to the throwExceptions() method:

class ControllerTestCase extends Zend_Test_PHPUnit_ControllerTestCase
{
    ...

    public function dispatch($url = null)
    {
        // redirector should not exit
        $redirector = Zend_Controller_Action_HelperBroker::getStaticHelper('redirector');
        $redirector->setExit(false);

        // json helper should not exit
        $json = Zend_Controller_Action_HelperBroker::getStaticHelper('json');
        $json->suppressExit = true;

        $request = $this->getRequest();
        if (null !== $url) {
            $request->setRequestUri($url);
        }
        $request->setPathInfo(null);

        $this->getFrontController()
             ->setRequest($request)
             ->setResponse($this->getResponse())
             ->throwExceptions(true)
             ->returnResponse(false);

        $this->getFrontController()->dispatch();
    }

    ...
}

The Dispatcher not only violates the DRY principle but also suffers from amnesia. The problem is that it doesn’t store the instance of the Action Controller, instead, it destroys it (Zend_Controller_Dispatcher_Standard Line 305). You can easily get around this issue by extending the standard dispatcher and overwriting the dispatch() method:

class Zf_Controller_Dispatcher_Standard extends Zend_Controller_Dispatcher_Standard
{
    ...

    public function dispatch($url = null)
    {
        ...
        Zend_Registry::set('Zend_Controller_Action', $controller);

        // Destroy the page controller instance and reflection objects
        $controller = null;
    }

This will allow you to access the view object after dispatching the request:

class ExampleControllerTest extends ControllerTestCase
{
    public function testDefaultActionRendersViewObject()
    {
        $this->dispatch('/');

        $controller = Zend_Registry::get('Zend_Controller_Action');

        $this->assertEquals('ExampleController', get_class($controller));
        $this->assertTrue(isset($controller->view));
    }

Links

PHPUnit: Testing Zend Framework Controllers
PHPUnit: Mock Objects

Written by Federico

November 1, 2009 at 1:33 pm

Symfony 1.3 Web Application Development

with 3 comments

Packt Publishing recently sent me a copy of the book “Symfony 1.3 Web Application Development” to review.

This book is not a reference guide, but an example driven tutorial that takes you through the process of building Model-View-Controller-based web applications. You will learn how to create and develop a simple online store application. It also covers best practices for better and quicker application development.

The book is based on the latest version of the Symfony Framework, and does a great job telling you what you get out of the box and how it works, which is perfect for hitting the ground running. During the development you are introduced to the concepts and features of the MVC framework. However, for those who want to know more about the framework, the book doesn’t explain how things work under the covers. This book is more for beginners who want to get started with Symfony 1.3.

One thing I didn’t like about this book is that it uses Propel instead of Doctrine as the default ORM framework. Apart from that, it does a great job explaining and demonstrating with practical examples how to build a Web application from scratch.

Overall, and considering that some of the topics in this book have already been covered in Practical Symfony 1.3, I rate this book 4 out of 5.

Symfony 1.3 Web Application Development

Written by Federico

October 13, 2009 at 11:24 am

Posted in Frameworks, PHP

Database Replication Adapter for Zend Framework Applications

with 10 comments

Last updated: 21 Feb, 2010

Database replication is an option that allows the content of one database to be replicated to another database or databases, providing a mechanism to scale out the database. Scaling out the database allows more activities to be processed and more users to access the database by running multiple copies of the databases on different machines.

The problem with monolithic database designs is that they don’t establish an infrastructure that allows for rapid changes in business requirements. Here is where database replication comes into play. Replication can be used effectively for many different purposes, such as separating data entry and reporting, distributing load across servers, providing high availability, etc.

Zf_Orm_DataSource is a Zend Framework Replication Adapter class flexible enough to support the most commonly used replication scenarios:

Single-Master Replication

In the simplest replication scenario, the master copy of directory data is held in a single read-write replica on one server called the supplier server. The supplier server also maintains changelog for this replica. On another server, called the consumer server, there can be multiple read-only replicas.

Configuration array:

$config = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'username'       => 'root',
    'password'       => 'root',
    'dbname'         => 'test',
    'master_servers' => 1,
    'servers'        => array(
        array('host' => 'db.master-1.com'),
        array('host' => 'db.slave-1.com'),
        array('host' => 'db.slave-2.com')
    )
);

// or ...

$config = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'dbname'         => 'test',
    'master_servers' => 1,
    'servers'        => array(
        array('host' => 'db.master-1.com', 'username' => 'user1', 'password'=>'pass1'),
        array('host' => 'db.slave-1.com', 'username' => 'user2', 'password' => 'pass2'),
        array('host' => 'db.slave-2.com', 'username' => 'user3', 'password' => 'pass3')
    )
);

In the setup above, all writes will go to the master connection and all reads will be randomly distributed across the available slaves.

Multi-Master Replication

This type of configuration can work with any number of consumer servers. Each consumer server holds a read-only replica. The consumers can receive updates from all the suppliers. The consumers also have referrals defined for all the suppliers to forward any update requests that the consumers receive.

$config = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'username'       => 'root',
    'password'       => 'root',
    'dbname'         => 'test',
    'master_servers' => 2,
    'master_read'    => true,
    'servers'        => array(
        array('host' => 'db.master-1.com'),
        array('host' => 'db.master-2.com')
    )
);

Using a distributed memory caching system

Database connections are expensive and it’s very inefficient for an application to try to connect to a server that is down or not responding. A distributed memory caching system can help alleviate this problem by keeping a list of all the failed connections in memory, sharing that information across multiple servers and allowing the application to access it before attempting to open a connection.

To enable this option, you have to pass an instance of the Memcached adapter class:

class Bootstrap extends Zend_Application_Bootstrap_Base
{
    protected function _initCache()
    {
        ...
    }

    protected function _initDatabase()
    {
        $config = include APPLICATION_PATH . '/config/database.php';
        $cache = $this->getResource('cache');
        $dataSource = new Zf_Orm_DataSource($config, $cache, 'cache_tag');
        Zend_Registry::set('dataSource', $dataSource);
    }
}

And here is a short example of how the Replication Adapter might be used in a ZF application:

class TestDao
{
    public function fetchAll()
    {
        $db = Zend_Registry::get('dataSource')->getConnection('slave');
        $query = $db->select()->from('test');
        return $db->fetchAll($query);
    }

    public function insert($data)
    {
        $db = Zend_Registry::get('dataSource')->getConnection('master');
        $db->insert('test', $data);
        return $db->lastInsertId();
    }
}

Source Code:
http://fedecarg.com/repositories/show/replicationadapter

Written by Federico

October 2, 2009 at 3:29 pm