Federico Cargnelutti

Database replication is an option that allows the content of one database to be replicated to another database or databases, providing a mechanism to scale out the database. Scaling out the database allows more activities to be processed and more users to access the database by running multiple copies of the databases on different machines.

The problem with monolithic database designs is that they don’t establish an infrastructure that allows for rapid changes in business requirements. Here is where database replication comes into play. Replication can be used effectively for many different purposes, such as separating data entry and reporting, distributing load across servers, providing high availability, etc.

In 2008, Paul M. Jones announced the release of an SQL adapter that allows Solar users to connect to master/slave database installations. My first reaction was: Great! This will inspire other FOSS developers to create similar components. And guess what, it did. I wrote my own ReplicationAdapter. It’s not great, but it’s flexible enough to support the most commonly used replication scenarios:

Single-Master Replication

In the simplest replication scenario, the master copy of directory data is held in a single read-write replica on one server called the supplier server. The supplier server also maintains changelog for this replica. On another server, called the consumer server, there can be multiple read-only replicas.

Configuration array:

$db = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'username'       => 'root',
    'password'       => 'root',
    'dbname'         => 'test',
    'master_servers' => 1,
    'servers'        => array(
        array('host' => 'db.master-1.com'),
        array('host' => 'db.slave-1.com'),
        array('host' => 'db.slave-2.com')
    )
);

// or ...

$db = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'dbname'         => 'test',
    'master_servers' => 1,
    'servers'        => array(
        array('host' => 'db.master-1.com', 'username' => 'user1', 'password'=>'pass1'),
        array('host' => 'db.slave-1.com', 'username' => 'user2', 'password' => 'pass2'),
        array('host' => 'db.slave-2.com', 'username' => 'user3', 'password' => 'pass3')
    )
);

Zend_Registry::set('db_config_array', $db);

In the setup above, all writes will go to the master connection and all reads will be randomly distributed across the available slaves.

Multi-Master Replication

This type of configuration can work with any number of consumer servers. Each consumer server holds a read-only replica. The consumers can receive updates from all the suppliers. The consumers also have referrals defined for all the suppliers to forward any update requests that the consumers receive.

$db = array(
    'adapter'        => 'Pdo_Mysql',
    'driver_options' => array(PDO::ATTR_TIMEOUT=>5),
    'username'       => 'root',
    'password'       => 'root',
    'dbname'         => 'test',
    'master_servers' => 2,
    'master_read'    => true,
    'servers'        => array(
        array('host' => 'db.master-1.com'),
        array('host' => 'db.master-2.com')
    )
);

Zend_Registry::set('db_config_array', $db);

Using a distributed memory caching system

Database connections are expensive and it’s very inefficient for an application to try to connect to a server that is down or not responding. A distributed memory caching system can help alleviate this problem by keeping a list of all the failed connections in memory, sharing that information across multiple servers and allowing the application to access it before attempting to open a connection.

To enable this option, you have to register an instance of the Memcached adapter class:

Zend_Registry::set('db_config_array', $db);
...

$cache = Zend_Cache::factory('Core', 'Zend_Cache_Backend_Memcached', $f, $b);
Zend_Registry::set('Zend_Cache', $cache);

Zend Framework Example

Here is a short and simple example of how the ReplicationAdapter might be used in a ZF application:

class SingleMasterDb extends Zf_Db_ReplicationAdapter
{
    public function fecthAll()
    {
        $db = $this->getConnection('slave');
        $query = $db->select()->from('test');
        return $db->fetchAll($query);
    }

    public function insert($data)
    {
        $db = $this->getConnection('master');
        $db->insert('test', $data);
        return $db->lastInsertId();
    }

    public function update($id, $data)
    {
        $db = $this->getConnection('master');
        $where = $db->quoteInto('id = ?', $id);
        return $db->update('test', $data, $where);
    }
}

Source Code:
http://fedecarg.com/repositories/show/zfreplicationadapter

A Data Access Layer (DAL) is the layer of your application that provides simplified access to data stored in persistent storage of some kind. For example, the DAL might return a reference to an object complete with its attributes instead of a row of fields from a database table.

A Data Access Objects (DAO) is used to abstract and encapsulate all access to the data source. The DAO manages the connection with the data source to obtain and store data. Also, it implements the access mechanism required to work with the data source. The data source could be a persistent store like a database, a file or a Web service.

And finally, a DataMapper is used to move data between the object and a database while keeping them independent of each other. The DataMapper main responsibility is to transfer data between the two and also to isolate them from each other.

Zend Framework Example

app/
    controllers/
        UserController.php
    views/
lib/
    Project/
        Dao/
            Db/
                User.php
            Service/
                User.php
        DataMapper/
            User.php
        Entity/
            User.php
        Model/
            User.php
    Zf/
        DataSource/
            Dao/
                Mapper.php
        Domain/
            Entity.php
        Db/
            Adapter.php

Database Table Structure

CREATE TABLE `user` (
 `id` int(11) NOT NULL auto_increment,
 `first_name` varchar(100) NOT NULL,
 `last_name` varchar(100) NOT NULL,
 PRIMARY KEY  (`id`)
)

The User DAO

The DAO pattern provides a simple, consistent API for data access that does not require knowledge of an ORM interface. DAO does not just apply to simple mappings of one object to one relational table, but also allows complex queries to be performed and allows for stored procedures and database views to be mapped into data structures.

A typical DAO design pattern interface is shown below:

class Project_Dao_Db_User extends Zf_Db_Adapter
{
    public function find($id)
    {
        $db = $this->getConnection();
        $query = $db->select()->from('user')->where('id = ?', $id);
        return $db->fetchRow($query);
    }

    public function findAll()
    {
        $db = $this->getConnection();
        $query = $db->select()->from('user');
        return $db->fetchAll($query);
    }

    public function insert($data)
    {
        $db = $this->getConnection();
        $db->insert('user', $data);
        return $db->lastInsertId();
    }

    public function update($id, $data)
    {
        $db = $this->getConnection();
        $where = $db->quoteInto('id = ?', $id);
        return $db->update('user', $data, $where);
    }

    public function delete($id)
    {
        $db = $this->getConnection();
        $where = $db->quoteInto('id = ?', $id);
        return $db->delete('user', $where);
    }
}

Source Code: Zf_Db_Adapter

The User Entity

An Entity is anything that has continuity through a life cycle and distinctions independent of attributes that are important to the application’s user.

class Project_Entity_User extends Zf_Domain_Entity
{
    public $id;
    public $firstName;
    public $lastName;
}

Source Code: Zf_Domain_Entity

The User DataMapper

class Project_DataMapper_User extends Zf_DataSource_Dao_Mapper
{
    protected $_map = array(
        'id'         => 'id',
        'first_name' => 'firstName',
        'last_name'  => 'lastName'
    );

    public function find($id)
    {
        $dao = new Project_Dao_Db_User();
        $row = $dao->find($id);
        if (!$row) {
            return false;
        }
        return $this->map($row);
    }
}

Source Code: Zf_DataSource_Dao_Mapper

The User Model

The following class represents the User Relational Model:

class Project_Model_User
{
    public function getUser($id)
    {
        $mapper = new Project_DataMapper_User();
        $mapper->setEntity(new Project_Entity_User());
        $user = $mapper->find($id);

        return $user;
    }
}

The User Controller

class UserController extends Zend_Controller_Action
{
    public function viewAction()
    {
        $model = new Project_Model_User();
        $user = $model->getUser($this->_getParam('id'));
        if ($user) {
            $userId = $user->getId();
            $userFirstName = $user->getFirstName();
            $userLastName = $user->getLastName();

            // get an array of database fields and values
            $row = $user->getRow();
        }
    }
}

Keep in mind that ORM tools such as phpDataMapper and Doctrine offer an alternative way of modeling data. ORMs and ERMs are popular with Web frameworks, and the combination of an ORM and DDD makes DAOs redundant, however, it has not been proven to be better than a straightforward approach of implementing a collection of domain-specific data access functions.

That’s all, I hope you’ve found this post useful.

Apache’s mod_deflate module provides the DEFLATE output filter that allows output from your server to be compressed before being sent to the client over the network.

There are two ways of enabling gzip compression:

1. Using Apache’s mod_deflate
2. Using PHP’s built-in functions

Encoding the output and setting the appropriate headers manually makes the code more portable. Keep in mind that there are hundreds of Linux distributions, each slightly different to significantly different. To allow portability the application should not make assumptions about the OS or config involved.

Using Apache

1. Enable mod_deflate

Debian/Ubuntu:

$ a2enmod deflate
$ /etc/init.d/apache2 force-reload

2. Configure mode_deflate

$ nano /etc/apache2/mods-enabled/deflate.conf

#
# mod_deflate configuration
#
<IfModule mod_deflate.c>
 AddOutputFilterByType DEFLATE text/plain
 AddOutputFilterByType DEFLATE text/html
 AddOutputFilterByType DEFLATE text/xml
 AddOutputFilterByType DEFLATE text/css
 AddOutputFilterByType DEFLATE application/xml
 AddOutputFilterByType DEFLATE application/xhtml+xml
 AddOutputFilterByType DEFLATE application/rss+xml
 AddOutputFilterByType DEFLATE application/javascript
 AddOutputFilterByType DEFLATE application/x-javascript

 DeflateCompressionLevel 9

 BrowserMatch ^Mozilla/4 gzip-only-text/html
 BrowserMatch ^Mozilla/4\.0[678] no-gzip
 BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

 DeflateFilterNote Input instream
 DeflateFilterNote Output outstream
 DeflateFilterNote Ratio ratio
</IfModule>

Using PHP

Create a gzip compressed string in your bootstrap file:

try {
    $frontController = Zend_Controller_Front::getInstance();
    if (@strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') !== false) {
        ob_start();
        $frontController->dispatch();
        $output = gzencode(ob_get_contents(), 9);
        ob_end_clean();
        header('Content-Encoding: gzip');
        echo $output;
    } else {
        $frontController->dispatch();
    }
} catch (Exeption $e) {
    if (Zend_Registry::isRegistered('Zend_Log')) {
        Zend_Registry::get('Zend_Log')->err($e->getMessage());
    }
    $message = $e->getMessage() . "\n\n" . $e->getTraceAsString();
    /* trigger event */
}

Reference

Use mod_deflate to compress Web content delivered by Apache

This class, a refactored version of Drupal’s format_interval function, makes it relatively easy to format an interval value. The format will automatically format as compactly as possible. For example: if the difference between the two dates is only a few hours and both dates occur on the same day, the year, month, and day parts of the date will be omitted.

class DateIntervalFormat
{
    /**
     * Format an interval value with the requested granularity.
     *
     * @param integer $timestamp The length of the interval in seconds.
     * @param integer $granularity How many different units to display in the string.
     * @return string A string representation of the interval.
     */
    public function getInterval($timestamp, $granularity = 2)
    {
        $seconds = time() - $timestamp;
        $units = array(
            '1 year|:count years' => 31536000,
            '1 week|:count weeks' => 604800,
            '1 day|:count days' => 86400,
            '1 hour|:count hours' => 3600,
            '1 min|:count min' => 60,
            '1 sec|:count sec' => 1);
        $output = '';
        foreach ($units as $key => $value) {
            $key = explode('|', $key);
            if ($seconds >= $value) {
                $count = floor($seconds / $value);
                $output .= ($output ? ' ' : '');
                if ($count == 1) {
                    $output .= $key[0];
                } else {
                    $output .= str_replace(':count', $count, $key[1]);
                }
                $seconds %= $value;
                $granularity--;
            }
            if ($granularity == 0) {
                break;
            }
        }

        return $output ? $output : '0 sec';
    }
}

Usage:

$dateFormat = new DateIntervalFormat();
$timestamp = strtotime('2009-06-21 20:46:11');
print sprintf('Submitted %s ago',  $dateFormat->getInterval($timestamp));

Outputs:

Submitted 3 days 4 hours ago

Java has no goto statement, to break or continue multiple-nested loop or switch constructs, Java programmers place labels on loop and switch constructs, and then break out of or continue to the block named by the label. The following example shows how to use java break statement to terminate the labeled loop:

public class BreakLabel
{
    public static void main(String[] args)
    {
        int[][] array = new int[][]{{1,2,3,4},{10,20,30,40}};
        boolean found = false;
        System.out.println("Searching 30 in two dimensional int array");

        Outer:
        for (int intOuter = 0; intOuter < array.length ; intOuter++) {
            Inner:
            for (int intInner = 0; intInner < array[intOuter].length; intInner++) {
                if (array[intOuter][intInner] == 30) {
                    found = true;
                    break Outer;
                }
            }
        }

        if (found == true) {
            System.out.println("30 found in the array");
        } else {
            System.out.println("30 not found in the array");
        }
    }
}

Use of labeled blocks in Java leads to considerable simplification in programming effort and a major reduction in maintenance.

On the other hand, the C continue statement can only continue the immediately enclosing block; to continue or exit outer blocks, programmers have traditionally either used auxiliary Boolean variables whose only purpose is to determine if the outer block is to be continued or exited; alternatively, programmers have misused the goto statement to exit out of nested blocks.

What’s interesting is that Python rejected the labeled break and continue proposal a while ago. And here’s why:

Guido van Rossum wrote:

I’m rejecting it on the basis that code so complicated to require this feature is very rare. While I’m sure there are some (rare) real cases where clarity of the code would suffer from a refactoring that makes it possible to use return, this is offset by two issues:

1. The complexity added to the language, permanently.

2. My expectation that the feature will be abused more than it will be used right, leading to a net decrease in code clarity (measured across all Python code written henceforth). Lazy programmers are everywhere, and before you know it you have an incredible mess on your hands of unintelligible code.

But what’s more interesting is that the idea of adding a goto statement was never ever mentioned.

Common sense perhaps?

When you profile a web page with Page Speed, it evaluates the page’s conformance to a number of different rules. These rules are general front-end best practices you can apply at any stage of web development. Google provides documentation of each of the rules, so whether or not you run the Page Speed tool, you can refer to these pages at any time.

The best practices are grouped into five categories that cover different aspects of page load optimization:

• Optimizing caching: Keeping your application’s data and logic off the network altogether
• Minimizing round-trip times: Reducing the number of serial request-response cycles
• Minimizing request size: Reducing upload size
• Minimizing payload size: Reducing the size of responses, downloads, and cached pages
• Optimizing browser rendering: Improving the browser’s layout of a page

Web Performance Best Practices

This manual gathers together the key insights into API design that were discovered through many years of software development on the Qt application development framework at Trolltech (now part of Nokia). When designing and implementing a library, you should also keep other factors in mind, such as efficiency and ease of implementation, in addition to pure API considerations. And although the focus is on public APIs, there is no harm in applying the principles described here when writing application code or internal library code.

The Little Manual of API Design (PDF)

Here is a categorized list of online programming books available for free download. The books cover all major programming languages: Ada, Assembly, Basic, C, C#, C++, CGI, JavaScript, Perl, Delphi, Pascal, Haskell, Java, Lisp, PHP, Prolog, Python, Ruby, as well as some other languages, game programming, and software engineering. The books are in various formats for online reading or downloading.

Free Programming Books

This book shows you how to build websites fast using PHP and MySQL. What’s more, it shows you how to supercharge your use of these technologies by taking advantage of a powerful, free web development solution known as the Zend Framework, which helps developers build websites with speed and efficiency.

This book embraces a teaching strategy of learning by doing, showing you how to build website features you’ll actually want to use within your own websites. Among other things you’ll learn how to manage data submitted through web forms, send unformatted and HTML e-mails through your website, manage user registrations, logins, and recover forgotten passwords, and even create the structure for a simple social network.

Along the way, you’ll learn how to take advantage of popular online services such as Google Maps, Amazon Web Services, the Facebook Platform, and PayPal to create even more compelling websites.

Easy PHP Websites with the Zend Framework

Come and join Yahoo! on the 9th and 10th of May in London for 24 hours of learning, hacking, networking and fun.

It’s a two-day event on Saturday 9th of May with a morning of Tech Talks covering a wide range of topics led by some of the Web’s most respected developers. Yahoo will showcase the newest platforms and developer tools for you to play around with, and there will be plenty of experts on hand to answer questions.

Sign Up Now