Federico Cargnelutti

Simple is better than complex. Complex is better than complicated. | @fedecarg

Archive for June 2008

Flickr Architecture

with 3 comments

Flickr Platform

  • PHP
  • MySQL
  • Shards
  • Memcached for a caching layer.
  • Squid in reverse-proxy for html and images.
  • Linux (RedHat)
  • Smarty for templating
  • Perl
  • PEAR for XML and Email parsing
  • ImageMagick, for image processing
  • Java, for the node service
  • Apache
  • SystemImager for deployment
  • Ganglia for distributed system monitoring
  • Subcon stores essential system configuration files in a subversion repository for easy deployment to machines in a cluster.
  • Cvsup for distributing and updating collections of files across a network.

Flickr Architecture

Written by Federico

June 30, 2008 at 9:11 am

A Modular Approach to Web Development

with 16 comments

MVC is about loose-coupling, and Modular Programming takes that concept to the extreme. A modular application can dynamically load and unload modules at runtime, completely separate applications in their own right, which interact with the main application and other modules to perform some set of tasks.

In this article you will presented with a different approach to Web development, one that doesn’t focus on a design pattern, instead, it focuses on a programming paradigm known as Modular programming.

Modularity

Modularity is an important design principle, its goal is to design systems so that modules can be optimized independently of other modules, so that failure of one module does not cause other modules to fail, and in general to make it easier to understand, design and manage complex interdependent systems.

Principles

  1. Modular Decomposability: A software construction method satisfies Modular Decomposability if it helps in the task of decomposing a software problem into a small number of less complex sub-problems, connected by a simple structure, and independent enough to allow further work to proceed separately on each item.
  2. Modular Composability: A method satisfies Modular Composability if it favours the products of software elements which may then be freely combined with each other or produce new systems, possibly in an environment quite different from the one in which they were initially developed.
  3. Modular Understandability: A method favours Modular Understandability if it helps produce software which a human reader can understand each module without having to know the others, or, at worst, by having to examine only a few of the others.
  4. Modular Continuity: A method satisfies Modular Continuity if, in the software architectures that it yields, a small change in the problem specification will trigger a change of just one module, or a small number of modules.
  5. Modular Protection: A method satisfied Modular Protection if it yields architectures in which the effect of an abnormal condition occurring at run time in a module will remain confined to that module, or at worst will only propagate to a few neighbouring modules.

Advantages

One of the advantages of modularity, where modules are self contained, is that you can replace or add any module without affecting the rest of the system.

Some frameworks allow you to re-organize your directory structure, but because they were not created with a Modular architecture in mind, they fail to provide most of the advantages that this architecture has to offer.

Example: Traditional MVC implementation

admin/
    controllers/
    models/
    views/
app/
    controllers/
        ContactUsController.php
        NewsController.php
    models/
        ContactUsModel.php
        NewsModel.php
    views/
        contactus/
            index.html
        news/

Using an MVC framework forces you to divide and organize your code according to the framework conventions. Presentation code goes to the view, data manipulation code goes to the model, and the request manipulation logic goes to the controller.

This is all theory, of course, and even though it’s not written anywhere that you should place 200 html files inside a folder called “views”, some systems encourage you to do it. And because everyone is doing it, you think is the right thing to do, right? Wrong.

Modular Programming

Modular programming is the breaking down of a website into parts that are of more manageable size and have a well-defined purpose. Modules should fit intuitively together into a system. In any development environment, modular programming is how larger, more complicated websites are constructed. The first step is to break the task into its basic parts, which leads to defining intermediate steps, and ultimately devising a comprehensive and efficient solution. Modules can be developed individually, validated, and then used throughout an organization, promoting teamwork, efficiency, and innovation. The advantages of modularity are obvious. Code only needs to be written once, which allows quick modifications. It provides a framework that dictates how subsequent programming should be incorporated. Modularity fits an environment where several programmers share work.

Modules

A module is a self-contained component of the system, which has a well-defined interface to the other components and can have its own MVC implementation. For example:

modules/
    home/
        controllers/
        models/
        views/
    contact-us/
        controllers/
            ContactUsController.php
        models/
        views/

A module can also contain configuration data, user defined routes, dependencies information and language translations:

modules/
    contact-us/
        controllers/
        i18n/
            en/
        models/
        properties/
            ContactUsConfig.php
            ContactUsRoutes.php
            ContactUsDependencies.php
        views/

Composability

Composability refers to a collections of modules that can be coordinated and assembled to produce new systems. Pushing this concept a little bit further, we could create a ModuleCoordinator component that acts as a connector between modules. This will allow us to assemble and coordinate new modular subsystems.

Separation of Concerns

In an MVC framework that implements a modular architecture, separation of concerns (SoC) is the process of breaking a module into distinct features that overlap in functionality as little as possible. A concern is any piece of interest in a module. Based on this concept we are going to separate the functionality of a module into two groups, front-end and back-end:

modules/
    contact-us/
        controllers/
            backend/
                ContactUsBackendController.php
            frontend/
                ContactUsFrontendController.php
        i18n/
            en/
        models/
        properties/
        views/
            backend/
            frontend/

The router of the framework will then determine which module, controller, and action of that controller should receive the request. For example:

Front-end Action Controller

The first segment of the URL path maps to the module’s front-end controller. The second segment is usually called “action” and maps to a method within that controller.

URI:        http://www.example.com/contact-us/support/product/iphone
Module:     contact-us
Controller: ContactUsFrontendController.php
Method:     supportAction()
Params:     array('product', 'iphone')

Back-end Action Controller

URI:        http://cms.example.com/contact-us/view-departments/10
Module:     contact-us
Controller: ContactUsBackendController.php
Method:     viewDepartmentsAction()
Params:     array('view', 10)

Content Management

Based on this architecture, you can create a Content Management System (CMS) that allows users to install, activate and assign modules to nodes (or pages). For example:

Page               Module          URL               Action
-----------------------------------------------------------------
- Home             home            home              edit | delete
- News             news            latest-news       edit | delete
- Products         products        products          edit | delete
  + Sales          products        on-sales          edit | delete
  + Shops          map             where-to-find-us  edit | delete
- Contact Us       contact-us      contact           edit | delete

In the example above, we created a vertical module structure, where the module “maps” depends on the module “products” and inherits its properties. They have a parent/child relationship. However, if you want to minimize dependencies between modules, you can create a flat module structure.

Systems like Joomla and other Content Management Systems have adopted similar architectures, this confirms that a Modular Architecture is a good approach to Web development and RAD.

Conclusion

A system architecture should not only be based on a design pattern, such as MVC, it should also be based on different programming paradigms.

In a typical website, a design is implemented so that it meets a set of requirements at the time of development. Often, after a website is delivered, the user will want added functionality, or different users will require custom functionality based on specific needs. In order to accommodate these situations without a complete re-write, a framework that allows for future additions of modules without breaking the existing code base needs to be implemented.

In theory, putting the models in the models/ directory, the controllers in the controllers/ directory and the views in the views/ directory makes perfect sense, but in practice, it doesn’t. Remember, designing a flexible and scalable system architecture is way beyond your directory structure.

Written by Federico

June 28, 2008 at 6:14 pm

Foundations of Programming: Building Better Software

leave a comment »

Karl Seguin has released the official, and completely free, Foundations of Programming eBook.

Although simplistic, every programming decision I make is largely based on maintainability. Maintainability is the cornerstone of enterprise development. Frequent readers are likely sick of hearing about it, but there’s a good reason we talk about maintainability so often – it’s the key to being a great software developer. I can think of a couple reasons why it’s such an important design factor. First, both studies and first hand experience tell us that systems spend a considerable amount of time (over 50%) in a maintenance state – be it changes, bug fixes or support. Second, the growing adoption of iterative development means that changes and features are continuously made to existing code (and even if you haven’t adopted iterative development such as Agile, your clients are likely still asking you to make all types of changes.) In short, a maintainable solution not only reduces your cost, but also increases the number and quality of features you’ll be able to deliver.

Foundations of Programming (PDF)

Written by Federico

June 28, 2008 at 1:02 am

Create professional-looking charts quickly and easily with pChart

with one comment

pChart is a PHP class oriented framework designed to create aliased charts. Data can be retrieved from SQL queries, CSV files, or manually provided. To have a complete overview of what pChart can do for you, we invite you to take a look on the on-line documentation which is trying to show all basic & advanced functionalities of this library.

Written by Federico

June 28, 2008 at 12:26 am

Posted in PHP, Programming, Tools

From Swift Mailer to Zend_Mail

with 8 comments

I’ve recently switched from Swift Mailer to Zend_Mail and to be honest, I’m loving it. Finally someone developed a lightweight, powerful and easy to use email component! Zend_Mail creates and sends e-mail messages with the format text or HTML and it allows e-mail to be sent in an easy form while preventing mail injection. Zend_Mime is a support set for handling multi-part MIME messages.

Simple email with Zend_Mail

$mail = new Zend_Mail();
$mail->setBodyText("This is the text of the mail");
$mail->setFrom("sender@example.com", "Some Sender");
$mail->addTo("recipient@example.com", "Some Recipient");
$mail->setSubject("Test Subject");
$mail->send();

For security reasons, Zend_Mail filters all header fields to prevent header injection with newline (\n) characters. You also can use most methods of the Zend_Mail object with a convenient fluent interface. A fluent interface means that each method returns a reference to the object on which it was called, so you can immediately call another method.

Zend_Mail fluent interface

$mail = new Zend_Mail();
$mail->setBodyText("This is the text of the mail.")
    ->setFrom("sender@example.com", "Some Sender")
    ->addTo("recipient@example.com", "Some Recipient")
    ->setSubject("TestSubject")
    ->send();

Zend_Mail messages can be sent via SMTP, so Zend_Mail_Transport_SMTP needs to be created and registered with Zend_Mail before the send() method is called.

Sending email via SMTP

$tr = new Zend_Mail_Transport_Smtp("mail.example.com")
Zend_Mail::setDefaultTransport($tr);

$mail = new Zend_Mail();
$mail->setBodyText("This is the text of the mail.");
$mail->setFrom("sender@example.com", "Some Sender")
$mail->addTo("recipient@example.com", "Some Recipient");
$mail->setSubject("Test Subject");
$mail->send();

Sending multiple emails per SMTP connection

$mail = new Zend_Mail();
$tr = new Zend_Mail_Transport_Smtp("mail.example.com");
Zend_Mail::setDefaultTransport($tr);

$tr->connect();
for ($i = 0; $i < 5; $i++) {
        $mail->setBodyText("This is the text of the mail.");
        $mail->setFrom("sender@example.com", "Some Sender");
        $mail->addTo("recipient@example.com", "Some Recipient");
        $mail->setSubject("Test Subject");
        $mail->send();
}
$tr->disconnect();

To send an e-mail in HTML format, set the body using the method setBodyHTML() instead of setBodyText(). The MIME content type will automatically be set to text/html then. If you use both HTML and Text bodies, a multipart/alternative MIME message will automatically be generated:

Sending an HTML email

$mail = new Zend_Mail();
$mail->setBodyText("My Nice Test Text");
$mail->setBodyHtml("My Nice Test Text");
$mail->setFrom("sender@example.com", "Some Sender");
$mail->addTo("recipient@example.com", "Some Recipient");
$mail->setSubject("TestSubject");
$mail->send();

How easy was that!? If you haven’t used any of the Zend Framework components yet, then this is a great opportunity to do so.

Articles

Written by Federico

June 27, 2008 at 8:59 am

Posted in PHP, Programming

Is this the future of Web application development?

with 2 comments

Drag and drop widgets to build Web applications, in minutes, with minimal code.

WaveMaker Visual Ajax Studio is an easy-to-use visual builder that enables the drag & drop assembly of scalable, web-applications using Ajax widgets, web services and databases. WaveMaker Studio will look and feel especially familiar to client/server developers who are used to working with visual tools. WaveMaker’s Studio enables data-driven and web-services based applications to be quickly created without complex code, forms, patterns or portal frameworks.

Features

  • Drag & Drop Assembly
  • LiveLayout
  • Push to Deploy: One-touch application deployment
  • Visual Data Binding
  • SOAP, REST and RSS web services
  • Leverage existing CSS, HTML and Java
  • Deploys a standard Java .war file
  • It’s free!

See it in Action

Written by Federico

June 25, 2008 at 10:49 pm

Posted in Programming, Tools, Web Apps

Learn to Build Robust, Scalable and Maintainable Applications using MVC

with 2 comments

MVC is about loose-coupling, and Modular Programming takes that concept to the extreme. A modular application can dynamically load and unload modules at runtime, completely separate applications in their own right, which interact with the main application and other modules to perform some set of tasks

This document (PDF) discusses the classes and interfaces of the PureMVC framework; illustrating their roles, responsibilities and collaborations with simple UML diagrams. The PureMVC framework has a very narrow goal, that is to help you separate your application’s coding concerns into three discrete tiers; Model, View and Controller. In this implementation of the classic MVC design meta-pattern, the application tiers are represented by three Singletons (a class where only one instance may be created). A fourth Singleton, the Façade, simplifies development by providing a single interface for communications throughout the application.

PureMVC Framework

Written by Federico

June 25, 2008 at 9:27 pm

Intrusion Detection For PHP Applications With PHPIDS

with 2 comments

This tutorial explains how to set up PHPIDS on a web server with Apache2 and PHP5. PHPIDS (PHP-Intrusion Detection System) is a simple to use, well structured, fast and state-of-the-art security layer for your PHP based web application. The IDS neither strips, sanitizes nor filters any malicious input, it simply recognizes when an attacker tries to break your site and reacts in exactly the way you want it to. Based on a set of approved and heavily tested filter rules any attack is given a numerical impact rating which makes it easy to decide what kind of action should follow the hacking attempt. This could range from simple logging to sending out an emergency mail to the development team, displaying a warning message for the attacker or even ending the user’s session.

Written by Federico

June 22, 2008 at 10:37 pm

Posted in PHP, Security, Tools

Agile Database Deployment Using Phing

with 7 comments

Phing allows you to use SQL to define changes to your database schema, making it possible to use a version control system to keep things synchronized with the actual code.

A common way to automate development and deployment tasks is by writing shell scripts, however, Phing provides some advantages over shell scripts for task automation. Most of the shell scripts I created so far help me automate the most tedious tasks, from configuring, building, testing and documenting applications to synchronizing files and migrating database schemas. But, having a large collection of shell scripts can lead to a maintenance nightmare, reason why I decided to port some of them to PHP as Phing tasks. I chose Phing because it’s simple, powerful and very easy to extend.

DbDeployTask

Phing offers an optional task for making revisions to a database, it’s called DbDeployTask. Dave Marshall wrote a nice article about it. After adding this task to my build script and testing it, I came to the conclusion that DbDeployTask is not very effective if you want to roll out incremental changes to the databases. Also, it introduces new problems to the database migration process.

For example, if you use DbDeployTask, the SQL script names must begin with a number that indicates the order in which they should be run:

basedir/
  database/
    deltas/
      1-create_foo_table.sql
      2-alter_foo_table.sql
      3-add_constraint_id_to_foo.sql
    history/
      up_20080622120000.sql
      down_20080622120000.sql
  public/

But, what happens when 2 or more developers are making changes to the same database design? How do they know which scripts have been executed? What if they don’t use a central development database, how do they merge individual changes? To be honest, I don’t know.

Agile Database Deployment

A more Agile way to manage database changes is for the developers to connect to a central development database. After making any changes to the database, the developers will have to:

  1. Update the local database/ directory before modifying any script.
  2. Copy the changes made to the database to the corresponding SQL file (up and down).
  3. Check the files back into the repository.

Finally, the changes can be propagated to other databases by executing a script or setting up a Cron job.

This is how the database directory can be structured:

basedir/
  database/
    down/
      alter/
      constraints/
      create/
      data/
    up/
      alter/
      constraints/
      create/
      data/
  public/

To put your database definition under version control, you need an off-line representation of that database. The directory up/ is used when migrating to a new version, down/ is used to roll back any changes if needed. These directories contain all the changes made to the database:

up/
  alter/
    foo.table.sql
  constraints/
    foo.table.sql
  create/
    foo.table.sql
  data/

This directory structure is based on the one suggested by Mike Hostetler, and it clearly offers more advantages. After the team has made all necessary changes, the next step is to deploy the schema. If you are responsible for performing that task, you can get the most recent version of the database project from version control, build the deployment script, manually update that script as necessary, and then run the script to deploy the schema changes into development, staging or production.

Learn More

Database Schema Deployment (PDF) – By Lukas Smith

Written by Federico

June 22, 2008 at 2:45 pm

Symfony’s plugins tell us a lot about what developers need

with one comment

If you want to use the power of the Rails framework without having to learn Ruby, then Symfony is the right framework for you. After spending more than 10 months playing around with Rails, I can say that Symfony is a great alternative to Rails for programmers who already know PHP.

What is a plugin?

A plugin is a packaged extension for a Symfony project. Plugins can contain classes, helpers, configuration, tasks, modules, schemas and model extensions of their own. The symfony autoloading mechanism takes plugins into account, and the features offered by a plugin are available in the project as if part of the framework. The PEAR packaged plugins have the advantage of managing dependencies, being easier to upgrade and automatically discovered.

Installing a Plugin

Since I last used Symfony, there have been great improvements, and the plugin system is one of them.

Stefan Koopmanschap wrote:

For symfony 1.1, the complete plugin system has been rewritten from scratch. This has been done to allow some serious improvements in the way plugins work, and to make it even simpler to work with plugins in your symfony project.

Installing a plugin has become much easier with symfony 1.1. The whole plugin system is now based on a full implementation of PEAR channels, which enables us to use all the advantages of this, such as the easy management (and even installation) of dependencies, or even the installation of plugins from different PEAR channels than the default symfony channel. You could even set up your own plugin channel to easily install the plugins you’ve written yourself!

Learn how to work with plugins in symfony 1.1

Popular Plugins

A quick look at the analytics of the Symfony website tells us a lot about what people need in addition to the Symfony core. Here is a list of the most popular plugins:

  1. sfSimpleCMSPlugin
  2. sfGuardPlugin
  3. sfDoctrinePlugin
  4. sfMediaLibraryPlugin
  5. sfSimpleForumPlugin
  6. sfFeed2Plugin
  7. sfThumbnailPlugin
  8. sfUJSPlugin
  9. sfControlPanelPlugin
  10. sfMogileFSPlugin

If you want to learn more about plugins, how they extend the framework and how you can package the features that you use across several projects into a plugin, read the plugin chapter of the Symfony book.

The plugin system ensures a great flexibility for distribution, upgrade and clean separation with the rest of the framework.

Once again, I can hear the Symfony play, and it’s wonderful.

Read More

Written by Federico

June 21, 2008 at 11:13 am

Posted in Frameworks, PHP, Programming

Follow

Get every new post delivered to your Inbox.

Join 524 other followers