Using Unison to synchronize more than two machines

Rsync is great, however, it only synchronizes files in one direction. Unison, on the other hand, synchronizes both ways. It allows two replicas of a collection of files and directories to be stored on different hosts, modified separately, and then brought up to date by propagating the changes in each replica to the other.

Why you should use Unison instead of Rsync:

  • Unison works across platforms, allowing you to synchronize a Windows laptop with a Unix server, for example.
  • Unlike simple mirroring or backup utilities, Unison can deal with updates to both replicas of a distributed directory structure. Updates that do not conflict are propagated automatically. Conflicting updates are detected and displayed.
  • Unlike a distributed filesystem, Unison is a user-level program: there is no need to modify the kernel or to have superuser privileges on either host.
  • Unison works between any pair of machines connected to the internet, communicating over either a direct socket link or tunneling over an encrypted ssh connection. It is careful with network bandwidth, and runs well over slow links such as PPP connections. Transfers of small updates to large files are optimized using a compression protocol similar to rsync.
  • Unison is resilient to failure. It is careful to leave the replicas and its own private structures in a sensible state at all times, even in case of abnormal termination or communication failures.

Links

Zend Framework Automatic Dependency Tracking

When you develop or deploy an application, dependency tracking is one of the problems you must solve. Keeping track of dependencies for every application you develop is not an easy task. To solve this problem I’ve created Zend_Debug_Include, a Zend Framework component that supports automatic dependency tracking.

We all agree that dependencies cannot be maintained by hand, it’s almost impossible, specially if you are using packages from Zend, Solar, Maintainable and/or Zym (if you don’t know any of these frameworks, check them out, they are pretty cool). Every time you add an include/require statement or create an instance of an object, you introduce a new dependency. And that’s why tracking internal project dependencies can become complex when using a framework.

The concept behind Zend_Debug_Include is that the dependencies for each source file are stored in a separate file. If the source file is modified, the file containing that source file’s dependencies is rebuilt. This concept enables you to determine run-time dependencies of files using arbitrary components. This solution is also useful if you are deploying your application using Linux packages. But, dependency tracking isn’t just useful for deploying applications, it can also be used to evaluate packages. Sometimes packages create unnecessary dependencies and this is something that we need to monitor.

Zend_Debug_Include comes with 3 built-in adapters: File, Package and Url.

Tracking File Dependencies

To track file dependencies you need to create an instance of Zend_Debug_Include_Manager and set the Zend_Debug_Include_Adapter_File adapter. This needs to happen inside your bootstrapper file, before the Front Controller dispatches the request.

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_File());
$included->setOutputDir('/var/www/my-app/dependencies');

/* Dispatch request */
$frontController->dispatch();

This creates a zf-files.txt in your output directory containing all the files included or required on that request. Every time the Front Controller dispatches a request, Zend_Debug_Include checks for new dependencies and adds them to the zf-files.txt file:

/var/www/my-app/public/index.php
/var/www/my-app/application/bootstrap.php
/var/www/my-app/library/Zend/Loader.php
/var/www/my-app/library/Zend/Controller/Front.php
/var/www/my-app/library/Zend/Controller/Action/HelperBroker.php
/var/www/my-app/library/Zend/Controller/Action/HelperBroker/PriorityStack.php
/var/www/my-app/library/Zend/Controller/Exception.php
/var/www/my-app/library/Zend/Exception.php
/var/www/my-app/library/Zend/Controller/Plugin/Broker.php
/var/www/my-app/library/Zend/Controller/Plugin/Abstract.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Standard.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Abstract.php
/var/www/my-app/library/Zend/Controller/Dispatcher/Interface.php
/var/www/my-app/library/Zend/Controller/Request/Abstract.php
/var/www/my-app/library/Zend/Controller/Response/Abstract.php
/var/www/my-app/library/Zend/Controller/Router/Route.php
/var/www/my-app/library/Zend/Controller/Router/Route/Abstract.php
/var/www/my-app/library/Zend/Controller/Router/Route/Interface.php
/var/www/my-app/library/Zend/Config.php
... (63 files in total) ...

To change the name of the file:

$included = new Zend_Debug_Include_Manager();
$included->setOutputDir('/var/www/my-app/dependencies');
$included->setFilename('files.dep');
...

Tracking Package Dependencies

Similar to Zend_Debug_Include_Adapter_File, but groups all the files into packages.

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_Package());
$included->setOutputDir('/var/www/my-app/dependencies');

The code above creates a zf-packages.txt file and adds the following data:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php

Now, if you introduce a new dependency, for example, Zend_Mail:

class IndexController extends Zend_Controller_Action
{
    public function indexAction()
    {
        $mail = new Zend_Mail();
        $this->view->message  = 'test';
    }
}

The next time the Front Controller dispatches the request and calls the index action, Zend_Debug_Include will automatically add the Zend_Mail package and all its dependencies to the zf-packages.txt file:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php
Zend/Mail.php
Zend/Mail
Zend/Mime.php
Zend/Mime

You can then use this information to keep track of dependencies and tell your build tool the name of the files and directories you need to copy and package.

External Dependencies

Here is where everything starts to make sense. Zend_Debug_Include allows you to search for external dependencies as well, you just need to tell the Adapter the libraries you are using. For example:

$libraries = array('Zend', 'Solar');
$adapter = new Zend_Debug_Include_Adapter_Package($libraries);

$included = new Zend_Debug_Include_Manager();
$included->setAdapter($adapter);
$included->setOutputDir('/var/www/my-app/dependencies');

The Solar packages will also be added to the zf-packages.txt file:

Zend/Loader.php
Zend/Controller
Zend/Exception.php
Zend/Config.php
Zend/Debug
Zend/View.php
Zend/View
Zend/Loader
Zend/Uri.php
Zend/Filter
Zend/Filter.php
Zend/Mail.php
Zend/Mail
Zend/Mime.php
Zend/Mime
Solar/Base.php
Solar/File.php
Solar/Factory.php
Solar/Sql.php
Solar/Sql
Solar/Cache.php
Solar/Cache

URL Adapter

If you want to create a different file for each request, use the URL adapter instead:

$included = new Zend_Debug_Include_Manager();
$included->setAdapter(new Zend_Debug_Include_Adapter_Url());
$included->setOutputDir('/var/www/my-app/dependencies');

The URL adapter maps the URL path to a filename. So, if you request the following URI:

http://my-app/blog/2009/02/01

It creates the file blog_2009_02_01.txt.

SVN

$ cd libraries/Zend
$ svn co http://svn.fedecarg.com/repo/Zend/Debug

Building desktop Linux applications with JavaScript

During his keynote presentation at OSCON last year, Ubuntu founder Mark Shuttleworth described application extensibility as an important enabler of innovation and user empowerment. Citing the Firefox web browser and its rich ecosystem of add-ons as an example, Shuttleworth suggested that the Linux community could deliver a lot of extra value by making scriptable automation and plugin capabilities available pervasively across the entire desktop stack.

Mark Shuttleworth also described his strategy for accelerating the adoption of Linux. He discussed the importance of extensibility in open platforms, contemplated the challenges of adapting conventional software methodologies so that they can be used for community-driven development, and contended that the open source software community has the potential to deliver a user experience which exceeds that of Apple’s Mac OS X platform.

Ryan Paul: Building desktop Linux apps with JavaScript

Top Posts 2008

I looked through this blog’s statistics to find out which posts got the most page views in 2008. Here are the top 10 posts:

  1. 30 Useful PHP Classes and Components
  2. Where is the include coming from?
  3. Open-source PHP applications that changed the world
  4. 10 great articles for optimizing MySQL queries
  5. 20 MediaWiki Extensions You Should Be Using
  6. Zend Framework Architecture
  7. Loading models within modules in the Zend Framework
  8. Scalable and Flexible Directory Structure for Web Applications
  9. BBC’s New Infrastructure: Java and PHP
  10. Code Refactoring Guidelines

Each post was written for a reason. “Open-source applications that changed the world” is my favourite one. It not only brings back good memories, but also shows our commitment to the open-source community and our passion for developing and supporting open-source software. Zend Framework was the most popular Web application in 2008. Yes, Dow Jones, HSBC and even the BBC is using it. In case you haven’t notice, I spent the last 12 months promoting the Zend Framework, reason why it gets mentioned in 6 of the posts. I’m also glad that “Code Refactoring Guidelines” made it to the top 10. It shows that the PHP community cares.

NautilusSVN: Linux TortoiseSVN Equivalent

Based on Stuart Langridge’s original script, Jason Field and Bruce van der Kooij created a set of Python scripts which integrate a load of Subversion functionality into the Gnome Nautilus browser. It’s basically a clone of the TortoiseSVN project on Windows.

NautilusSVN currently supports the following functionality:

  • Checkout
  • Commit
  • Revert
  • Diff (using Meld or gvimdiff)
  • Log Viewer
  • Revision and SVN User as columns in Nautilus views
  • Emblems to show file status (though buggy)
  • SSL authentication (buggy)
  • Username and password entry dialog
  • Editing Properties

NautilusSVN Project

Netbooks: Microsoft’s biggest worry

Netbooks will account for about a third of all PC growth this year, according to Citigroup. They are a real threat to Microsoft. Clearly, the future is in netbooks and that has Microsoft worried. Microsoft isn’t just worried about ceding 30% of the netbook market to Linux, it’s also worried that if people get used to running Linux on netbooks, they’ll consider buying Linux on desktop PCs as well.

According to Dickie Chang, this gives users a chance to see and try something new, showing them there is an alternative.

Happy New Year!