I needed to automate the task of synchronizing files from one server to another, so I wrote a Phing task. Finally today I found some time to finish writing the documentation.
Overview
FileSyncTask is a Phing extension for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer. FileSyncTask can copy or display directory contents and copy files, optionally using compression and recursion.
Rather than using FTP or some other form of file transfer, FileSyncTask uses rsync to copy only the diffs of files that have actually changed. Only actual changed pieces of files are transferred, rather than the whole file, which results in transferring only a small amount of data and are very fast. FTP, for example, would transfer the entire file, even if only one byte changed. The tiny pieces of diffs are then compressed on the fly, further saving you file transfer time and reducing the load on the network.
FileSyncTask can be used to synchronize Website trees from staging to production servers and to backup key areas of the filesystems.
Usage
There are 4 different ways of using FileSyncTask:
- For copying local files.
- For copying from the local machine to a remote machine using a remote shell program as the transport (ssh).
- For copying from a remote machine to the local machine using a remote shell program.
- For listing files on a remote machine.
The SSH client called by FileSyncTask uses settings from the build.properties file:
sync.source.projectdir=/home/development.com/public
sync.destination.projectdir=/home/staging.com
sync.remote.host=server.com
sync.remote.user=user
sync.destination.backupdir=/home/staging.com/backup
sync.exclude.file=/home/development.com/build/sync.exclude
Listing files
The “listonly” option will cause the modified files to be listed instead of transferred. You must specify a source and a destination, one of which may be remote.
<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
sourcedir="${sync.source.projectdir}"
destinationdir="${sync.destination.projectdir}"
listonly="true"
verbose="true" />
Excluding irrelevant files
To exclude files from synchronizations, open and edit the sync.exclude file under the build/ directory. Each line can contain a file, a directory, or a pattern:
*~
.svn
.htaccess
public/index.development.php
public/images/uploads/*
build/*
log/*
tmp/*
Copying files to a remote machine
The following task definition will transfer files from a local source to a remote destination:
<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
sourcedir="${sync.source.projectdir}"
destinationdir="${sync.remote.user}@${sync.remote.host}:${sync.destination.projectdir}"
excludefile="${sync.exclude.file}"
verbose="true" />
Example
Directory structure
In order to separate the sync settings from the main build file, I’ve created a file called sync.properties:
development.com
|-- build
| |-- build.properties
| |-- build.xml
| |-- sync.exclude
| `-- sync.properties
`-- public
`-- index.php
XML build file
Phing uses XML build files that contain a description of the things to do. The build file is structured into targets that contain the actual commands to perform:
<?xml version="1.0" ?>
<project name="example" basedir="." default="build">
<property name="version" value="1.0" />
<!-- Public targets -->
<target name="sync:list" description="List files">
<phingcall target="-sync-execute-task">
<property name="listonly" value="true" />
</phingcall>
</target>
<target name="sync" description="Copy files">
<phingcall target="-sync-execute-task">
<property name="listonly" value="false" />
</phingcall>
</target>
<!-- Private targets -->
<target name="-init" description="Load main settings">
<tstamp />
<property file="build.properties" />
</target>
<target name="-sync-execute-task" depends="-init">
<property file="sync.properties" />
<if>
<not>
<isset property="sync.verbose" />
</not>
<then>
<property name="sync.verbose" value="true" override="true" />
<echo message="The value of sync.verbose has been set to true" />
</then>
</if>
<property name="sync.remote.auth" value="${sync.remote.user}@${sync.remote.host}" />
<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
sourcedir="${sync.source.projectdir}"
destinationdir="${sync.remote.auth}:${sync.destination.projectdir}"
backupdir="${sync.remote.auth}:${sync.destination.backupdir}"
excludefile="${sync.exclude.file}"
listonly="${listonly}"
verbose="${sync.verbose}" />
</target>
</project>
Execute task
$ phing sync:list
Outputs:
Buildfile: /home/development.com/build/build.xml
example > sync:list:
[phingcall] Calling Buildfile '/home/development.com/build/build.xml'
with target '-sync-execute-task'
example > -init:
[property] Loading /home/development.com/build/build.properties
example > -sync-execute-task:
[property] Loading /home/development.com/build/sync.properties
[echo] The value of sync.verbose has been set to true
Execute Command
----------------------------------------
rsync -razv --list-only -b --backup-dir
Sync files to remote server
----------------------------------------
Source: /home/development.com/public
Destination: user@server.com:/home/staging.com
Backup: user@server.com:/home/staging.com/backup
Exclude patterns
----------------------------------------
*~
.svn
.htaccess
public/index.development.php
public/images/uploads/*
build/*
log/*
tmp/*
(list of files that have changed)
BUILD FINISHED
Total time: 1.9763 second
More information
Related Articles
