When you’re building an infrastructure that is distributed all over the internet, you’ll come to a point where you can’t rely on synchronous remote calls that, for example, synchronize data on 2 servers:
- You don’t have any failover system that resends messages if something went wrong (network outages, software failures).
- Messages are processed over time and you have no control if something goes overloaded by too many requests.
Even if you don’t have to send messages all over the Internet there are enough points of failures where something can go wrong. You want a reliable and durable system that fails gracefully and ensure.
Solutions
Dropr
Dropr is a distributed message queue framework written in PHP. The main goals are:
- Reliable and durable (failsafe)-messaging over networks.
- Decentralized architecture without a single (point of failure) server instance.
- Easy to setup and use.
- Modularity for queue storage and message transports (currently filesystem storage and curl-upload are implemented).
Beanstalkd
Beanstalkd is a fast, distributed, in-memory workqueue service. Its interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running most time-consuming tasks asynchronously.
It was developed to improve the response time for the Causes on Facebook application (with over 9.5 million users). Beanstalkd drastically decreased the average response time for the most common pages to a tiny fraction of the original, significantly improving the user experience.
Zend Platform Job Queues
Job Queues is an approach to streamline offline processing of PHP scripts. Job Queue Server provides the ability to reroute and delay the execution of PHP scripts that are not essential during user interaction with the Web Server. Job Queues improve the response time during interactive web sessions and utilizes unused resources.
Memcached as simple message queue
In this post, Olly explains how to use memcached as a simple message queue:
Some months ago at work we were in the need of a message queue, a very simple one, basically just a message buffer. The idea is simple, the webservers send there messages to the queue, the queue always accepts all messages and waits until the ETL processes request messages for further processing. As the webservers are time critical and the ETL processes aren’t you need something in between.
Federico Cargnelutti 
6 responses to “Getting Started With Message Queues”
What about http://www.danga.com/gearman/ I find that to be the best free one especially when used with the PEAR lib for it.
Yes, I missed that one. Thanks Jon :)
I’ve had a great experience using Dropr — scaling to 10M messages /day from my PHP site.
Zend Server 5 has re-implemented the Zend Platform Job Queue. It’s faster and more stable than the previous incarnation.
http://www.eschrade.com/page/queue-introduction-zend-server-queue-4b8eef5c
thanks for sharing
i am still looking for my weapon of choice when it comes to queues :)
More info here:
Quora: What is the best queueing or enterprise messaging system out there right now?
http://qr.ae/RSZ