Wednesday, September 14, 2011

Celery task queue with PHP

We have released a Celery client for PHP some time ago. What does it do and why is it useful?

Celery is a piece of software that helps you easily run time-consuming tasks. If you have ever used system() or exec() to create a big ZIP file or encode a video, you probably already know what the problem is. If you didn't - well, time-consuming tasks executed from within a PHP script (or any other web application) usually generate many problems, unresponsive user interface and timeouts being two major ones. There are many ways to overcome these issues, but face it - spending many hours and hundreds of lines of code just to create a ZIP file isn't especially effective.

So how does a basic Celery application to create a ZIP file look like? Let's see:

from celery.task import task
from subprocess import Popen, PIPE, STDOUT

def create_zip(zip_path, files_path):
    command = ("zip", zip_path, files_path)
    return Popen(command, stdout=PIPE, stderr=STDOUT).communicate()[0]

The above code calls the command-line zip command with two arguments: path to a zip file and path of the files to be archived. "def" is Python for "function". (this is just an example, in real world you will probably want to validate paths and replace zip command with Python's zipfile module)

Now you will probably want to call that code from your PHP application. First schedule the task for execution:

$c = new Celery('localhost', 'myuser', 'mypass', 'myvhost');
$_SESSION['zip_result'] = $c->PostTask('tasks.create_zip', array('/home/user/', '/home/user/images/'));

Then you will want to somehow asynchronously display the result to user. You can use AJAX or some other technique to display the result of this script every second:

$result = $_SESSION['zip_result'];
    echo 'Please wait...';
    echo '<a href="/">Ready!</a>';

That's it, as easy as this. Check out the Celery-PHP documentation and try it out.

Also, the examples above are oversimplified - make sure you validate your input data, handle errors and don't unnecessarily poll the server.

No comments:

Post a Comment