Php Sync

PHP’s on a public webhosting platforms almost always limit the means of communication between php-servlets, which are needed if one servlet should be notified of changes made by a second serlet. How can one emulate such a notification by other means?

Ideally, the web application employs a message queue and there are subscribers and publishers. Nothing of that is available on cheap web-hosting.

The challenge #

Web-applications with long polling might need to wait for changes, which then should be reported back. On the server one could poll all relevant resource or follow a message queue. As usual, one emulates a message system using a database and an event table. However, most databases don’t offer a “wait for new rows” or “select at_least(1)” construct that would block the request until rows are available. And polling a database is not good.

Pipes #

Second to plain files, pipes are always available. When they exist, normal file operations are sufficient, the call to mkfifo might not be blocked. Pipes are used to transfer data between processes and offer blocking behavior. Content is not stored on disk and readers have to wait for writers.

Usually, there is one process writing into a pipe and another process reading the pipe. However, there can be multiple writers and multiple readers. When a writer flushes a block, that block gets moved to the input of the “next” reader. It might depend on the actual kernel implementation which reader will get that block. Fortunately, it works in my favor under Linux.

Behavior and rondezvous #

A reader can open a pipe for just reading - then the opening call blocks until a writer is available. A writer can open a pipe, that opening call returns immediately, but writing gets blocked until a reader is there.

However, one can open a pipe as a combined reader and writer. The first write will not block, since the written data is buffered at some place. I do not know the internal workings, but it looks like the buffers are provided by the readers, hence they are getting data in the same order as they provided buffers (maybe by the read requests).

Synchronization by pipes #

Here, synchronization is by numbers, newer messages have a higher number. Listeners are up-to-date until a certain number and are waiting for messages with a higher number. Publishers push new messages into a database table and then announce the new highest number.

Implementation #

1. All parts open a pipe in read-write mode.
2. Publishers write the new highest number into the pipe, close the pipe and exit.
3. Listeners write their highest found message number into the pipe and enter the following loop.
4. Listeners read the next number from the pipe - this will block after a while.
5. If the number is the same or lower - just jump back to 4.
6. New highest number received, write it into the pipe for the next listener.
7. Exit the loop.
8. Do the stuff that the listener has been waiting for.
9. Listener exits.

Modification #

A publisher can try to open the pipe for write-only. Writing would block if readers are absent. If possible, that write operation could time out and the publisher exit. That would prevent the edge case that the publisher is writing into its own read buffer even though other listerners are present. Same for listeners, who could open the pipe again in write-only mode to prevent this.

How it works #

When a number is written into the pipe, it circulated throught the listeners in the same order that the linux kernel organizes pipe-readers. The readers that have been notified, put the number onto the pipe and then leave the circle of pipe-readers. The number then gets swallowed by a reader that already has this as its highest number. The pipe is empty and all listeners wait for the next publisher to arrive and push the next number. When no listener nor publisher has opened the pipe, all buffers are dropped.

Snippet #

The following snippet works almost all the time.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

<?php
$stat=intval($argv[1]);
$pid=getmypid();
print("$pid called with $stat\n");
$f=fopen('p1','r+');
fputs($f,"$stat\n");
while(true){
    $l=fgets($f);
    $val=intval(trim($l));
    if($val > $stat){
        print("breaking loop $pid\n");
        fputs($f,"$val\n");
        break;
    }
    print("$pid swallows $val\n");
}
print("$pid from $stat, continue with $val\n");

This is a short PHP script that can be called like php script.php <number>&. Each invocation will start a background php process. All processes with the same highest number will wait. The next call with a higher number will end all previous processes.

However, sometimes… #

In some cases, the new php process reads its own number and swallows it. Then the other listeners are waiting in vain. This is the case here, but in the algorithm outlined above, the publisher never reads its own number. I am not sure if the buffer might be discarded in that case or if other listeners will get it.

Not perfect, but good enough #

In a web chat application, each participants wants to see messages of the others as soon as possible, but if the chat hangs for one post, it is not critical. The reading from the pipe is a good alternative to frequently polling a database.