We examined a few possible solutions such as Gearman, popen, and multi curl. However all of these methods seemed to require more overhead than they were worth. What I really wanted to see was something more along the lines of POSIX threads to distribute the work load and shared memory for passing data between the parent and child threads.

After some searching through PHP extensions and the official documentation I ran across PHP’s Process Control Extensions suite which contains PCNTL functions, one of which is pcntl_fork. Combined with PHP’s Shared Memory Functions, this promises to fit the bill of inexpensive distribution of processing tasks along with low-overhead inter process communication.

Here is a sample proof-of-concept script. I’ll outline what it does below:

$data = array();

echo "Parent PID: ".getmypid().PHP_EOL;

function forkTest(array &$data) {
	$pids = array();

	$parent_pid = getmypid();

	for($i = 0; $i < 10; $i++) {
		if(getmypid() == $parent_pid) {
			$pids[] = pcntl_fork();
			echo "Forking child, \$pids now has ".count($pids)." elements".PHP_EOL;
		}
	} 	

	if (getmypid() == $parent_pid) {
		/* Parent thread */
		echo "Hello from parent: ".getmypid().PHP_EOL;
		array_push($data, "parent".getmypid()); 		  		 

		/* Process childrens' results as they exit */
		while(count($pids) > 0) {
			$pid = pcntl_waitpid(-1, $status);
			echo "Attempting to open memory with pid: ".$pid.PHP_EOL;
			$shm_id = shmop_open($pid, "a", 0, 0);

			$shm_data = unserialize(shmop_read($shm_id, 0, shmop_size($shm_id)));
			shmop_delete($shm_id);
			shmop_close($shm_id);

			$data = array_merge($data, $shm_data);

			/* Hunt down and remove pid entry */
			foreach($pids as $key => $tpid) {
				if($pid == $tpid) unset($pids[$key]);
			}
		}

		echo "All children exited, \$data now has:".count($data)." elements".PHP_EOL;
		$pids = array();
	} else {
		/* Children threads */
		$pdata = array();
		echo "Hello from child: ".getmypid().PHP_EOL;
		array_push($pdata, "child".getmypid());
		$data_str = serialize($pdata);

		$shm_id = shmop_open(getmypid(), "c", 0644, strlen($data_str));
		if (!$shm_id) {
			echo "Couldn't create shared memory segment".PHP_EOL;
		} else {
			if(shmop_write($shm_id, $data_str, 0) != strlen($data_str)) {
				echo "Couldn't write shared memory data".PHP_EOL;
			}
		}

		sleep(rand(1,10));
		exit(0);
	}
}

/* Run the test 10 times */
for($f = 0; $f < 10; $f++) {
	echo "Running $f forkTest()".PHP_EOL;
	forkTest($data);
}

echo "Fork test finished, \$data now contains ".count($data)." elements".PHP_EOL;
echo "\$data:".PHP_EOL.json_encode($data);

This code describes a function that spawns 10 child worker threads, each of which gets a reference to the global $data array. Each child thread pushes a string element containing the child thread’s process identifier into the array, serializes it, and then places it into a shared memory slot with the process id serving as the shared memory id. The parent process waits for each child thread to exit, gathers the data from shared memory, clears the shared memory, and then combines the results into the master $data array. My test application runs through this function 10 times to demonstrate how forking in PHP can be safe and memory efficient. The result should be a $data array with 110 elements in it. I’ve thrown in sleep commands with a random time between 1 and 10 seconds to show how threads can return at different times.

No doubt optimizations can be made but this should serve as at least a rudimentary example of true and efficient threading in PHP. Well, provided that the work you are planning on doing is worth the overhead (which, small as it may be, still exists and should be factored in) and provided that you do not mind locking your application down to a POSIX environment (meaning the above code will not work on windows platforms).

I didn’t wan’t to pull this data out of MySQL every time I went to build a map, I wanted to use one giant static array instead. So to write the array back out in a re-usable fashion I developed the following code which can take any array, of any depth, associative or index based, and spit out corresponding PHP that you can copy and paste back into your script.

function phpPrintArr(array $arr) {
	echo "array(".PHP_EOL;

	$c = 1;
	$total = count($arr);

	foreach($arr as $i =>$v) {
		$iq = is_numeric($i) ? '' : '"';
		$vq = is_numeric($v) ? '' : '"';
		$e = $c < $total ? ',' : '';

		if(is_array($v)) {
			echo $iq.$i.$iq.'=>'.PHP_EOL;
			phpPrintArr($v);
			echo $e.PHP_EOL;
		} else {
			echo $iq.$i.$iq.'=>'.$vq.$v.$vq.$e.PHP_EOL;
		}
		$c++;
	}

	echo ")".PHP_EOL;
}

Oh, and for anyone who is interested in the array of country polygons this produced from the Thematic Mapping Engine data, here is the rather large array. Feel free to use it in your own KML project.

Here is the video of the initial announcement along with some juicy technical details:

If you are interested in seeing how your PHP project fares when run through HipHop, check out the official HipHop project page.

With tools like HipHop, scripting languages like PHP are no longer subject to the charges of gross computational inefficiency. In short, it’s a great day to be a web developer.

In less than a week I’ve grown to love the Android platform and I’ve already got a few ideas for some Android apps to write. But first I wanted to make sure I could post links to my apps using handy-dandy QR-encoded images easily within WordPress.

So, borrowing somewhat from the WP-Footnotes plugin I set about to create the first rendition of the WP-QREncoder plugin for WordPress.

This plugin is capable of encoding any string of text (specific use case is a URL) and is still in it’s infancy so I would appreciate any feedback you might have.

Download WP-QREncoder plugin here.

For more information (including usage) see the plugin’s permanent page here.

Oh, and here’s an example of the plugin in action: