Mean in green
I'm Kevin. I live in Salem, Mass with my wife and little boy and I build software.

Keeping an eye on memory usage

Friday Jan 08, 2010

So, while working on a project recently we were faced with the reality that Drupal was using too much memory for our server to handle. The culprit was an AJAX refresh script that helped us to keep the page content up-to-date without refreshing. We had spent a good amount of time tuning the amount of processing and bandwidth used by the module, but the Drupal full bootstrap was just killing us with a 10 second refresh rate.

The solution we settled on was to not bootstrap Drupal if it wasn't necessary. We implemented a preliminary PHP script that the AJAX request was first passed to. This script opened a MySQL connection and ran a single query to check for updated content. If there was none, the script returned empty JSON. If content was found, then Drupal was fully bootstrapped to load the content and return a full JSON structure with the new content.

The results shouldn't be surprising, but still blew me away. Using the PHP function memory_get_peak_usage() we were able to see just how much memory the scripts were using at the point where the JSON was returned. When new content was found and Drupal bootstrapped PHP used 16961212 bytes (16.18 MB) of memory, but when there was no new content, the intermediate script only used 86968 bytes (84.9 KB) of memory.

Read more for the code...


/* * @file * This file handles the initial AJAX request to ensure * that we are not doing a full Drupal bootstrap just to * find out that there is no new content. If the initial * query fails, immediately return a failed JSON response. * Otherwise, pass along to Drupal for new content. * * Note, this file should be in the same directory as the * Drupal docroot (same dir as index.php). /

// Set up database connection $db = array( 'user' => 'myuser', 'pass' => 'mypass', 'host' => '', 'port' => ':3306', 'db' => 'mydb', );

$link = mysql_connect($db['host'] . $db['port'], $db['user'], $db['pass']); mysql_select_db($db['db']);

if (!$link) { die('Could not connect: ' . mysql_error()); }

// AJAX request variables for initial check $id = $REQUEST['id']; // Either a nid or a tid $mtime = $REQUEST['mtime']; // Last known update timestamp $node = $_REQUEST['node']; // If it is a node or a category

if ($node == 'true') { // If they are looking for a changed node $query = sprintf("SELECT changed FROM node WHERE nid = %d AND changed > %d", mysql_real_escape_string($id), mysql_real_escape_string($mtime)); } else { // If they are looking for additions to a category $query = sprintf("SELECT changed FROM node n INNER JOIN term_node t ON t.nid = n.nid WHERE t.tid = %d AND n.changed > %d ORDER BY changed DESC LIMIT 1", mysql_real_escape_string($id), mysql_real_escape_string($mtime)); }

$result = mysql_fetch_array(mysql_query($query));

if ($result['changed']) { mysql_close(); // If we found an update, set the querystring and // pass along to Drupal $_GET['q'] = 'refresh/page'; require_once './index.php'; } else { // If not, we'll simply return mysql_close(); header('Content-Type: text/javascript; charset=utf-8'); // Note the memory_get_peak_usage() used for testing print json_encode( array( 'mtime' => $mtime, 'mem' => memory_get_peak_usage() ) ); exit; }