Drupal makes your life very easy, but that's no excuse for being lazy. While profiling scripts today, I ran into something that, for whatever reason, never clicked before. I was running Xdebug and KCachegrind on some complex pages to get an idea where I could optimize. Xdebug has a profiler built in that creates files digestible by something like KCachegrind. With profiling on, you get a profile dump file for every PHP script you run. You can in-turn open that file with KCachegrind to get some helpful analysis.
Profiling is a great way to expose those problems that I often ignore...
So, I ran a few scripts and noticed that there were a whole bunch of profile dumps - way more than the number of Drupal pages I profiled. After a quick inspection, it was apparent that these extra files were coming from 404 "Page Not Found" errors generated by Drupal.
The errors were coming from a CSS image that wasn't found. Normally, an Apache generated 404 would be a trivial issue. However, since Drupal rewrites all requests that are not for files or directories as "q=$1" it does a full bootstrap for every one of those 404 pages. So, in my case, I was doing an extra full Drupal page load for every page request.
So, the moral of the story is: Do not ignore 404 errors being issued by Drupal. Watch those logs like a hawk! With a little bit of cleanup, you can drastically reduce the load on your servers.
Happy day!