I believe that presidents should be measured against the counterfactual of what would have happened had the other guy won - using this heuristic, most presidents don't look so bad.
For example, one of the biggest failures Bush is criticized for is the financial crisis in 2008. While criticism can rightly be levied at George Bush for the massive regulatory cuts and the systemic - almost willful - failure of his administration to recognize the dangers of mortgage-backed deriviatives, there's simply no reason to think similar failures wouldn't have occurred under a Gore Administration or Bill Bradley or John McCain or really any other mainstream politician from the time. Many of the policies that led to the housing bubble in the first place had their genesis as far back as the Reagan Administration and thus can't be laid entirely at Bush's feet.
Similarly, the Bush Administration receives a lot of justified criticism for the massive failures in the response to Hurricane Katrina, failures which critics say had an unmistakable racial quality. However, and again, the failures with the levee system predated the Bush Administration by decades, and while his messaging in the days after the storm was almost horrifyingly tone-deaf, immediately following the storm, many in the media joined Bush in proclaiming that the fears about the Hurricane had been overstated and the response sufficient. Days later, when Bush was unwisely congratulating the FEMA head, the damage was already done, and while the response, even after the fact, is widely considered to have been insufficient, it's not at all clear any other Republican president (or Democrat for that matter) would have done much better. Terrible as it is, Republicans simply don't have a lot of political incentive to spend resources in places like New Orleans. A sad fact to be sure, but a systemic one not limited to George W. Bush.
To this point, I would argue the above complaints along with other things people have mentioned - absolutelyexploding the deficit from the first surplus in decades, greatly exacerbating income inequality with regressive tax cuts aimed primarily at the rich, the aborted Social Security reforms, the Harriet Miers Supreme Court nomination and a few other things - I would argue that these criticisms are mostly partisan in nature. An intelligent person could make a strong argument for limited privatization of social security, or stimulus through cutting taxes, or increasing deficits in lean (though not catastrophic) economic years. That doesn't make these criticisms invalid, but they wouldn't, for me, cause Bush to rise to the level of one of the worst presidents.
Why I think George W. Bush is arguably one of the five worst presidents in our history and inarguably one of the ten worst is one thing: The Iraq War. The Iraq War was a massive failure of the intelligence community, intellectual honesty and transparency in our political system and good common sense. Using my method of measuring the quality of presidential candidates, the Afghanistan War likely would have happened in some form or fashion under a Gore Administration or a McCain Administration or really anyone in power at the time. However, no other politician -none in the 2000 presidential field (Edit: based on subsequent conversation it appears there's at least some possibility the Iraq War happens under McCain) - would have moved for war with Iraq which was widely considered - even at the time - a distraction from the War on Terror and it's not like the War on Terror couldn't be criticized in its own right. The various failures of the Iraq War have been recounted in numerous places: Abu Ghraib, crony contracts to companies like Halliburton, the privatization of key American military operations. But those are all somewhat narrow execution concerns in a chaotic time - war is often messy. To me, the most unforgiveable facet of the Iraq War is that it happened at all.
This is the main crux of why I consider George W. Bush such a bad president: the morally unjustifiable doctrine of preemptive war. Make no mistake, The Bush Doctrine of preemptive war represents the single largest change in American foreign policy since Woodrow Wilson and, along with the Monroe Doctrine, one of the three most dramatic turns in American foreign policy ever. Other presidents who rank among the worst all time usually harm America on the basis of domestic policies: James Buchanan (who failed or address or even properly identify the major cleavages in the lead-up to the Civil War), Warren Harding (who really laid the groundwork for the Great Depression), Andrew Johnson (who tried to roll back basically all of the civil rights accomplishments from before Lincoln's assassination), and Franklin Pierce (who laid much of the groundwork for the Civil War, much earlier even than Buchanan).
Bush is the first to have done his harm primarily by virtue of his pursued foreign policy. George W. Bush sent thousands of Americans and tens or thousands hundreds of thousands of Arabs to die while spending trillions of dollars on a war that is unlikely to have happened if almost anyone else was president (including members of his own party and indeed, even his own family), undertaken in service of a foreign policy that was way out of the mainstream and anathema to 225 years of stated American foreign conduct. However you do your rankings, I have no doubt that one hundred years from now Bush will be regarded as one of, if not the worst post-Depression president.