I did the .htaccess thing I found in this post Won’t install EE 2.1 and suddenly it worked. So, problem solved.
EDIT: Just to clarify for people who run into the same problem trying to install ExpressionEngine on Dreamhost: all you need to do is make sure you have an .htaccess file with the line
AddType x-mapp-php5 .php
inside the root folder of your website, that’s the folder named <yourdomain>.com. If you want to place the system folder above the root, place the .htaccess at the same level. For convenience’s sake, I’ve attached an htaccess.txt file to this post. Just make sure you rename it to .htaccess once you’ve uploaded it or else it won’t work.
Other than that, just follow the instructions in the user guide, the links to which I included in my original post. Easy. Perhaps someone could add this to the user guide? My reputation isn’t good enough yet.