Check out the apache config files. For Debian/Ubuntu theyre in /etc/apache2/sites-available/ for RedHat/CentOS/etc they're in /etc/httpd/conf.d/. If you've just installed it, the file in there is probably named default.
Make sure that the config file in there is pointing to the correct folder and then make sure your scripts are located there.
The line you're looking for in those files is DocumentRoot /path/to/directory.
For a blank install, your php files most likely needs to be in /var/www/.
What you'll also need to do is find your php.ini file, probably located at /etc/php5/apache2/php.ini or /etc/php.ini and find the entry for display_errors and switch it to On.