A lot of web hosting companies are offering unlimited storage space, databases, email addresses, domains, and bandwidth for <= $10 per month. This includes:
WebHostingGeeks has comparisons and awards for many different hosting companies.
Other items to take note of:
Carbon Offset
Do they offset 100% of their carbon footprint? 150% of their carbon footprint? 300%?
Support
Do they have 24/7/365 support? Do they have phone support? Can you, as a non-customer, call their support and get a person on the phone within 5 minutes? Do they have a comprehensive online documentation system (wiki is usually effective here)?
Domain Registration
Do they offer a free domain registration (this is usually ~ $10/year additional to the cost of hosting)? Can you have multiple domains registered simultaneously? How many sub-domains can you create?
Cost
How much is the hosting per month when paying monthly, yearly, etc? Does the price change after an introductory period?
If you need a public encryption key (e.g. for an online store), how much is an annual SSL certificate?
Administrative Features
Do they provide SecureShell access (SSH)? What about Secure File Transfer Protocol (SFTP)? Both of these tools make site administration efficient and secure. I use SSH on a daily basis and find it indespensible.
Can you configure Cronjobs? This is usually possible using SSH or from your hosting configuration page.
Do they offer free backup services for your files and/or databases?
Server Side Software
Do they offer easy installation/upgrade tools for common website software such as WordPress, Drupal, Joomla, ZenPhoto, MediaWiki, etc?
What version of PHP is installed ? What version of MySQL is installed? Do they support Python, Perl, Ruby on Rails, etc (depending on your interests)?
Do they support alternative databases such as PostgreSQL, PostGIS, etc?
Can you have long-running processes (this is important for software such as Django, Rails, etc)?
Basically, make sure that the hosting company supports up-to-date versions of common server side technologies. Check the respective websites for each server side technology for up-to-date version numbers. If this is confusing, ask the sales or support representative how up-to-date the servers are.