A short list of helpful services and tools for founders. Please respond in comments if I am missing something that should be added to this list. Feel free to share if this is helpful.
Name and general setup
Instant Domain Search and Panabee are both great tools to help you as you decide on a domain name personal tip: avoid hyphens, words that are difficult to spell or pronounce. Ideally you want a name that is easy for transmission via word-of-mouth, easy to type in on a mobile phone, and easy to remember.
Use Namecheap to purchase your domain. For hosted mail, you can easily sign up for Google Apps for $5/user/mo, which gives you shared storage space via Google Drive, Gmail accounts, and shared calendars.
Use Google Voice (free) to set up a business phone number—you can set the number to forward to all of your phones and send voicemail transcripts to email. For a more full featured service, use Grasshopper (25-50/mo), which has advanced call routing, extensions, and more.
Hellofax for a fax number and faxing through the web.
Testing and analytics
Payroll and benefits