A domain is how the “name” of a website is called. When you start your startup, I’m sure you want to use www.mystartupname.com. And you are right; it is good to have the name of your business on your web page. It is really important to buy it at the beginning of your business. The main reason is not for having a website.
Having a domain allows you to build a page online, but also to have a professional e-mail. Starting your business with email@example.com might be a problem in the long run. Moreover, it is cheap to buy a web domain and to have a professional e-mail; it is not worthy of starting with a public domain. For $12 you can buy it at Godaddy or NameCheap. Google will charge you $4 a month for the e-mail.
Having a domain will allow you to create businesses accounts in different platforms, keeping personal and business things separated is essential. Otherwise, you will spend some time fixing this in the future. Paying those $5 a month will also give you more Google cloud benefits; Google drive offers you 30Gb of storage, instead of the 15Gb of the free accounts.
In the end, it is all about organisation and cost benefit. A cheap action can save you a lot of time, so no excuses. More than that, contacting people with a professional e-mail will give you credibility. When you are starting, and you have almost nothing to show, credibility is gold.