That's fine. I didn't like a lot of the results, but I am quite happy with the one we chose. For $30, it was a really low risk experiment to begin with.
It may cost you more in the long run if you update/replace the logo you got for only $30 now.
The branding and visual elements for your site and company should work with your logo; so whatever branding assets you have (site, business cards, stationary, ads) will all have to be updated if you change the logo later.
I would not consider this to be a low risk experiment.
abeh,
You discount the value of time. If the startup fails anyways, it's a moot point. If it's successful, I will be able to afford it. I'd rather need to pay more money later for a startup that is doing well than spend a lot of money now on a speculative return later.
Sounds logical. If it fails, hopefully the reason for failure is not the logo, or an accumulation of things done quickly and cheaply.