Think about the output your API will produce. JSON and/or an XML flavor? What about JSONP support?
And do you want users to identify themselves? How? An API key is the usual way, but Digg has a neat alternative (which I'm using for inspiration on a soon-to-launch API) of each API call providing an app identifier in the form of a URL of a web page describing the app.
Also, don't forget to version your API so that in the future if you implement a change that breaks the default or usual behavior, you can have a chance of maintaining backwards compatibility.
Rate limiting? If you want that, use memcached or Redis or any super-fast in-memory store.
Finally, log everything on the server.
Start with basic GET services. Have users do admin stuff on your website. Then create embeddable widgets that present how to use your API to GET and display data from your service. Use javascript to fetch the JSON, and add it nicely to the DOM.
It's pretty fun actually.
Delicious used simple GETs with a few parameters and you could talk to it with a shell script.
Unfortunately, that was before the days of OAuth, which you really need to support these days. If people build apps that interface to your app, you don't want people typing their passwords in to the other app.
Use JSON and as consistent an interface as you can come up with.
Think through how you'd rebuild your website using only your APIs.