Both look promising but I've found that Starlette on it's own is very lightweight, well-designed, and straightforward to work with (coming from many years of flask/django).
Some nice features of FastAPI are the automatic response type handling, e.g. "return {'json': 'yes'}" vs "return JSONResponse({})", and the flask-like route decorators. I ended up building the former manually into my project.
Both Starlette and FastAPI are worth checking out, especially if you've spent time building python web apis with django or flask in the past.