We know aluminum is harmful and implicated in multiple serious health conditions. The only unknown here is the degree of aluminum leeching from the pot to the coffee. For something you do twice a day, this is a different kind of preventable exposure to risk than eating red meat or living in a polluted city.

IMO one would be a fool to accept this risk, even if small.