There are definitely cost and other considerations you have to think about when going multi-AZ.

Cross-AZ network traffic has charges associated with it. Inter-AZ network latency is higher than intra-AZ latency. And there are other limitations as well, such as EBS volumes being attachable only to an instance in the same AZ as the volume.

That said, AWS does recommend using multiple Availability Zones to improve overall availability and reduce Mean Time to Recovery (MTTR).

(I work for AWS. Opinions are my own and not necessarily those of my employer.)