The only downsides would be if you need to produce or deal with architecture-specific binaries. For example if you build container images or native code on your machine and ship them up to AWS, Google, etc. to run your service.
With the change to ARM you'll now build an ARM container (or exe, etc.) and it won't run on Intel or AMD-based servers. You'll need to invest time figuring out Rosetta 2, cross-compilation, QEMU, and other multi-architecture workarounds to produce non-native binaries. It's not difficult or particularly confusing once you understand all the moving parts, but it has tripped a lot of people up who thought things would just magically work.
If you're dealing with particularly gnarly messes of complex build systems with obfuscated and obtuse decades old makefile messes it can be quite daunting to add cross compilation and multiarch support. But most codesbases aren't like that and you'll probably be fine. Interpreted and JIT-compiled languages like Python, Node, etc. are generally smooth sailing (unless you link to native code). Modern systems languages like Go, Rust, etc. are very easy to cross compile. With C/C++ you're up against a bit more pain depending on how nice your build system behaves (but let's be honest if you're dealing with a C/C++ codebase in 2021 you're used to some pain).
The inverse problem is also now true. If you pull down a container image, download a binary, etc. it ideally has to be ARM to work. Rosetta 2 can help in the short term to get Intel dependencies working, but again there will need to be some work done to get the dependency moved to multiarch and ARM support.
There's also a major gap in this story if anyone on your team is using Intel machines. There is no good equivalent to Rosetta 2 for Intel--i.e. those Intel users are going to have trouble making an ARM binary or container that your mac can consume. You're going to have to coach them up and walk them through using something like QEMU to achieve this goal (this might not be easy, be warned). IMHO it will be easier to push this load onto your CI system (assuming it can support building ARM) so that people just push source code from their machines and CI takes care of ensuring Intel, ARM, etc. builds are available to everyone.
If this sounds like a lot of issues, try to stay on the happy path of using one architecture everywhere. M1 macs + AWS Graviton 2 instances in AWS is 100% ARM64 all the way and really something incredible once you live in that world. Other cloud providers (ahem Azure?? Google??) need to get their act together and get ARM64 instances available yesterday.