You don’t need a GTX 1080. Any modern low-end GPU like RX 550, GTX 1030 or the new Ryzen APU would be fine. Even higher-end Intel iGPUs should work[1].
Basically all you need is at least DisplayPort 1.2 (or 1.2a if you want Freesync) for 4K @ 60 Hz.
The RX 550 has 1.3 support, so it should handle up to 2x4K @ 60Hz with MST (daisy-chained or with a splitter) or a single 5K @ 60Hz.
Some 4K monitors can also work with 2×HDMI, but I don’t think you’ll find that on any low-end card. Maybe high-end used.
As for actual performance, there is a huge gap between what you need for gaming and what you need for office work.
A low-level graphics card from few generations ago with 1GB of VRAM would be fine even (like Quadro 2000), except that it will only be able to drive the display at 30 Hz.
The biggest gotcha with a 4K display is actually your host RAM. If you try to run anything like GIMP, Photoshop, or vector editors in particular, it will gobble up much more memory than usual. Your regular web browser will as well, though to a much lesser degree.
Getting something just a little better, like an RX 560, might be a good idea though, this gives you enough GPGPU power to run most expensive video filters and scalers at native 4K[2].
[1] My Apollo Lake netbook with an entry-level iGPU can handle a 3K display easy
[2] Your source material might be 1080p but you end up postprocessing it at 4K. This might surprise some people.