Strange. For me, it was the installation and setting up of CUDA on my long-in-the-tooth Ubuntu 14.04 LTS box that led to me upgrading to 18.04 LTS (along with a hand-munged version of the latest gcc, plus an ill-advisde NVidia driver upgrade after both of those that hosed things).
That said, while I do have the latest NVidia drivers now (albeit on an old 750 ti/sc card), I haven't tried to re-install CUDA or other ML tools that I had before (it was done for Udacity's "Self-driving Car Engineer Nanodegree" - to compile code for the assignments, and to utilize my GPU to the fullest, I needed to do a bit of a mashup that subtly broke things - but it worked well enough for the course).
Maybe doing that might show something similar to what you describe?
I know when I did it the first time, I was kinda surprised at how easy it worked out; I had CUDA working with Tensorflow quickly, and it all played nice with the GPU rendering at the same time (we had to do things like train a model using data from a Unity-based 3D driving simulator, so when you ran the model to drive the car, it had to use both CUDA for the model and GL for the graphics, and amazingly it all worked on that poor 750 - and had a decent frame rate!)...