What plotting library were you using? The "first plot of the day" is a bit slow for me, but certainly it doesn't take minutes. I just opened a fresh repl and "using PyPlot" took a little over 5 seconds, and the first plot took around 1.2 seconds. Plots after that are around 0.001 seconds.
Granted, if you were plotting in a small script, I guess every run would take about 7 seconds to get the plotting functionality.