Of course cost goes up fairly linearly with size. More rebar, more pipes, more materials, more diesel in the bulldozer, more man hours, more welding rods, more welds to inspect, more money.
But there are also a ton of one per instance overhead paper pushing costs, especially with something as onerously regulated as nuclear that are a large fraction of overall costs and will be roughly the same whether you're building a 1kw facility or a 1gw facility. All of that has to be amortized over your reactor. So if you build a small reactor each dollar of reactor might have $10 of overhead it needs to amortize. If you build a big one each dollar of reactor might only have another $1 it needs to amortize.
Of course there's a sweet spot before the reactor is so massive the part count grows so high that the MBTF starts getting you. That sweet spot also has a low end below which the fixed regulatory and palm greasing costs are so high that your tiny reactor can never hope to pay them back in it's lifetime.