Capacitors can be embedded in chip packages, but it adds cost, size and complexity. Bonding two chips inside of a package increases the overall size considerably. You would have trouble connecting from the main IC to pins on one side of the chip, crossing over the capacitor, for example.

Every production board will have capacitors in other places, so removing one of them at the expense of increasing chip size and PCB area isn't a good trade.

For high density designs with high budgets, you can embed capacitors and other passives directly into the circuit board in cutouts. You can also use special capacitance layer substrates to form a big distributed capacitor between two copper planes in the PCB.

There are a lot of options out there, it just doesn't make sense in most cases because putting a capacitor that costs less than $0.01 around the chip is trivial.