That's right, Cosmic JS apps are fully-built websites and apps.

Extensions are more like WP plugins: https://cosmicjs.com/extensions/