Right - it's the barrels that infuse a lot of the flavor.
> Many whiskies, especially those that are made on the Scottish island of Isley, have a typical smoky taste that develops when malted barley is smoked on peat fire. Chemically, the smoky flavour is attributed to phenols, and in particular guaiacol, which is much more common in Scottish whiskies than in American or Irish ones.
Other barrels used for scotch are typically fortified wine products, like sherry or port. Occasionally limited releases will be aged in things like ex-rum or ex-brandy casks.