Not quite. A referentially transparent expression (E) is one where you can replace any of its subexpressions (A) with another (B) that has the same meaning (not value!!!!) as (A) without changing the meaning (not value!!!) of E. However, in purely functional languages, the meaning of any expression is a value, but that's the important thing about them, not the fact that they're referentially transparent as imperative languages equally are. We often use the word "semantics" or "denotation" instead of "meaning" in the above, and we say that a pure functional one is one that has "value semantics", i.e. one where the meaning of an expression is a value.
Most programming languages are referentially transparent when not using macros (that was the whole point of talking about referential transparency in programming in the first place), and that's important because it demonstrates both the expressive power and the complexity of macros.