If you define u to be a column vector, and u' to be its (conjugate) transpose, and define matrix multiplication as usual, then u'v is a dot product and uv' is an outer product. What you mentioned about the standard basis vectors in R^3 works for standard basis vectors e_i in R^n using this notation; e.g., e_i e_j' is an all zero matrix with a one in the ith row and jth column.
If you have a hard time remembering whether u should be a column vector, you can simply identify things that look like u with vectors (|ket>), and things that look like u' with linear functionals (<bra|). Then u'u is just a dot product.
There are other notations, too... Sometimes (u, v) and u \cdot v are inner products, sometimes u \otimes v is an outer product.
Generally linear algebra-style notation is terser and has less line noise. IMO, some manipulations are simpler to think about using this notation than with the bra-ket notation, especially when you're doing numerical optimization or numerical linear algebra.
Maybe another reason to prefer it is that modern notation for operator algebra follows this notation and not the bra-ket notation (although I think bra-ket notation was invented for this purpose...). If L is a linear operator, L^* is its adjoint, Lu is the application of L to u, etc...
Of course, what notation you should use is determined by what community you belong to. If you're publishing papers on numerical linear algebra, it would be very strange to decide to use the bra-ket notation instead. The purpose of notation is communication. The goal is to communicate ideas to other researchers. It's good to develop the ability to be flexible with notation and go along with what other people are doing.