The function of a peer reviewed publication is to formalize and verify that some result is correct, reproducible, and relevant (and to advance tenure). One would hope that whether you publish or not is mostly orthogonal to how you conduct your work. If you're doing data analysis in any professional capacity, you should be using mathematically sound techniques, and you should understand those techniques. There's no other way about it. I'm not sure what your line of work is, but I can't think of any technical field where one's analysis techniques being "scientifically sound" is anything but paramount. Why else would the work have any value? Perhaps it helps to think about publishing your work to be more like deploying your code, rather than testing it.
That isn't to say that every step of the way you have to conduct yourself with the utmost rigor. It doesn't mean that you must prove every theorem each time you use it, nor that you have to be laying out strict tests and hypotheses every time you load a data set. But to do data analysis in a scientifically sound manner does mean that you have to understand the background of the techniques you're using, and how they apply to your data. It does mean that you have to periodically bring your analysis back to basics and make sure your ducks are still in a row; that you haven't fallen off the assumption wagon somewhere along the line. It does mean that you should know why something is working, or why something is broken, not just that it is.
"Statisticians" don't sit around all day laying out hypothesis tests in latex. They do the same thing you do; they fire up R, load the data and start playing with it. It's all about the context you have in your brain when you are playing with it. This, I think, is where the formal training becomes quite important. Because as with most things, it's the unknown unknowns that will destroy you, and having the broad formal foundation gives you the tools to protect yourself from walking into a minefield problems you didn't even know existed yet.