spaznode

Sounds like a good article, wish I could read it on my mobile iOS device.

YAWN

C. For the non-control and more fucking your shit up systems

"You can screw around with that and come to realize that you're completely out of control, you don't know how many threads are involved and exactly how"

I think this was the turning point for me, the day I realized someone had created this large concurrent system with absolutely no idea or care as to what was going on. (the not uncoincidentally same day qa discovered some serious problems making the app become almost useless with extremely high CPU loads)

Python has been by far the most straightforward and easy language for me to pick up and just code and also read out of anything I can remember. Not that I use it a whole lot, it's still beautiful to me.

"That's just conpletely false. I don't know where your assertions are coming from but language design absolutely has a great deal to do with what kind of tools / developers you can expect to find to support the language. I'm all a-huff with the sheer gall and ignorance of your assertion that I'm just not even sure what to do. Bad developer. >:( bad"

Revised: fine ok maybe a bit harsh. My opinion is that language design would be extremely influencial over how people implementing collections/tools/compilers go about doing their work and what the end result of their efforts would look like. In fact they probably are more telling of a design than anything else. That last part was sort of bs guesswork but it sounds right. Happy now?

When's the last time a development team making an internal technical decision became something a company chose to "officially announce"?

"ÜberCorp announced to shareholders today that it decided once and for all curly braces belong on a new line. And that only fumbling retards would choose vi over emacs.."

That's a perfectly good response. Just wanted to know if this was a "people quietly shuffling and re-implementing some core algo" kind of announcement or more general scientific theory.

So, thank you. By all means keep researching and advancing.

Forgive my ignorance, but does anyone know any real world examples where something like this might improve a certain type of work?

I'm sort of fuzzily half-assedly thinking this might be applicable to image - and by extension video - work but maybe that's not the case at all. I want to be more excited, please enlighten me. :)

I work with multi terabyte data sets myself as a software engineer, but after working with these people am not under the mistaken impression that I'm a data scientist of the kind described in the article.

I wouldn't have known this about myself before working with and seeing what they do firsthand of course. That's kind of my main point.

You're right, it was an insult.

It is nonetheless still true.

The thing is that you really ~can~ just say that. Statistical analysts aren't f-ing around when it comes to analyzing data and are capable of some extremely magical things that you can't just "pick up" randomly simply by working on data a lot.

Just like you can't randomly understand the fundamentals of software engineering like data structures & algorithms without first studying the concepts directly.

Hey I'm totally fine if people don't like what they're reading though. It gives me a competitive edge for any company I work for if everyone else is against the idea. ;)

Since I work daily with a group of these "data scientists", I can only assume your comment indicates that you have no idea what you're talking about.

Actually that's exactly the opposite of true. In fact you -need- old data to validate that your mathematical model actually works by applying it to whatever historical timespan of data you are trying to derive intelligence from. At least if any sort of unit of time is applicable to any of your models.

Of course just sitting on the data doesn't do anyone any good at all..and randomly picking at stuff isn't likely to do much either unless you go in knowing what you want to find.

Fuckin god damn right. Use some common sense m-fers. ;)

We wouldn't be having this discussion on the Internet if it weren't for our love of "tits and ass".

Why are we acting so prudish about sexuality? I know my wife wouldn't care about a reference to celebrity t&a. What kind of girls do take offense? I'll tell you what kind, ugly girls. Had to be said.

Probably a good article if it weren't impossible to read due to broken mobile detection "logic" in infinite redirect.

Answer: scroll bars suck

I think his main point was that people are already hoarding but - like most knowledge - don't know what they're missing/clueless about without some good statistical analysts.

My employer has a whole department of these analysts and after working with them closely it is obvious to me that they are the most important / valuable to ourselves and customers asset in the whole company. Engineers can pick up on some of these things but these guys aren't just picking stuff out of their ass or anything. I'd have to agree with the author that any company storing or collecting any significant amount of data should give this stuff some serious thought.

How can we honestly accept anything coming from the people who gave us eclipse? What a giant stinky pile of pooh that turned out to be.

I think that the current stewards of the java language are actually going down the most helpful/likely to succeed path to helping fix java by fixing java. What a novel f-ing concept. As much as it can be anyway, and despite the serious reservations of the giant doucheball they inherited with most of oracle. (despite Larry himself possibly being a pretty cool/legit engineer himself)

I was mildly interested until I discovered that the blog was void of any useful content other than an attempt at re-defining craftsmanship to mean whatever TDD mantra the poster believes in.

Don't know why this is on the front page.

Lol! For the first time ever I ran across a reasonably sized perl based project where I actually understood and could read the flow of things and I've written plenty of perl many ages ago. (even that god awful object oriented style perl they made me use at bellatlantic for "Watson" )

Never had any issues playing around with or reading others emacs lisp macros though. After the initial within within brain fuck part is over at least.

Just look at this. He keeps rolling out what is usually hard earned wisdom gained over years of time of experience while constantly striving to improve yourself and any software you work on.

Do yourself a favor and take the shortcut of listening to this talk..not to say he may not join a cult religion at some future point in time and come out with crazy crackpot ideas then but everything I've seen and read so far are things that all senior+ quality engineers should find some common agreement with.

I can't add anything more to what @jsmcgd said but Rich's strange loop talk really brightened my day and more importantly gave me the tools to express what I sometimes try to share with other developers in the clearest way possible. Thanks man, really awesome talk. Invaluable if more people could start thinking this way. (which it sounds like oracle/java 7,8,.. will also help to do whether they like it or not and that's also awesome for that general clump of dev brethren)

Did you watch the video? He wasn't talking about tools and he wasn't talking about clojure. It was programming in general and if you have to ask if it's worth your time the obvious answer is no.

Ouch! (regarding getting fsck-ed with command line options)

Before current job I built a fairly large more general media management system which included tons of video related work (and encoding ;) ) using ffmpeg for some fairly prominent (something something geographic something / etc) so I definitely appreciate how powerfully awesome ffmpeg is once you get over the initial overwhelming hump of having to learn all the more general video concepts involved outside of ffmpeg.

Which is why seeing a "performance comparison" blog post was immediately extremely suspicious sounding for the very valid point that everyone has brought up that speed isn't all that matters. From my experience speed is one of the lowest concerns anyone doing anything professionally with video has. Quality / reliability / tweak-ability are all much more important.

If you're writing software you intend to sell for money for anything other than cat videos uploaded to youtube - do yourself a favor and make sure you're informed about all the core concepts before randomly picking someone. You'd be surprised at all the hiccups and encoding issues you run across once you have a large enough sample set of "problem videos" to test against.

Where's the shout out to ffmpeg? Don't tell me that almost all services involved aren't built on top of it.

It's true, and we could always use more development help if you're in dc @catalist.us among others..

Not sure what you mean..rtfm

I don't really like to admit it, but anyone who thinks it's just the "poor old algorithm" doing what it thinks is fair is misinformed.

Part of my core day job work involves identifying real vs not real people and also uses the same technology to drive everything - lucene. An algorithm that refuses to attempt identification beyond what data yelp itself stores is clearly in denial of the "mountain" of data available on people in the form of public web apis on the internet. I could write an "algorithm"/analyzer that does a better job than what yelp is currently doing in a day.

At the end of the day the biggest loser in all of this is probably yelp's users. Inaccurate reviews means you're getting inaccurate results on finding places local to you...which means the tool isn't nearly as useful to you as an end consumer. They might want to just sit down and find another way to get the same sales figures without sacrificing the quality of their product. Otherwise all it will take is someone else to come along and give people the right product to put them out of business. These stories that keep appearing on the internet about Yelp are increasingly becoming harder to ignore for everyone, which is sad for all the hard working engineers at Yelp who's only desire is to deliver a kick ass product. Lame