mastratton3

I've always wondered if consistency would add significant value but it is hard to argue with the cost of free solar power.

I don't know - on the other side of the coin imagine if no data was collected. It would be near impossible for companies to do any sort of troubleshooting or QA.

Taking it a step further, it'd also be significantly more difficult for them to understand their audience and users... which could actually make the experience worse for those users.

Even in the more nefarious case of ad-tech, I'd personally prefer more relevant ads than generic non-relevant ones.

I'm not saying all data collection is fine and well intentioned, but I also don't think it is necessarily as zero-sum as people think.

I do think when data gets licensed or shared w/ 3rd parties it should be more clear how it gets used.

That's not exactly accurate. Transocean was the contract driller on deepwater horizon but BP still took the brunt of the liability and PR backlash. The contract structure actually hasn't changed much in the past 20 years.

That occurred during drilling not production so all the standard well head facilities were not in place.

^This.. I don't see why people thinking hoarding cash when capital is cheap is a bad strategy. It totally ignores the fact that ease of raising and cost of capital is time dependent. If you had a war chest of cash in 2008/2009, that'd be insanely valuable.

This is really good advice. I'd say it generalizes to learning most engineering challenges: Pick a problem you're interested in and solve it from top to bottom, tweaking all components as you move forward.

Isn't something about to be open source the opposite of vendor lock-in? Couldn't you just use databricks and then pretty easily migrate over to your own managed spark cluster if you so desired?

Honestly, if you really rationally break it down, starting a business rarely makes financial sense when factoring in the risk. If you want to maximize guaranteed lifetime income, having a job w/ a high demand skill (That can remain in high demand) is probably the way to go.

except it does carry actual health risks?

lol yes, whats the quote on "Don't assume bad intention when incompetence is to blame"?

After seeing how people write code in the real world, I'm actually surprised there aren't more outages.

This is more of an opinion, I'm sure there are plenty of cases where the founders didn't know each other directly but were introduced due to common interests and it worked out just fine (My current company included). Yea, you probably shouldn't jump right into it, but I don't think it is a bad idea. Also - if you're sticking to your current network, I imagine the chances of finding someone who has more complementary skills is lower (ie: I'm an engineer & most of my network is engineers, what I'd really need though is more of a business/sales guy)

The interesting thing here is that I find many young engineers want to work on the newest tech, however, they also want the 10-5 schedule.

If you want to work on new and exciting, you have to be willing to own the consequences and stay up late digging into the bugs, learning the ins and outs, and committing to delivering what you said would be delivered. </endrant>

TLDR: If you want to work on interesting, it'll take more commitment.

Exactly, would allow companies to hire from pretty much any major city and would allow people to relocate back to where they are from and maintain a great lifestyle and still have some level of camaraderie at work.

I actually think we'll end up somewhere in the middle, where people work remote most of the time but companies may have more smaller offices for people to go into.

My wife works remote a lot of the time (But has the option to go to an office) and when she goes weeks w/o going to the office, the lack of interaction can get to her.

I mean $2 seems fair for them to hold it in inventory and take the risk.

The Wonga Coup - https://www.amazon.com/Wonga-Coup-Ruthless-Determination-Oil...

I do highly recommend the book though, easy read during travel.

I've been thinking about this for a while actually. A lot of the data is available but generally requires going through FOIA which is difficult to do at scale.

This makes so much sense and frequently overlooked. I find even in other areas (such as sports), what makes someone seem really impressive generally isn't what is actually important.

A dumb example is from cycling, people frequently over-practice the ability to sprint at the end of a race but what is really important is the ability to conserve energy throughout the race... Really dumb example, but I think it is somewhat similar.

Meetings - I think anytime someone mentions they won't be able to get things done because of meetings, they should reassess if the meeting is needed at all.

I agree with this mostly, but I've also heard that algorithms questions can be a good proxy for actual software engineering skills. I've never validated this claim but I'm curious if anyone has any research on either side of the argument.

Coming from the other side of the process, even if a question may be arbitrary (Although, I'd argue a lot aren't) it can still be helpful in evaluating a candidate's thought process. Additionally, using the same questions on multiple candidates is a good way to actually cut bias out of the system as you have a lot more data points to evaluate responses against.

Agreed, it'd be tough to leave a company where things are going well to become a contractor at a no-name company. I also think I'd be much less willing to make a jump unless I had met a number of the current employees and founders and felt it would be a culture fit.

Oh great, thanks for the reply. I think thats about where I think we'll land... keep S3 as the primary source, but have HDFS be used for intermediate jobs.

We're actually having a debate now as we're starting to process larger datasets as to whether or not we should keep everything on S3 or start using HDFS w/ Hive. I'm curious if you guys considered HDFS and why you decided to go strictly with S3, and additionally, are there any issues you encounter with S3.

I wish they would have shown comparisons to other states that didn't legalize as opposed to trying to draw conclusions based on the same states prior performance. In some of these cases where legalization didn't cause significant increases, how does this compare nationally?

IMO PhD is useful as a proxy for people who are intellectually curious and are able to dig into a problem until they reach the bottom of the rabbit hole.

I personally think these same skills can be found outside of PhDs.

Interesting, can't enter a birthday before 1916. What if I was over 100?

Yes, there are desktops with high amounts of Ram but to buy a machine like that would probably be more than setting up a hadoop cluster on commodity hardware. And for embarrassingly parallel problem, hadoop can scale semi-seemlessly.

In reality, it still takes work... but can be done.

That answer could go a number of directions depending on your level of experience. I would say find a problem you need a distributed system before and then as said previously, use AWS EMR. If you're more interested in the infrastructure side of things than its always a good experience to setup a cluster from scratch.