alanctgardner3

Where do you live in Canada? In southern ontario (where Toronto and most of the people are) there's no snow for 8 months out of the year. And backyards/sidewalks are really only impeded for maybe 2 months (if you're able-bodied). Meanwhile you can totally have a pool, a vegetable garden, a place for your kids to play, etc. the rest of the time.

I recently moved from an apartment to a house downtown with a tiny backyard, and it's a huge improvement. We didn't really use it in February/March. But the rest of the time it's awesome.

Alcohol sales (and drinking age) vary across provinces - I don't think any will let you sell your homemade liquor, but Ontario is very restrictive, while Quebec (for example) is relatively relaxed.

The repo maintainers are going to be on the hook to rebuild every dependency every time any package in the dependency chain changes. That sounds like a nightmare versus the current scenario where only one package gets revved when a library has a bug.

It's important to have role models you can identify with. Recognizing and celebrating the achievements of female directors is important. When you don't include female directors in lists of great movies, when female directors don't win awards, it reinforces the idea that women are allowed to enter the industry, but they can never really thrive and succeed in the same way a man can. Scholarships and quotas help at the top of the funnel, but if the middle of the funnel is still awful and discriminatory you're just feeding more people into the meat grinder.

> I'm a clinical psychologist who was in charge of the largest randomized controlled trial of online cognitive behavioral therapy for depression (n=500).

Peer-reviewed study, please?

> every person is a special snowflake argument

Right off the bat, you sound like a condescending asshole. I would never go to more than one session with you if this is how you treat patients.

> Exercise works really well for curing depression.

You really think you _cured_ someone's depression by telling them to take a walk sometimes? Granted, I find strenuous exercise is really helpful, but there are also days where you cannot will yourself out of bed to go to the gym. Having some asshole psychologist tell you that it's your fault just makes things worse.

> Did you know that by breathing calmly you will cure yourself from panic attacks? Forever.

Yeah, tell this to my girlfriend, who gets _more_ anxious when she's trying to take deep breaths.

> I've been a fairly happy person all my life

Because you never experienced significant trauma, you don't have an imbalance of chemicals in your brain, basically you just won the goddamn lottery. Do you take advice from people who win the Powerball about how to get rich?

> I'm actually working on building free online treatment for depression right now

This sounds really admirable. Looking at the website, it's so cluttered and confusing I don't think anyone could benefit from it. And it seems like you're recommending that people contact you over Telegram? Like anyone? Just your formal patients? This is a giant ethical and HIPAA minefield, offering professional medical services via text message.

Do you have any insight into when Zeller's was good and what went wrong? HBC seems to have done a much better job turning around The Bay locations (albeit very gradually) and apparently they also own Saks?

I'm pretty young and all of my experiences were a combo of dirty, badly stocked stores, products that I didn't care about, bad fluorescent lighting, and employees who didn't give a shit. I also don't really shop at Walmart, but Walmart seems kind of sterile and organized, at least. I honestly did not know why anyone would ever go into a Zeller's - they had clothes and housewares and groceries and stuff, but somebody else seemed to do all of those things much better.

Reach out to some local companies that look interesting? Job search sites are a pretty poor resource in most cases - you'll do better from referrals within your network. Even cold-calling/emailing/LinkedIn messaging people at interesting local companies and trying to meet with them would be worthwhile. They might not have a role, but they'll have connections in the industry who can help you.

If you don't have any EE experience, it might be worth doing a second degree - graduate or undergrad. EE is pretty different from writing code.

I love that people who don't work in tech can still afford to live in Toronto. I don't think we're doing as well as we could to combat income inequality, but I'm glad we aren't throwing billions of dollars at overinflated startups. Hell, the biggest recent Canadian startup success story is Shopify, which actually sells real products and has revenue. Not just "slap some ads on it and we'll turn a profit".

For reference, ~100 people died in terrible conditions building the Hoover Dam. It was a tremendously unsafe worksite. The environmental impact of the project was barely considered.

I think all things considered we've made good progress.

What part of sending things to space isn't "Engineering"? Are you implying SpaceX, ULA, et al. are just winging it sending things to the ISS?

In my experience, almost all overhead in Go programs will result from being bitten by memmove, alloc and GC. Profiling any program that uses interface{} and byte buffers will quickly expose you to how much the runtime loves to reallocate and copy objects. In one instance I got 10x throughput improvement by switching from a generic interface (take an interface{} and detect the type) to using typed methods and hand-rolling a shim that called the appropriate methods.

Powershell!

Seriously, MS doesn't do everything great, but Powershell has typed pipes (typed everything, actually, it's not just stringly like *sh) which are seriously awesome.

French in Quebec is kind of similar - to avoid being assimilated into English Canadian culture, it's very resistant to change. French speakers from Europe tend to find it kind of quaint because it's been developing independently since the 1700s.

I'm pretty sure this is a rephrasing of a much older, well known principle in psychology (they cover it in undergrad), that work satisfaction is a function of: autonomy, feedback and variety.

In other words: people want to control how they do work, and feel they have some flexibility. They want to recieve prompt feedback so they understand when they're doing well or badly. And they want to encounter a variety of challenges.

Gamification is just building a tight feedback loop and providing sufficient variety. Autonomy is much harder to automate, because it seems to be the opposite of building an 'on-rails' experience guaranteed to please someone.

At least for the running example, running marathons is extreme. A lot of people run 6-10 mile distances, precisely because it doesn't take over your life. You might be surprised at how many people you know run and don't talk about it ;)

More broadly, I think you're falling into the trap of assuming the loudest/most prominent people represent the average. Just like there are a lot of people who run and also do other things, there are lots of people who are quietly religious.

I don't even get where being fat comes into this? There are a lot of factors that go into Americans being fatter on average than Europeans, but your list devolved into the typical one-dimensional "I hate North Americans because I moved to Europe and I'm so continental now" rant.

> Aim to be more well-rounded

Good advice for everyone, regardless of where they live.

All major corporate open source projects have a contributor license agreement (CLA) which contributors must sign. Regardless of whether they intend to close the source in the future or not. See the VLC license change fiasco for why CLAs are very good: http://www.jbkempf.com/blog/post/2012/How-to-properly-relice...

Two things stick out:

Catching unused variables is pretty essential, it's easy to shoot yourself in the foot by redeclaring a variable with := inside a block.

>> which means you have to need separate storage for the actual RaceCar and GetawayCar values, either on the stack with a temporary variable or on the heap with calls to new

Maybe I miss his point here, but in Go it's totally valid to just create a new *GetawayCar with &GetawayCar{} in any scope. You can return that pointer from your current method. It's not necessary to explicitly put your GetawayCar on the heap with new(), Go will decide for you with escape analysis.

There's a big difference, calling someone "boy" or "girl" instead of "man" or "woman" is demeaning. How often do you honestly refer to male engineers as "boys"?

> don't care enough to pay attention

Surprisingly, it doesn't matter how much you care.

Just to clarify, in your view women: don't work, can't be engineers, always want to be in a relationship, and are terribly unfaithful and will leave you for somebody with more money? Also, maybe stop calling adult women "girls".

Okamoto (based in Japan) makes some good ones that can legally be sold in the US. http://www.okamotousa.com/

> If Leap schedules things correctly, in part because they are a private company and have incentive to do so

I don't know what you think the TTC does all day, but it's not sit around and say "if only we had competition, we'd make the busses better". It may not be possible to schedule to avoid busses bunching up during peak times, if that's how traffic behaves. The only way to fix it might be to run an excess of under-utilized busses, which cuts into profit margins. Which is something a private company with higher rates might be able to do, but the TTC is limited because service has to be accessible to everyone.

The risk of a private company like this showing up is that it'll decide to focus intensively on the 20% of routes that yield 80% of profit. This bleeds the public transit service of funds needed to run less profitable services at off times that are used by people without 9-5 jobs, or people in less privileged areas. So the rich get better bus service, and no longer subsidize the service for the poor.

Maybe it's just me, but I found the copy insufferable. I don't think I'm the target audience, but there's some irony in hitting a wall of text complaining about how people write too much and think they're Shakespeare. It could be a lot clearer and more direct.

Posted down-thread as well, but Google Wallet worked in Canada when it first launched. It wouldn't accept a Canadian address, but you could pay if you could get money onto the prepaid card. They added progressively more limitations, until eventually it started erroring at Canadian merchants. It's completely on Google, there's no technical incompatibility.

Nope. I had Google Wallet working on my Neuxs S in Canada two years ago, you can use it at any Tim Hortons ;)

Canadian banks do support their own mobile wallets (similar to the ISIS model), but only on a few cherry-picked devices, and with a custom SIM. See http://www.rbcroyalbank.com/mobile/wallet/

All provinces have a comparable professional organization, all of which are members of Engineers Canada. We have the notion of "self-regulated professions", where the government gives a charter to the professional organization to regulate use of their name, their members, etc. For Americans: it's not comparable to the IEEE, it's comparable to the College of Physicians and Surgeons (also a Canada/UK thing the US doesn't really have). So they're given the same power to penalize people "impersonating engineers" as the royal college can penalize people impersonating doctors.

My understanding is it's all corporate accounts. Almost nobody actually goes into the store, but my girlfriend's work regularly puts together orders and has them delivered for any office supplies they need. The retail stores are just a big distribution network, and customers are reluctant to go to Amazon or a smaller supplier because a) nobody ever got fired for buying Staples b) they already have an account there.

> The biggest issue with big data is most of it sits unused

This is really variable. If you're at a place where they jumped on the bandwagon, then yes. There are also lots of companies (and not just Google/FB/LinkedIn) that build mission critical reporting and ML infratstructure on Hadoop. These companies appreciate the value of workflow coordination, and they wouldn't move ahead without (at least) Oozie/Azkaban in place to give some visibility into their workflow.

> But, in the long term, there will be a big change.

I think more types of work will become commoditized. If you just want log processing, there are lots of on-premises and cloud options. Splunk has been doing this forever. Ostensibly with good-enough BI software you could just focus on ingest, and everything else is drag and drop. On a long enough time frame, hand-rolling pipelines will become obsolete. This is like a 10+ year timeline for any player to get significant market share. In the meantime, people have to actually get stuff done, and their skills will be transferable because they understand distributed systems, ETL, warehousing, and a lot of other stuff that hasn't really changed in a decade.

> Becoming an expert in a particular data engineering component

Are you advocating that nobody writes Spark Streaming jobs, because they should rewrite Spark instead? Don't learn to work with Impala, learn to rewrite Impala? I disagree, the tools are only getting better, and it's going to take more and more work to replace the entrenched players. Working on top of solid tools will make you far more productive than engaging in NIH and making your own SQL engine.

> Becoming an expert on quickly and effectively deploying cloud services to get the job done

Like RedShift, EMR and Amazon Data Pipeline? They're hardly turn-key solutions. Amazon's Kinesis is just Kafka with paid throughput - you can absolutely re-use your skills in the cloud, without having to cave and get locked in to a single vendor serving one specific use-case.

> What not to become is one of these OSS DIY bigots: not good enough to build truly differentiating technology, but adamant about building and running their own <up and coming OSS technology>

So in your mind you either pick a vendor to handle all your data for you, or you're an "OSS DIY bigot"? Something like owning your entire user analytics pipeline isn't mission critical for a startup, it's stupid to build it yourself?

> These folks will be wiped out in the next decade or so.

Even though Oracle is amazing and great, lots of people still use Postgres, MySQL, etc. There's always going to be a continuum from "We should buy his turnkey thing" to "we started by rolling our own SQL query engine". You need to be able to identify when each is appropriate, not shoehorn in a one-size-fits-all solution.

There's a lot of stuff under the "Big Data" umbrella: I focused on Hadoop below because that's my focus right now. I'm sure I'm missing some roles here, but the specialities I can think of are:

- getting data out of production systems and transforming it (infrastructure or ETL) - analytical querying and reporting - system administration - machine learning

There's also the wide world of NoSQL data stores, which people lump in with big data, but which require vastly different skills.

The Hadoop VM I linked to above is good for working through exercises for all of the above.

As a starting point, this book[1] walks through the motivation behind Hadoop, and then gets a little into internals and use cases. It's out of date, but you can work through it and get into the right frame of mind, understand HDFS, etc. It's a good starting point.

AMP Camp (that I linked to above) is an introduction to Spark for people with a little Hadoop experience. Spark is getting a lot of attention, you could run into it in a number of roles.

If you're going to be planning the whole pipeline, or doing any sort of infrastructure role, I recommend Hadoop Application Architecture[2] for more modern tools and design patterns. This blog post[3] is a pretty good overview of distributed logs, which are essential for horizontal scale. Understanding Kafka and ZooKeeper is really useful for infrastructure roles, maybe less so for admins.

If you're planning to be in the reporting layer, having a deep understanding of SQL and data warehousing is useful. This book[4] is old hat, but I would say it's expected knowledge for anyone planning a warehouse, and it's interesting to understand best practices. Most places will also expect knowledge of Tableau or a similar BI tool, but that's tougher to learn on your own since licenses are brutal. Visualization with D3 is nice to have in this space, especially if you're coming from a web background - Scott Murray's tutorials [5] are a good starting place.

It's harder to point to resources for sysadmins - if you weren't a sysadmin before, you need to understand a lot of other concepts before you worry about Hadoop stuff. ML is similar - you need to understand the principles and be able to work on a single node. There's lots of good resources out there about getting started in data science.

1. http://shop.oreilly.com/product/0636920021773.do

2. http://shop.oreilly.com/product/0636920033196.do

3. http://engineering.linkedin.com/distributed-systems/log-what...

4. http://ca.wiley.com/WileyCDA/WileyTitle/productCd-0471200247...

5. http://alignedleft.com/tutorials

alanctgardner@gmail.com, feel free :)

If you want to work in a "big data"-type role as a developer, I wouldn't worry about finding huge data sets. There's a dearth of candidates, especially ones who actually have hands-on experience, and having deep knowledge of (and a little experience with) a broad range of tools will make you a pretty good candidate:

Fire up a VM with a single-node install on it [1] and just grab any old CSVs. Load them into HDFS, query them with Hive, query them with Impala (Drill, SparkQL, etc.). Rinse and repeat for any size of syslog data, then JSON data. Write a MapReduce job to transform the files in some way. Move on to some Spark exercises [2]. Read up on Kafka, understand how it works and think about ways to get exactly-once message delivery. Hook Kafka up to HDFS, or HBase, or a complex event processing pipeline. You'll probably need to know about serialization formats too, so study up on Avro, protobuf and Parquet (or ORCfile, as long as you understand columnar storage).

If you can talk intelligently about the whole grab bag of stuff these teams use, that'll get you in the door. Understanding RDBMSes, data warehousing concepts, and ETL is a big plus for people doing infrastructure work. If you're focused on analytics you can get away with less of the above, but knowing some of it, plus stats and BI tools (or D3 if you want to roll your own visualization) is a plus.

[1] http://www.cloudera.com/content/cloudera/en/downloads/quicks... [2] http://ampcamp.berkeley.edu/5/