tom_b

I've never come across a great book of relational models specific to domains.

My favorite general db book is Database Systems: The Complete Book (DS:CB), by Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom. This is a good book for learning about relational modeling and how to normalize a model so that your eventual schema has as few dependencies between entities as possible. I taught an introductory class where the required (by the department) textbook was Foundations of Database Systems by Elmasri. It's definitely not my favorite book, but if you skim chapter 3 on entity-relational modeling and then chapter 9 on converting an entity-relational model to a relational model suitable for an RDBMs, that is a useful starting place. When I was teaching the class, I often had to go back to the Garcia-Molina,Ullman, Widom book for simple and clear language around concepts for my lectures.

I'll second the recommendation for Joe Celko's books for general SQL programming.

I personally like collection+json as a response type for REST. I will say that my own efforts at using a hypermedia-containing response type for a REST api shared across several teams have been mostly wasted effort. Consumers of the api treat the responses as plain-old JSON which somewhat kills the advantages of having a proper REST response in the first place . . .

Requirements were to allow internal and external teams the ability to CRUD a small set of resources using multiple programming languages, OS, and include command line scripting ability.

Direct access to our HIPAA/IRB database instances are based on white-lists - most internal and all external teams are prohibited from direct database connections. So for those groups, there is no possibility of any SQL access to schemas.

We use API keys with a shared secret (like AWS) to sign REST requests. If internal and external groups need to work together to jointly CRUD some resources for a specific project, they can share an API key dedicated to that project.

There was also a push to switch from an app-centric view of data management to an API-driven approach serving hypermedia (collection+json). That worked conceptually, but most API users ignore the embedded hypermedia links. I find that too bad, as those links make generic API consumption clients much more resilient to change - following a link embedded in a response rather than relying on POST to a well-known URI is a useful abstraction.

I have had some success stealing the immutability ideas out of Datomic and sticking them into normal RDBMs. I generally add a timestamp and a deleted flag to normal entities in the schema. Updates or deletes result in new tuples with later timestamps - in the case of deletes a "deleted" column gets updated to 'Y'. All read queries are actually executed against views defined on top of the entities where the max(timestamp) is used to select the most recent instance of an entity along with a check for deleted='N'. If the most recent tuple for an instance has been deleted, it won't show up in the results.

This actually works pretty well, but I have a REST API fronting the whole setup - programs are not directly accessing the underlying table or views directly.

I also did not hear back after doing the challenge in December using Clojure. But I also indicated in my cover email that I probably wasn't a good candidate if relocation was a must and that could have easily been a deal-breaker for them.

My experience using environment variables as recommended above has been positive. I currently maintain a Clojure web app (a REST server really) and have found that maintaining configuration options, like a keystore and keystore password, in environment variables is really nice. Having the exact same code base in dev, test, and production with separate configs specific to the environment simplifies dev life.

The one issue that has popped up here with the env var approach is that our system administrators are not used to the idea and this creates friction when deploying new apps or updated certs.

Cool - thanks to all three of you for Starfighter. I'm sad it didn't work out as envisioned. Looking forward to the eventual write-up when it fits into your time.

The founders wound it down. Thomas is threatening to actually do a write-up about the experience sometime (poke, poke).

I think the clearest answer is Dweck is saying that misunderstanding her concept of the growth mindset leads to a "false growth mindset" rather than strictly defining the term.

FTA:

  But Dweck recently noticed a trend: a widespread 
  embrace of what she refers to as “false growth mindset”—
  a misunderstanding of the idea’s core message. 
  Growth mindset’s popularity was leading some educators 
  to believe that it was simpler than it was, that it 
  was only about putting forth effort or that a teacher
  could foster growth mindset merely by telling kids to 
  try hard.

The larger concern seems to be that well-meaning educators have co-opted the growth mindset terminology and warped the idea in ways that are directly problematic or directly fixed-mindset in nature.

Just anecdotally, I have friends who have been very successful with buying and renting condos/apartments in a college town. Demand for rental units stays high (well-known, reputable public university).

Oddly enough, they have generally negative experiences renting to parents of students instead of directly to students.

I will say that my friend works full time as the property manager for their units. It's not a passive investment.

Several years ago, I talked with an accountant who had clients who owned rental property about the ROI. The feedback was that if you had to hire a property manager and outsource all repairs/maintenance, the value proposition of rental property was greatly reduced and closer to neutral than many owners initially assumed.

It looks like it is working out well for my friends though - they were able to leverage their financing as they acquired new units because interest rates have been so low. They have been cash-flow positive for many years. But they also capped themselves at about 20 units under management.

I was taken aback when I realized just how few people understand the United States progressive tax system and how income tax brackets work. That misunderstanding is so pervasive (in my experience) that I can't even imagine people having a general understanding of how deductions work at all.

Oh, I default to using the hacker label as a compliment to programmers here on news.yc. I tend to assume (based on reading bits of code from various news.yc contributors) that many devs (carefully not bucketing myself in this group) here exhibit a level of mastery above a run of the mill corporate programmer.

I laughed too - but also wouldn't mind working professionally in a Clojure-oriented shop despite having some freedom to use Clojure now. But I'm a lone wrangler of parentheses in my current role and if I could (in my location) join a team of better Clojure hackers, I think I could take my skills up a level or two even if all I had was the opportunity to talk shop in the hallway.

Interesting that Bay Area hackers make more than local hackers when they relocate outside the Bay Area.

FTA:

  A 2015 report by Hired found that when engineers from 
  the Bay Area relocate to other areas, they out-earn
  engineers on the local market. Experience in the Bay 
  Area seems to advance careers. Engineers moving from 
  San Francisco to Seattle make an average of $9,000 
  more than others who get offers in Seattle. This Bay 
  Area premium is even higher in other cities: $16,000 
  in Boston, $17,000 in Chicago, and $19,000 in San Diego.

[found slide at http://get.hired.com/rs/348-IPO-044/images/Hired-State-of-Sa...]

Bay Area hackers are more valued in different markets than local hackers. I would love to see the raw data for the "relocating" hackers and local hackers. Is it a question of applied experience opportunities in the Bay Area hackers? Is just startup afterglow? Are relocating hackers better than average pre-Bay Area experience to begin with and this shows up when they migrate away from the Bay Area?

As a youth baseball coach, I have noticed that none of the players talk about a favorite baseball player or team. We're cord-cutters and my own kids never even mention watching a game. Not a good sign for MLB.

Lack of obvious career development opportunities. Concern over the viability of the business. Discomfort with idea that projects were frequently cost-centers without tangible benefits for customers.

If it is an option for you geographically, look around at college town school districts where maybe there is some remaining socioeconomic diversity (e.g, the whole school district isn't made up of employees of the university).

My kids are in a school where more than half the kids are non-white and around half the kids have low socioeconomic status (SES) based on free/reduced-price lunch numbers despite our district being high-SES compared to other districts in our state. We have a good base of racial diversity although that diversity is mostly Hispanic (dual-language school).

A teacher at our elementary school once said that socializing across SES normally happens only if parents make a point of providing opportunities for that to happen with the kids. Kids have lots of birthday parties at local parks - I've attended several of these gatherings where the kind-hearted bilingual parents switched over to English so that I could be a part of the conversation - while my kids are bilingual, I am a typical American and only speak English.

As both a parent of participants and as a coach in youth sports, I'll say the rec leagues sometimes provide good opportunities for that mixing of different SES kids. It often falls apart though - we have youth sports leagues where the participation and lost opportunity costs due to heavy practice/travel time seem to screen out low-SES kids.

It's kind of funny, but my family basically lives in small-town America but has wound up in a better school district in our state that also has some racial and SES diversity in the population. It's working as both a good school district and opportunity for kids of different SES and race to grow up together. Longer term, problems with affordable housing are probably going to blow some of these opportunities away.

Anecdotally, my personal network and I have found the reluctance to direct hire into senior dev jobs the same across the country in RTP, NC. I'm not sure it is really related to proprietary tech here, but rather reflects uneasiness in hiring orgs to trust their hiring process to actually identify senior-level/capable devs.

Hiring groups are just much happier to hire at lower levels and promote up to retain better devs.

I have semi-copied what AWS did a couple of years ago. Each api consumer has an api key with a private key known to the consumer and server-side. Requests are SSL only and are signed with the private key by the consumer side (including any parameters and a client-side timestamp). For example, using curl:

  #!/bin/bash

  TS=$(date +"%s000")

  SIG=$(echo -ne 'GET\nhttps://servername:port/api/resources?api_id=1&ts='$TS | openssl dgst -sha256 -hmac "secret-key" -binary | xxd -p -c 32)

  curl -H "Content-type: application/vnd.collection+json" 'https://servername:port/api/resources?api_id=1&ts='$TS'&signature='$SIG

I have consumers of the api using node.js and Java as app dev languages with various http client libraries successfully. The actual server-side api is written with Clojure using the Liberator library (highly enjoyed working with this combo).

Server-side uses the api_id to check the signature using the shared secretkey. (Edit to clarify). There are two timestamps flying around here. One is a timestamp from the client call to the api ($TS above). This timestamp has to be "close" - completely configurable on server-side - to the server-side time or the request is not valid. A little more subtle is that each resource has a timestamp that is the last time the resource was changed server-side. As part of the authorization for the api call to change a resource, it has to be make the call to the api with the resource's most current value for that timestamp to succeed. This is a little goofy logically.

This approach works very well when you have a requirement to allow multiple different apps to call the api rather than say, allowing users to call the api from their browser. I don't have a case where users are in a web browser app that directly calls the REST api.

By the way, if you are using SSL, you don't have to worry about the api_id being exposed in either the query parameters or the request headers.

I agree with the above recommendation.

I have read John Cook's blog for some time and think he might be a good fit as a consultant for bringing you up to speed and reviewing your current teams' work.

http://www.johndcook.com/blog/

I suspect it would be worth your time to get a consultant like him to come in for a week, review all the work going on in your org, and maybe lay out a learning plan or crash course in developing a safety net for you.

Adjunct side-gigs can be fun. Teaching students who started at zero and wound up quite capable of sophisticated hacking was very rewarding for me. If you have been curious about teaching and find an opportunity, I highly recommend trying it just for this aspect of teaching.

But I taught a course as an adjunct and the preparation time totally killed me. I wound up re-working the entire course from scratch. Worse, the course started almost as soon as I was asked to teach. By the end of the course, I was often prepping for the class the night before lectures.

Despite promising myself to not repeat the experience, I have just accepted another adjunct lecturer side-gig for the spring. The main difference is that excellent and up-to-date materials already exist. I'm hopeful these materials make the course prep less intense.

I am curious about demand for this skill in the market.

But I just don't see it - machine/statistical/deep learning gigs just seem really rare.

I know this isn't a great metric, but searches on Indeed.com:

  "deep learning" - 873
  "machine learning" - 9,762
  "statistical learning" - 65
  java - 72,802
  javascript - 43,785

Same searches on LinkedIn:

  "deep learning" - 646
  "machine learning" - 6,952
  "statistical learning" - 34
  java - 43,845
  javascript - 30,818

Even the "machine learning" search on Indeed, with 9K+ results has 1300+ from Amazon, followed by a much smaller number (in low hundreds each) from Microsoft, Google, others (including some that look like staffing companies).

Even on HN's who's hiring Sept 2016 thread phrase counts: 14 "deep learning" 79 "machine learning"

I completely agree with the idea that being able to use some deep/machine/statistical learning is going to be a toolset that data hackers need to have. I even think that there is a bit of the "build it and they will come" magic waiting out there.

But I think the best way forward is to be working in data and figure out how to generate value with deep learning - this will be much more productive than trying to seek out a deep learning gig in terms of promoting deep learning in the workplace. Heck, that's a suggestion I would be wise to take myself . . .

Hi,

I found Larry Harris' Trading and Exchanges: Market Microstructure for Practitioners a solid introduction to market making and trading. Terms and concepts are easy to pick up from the text. I was comfortable enough after reading it to skim stats journal papers talking about market making models. The Stockfighter team had mentioned it in older threads here. It's expensive, but I just borrowed it from the library at my university instead of buying.

I also like The Elements of Statistical Learning which is free from the authors (http://statweb.stanford.edu/~tibs/ElemStatLearn/download.htm...). Although it isn't specifically about economics or markets, you should at least read it.

I'm at a loss on general economics books.

No, not really. When my purchase was made, the Macbook Pro had the nicest screen and form factor with 16GB of memory.

I debated the Carbon X1 at the time, but I recall it was limited to 8GB of memory. I regularly work with Java-based ETL tools and datasets that spill out of 8GB, so having 16GB was important. I could have gotten more memory if I wanted a bigger form factor but I really didn't want to lug a "workstation replacement" around.

A couple of years ago I said if the Macbook lasted as long as my Thinkpad X200 did (4-5 years), it would be a reasonable purchase.

If I was buying today, I might just buy some type of large screen Chromebook and do everything on some cloud VM instance with even more memory.

I run Debian and XMonad on a Macbook Pro with 16GB of memory. Nice machine. I guess it would still dual boot to OSX, but I haven't tried for more than a year.

I would recommend starting with Stephen Few's books: Show Me the Numbers and Information Dashboard Design: Displaying Data for At-a-Glance Monitoring to get an idea of visualization from a business needs perspective. These are easy reads.

Few is heavily influenced (and cites) Edward Tufte - Tufte is probably the definitive reference for communicating data with visual techniques. If you are interested in pursuing scientific visualization, you probably need to spend some time with Tufte. I never have properly done this myself.

These sources will give you a solid foundation without being tied to specific tech. You'll probably find lots of libraries built on top of d3.js that implement the basic ideas of both authors. I have been meaning to look at data exploration examples (e.g., stuff with visualization examples in R) and translate to different front/backend. That might be neat too.

Build some examples where the visualization provides real insight to a defined business problem and you will probably score some new opportunities for yourself.

Good luck.

Thomas, were many starfighters you referred not getting offers? I love the idea of a (properly scoped) project/challenge in the hiring process and find myself profoundly worried when no code gets discussed in tech interviews.

I also really enjoyed stockfighter challenges. Between those six and my university's copy of Harris' Trading and Exchanges, I learned quite a bit. I bailed out on the AVR challenges though.

From a purely selfish viewpoint, I also want to work at places where the hiring process isn't really just whiteboard or gotcha questions, but instead actually has some consideration that, you know, demonstrating the ability to produce working useful code matters. Maybe I'm just nuts. The last interview I was in (for what people here would say is data science/engineering) involved a couple of big-O questions and two ill-specified SQL queries . . . sigh.

Ignoring the breathless nature of the article, this is a buzzword label for a commodity skill set that pays a commodity salary in tech. It is also the commodity skill set that my employers have all paid me for.

There has been for a long time hype around new technology and labels for business intelligence, data warehousing, big data, and now data engineering/science. I'm not saying there are not some roles in this space that return huge value to organizations, but that these opportunities are much rarer than the buzz indicates.

I wonder if the perceived shortage is mainly hype as the shift to new cloud technologies makes many of the older ideas a little less useful - if you are plowing data into BigQuery, you probably aren't so worried about your star schema data model for reporting.

I would strongly advise people that look at these types of articles to look at the roles in question and ask "Is this role on the critical path to customers paying us?" My experience has been that the answer is often "No." This is bad. I have also seen situations where businesses that do rely on smart data integration can show that they are selling dollar bills for ten cents that still have trouble getting customers on board with spending that ten cents. Business is weird.

Credits rarely transfer into a major or minor program of study regardless of source institution. Waiving of requirements to opt into higher-level coursework is fine. That's rarely what students want.

ITT is just the worst example of crappy education. There are plenty of schools where a crappy education is still available. Federally backed (edit: guarenteed) money for student loans intuitively seems problematic.

Clojure.

At stockfighter.io and HackerRank, I enjoyed practicing my Clojure hacking skills. HackerRank supports Clojure as a language that is submitted and run against automated test suites for challenges. At stockfighter.io, you can choose any language since you are submitting answers to a REST API.

My Clojure chops increased and I now use the language at work for new development.

I am debating a switch to python for machine learning challenges (maybe Kaggle?). If I decided to return to the enterprise world, I probably should drift back to plain Java for challenges.