Interview with John Cook: mathematical consultant | Hacker News Reader

Interview with John Cook: mathematical consultant

sinews.siam.org

179 pointsjgamman25 comments

Highly recommend his blog [1] - where he often writes bite-sized math-y articles. I have personally benefited from it; I was looking for ways to sample quickly from a mixture of Beta distributions when I stumbled upon his post on the Kumaraswamy distribution [2], which takes on Beta-like shapes but has a closed form expression for the cumulative distribution function (CDF). The latter property makes generating samples from it convenient and fast.

[1] https://www.johndcook.com/blog/

[2] https://www.johndcook.com/blog/2009/11/24/kumaraswamy-distri...

In addition to being a great resource on mathematics and scientific computing in general, if you're working in the scientific python ecosystem, his blog (and assorted similar content on twitter/etc) is a must-read. There's often a lot of great scientific python content there too.

I can't count how many times I'd be looking up a statistical or applied mathematical concept and came across an article he'd written on it. Great resource.

I love his blog. It has wealth of information.

+1, learnt soooo much from it. I have this mentally categorised next to Paul Bourke's website, which was one of the best sources of information in 90s internet: http://paulbourke.net/

> Once I started consulting, several of my initial clients offered to hire me as an employee; my backup plan was to pursue one of those offers.

I've been consulting for ten years in software/AI/data science. One of the largest difficulties is that many potential clients actually want an employee, not a consultant (perhaps hoping I'll join full-time if it works out, similar to above). In fact, most people who reach out are only looking for employees.

Nowadays it's also become more difficult to independently consult in California, since there are tighter tests for independent contractor vs. employee. Even forming a single-member LLC may not be enough to get around the distinction for some types of work. Due to this, one of my 5+-year clients was forced to reconsider our contract, and ended up only offering employment.

Supplementing income with part-time employment could be perilous for a consulting business (note: ianal, consult your own lawyer). For example, there is potentially an explicit "duty of loyalty" not to work with "competitors." And, the IP ownership clause could include any work "related to" your employer's business -- even when created on your own time.

Companies now actively seek or mention criteria that they were not hiring CA based freelancers.

California's IC laws are a testament to the idea of excessive government overreach creating a mess out of a functional system. At best with good intentions they implemented poorly written laws. Or the cynical view, they implemented a set of laws without genuine concern, simply because these companies were politically beneficial to oppose (the gig economy companies in this case).

Same goes for what has been happening in the U.K. with regards to the off-payroll legislation or “IR35”. It has now become near to impossible to work as an independent consultant via a personal service company. Vast majority of companies now just want bum-on-seat contractor on fixed term contract and company payroll.

I was excited to read this as someone with a mathematical background and is now an engineer and software developer. However, I think this just calls out all the open questions about this style of consulting and doesn't answer them. How does one get clients, how does one keep a steady income, how to invest for retirement, how does one convince clients you have the right skills, how do you know you have the rights skills, etc.?

Also, there are conflicting answers in that he wanted better control over income than traditional employees but then later says income can be infrequent, saving is hard, and payments come late. I have none of those issues as a salaried employee and also have retirement benefits. He also mentions that finding work is no harder than finding employment but then later said that finding work is harder than doing the work.

> Clients often need to have a conversation with someone who thinks like a mathematician. This means carefully defining terms and focusing on the largest ones, making implicit assumptions explicit, knowing when an approximation is or is not good enough, and so forth.

I love to see this mentioned, but in my personal experience, this has not been valued at all. I have tried it and discussed it, but nobody has appreciated it. This is really what domain-specific design tries to get at in software engineering, but almost no one practices it.

I am in a similar situation to you. I've been lucky to work on a few projects where I've been able to use my skills in applied mathematics—often a bit of statistics or optimisation—but I'd love for my job to have a stronger focus on applied mathematics.

> However, I think this just calls out all the open questions about this style of consulting and doesn't answer them. How does one get clients, how does one keep a steady income, how to invest for retirement, how does one convince clients you have the right skills, how do you know you have the rights skills, etc.?

He touches on some of these issues in a podcast interview with Radim Rehurek (of Gensim fame).

https://rare-technologies.com/rrp-2-john-d-cook-consulting/

I'm an applied mathematician and have been working independently as a consultant for nine years now. I don't have all of the answers for you, but perhaps some of the following may help.

- Regarding retirement, those who run a business get access to a SEP IRA, which has a contribution limit higher than a 401k. It's safest to talk to an account when first starting off, but the IRS posts their rules here:

https://www.irs.gov/retirement-plans/retirement-plans-faqs-r...

Long story short, you can contribute the lesser of 25% of income or $61k. At a bare minimum, I contribute this amount to minimize my tax liability. The amount changes year by year and I wish I had more, but the gains have been fine. I just use index funds. Also, I have been told there is a way to convert the SEP into a 401k, which then has certain tricks such as being able to loan yourself money in order to smooth out your income if you can pay back the money within a certain time frame. I have not had to do this and, again, I would encourage a conversation with a good accountant. I mention this because there may be a way to use your retirement account to smooth out your income stream if your work tends to be of the feast or famine variety.

- Regarding clients, my experience was that the initial set of clients came from people who I knew needed help that I had met while working as an employee. From an ethical point of view, I do not believe in reaching out and attempting to setup a contract while still working for your employer. This may also be legally dubious. However, I do think a tasteful email after separating from the employer is reasonable modulo non-competes. If the work you do for them is good, they tend to bring you back for more work. They also tend to refer you out others. Professional organizations and meetings sort of works sometimes. It's a massive time sink. I've never been able to show up and land a contract one a one shot. It takes a lot of personal time. I've never had a cold call work.

- If you ever want to attempt a federal government contract like an SBIR, STTR, or BAA, it's essential to take a class from your local chapter of the National Contract Management Association (NCMA):

https://www.ncmahq.org/

This is a bias of mine. That said, I learned a lot about how to read and write contracts and my class was something like $200 and worth every penny. Ultimately, you could have a good proposal and have it screened out because it doesn't use the correct verbiage that the contracting people need to see. It's not worth the mistake, so just take the contracting class.

- Does it pay better? Hourly, yes. Overall, no. Lifestyle is better. Potential upside is better. Ultimately, I've never been able to put together enough contracts where I'd make more than what I believe I could make as an employee. That said, there's a balance between contract work and product development. The products, meaning software, intellectual property, and algorithms, can be used on contracts to pay the bills. However, they can potentially be monetized independently and sold as a product. Maybe that's licensing. Maybe that's a finished software product. Personally, this is the direction that I've been heading because it's not clear to me that pure consulting is the way to retirement. That said, the product development has been very important to me, so I have not focused solely on developing the consulting business.

Anyway, this isn't meant to be a comprehensive list of how to be successful. In truth I don't know, but I enjoy hearing stories from others on what worked for them. If anyone else has some experiences, I'd love to hear.

Thank you very much for this information! This is actually quite helpful.

The NCMA link is appreciated, as I have been curious about SBIR contracts. However, it is quite difficult to even navigate what is being called for much less understand how to properly submit a proposal.

Thanks again for the detail.

Not a problem! And, yes, federal contracts are incredibly confusing and full of terminology that most people outside of that line of business do not use. That said, one more resource that may be helpful is your local SBA (Small Business Administration) office. There's some information online:

https://www.sba.gov/federal-contracting

However, some of the local reps can be good. If your business qualifies for 8(a) status, there's much more help that the government can provide that can also be helpful. There's information on the SBA's website as to what that means and how to register if you qualify.

That said, I'd start with a local chapter of the NCMA. That will help demystify much of the process. Then, I'd talk with the SBA to see if they can provide further assistance.

John D. Cook seems like the type of person who would inevitably rise to the top of their chosen profession, regardless of which path they took to get there. He is very intelligent, creative, and determined to do things his own way.

"Academics are often disappointed if a solution is too simple because that means it might not be publishable, but private sector clients are delighted with simple solutions."

As a B-school prof, I followed a few consulting opportunities. Results:

(1) Not always easy to get paid.

(2) Medicine and some fields of engineering are recognized and accepted professions where both the professional and the customer know something about what to expect. Less so for applied math. In simple terms, so far applied math is not yet much of a profession.

(3) Really, all the business came via the B-school. But the academic, research university culture of the B-school regarded any consulting or applied work as neglect of research or teaching.

(4) Somewhere have to get past the norm that a pure/applied math Ph.D. should not make money enough to buy a house and support a family. A lot of people with no graduate degrees, in nice houses, with late model cars, doing well supporting a family, are laughing at how smart math Ph.D.'s are. And, if business person X meets a pure/applied math Ph.D., e.g., in just a business meeting, person X can resent that the Ph.D. may know some technical material they, person X, does not know and that might be important for the business of X. A person with cancer does not resent the knowledge about cancer held by an MD treating that person for their cancer.

(5) Broadly, a medical school is clinical, sees patients, and the profs, MDs, are expected to continue to practice their profession, that is, continue to see patients. Again, the usual research university math/science culture would regard the clinical work as neglect of research or teaching, but, really, medicine is one of the most active and productive fields for research.

(6) At times there are opportunities, with eager, well funded customers, for computing with some applied math in US national security around DC. To be blunt, pure/applied math in the research universities gets funded nearly only by the US federal government and for basically one reason, US national security, and because of some of history, cryptography and The Bomb, with some respect for applications of antenna theory.

(7) My view is that some guy in a big truck/little truck business -- buy from a few big trucks, sell from lots of little trucks -- is right: That is, have a business, an actual business, with products and/or services, customers, revenue, and earnings. For more, now, there may be good businesses based on computing and the Internet, for more than just Intel, AMD, Cisco, Qualcomm, Microsoft, Apple, Google, Facebook, and Amazon. And some pure/applied math somewhere in such a business, likely programmed and buried somewhere in the infrastructure, might be a crucial advantage.

Else, what are we racing toward, a few really big companies making robots and everyone else with nothing to do?

Regarding 7, working in such businesses I've often felt a tingling of "someone with greater mathematical sophistication could cut through the complexity I'm getting bogged down in and/or improve performance."

If I had the same feeling about a Java consultant I know about how and where to find one, how to present the problem and what form their solution might take.

I think spreading the word about how to do the same with an applied mathematics consultant would be very useful in expanding the market.

You use many MD analogies. Let me assure you that patient _do_ resent knowledge about their disease. Also, your assumption about MD academia is unrealistic. Universities are driven by research citations and funding, and this is also true of medical schools. Many professors basically hold the belief that even a chimp could do clinical work, while only the true elite can do research.

He and Stephen Wolfram. Both same sort of style. Their math is more on the applied side than purely abstract.

He doesn't have the ego though.

What's wrong with stephen wolfram?

> What's wrong with stephen wolfram?

The ego.

Examples?

His argument about employment is weird. I am 99% sure he could get paid more with less lead-chasing if he worked for one or more tech companies as a researcher, analyst, or developer. Maybe his consulting work is more fun and interesting, but “agency over income” is doubtful.

Reply on news.ycombinator.com