logancg

And, consequently, this may have serious implications for language models that use the Scots Wikipedia corpus. [0]

[0] https://twitter.com/r_speer/status/1298297872228786176

We do. If you’re interested in the architecture particulars (I.e. which layer types, layer shapes, layer parameters, activation functions, etc) you’ll commonly see these in deep learning papers as a directed acyclic graph with layers represented by a collection of vertices or rectangles. Other architecture specifics are sometimes noted with text alongside layers. We also have more general representations of deep models which look like, and sometimes are exactly, Probabilistic Graphical Models. PGMs have their own formal language which makes it quite easy to write complex models describing a joint distribution very simply.

However, this is different than interpreting how a deep model works. “Interpreting” is an overloaded and poorly-defined term of active research. The sense I get from my friends in the interpretability/explainability research world is that despite the buzz, there is no common definition that lends to a clear set of requirements for an acceptable interpretation of a neural network.

I'm econ undergrad -> DS -> Machine learning. Econ is very useful for data science if you focus on the right subjects: statistics, math, and experimental design. You get all the hard skills you need to interact with data that a statistician or computer scientist gets, with the (significant, unique) benefit of learning how to ask the right question or design the right experiment given what is likely a messy, weird, social scientific question.

On the other hand, if you don't do any quantitative, empirical, or experimental economics -- i.e. you only do theory or political econ -- then you won't pick up these skills (as much).

How do you come to the conclusion that ML is a buzzword here? It's natural to publicize interesting new research that grounds a field -- as statisticial learning theory does for machine learning. Historically, Ben-David and co-authors have produced foundational work on the implications of SLT in ML.

Deep Graph Library (DGL) [1] came out in November and I've heard good things. It looks easy and intuitive.

[1] https://github.com/dmlc/dgl

For more, I highly recommend looking at the accepted papers from the NeurIPS 2018 Relational Representation Learning workshop. [1] I really enjoyed the workshop and I hear workshops tend to represent a (rough) frontier of the subfield.

[1] https://r2learning.github.io/

There are a lot of talented ML researchers in China. This is a product of (a) the government and major companies (i.e. BAT) investing heavily in fundamental ML research (b) the population size (c) a long tradition of STEM-focused education in China. So, it's not surprising that would be the case.

The interesting questions are if China is uniquely focused on deep learning over other ML techniques, and Chinese research compares in terms of quality. Anecdotally (speaking as a researcher in the field) papers from Chinese institutions seem disproportionately focused on deep learning (whereas, for example, the UK does great work in Bayesian ML and the US does disproprotionately well in NLP). I'm not a deep learning researcher so I can't judge the technical merit, but I was just at NeurIPS in Montreal, and I saw about equal representation of Chinese institutions as South Korean ones. South Korea, with ~1/25 the population, punches way above its weight per capita.

I'm curious: if you think so much about climate change and its effects, and even plan your actions and location around it, have you considered directing your energy to combatting climate change with your skills on a systemic level? (For example, building sustainability companies, being part of climate activism, earning-to-donate?)

I'm in a similar spot but optimistic about systemic action.

Sounds interesting. The two big barriers to this system are:

1. Political buy-in to set a cost of carbon emission 2. Economic viability of removal (often referred to as carbon capture and storage, or CCS)

1 is non-trivial, but happening. Canada has a provincial carbon tax measure which requires setting a price. A price is also defined in the European cap-and-trade system.

2 is also non-trivial, as the economics are pretty bad right now. However, CCS costs are projected to decrease rapidly. [0]

Right now, integrating CCS is on average more expensive than purchasing carbon credits. Carbon offsetting companies are, to your point, an example of private enterprise filling a gap.

[0] http://www.ccsassociation.org/why-ccs/affordability/

This is a great idea Ben, and I appreciate the work you do. Do you see Kaggle datasets as a tool to encourage better data formatting, or are you also thinking about building tools for automatically visualizing, cleaning, and organising data?

The link at the bottom should be emphasized: https://github.com/awesomedata/awesome-public-datasets

It is a very expansive collection of datasets, some well-prepped for ML and most not (which is part of the fun of it, anyways).

Totally agreed. For the thought experiment, make the super liberal assumption that most value was produced and realized in the past year (or a single year), and it becomes comparable. Otherwise, the inefficiency is a hard lower limit. So in the extreme case, the lower bound is 5x inefficiency. If we assumed a value equivalent to 1-5% of market cap is realized every year, it becomes 100x to 500x inefficient.

A thought experiment: how much energy should Bitcoin use?

Bitcoin's mkt cap is ~$140B [0]. The world's GDP is ~$127T [1]. If we assume: 1) BTC market cap is representative of its economic output in accounting terms, or is at least an upper limit 2) Energy use by product should be proportional to its output

Then at 0.11% of world output, Bitcoin is at least ~5x more energy intensive per unit of value than the average product.

(This is obviously not wholly accurate. For one, market cap != annual value. If accounted for, that might make Bitcoin several orders of magnitude less efficient. And assumption 2 is probably a linear approximation to a highly nonlinear relationship. But I propose this as a fun thought experiment that questions the energy-value relationship.)

[0] https://coinmarketcap.com/currencies/bitcoin/ [1] https://www.cia.gov/library/publications/the-world-factbook/...

This was awesome. Thank you! It’s incredible that it only takes ~70 people’s lives, end to end, to get back to the invention of cuneiform.

Imagine all those people in a room. You could fit them in a small banquet hall. And between the oldest and youngest is the difference between the invention of writing and the utterly complex, global economic flow that we’ve networked ourselves into today.

I think Cassidy would agree with you. I think it's strategic wording to sell his mission and motivation better.

It also potentially engages people who continue to think it's a subject of belief. That may provide an opportunity to educate them.

Not in SF, but just wanted to pass on that I took a look through your personal site and really enjoy it. Best of luck with Impossible Labs!

This would be a fantastic idea.

SEWTHA made me see the world in a completely new way. When I finished it, I started seeing everything I use, see, or have as an energy process. I felt like I learned a first-principles toolkit I could use to break down anything in terms of energy.

I highly recommend this book to everyone – especially if you're serious about thinking about climate change. No BS, no platitudes – just energy from a practical physics perspective.

Updating it today could make it an even more important educational tool for thinking about climate change.

Fantastic, thank you. I didn't know Robinson had a (deliberate? unlike the Mars trilogy) cli-fi novel.

Aye – the list is great. I always admire personal recommendations! Will take a look at Bacigalupi.

Hey guys! Thanks for sharing this. I'm actually in the process of starting something similar. What is your approach to ensuring data security for your clients? That's one of the biggest open questions we're always improving.

This is an extremely interesting example of how a technological fix can emerge in a difficult situation. Transferable to the US context, too. Thanks!

During the speech [1], Yann (surprisingly) didn't mention consciousness at all. The focus of this segment was the need to "imagine" the future. The premise is that "common sense" – Yann's big theme of the talk – is about "filling in the gaps" of incomplete information. We fill in the gaps by imagining the future.

So consciousness was not raised at this point. But that doesn't mean that it couldn't be an emergent property.

[1] Am at NIPS and attended the speech.

Sounds cool. But the website says the topic is "A Trump presidency will be good for America". Which is correct?

Click on the "web" link under the link above the comment section. You can click on the article from there.

Ideally we'd use nothing besides ourselves, but dice is a good idea.

The CEO (one of the co-founders) is 18 years old – and already a prolific entrepreneur.[1] He did a great talk at TechCrunch Battlefield.[2]

Impressive work by the Beam team. I'm not a gamer (at all), and even I have way too much fun watching streams on Beam.

[1] http://www.geekwire.com/2016/beam/ [2] https://www.youtube.com/watch?v=IRpgU2tTRWQ

I spent some time in Singapore. The startup ecosystem has critical pitfalls and successes.

There is a lot of money (Government and private), accessible accelerators, mentors, and business ecosystem support. Taxes are low and registering a business can take only 1 day. [1]

But the ecosystem lacks genuine technological development and experimentation. Most startups I saw were ineffective copies of semi-successful American startups. My entrepreneurial friends there lambasted their friends' desire for stable careers at the expense of creative engineering.

If anything, it'll become a home first for lifestyle entrepreneurs. It's a lovely place to live, it's easy to start a business, and may become a great 3-year "start a business for fun" destination for expats.

However, Singapore has top-tier scientific institutions (NUS in particular) and the government is ruthless in attracting talent internationally. [2] This may be a good sign for proprietary science startups.

[1] https://www.quora.com/Is-starting-a-business-in-Singapore-di... [2] http://www.straitstimes.com/singapore/three-top-singaporean-...

Try ISLR (for learning/prediction-based stats). Probably the best for someone of your level. Incredibly well written, based around examples, reproducible code. The complex parts – the parts where you need math or abstract concepts – are VERY intuitive.

http://www-bcf.usc.edu/~gareth/ISL/ISLR%20Fourth%20Printing....

Free download from the authors, too!