You created the throwaway account > 1 year ago.
Must have seen everything coming.
4,211 karma · joined April 16, 2018
Must have seen everything coming.
But do consider subscribing to QZ if you like their articles, to support them.
Abstract
Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpretability. Through the lens of attention, we analyze the inner workings of the Transformer and explore how the model discerns structural and functional properties of proteins. We show that attention (1) captures the folding structure of proteins, connecting amino acids that are far apart in the underlying sequence, but spatially close in the three-dimensional structure, (2) targets binding sites, a key functional component of proteins, and (3) focuses on progressively more complex biophysical properties with increasing layer depth. We also present a three-dimensional visualization of the interaction between attention and protein structure. Our findings align with known biological processes and provide a tool to aid discovery in protein engineering and synthetic biology.
“It is not without cost for the head of a global business to take this kind of stand, which is why you see so few doing it.”
Harvard Business Review with four suggestions for U.S. companies:
1/Reduce Hong Kong presence
2/Relocate supply chains to politically safer countries
3/Reevaluate relationships with Chinese companies, universities
4/Factor in geopolitical investment risk.
The page mentioned that: 10th generation Intel® Core™ 10nm mobile processors and up to 32 gigabytes of RAM
But on the product page, we can only get either 8 or 16 GB of memory. Do you know if it's possible to buy these with 32GB of RAM from Dell?
Original title:
Senator Hawley to Zoom: “Pick A Side: American Principles and Free-Speech, or Short-Term Global Profits and Censorship”
Link to no paywall version: http://archive.is/qstnY
The karma stuff is the last thing I care about. I just want to bring this story out to the HN community since it is something worth discussing about.
Cheers.
I wouldn't be surprised to see this story slide off the front page real soon.
This part may be relevant to add to the existing discussion on HN:
More likely, the explanation lies in the nature of the software Google uses to moderate content automatically, which uses a set of computer-science techniques called machine learning. Such software can update itself based on how users interact with the website, without any intervention from human programmers. This automated nature, combined with the software’s complexity, make it plausible for errors to arise in ways that are difficult to understand.
For example, if YouTube comments about Wumao and other ccp-critical phrases are flagged enough times by enough users as spam, hate speech or bullying, then the system could start removing them automatically. This could be the result of something as harmless as a furious comment war between pro- and anti-China factions, or of a campaign designed to influence the moderation software. Google says this was not the source of the error, but would not say what was.
abstract:
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions – something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.
What’s wrong with single authorship and no affiliation?
At the end of the day, if the paper proposes some idea or method, and achieves the stated claims (with reproducible code), then I don’t care who wrote it, how many authors there were, and who the authors work for.