HNHacker News
TopNewBestAskShowJobs

wuthefwasthat

51 karma · joined July 23, 2014

submissionscomments
wuthefwasthat··on Dear OpenAI: Please Open Source Your Language Model
Hi! OpenAI employee and paper co-author here, speaking entirely on my own (nobody else in the company knows I'm posting this). Occam's razor is a great principle, and I completely see how it might point towards your viewpoint. But I'd like to make a few points which will hopefully correct some misconceptions:

1. When we made the decision to do partial release, the talking points were about dangers and benefits of release - reputational benefits was one of the four main benefits we listed and not talked about much. Everyone in the room, including mostly technical non-comms folk, agreed that caution was a reasonable decision, given our uncertainty about malicious use.

2. I was at every meeting we had with the press, prior to release. We had written and rehearsed our major talking points beforehand (and stuck to them). Here they are: what we did technically, how it's different/new, how it's still limited; generality and zero-shot behavior of the model; how the trends of scaling are steady; the realism of the synthetic text, and how we're potentially on a trajectory similar to the synthetic images (we specifically compared GPT-2 to Alec's 2015 work on DCGAN, not the photorealistic stuff we see now); malicious use cases and policy implications; framing our partial release as an experiment and a discussion-starter. Ultimately, we didn't have fine-grained control over the actual articles written.

3. My impression is our samples are a qualitative jump from anything publicly known prior to our release. Most people (whether academics or journalists) who have seen our demo come away impressed; a small number had more muted responses. I would be pretty surprised if there weren't use cases this immediately enabled that previous language models didn't; whether they are more economically-valuable than malicious, I'm unsure (although I have a guess).

4. I personally don't have any evidence that the people I worked with on the release aren't well-intentioned, aren't thoughtful, or aren't smart.

Stepping back a bit, there are definitely aspects of what we did that I already regret, and perhaps more aspects I will regret in the future. I wish we'd had more time to think through every aspect of release, but opening up discussion to the community doesn't seem like a terrible outcome, regardless of how well thought out our decisions were.

wuthefwasthat··on Better Language Models and Their Implications
Yeah, there are definitely still places the samples fall short! Keep in mind we're still using very naive sampling techniques.

RE Winograd: WNLI is different, see https://arxiv.org/pdf/1804.07461.pdf

wuthefwasthat··on Better Language Models and Their Implications
Those samples are from the large model (GPT-2)! Regarding memorization vs. generalization, see our paper for more analysis.
wuthefwasthat··on Better Language Models and Their Implications
Thank you! We've released 500 random unconditional samples from GPT-2 at https://github.com/openai/gpt-2/blob/master/gpt2-samples.txt
wuthefwasthat··on The Hanabi Challenge: A New Frontier for AI Research
Never thought I'd see my github repo cited in a DeepMind/Brain paper. I still have SOTA for >2 player by a large margin :)

I'd be pretty impressed if they beat my 5p strategy without Hanabi-specific stuff, but I assume that's not their goal. Good luck to DeepMind/Brain either way!

Authors: would love to chat sometime if you're reading this! If I get back to Hanabi botting, I might try to use a learned policy to replace some of the random (very non-optimized) heuristics my bot had.

wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
Yeah, I'd add it if it helps other developers get started with the code base. Happy to chat with you about it, though, too! Feel free to email me.

That said, I'm not sure there's anything particularly interesting or good about the architecture though - in fact certain things I know are probably badly designed

wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
Thanks!

The lag was for initial page load, and is just a historical footnote, probably. Here's what I dug up from past chats: In May 2015, for a 250k bullet document, it used to take more than 5 minutes. Apparently by July, it was down to 30s-60s. I'm sure it's much better now!

wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
It basically comes for free if you implement the tree operations efficiently :)
wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
Let me know how it goes!
wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
I made ? a bit more discoverable. Thanks for the feedback!

It's pretty easy to experiment with a new data backend, as long as it's easy to implement GET and SET for a key-value store on top of it. It took me less than an hour to get the first version of Firebase working. Most of the work would be implementing the auth, UI, documentation. I've planned on experimenting with Dropbox/OwnCloud/etc at some point

wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
You're welcome! Let me know what you end up thinking. I'm quite curious, since I came from a similar situation as you, but went less far with org mode (I've tried the spacemacs org-mode layer but didn't like it that much).
wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
It is possible to customize the shortcuts, though it's a bit cumbersome. In Settings under the Hotkeys tab, you can download a JSON file describing your mappings, edit it, then re-upload it.
wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
Those both work for me. What browser are you using? (Are you using vimium or vimperator maybe?)
wuthefwasthat··on Vimflowy – A productivity tool drawing inspiration from Workflowy and Vim
Author here! Didn't see this coming - time to update all the documentation :)

Thanks for the comments, everyone, and feel free to ask questions here.

Some background: I started working on Vimflowy after a conversation with a friend. I'd seen many similar apps, but the vim part was important to me. I've tried the spacemacs org layer, but it didn't work out. My friend had a huge document on Workflowy which started getting laggy for him. Vimflowy lazy loads, and deleting and pasting large subtrees is efficient, which went well with the vim concept. So it started out being a tool mainly for me and my friend. At this point, I'd be happy if others found Vimflowy useful, though I don't have grand aspirations for it. I've tried to make it so that developers can customize it to their liking without much trouble. I would consider making a second set of non-vim bindings out of the box if enough people seem to like it - it'd be relatively easy to implement, but the space seems crowded enough as is.

wuthefwasthat··on The Neural Network Zoo
You say it's impossible to prove a universal negative? I'd love to see a proof of that
wuthefwasthat··on Show HN: Git alias commands and shortcuts
aliases for opening files, which i find quite useful

  # open file by name
  op = "!f() { $(git editor) $(git ls-files | grep -i \"$@\"); }; f"

  # open currently edited (unstaged) files
  eop = "! $(git editor) $(git ls-files --modified)"

  # open files for conflicts (good for rebase/cherry-pick/merge)
  mop = "!f() { $(git editor) $(git diff --name-only --diff-filter=U); }; f"

  # open files present in diff
  dop = "!f() { $(git editor) $(git diff --name-only $@); }; f"

  # grep and open files containing
  gop = "!f(){ $(git editor) $(git grep --name-only \"$@\"); }; f"

  # "show and open" commit
  shop = "!f() { $(git editor) $(git show --pretty=\"format:\" --name-only $1); }; f"

  editor = config --get core.editor
I wrote a bit about them here: https://wordpress.com/post/wuthefwasthat.wordpress.com/19

And all my aliases here: https://github.com/WuTheFWasThat/dotfiles/blob/master/.gitco...

wuthefwasthat··on Terminal – Virtual computers that start, resize, and scale in seconds
we just added some official base instances. check them out!

http://www.terminal.com/explore#query=username%3Aterminal