Build Impossible Programs
jvns.ca
jvns.ca
In 2014, for the Google Summer of Code program, I applied to build a JIT compiler for the MoarVM virtual machine. (It is actually more correct to speak of my work as a JIT backend than a full-fledged compiler - the 'frontend' of the compiler was already under development when I started). At the time, I knew just short of nothing about compilers, let alone JIT compilation. So what I did instead was learn, and make lots and lots of mistakes.
And it worked. The JIT backend I created has been in production since 2014, with a new backend since 2017, and a small (but thriving) community of contributors. It is far from the worlds most advanced JIT compiler, but it does provide a nice speedup on real-world perl 6 programs. All the while I've continued learning.
So... definitely try ambitious things :-)
1) Stop and spend time trying to get a clear mental model of what you are confused about
2) Push forward and try to build despite not understanding
In the past, when I tried doing super-ambitious things, I used the second strategy. Many sleep-deprived (tip: “sleep is for the weak” is BS) nights in which I accomplished nothing left me with a strong feeling that I should not try to do impossible things. However, since I learned the first strategy and have been applying it successfully to projects at work, I am rethinking this. I have increasingly been feeling like I would be able to be surmount large challenges if I first set myself up for success in terms of resources and people to ask for help.
Questions:
A) Do you apply a “step back and study” strategy to tackle confusion? Have you found it successful?
B) Do you have any more detail you could add on how it works?
C) How do you set yourself up for success outside of a work setting? For instance, I would like to write a server-side debugger for nodejs that is independent of google chrome and works toward a pry.rb-like developer experience. However, I haven’t written C since by OS class in uni and I don’t know the internals of node.js. How would I go about building trustfull relationships with people who know the internals of node.js so that I can ask them questions in the style of https://jvns.ca/blog/good-questions/
So when you pose the question, whether to push or take a step back, in my experience, it's better to pick clarity over aggressive pushing. It's subjective though. While I think there are overarching principles, I also think that different people react to these situations at least a bit differently.
To say a few words about confusion when confronting something you don't fully understand, I think the more confusion you can take, the better. Ideally confusion should be second nature to us, too often we jump to something prematurely just to escape it. When you think about it, we never fully understand all the things at play. It's about not being paralyzed by that.
My condensed advice is:
1. Are you still making progress? Keep going. Don't give up.
2. Are you spinning your wheels? Take a step back, re-evaluate what you know, investigate things you are not clear on.
3. Still stuck? Ask for help. Don't panic or be ashamed. Show the progress you've made in step 1 and share the things you studied in step 2.
a): 'Studying' is definitely very important to get an idea of the context of the thing you are trying to solve. (I did a lot of reading both before and during the project). But, it doesn't always relate easily to the problem you have right now. (E.g. I'm using linear scan allocation, and the papers on that method all glossed over the nature of a 'live range', which took me 3 iterations to figure out). And without a practical problem to guide you, it is easy to get lost in literature.
b): What I find is that before I try to do something, I typically put together a high-level overview of how I think I can achieve it (what stuff to change where, and what I will need for that, roughly). But I'd classify that more under planning than under studying per se.
What I also do is take lots of notes. I have several large org-mode files (big fan) and before that paper notebooks in which I write down thoughts as they come up. I think org-mode is superior because it lets me organize things after the fact. Notes are a crucial bit of making progress for me, because I don't need to revisit the same things over and over.
The other thing that I find helps very much (and which I wasn't very good in at first) is to do incremental, small-step improvements. Move a function here, change an struct there, change an interface, move some process earlier or later, split it up. Simple things that you can do and that you know will not go wrong. Many problems can be broken down in that way, and the few that can't can at least be 'isolated' so that you can fix them 'atomically'. I wrote a blog post about this some time ago, if you'll excuse the link: http://brrt-to-the-future.blogspot.com/2016/06/in-praise-of-...
c): I was lucky to be able to join the perl community as part of a GSoC project, and fortunate that the perl community is altogether friendly and welcoming. And I really can't speak for the node.js people at all. But what I will say that anyone with a genuine interest in contributing (as you seem to have) and a willingness to do the work to learn (and to listen) will find a warm welcome from nearly all open source projects, because all of them are relatively understaffed compared to their ambitions :-)
So in your situation, I'd start with checking out the node.js source, start with trying to figure out how a debugger could work (there is lots of prior art here, of course, so you can check out the chromium source as well), and ask your questions basically as the jvns comic demonstrates.
I hope that answers some of your questions.
I have a simple little project I'd like to make to manage movie night with my friends. Right now we do it all over email and it's a confusing mess. Since I'm new to the world of web apps (and looking for an excuse to dive in), I needed to learn some background and now I'm firmly gripped by analysis paralysis. I've learned a bit of Go, of Ruby and Rails, Dart, Angular, and some others. I've read through most of the HTTP spec and I have Roy Fielding's paper where he defines REST in my Pocket account.
This morning I'm thinking I should check out Azure or maybe Google Cloud. But also I've done work in Java + Tomcat... maybe I should dust of my JDK?
I need somebody to say "here are the four things you need to make a one-page app that lets users express some preferences and constraints and a little engine in the background can tally the results".
Don't be lured by the latest and greatest HN koolaid. Stick to the bare minimum. The more complexity you add to a project the harder would be to maintain.
To make a food analogy, if you're trying to learn how to cook, you don't need the latest kitchen gadgets.
Just ingredients, pots/pans, and a reliable, controllable source of heat are enough.
Adding extra pieces (especially when you're getting started) is only going to make your learning curve steeper.
rails new && rails g scaffold Movies title:string && rake db:migrate && rails s
Visit localhost:3000 in your browser and there you go. With that short line, you can CRUD Movies including a functional UI. Obviously “real” apps will require a bit more work, but not much depending on how complex you need things to be. The Rails Guides are super helpful.
If someone wants a more robust and production-ready intro to Rails, Michael Hartl’s free Rails Tutorial is exceptionally good.
Other than that, no it isn't necessary. You could start with plain HTML and JavaScript only. But sooner or later you'll have to choose one tech for backend.
RoR feels like something I really should have in my toolkit anyway.
You should work through the book Rebuilding Rails if this is your goal.
For the client side I would recommend vue, but react is also very popular.
Once you have chosen the backend/frontend technologies then you can start building. If you don't know anything about rest learn how your apecific framework does it. You don't need to learn rhe whole spec. At least not in the beginning.
I really just need to stop considering...
I love to throw out crazy ideas. Most of them get filtered away, but I've gotten to prototype some initially unapproachable stuff:
* Generate a dsl for interacting with web pages in ui tests from the frontend framework templates
* Write a test recorder that lets you step through the browser state
* Write a Jenkins plugin to track who's probably broken what test
* Write a VSCode plugin to add run buttons to CodeceptJS tests
* Write a git hook that maintains two way synchronization of a folder between two separate git repositories (by rewriting & copying commits).
My co-workers see me do crazy stuff, but in actuality, it's not that hard. I would say that VMs are definitely harder than what I've done, but I think the concept is the same: the unfamiliar seems unapproachable.
I should add I have had support of more people than I can name :-)
You do that enough times and coding becomes like real-world magic. You know you can do it, you don't know how yet.
Yes, there are places you can get stuck. Rules-processing, unstructured data, Markov Chains, ML, and so forth. But even that's a win. You start understanding the various classes of problems to be solved and how best to approach each one of them. That kind of meta-knowledge is really difficult to get without wading into things.
At the end of the day, you realize that there's no such thing as impossible problems in coding. There's just problems that people can't figure out how to produce optimum results. A lot of the ML stuff we're seeing now is more like "make the best of things" and less like math.
Programming is a hoot.
Why I'm pretty sure I'll stay a software engineer for life.
I thought the comment was both interesting and illustrative. Sure, in CS there's a ton of things still to do. But my comment wasn't about CS, it was about programming -- making things for people. The programs don't have to be provably correct or even consistent. That kind of stuff is the science part of programming which is completely different from the applied philosophy that is programming computers to make stuff people want.
It's a great scheme, and I applaud them for it, and it's a hike up if you're non-Senior, but for about 10%-20% of engineers I know it's a pay cut for 3 months. That's OK, you get to work on something awesome. Not bad, really.
I suppose I should be spending more time looking out for this kind of thing. It's possible there is more money in open source development than I had previously considered.
Innovation is NIH somewhat, somehow and that is a Good Thing.
I hate working at a place where people are so busy being clever that they reinvent everything. You steal from the younger devs an opportunity to learn, and you externalize the cost of your poor documentation and interface consistency onto your coworkers instead of spreading it out over the entire industry. This is NIH.
What you should do is pick a segment of your problem space that has mediocre tools and make something much better. The 80/20 Rule seems to work quite well here, when I see it applied, which is not often enough.
That is 20% of the effort can deliver 80% of the features?
Your link works! Many thanks.
EDIT: M3U8 link: https://skyfire.vimeocdn.com/1537357126-0x19388e9cb2a5d6095b...
I get this when opening that link:
TypeError: "play" is read-only player.js:2:3975
VimeoPlayer< https://f.vimeocdn.com/p/3.3.17/js/player.js:2:3975 <anonymous>
https://f.vimeocdn.com/p/3.3.17/js/player.js:2:260
TypeError: VimeoPlayer is not a constructor 290376045:1:9150 <anonymous>
I still don't know how to fix that except to practise and memorise what I'm going to say, and then just read it. But then it just becomes very monotone-y.
I wouldn’t write a script and stick to it word for word, I’d make slides and general points and then practice them out loud until I got them right.
After a few years of doing this, I only had to practice a few times.
Today I can stand up anywhere and talk about anything to anyone without notice. But it was a long road getting here.
I still get the same feelings of anexiety, but the way they effect me has changed with the practice. Where they’d once made me mumble and lose my train of thought they now motivate me. I still get red cheeks once in a while, but I’ve learned to laugh it away.
So practice, practice, practice, but it’s a lot of hard work. I mean, practicing a 30 minute presentation 20 times takes maybe 12 hours, and if you’re anything like I was, there is no short cuts.
You can be comfortable doing it, but doesn't mean some practice wouldn't make it more enjoyable for the listeners.
For me though, the side effect of practice has been how I can now naturally put my thoughts to speak and deliver the points I want the way I want. Both of these things seemed almost like magical abilities to me 10 years ago.
Anyways, record yourself practicing, it's a great tool.
More archived videos are available here: https://www.youtube.com/channel/UCDGknzyQfNiThyt4vg4MlTQ/vid....
Her enthusiastic quirky style reminds me of Clifford Stoll's (https://www.ted.com/talks/clifford_stoll_on_everything)-- very warm, humorous, and humble.
Also a good strategy to use when preparing for interviews.
It's strange but it feels more natural to have a real person behind the camera and pretend you're talking to that person instead.
Your mind and heart races and so does your speech, so that trick is good for when you're nervous and have helped me several times when giving presentations. You will think that you're talking in slow, but in reality you are talking normally.
https://gist.github.com/AndreasS2501/2dc6c5813f5fd8abc79aad4...
Philosophy is pretty cool?
> So lets focus on that, the impossible thought: How can we live together in peace and prosperity with each other and nature?
By fixing ethical misstandings, improving laws and destroying borders where none need be.
> So again, blockchains and ethereum are software and you can do any thing with them.
You cannot fight racism with it. You cannot fight prejudice with it.
If all you have is a hammer, lots of things start to look like nails. Sure, you might be able to hit your girlfriend with the hammer, but the relationship would probably improve more if you talked about the problem.
Or, rather, if the relationship ceased before violence entered the equation.
(In case anyone complains about pessimism, a friend of mine says often: "Relationships can only end in one of two ways: death or breakup." I'd rather think of it as a stoic position.)
I absolutely love everything Julia writes. She writes in a way that makes absolutely zero assumptions about your level of technical competence. You could be a junior, mid, senior and still get something from most of her articles.
Anecdotally; Almost every female engineer I work with writes and communicates in this fashion and I really bloody wish more of my male counterparts would speak with less jargon/acronyms for the sake of new starts/non-engineers.
It’s not a good idea to generalise and push gender stereotypes in this way. You’re saying that female engineers have better communication skills than male ones, setting the bar higher for female engineers.
You might think you’re improving gender equality with statements like this but really you’re setting a different bar of basic competencies for males vs females.
I absolutely love everything Julia writes. She writes in a way that makes absolutely zero assumptions about your level of technical competence. You could be a junior, mid, senior and still get something from most of her articles.
Anecdotally; Almost every female engineer I work with writes and communicates in this fashion and I really bloody wish more of my male counterparts would speak with less jargon/acronyms for the sake of new starts/non-engineers.
- without that apparently not being allowed:
It’s not a good idea to generalise and push gender stereotypes in this way. You’re saying that female engineers have better communication skills than male ones, setting the bar higher for female engineers.
You might think you’re improving gender equality with statements like this but really you’re setting a different bar of basic competencies for males vs females.
The problem is you chose to group your coworkers by gender in the first place, and then assign some trait to that group.
Your comments just end up reaffirming confirmation bias. It also reveals your own bias to automatically associate traits with a certain gender.
Why not group people by color of their eyes, handedness, tallness, and so on? I'm sure if you look at the coworkers with "good communication" you could find common traits that are not related to their gender. So why imply gender as the cause?
Anyways, I think it's something to be avoided and be aware of in your communication.
it’s sexism
What an utter load of rubbish. Hard to know where to start with all the misguidedness in there. (It reminds me of hearing someone say once that to say that men have penises and women vaginas is sexist.) Your reality is clashing with their theory, so your reality must be the thing at fault. Insanity. Does appalling nonsense like this come from vaguely hearing some gender studies stuff at uni? It seems they think they're doing good by their ignorant, condescending attacks on what was enthusiasm + observation.
I wasted my time writing it, but I realise its not worth engaging or even having it open to engaging with further. It's appalling that it devolved to this point. There is something happening with us all, it's as if any form of online discourse has to be bucketed into some form a political container in the minds of some people.
They've got a filter of sorts that scans for words and outputs a response in the form of "This relates to <INSERT_MEME_HERE>".
A program of sorts that states; Your sentence contains the word women and men, oh okay that comment is certifiably a post that is pro liberal agendas, and the poster is 100% guaranteed to be anti conservative. NO! Its a fucking post about the fact that SOMETIMES there are observable differences between different groups of people.
Why can't there be any nuance any more? :(
I don't see this happening on here much, though, thank goodness. It did make me think of the documentary The Red Pill, and although this wasn't directly a 'men's issues' matter, you see the same blind—to use that word—SJW mania. So sure they're on the side of right that they don't need to check if they actually are. (I don't fully understand all this, maybe no-one does.) It's much easier/more exhilarating to join a crusade or lynch mob than to put the time in to judge for yourself - thinking is hard work and people don't like doing it.
Anyway, I'm glad you stuck up for yourself, it's a shame you deleted that comment though; it was admirable.
I work with a lot of gay men and I notice that they can often be more empathic and sensitive than straight men. Different groups behave differently and individuals can act however they like.
It's not sexist or homophobic to notice trends. We stand to gain a lot from noticing how people behave and reflecting on how we act ourselves.
You can’t just say women are like X and men are like Y unless you actually have reasonable evidence to back it up. Prefixing it with the word it’s an anecdote is just a get out clause that makes it seem ok. In the end the effect is negative on both men and women, even if it’s a positive trait.
You probably aren’t noticing actual trends, you’re just reaffirming thoughts you already have by picking specifics traits out. It’s perfectly natural and normal behaviour, but it’s still not right and we should avoid it where possible.
I get that they can affect expectations but we shouldn't allow ourselves to be paralyzed by the conceivable negative effects of our well-intentioned comments.
Your comments just end up reaffirming confirmation bias. It also reveals your own bias to automatically associate traits with a certain gender.
Why not group people by color of their eyes, handedness, tallness, and so on? I'm sure if you look at the coworkers with "good communication" you could find common traits that are not related to their gender. So why imply gender as the cause?
Anyways, I think it's something to be avoided and be aware of in your communication. I assume you didn't do it on purpose.
The talk presented here is a great example of what we could be doing instead.
And yes, this talk is very very good.
https://jvns.ca/teach-tech-with-cartoons/#tools-that-make-it...
Presumably, she uses the same to do the slides. Here she talks some more about her tools:
She actually wrote a blog post about reverse engineering the file formats when transitioning between them: https://jvns.ca/blog/2018/03/31/reverse-engineering-notabili...
(btw, if you go through her drawings over the years, they got better and better. I think a lot of what makes them great nowadays is practice, practice, practice)
I'll check both out, going to give that presentation style a try.
Cheers for the links.
These poor explanation and documentation patterns I find most often in open source (though it does indeed occur in industry software as well). A huge number of GitHub READMEs reflect this notion (even in npm packages, for example, that are downloaded 1000+ a week!).
To the author of software, the README may be perfectly clear, but for first time users of the software, it can be totally lacking and/or just plain confusing. I always try to be explicit as possible in my instructions and documentation in hopes to battle the field's poor explanation reputation. It's definitely a skill I always try to work on and get better at.
I'm not saying that all males are exhibiting these behaviors but the ones that do are prevalent (and it seems that by doing so they're gaining an edge which makes them successful memes that stick and expand).
I think this means "the assumption of zero knowledge." I love when Scientific American or articles on the web try to give the reader an idea of a recent advance in physics, say, by starting to explain that "atoms are these tiny thingies which are so small that you cannot see them with a naked eye and which were first conceived of by the ancient Greeks." In certain contexts, you cannot make an assumption like that.
"As usual these days I drew the slides by hand. It’s way easier/faster, and it’s more fun."
Hence the topic should be "Build possible programs".