Challenging software projects some programmers should try | Hacker News Reader

Challenging software projects some programmers should try

andreinc.net

70 pointscmpit35 comments

To add onto this list (unless I missed it), I would recommend interpreter, compiler and assembler. An interpreter is a fun way to learn a new language and can be done creatively and artistically which allows you to exercise the art of programming in expressive ways. Lexing for optimal string manipulation and parsing, AST construction for fun ways of building and creating the grammar and interpretation which can be done any number of ways.

Also, it’s not as hard as it seems if compilers in college was your last attempt in this area.

- crafting interpreters

- writing and interpreter/compiler in Go

Both good books on the domain for an amateur compiler writer.

For lower level assembler and chip design stuff/logic gates etc. obviously I recommend Nand2Tetris

Yea! And for the same amount of effort and difficulty I can recommend a couple as well!

- build your own kit car. Maybe start with something easy like a Ford GT with a V8 from a donor Mustang!

- build a kit airplane! Make it a jet for a little bit of extra challenge!

- make a rotary engine from scratch! Make your own materials!

- now a figure 8 propeller piston engine!

- create your own chip fab facility!

These articles are weird, delusional flexes.

All of these ideas sound really fun

As far as writing an interpreter or compiler are concerned, what you're describing is more like making one for an existing production grade language that's compatible with it's wider ecosystem.

These exercises are more like building a rubber band powered toy car. Useless (but not worthless), fun to build (if building things is something you enjoy), and something you can learn fundamentals of the domain from.

Also, I wish I could take a swing at any of what you mentioned for as little up front investment in money or space as giving these software exercises a shot, it'd be an incredible learning experience and awesome fun!

Well, my previous trolling aside, you can build an internal combustion (or electric!) go-kart type deal on a mid software developer budget. There are plenty of university students doing so for under a couple of thousand USD. You can also experiment with RC planes.

I do realize how relative money is - it's "nothing" to a FAANG dev (though they probably don't have the time), it's potentially years of savings for someone working in the the 2nd / 3rd world.

For sure! My current personal enrichment hobby of choice is machining metal. Tons to learn and great fun, but an engineer's salary definitely helps as you say.

It might sound humorous, but I’m not 10x programmer. An educational lexer and interpreter is something you can study and implement in 2 weekends. Assemblers are very very small amounts of code that spit out predefined binary from already made instructions (chip—8 is good for this). I think there should be MORE resources around simplifying what people think is incredibly complex. Production grade compiler tools like LLVM are incredibly complicated, but learning how the systems work isnt. While your comment is humorous, it’s toxic for those who actually want to understand fundamentals well

So if you're not 10x, what's the point of you having built a mediocre educational lexer and interpreter?

Because its fun. I'm not a 10x (not even close), but I dabble in languages, parsers, interpreters, etc. for fun.

Personal growth and enjoyment mainly. Understanding how languages work at a deeper level. Pure enjoyment of learning my craft as is any project I do involving programming outside of work.

When you understand your tools, you understand your job more. You also can appreciate things more from a different perspective.

I agree with you bosch_mind. Building an interpreter is not hard, building a well optimized interpreter definitely is.

But that is not needed for an educational project. I encourage people to do things out of curiosity. Just ask yourself, what else is there that I could do that I haven't yet tried? (if you have the time)

It's not a flex or resume building or whatever (who cares about that?), it's a learning opportunity. I do think that sometimes there is a collective "this is hard" stance for things that we haven't had to do in a while because maybe they have been abstracted away or good solutions already exist. But the "this is hard" stance is actually harmful if it scares one away even though things turn out to be totally doable once one overcomes that mental hurdle.

Flexes, maybe. Delusional? Not at all.

Here's a few stuff I've worked on for my own education over the years:

- Raft: https://github.com/eatonphil/raft-rs

- BTree: https://github.com/eatonphil/btree-rs

- A little Lua VM: https://github.com/eatonphil/lust

- A little Postgres clone: https://github.com/eatonphil/gosql

- A little Elastic clone: https://github.com/eatonphil/docdb

- A little amd64 emulator: https://github.com/eatonphil/go-amd64-emulator

I've never considered myself the "flexer" type, but maybe the tone of the blog post was a little misinterpreted.

PS: Great stuff you've shared. I remember in the past I've spoke with you about b-trees.

I think the tone was fine. :)

We did many of these in the course of a CS degree 25 years ago, IE: scientific calculator, a server, interpreter, a VM… I would expect people working in industry who have completed a degree have done most of this list in the course of their studies?

Most universities have defaulted to standardized materials to one of the big publishers and have largely removed most of the difficult parts because uni's want students to have a "broad knowledge" of other subjects as well as large amounts of faculty turnover. Often with the intent that those students complete politically motivated coursework, some is now a requirement to graduate.

These project types are generic and avoided because students will just copy+paste others work. It provides no signal to the quality of students without digging into work and figuring out if they actually built it. Seemingly nobody has time for that, it's easier to pawn it on Title IX leetcode and nepotism because there isn't any shortage of candidates.

Meh, I'm almost graduating from my CS degree. It was... underwhelming... to say the least.

We did have data structures and algorithms at some point, also learnt a bit about assembly and a /tiny/ bit of processor architecture.

But ultimately, my courses felt, like so little for how much I paid and the supposed "level" of my university.

The last 4 semesters or so could be summarised as, role playing agile development while making awful websites for awful companies (we work with real companies which is kinda cool except it sucks so much).

I don't know if it's just my uni, or my country, or my campus, or whatever, but I /wish/ I had the fortune of getting to actually build this stuff for class, instead of boring web development every single day.

CS programs have been moving away from computer science education and moving towards teaching how to make webapps because students are more interested in resume development to get a job than getting an education and universities want their tuition dollars. Given the cost of college and the realities of the job market I cant blame the students either.

My last semester was marked by operating systems (Unix V6, which you can find the C code for still floating around) and interpreters - in which we implemented a simple python like language called "Wumbo".

So here's a question: Why aren't students doing these things, in either high school or college, regardless of what's being taught in our institutions? I'm fairly certain we built a scientific calculator in pascal while I was still in high school and then did it again in college in C++ or maybe other languages. Building a simple http server 25+ years ago wasn't exactly novel, so today I would think it would be an appropriate high school level project that a motivated student could do on their own.

Because there's a broader range of things you can make, and the web is the largest source of inspiration so takes up the lion's share of projects contemporary students produce.

I got to the "lesser known data structures" section and was already questioning the rest of the article:

> B-Tree

> Circular Buffer

Are either of these "lesser known"? Maybe if all you ever program in is higher-level languages.

B-trees are damn near ubiquitous in OSdev, file systems, databases, or pretty much any indexing system.

Ringbuffers are a hard requirement for pretty much any modern hardware interfacing. All Nvidia, AMD, etc graphics commands are issued via constructed ringbuffers. They're also pretty common in network programming, editor/IDE development, gamedev, and certainly OSdev (outside of drivers).

> Cuckoo Hash Table, Open-Addressing Hash Tables

These have the opposite status to the previous ones. They're not ubiquitous because the alternatives are better.

Cuckoo Hash Tables exist almost singularly as a teaching paradigm about what not to do, because it's overly complex for little benefit. Open-Addressing Hash Tables are avoided (but definitely in use out there) because they require a very good hashing algorithm to avoid clustering+collisions and a large static address space.

That all being said, it picks up and the later sections are much better.

B-trees are damn near ubiquitous in OSdev, file systems, databases, or pretty much any indexing system.

Most software developers are not working in any of these domains!

I believe most of the most efficient hash table impls in cpp use open addressing (swisstables and similar). The ones in std:: don't mostly because of pointer stability.

Thus the:

> (but definitely in use out there)

Additionally, I didn't say they were less efficient. I said they require more logical complexity and upfront storage to be efficient.

> Are either of these "lesser known"?

Are any of these?

It's odd, I have done a number of the things listed throughout my development as a developer, and for many, I would attribute a great deal of my knowledge to working on them, but, in each instance I came to those projects by my own necessity or transient interest.

Something about seeing these all lazily listed here together frustrates me, and leaves me wanting to just say to anyone who questions their interest in programming because they are unable to see themself as someone who does all or even any of the things listed, "forget that, follow your passion".

Save your time: work on stuff you _like_, or work on Leetcode for employability. Don't work on things you don't like for employability (unless it's Leetcode).

Mostly agree, but will say that there doesn't seem to be a silver bullet. Small project weekend stuff doesn't seem to phase employers as they read like dime-a-dozen undergraduate projects, while ambitious projects can require an incredible amount of time and breadth of knowledge, time that someone on the market wants to spend efficiently.

Exceptions (for small projects) might be highly sought-after skills (devops/ML/Cloud/security), but you can also get certification in any one of these. Maybe trying different stacks for web counts for something.

I know a couple of jobless people stuck in the Leetcode tar pit fetish. It is heartbreaking to watch someone struggle so much just to get 5% of the skill they need in a real programming job.

Instead of cloning well known sites I would talk to people and try to solve their problems. There's a big difference between people telling you they'll use your product and people actually using it. The bar is extremely high since your product has to solve their problem, it has to be better than the alternative, they have to remember to use it (or you have to remind them) etc. You will learn a ton and might produce something useful. Wish I had done that more especially as a student!

Build a GUI program with clearly visible and easy to interact with, GUI elements: text input, labels, buttons, menus, scrollbars, visible titlebar.

I know: i'm old.

Not directly related but the author writes:

> However, it’s important to note that most of these ideas may not be relevant to today’s job market

Then advise to clone the frontend/backend of well known site.

What other project HN would advise to do that could increase attractability for the job market, especially in python (the language I know) or C# (the language I would like to learn). I'm not looking for a job currently or in the foreseeable future, but I'm not against having a step in advance in case things go wrong

When I taught web development at university, we made sure to focus on hireable skills. Back then, that involved making a website based on a Photoshop export. That proved to be very successful, and we hired a bunch of those students over the summer since they had skills we could actually use.

That said, I think a lot of theoretical practice tasks are good for stretching knowledge and skills, so they have value too.

Learn typescript. You can do backend and frontend (and even IaC) in typescript. First-class VS code support and sprawling toolchain support.

>What other project HN would advise to do that could increase attractability for the job market

Leetcode. Anything on your public portfolio that "opens a door" usually only opens a door to being whiteboard interviewed.

Reply on news.ycombinator.com