Developers spend most of their time figuring the system out
blog.feenk.com
blog.feenk.com
You can learn enough of a language to be dangerous in just a couple weeks. Enough to be productive in 2-3 months. However, there are 3 other things you have to learn:
1. How the code for the project you are hired for works. How it's laid out. All the weird shit about it.
2. How the framework/libraries used by the project works. Nothing to do with the project, but maybe you haven't used Angular, Swing, or Boost before.
3. How the data is laid out, and how it gets in/out of your system. What database tables are there, what's the workflow.
Those 3 things are the hard part and what take 6-12 months. Learning Javascript, Ruby, or whatever language is the least of your worries.
But there are a couple of reasons why I hire for for expertise in the ecosystem. Granted I'm in .NET which is a very homogenous ecosystem, so framework/library/tool knowledge bifurcates into people who are experts on every tool/library/framework we use and people who've never used one.
1. For the first 6 months you're paying a sr. engineers salary to get jr. engineer productivity. Given that most devs switch after 2 years this is a huge productivity hit and it's all front loaded.
2. While great devs are great, it's a lot easier to figure out if someone is a framework/language expert than if they are an intrinsically great dev in an interview. i.e. ("Do you know C# well?" is easier to determine than "Are you a good problem solver?")
3. Being an expert on an ecosystem pays dividends even after the first couple of years albeit smaller than the first 2.
4. It's not that much harder to hire someone with expertise in your ecosystem than not.
5. Most devs are searching for jobs in the ecosystems they like working in. But a lot of devs who can't find a job get desperate and start applying to everything, which can bring down average quality of people who don't have expertise.
1. Senior engineer productivity
Most companies have zero idea what to do with a senior engineer and they measure them against a junior engineer. The junior engineer usually does the following:
- works a tonne of hours - closes a tonne of bugs - makes a tonne of mistakes - asks a tonne of questions - uses up a tonne of senior engineer(s) time
And you end up with:
- a high ticket/bug fix rate per day from the junior - generally horrible and poorly designed code from the junior but the code works - few big fixes from the senior - what the senior does produce is generally better designed, fewer bugs and is easier to maintain.
If you have hired a quality senior engineer it shouldn't take too long before your junior and mid level engineers start soaking up the time of your senior engineer. It shouldn't take too long for your senior engineer to become the subject matter expert in several areas of your product which takes up even more time.
I am usually hired because things are a mess and they need to be cleaned up. Years of junior engineers doing the best they can with limited knowledge and experience tossed in with some management pressure leads to unmaintainable code/environments.
The largest impact I have ever had was when my manager put some trust in me, gave me autonomy and a huge project. I think this is what you should do with your senior engineers. Give them the space to make an impact.
The industry at large does not realize how much mentorship a beginning software dev needs, or how much damage a poor engineer can cause. I know of a startup where an eng with too much appetite for shiny new tools caused probably $500k in lost runway, and I don't think that was exceptional.
Full stop, this is the _only_ way I think to allow an engineer the ability to actually affect change.
There's something wrong with your hiring pipeline and IC career path if that's what you are observing.
Try paying more for new hires.
> 2. While great devs are great, it's a lot easier to figure out if someone is a framework/language expert than if they are an intrinsically great dev in an interview. i.e. ("Do you know C# well?" is easier to determine than "Are you a good problem solver?")
It's easier to grade multiple choice exams as compared to essays. But the former mostly measures how well someone can rote memorize.
> 5. Most devs are searching for jobs in the ecosystems they like working in. But a lot of devs who can't find a job get desperate and start applying to everything, which can bring down average quality of people who don't have expertise.
Get a better hiring pipeline. If you start dealing with too many clueless candidates it's time to look out where they are coming from (which recruiter, school, whatever) and switching source.
> It's easier to grade multiple choice exams as compared to essays. But the former mostly measures how well someone can rote memorize.
It seems obvious to me if you have two equally useful attributes, and two measures. You should rely on the measure that more accurately reflects the actual attribute. Finding and hiring great problem solvers is a notoriously hard problem.
> Get a better hiring pipeline. If you start dealing with too many clueless candidates it's time to look out where they are coming from (which recruiter, school, whatever) and switching source.
What pipelines would you suggest? We used recruiters, indeed, monster, hiring, hacker news, stack overflow, etc... None of them were a silver bullet. They all had drawbacks.
I read two things: Not FAANG and national average.
If you are competing for FANNG talent that's where you should aim. Whatever average, that includes all bodyshops and bootcamp grads is a useless measure. Average Football player in the US and NFL football players have very different compensations.
> What pipelines would you suggest? We used recruiters, indeed, monster, hiring, hacker news, stack overflow, etc... None of them were a silver bullet. They all had drawbacks.
For college hires go directly at the source. At which university do you recruit?
Ouch. While I myself have been guilty of that 2-3 times during my sadly way too rich on customers and employers career, this is definitely the exception and something with your hiring filters is very wrong.
Combined with your disclosure that you pay about $200K but still have problems with good programmers then I'd think you are letting people in either way too easily or you don't aim well. Which brings me to...
> 2. While great devs are great, it's a lot easier to figure out if someone is a framework/language expert than if they are an intrinsically great dev in an interview. i.e. ("Do you know C# well?" is easier to determine than "Are you a good problem solver?")
I will respectfully disagree with the last assertion. You or the people who interview devs must reach outside of the usual leetcode / whiteboard tests and do some research on more general interviews and learn techniques through which you will get a very good idea if the interviewee is a good problem solver.
Trouble is, most tech interviewers view gaining those extra evaluation skills as a waste of time.
Piece of advice: give the candidate 10-15 minute and an easy coding problem they could solve in front of you, or give them a really small homework (not this "you'll need 1-2 days at home to solve this homework" nonsense) -- but don't make the tech aspect of the interview the dominant part. Talk with the candidate. Give them a few examples of tough situations from the past. Chat with them about how would they approach the problem(s). Ask them what they know about CI/CD, about best practices in framework X or Y, what project or task that they did makes them proud of themselves. Many other such great interview questions exist.
Not meaning any disrespect to you, you seem to be trying hard to get good devs. But from what I read here it does seem like the aim of your company isn't good and it's mis-firing the hiring gun.
$200K will tempt a lot of people to lie and "wing it" when they get hired. But you should leave the high remuneration in place. That way when you get a good professional they'll have one extra -- and very fat -- reason not to leave. :)
To elaborate a good engineer will be able to read a well designed system and almost immediately start mimicking what needs to be done when adding new features regardless of how much experience they have in the given language/framework.
Then you are doing something wrong or have a watered down notion of "senior engineer" (as in "junior engineer" that spent X years on the job, as opposed to someone who is actually progressing well through his career in that time). Senior engineers should IMMEDIATELY outperform junior engineers. Knowledge of the system is largely unnecessary. What senior engineers bring to the table are completely different skills than junior engineers. You can have 100 junior engineers and they will still not be able to figure out the things a good senior engineer will do for you.
Probably you are not utilizing them correctly or giving them the right problems to solve.
So I'm comparing someone who has been professionally developed software on a different stack for 10 years compared to someone who has developed software on an identical stack to yours for 1-2 years.
Believe me they are very different.
Or do you want the surgeon who has jumped from area to area and has no more than a year or two of experience in each area? The experience doing knee replacements will be invaluable in your heart surgery. Right?
It is so weird that this meme of "10 times one year of experience" is somehow become a negative.
I was teaching my self from a 1964 vintage IBM manual that my father had kept :-)
That's a lot to learn.
What they're going to spend more time on is: learning how you decided to solve your problems with your knowledge of those tools and how you decided to fit it all together.
The frontends are actually the hardest to stick with because they turn over fastest, probably followed by distributed-compute solutions. C# is a short walk from Java, SQL has been around for decades(even given dialect differences that change best practice).
If you're a veteran with Javascript, and Angular, and Node, and MySQL, you will scarcely be able to add something as simple as a "Birthday" field to the user profile pages of your new employer's SPA on your first day or first week on the job. That background will give you an idea of how you would have done that on previous applications, and give a slight speed boost as you try to skim the project for tables and functions that are related to user profiles, but the new application is almost certainly different. If you knew C#, SQLite, and desktop application development instead, or, heck, were fresh out of college with nothing but some toy applications that used Java to write some CSV files, that would scarcely matter, you'd still spend that first day reading through the codebase to understand how user profiles work, and regardless of your experience you're probably merely going to copy-paste and modify the "Occupation" field anyways.
There are a couple minutes of that first day where you're editing a bit of code and it will go slightly faster if you can remember off the top of your head that in Angular, the keyword to parameterize an input[date] field with a maximum value so your users don't accidentally claim to be infants is "ngMax". Or you could go to docs.angularjs.org and look that up in 5 minutes or less, which is a delay, but that doesn't matter much if the other 7 hours and 55 minutes of that first day (and much of the coming weeks and months) will be spent learning the domain-specific details of your business, navigating the project hierarchy, and memorizing the most common table columns and class names.
I think too many hiring managers are unwilling to voice and reason about their subconscious expectation that they're going to hire someone who is a clone of their existing employees. The only developer on the planet with 5 years' experience building a CRM and scheduling tool for landscapers in West Virgina using React/Node/MySQL with database layout and code architecture that matches what you have is named Dave, he's already in the office down the hall, and yes he's behind on v3 because he's swamped with tickets from customers on V2 right now, he needs you to hire someone to help. Just get anyone who's reasonably technically competent and good at problem-solving, and they'll pick it up as has always been done.
Nothing simple about this, considering that Omega Star doesn't support ISO timestamps.
The first place I worked as a dev, they had copious amounts of documentation and such an expansive wiki, every time I asked something, the common response was, "fates, it's in the wiki."
I did have quite a bit of onboarding, but after a few weeks, the purpose of handing me off to their documentation was to make me more independent and solve the problems myself
Conversely, I started a new FTE role at a large health care corporation. No training, no onboarding, not even a cursory tour of the network, what source control they use, literally nothing.
The amount of time I wasted learning all of this I'm sure cost the company more money than having a proper onboarding and training program. I pushed my manager and his manager on the importance of onboarding (we were hiring a lot of new devs at the time) but his retort was "it's not in our budget, you just have to learn as you go." Ironically, having spoken up, every time a new dev joined our team, they were pawned off on me to bring them up to speed on everything they needed to know - all the while handling a full load of project work.
Not properly onboarding developers has a huge impact on multiple areas of a business. I was shocked a smaller company knew this, but the larger, entrenched corporation wanted nothing to do with addressing this.
Only person that's not a grad, expected to deliver sprint one.
Example: A decade ago I hired a Designer who had the potential to become a decent JS/UI coder, but wasn't one yet. He upleveled his own JS skills to where he was quite competent. But more importantly, we made sure he learned WHAT THE SOFTWARE DOES IN THE REAL WORLD. If he doesn't know that, and how the people that use it think about their own jobs and environments, he can't possibly design and write good interfaces. (This was software to manage, operate, maintain, and optimize utility-scale solar power plants.)
When we merged with another company in the space a year or two later, he came back and reported that he was stunned that he (the UI designer!) knew MUCH more about how solar power plants actually operated than any of their "engineers" writing their application code. (And yes, he did...)
It's not about solving the problem in the currently "right" or trendy way - it's about doing a good job of solving the right problem in a really useful way!
(FWIW, I no longer care at all which tools, languages, or frameworks are used - if you care about that, you're focusing on the wrong stuff...)
If you know a variety of languages already and your IDE is hinting, linting, and suggesting idioms, you can be productive super quickly.
(I've now been writing C# for years and can confirm that my understanding of the language has not changed dramatically since my first week with it.)
For example, I was making games professionally using C#/.Net for several years before getting hired in enterprise app development. I was able to contribute right away, but there was a ton I wasn't familiar with and took time to understand, from SQL Server (especially SQL optimization, transactions, triggers, etc), to BLL/DAO, dependency injection, CQRS, Unit Testing, Octopus Deployment, Powershell, Active Directory, TFS Server/Build, Windows Services, XAML, Microsoft Azure, IIS, Entity Framework, .NET Core, and a bunch of other things.
Six years into enterprise/web work on the Microsoft stack and I'm still learning new things here and there.
All that being said, I will reiterate that I was able to work on bug fixes and new features within a week of being hired at both companies, despite not knowing hardly any of this stuff at the beginning, and most of that time was spent getting familiar with how everything was structured and flowed in all their various software.
That was Java 5, though. And frameworks is a different story either, I spent lots of time trying to figure out what J2EE is.
did that include reflection ? P/Invoke, and support for native pointers ? async/await and coroutines ? differences between structs and classes ? how generics are implemented ?
Fortunate are you to have worked with people to give you such a point of view.
Our team chose Ember since we didn’t have extensive front end experience and opinions, and it’s worked out well for us overall.
Though I would have it as #1 in any job. Having some idea of what the business using the software is trying to achieve (at both the macro and micro levels) is far more valuable.
Basically knowing the why, you can use as the frame for the what and how. Some of those quirky pieces of software could be misguided, unnecessary or counter-productive - and this is where you become a value-add to the organisation.
I implore anyone serious about changing how software development works to digest Andrei Chis' thesis Moldable Tools: http://scg.unibe.ch/archive/phd/chis-phd.pdf
Once you understand the consequences of reducing the cost of specialised tool development by orders of magnitude, it becomes obvious that a qualitative change in experience follows, that brings us much closer to what people like Engelbart were searching for.
Ooooh, now I am interested. I really enjoyed exploring Pharo but couldn't combine it with work. But getting to play with Pharo for a work-environment while still being able to do my day job sounds very appealing
I am actually working on my own vision of this, but it is slow going.
Good luck!
So I decided to document main flows and designs in PlantUML diagrams. having these diagrams greatly improved onboarding process, cause you can quickly glance what component does what and what are the dependencies (the code base was in JS, so it is usually quite limited on refactoring/figuring out wtf is going on).
But the problem with such approach is: diagram quickly gets out of date. Someone makes the change and the diagram makes no sense at all now. With what I saw in Gtoolkit, you can always query the real source code and build custom dev tools that always produce current and real overview of the system. I would love to have a starter kit for JS projects that you can drag and drop and start building your own tooling for your product.
Manually drawn pictures should only be acceptable for depicting future intent, but not for depicting the existing system.
With Glamorous Toolkit we want you to build your own environment specialized for your own system showing what you say is important.
For JS, you can start from here: https://gtoolkit.com/docs/analyzing-js/
https://ballerina.io/why-ballerina/sequence-diagrams-for-pro...
Can you elaborate on that?
> But the problem with such approach is: diagram quickly gets out of date.
True, but using something lighter-weight like Mermaid.JS helped me in these cases.
One of my own projects in this area is a prototype where I made the best possible explanation of a JavaScript library I could: https://glench.github.io/fuzzyset.js/ui/
and also Knuth agreed, and that is why he invented literate programming.
But why literate programming isn't more popular is beyond me.
I saw this topic brought up in a video I was watching yesterday https://youtu.be/SzA2YODtgK4?t=1519
There are 2 reasons:
1. Programmers and code review tools are not very good at making sure comments and documentation are updated when code is.
2. People see literate programming as primarily for documentation-focused or teaching purposes.
The statistics in the article we're commenting on suggest that more software systems should be documentation/teaching focused, since the "learning the system" phase is where most of the time goes.
I agree. Specifically, it's for teaching my future forgetful self what the hell this bit of code is supposed to do. Especially if waterbed theory appears to apply and it's inherently a juggling act of interlocked complex systems.
One issue with literate programming is that it advertised a single narrative for code. Only code is data, and there are always many narratives about data. This change might look cosmetic, but it is actually fundamental.
In Glamorous Toolkit, beside having multiple views, we also embedded an interactive notebook right in the development environment and through it we tell interactive stories about the inside the system.
And shareholders to make money.
Agreed.
I think it was Fred Brooks who argued that the number of bugs in a system correlates with the number of lines of communication between the coders of the system. And that number grows exponentially as you add coders.
Fixing the communication has the most immediate impact on bug count.
Obviously, your method of communication needs to adapt to the subject at hand. Talking about a website design certainly seems substantially different enough than talking about web assembly concurrency semantics to warrant different modifications of English.
It’s no accident the the Glamorous Toolkit has been implemented in Smalltalk; it is part of the flow of Smalltalk culture.
Instead of slumming of trying to make JavaScript, a truly awful language for humans with the most contradictory base library I know of, faults beautifully demonstrated in several presentations, try using an open source Smalltalk, Squeak, Pharo and indeed GT itself.
Often the most difficult situations are not complex due to the use of some advanced technology or business need - but instead because of the sheer amount of components in play that you need to understand in order to add any value.
The ramp up time just to understand WHAT components make up a given system let alone HOW they work seems to have shot through the roof over the past 5 or so years.
With once monolithic systems being broken down into distributed microservices, service meshes being widely deployed, everything-is-an-API architecture and other good things - an unfortunate side effect (when combined with seemingly growing expectations on engineers to be 'full stack') is that cognitive load has shot through the roof.
While compared to 10 years ago - it does seem like most systems have better uptime - I'm not convinced they're easier to support and aren't over-engineered most of the time.
Edit: spelling, grammar (sorry it's midnight and I'm falling asleep)
Unless your business requirements change drastically every 2-6 months, then it doesn't matter what architecture you use - you'll have to redo most of the stack in a monolith or distributed microservice arch.
Are these good things, or just fashionable things? Is optimizing for five nines, at the expense of many other metrics, ever the right choice if you're not Google? Or is it just a fad? Especially if and when the implementation is just pure blind cargo-culting?
- if you're slow and careful, you're not worth your salary
- if you're skilled and finish fast, that job was easy and not worth being paid a lot
I try to charge more because I do move pretty quickly, but I have been replaced by someone "A third of my cost" more than once. If you took the triangle for "Fast, good, cheap" and graphed the market demand for each, my gut feeling is that "Fast, cheap" is the leading demand by a long margin. To someone hiring a freelancer, "Fast, cheap" looks close enough to "Fast, Good"
Hence the advice "charge more". Never try to compete on price, it's a race to the bottom that commodifies your work, and you're competing against people able and willing to work for a pittance. Be strategic, be viewed as strategic, and price accordingly. Also, consider a hybrid with a fixed-price base (esp. w an initial "discovery" phase, priced on its own), and explicit provision for time and materials overages. Mostly, figure out what works for you, and do everything in your power to deliver value for your clients.
I say all this having used a few different models over the years but also presently engaged in a long-term (well-paying) hourly contract that's honestly been a welcome break from the sales/marketing/bizdev aspects of consulting.
Sometimes fast and cheap is the right answer. And there are devs who enjoy that role (although they often don't have the self-awareness to realize this, in my experience working with them).
There were SO MANY things to figure out, and I had no experience in Kubernetes (and only surface experience with Docker, Tilt and Helm) yet I was expected to be productive in 1-2 weeks after onboarding.
The main reason why I started taking too long in doing any tasks was that I grew more and more reluctant to ask questions. They were always answered extremely shortly and unhelpfully and left me with the impression that nobody cared to help me start becoming productive sooner. Also you were expected to hang out in a rather unofficial Mumble server the entire day if you have questions. What? Are we working remotely or are we simulating an office? Seems it was the latter.
So anyway, I took the long route and started exploring a lot while working on my first tasks.
Needless to say, this got me fired. I regret losing the very respectable salary but beyond that I am actually happy that it didn't work out.
It's like, I get it, you guys are all busy, but if you took one or two weeks to hand-hold me every day then we wouldn't have a discussion at the 4th month mark about why am I so slow and that I have to be let go.
It was rather sad because I kind of liked the guys in the team -- but they were not helpful and you were expected to wing everything yourself. Which is fine, AFTER you receive meaningful initial help. Which I never did.
Sorry for rant. Reminded me of this HN comment (disclosure: not mine) which IMO nails it: https://news.ycombinator.com/item?id=25800104
But, on topic again: I wasn't able to express that message properly some of the places I worked at. There are people who were receptive to it (it == "you need to properly onboard people even if that means some of your business-critical devs work at reduced capacity for a few weeks") when I said it in the past -- and most are receptive to it in my current job as well -- so it seems it really depends on the people themselves and/or the culture of the company.
I guess it's kind of like hiring: it's much more randomness than anything else.
Yes, and:
> why I started taking too long in doing any tasks
I actually test my code, which makes me appear A LOT slower than my "fast" coworkers. So while I rarely do rework, my apparent "velocity" (Agile FTW) looks much worse.
I mean actual tests. Including negative tests. Which requires knowing how the system I'm changing actually works.
My last few gigs, I don't recall any one else testing their code, much less doing negative testing.
Multiple times, coworkers will discover that their code in production didn't actually work. Could never work. Ever. Like the "cache" which never updated its entries.
One of the curses of having been a QA Manager, doing actual QA & QC & Test, is I'm embarrassed when my code doesn't work. Which has become a career limiting character flaw.
I now ask exactly how management attributes time for bug fixes. If they aren't allocated to the person, team, and story point for the original feature, then I explain to them my story and why I won't work for them.
I've had colleagues in previous jobs that were times better than me and they got jaded by such attitude and left, leaving EVERY single manager "very surprised". But they were also kind of snarky and dismissive about it. Oh, they joy when they finally admitted 6 months later that they made a mistake by letting him quit and then the second negotiations when they wanted him back! He told them "eff you" twice in a row but eventually accepted 6-months contract... at 3x his older salary.
I feel that the under-appreciated engineers must learn to punish the companies with their absence much sooner. But yeah, a lot of programmers are rather introverted and shy and don't understand the leverage they have so sadly what we have as the status quo is pretty normal...
Unfortunately most software companies (like all companies) are based on management ego and rah rah rah. This is an infamous problem which is part of why software quality is so low.
And it's still under-appreciated as a skill to this day even if I got handshakes and many "good job!"-s in chats.
Now they're established, they've got their place. That boss that they struggled under is still there and they're going to do to you exactly what he did to them. It's like hazing. Sure one of them might choose to break the cycle and take you under their wing but they're taking the risk that after you learn the ropes you'll go to their sadistic boss and stab them in the back. You know you'd never do that but this is a workplace where everyone stabs everyone else in the back so why not. Also don't think that there hasn't been someone that's already tried that and got stabbed in the back so everyone has seen it and knows what can happen so they're not going to do that.
What they were probably looking for were two things. You could take a certain amount of shit and not fight back or at least fight back but only to a limit. They wanted to know that you knew your place. The second thing is they wanted to know if they could trust you. Until they know that you're not going to go to their boss and say something like, "I have no idea what they're doing. They don't even know XYX hot new tech" you're an enemy.
Not staying there very long was probably a blessing. Working in these environments can leave some serious scars. You read about stuff in history books were you think, "wow, how can someone be so angry that they do such terrible things to other people" then you work at a place like this and you find yourself thinking, "if I came across him in the parking lot having a heart attack I'd step over the body and smile the whole ride home." Then you'll know.
I did get the general vibe that this is some kind of a rite of passage -- the unsaid message of "figure it all out by yourself or you're not worthy". I also did get the vibe they are tired and overworked (hence they needed several new people in the team). But the strongest impression was of a formerly tight-knit team of people who are now forced to work remotely and who are begrudgingly admitting they need help, and were also strongly introverted -- to make matters even worse.
That mix led to the situation described previously: I got more and more paralyzed and hesitant to ask for help as time went by, and my daily efficiency dropped to almost zero.
The thing that hurt me was how annoyed the team acted when I asked every question. One nasty (but pretty normal) response was "read Tilt docs" and it really didn't help me when it turned out that a special Python code line had to be put because my local k8s install refused to even initialize its network interfaces without much of an error indication until I drilled way down.
I did not feel any triumph when the guy I did a screen-share session with finally admitted that they might have thrown me into too deep a pit and that yeah, my setup turned out more difficult than theirs and that yeah, last they did it the cluster was smaller and easier to configure.
I never did once thought to say: "HA! TOLD YOU SO!" -- I was just very saddened.
It's easy for one to give in to negative thoughts and to second-guess themselves but discussing with people -- including yourself -- in this sub-thread did give me more clarity and showed me that it wasn't only me who was at fault.
Thank you.
Ya, that's a pretty good list.
> ...not too much catering to “super stars”. 1-2 heros does not a team make, the senior people make it their job to lift everyone up. The team doesn’t obsess over their high performers.
Amen.
A team is only as fast as its slower member.
h/t The Goal, theory of constraints
My general impression after I was sacked was that the team (and the smaller company) were resisting to get their culture changed with all their might (example: sitting in a voice chat room for the entire day, seriously, we work remotely and somewhat asynchronously nowadays, so why?).
But I didn't give it much thought because in the end it didn't matter: they made up their minds without discussing with me, and even if I re-applied it would not get me anywhere.
What really got to me however was that I was fired shortly after I finally took the company's mission and challenges to heart and started working VERY hard. Just 2-3 mere weeks before I got fired. To be let go almost exactly after you muscled through a mountain of obstacles and started loving the job and the people... that definitely did hit a vulnerable spot inside of me, I admit.
> Also, I'm pretty sure I've been in that gig and it bites. I suspect you dodged a bullet even if you got grazed.
Long-term I am sure I'll think the same but in the meantime my income took a hit. Sigh.
As outlined in more detail in another sibling comment, I had a lot on my plate in my personal life during the gig and I couldn't pull through in time (in their eyes at least).
I am glad and proud that I managed to overcome literal dozens of obstacles -- most of which with tech that's extremely hard to master, with Kubernetes at the top spot -- but I still wish I could work with them again. But with some culture modifications. Which, I realize, won't ever happen. Plus most companies and teams never change their culture.
I was painfully aware that I had to put in 12+ hour working days until I show a productive output, yep. But sadly the stars aligned against me: during the same period my wife had severe depressive episodes I had to help her through, my mother almost died and me and my wife took turns "patrolling" the hospital where she was laying for days, had a huge fall out with my brother during the same time, and I finally cracked under financial and emotional pressure (not going to bore you with my life story but let's just say that the last several drops made the cup overflow). On top of that I was asked to comply with weird culture and practices that put extra pressure on me.
So I wasn't able to do what I was very keenly aware that I should do to keep the job. I am still a bit sad about it because I know for a fact that the whole thing actually started taking shape and I found my motivation and energy and desire to work on the problems in detail and with good craftsmanship... but it was too late at that time, apparently.
it's not your fault
that you were subjected to that, or that you did not match their terrible standards.
For instance, when you're talking about time/effort estimates, if you include the time it's going to take you to get onboarded with a new system, this is a sign you are more professional, not less.
However, I am saying we can improve the effectiveness of the time we spend by an order of magnitude without much effort. We just need to want it. Interestingly, I also observed that solving problems without reading code leads to increased happiness, too.
Thing is, nobody wants to produce such tools for free. Plus, they'd be a huge competitive advantage so even if a company invents such a tool they're very likely to use it for their own gain and not to help the entire programming area at large.
Consider a closely related topic: source control. Nowadays, not using any form of control sounds crazy, but there was a time when that wasn't the case. What was standing in the way? A subset of working developers who just wanted to code without having to think about the task of systematically capturing a record of a codebase's history. But having a reliable, exhaustive record of changes is more useful than not having it, and the default stance today is that you need to use source control. What did it take to get here? Programmers getting over themselves and putting in upfront work to attain a level of competence where basic source control operations are natural.
The tools that the author has created and the work they're doing is in "clearly better than what many people are doing now" territory, but it's completely unworkable so long as programmers are unwilling to get over themselves and keep opting instead to just do things the way they've always done them.
I think it's economical incentives and supply/demand games above all else these days though, not so much about technical prowess (which is IMO aplenty in most teams I ever have been in). I have met some extremely intelligent lawyers and business managers and a good chunk of them are VERY KEENLY aware that most programmers are of low quality and are unwilling to change their (barely learned and never revised) ways. They know, trust me.
But they sleep better knowing there is a bigger supply pool out there because they mostly care how do they manipulate the tech worker into their agendas and objectives -- and thus [want to] view IT workers as replaceable cogs, even if we all know (them included) that this is factually untrue.
That's a huge cultural problem. Not because I feel threatened by some 20 y/o smart guy who feels like a God after two JS hackathons, no; it's mostly because the shot-callers play on people's egos. Meritocracy is still mostly a theoretical construct.
---
RE: "we need to want it bad enough", I do want it very badly both ways: (a) have intelligent AGI-level of tooling and (b) people not shoving their current hype-train ideas into a 5-year old project -- but I personally am not willing the put the work in both because both are not my job. And even if they are made my job, I am 99% sure I will not be paid enough to (1) deliver direct business value with my tech expertise, (2) mentor youngsters and (3) work on automating myself out of the job, all in parallel.
So while I do agree with you on all accounts, I think the incentives outside of our tech bubble are wildly misaligned with our interests and objectives.
And you are right in saying that it can provide a huge competitive advantage.
The only difference is that the tooling does not give you meaningful insights unless you ask it questions. Well, unless you program your questions. But, that is actually much less difficult than it appears.
A couple of such terms that I use[0] are “knowledge transfer” and “[initial or in-depth] audit”.
Typically I’d be communicating with a non-technical decision maker on the other side, and if there’s an existing codebase (or third-party systems that need integration into what I build) I may request:
1. An initial audit before I agree to the larger body of work. Audit will require access to systems in question and the ability to talk to the people in charge of or using them. Initial audit may already warrant an NDA, but whether I can accept further work will ultimately depend on the outcome of this initial audit. The deliverable of the initial audit may be a brief document or a scope/statement of work.
2. Knowledge transfer or in-depth audit. If knowledge transfer is impossible, during the previous step I’m supposed to get a good understanding as to why (does the previous owner refuse to communicate? which factors have caused this situation?), and instead of knowledge transfer allocate billable time for an in-depth audit. There would be a separate deliverable of this step as well, which would include some diagrams or documents describing the systems in question. A separate deliverable is useful both to myself and to the customer (if they choose to hire someone else to work on this), and makes it clearer what the customer is paying for.
[0] If there’re better alternatives, I’d be curious to hear (though I’m not holding my breath, responding to this thread 2 days late).
One thing that I've tried (to varying levels of success) is to use that time writing unit tests. It's visibly productive, nominally helpful, and incredibly useful in picking apart what's actually going on under the hood. The two biggest challenges I run into are managers (including engineering managers) who insist that unit tests are a waste of time and codebases that retard attempts to break things down to workable units.
For contractors too the learning curve is part of the deal. The contractors get paid by the hour when they are climbing the learning curve. Managers seem fine with it.
And the rate of change depends on the person as much as on your internal structure, documentation, scope, etc.
How long does it take to do a crossword puzzle or play a game of chess? As with everything, it depends.
I think we first have to talk about it at all. Doing that, we can learn how to optimize this part of the work quite significantly.
But, would you like to detail what specifically you find hard?
Most managers will not look into the matter too closely, because if they find no wrongdoing then it was wasted time and if they do find it then they have to spend even more time to find a new freelancer. (And possibly take a reputational hit for hiring the wrong person in the first place)
Part 1 https://vimeo.com/496004749 Part 2 https://vimeo.com/498735070
For me, this also calls to mind an old blog post from Peter Seibel about a disconnect where many of us think that we should all be reading code for our own understanding, somewhat like literature, but very few of us do and it rarely yields much. And one of the reasons why it seems that style of reading is ineffective, is that coming to an understanding of code is more like a scientific investigation than just reading prose.
http://www.gigamonkeys.com/code-reading/
I agree with that point. I also agree that we need to be able to create tools that deal with software systems more easily.
But I also think two other perspectives are important:
- One is the historical perspective. The code-base is rarely a coherent whole. Two different areas may accomplish similar things with different tactics. The design of a component may be ill-suited to the way it is now being used. What is an intentional choice, and what is an accident? Which past choices should my current project align with? We typically read the code as it exists currently, and look at specific parts of the history only as a supplement, because even visualizing history is complex. But understanding why choices were made, and in what context, can be critical to knowing which things can now be changed.
- The other is that reading code shows us a complex intensional definition, and reading tests gives us a partial view of an extensional definition (in case X we get behavior Y). But to "understand" programs enough to proficiently change them, we have to grokk something like the neighborhood around our current program: How would a given change in the code change the behavior? Being able to interactively change and re-run a program, and compare behavior before and after is in some sense like doing finite difference method differentiation.
The post explicitly says that assessment (comprehension with the purpose of making a decision about a situation around a system) is not reading. However, it does says that people currently conflate the two because nobody talks about them to the point that reading is a proxy to measure comprehension effort.
You are correct in saying that I argue that it is not appropriate to employ reading as a main means for assessment.
Code is certainly not literature, but it should still be studied. In fact, assessment specifically talks about the intent. If the intent is different, such as learn a new language, reading is appropriate. Reading is also appropriate when the problem fits on one screen. It starts to be inappropriate as soon as you start to scroll.
I also do not say that tools should be limited to code either. Every aspect of a system, including its history, runtime, tickets, customer feedback, is data, and it's all relevant. We should be easily able to integrate any of these in our reasoning.
I agree with the observation that code can vary greatly. In fact, it is for very reason that out-of-the-box-clicking-tools will always fail to provide meaningful value. They bake the question in the click, but because of context is unpredictable, we simply do not know the question before we have the problem. That is why the specific tool we'd need should come after the problem, not before it.
And yes, a system is a phenomena that should be approached through the scientific method (this is the essence of what moldable development is). Developers are already doing that implicitly. We should just make it explicit. All sorts of possibilities will arise after that.
I guess I would only add the distinction that you're discussing "comprehension with the purpose of making a decision about a situation around a system". But sometimes we legitimately want to build comprehension without yet having a specific purpose or decision (e.g. when onboarding a team with an existing code base, or trying to understand how a technique works), but even then reading is a tempting but inadequate path to building understanding.
You are raising an important point. When you do not have a hypothesis, the first thing you want to do is get one :). It's like in research: the greatest problem you can have is not having a problem.
Now, how do you get a hypothesis?
You can start from some generic visualizations. The goal here is not to gain understanding, but to poke at finding interesting initial questions.
But, you actually always know something. You likely know the domain. Or you know the last tickets that are in the work. Even listening in the casual conversations is a good starting point.
When we train people, we literally start from the very issue they work on. Within 15 minutes, we typically find an interesting hypothesis to check for. For example, a dialog could go like this:
A: What do you work on?
B: A UI refreshing bug.
A: What do you think happens?
B: I do not know.
A: Why are you looking at this specific screen? (this is a key question. people often do not know why this screen and not another. If you have a 250000LOC system, you likely have some 5000 other screens you could potentially look at. Not knowing why this one is potentially interesting is not a good thing)
B: Because I think maybe it's related to how we subscribe to events.
A: How do you expect the event subscription to be like?
B: It should always happen in a method called xyz that is provided by the framework.
A: In all classes?
B: A, no. Just in components.
A: Ok, so you want to know the event handling that are not defined in xyz in subclasses of the component superclass.
B: A, right.
It's actually remarkably straightforward. Just try it.
What I took away is the faster I can recognize I'm in confusion, the faster I can get out of that state--it means I really need to focus on learning what I'm not understanding, and then come up with a plan to change the actual system into what I want it to do.
Another thing is, though I've really refined text editing skills, it's interesting to see yourself struggling to _simply edit text_. So while watching myself, I jotted down a few ideas of shortcuts that could help.
Final thing, that I don't know how to fix, is sometimes I'm not motivated, or not in the mood, or scatter brained. Honestly, looking at myself, I don't know what my deal is. If I can just force myself to turn on the stream, then my friends will pop in and encourage me, so that has really helped. Though I'll still hit work that I'm like ugh.
It should also be said that Glamorous Toolkit works for other technologies, too.
Compared to who I was, say, ten years ago, I'm better at grokking the system in general. But for many projects keeping the entire thing in my head is impossible, so another thing I feel I've gotten better at over time is how to figure out which part of the system I need to hold in my head as I work on things.
Those people are practically exalted today, whereas their existence should indicate that for the vast majority of people, computers are inhumane and inefficient in the Bret Victor sense: http://worrydream.com/TheHumaneRepresentationOfThought/note....
There's another interesting program comprehension tool based on dynamic analysis (as opposed to Sourcetrail's static analysis), which I've yet to try: http://findtheflow.io/
https://www.chris-granger.com/lighttable/
Although as ground breaking, it does not seemt to have 'taken off' as much as it should?
They need an engineer, but think they only need an operator, and they schedule (and sometimes pay) accordingly.
OK, I only skimmed the site, but mostly because above question keeps nagging me and the mind boggles.
IOW, gtoolkit is today's sexy emacs (or, it claims to be what emacs claims/ed to be): Here's a toolkit for writing/extending/MOLDING your tool. Or, acknowledging it's smalltalk-heritaged, it's yet-another-smalltalk-IDE.
It's not like the "extra work" of making your product explorable disappears with glamorous toolkit, the silver bullet. gtoolkit just offers a streamlined set of APIs for you to (comprehend, and navigate, and then) use to make your product explorable, doesn't it. So we add another layer of abstraction, work, and potential for error? What was that with complexity and abstraction? Hmm.
The parallel to Emacs is quite on point. Emacs was a great for text, but it's about time to outgrow that medium.
Indeed, Glamorous Toolkit is a Smalltalk system, but the target is for it to work with all sorts of other technologies (and it does already).
And yes, the claim is to spend that extra work to build tools. I understand how that can appear as coming at an extra cost. But, here is the thing: the budget for figuring the system out is already allocated.
Just in the same way as it was allocated for testing. When automatic testing became a talked-about proposition, people claimed they do not have time to spend doing it because they are already busy clicking around. It turned out that automation freed much of the energy to allow people to focus on more rewarding activities.
Now, it's code reading's turn to be automated. Not all of it will. Code reading is still meaningful in the small.
Glamorous Toolkit is a first technology that shows how this works in practical settings. It's not theoretical. Of course, it comes with a learning cost. We estimate: - about 1 week to learn how to learn (yes, the technology can be used to learn the technology, too :)), and - about 1 month to get reasonably fluent with initial analyses.
This is an investment that should be judged like any other investment. The promise we make is that the investment can be utilized over and over in many different circumstances because moldable development is universally applicable.
You should not believe it. We made the technology free and open-source together with all the material around it for people to evaluate it themselves.
1. Begin by learning all the end-to-end product flows this API is going to be invoked within. This is the context in which your API is operating in. It also sets the stage for non-functional requirements such as latency, and availability requirements. How will end customer and business be impacted if this API were to misbehave?
2. Next speak to all the customers of this API. How do they produce the data for your API's inputs and how do they consume your API's output. You will be surprised by all sorts of creative ways in which an API gets used, not necessarily what the API was intended to begin with. But you need to support all those clients so better get to know their use cases. Also, understand how they handle your API failures. The consumers tend to make unstated assumptions about input/output validations and invariants. It's not documented anywhere but floats around as tribal knowledge. Extract that knowledge and document it somewhere. Make sure you are very clear about every single input parameter, how it's produced and its expected values as well as every single output parameter and how it's consumed. Don't forget about exceptions. Often times customers treat exceptions as just one of the expected output parameters and so actually depend on the API throwing that exception.
3. Finally, take a look at the API's implementation. Don't get frustrated if you don't understand more than 50% of the code. It's perfectly fine. Make notes of the parts that make sense and also note down those questions. Ask around. Often some parts makes absolutely no sense to you. But it is there for a reason, someone as competent as you put it in the first place. Again, ask around both within your team and the API's clients. Pay attention to the API's downstream dependencies and how their failures are handled and bubbled up to your clients. Your API's SLA depends on your downstream APIs, learn about them and ensure that it matches the expectations.
4. You also have indirect consumers of your API who depend on the side effects produced by it. Make sure you learn about them. If this API causes a side effect (such as DB update, publish a message to Kafka) then learn how those side effects are consumed. Does someone depend on it? If yes, how?
Though I've described these steps in sequence I find it useful to approach all of them in parallel and over multiple iterations. At first everything is blurry and after first pass you penetrate through about 10% of cloud cover. After 4th or 5th pass you will have a reasonable understanding (~85%) to make small bug fixes.
Right off the bat, I think you hit the nail on the head whereas I have only slowly started to think about how API's are used, rather than just how they're written.
1 is a great point. I'll take some time to get more info on the end to end flows, and I think that will provide some great foundational understanding before even going in.
2. Is a whole knowledge bomb in itself. I hadn't even thought of many of these points. Talking to the consumers is another great idea and I'll make sure to do that.
3. I appreciate the reassurance. Part of why I never built up the skill so far is because it's, frankly, intimidating to start poking at some big system from scratch. Thinking about downstream APIs is a smart angle.
4. I like this point a lot. Seems like a way to level up and start thinking about the bigger picture.
I can already see the benefit of these strategies and how they'll help me develop my understanding. Now I feel equipped to tackle our team API's, not as daunted as I was feeling before. Thanks!
As you go through this exercise I'm sure you will gain new insights into the process and the API itself. Please document them for the benefit of others around you. The way I see it, you are anyway putting all the effort to learn the system, you might as well put in a little bit more effort to document and get your peer's appreciation in return. Also, you will have something concrete to show for all the time and effort you put in over 2-3 weeks, as opposed to just saying "Right I'm now ready to maintain/change this API".
Written communication and specifically documentation is a very under-appreciated skill among software engineers. But it's a great way to make yourself visible among your peers and spread the knowledge of the system.
The software you write will have impact on customers, but they are far removed from you. The documentation you write on the other hand is beneficial to your peers with whom you work and interact every day so this is a good way to be in their good books. They will appreciate and respect you for going the extra distance to spread the knowledge.
Good luck!
I also like your point on visibility a lot. That's something I've only really starting learning about recently at my current org, and you make great points there.
Thanks again!
I have no stake in the the companies future? OK, add another layer of abstraction to 'fix' that nonissue. Whatever. I can avoid it for as long as I can see to care.
That was the most blatant blogversising I have ever seen.
Rust is ideal for that if you have the opportunity to use it
I would argue the opposite. It's easier to understand what's happening when you know at a glance what datatypes are involved, whether they're used by value or reference and how each case is handled. When I last worked with Python it was rather time consuming to fix bugs that crashed a script at some point after 10 minutes of processing, caused by code that would have thrown a clear compiler error in some languages and would have been fixed in seconds.
And when I look at e.g. a Rust function signature I immediately see what kind of data is used as arguments and what's passed back in the result. Meanwhile in JavaScript you just need to forget a symbol in a check and suddenly it won't know the difference between "false", "null" and "undefined".
You are right about integrating third-party systems, but I still prefer the approach where possible issues are discovered as early as possible.
If I know how the external system will behave then yes Rust is better, obviously. However I was talking on the case of figuring out, as the article says. Sometimes you are not even there, you integrate with an external (legacy? undocumented? buggy?) system and you need to understand what's going on. In these cases a dynamic language is so much more productive than Rust and any Rust-like language.
Or even better, if both is the same. That's the reason why, as much as I like Rust, I would never use it for a project where performance is not critical.
That makes it just less transparent. You still have to be careful and know the difference or one day you unintentionally create a shallow copy of an object and now you've got a bug the runtime won't warn about. Yes, Rust is more complicated and harder to get into in those areas, but in return it's not as ambiguous. I know that calling `clone()` will always do just that, no edge cases, and even if I were wrong the compiler would immediately tell me because of a type mismatch.
Realistically I don't think any of the two approaches is significantly faster. But I know what I find more consistent and less frustrating to debug.
However, if you forget about Rust specifics, then things look different. E.g. by simply having the constraint of complete immutability, there is no reason to differ between an object identity and its deep content anymore - it will just be always the same. Of course that means no mutation and hence reduced performance for certain things, which is why Rust doesn't do it.
I can only hope for Rust 202X edition that can introduce new syntax / deprecate another and make certain things clearer (even if that means explicit with a little writing here and there). Lifetimes and traits in particular need a lot of upfront investment to grok intuitively, and the fact that some core aspects of the language are implied can later hit your assumptions really hard and confuse you for a while. At least that's happened to me, maybe I am just dumb and mediocre though.
Actually, maybe my post gave a wrong vibe. I like Rust and I think the team put a lot of effort into programmer ergonomics.
It's just that the language is focused on performance a lot and compete with C++. And so they sometimes make tradeoffs in favor of performance instead of expressiveness or simplicity.
That is understandable - but 95% of the projects I(!) have worked on don't need this - I can just give it a GB more RAM and a bit more CPU and write software quicker because I don't have to care about certain details.
> At least that's happened to me, maybe I am just dumb and mediocre though.
The fact that you used Rust probably puts you in the upper 10% - rough gestimation. I'm not telling you which 10% though. :P
Yep, exactly my feeling. When I get back to writing Elixir at my $day_job I am just blown away how I can achieve most of the same results (for 100X less performance of course) in like 20x less coding lines... :( Not a direct comparison with a dynamic language is possible of course, but I too wish the Rust team starts sacrificing something for a bit more expresiveness and code conciseness.
> That is understandable - but 95% of the projects I(!) have worked on don't need this - I can just give it a GB more RAM and a bit more CPU and write software quicker because I don't have to care about certain details.
Both what you describe and hand-crafted and ruthlessly tested C/C++ code that's maximally efficient have their place. But I definitely don't belong the the "machine efficiency at all costs" tribe and I get worried at any potential signal that Rust is headed in that direction. Which it might not be. We'll see.
> The fact that you used Rust probably puts you in the upper 10% - rough gestimation. I'm not telling you which 10% though. :P
<Saruman voice> YOU HAVE NO POWER HERE!
...I mean, I am myself's worst critic. It took me a while to get comfortable with Rust and even if that means I am a below-average programmer, I don't care. I am taking my time and I can objectively measure that I am getting better with time there.
I still do agree that Rust does require time and persistence however. That is irrevocably true. Here's to hoping the team will make it consume a bit less characters (and thus typing) and improve the compiler and the tooling further. I am rooting for them with all my heart.
I think they do - but that's good! We need a language like Rust to write operation systems, databases, proxys, webservers, hey maybe even browsers. All the things that are widely used and need to be high performant and secure.
Maybe you are using Rust, but you actually really want a different language, one that doesn't focus so much on low level / performance?
Haskell or Scala or F# come to my mind. I'm listing statically typed languages, because I assume you like those (otherwise, why Rust and not sticking to Elixir).
That is very possible. But utilizing my experience and intuition, very rarely have I seen such meticulous and relentless pursuit for efficiency and a compiler that will kill most of your bugs after it successfully compiles your program like Rust. Maybe Haskell and OCaml are it as well but they have plethora of problems that Rust doesn't have. Maybe Nim and Zig? Only heard good things about those but never tried them.
> Haskell or Scala or F# come to my mind. I'm listing statically typed languages, because I assume you like those (otherwise, why Rust and not sticking to Elixir).
Personal / professional development. I started with C/C++ and Java 19 years ago and moved to dynamic languages at least 12 years ago and I felt that I want to have such a powerful language like Rust in my toolbelt again.
Scala (which I use professionally) comes close to Haskell, but you need more discipline, because it has e.g. the concept of "null" and you have to avoid it.
What I like about Scala is that it has a sweet spot in the sense of a good number of jobs (way more than Haskell or Rust) and also having very good tooling (better than rust, not as good as Java though).
And Scala gives you this "if it compiles, it works" feel. But it has a steep learning curve.
I think F# is also great and underrated - same for OCaml. But because the languages are even more niche, they have less good tooling etc. What plethora of problems are you referring to btw?
Nim sounds exciting, but I've never used it either.
> I felt that I want to have such a powerful language like Rust in my toolbelt again.
If you are up for systems development, I would stick with Rust tbh. I think it will offer you some good job opportunities down the road and in general have a bright future. I don't think other languages like C++ or D can really compete with Rust in the long term.
Otherwise, I recommend to Haskell or Scala a try, depending on if you favor the learning experience or the practical gain.
- Haskell: literal hundreds of possible combinations of compiler variants. Immediate turn-off.
- Haskell: several String types. I understand the lazy / non-lazy distinction but I can't understand why in 2021 you have C strings and UTF8 strings separately. I am not seeing much Haskell adoption in embedded contexts where every byte counts. Felt like a meaningless academic pursuit and not a practical concern.
- OCaml: lack of actual parallelism. I am following the Multicore OCaml monthly reports but at this point I accepted that it's best to just wait for OCaml 5.0 which promises it will have Multicore baked in (earliest timeline: end of 2021, so I don't know likely mid-2022?). Also I don't like the mixed paradigms. Even if I would appreciate using a `for` loop every now and then I think I shouldn't be given that freedom. But that last one is a minor gripe actually.
- OCaml: strings again. Having UTF-8 strings there is a challenge. In 2021 there is absolutely no excuse to introduce friction on such topic. UTF-8 strings must exist. I know I can use the libiconv bridge and it's not what I am talking about. I am talking first-class support.
- Haskell and OCaml tooling felt behind excellent tools like `mix` (Elixir) and `cargo` (Rust) but I hear that they are constantly improving and are easier and more intuitive these days. Hope my impressions are outdated there!
There were others but I only managed to remember those above.
Languages like Rust or Go will find themselves in a spot where improving the language will become hard - same for Java and look how slowly the language improved and improves still.
Can't say so much to your remarks about OCaml, but was interesting for me to read.
So, why not try Scala or Crystal? :)
Another factor is the so-called T-shaped skills. I feel I've been going wide (learning every possible technology and paradigm under the sun) for waaaaaaay too long. I now want to focus on several skills and learn them to near perfection before going wide again.
If you already learned pure functional programming through Haskell, then I would not recommend to learn Scala to broaden your overall language-skills.
If you haven't really done the pure functional programming thing, then I recommend to learn it. Especially the way of doing concurreny would be very very different from how it's done in both Rust and Elixir. Scala also has actors, but the other way of doing concurrent programming is more interesting. For example, check this here: https://zio.dev/docs/datatypes/datatypes_stm
This is something you don't have in Rust or Elixir at all (to my knowledge).
But if it's really just for the sake of learning, I think choosing Haskell is better - who cares about strings when you learn these things.
If you want to specialize, learn Rust in and out. :D
> This is something you don't have in Rust or Elixir at all (to my knowledge).
Maybe I am misunderstanding you but Erlang -- and thus Elixir -- has the best actor system invented so far. Message passing, copying data between actors, immutability, Erlang's OTP (fault-tolerance and tunable restarts of crashed actors), all of those things were the entire reason I moved my web work to Elixir at all. Well, the amazingly well done build and task executing tool `mix` turned out to be a huge and pleasant bonus, not to mention the very welcoming community and top-notch docs and best-I-ever-seen REPL experience.
In fact Erlang's actor system is so good that those in Scala and .NET were very heavily inspired by it. Akka in Java land as well.
Rust is getting there too -- the async semantics, the const functions and the various runtimes definitely are converging to much more efficient and machine-native actors with zero copying semantics and dynamic multiplexing on all CPU cores. I am extremely excited to see where Rust is headed in the next 5 years. It has the potential to get very close to the end-all be-all language.
> If you want to specialize, learn Rust in and out. :D
Completely agreed! There's so much work to be done out there that requires efficient use of hardware. So many companies have legacy systems still limping on ancient C/C++ monoliths and 2-3 brave souls are maintaining them, but the business wants either new features or the tech debt is preventing any improvements -- reasons abound.
Rust is extremely well-positioned to disrupt a lot of companies with legacy systems. I am planning to cash in on these opportunities. So it's a good advice from you, thank you.
I would sigh off on that, and yeah, I think you misunderstood me.
> Yep, I've been quite exposed to [almost] pure FP for 4 years and something now -- by working with Elixir.
That surprises me. I'm not sure we use the same terminology. There is no "almost" pure. Immutability and passing around functions is nice, but it is really only 10% of functional programming. Mind that many languages call them "functional" nowadays, but the original meaning is actually different - it's about referential transparency. I'm not aware that Elixir supports that in a meaningful way, especially since you said you did it for 4 years.
See also: https://elixirforum.com/t/thinking-functional/1771 or https://news.ycombinator.com/item?id=14849474
So for what I'm writing next, I'm just assuming that what you did is using immutable datastructures and avoiding mutation of variables etc.
If someone comes to me and wants to learn actorsystems, I can direct them to either Erlang/Elixir or Scala (with Akka). But honestly, Scala+Akka makes it difficult to fully embrace actors and is just... inferior. I would always recommend Erlang/Elixir and would even go so far to say, that most people probably don't really learn to think in actors if they pick Akka.
For pure FP however it is the opposite. If you have not written any slightly bigger program in Haskell or Scala using FP, then you also haven't understood the concept. If you don't already think in pure FP, then you will have a hard time to learn it when writing Elixir.
In Scala, the situation is much better, but still not optimal _for learning_ pure FP. I suggest you to look into Haskell again when you feel in the mood to tease your brain a bit with a new style of programming. Pure FP is as different from using Elixir/Actors as Exlixir/Actors is different from writing python. You have to think different.
For me, both Actors and oure FP are actually techniques that complement each other very well. Pure FP is good for writing all the code _inside_ of an actor. It makes reasoning and concurrent programming much easier compared to having a lot of really small actors. On the other hand, pure FP does not scale - once you cannot stay in your own small "bubble", you need a concept to go beyond. Be it to work with multiple machines, deal with elegant recovery from hardware problems or network problems between bigger parts of the system, or simply load and messaging problems. I don't know anything that is better suited than the actor model here. I hope that eventually we will have a VM like the Erlang VM with a language that supports pure FP as well as Haskell does. :)
In fact, I argue that people should always choose a technology by how easy it is to figure a problem out two years into development.
If you find Pry useful, I'd be curious what you think about Glamorous Toolkit.
A lot of time spent maintaining / understanding the codebase is time spent thinking about edge cases and about what can go wrong.
I had a similar experience with Haskell, but in Haskell you also have to think about laziness and what the computer will actually try to do, which complicates things a bit.
The amount of brain wankery you initially put in your architecture is inversely proportional to onboarding time for new staff. This problem is no stranger to Java (and Rust, it seems) projects.