The perils of programmer education in the bazaar
blog.practicingruby.com
blog.practicingruby.com
Improving writing education can increase the pool of software developers who write/teach well. Teaching is often an excellent way to learn, because writing helps to structure thought.
> Producing high-quality educational resources is obscenely, ridiculously difficult and time consuming. So the people who can invest the effort are typically either from a position of economic advantage, or are backed by monied interests.
This is a challenge in any "commons", where code/prose content (marketing) is often funded by ancillary revenue/income/objectives. How are libraries funded and how do librarians decide which topics are curated by those funds?
> Those who are doing original research, particularly things that are experimental or exploratory in nature, are not well supported at all.
This is a subset of the broader challenge of long-tail discovery, which afflicts many smaller code/app/media publishers who lack marketing budgets and expertise.
> We must find a way to bring programmer education out of the marketplace, and into the commons. How should we go about doing that?
How about finding ways to compensate those who already produce valuable content in the commons, so they can have more time to do what they already do well?
Related to your last point:
> How about finding ways to compensate those who already produce valuable content in the commons, so they can have more time to do what they already do well?
This is actually what I've been trying to do with my own reader-funded project (http://practicingruby.com). I've been modestly funded for five years now to produce free documentation for Ruby programmers (and really anyone who is interested in software development that can read Ruby).
Occasionally, we've paid contributors to help us publish articles. Right now I'm trying to do a Kickstarter to give us some seed funding to really expand the number of contributors we can work with, and my existing subscription revenue will fund me so that I can assist these folks with editing, research, code cleanup, etc. That helps expand the pool of potential contributors, because folks don't need to be expert programmers or writers, they just need to be working on something interesting and willing to try to share their knowledge and experience with others.
Those who want to support the contributors fund are welcome to do so here: https://www.kickstarter.com/projects/828998531/practicing-ru... -- 100% of this money will go to the authors of the articles, I'm not keeping any for myself.
What do you think of the economic boundary between timeless content (foundational principles) and time-sensitive/perishable content? LWN segments their free/paid offerings by time. One could argue that there's another, missing category: review/summary articles which contextualize the perishable content and extract implications for long-lived assumptions.
On one hand, there are ecosystem-wide benefits from removing financial barriers to entry, e.g. free and high-quality content on foundational principles, proven over time by competition between many alternatives. On the other hand, there is economic value to be created from high-quality synthesis, for audiences which can afford and apply this synthesis to immediate commercial problems.
How can we segment these two audiences? Softcover.io seems to have been successful, with the Rails tutorial as one data point with substantial revenue (6 figures) even though the core content was free to read.
In other words, if I can look at my archives as being a body of materials that mostly holds their value over time, then supporters can be comfortable knowing that their money is stretching far and wide.
Also, although we don't do it as often as I wish we could, I like the idea of periodically going back and revising or replacing old works with updated content, to extend their life even longing and take advantage of new thoughts that come along. Having paid supporters really helps there, because it feels less like a chore and more like a service.
As for perishable / time sensitive content... I don't really know. I guess in that case you're funding people for their time and expertise rather than the lasting value of the work they do. I think this would come with a very different set of challenges, but I'm not seeing where the economic problem would be.
In the first couple years of Practicing Ruby's development, we did release the content only after it had been available to paying subscribers for a while. (There would be 10-20 articles available to subscribers only at any point, with a sharing mechanism that allowed folks to post links for others to jump the paywall)
When we stopped that and moved to directly releasing content when it was ready to be published, there was no major change in subscribers immediately, or even within a six month window after that decision was made. Actually, the cancellation rate went way down, so that might be a sign that the supporters were supportive of the idea.
The problem (and I'm sure some folks here may have guessed it), is that once everything was made available for free right away, the number of new subscriptions went way down. Whereas before we might see 30 new subscriptions a month and lose 20 people, we started seeing 1 new subscription a month and lose 2 people.
A sequence of events that happened since then has slowly bled the business of its revenue, although our traffic improves every week, even without new content being published for several months at a time. The most notable one is that I decided that the business ought to run on its own revenue, something that is ultimately a good thing but was a harsh change because previously I'd work 120+ hours a month on the project and only just barely squeak by on a subsistence level.
So basically, the project went dormant for the better part of a year, and although I did some small things from time to time, I mostly saved up the revenue so that I could pour more dedicated time into things down the line. In that process, we understandably lost supporters over time, but the main issue was bringing in new subscribers.
At this point, I'm burning the modest reserves I stored up, and that's given me a few months of runway before the project needs to go dormant again. If the collaborator model works and the kickstarter I set up gets funded, I think I'll be able to stretch things a lot farther. I do a very intensive process with anyone who contributes their work, but at this point it's still much easier for me to collaborate with someone who's contributing ONE article to Practicing Ruby than it is for me to research and write ten more pages when I've already produced something like 1000 pages of work.
So in a way, the death knell rang, and that's what motivated me to try to fix the business problems. In the process of doing that, I remembered the motivation for doing the project in the first place (which was to provide a functional example of a top-notch free documentation project), and that's what got me excited about things again.
Who knows what will happen from here? Maybe the project will still die. But either way, I'm sure I'll learn a lot from it.
However, the main feedback I got from people (and it happened often enough where I'm convinced it was a common line of thinking) was that folks just weren't involved in Ruby anymore, or weren't doing a ton of online reading because got busy with other things. When the average age of our remaining supporter accounts is something like 3-5 years), that's understandable.
I've had two children since Practicing Ruby was started. Things were hard for a while. I wanted to quit the business many times, but left it in a zombie state in the hopes to do right by my subscribers sooner or later.
Now things are getting a little better for me, so that's what I'm going to try and do.
As for eBooks, I've thought about that 100 times. I would love to get some of the existing content into a nicely tied together collection and release it as an eBook. But it's a big enough undertaking where I feel it'd take me away from other more valuable work I could do.
If we get our cashflow situation to be even a little better where I could fund a couple days a month to work on that, I'll go ahead and do it. But for now, I only really can afford to fund about 1.5 days a week of my own work, so that isn't much to spread around.
The tech community has seen to it that this doesn't happen with: file sharing of copyrighted material and adblock.
You can't first destroy entire industries in the name of 'freedom of speech' and then cry about how you can't make money in those industries any longer.
Content is now cheap and it's only getting worse. Which means content providers will be forced to take on jobs that they don't actually want to do (or are necessarily good at) to survive.
We do have Wikipedia. It would be nice if there were hundreds of such examples out there.
There's a natural alignment of incentives when you're working on the same codebase as someone who actually knows what they're doing. Any mistake you make, they have to fix, and so they have good reasons to see you get up to speed as quickly as possible.
Another good heuristic for online forums and comments: the shorter the comment, the greater the chance that the person actually knows what they're talking about. Experienced people often might throw out a quick correction because it costs them nothing, but have no desire to get into a big debate because their time is more valuable than that.
A smart high-school student who enjoys a challenge comes up to you and asks which projects to look at, and (if you say more than one) which one to start with. What do you say?
If you can't get a job and need to go the open-source route, then Rust, Chromium, V8, Django, and the Linux Kernel are all good options. Again, the guiding principle is to pick software where you're familiar with the outside, so you have context when you go look at the inside. Reading software in isolation is hard, so you're often best off volunteering to fix a couple low-priority bugs on the bug queue, and then diving in to start with them. Existing volunteers will often point you to the right spot in the code to get started if you're doing productive work for them.
2. What, specifically, about a flea market is bad. If the complaint is about quality, I'll refer you to Sturgeon's law.
3. You're going to need to explain the competitive vs cooperative comment, right now I have no idea what you are saying.
4. Your comment about "blogging, teaching and tweeting" is true, but I think it is clearly an improvement compared to the previous system. Portfolio based systems are almost always better than credential based systems.
5. I'm not sure who's making money selling pick axes to starry-eyed bumpkins, but all of the development tooling I use is free. Almost all of the online tools I use are prices such that individual use is free and corporate use costs money. Not sure what you're talking about here.
In summary: It seems like your basic thesis is that the world isn't equal because some people are better at communication and self promotion than others. I'm not really sure why you think the world could be another way.
Yes, at about 5% of the speed you could learn by actually working on your communication skills with your colleagues on a day to day basis.
Basically, if you want to ever be more than a code monkey, you need to speak up both in person and online.
So is writing an email. In a typical day I probably send 5 important emails. Writing opportunities abound!
Not to even start on writing designs, memos, technical briefs, position papers, meeting notes, technical docs, user documentation, etc, etc.
The opportunities to practice your writing skills at work are limitless. I'm not opposed to blogging in any way; let's just not pretend blogging (or tweeting) is in anyway important or necessary for building communication skills.
Are they really though? On what basis?
99% of my best work can't form part of my portfolio. I spend 2 hours of my own time on L&D each week day. Maybe 30 minutes of it is on projects I could present. In a year that's ~120 hours max. I'm not going to be doing anything really impressive from a portfolio POV with less than a month of full time work put into it.
Even worse is the trend I'm seeing where people drop the Learning from L&D altogether so they can invest more time into presentable projects. It is stunting their growth as developers... even as it improves their hireablility.
There are certainly problems with the portfolio model, but I've never heard anyone seriously argue that it's worse than credentials.
If you have the credential of an undergrad degree at MIT with a good GPA, combined with a credential of working for Google for 5 years, there is a very high likelihood you are a good engineer.
On the other hand if you have made serious contributions to the Linux kernel, there is a very high likelihood you are a good engineer.
But most developers don't go to MIT and work at Google. And they also don't make serious contributions to large open source projects. For most developers credentials and portfolios are not effective methods of hiring.
> 1. Community is absolutely a weasel word when you're using it to lump together people from many different backgrounds and speak on behalf of them, or to act as if your particular sense of identity is representative of a whole group.
Help me find a better word for what I'm trying to say here: When I say "as a community" I mean, "as people who ought to be concerned about the social impact of our decisions as much as the economics and personal motivations"
> 2. Nothing is bad about a flea market. I just would like to be able to also have public non-commercial spaces for learning, and have them be well funded and supported too. I meant it literally when I say the current model works too well economically, and so there is less incentive to make long-term investments in educational resources that are explicitly for the common good.
> 3. It comes down to economics and social capital. Those who have gained notoriety through luck, connections, or money, tend to have their signal amplified indefinitely.
Yes, if someone really starts being malicious in what they do they will eventually lose their reputation, but it's a slow process and so those who get into that position tend to stay there. This is the nature of a competitive environment, whereas a cooperative environment intentionally makes adjustments to mitigate that effect. It's a little hard to explain because we have so many examples of the former and so few of the latter.
> 4. We agree, and I said that a few times in the article. It wasn't just for rhetorical purposes. I'm saying, if the current model is an improvement and evolution of a worse model that existed in the past, maybe we can still think about a new paradigm shift that is much better than what we have now.
But honestly, I'd be perfectly satisfied with comfortable co-existence between the commercialized education model. Even if 80% of our work was indirectly or directly tied to commercial promotion, it'd be nice if 20% of our resources could exist in the commons.
> 5. Not referring to developer tools here, I'm talking about developer education. There are plenty of "learn to code" things that are not effective, but because they're marketed well, they make plenty of money. In addition of these, there are plenty of "how to make money as a coder" things, etc.
There are also plenty of examples of open source tools that are poorly documented or designed, yet due to their popularity or entrenchment, spawn a business of consultants and educators to teach people commercially to use these things, without contributing the materials back to the commons under free documentation licenses. This is valuable work and shows that our marketplace is semi-efficient, but it's not socially optimal. The socially optimal result is for the information to be released as free documentation.
> In summary:
I guess my thesis missed you, because that's not what I meant to say at all. What I said is, "the world is unequal, and so it will tend to favor a tiny minority of privileged folks unless we actively seek to balance things".
For my own part, I've spent the last five years creating what may be the largest collection of open source learning materials for Ruby programmers in the world (find it at practicingruby.com). If you google "infrastructure automation", one of our articles is the top hit. If you google "actor model", one of our articles is on the front page. Neither of these articles were written by people with sizable followings or influence on the internet.
I paid these folks for their work, helped them shape and edit their content, and gave them access to a much larger audience than what they would have reached on their own. My own name does not appear on their work, nor does anything promoting some third party product or service.
People do pay for Practicing Ruby. Not commercial sponsors, but readers and community supporters. It'd be nice to see a few dozen similar publications on various topics, and for the model and motivation to be well understood and supported.
The problem I have is that this kind of thinking about publishing is something people never even give a passing thought... so as someone trying to do something different, I am constantly fighting an uphill battle.
I don't want to overstate my own role in making a difference here... I view my own project as a baby step, one of many necessary to make the kind of changes I'd like to see in the world. But at least I'm trying to stand up for my principles by putting the ideas into practice.
"Profession" is the word you want here, I think.
People are learning to code in record numbers
Teaching someone else to code has never been easier (and is getting easier all the time)
What exactly is going wrong that needs fixing?
This economic model is so ingrained in our DNA, that we hardly ever question it, and many would go as far as to hold it up as a triumph. But the truth is, even if this system works at the grand scale, it’s hurting both learners and teachers in countless subtle and not-so-subtle ways. Here are just a few:
1) The system we’ve created is assumed to be a meritocracy, but is actually driven far more by popularity and market trends than it is by the quality, relevance, depth, and effectiveness of our learning materials. Although it is far from a zero-sum game, the current model is much more competitive than it is cooperative, and so suffers the consequences that come along with any competitive environment.
2) Because being highly skilled at both writing and software development is rare, those who can do it well are often get the most attention and influence from the software world, turning these folks into “thought leaders” that drive the overall direction of the community.
3) Producing high-quality educational resources is obscenely, ridiculously difficult and time consuming. So the people who can invest the effort are typically either from a position of economic advantage, or are backed by monied interests.
4) As a result, the materials that get produced, and the topics that get covered widely are based on where the money and social buzz is at right now. Those who are doing original research, particularly things that are experimental or exploratory in nature, are not well supported at all.
5) Because we’ve tied blogging and teaching and tweeting to hireability, plenty of people enter into the bazaar not because they particularly want to, but because they must do so in order to “prove their worth” to potential employers or clients. Again, this clearly benefits those who have lots of time and resources to burn, and burdens everyone else with massive amounts of shadow work. We cannot be surprised at the lack of socioeconomic and cultural diversity in light of this point.
6) The success of the few enables the selling of dreams to the many, creating a cottage industry of infomarketers. In effect, these are the folks who sell pickaxes and maps to gold prospectors, and it’s no surprise that their business is booming. Some of these folks are brilliant and insightful, but many are hucksters.
In light of all this, we cannot say “this is just how the internet works”, as true as that statement is now. We have to ask, how do we make things better for our future selves and those who will come after us down this road?
People who are good at writing educational materials need to be paid to do this, because it's hard and takes a long time. I don't see any possible objection to this.
Since money and social "buzz" follow the interest of many people, it seems obvious to me that the educational resources should be focused on those areas instead of periphery areas that few people are interested in.
Other people have said this, but favouring people with good communication skills in the job market is part of a functioning meritocracy.
The reason for the lack of diversity has very little to do with educational resources available on the internet and much more to do with cultural attitudes in the West. SE Asian countries have much greater participation by women and they use the same teaching resources.
I smell a political ideology being thinly justified by some talking points here. But I'm open to being wrong...
I have been paid to do this for five years with practicingruby.com, without ever relying on commercial support. It's 100% reader funded and reader focused, and it's the largest open-access library of Ruby-related articles you'll find in the world.
> Since money and social "buzz" follow the interest of many people, it seems obvious to me that the educational resources should be focused on those areas instead of periphery areas that few people are interested in.
Everything that is popular and commonplace now was a sparsely populated frontier before. Some folks need to venture into that territory now so that we all benefit from their explorations in the future. It would be nice to keep them well supported, if we want to maximize the benefits we'll reap in the future.
> Other people have said this, but favouring people with good communication skills in the job market is part of a functioning meritocracy.
Helping people with good ideas and a good work ethic is a job we can and should do if we possess the means to do so. The exception may not prove the rule, but I suppose you can say that my communication skills are adequate given that this is my essay we're discussing on the HN front page.
The only way I got to where I am was through support of mentors and colleagues who had helped me in the early 2000s, when free and open source software was predominantly a hobbyist activity and not a commercial activity. There was at that time a real sense of doing things for the common good, rather than just to serve your own interests.
And as I've said before: we rank on the first page at Practicing Ruby for the google search term "infrastructure automation", we're on the first page for "actor model", first page for "Law of Demeter", etc. Several of the contributed articles we've managed to drive so much traffic to are by people who consider themselves intermediate developers, and amateur writers. I've used the money from my paying supporters to: pay contributors for their work, help them edit, revise, and polish their work, and reach a larger audience than they would have on their own. This model does work! It's just hard to sustain because it's so foreign to most people.
I won't even entertain your comments about the lack of diversity, because it reveals your ignorance on the topic, and lack of concern.
1) If you don't think that gender diversity is strongly related to education, why are you participating in a group that's designed specifically to address that problem?
2) Are there not qualified female candidates for running RailsGirls in your area? If not, why do you think that's the case?
You're doing good work, keep it up, have a great life.
1. Create a "subweb". It is composed of the following
1A. A new specification for the publishing of information in complete form ( read structured content published together with templates to present them )
1B. Both server and client systems that implement the new specification over some sort of modified http but without using current DNS
1C. A free distributed un-filtered un-policed distributed DNS alternative
1D. A complete hierarchical index of all content existing in the new system
1E. A free distributed search engine allowing submission of content within the new system.
2. Make it clear the new system is for free content only
3. Disallow advertisements on it entirely
4. Tie in accurate hashing and allow people to dedicate as much hosting power as they are willing ( either to the content itself or index data to what hashes represent what )
5. Make it entirely secure with a public/private key system allowing people to establish meaningful identities within a system where their content is copied eternally.
6. Use the new system yourself to publish meaningful non-shit information and encourage your friends to do the same
By the way I really like your article and I think you are pointing out a crucial problem spot on that most people are oblivious to.
The easiest way to think of what must be done is just to look at what libraries are. Organizing information has been done by libraries for thousands of years and has a fairly established sensible process. ( which is being ignored by the internet )
1B: named-data networking, camlistore, ipfs.io, bittorrent's web -- very easy to censor since all content is uniquely identified by a hash
1C: ?
1D: Usenet, Wikipedia, Wikidata
1E: technically difficult, see history of YaCY and http://juretriglav.si/an-open-distributed-search-engine-for-...
2: Wikipedia, but content is rejected under arcane citation policy
3: Wikipedia, but content is rejected under arcane citation policy
4: http://ipfs.io & https://crowdprocess.com/
5. blockchain publishing
6. standard challenge of bootstrapping user-generated content in any new online network
The role and funding of libraries have changed over time and across cultures, e.g. being associated with cities or nations, being associated with universities and churches. Current libraries are struggling to retain access to book stacks, e.g. look at the history of the successful community fight to prevent the flagship NYC library from removing many of its books.
The system has to be brainless to use if it is going to succeed at all, and it has to have enough initial buy in to be worth bothering with initially.
Wikipedia as you have pointed out has arcane crazy policies that restrict it from having user generated content. It claims to only be a replication of printed articles. That is lies, but it is a pain in the butt to actually get valuable content into it and not have it removed.
Usenet suffers from not having structured data, being just chunks of text.
Github suffers from refusing to host binaries except for in weird cases ( there is a binary build hosting I think?? )
Bittorrent suffers from still being attached to hostnames for the most part ( I'm aware of the distributed system it has too but you typically can't get many seeds through it )
Most systems that allow content suffer from having copyrighted data on them. The goal of this would be for it to be publicly known that there is no copyrighted data, such that universities and such would be willing to run the distributed server, and censorship could be stopped by enough people running it globally.
I'm not focusing on funding of libraries so much as the fact that they have an established set of categories to put information into. There is no such standardized list of categories for websites to go into, and the creation of such is important to the future of the internet.
On the topic of categories: once upon a time, dmoz was an open competitor to Yahoo, when hierarchical categories were used for discovery. It's now the Open Directory Project: http://rdf.dmoz.org/ . Google acquired the best-in-class open data graph FreeBase and later killed it. The data has since moved to Wikidata, which is the closest we have to an open version of the proprietary semantic graphs which have been constructed by Google, Facebook, Bing, Diffbot, etc. Some related ideas at http://openknowledgegraph.org/
Agreed that end-user deployment and usability need to be brainless and the system must be initially seeded with high-quality content. Usability is expensive and often specific to vertical market segments, hence would be best implemented by competing commercial applications which operate on the public, open, distributed data. But there would be the risk of commercial centralization (like Github/Gmail) of the open protocol.
It will be a challenge to orchestrate the boundaries between commercial software and open content, but this dynamic is arguably responsible for the construction of the entire Internet. We need to find balanced and stable boundaries between commerce & commons, because neither is going away.
http://en.wikipedia.org/wiki/Structure_and_Interpretation_of...
SICP is an extraordinarily difficult book; "Simply Scheme" by Brian Harvey and Matt Wright is a good stepping-stone.