Python 3 at Facebook
lwn.net
lwn.net
I wonder why that is? My theory is that it has to do with the new Unicode encoding; if 99% of your strings are ASCII compatible, the new encoding will be twice as efficient. Seems unlikely that would account for all of it though.
One of the interesting side-effects of this, is in 3.7+ dictionaries are now ordered by insertion, which I'm sure will help fix tons and tons of subtle non-deterministic bugs (and probably expose a few too).
Although I'm not 100% on the performance characteristics of the new unicode implementation. The unicode first string handling is a massive improvement when it comes to non-English languages. So IMO it's worth a hefty performance hit, based solely on it's own merits. In practice, it hasn't seemed to make a meaningful difference to me. So I'd wager it's good enough, unless you're hitting some sort of use case specific bind. In which case, it's probably time to leverage the C bindings anyway.
edit: Ah, I remember. His talk uses slides that were created in sphinx/readthedocs, but IIRC, the link to those docs never worked for me: https://twitter.com/raymondh/status/867021035719110656
and
https://www.python.org/dev/peps/pep-0468/
The implementation is based off of PyPy's "compact" dictionary implementation.
The change to dictionaries seems a lot more likely to make a difference, although it was introduced much later.
3.6
> The unicode first string handling is a massive improvement when it comes to non-English languages. So IMO it's worth a hefty performance hit, based solely on it's own merits. In practice, it hasn't seemed to make a meaningful difference to me. So I'd wager it's good enough, unless you're hitting some sort of use case specific bind. In which case, it's probably time to leverage the C bindings anyway.
That's because everyone had to use Unicode strings on Python 2 already, so the en/decoding and Unicode handling overhead was already there. Applications not using unicode strings were mostly just buggy or didn't work outside ASCII+. The major difference is that Python 3 string code is a lot less fragile than Python 2 code, because it either works and does so for non-English text, too, or it doesn't. Meanwhile Python 2 code would appear to work fine until you started to drop those sweet umlauts and got exceptions all over the place.
>3.6
It was an implementation detail in 3.6, 3.7 made it part of the language spec.
Which is good, can finally get rid of the metaclass I hacked together for my spark (earley parser that used to be used in the python build process to parse ASDL) shenanigans.
After you have used it for a while, it is a great feature. I would now be happy to take a small dict performance hit in order to preserve the ordering guarantee.
First implemented in pypy, then ported to cpython 3.6.
Talk by Raimond Hettinger about the new implementation with several other implementations:
The rationale is that even if they clearly said "we might make this non-sorted again later", in practice too much code would end up relying on the new behaviour (whether by accident or because people didn't pay attention to the warning).
So if changing the behaviour is likely to become impractical anyway, they might as well make it a promise so people can benefit from the guarantee.
They made essentially the same choice when they changed sort() to an algorithm that happened to be stable, about 15 years ago.
Being from Europe, my strings are still 99% ASCII compatible. But 1% is about four orders of magnitude too big for a failure rate. Every single string that goes through an application and isn't ISO-specified (bank account etc) will sooner or later be hit with unicode. Just wait 'till bank accounts allow emoji identifiers.
Isn't it great when engineers upgrade your codebase in their free time.
I always take the opportunity to identify some long-term-beneficial changes which would look good on my CV, and run with them even if it means investing my free time before I can present the potential to the business.
Probably worth adding that we ultimately created a python foundation team, of which Jason is a member.
Here's something dumb I created in my "within working hours" free time that Facebook let me open source: https://github.com/facebookresearch/py2bpf
More companies should encourage hiring developers who would solve problems in their spare time. When I applied for an apprenticeship at 8th Light, they had me write a tic tac toe game, and I wrote it in 4 or 5 different languages, just for fun. I think that's why they hired me.
Man, it would be good to have an employer. Any takers?
I wasn’t aware that Facebook used python so heavily. Presumably much of their code is in Js these days?
I’m so tempted to try and get a job in software eng sometimes, it’s by far what I have the most fun doing.
If you are at school, university, whatever that is, you need to be exposed to concepts. Those will rarely change. For instance, Lisp already had garbage collection and object orientation decades before they were available in mainstream language, and the basic concepts still have not changed.
Once you understand then, learning a new programming language is done in a matter of days (libraries and common idioms takes more time). You are basically transferring concepts you already know to a slightly different implementation, but there will be hardly anything entirely new. More so for "mainstream" languages.
Then you need practical experience, which do not require teachers to be involved.
I think the idea is that you should be able to pick up the language and framework as you go, if you do actually have the concepts down. If your generalised knowledge isn't to the point where you can apply it to some arbitrary framework/codebase in some arbitrary (imperative) language and start being productive more or less immediately (with documentation open in another tab), then your generalised knowledge just isn't at the level where people care about it, yet.
edit: That generalised knowledge they're expecting absolutely might go beyond what you're shown in school (it depends on the school). If you draw the line of "fundamentals" at "what was in my curriculum", then that's an arbitrary line you're drawing, and it's not the line that most of the industry is using (even if people are using their own coursework as their meter). If you consider (eg, in a web dev context) HTTP requests and MVC "fundamental", then you should be able to pick up Rails or Django or whatever without much trouble. Whether or not your school had classes on that doesn't really matter to anyone.
No, I got that. My point was that no one cares that you completed an algorithms and data structures course, they care that you know the fundamentals of (the subgenre they're interested in of) software development. You're just using a definiton of "fundamentals" that excludes the (entirely language and framework independent) concepts they care about that you don't know.
A take home project using a specific language and framework does not require you to know either to complete it before you start; if you know the concepts, you can pick them up as you're writing it. If you've never seen MVC before (and the knowledge you have doesn't allow you to grok it fast), then in a web dev context I would argue that that is a fundamental concept you are missing, which was my point.
Interview questions can be different, but you didn't mention them in the post I responded to.
So, while a student may have strong CS theory, they may lack industry specific "fundamentals" and companies may discriminate against those candidates because they lack that industry specific skills (that otherwise could easily be picked up).
I argue that traditional fundamentals as defined above are more important than industry specific tooling or frameworks, since the tooling or frameworks change but the core theory does not. I would like industries to not discriminate against graduates that come from a strong theory background but weak industry specific background, since these people should be able to pick up whatever tooling or frameworks on the job.
What does "otherwise" mean, here? The context we were discussing is that they asked you to quickly pick it up in a take-home assignment, and you couldn't pick it up that quickly. I don't see how this is them being shortsighted; it sounds like you just not delivering on the promise of your "fundamentals".
> the tooling or frameworks change but the core theory does not.
I 100% agree: someone who learnt MVC with Smalltalk in 1979 won't have any trouble picking up Rails' or Django's or whatever's version today. The concepts are timeless.
edit: to boil it down:
> these people should be able to pick up whatever tooling or frameworks on the job.
Yes, and if they can, they won't have trouble picking it up in the process of writing a take-home assignment. If they can't do that, then they just failed a litmus test.
I never failed any take home assignments, so this isn't about me. I am speaking more generally and not about me personally. My point is fundamentals don't change, frameworks do.
Edit: For some reason I can't reply to your response of this question. But no problem. I see your point and I do agree with what you're saying. Thanks for clarifying.
It was generic "you", the hypothetical person who was having trouble with take home assignments but not interview questions that you were talking about initially. All of my points were about that hypothetical person, and not you personally. Sorry for the misunderstanding.
> My point is fundamentals don't change, frameworks do.
Yes, that's what I was addressing. Companies aren't asking about frameworks when they give you a take home question, they're asking you to learn the framework if need be, which will be easy if you know the fundamentals. I'm not sure where you're getting lost on this point.
Most companies have very short-term thinking and are looking to hire for the task at hand (to be fair to them, it's not always good to over engineer or over invest). If you do get a job, the problem can be in 5-10 years when your company's needs change or you want to move up in your career.
It may not apply equally to tech or could be a generational thing, but generally trade schools taught specific job skills while colleges taught concepts (and usually omitted a lot of the specific operational skills). I've heard my dad describe it as a familiarity with the wider discipline and the knowledge of where to go for a deeper understanding, when needed. A lot of the baby boomers I know went back to school later for their Masters or PhD because that was a limiting factor for their career advancement.
Less and less often companies are investing long-term into employees. It's up to the individual. That being said, not everyone has to be a lead or a CEO. You can have a long and fulfilling career being a really good mechanic or code monkey. It's such a well earned stereotype to see people go into management and really miss the day-to-day coding work.
Industry seems to expect CS degrees to be vocational degrees.
E.g. when I was at FB I joined a C++ team while having written about 200 lines of C++ before in my life — we also had proper C++ experts in the team, and they cared more about experience in the problem domain. All of these conversations were held _after_ I was hired though — the pipeline is a mixture of hiring experts for specific positions and "hire people that seem good enough, and let them find their own teams"
I think this simplifies the truth too much. At least in my limited experience, you work in whatever language your team is working in.
Its not just "what your team works on", but what fits the task/what the existing tooling is in. If you're only working on a single product/project ("I develop the android app for XYZ product"), you likely work in the same language or two as the rest of your team. But that isn't always the case.
In more practical terms, you are not going to write a cronjob in java just because that's what the main system is written in.
Many of the "crappy" companies do look for "X programmers" even when there is no need, just because of the tech they are currently working on at that specific point in time. Even when they are large (especially when they are large, it would seem).
https://www.payscale.com/research/US/Job=Software_Engineer/S...
The prominent languages in each layer have pretty substantial code-bases.
More importantly, there's an engineering culture of eschewing "code ownership" and encouraging "code upgradability" so that motivated folks like Jason can drive these large changes.
Nice. That's cute.
Very roughly speaking, a way in which a script can invoke a piece of Python code.
Turns out there is another person in Facebook with the same name.
Per LWN.net:
“The "subscriber link" mechanism allows an LWN.net subscriber to generate a special URL for a subscription-only article. That URL can then be given to others, who will be able to access the article regardless of whether they are subscribed. This feature is made available as a service to LWN subscribers, and in the hope that they will use it to spread the word about their favorite LWN articles.
If this feature is abused, it will hurt LWN's subscription revenues and defeat the whole point. Subscriber links may go away if that comes about.”
It's rare that we rescue an older story and while it's waiting to get re-upped another repost gets traction, but that's what happened.