At least one of the three OPs is a foreigner. The problem is much larger than the US.
645 karma · joined September 6, 2012
http://research.microsoft.com/en-us/um/people/lamport/pubs/h...
The article was more about: Python programmer meets C, C introduces him to malloc(), programmer falls in love with malloc, C gets jealous and in a fit of rage deallocates programmer.
I think this extended love affair for C comes from the fact that it's the least-common-denominator in terms of what our OSes are written in. At some point everything has to make system calls that are using C calling conventions and data structures. It makes sense that the base layer of everything will be written in C. It's an historic anomaly, not some fundamental awesomeness of the language itself.
That said, it's a fundamentally awesome language, especially to build OSes in, but that's tangential.
It's speed.
It is also almost always interesting. It reveals a boundary that has been crossed, a bottleneck plugged. I think that us HLL devs always like to think we have C in our back pockets. We say, "well if performance is bad we will profile then rewrite that bit in C". Then, when faced with the reality of actually having to write something in C, well it's a different story. We think the performance increase will be dramatic, but often we are left with something that performs worse than the original HLL equivalent, and that leaks memory and frequently crashes to add insult to injury!
At the beginning of my career, I spent fourish years professionally writing K&R C. Before that I had convinced my profs that I could submit my assignments in C rather than the mainframe Pascal that was being used by the other students. The balance has been spent in Smalltalk, Java, Python, etc.
I must say that I really like the re-adjustment. Even though there are 'endless banks of RAM', C forces the programmer to be stingy with memory and come up with neat hacks, simply to avoid the enormous pain of being too clever.
Using malloc() is like lying. You've got to remember them all.
Yeah, just exit! That's the kind of cool thing about these small, short lived Unix programs like grep and cat - chew memory (stack or heap, up to you!) and bail out - let the OS clean it up. It is essentially the same argument as "prefer large mallocs to small" - the process itself is a 'large malloc'.
"Every day, we collect around 8 million data points on exercise and video interactions, and a few million more around community discussion, computer science programs, and registrations. Not to mention the raw web request logs, and some client-side events we send to MixPanel."
OK - 8 million records per day. Let's double that for the argument's sake.
Even if they were fairly fat records (1Kb), that's only 16Gb / day. That makes it around 2 months / TB.
I can easily put together a machine with 20TB of storage and run a traditional free relational DB (or even a single free node of Greenplum) and store more than 3 years of this data.
Then bang against it with SQL. Transactions are free.
We are hiring for quite a few engineering and non-engineering positions, but I would like to highlight the roles available on Chango's Large Scale Data team where we have two open positions for intermediate/senior software engineers.
At Chango, we process 20 billion or so unique data points per day, and are trying to minimize the latency between times when we collect the data and make it available for the real-time distributed systems that need to use it.
We use traditional SQL databases (Postgresql), columnar data warehouses (Vertica), networked/local key/value stores (AeroSpike, KyotoCabinet, LMDB, memcache), as well as a many node petabyte scale map/reduce cluster (Disco/DDFS).
You will be programming in Python, Cython and C with forays into Go and Erlang. You are comfortable with your code running on 100s of nodes in multiple data centres. You have solid knowledge of Linux internals and tools, networking, disk, cache and memory subsystems as well as techniques to measure and optimize all of the above.
You have thorough familiarity with relational technology/theory, ACID, transaction processing, networked and local key/values stores, data warehousing and columnar databases. You should have experience with map/reduce and distributed file systems (Hadoop/HDFS, Disco/DDFS).
You have a firm understanding of the various and sometimes exotic data structures and algorithms used for processing large data sets. You will use bloom filters, hyperloglog, skip lists, b+ trees, prefix tries, hashing in all it's multitudinous forms, MVCC, two-phase commit, data compression/encoding, etc.
You are familiar with and are eager to build distributed systems. You are familiar with shared-nothing architectures, replication, fault tolerance, queueing, messaging models, CAP theorem, and distributed locking.
Competitive salary, equity and benefits.
contact: tim AT chango.com or on the site
In the late 80's and early 90's, rapid prototyping was very fashionable in the 'anti-waterfall' software engineering schools of thought. The idea was to use dynamic languages with good development tools (various Lisps, Tcl/Tk, Smalltalk) to rough out the idea, which would then be rewritten in 'real' languages.
Of course this tactic failed successfully! We shipped the prototype. Nowadays, it would seem absurd to develop a web application (for example) written in a 'real' language like C++ or PL/1.
I have seen so many furious sprints towards the 'finished' system with so little knowledge of what actually needs to be built. The only part that has changed recently is that there is now typically a unit test suite along side that correctly tests the wrong thing.
15 minutes of US tweets would fit on your phone :-)
Windows 3.1 was released in 1992, which was the real starting adoption point for the majority of the market.
By that time, the NeXT computer had already been out for four years, the Mac and X Windows were eight.
I was actually just pointing out that software development, as a discipline hasn't advanced that much since 1990. The number of KLOCs freely available to use within your code has increased dramatically. The documentation (which was on paper pre-WWW) has improved exponentially. The tools, OSes and languages? Pretty much a plateau there since the mid 80s peak.
In the early 1990s, many developers were using an OS called Unix, with esoteric tools like vi, make, sh, and emacs. They wrote software in ancient languages like C, C++, Python, and Objective-C. Their programs communicated over TCP/IP networks using sockets. If you were lucky enough to be working on a military contract, you would be exchanging documents using XML and HTML papa SGML.
Everyone suffered until the major breakthroughs of Javascript and PHP.
The best advice for a young Jr anything is to do the job you were hired for exceedingly well. If the words obnoxious or needy are ever used in the same sentence as your name, you are doing something terribly wrong.
I would say the downside of this 'rich' environment is that your tools tend to become language dependent. Early Smalltalk code versioning tools were fairly easy to build (I'm guessing), but would definitely be Smalltalk centred. The unfortunate advantage of an ASCII text file is that it very difficult to have more than one interpretation of it's non-existent meta-data.
def f(x=()): x = list(x) ...
def g(y=(())) y = dict(y) ...
f() is the pattern for lists, g() is for dicts. Note that you can pass in regular lists to f() and regular dicts to g().
I am not sure how other teenagers made this awkward transition, but I did it by writing a tiny assembler in Basic. It was fairly easy because Basic had PEEK (get contents of address), POKE (set contents) and CALL (jump to address) that everyone was used to using to get anything useful done. I think I just got tired of hand assembling programs into POKE calls.
I remember getting a thin wire-bound book with a detailed disassembly (with comments!) of the ROMs and DOS as a birthday present. I think one of the thing that inspired programmers on the Apple ][ was the amazing system code that Woz had written. He set a very high bar for assembly language programmers. His code was economical, fast, smart, and tiny. He demonstrated how to squeeze the most out of limited resources.
I think it will do two things to Vancouver's startup ecosystem. In the short term it will be painful as you lose talent to the better funded new competition. In the long term it is a win as the big companies validate the Vancouver 'brand', increasing exposure and attracting new investment.
There is a lot of responsibility on the shoulders of language designers. There is the notion of technical debt that we all deal with as programmers, but the meta-debt the language designer accumulates is astounding.
Let's meditate a while. As we read this, some young, well meaning C++ programmer is writing another String class.
I don't even know if talking about Tony Hoare's NULL story is really that relevant. After all, that's a single feature of a programming language. In the context of C++, talking about NULLs at the language level is like arguing about the colour of your Edsel.
The standard now known as C89/C90 had been in committee for many years before being 'released'. This didn't stop the tool vendors (like Borland) from supporting the 'proposed' standard much earlier than 1989.
Unfortunately, commercial UNIX vendors (like HP, in my case), were very slow to adopt the standard and update the cc compiler in their distribution. This forced us to work in K&R for a good time after 1990, all the while grumbling that $150 MS-DOS compilers already 'had ANSI'.