fullwedgewhale

Not being BSD license I would expect it never to get beyond the ports tree.

I think that argument is only correct if you have fairly well understood pieces of work. I'm going to build an another on-line social recipe app. People type in recipes, they share those recipes, make comments on those recipes, maybe review the recipes, etc. I think their metrics break down very quickly when you start doing more interesting things like, taking automatically finding similar recipes based on the descriptions of the steps in the recipes. Or maybe performing a recommendation of recipes based on a user's rating of recipes.

Or, for that matter, correctly internationalizing the recipes and conversion of measurement units. (You don't want to have add 3.43272 cups of flour to 0.2347 gallons of water, when translating metric to imperial - you'll need logic to scale up/down quantities and round within a given tolerance - which is often narrow when you start baking pastries). Or there might be a service for this specific example that I'm not aware of, but the point is when you go beyond a very narrow scope, I believe this tool breaks down.

I view it as dangerous because it spits out a definitive answer without clear explanation of the model used to generate the estimate. If you look at other cost estimation tools (even closed source tools) will explain the model and assumptions used to generate the estimate. You can then make the determination if the model and assumptions apply to you.

It also doesn't indicate the kind of application (which is probably a largely stand-alone application). You can't stop some clients from going on to these sites and looking up a cost and not understanding what goes into an estimate. What would be even worse, in my mind, is to have other developers use a tool like this to generate a cost and then fail because they grossly underestimate complexity.

So having done cost estimation before, this is horrible. About all it would be good for is the initial swag at brochure ware. Generally speaking cost estimates should never be a fixed point and should be done in a range, with assumptions (that when violated) would allow the cost numbers to change. In addition, there's no measurement of complexity through requirements. Basically all these estimators are useful for only the simplest software projects with extremely limited requirements and scope. The cost of the software is a function of the known requirements which are carefully enumerated during costing to generate some sort of complexity model. (For example, function point analysis). That's fed into a cost model that translates that complexity model into a dollar range. That interval is then used to generate a final, contractual, number. But notice all the work that went into that estimate. What's more is that the estimate is refined during the work such that new data is incorporated into the complexity model, allowing changes in scope or requirements to be re-priced.

But what if it's 200% bigger? I remember reading an article that looked at the time it takes to load MS office under Windows. Even though computers over the last 15-20 years or so are much, much faster, the wall-clock time to get Word or Excel up and running has pretty much stayed constant. Granted, disks haven't kept pace with CPUs, but when your computer is 100 times faster and it still takes just as long... Is that a good trade off?

That may be true in Node, but as a counter example, in Ruby, I saw dependencies creep into projects where there would be some minor point like "I need to do X" and that's done by library Y. In addition, Gem Y does A, B, C and D. In order to do all that it drags in several dependencies which are not directly needed. When you actually look at X, you realize it's not that difficult to do yourself. So at what point should you just write the functionality yourself and at what point do you rely on external libraries (and any baggage they can bring with them.) You have to maintain that code (even if it is fairly trivial), but you then have to maintain your dependency (keeping the gem up to date, maybe making small code changes to accommodate breaking changes in the gem). It can be a real mess.

One thing that strikes me is the explosion in dependencies in most software. I'm guilty of this, too. I've seen plenty of examples where an entire library or framework is added to a project just for a couple of features. Add a few libraries like that and suddenly you have a few megabytes of additional libraries, where maybe 90% or 95% of the features will never be used. A good article a while back looked at common unix utilities, comparing the size of commands like cp from the 1980's to the present. Most of the bloat had to do with features that almost nobody ever uses. It wouldn't be so bad if everyone used the same set of libraries. For example, almost all applications have a dependency on certain core libraries like libc.

But we often use different libraries that do essentially the same thing or different versions of the same library, so instead of 1 copy of libfoo.jar, I have 2 copies of libbar.jar and 4 copies of libfoo.jar that may all do essentially the same thing. Then I have essentially the same functionality in C++ (some libraries that wrap collections), Python (where maybe one of they python versions wraps one of the C++ libraries, but a different version). And of course I have a version installed in each ruby environment. Add to that their dependencies, and the dependency's dependencies, and you have a perfect storm of craptastic. So libfoo.jar version 1.2.3 depends on libbaz.jar 2.3.4 which depends on libqux 1.5.7. Let's say each one is 250k, and all I ever used was some list sorting utility in libfoo.

But I don't know what we could really do about it. You can't force everyone to program in C++ or limit them to a set of blessed libraries. I think maybe developers could be more judicious about when they could add a few lines of code and when they actually need to bring in a hard dependency an an external library. And it happens with commercial software as well. Maybe this is just the way the world will be.

But not all over the world. In some countries, even in Europe, you can be a physician and do 'okay,' but being a computer programmer would be better.

I think top talent competes globally so it's able to either move to where it makes more money or demand higher rates. It's the anonymous guy in the cube whose skills may range from incompetence to excellent, but you don't know what you're getting at first.

I think one difference that I've seen is that the US has a very different attitude toward computer programming. In the US it's more glamorous? That's not the right word, but it seems like when I talk to a number of foreign developers who emigrated to the US, it's more about being a good job than it is about being a genius hacker, rich by 25, or being a technical tour de force. I see a lot more women overseas who become programmers. Maybe because the culture is different, and becoming a programmer is like becoming an architect or engineer?

I think that different culture might self-select US developers the same way top schools produce highly motivated, successful people because they recruit highly motivated, successful kids. Not to say every US developer wants to climb to the top of some technical or economic mountain, but it seems like maybe there's a smudge more passion among US developers. Also, intelligent, math oriented individuals have other avenues in the US, like finance, (and unlike the rest of the world) dentistry and medicine are lucrative careers. Meaning you choose programming more because you want to program, but in other countries it's because it's a better paying job.

I've also noticed that in some countries, once you've done a few years writing code, you quickly want to join management ranks and develop a coding allergy. More so than in the US, where it seems like 50 year old developers still want to write code. I get the sense that, in some countries, if you don't get into management then you are a failure at some level. So in the US you can find someone with 10+ years experience developing software, but in other countries you just have people who've stagnated and never moved up.

I dunno.

Excellent suggestion. I highly second that. Yes, even though more and more vendors are making that harder. For example, claiming that reinstalling the OS voids your warranty.

Google puts out free or cheap services and devices because they're trying to draw users to their platform, which gathers metrics about usage. These feed into the services, but they also feed in to Google's need to be a better advertising platform. Although we think of Google as the company behind the eponymous search engine and Android phone, it's actually an advertiser. I'm not sure if it's still in the 95% range, but it's still high. I have a 7" Nexus tablet which (for the time) was a great tablet at an insanely good price. But I knew I was essentially giving up information about what I read, what I listen to, and what apps I used in exchange for the free services and discounted device.

In contrast my Apple devices aren't cheap, and there is less collection of data with the intent of selling my attention to third party advertising clients, but they are more expensive and I pay an iCloud subscription fee. For the most part the Mac software doesn't suck and works reasonably well. I don't love everything about it but it's more appealing than Picasa for photos. I do pay for extra storage on Google to back up my pictures (over 100 gigs of family photos). But I am definitely paying more Apple in a direct sense (buying computers and phones + iCloud) than to Google (Google Play and 1 device). Apple's margin on hardware is high, but they make a quality product (for the most part).

What I give Google is information about myself, my work, my family, the music I listen to, the books I read, and anything and everything I ever searched for. They're pretty open and transparent about this, and I pretty much understand what I'm turning over in exchange for their free stuff. See the Google dashboard on you. If there's an application or service hosted by Google, I pretty much understand their intent is to extract data from that service or application.

That works for most people, even though they really don't understand this exchange or think about it. Free is a powerful lure and most consumers have come to expect not to pay anything at all for software and services. That's why they download a free game and keep playing it even when they find out it sends the contents of their address book off to some weird holding company. It's also why they get suckered into free games (because 4.99 is just too expensive on an App store) and then spend multiples of that amount in in-game purchases. It's why they're willing to turn over even extremely intimate details of their life for free e-mail or messaging services.

But the incentive for Google is not to make great software or hardware because that's how they make their money. Their incentive is to roll out services which help them collect information to feed their need for information about your tastes and behaviors. If that photo app you're using doesn't really give them more information, there's no incentive to make it better. Apple has an incentive to make software good enough to support their device/computer sales. Microsoft (arguably) has an incentive to make software good enough for manufacturers to license it. Apple and Microsoft, no matter how much people love to shit on them, actually have more straight forward, simple motives. They have an actual financial incentive to care about the quality of the product they deliver because you are paying them (directly or indirectly).

tl;dr Google only makes software good enough to draw you in to collect data. Other vendors are selling yous something and therefore have an interest in making that thing better.

I have to think one major issue for a German startup (for example) would be that their ideas might not work in France because of French laws which might prohibit some feature of that German service. So, the Market for a French firm is (at least initially) France. That's significantly smaller than the US. Coupled with the fact if feels like most US managers look at a laptop as a basic tool while many European managers seem to regard it as a tool for their subordinates.

http://eclkc.ohs.acf.hhs.gov/hslc http://en.wikipedia.org/wiki/Head_Start_Program

It's an early childhood learning program. Nothing's perfect but it has a pretty good track record.

The problem I have with the talk is that he's focusing on one narrow part of the economy and then using that as a proxy for the whole economy. It's also obvious he doesn't look at some of the negatives related to income inequality. For example, high levels of income inequality correlate with high levels of political instability and high levels of corruption. Countries with high income inequality are rarely gleaming examples of economic efficiency. They're often not very nice places to live. Countries with low levels of inequality (like much of Northern Europe) are nice places to live.

As Paul understands that we're all connected (you can't pull on one part of the bed-sheet with out pulling on the rest of the bed-sheet), he hasn't made the connection that if one side of the bed is dragged into being a shit-hole, the rest of the bed will eventually be a shit-hole, as it's all connected. In South America, for example, kidnapping is such a huge problem. The effect is to make these places less nice for the wealthy as well as the poor.

A lot of times the policies that reduce inequality aren't kill all the start-ups/crazy take all the money from rich people policies. Often their simple things that don't take a ton of money like head start funding, lower the cost of post-secondary education and training, and providing certain basic services like health care. What the US has done is to look at balancing the budget by reducing taxes on the very wealthy (upward pressure on income inequality) while taking money from programs that reduce equality.

There's a belief that most people who are poor are not working. Actually, many of them work quite hard. But washing machines work hard, too. Simply working hard doesn't get you ahead. In some cases there's a fair amount of luck involved. I sometimes look at the startup economy a little bit like pulling the slot lever. You can do everything right, but a lot of things still have to go well (some of which are outside of your control) before you cash out. Paul rightfully calls this risk, meaning that it's not guaranteed that an investor will get his pay out and it's far from guaranteed a founder will get rich. But it's bizarre to think keeping the US from sliding into an economic hell-hole will somehow destroy that slot-machine economy.

Under the DMCA it's illegal to root your devices: http://www.wired.com/2015/04/dmca-ownership-john-deere/?mbid...

Got it, thanks!

It raises some good points. It makes me think of other projects I've been a part of where the working, existing system was starved for resources in favor a new system which didn't exactly dominate in performance or capability.

I've been programming for almost 20 years but am new to Python. I'm currently working on an API for a product and testing it against 2.7 and 3.4. It's not the end of the world, and if it works for 3.4 it pretty much works in 2.7. Granted, it's not the world's most complex or deep piece of python, but is there a real problem in making 2.X code run on 3.X? Is it really that big of a change?

I'm sorry, but the switch over cost to setting Bing as your home page instead of Google, or using Firefox which defaults to Yahoo, just isn't that much. It's one thing to have a lock on the desktop market where switching over your desktop software (or 10,000 company desktops) is a significant cost.

For users with IE or Firefox they have specifically set the search engine to Google. That means users are actively seeking out Google. Maybe require all French citizens to switch search engines once a year? Make browser vendors randomize the choice of search engine?

I would imagine that people would still use Google, regardless, even if links to Bing, Duck Duck Go and Yahoo were on the page. I like Duck Duck Go, but every once and awhile I go back to Google because they do a better job.

Things I'm more worried about than Google's search hegemony:

1) The fact that new computers may soon be unable to load unsigned kernels

2) You buy a device, like a console, it is illegal for you to root it.

3) Content is locked out region by region, and VPN users are considered pirates.

4) Governments want to incorporate back doors to encryption - leaving all less secure

Thanks, I'll take a look at that.

Just in time for the X-Files coming back this summer!

I've got the book in my queue to read, but parallel programming is hard. I think it pretty much still stands that there is no way to show that parallel code works, other than to do a formal proof of each section of the code.

I think it would be in everyone's interests to require all publicly traded companies to put out 4 press releases during the year (one per quarter) with an obviously ludicrous claim. Either the algorithms would get better or they would go away.

Let's step back for a second and realize there's nothing magical about college. We can make the same arguments about the validity of a high school education. In fact, I can take your argument and say that providing everyone a high school education lowers every high school graduate's wages. We can then apply the same argument to middle school or any even providing any education at all. If we teach everyone to read and write, under your model, it devalues reading and writing. Because fundamentally, your model is one where there is a fixed pool of demand for educated people and beyond that any additional education add little additional value. And that is a completely wrong assumption.

The reason why education works is that it makes all parts of society function better. Having an educated population produces a higher functioning society. That's a society that, more often than not, is able to produce more wealth faster. In fact, it's in the enlightened self interest of society to have a broadly educated population. Basic research winds up driving GDP growth as science goes from the lab to products. Having more engineers and scientists is important, but so is having musicians, writers, nurses, and so on. All these types of education are important in a modern society. What's going on in the United States is that we are slowly short changing ourselves by making it harder and harder for people to get that additional education.

But in the process we're also creating a more stratified society. A lot of social mobility comes from the "first generation to go to college." By making college education less affordable, we keep people in the lower and lower middle classes that could have a transition up the income ladder from lower skilled, lower paying jobs to white collar professions. That's just one way we're becoming a society with little social mobility. American has moved away from the meritocratic American ideal. Now, even if you're an idiot, if you start out rich you'll probably stay rich. If you're poor, even if you're smart and capable, you'll probably stay poor.

Let's say you have a business process, like if a shipping manifest goes through any one of the following 3 cities, then you need to file form XYZ, unless the shipper is one of the following official government agencies and they've filed forms ABC and DEF. That was the original requirement in 1980. It was documented, put in a series of binders, and placed on a shelf.

1982, Adds another port to the list of special port cities, but only if shipping goods of type JKL or MNO. That change was documented in an inter-office memo and filed away. Except the only time you have the type of goods information is in a different module - so even though it pertains to the original business process, it's in another module that prints the ship's manifest to (physically) mail it to the insurer.

1989, the original requirements binders are moved to a storage facility.

1992, The memo is also sent to an archive facility. Original manuals have been destroyed because the records retention policy is 10 years.

1994, There's a change in the law and an emergency fix was put in, and the comments were put into the source code.

1995, The source code with the comments is lost, so an older version of the source code is recovered with the just the code change.

And so on and so on

Until 2015. You have 5,000 to 10,000 lines of code that deal with the original requirement. They're split into multiple modules. They reside in a source code base of 5,000,000 lines of code. The people that use your software have a combination of the software + a whole bunch of unwritten rules like: "If it's this country, and this port, and this port of origin - PF10 to get to the override screen and approve the shipment. Add 'As per J. Randal' in the comments."

The point of the article is that the meritocratic system in this country is broken. In part this is because we have systematically transferred wealth out of the bottom to the top. For example, focusing on tax cuts that benefit high income individuals while cutting support for public colleges. These two things are usually done at different times with tax cuts usually to stimulate the economy and spending cuts coming to reign in budget imbalances. The result is that you could work your way through college in the 1960's by working about 20 hours a week. Now you'd have to work 40+ hours a week to pay tuition at a public university. The difference income throughout someone's working life between a college degree and no college degree is about 1,000,000 dollars. So the middle and lower middle income people don't see the benefits of tax cuts but they bear the burden of college tuition increases. And their long term earning potential suffers. And the country as a whole suffers because we have a less well educated work force.

You can repeat this pattern with a whole host of other programs and issues. For example, the minimum wage is still behind where it needs to be if it were adjusted for inflation. We're becoming a casino culture where people keep playing the game because they believe they're going to be the ones that turn out okay. For the most part, your ability to leave your social class is pretty close to zero, with the rich staying rich because their wealth can insulate them against bad decisions. The only real mobility for most people has been down.

I'll throw another one down your way. An organization I worked with had a about 5 million lines of COBOL in one system (they had several more and this one systems was only about 15% of their total transactional workload). It used a proprietary pre-relational database that allowed users to do both queries (of a sort) and do things like the value from the query result + 1500 bytes.

They tried re-writing pieces in Java at a cost of tens of millions of dollars. Java was the new hotness. In addition, they built out a Java hosting environment using expensive, proprietary Unix hardware to reach the same production volume as the mainframe. However, it was grossly under-utilized because the Java code couldn't do much more than ask the COBOL code what the answer was to a question by using Message queues. More millions of dollars went to keep up licenses and support contracts on essentially idle hardware.

They tried moving it to Windows, using .NET and MicroFocus COBOL. But the problem was they would still be tied to COBOL, even though they (conceptually) had a path to introduce .NET components or to wrap the green-screen pieces in more updated UIs. But that in itself was a problem because all their people knew the greenscreen UI so well it was all muscle memory. Several workers complained because new GUI actually made them slower at their jobs.

They were stuck because they had no way to reverse engineer the requirements from the COBOL code, some of it going back 25+ years. Of course it wasn't documented, or if it was, the documentation was long gone. For the most part they were tied to that COBOL code because no one understood everything that it did and there were only a handful of COBOL programmers left in their shop (I think 6) and they were busy making emergency fixes on that + several other millions of lines of code in other systems.

They were, however, looking for an argument to retire COBOL and retire the mainframes. The cheapest solution would have been to stick with COBOL. Hire programmers. Teach them COBOL (because it was painfully difficult to find any new COBOL people and for various reasons they could not off-shore the project). Continue to develop and fix in COBOL (especially before the last remaining COBOL programmers died or retired). If you cleaned up or fixed a module, maybe move it to Java when possible.

The long story short is the decision to introduce a new technology, even in the face of an ancient, largely proprietary (since it's really about IBM COBOL on mainframes), and over-priced solution can actually lead to a worse outcome. Had they stayed with boring technology. Had they in-sourced more of their COBOL workforce. They might not have felt happy, but they would have been in a much strong, better position. Instead they were paying for a mainframe, and a proprietary Unix server farm, and software licenses on both Unix and z/OS.

When I last was there they were buying a new solution from Oracle which was supposed to arrive racked up and ready to go. Several weeks in they essentially said it would take months before the first of the new Oracle servers would be ready for an internal cloud deployment on which to try to re-host some software. I'm not even sure what they think they would be re-hosting but they talked about automatic translation of COBOL to Java.

Come Monday, here's what you'll get: "GitHub attack shows the dangers of relying on cloud." Or "Government software to protect children from Phil Robertson held hostage to attacks on a popular cloud service. This demonstrates the dangers of cloud. How can we be protected from the cyber criminals and pedophiles in their clouds? Here is are usual panel of people who didn't quite rate their own panel show. None of them know anything about computers, really and at least one uses their iPhone to prop up the short leg of their kitchen table. Panel, what do you think?"

"It's Obama's fault."

"It's the republicans."

"I make pee pee."