d4nt

Your point is a good one. Without wanting to get too deep into a design for some hypothetical system, I guess I was thinking that I'd either try and modify the current request such that it can return something to its caller (e.g. Render a web page saying "Loading..." with a setTimeout call in it). Or use .NET's HTTP request library which has a delegate callback option that lets you get on with other stuff.

I think async is a systems programming tool rather than an application programming tool. If you're writing a web server or a web browser, it'd be useful. But for a web application, you're already sitting on a highly scalable and robust async library called IIS so when you need a background task the easiest thing to do is to make it a RESTful API call. I find many web developers now intuitively break up their application into parallelizable chunks without ever really thinking about async or threads, they just just say "this web page is taking a long time to load, maybe I can defer some of the work until after the page load" or (as you seem to have done) "maybe I can use tools like AppFabric Workflow to orchestrate all these small tasks".

Part of the problem here is that any open [web] community will eventually contain more people who've joined after you than people who joined before you. Consequently, stuff you learned long ago is still new and interesting to the majority of the population.

I had a interesting discussion with @ColinWright a few months back about how you might design a forum based separate cohorts for each time period in which people joined, and on scoring each post separately within each cohort - if I had a few months to spare I would love to try building & launching that. Sadly, I don't.

But this is the public debate that Snowdon called for. The leaks put pressure on tech companies to defend their reputation and when they do, each new development keeps the story going.

You're right about Windows being very profitable up till now, but there's something clever in this idea...

Businesses pay for Windows, but ordinary consumers by and large don't, they get it for "free" when they purchase a computer. But hardware is now getting so cheap, that Window's license cost is really starting to show. This will only get worse.

To stay competitive Microsoft will have to do two things. 1) Compete in the consumer marker with attractive devices (phones, tablets, Xbox, netbooks), and 2) Remain the "obvious" choice for business.

In order to do #1 they either need to copy Apple and build premium devices by themselves or compete with Andriod/Chrome. Either way, their current OS revenue model of selling to OEMs is going to die.

To do #2 they need to stay as ubiquitous as possible. You may not think that having a few MacBooks and iPads around the office matters to Microsoft but once they're there all of a sudden things like SharePoint and Lync and maybe even Exchange start to look less attractive because a big part of their appeal is built on how it all works together with Outlook and Windows. A few years ago quite a few Microsoft products had web interfaces that only rendered right in IE, they've had to stop that kind of thing more recently because non Windows devices are becoming more common - that trend is only getting worse.

Open sourcing a Windows core and then selling server management tools and premium editions enables them to keep charging business while ensuring Windows stays relevant and as widely used as possible.

Yes, Azure. It's developed into a good offering and I'm pleased for them that they're doing well.

But, there was a time when Microsoft entering a market would send all the existing players running for the hills. They used their OS dominance to control what software OEMs installed and killed off every desktop software competitor they ever had. Your best hope was to sell your company to them. But they couldn't do that with the phone, tablet, search or cloud markets. They've built some good products and some of them are doing ok, but they haven't had the kind of automatic win in those markets that those of us around in the 90s remember.

I've always felt MS is two businesses, an enterprise systems (Office, SQL Server, Project, Dynamics, Azure) business and a consumer devices business (Windows, Xbox, Phone, Bing) which should have started building hardware sooner than it did.

They really need to either spin off a separate business, sell some divisions or just stop trying to compete in some areas. Experience shows their OS dominance doesn't count for much in the phone, tablet, search or cloud markets.

I think some business models only make sense at scale. While the caveat does apply to facebook (they probably could have been a profitable niche student social network) occasionally I see businesses that it just doesn't make sense to even start unless you plan on getting really big. Examples would be PayPal, Google and Spotify.

This got me thinking. I share many of the authors frustrations, but I realise that people are just trying to get a thing done, and they really just want to outsource all the IT knowledge, just like I want to outsource car maintenance and food production.

The real problem here is that the IT literate have historically been very bad at communicating how valuable their knowledge is to others. Just like the author, I do a hell of a lot for people for free.

I think the issue stems from the newness of IT, most IT literate people grew up in families where they were the computer whizz kid, and enjoyed showing off what they knew to their extended family, friends and neighbours. When we were 12, the praise, and maybe a bit of pocket money was all the thanks we needed.

Actually though, comparable fields of expertise charge a lot of money per hour and I therefore tend to approach them with respect. When I want a lawyer to arrange a house purchase I expect to pay a lot of money and even though I may just want the darn thing sorted, I know I have to listen and fill in forms correctly because holding up a whole house purchasing chain can have big consequences.

Like the author, I have been too willing to insulate others from the consequences of their computing mishaps without charging them for my time or making them listen to me while I explain what they should do next time. If more of us did that then perhaps people would be less casual about dumping their problems on us and expecting it to be fixed.

In the early days, Linux was not the only open source kernal out there. It's possible that Linus's blunt approach is actually a key factor in Linux's success. Society hasn't yet had long to establish a set of norms for working in distributed teams like this (where the members aren't compelled to do as their told by traditional employment contracts). Maybe he's right and removing the offensiveness would just bog Linux down in endless political games.

While I agree with the points you make, there are no easy answers here. The problem with technology concentrating wealth in the hands of fewer and fewer people is that it gets really hard to tax those individuals. Partly because these people can move around easily and avoid your taxes, partly because it becomes morally difficult when the amount you're taking off one person is literally 100,000x what you're taking off someone else but also because, unlike the middle classes, the sums involved make it worth those individuals' time to hack the tax system to avoid taxes. Companies at least have offices or retail outlets that are stuck in your territory and can be inspected by tax inspectors and have turnover that cannot be disputed. So successfully getting at the proportion of GDP that the government needs to be able to give everyone a basic income may well be easier if you go after companies.

Agreed, the idea that whole teams of people were employed to basically prevent the government being embarrassed, and that they view their role as critically important, suggests that the state has grown way to big in the last 100 years or so.

The definition of "anyone who does anything that might piss off a government" includes a lot of people. What if I become a political activist in 20 years time, I might well piss of a future government, should I stop using gmail now in case that happens?

Sounds exactly like Droplet (http://dropletpay.com) which is already live in the UK.

The value of a lot of these apparently lightweight B2C apps is derived primarily from the attention that they get from their users. If you can build something that gets attention, then that's worth a lot of money.

In a world where people are spending less and less time watching TV there is now a huge imbalance between corporations who have lots of money and want attention, and huge numbers of people with smartphones and limited attention spans. It's like a thunderstorm, all these electrons trying to get to Earth, then suddenly, Bang! Instagram. These valuations, and the developer salaries they fuel are just a by-product of all that money trying to get from point A to point B.

Google got huge by inventing one new channel for that money to flow through. And a pretty brute force one at that: You searched for "laptop" here are some ads for laptops... It's still ahead of everything else though, which is basically a re-implementation of the old TV campaigns but on the web. If someone could only figure out how to show you laptops just before you thought of searching for laptops, then they would be even bigger than Google.

I'm using GAE for QueryTree (http://querytreeapp.com) which is a data analysis tool.

The GAE data store is really nice for persisting objects that come from users filling in HTML forms, or equivalent. One user can only type in data so fast and that kind of volume is fine on GAE. Plus you get great read scalability.

However, if you're creating data in some sort of simulation or loading it in from elsewhere, use a relational database (you can connect to external servers from your GAE app). Loading a few million rows into the GAE data store would cost you a fortune and the lack of joins or proper SQL probably means you have to get them all back out into memory again before you can do anything useful.

QueryTree uses the GAE data store as a central hub for user account info and settings, then shards lots of MySQL instances for actual data work, which seems to work well.

I don't make a habit of spamming discussion threads but... my product QueryTree (http://querytreeapp.com) will load JSON files even if they have lists at the root, you can use the Append tool to bring multiple files together, and you can then use Filter, Sort, Join and Group without ever having to define a schema.

Admittedly, it's aimed at novice users so if you're comfortable with SQL it may feel limiting.

This is spot on, in many engineering jobs getting the right technical decision made requires you to exercise influence over management and peers. And having some influence in the bank involves a careful tending over a long period. If that makes you unhappy, try to think of it as building up karma so you comments go straight to the top of the list.

I'm sure this works for things which you might buy having only see the website (and maybe a few screenshots) but beyond a certain level of cost/complexity you actually need to have some real software for them to try before they'll pay.

I think big data is a bad term, the key issue is whether you're using data to drive your decision making, not how much of it you have or what storage technology your using. If you're Amazon or Facebook then, sure, you've got to crunch through many TB to get to an answer, if you're an insurance broker turning over $100m from 2000 customers then there's no way making a business decision should need more than a few SQL queries on relational databases. But saying you're into big data is now almost a shorthand for saying, we analyse our data and use that to drive our decision making.

A common cause of slowness in .net web app is multiple database round trips during a single request, all happening in sequence. You can write async database calls in .net but they're not the standard / path of least resistance, and by the time you app is hitting these issues, refactoring it all is a big undertaking.

The results do not surprise me, but what I find more interesting is how different ecosystems and frameworks can encourage your average developer to write scalable or non-scaleable apps. And I still think node's model has a lot going for in this regard.

You're assuming an lot of clever forethought on the part of Yahoo. I find it hard to believe their PR team are that cunning. It's more likely that Yahoo are desperate to rejuvenate and buying up the nearest hot startup (at whatever price it takes) is politically easier than forming an internal skunkworks to build something cool, which would be way cheaper.

As one of the 500 million people wants to "work less and earn more", I'd just like to point out that all Americans are gun slinging Christian fundamentalists. Oh wait, did I just make an assumption about a rich and varied culture all conforming to an exaggerated stereotype?... you know I think I did ;)

Yes, the relationship between page load speed and bounce rate is well proven: http://blog.kissmetrics.com/loading-time/?wide=1

> He said that the best entrepreneurs, like Zuckerberg, have a definitive view about the future and plan for it; they don't willy-nilly chase luck--using statistics, probability, and iterative processes--to stumble upon something, anything that flies.

I see no reason to lionize this kind of narrow mindedness in entrepreneurs. In almost every case, approaching a situation with this mindset will result in failure. I meet several entrepreneurs with this mindset every year, they are almost always convinced that they're the next Jobs/Zuckerberg and that everyone will be using their "Facebook but for {{ Social Group }}" within 6 months, despite the fact that they've been "working" on it for 3 years (part time) and so far haven't found a technical co-founder. The world is full of nut jobs like this, the fact that one or two of them happened to get fabulously rich doesn't even suggest a correlation, never mind causation.

EDIT: I respect Zuckerberg, he has many good qualities, he's focused on innovation and on maintaining engagement with his users, he's done a lot of experiments along the way, some of which have worked, some of which haven't. I do not ascribe his success to a refusal to deviate from the original plan and I'm not sure he would either.

No, step #1 is to put up a launchrock page promising FizzBuzz'd numbers and seeing how many signups you can get.

I've worked on a couple of these systems over the years and I think it's a combination of the philosophy of those languages: http://steve-yegge.blogspot.co.uk/2006/03/execution-in-kingd...

And partly a correlation between the type of company that chooses a "proper", statically typed, "enterprise ready" language and the type of company that hires "safe pair of hands" programmers who value predictability over getting stuff done.

I sense that, for a long time, the world of journalism has told itself that it is a fundamental pillar of democratic society. While true, the implication is that society owes it something. You might even extrapolate that a journalist doing an investigation that did not result in a published story, has still added value because they have kept the authorities in check.

However, the funding for this type of work only really existed as a quirk of the technology that newspapers used and now that the public has the opportunity to only pay for the stories they're interested in; I think society will need more constitutional rights regarding access to government data and a crowd-sourced monitoring effort to prevent corruption.

I, like you, don't feel a moral imperative to pay someone for putting in a days work regardless of what they produce at the end of it. I would donate to political campaigns regarding constitutional rights to open data though.

It baffles me how journalism hopes to survive the disruption that the Internet brings with stunts like this, and the odd pay wall.

I'm doing a bit of work with journalists at the moment, and while I think many of the individuals would like to innovate, many of the structures and institutions that exist do not seem willing, they insist in framing the issue as "how can we get someone else to pay for us to keep operating as we are now".

I long to see journalism freed from the constraints that a daily print run onto dead tree once imposed on it, and re-imagined in an always evolving, interactive and yes, paid for, model. Just don't make me buy a load of content I'm not interested in, and don't try and charge me for a floor full of people who're re-hashing syndicated stories or writing fluff pieces.

I think an important aspect of what makes Excel formulas (relatively) useable to people who don't think they're programmers is their functional nature. You don't have to simulate the program's state in your mind because there is no state. You hit return and it shows you what the result is, and if your formula gets too long you can break it up into multiple cells. Giving users a procedural programming language is only going to help a small number of Excels users - the ones who like thinking in a procedural way.

For me, the real pain point in Excel's formula language is in how the path of least resistance leads you towards unmaintainable code. "SUM(A3:K754)" is not very meaningful. If only named ranges were more discoverable and useable, we might end up with more examples like "SUM(ProductSales2012)".

Shameless plug: I'm working on a visual, functional programming tool for querying data that's aimed at people who currently only use Excel: http://querytreeapp.com. I think the advance in browser technology is going to bring a number of tools like this to market to challenge Excel's dominance and hopefully improve to quality of "shadow IT" in business.