Rosetta: ESA’s ‘sleeping beauty’ wakes up from deep space hibernation
esa.int
esa.int
Boggles my mind what a group of engineers can do. Make you wonder if our industry's mind share being wasted on web apps and consumer gadgets because of the profit incentive associated? Just seems like an unbelievable symphony of fields all playing in concert, let alone having it all work as expected in production! Probably the most stunning example of this was Curiosity's Landing on Mars (the tethered landing blew me away!) [1].
[1] http://www.nasa.gov/mission_pages/msl/news/edl20120809.html
We can believe people like Elon Musk will be the ones guiding humanity towards space conquest, not state programs.
We need both. The deep space research that's only possible with $1e9-$1e10 budgets has to precede scaling up and commercialization. Furthermore, there's little commercial incentive, even for people like Elon Musk, to invest in basic science research like Hubble, JWST, the LHC, etc.
There is an expression I've heard jokingly about that, something like, "The brightest minds of our generation are working on click-fraud prevention."
I saw the best minds of my generation destroyed by adwords, starving hysterical naked,
dragging themselves through the back-linked sites at dawn looking for an angry fix,
Angel-headed hipsters burning for the ancient heavenly connection
to the starry dynamo in the machinery of night...
I survived through the Techstars Austin program and have been apart of the Austin startup scene for quite some time and what many of the entrepreneurs, whether small or big, are doing is much more important as whole than what your statement implies.
You should see the list of companies selected for the 1776 Cup; some were "appy" (but solving REAL problems!), others were hard science (new form of carbon to make capacitors more efficient to replace batteries), and some were humane (cheap, portable, and easy to build housing for displaced people).
In fact, I think it's myopic to assume that a business has justified its existence solely on the basis that it has experienced success on the market. A method to make a well-to-do individual or company even more well-to-do, despite a 2x or 10x cost to the competition or to society, is a winning business proposition. "Create the problem, sell the cure" is a valid business model. Feeding the poor is not. Like most of us here, I believe that in total the market is a force for good, but let's please not pretend that market forces are self-justifying or beyond reproach. In an adversarial system, the solution to one person's "real problem" can be a real monster to the rest of us. We have to make the good/evil call on our own.
Most of us are familiar with the game-theoretic concept of Nash Equilibrium and how, with the right rules, you can drive a local optimization scheme* to a globally anti-optimal outcome (i.e. the prisoner's dilemma). Since our economy runs on a local optimization scheme, it's entirely reasonable to wonder if there are any market segments where the Nash Equilibrium is particularly objectionable.
The marketing industry is a great example. I've heard it claimed that its purpose in society is to convey information to the consumer. But that's not how it makes money. It makes money by attracting customers, and indeed the ads that you and I see every day are so acutely tuned to this goal, to the near total exclusion of all others, that I doubt there are many of us who believe that watching ads helps the average person make better business decisions.
"Captain Morgan has the best brand of rum!" and "Dawny makes the softest tissues" are not messages that are of any use to society. Just the opposite -- they're probably false or, at best, misleading. Both customers and manufacturers would benefit from cutting the advertisers out of the equation. But they can't, because that's not a Nash equilibrium, and so the arms race continues. It's not as expensive as the actual wars we are fighting, but it's not too far off either.
Look, you're right that the whole "all industry does is churn out socially worthless CRUD apps" criticism is unfair. Industry has solved (and will solve) an enormous number of practical problems. But it's not myopic to notice when a business empire is built on a strategy of dubious social value or to notice when the market is systematically undervaluing some kind of human activity. Those things happen, and it's reasonable to complain when a petty/artificial "real problem" gets enshrined while the most glaring "real problems" get swept under the rug.
The scientific and humane startups you mentioned sound like steps in the right direction. I just hope that they experience a nonzero success rate. Market forces have a long and ignominious history of serving those sectors poorly. I don't have any great answers (apart from the obvious "don't defund science or social nets"), but the only way we can hope to find better options is by identifying what's wasteful, calling it that, and trying to figure out new ways to address the problem.
* While I sincerely apologize to any game theorists who object to my inaccurate terminology, I must point out that it was you guys who decided to call this kind of optimization "greedy."
I also think, despite its many warts, our environment (that of the USA and the tech / startup industry) is an arena within which agents can operate competitively and towards their own ends.
Is what we are doing the right way to do it for everyone? I'm not sure and it's also not my place to make that sort of judgement. But I do know it is working exceedingly well for many of us, improving the quality of life and advancing the state of our society.
Yes, I understand that. Sometimes this leads to a healthy, competitive environment with some seriously awesome features (the ability to replace incumbents, supply/demand matching, abundant opportunity, etc). But sometimes it doesn't. Sometimes it does crazy, stupid, wasteful things. Sometimes market forces are smarter than political forces in that they force people to deal rationally with unpleasant truths. Sometimes markets are dumb as bricks and create unpleasant truths all on their own.
My post did not propose any form of socialism. In fact, it didn't propose any solution at all. My post merely asked you to not pretend that the market's shit doesn't stink.
> your statement that the market's allocation is a waste is shortsighted because an environment in which someone decided what people should or should not be working on is an environment I want no part in.
I don't see how your desires relate to the shortsightedness of the plan I never proposed.
> I do know it is working exceedingly well
Is that a base rate fallacy or a "screw you, I have mine"? I can't tell.
> My post did not propose any form of socialism. In fact, it didn't propose any solution at all.
Looks like I read too much into your post I suppose and you're right that you didn't propose a solution. In that case I think I reacted to the idea I projected into your comment that the "solution" to the problem you're referencing was external resource allocation (rather than that determined by the market) - which is something I disagree with but maybe there's something there I should analyze about myself.
> I don't see how your desires relate to the shortsightedness of the plan I never proposed.
Again, you're correct here. It's obvious to me at this point that I was too emotionally involved in my interpretation of what you wrote.
> Is that a base rate fallacy ...
There was "screw you" content in there because I was projecting on to you something to resist and push against but I'm aware now of where my fault was. I'm intelligent enough to know that it is indeed a base rate fallacy so I have no excuse.
Thanks for taking the time to reply and thank you, also, for your candor.
While I would not excuse our resource allocation system from the list of things that should be justified... and notwithstanding that a key resource allocation component, the Government, is subject to a wild variety of inefficiencies...
I think that it ALSO takes a lot of hubris to tell the nation's 300 million people or so+ that you know better than they do, and that they should be sacrificing for the Noble Cause of putting people in space, instead of spending it on the things they feel like make their life better, even if it DOES happen to be the brand that spends more money on advertising instead of a similar-quality generic. (Also, for what it's worth, you should be aware that your interpretation of advertising is a pretty tendentious one.)
So there's that.
(+ I say the Nation to avoid extreme comparisons in living standards which may occur with a more global view, to say nothing of the diversity of economic systems.)
Of course he knows better than they do, given that, as you said, the choices are based on "more money on advertising", and that there are no real choices available, but only those designed to cater to the most selfish and short-sighed instincts of humans (i.e. that can sell easiest). Groups of people this size are not a collection of thinking individuals, they're a dumb fluid following well-understood rules that are exploited by individuals who understand them. The choice is an illusion here.
Of course. I'm not proposing a command economy. In fact, I wouldn't even propose to legislate away the industry I just maligned.
I do believe (in agreement with Adam Smith) that the market is not a suitable model for all kinds of human activity. Science, for instance. You can't determine the value of an experimental result until long after you have given it away for free, making it nearly impossible for scientific organizations to capture any of the value they create.
Next time the doctor tells you (or a dying relative) that you don't have any good options, reflect on the market's wisdom in spending only 2% of health care revenue on research. Does this figure represent some hidden insight or does it represent a boring fact about the way markets work that has nothing to do with optimality?
> It's not as expensive as the actual wars we are fighting, but it's not too far off either.
It's not that much different as people may think. What the real arms race and marketing have in common, is that they're at their essence a bare, exposed* positive feedback loop. Marketing done right seeks to cancel itself out; if two competing companies do their marketing in an optimal way, it will leave the customer in the same state as if no marketing effort ever occured (ok, maybe a little more tired from listening to all that crap, but otherwise his/her decisions will not be influenced). In doing so, it can in principle use up (i.e. waste) unbounded amount of resources. That's why I sincerely consider marketing to be a parasite and a plague of our times.
You know one other thing that work in the same way? Political campaigns. Each year each side hogs up more and more resources for their campaign, but done right, they just cancel each other out, ending up providing zero value for all the resources^ wasted.
So yes, I strongly believe we have a problem here.
* - think about looking at a bare, exposed core of a working nuclear reactor. Not fun.
^ - I keep using word "resources" instead of "money", because the latter invokes some weird brain failure modes in people, that make them think there's no problem here. Positive feedback loops like those don't waste abstract numbers, they waste energy, fuel, mined resources, paper, human time and human lives, and screw up our planet even more than it is already.
I am wondering what would be the impact of a law to limit the advertising budget of companies. I will be hard to apply it, but the goal is to force companies to get out of the Nash non-optimal equilibrium that forces them to spend resources on marketing just because if they don't, the competition will.
Marketing covers more than this. It isn't just how the pie is divided up, it is also about increasing the size of the pie.
In fact, if it increases the number of potential consumers, marketing can be rational and profitable even if it reduces the market share of the marketing company (50% of 10 consumers is less profitable than 40% of 100 consumers). There is a real rising-tide-raises-all-ships effect, especially in discretionary goods (eg, jewelry, many electronics, financial services, the list goes on).
Yes, but the size of pie is bounded, and at some point you hit the limit of customers you can reach (most companies service local population) and/or limit of the market (there are only so many people with enough discretionary income to afford whatever you're selling). Then you're back to fighting for division of pie.
A 10x cost to competitors is great (assuming no broken laws).
You were wise to qualify with "assuming no broken laws" because the schemes I was thinking of weren't legal, just almost impossible to prosecute ;)
> Make you wonder if our industry's mind share being wasted on web apps and consumer gadgets because of the profit incentive associated?
I find this sentiment strange. To me, your kind of excitement, which seems to be pure joy over incredible engineering feats, seems nihilistic. There are serious terrestrial problems happening right now. Massive environmental destruction, humanitarian disasters. Even in relatively stable places like the Bay Area I know many people who are teetering on personal disaster. These are extremely complex problems and engineers have extraordinarily powerful skills to help solve them.
I understand why people are captivated by things like comets and why they'd want to attempt these extraordinary feats. It's exciting and fun and there are few joys that can match doing something that's never been done before.
For that reason I don't begrudge anyone who chooses that kind of work. Earth's crises are incredibly depressing and there's largely no glory in helping a few people stay out of homelessness, or helping a kid get through college. It's risky, demoralizing work.
But I don't really understand why a mission like this would be considered a noble pursuit. Fun, sure. But not especially noble. Yes, basic space science has some value to society in the long term. But we have a MASSIVE backlog of basic science that is being unused. We have journals chock full of solutions to problems that are going unimplemented.
Even something as simple as logistic regression, which has been around for 70 years, is vastly underused. People get excited about bayesian models and neural networks and all this great research, but there's still a massive backlog of problems that could be solved with a simple regression or an ANOVA. There are no basic questions to solve, just the work of obtaining and cleaning data, doing the analysis, and packaging it in a tool that people can actually use.
You seem to wonder why more people don't take on extremely difficult challenges on the edge of possibility. But I wonder why more people don't use existing technical tools to solve the massive backlog of problems that are being ignored.
>Massive environmental destruction, humanitarian disasters.
Right?
How many time do you think you have?
Back to space. There are some long shot wins with research like this, things that are pure research prerequisites for big changes. As in, once we've looked at and followed a comet, we'll have the experience doing remote landings on small rocks in space. Why not do that to asteroids? Perhaps we can find an asteroid with an unusually high concentration of rare earth metals. That could bootstrap fuel cell research by dropping the price of rare, useful materials like platinum. Or gold. Such tech would help alleviate many problems at once. It might break economies, but it may also put us post-scarcity for manufacturing tech.
I don't mean to make it sound like I don't think it's not worth doing just for its own sake. I really do. But I also think that the potential wins we're setting up for the next generation or three down the road make starting now worth it. Being slow and thorough is a risk of lost opportunity, potentially huge loss. I also think there's plenty of brilliance available to work on the other problems, too, and I am often confused that we feel a scarcity in that respect (like how there are food shortages despite incredible productive ability - I just can't properly wrap my head around the subtle complexities that cause such strange things).
So I think you have a point, but I think it's a bit harsh to call it nihilistic. That's certainly there, but there's more good to be had, too. Can we do better? Sure, but I'm cynical enough that I'll accept the occasional wins for what they are, while secretly hoping it changes to world for the better.
While I completely agree with you on the massive backlog of problems that could be solved with very simple data analysis (this is the reason I am in the field), your thoughts about data modeling techniques are very much off the mark and sound like something read from a dated statistics textbook.
First, on logistic regression:
-I can tell you with certainty that logistic regression is not vastly underused, and is consistently among the most popular machine learning algorithms [1]. From personal experience I think it's likely the most powerful model there is.
-People are excited about Bayesian models and NN because they work. Actually, most logistic regression in practice is Bayesian by nature (regularization much?). More so, logistic regression is actually a piece of what powers NNs, and it's incorrect to directly compare the two. To put it more blunt, these are drastic improvements to logistic regression that allow the technique to solve greater problems.
-You are trivializing the amount of training needed to properly apply a classifier by calling it simple.
Second, there are plenty of basic questions in the data field to answer [2]:
-Doing anything with large amounts of data is still complicated and hard
-Unifying multiple data sets
-General model comparison
-Most modeling assumptions and as a result conclusions reached from them are wrong
-Communicating results to people. This is the most underestimated. One of the reasons we have such a backlog of problems that haven't been touched by simple data analysis techniques is the difficulty involved in explaining results. P-values are fucking meaningless and massively misinterpreted and misused. I won't start on frequentist statistics.
1) http://www.quora.com/What-are-the-top-10-data-mining-or-mach...
And the computer scientist who eats Debbie's waffles every Saturday spends his weekends reading about SpaceX instead of pulling her data into R and giving her some insights that would give her a much safer retirement.
I disagree. A downvote is a signal, "fewer comments like this one please". Sometimes it's obvious what that means, but sometimes it's not. In that case it's perfectly reasonable to ask, what was it about my comment that you disliked? A feedback signal is only useful if understood, and sometimes to be understood it needs clarifying.
I Think you over-estimate the capability of "our industry". I don't want to pop your bubble but the people required to make those space machines are not you typical web 2.0 programmer. They are the best of the best and many have half a century of experience. Also they have way more budget. Space is hard.
He quit after a few years and now works at Spotify since it pays way better and is more fun.
I work for ESA space missions (not Rosetta). Although my colleagues are certainly well qualified, I am not sure if we are really the best of the best. (we are not the typical web 2.0 programmers though, this is certainly true :o).
Usually, the software for space missions is written by electrical engineers or physicists/astronomers, who somehow specialized in software after their master or phd. Indeed, the style of that software is quite different from what is usually developed in business setups. No unnecessary complexity, just straight forward paths to the solution. We use more or less exclusively open source tools. No windows too. Works pretty well. Our results are also open source, btw. Just to answer some guys who believe, this is all military, secret and disclosed.
But we also have to fight cowboy-coding and design-pattern-gurus. We usually don't have to fight software managers though. Everybody is so well qualified, that we use to discuss once a week what we do, where we want to go and then just do it.
Worked very well, so far. My last mission was a huge success :o). The real innovation may indeed be done at universities and science institutes, not so much in startups and business. Which startup can afford to hire 200 scientists and invest a billion to do something new? I think there is indeed no competition on that level.
But we also give a lot of tasks to industry (we have to, because of the public funding). In some cases, this caused trouble, because their main focus is to make profit and still be good enough to avoid problems. Also, they usually keep their methods secret for legal reasons. If something does not work in space, it can be hard to get the necessary information to fix the problems (while the time is ticking, because many missions have finite lifetimes).
Our goal is usually 'as good as it gets, outstanding, if possible'. We can afford that and that is really cool. It doesn't work in a business setup, because it usually is expensive.
I've never seen code intended for space missions. I guess they don't share it as most of it can't be reused for other missions so there's really no point in doing it.
They also seemed to run their own summer of code in 2013: http://sophia.estec.esa.int/socis2013/
My experience is that ESA lets the industry (and maybe universities) do most of the SW/HW development and ESA does stuff like requirements and high level management. But I have never followed a project from start to finish, only been in the aforementioned industry :)
I develop data processing software.
You are right, ESA usually oversees the projects and hosts the main data archive. Esa also takes care of the launch vehicle and interfaces with Arianespace for example.
The hard- and software development for the payload is usually done in numerous science institutes, part of them also universities. They usually have subproject-leads, while they again subcontract to external companies, as needed.
A complex structure that has to do mainly with the way how European projects are funded.
What did you do for Esa? What is your view on these projects? We always had the impression, the companies are under a lot of budget pressure during the missions.
A readable article on the group that developed the Space Shuttle's software is "They Write the Right Stuff" by Charles Fishman [1]. It has some interesting insights into the group's development culture, as well as the type of people who do that job (tldr: they're not ninja rockstar js hackers).
Edit: and downvoted because I want Hacker News comments to be about hacking. Maybe it's time to change the name of the site to Rich People Angst and Bad Economics and Politics News.
Engineers that work in defense/nasa/etc generally aren't on twitter, linkedin or facebook broadcasting, well, anything - you just don't hear about them unless you personally know some of these people. It's part of the job description to be private and avoid becoming a target of the media or a foreign nation. It also makes it difficult to stay in touch with colleagues when you're relegated to the rolodex-era of communication.
Quite contrary to your statement, this world might be 1000 times better if the same engineers working on government programs were to work in the commercial sector, pushing the boundaries of tech that is accessible by everyday people.
The point I was trying to make is that I imagine that these brilliant engineers wouldn't be focusing their efforts on iterative improvements to vacuums and tablets.
To be a bit more serious: I do consider the notion that private enterprises always are better to be stupid. For large projects the private bureaucracy are usually as big as the government ones. The private enterprises are with a few exceptions very shortsighted; that is why I joke about another tablet. Show me the company that put serious money into e.g. fusion.
I volunteer for the Long Now Foundation. [1] One of their amazing efforts is the Rosetta Project [2], a set of parallel texts and linguistic resources so that future civilizations will be able to decode anything written today.
The ESA put one of the Rosetta Project disks on the space probe [3], so a bunch of information about our languages is now way the hell out there. You can think of it as very long term backup. [4] I'm really happy for the Long Now folks today.
As an aside, I strongly recommend their series of talks. It's one of my favorite nerd events in SF, and members can stream them live. The old talks are also available via podcast.
[1] http://longnow.org/ [2] http://rosettaproject.org/ [3] http://blog.longnow.org/02013/12/16/wake-up-rosetta/ [4] http://blog.longnow.org/02008/08/20/very-long-term-backup/
Wonderful to think of the disk out there on a comet though, forever circling the sun.
They are definitely looking for ways to bring down the cost. I'll suggest to them that they start a waiting list. In the meantime, I'd suggest you follow the Long Now blog or the Rosetta Project blog. You can be sure that they will mention a 10x drop in cost on both of those.
Unfortunately it seems the previous submission didn't get a lot of love for some reason: https://news.ycombinator.com/item?id=7091027
Come on people, if something is hacker news worthy it has to be this!
I am impressed.
That being said, a hundred other things could have gone wrong and mission control would have had no idea that something happened. Meteorite impacts, misalignment of the solar panels, software bugs, radiation damage and a faulty wake-up sequence are just some things that spring to mind.
Bad thing is that most of these could result in the spacecraft not even being able to send out a ping. Or any information that would tell engineers what went wrong and how to fix it for future missions.
Imagine just listening to silence and not being able to find out if any of it was your mistake.
That sounds a bit like a mod for the game Kerbal Space Program (KSP). There is a planet called Inaccessible which has an equatorial rotation speed greater than the escape velocity. As such to 'land' you need to continuously be thrusting towards the planet (as opposed to away to slow your descent) - or just land at the poles.
But it is certainly custom hardware and a custom OS, because you need radiation-hardened chips (even for earth-satellites) and extreme reliability.
Are there any space exploration vehicles running Linux?
No idea about OS.
This is quite a feat of engineering. How did they compute the trajectories? Most things if you throw them into space will never be seen or heard from again.
Play a little bit of Kerbal Space Program if you want to get a feel of it :).
Also I am not sure that we know all the weights of the celestial bodies with enough precision not the introduce an error while performing a 'slingshot'. Rosetta has propulsion and my guess it that it can be used to actively correct small mistakes in positioning.
We've been doing gravity assists for some time now, so I guess this topic is mostly figured out, though probably some active corrections are required. Again, maybe the ESA guys will chime in. Or InclinedPlane ;).
I imagine finding the best trajectory in the first place to be the more difficult and interesting problem.
(Not an ESA guy, just wanna-be-condensed matter physicist.)
To give you an idea of the magnitudes involved, consider that GPS satellites do need to be aware of relativity. GPS depends on extremely accurate clocks, to the extent that relativistic time dilation due to the altitude of the satellites becomes important.
However, the necessary correction for relativity is only 38 microseconds per day. That's about 0.00000004%. It matters for GPS, but that's a special case.
For another example: the orbital precession of Mercury was one of the first indications that there might be something beyond Newtonian gravity. Newtonian gravity would have Mercury following a steady ellipse, with small changes due to the gravitational influence of other bodies and the fact that the Sun is not a perfect sphere. However, Mercury's orbit precesses (which is to say, the ellipse itself rotates slowly) more than could be explained by influences from the Sun and known planets. This was initially thought to be due to an unseen planet called Vulcan, but none could be found. Eventually it turned out that general relativity explained the discrepancy perfectly.
The total precession of Mercury's orbit is about 574 arc seconds per century. The amount of that due to relativity is about 43 arc seconds per century. That's about 0.12 arc seconds per year, and it means that if you used pure Newtonian mechanics to predict Mercury's position one year from now, relativity would cause your prediction to be off by (very) roughly 17km. Not much, and trivially corrected for using the spacecraft's thrusters while on course. That's equivalent to a velocity error of 0.5 millimeters per second on the spacecraft's part, and I don't think rockets are that accurate to begin with.
I guess what I'm saying is I'm a bit jealous of the team effort and environment.
http://www.esa.int/esatv/Videos/2013/12/Rosetta_s_Journey_B-...