Algorithm detected over a hundred asteroids after studying old telescope data
natureworldnews.com
natureworldnews.com
From the paper, previously you needed a sequence of images from a single night to generate a "tracklet," which lets you estimate a guess of the orbit of the asteroid. The guess can be compared with images from other nights, to confirm the existence and orbit of the asteroids. Without the tracklet, the asteroid could be heading in any direction, meaning that the set of possible orbits is huge, and there would be many many possible matches to other images.
The key insight of THOR (their algorithm) is that even a mediocre guess of the orbit makes the true orbit stand out. From an observed object, they predict a test orbit. Then they take all other images that could be along that orbit, project all objects in each image to the original image according the orbit, and then stack the projected images together. If their guess of the orbit was perfect, there should be a clear dot in the stacked projections, where that single asteroid was seen in all the images. But even an imperfect guess is still useful. Any object that's somewhat close to the guessed orbit will show up as a line in the stacked image. So find the lines, use some math to refine the estimate of that objects orbit, and now you've got the object.
(I'm not in this field, so take all explanations with a grain of salt please).
Sounds like a great way to search for asteroids, never would have thought of it.
It's essentially just a coordinate transform that lets you add up evidence for a particular object, and the objects stand out as little bright points.
I'm sure they're doing lots of clever things on top of it of course though.
NASA has had a dedicated search for many years: https://cneos.jpl.nasa.gov/
Just detecting these objects in past images isn’t helpful if the orbit can’t be constrained enough to continue tracking it!
> The THOR team has shown the potential to map out the trajectories of numerous asteroids in our neighborhood, but there are few key caveats. Since their images date from a few years ago, asteroids that haven’t been reobserved lately have effectively been lost at this point, although they could be picked up again in newer images down the road, says Paul Chodas, director of the Center for Near-Earth Object Studies at NASA’s Jet Propulsion Laboratory in Pasadena, California.
Often I find lately the barrier to a project is sociological, not technical.
Trick is now, how to put together these possible groups of 4 for detection. Because they say they have 68 billion observations, certainly combining each to each 4 times is not possible. There must be some heuristic and/or trick to it.
Maybe define heuristics for what ISN'T useful first to pare down the data set (looking for negatives), then focus on the sort where you're actually looking for positives?
When I'm sorting photos for art purposes, I shoot a lot, years at a time, then spend a month or two sorting into albums, I actualy lost hours of work because I had to restore my phone from a backup recently.
But the general idea, when doing a completely manual process, is you grab say, 1kish of 20k images, then from there further split them into 3-4 albums.
Now if they were astronomy photos then you could go into that set of about 500 and manually look, think what's a heuristic to get me from where I started to here more quickly.
(Sorry if articulating myself poorly, this site spikes my anxiety -- if I give useful information people just siphon it up, but if you give an opinion they don't like, folks get real ornery.)
- Detect any protracted images on the photos which shouldn't be protracted
- Interpret ends of them as 2 observations and dots at 1/3 and 2/3 as 2 others (exploiting the fact that speed of angular motion at such a short interval is essentially constant).
- Run a (very crude) orbit detection using these
- Look for any suspicious dots on other images within the margin of error provided by this orbit (that is, where the object with that orbit should've been if it was real, at a time each other shot was made).
- If found, use these for orbit refinement, rinse and repeat until on all shots where the object should've been apparent, it is present, or reject it as a false detection if that doesn't happen.
Many (maybe most) researchers are terrible programmers, it's a non-issue. If you want to learn how to do this kind of work you have to learn the science.
For example I know different types of stars and how their compositions differ, but not detailed mathematical models. If by "the science" you mean the latter, great, I can drop the idea, if the former I might be able to add value.
(For example, I've read a lot about black holes, Hawking radiation etc, but not in a detailed mathematical sense, just more general ideas like the concept of an event horizon.)
(That's why I dropped out, worked briefly in policy, learned to hack, then sold stock out of my retirement fund to get an open source intelligence certification from Bellingcat -- because THAT MANY sets of people refused to follow basic social norms.)
Research is not a spectator sport, it's highly specialized and competitive. That's not to say you can't contribute, if you want to participate the prerequisite is being willing to put a significant amount of time and effort into learning the skills of research, and learning about the specific domain you are interested in. You'll need to go from the basic theory to more specialized topics, read review papers, understand where the state of the art is, etc... As you read more and more you'll get insights on what's been done and what hasn't, you might find a niche that you can contribute to. In some fields the obstacle is gathering data and doing field work, you could make a contribution there as well. You could also contribute to open-source scientific software, I'd imagine it's be less competitive than doing research, you'd still need domain knowledge though.
Astronomy is currently a highly accessible field. Amateurs are still making meaningful research contributions. There is also a lot of grunt work available using other people's methods.
Edit to add: You don't even need to gather your own data. There are many, many datasets lying around waiting to be processed by new techniques.
A lot of the papers I read are implemented by adding a single line of code to a model that someone with an undergrad knowledge of probabilities could have written. Except they couldn't because to get that line of code you need a deep knowledge of the prior work, good fundamental knowledge of the discipline to even come up with such an idea, knowledge of the datasets that are in use, the problems that are interesting to work on and those that aren't, etc ...
> No one here could have divinated that algorithm without doing the legwork.
No one here could understand how to combine rudimentary image processing in such a way as to track a moving object over time? It's impressive, certainly, but there's more data processing here than astrophysics.
Most people with a BA in science or math would have enough requisite knowledge to pick up what they would need for this paper over the course of a weekend. The part they couldn't do that with would be the image processing and data science. If they already have that? Yes, this is absolutely achievable. Just copying this work on a different data set? Also very achievable. The knowledge of which datasets are in use? In this field those are both mostly public and easy to find.
Yes, those who don't have some of this knowledge will have to put in more time than others, but the implication you make about the sheer amount of time needed and what would actually be required just to get involved just doesn't apply here. There was a time half a decade ago where someone like the original poster could literally get a job doing this stuff. That may still even be the case.
You make it sound easy, I don't know why those 10 authors even wasted their time publishing this. It reminds me of a joke about getting grants, you show someone your proposal and they'll tell you it's a bad idea and will never work, you show them your work after a few years and they'll tell you the results are obvious and everyone already knew.
I didn't tell him it's impossible, I said it requires a lot of work beyond scripting and specifically listed some of the aspects of the work that are easier to get into.
Of course they could get a job doing it, that would involve spending 8 hours a day working on the topic and doing all the things I described. You word it like you're disagreeing with me but go on to list skill after skill that's required to actually achieve this. A bachelor of science alone is three years of studies, data science and image processing are very specialized scientific skill-sets that take a long time to acquire. I've seen plenty of people take a 6-months bootcamp who are barely able to manipulate a matrix at the end. Knowing which datasets and problems to work on requires a scientific culture which you acquire by reading the literature, not just googling. Combining all these skills into an idea that results into a quality publishable scientific output is a job and it requires as much work as any job does.
By all means, let people do the work and contribute if it's interesting to them, but everyone here feels the same way when we see a paper come through here in which they "just" combine some Hough transforms and some observations "oh I could have done that." It doesn't work that way.
As with so much development there is a massive gap between 'I can do that now, someone showed it worked' and 'I could have come up with that before anyone showed it worked'.
Leading edge science becomes demonstrations that undergrads do after a time. Plenty of fields seem to progress when someone realises a simple idea can be applied in new way.
6-months to learn matrix arithmetic algos sounds like that person just wasn't cut out for that task.
Best option here seems like encourage the person to try applying their domain knowledge from CV to the field at hand and ask them to report back how they get on. If they've applied Hough transforms before, perhaps, they might be able to refine the algos here, for example.
tl;dr I think you're being a bit precious about it all.
I've read hundreds of papers on security and privacy. People make flippant comments like "You can't work on my team unless you have a PhD" followed shortly out of band by comments like "I'm the head of the Montreal office".
I currently find it difficult to study those topics anymore, since there seems to be some fundemental disconnect between my actions and results.
(And then, in parallel, thanks to deleting Facebook, I know a lot more about celebrity gossip and geopolitcs than I used to, so sometimes the old self soothing method of watching a movie isn't as fun, depending on who was involved.)
If you have suggestions taking into account the above, feel free to reply here or 1:1 to the email listed on my profile.
You start with a literature review and textbook search. read 5-10 different undergraduate astronomy and physics text books, then undertake a current literature review. Do searches in science journals that you might get free from your local library, or the library associated with your alma mater if you have connections there still.
That gets you the basic knowledge needed.
Next is the hard part, using current state of knowledge, finding current data sets, finding your own algorithms for "scratching your own itch"
Writing the script/programming is the last step - the least skilled step.
Imagine you are John Carmack writing Doom for the first time. He had to know the math, look for "at the time state of the art" university dissertations regarding 3d rendering and occlusion, etc and combine that with his own knowledge about coding, assembly language, and put all that together to make Doom run on the hardware he had access to at the time.
It's exactly the same for any field. Be humble, learn the basics, get caught up on current research, read deeply, read other people's disserations and ideas, then using what you have, and combine it with other people's ideas, and then you start the programming part (or you can hire people to code it for you, nothing wrong with that). and that's how it works.
We all stand on the shoulders of giants who come before us.
I would have done that, but my future advisor repeatedly broke their promises about basic things like whether a thesis by publication (several papers vs one on a common theme) was an acceptable path, then when I tried to pivot to industry folks made emotional decisions any time I interviewed for a role that was not a precarious smart money position.
>Be humble, learn the basics
I've made career decisions for over a decade because anytime I encounter complex math, the person instructing me has an attitude like yours, paired with a complete inability to teach.
(For example, I took a course that was on statistics specifically for use in psychology... we did many, many exercises with specific software and techniques, then did an exam using... none of them. That kind of rapid shift, paired with an attitude of "if you can't learn from my schitzoid teaching style, you don't belong here" drove me into a deep depression, since everyone I knew who had about the same level of statistics and programming knowledge as me had years of teaching in K-12, not a smarthering of summer camps and self learning in between a school year where if you used the wrong tone, they'd draw blood on you.
>We all stand on the shoulders of giants who come before us.
Despite the harsh tone of the two comments prior to this, I agree, but be very mindful if you say things like "learn the basics" you can connect someone who is willing with a teacher who will do so, or be prepared for a string of angry internet comments on nights when someone doesn't feel like quietly watching pirated movies another night in a row to self soothe their economic precarity.
Aim to understand things for your own pleasure rather than being useful.
I know a lot of things for pleasure, but it always felt like folks would zero in on whatever I didn't, to the point it destroyed my interest in a few things that didn't require calc.
https://b612foundation.org/open-positions/ https://www.lsstcorporation.org/lincc/job_opportunities
(It's a common issue for us security folks: we're good at writing out pretty code that automates things on the command line with bash and a few esoteric CLI tools, but a full stack developer, I am not.)
Veritasium video with Dave Jewitt of UCLA explains a lot about what were doing about asteroids.