Hacker News Reader

Comment by binarystargazer | Hacker News Reader

I’m the person in the chain that takes the scientific data and maps it into these color images used in the viewer (among other responsibilities). I spent a long time researching and developing the software that as close as possible maps the data into human perception. It means so much to see people out there appreciating our work, and following what Rubin will achieve. So that you for taking your time to check it out!

View on HN

Hey! Former astronomer here; I used to work a lot with SDSS data products. Do you happen to have a link to any sort of photometry DB/catalog for Rubin? Basically I’m looking to grab a ton of data but I’ve been out of the field for a while and I’m not sure where they publish their data sets and which ones you’d recommend as being relatively accessible.

You want to check out the early science release info at https://rubinobservatory.org/for-scientists/resources/early-... . This is all early science from preliminary data. It is mostly to help people ramp up developing their software against the type of things they will be seeing in the future. There some known issues and gotchas outlined in the release. Hopefully this will be enough to whet your appetite. The data seen in this sky viewer is part of the Data Preview 2.

I hope the data releases are a bit better than what I've seen from recent telescopes.

The common pattern seems to be that the majority of data is locked away behind multi-year embargo periods, or never released to the public at all with no explanation why, with only the smallest little crumbs of raw data actually released to the public, and usually years out of date. Also you have to go through some kafkaesque chain of technologies/servers/bureaucracy to get access to that little bit of data.

It's a little depressing to me when the public funds these telescopes and then get snubbed when it comes to getting access to the data they produce.

We are getting pretty far outside my area of expertise / knowledge on this topic here, but I know I and others agree with you here. There is a Rubin Observatory data access policy out there. Don't quote me on the exact specifics but I believe that most colleges and research institutions already have a mechanism to enable data access setup, or can get one. This is mostly an id auth type thing to make sure those accessing have data rights. There is a policy where people not associated with those can request access too, but I am not sure the procedure and would not be qualified to weigh in on anything more specific.

Curious what were your thoughts about this story when it broke: https://www.theatlantic.com/science/archive/2024/12/vera-rub...

And whether it impacts your work? For example, if the unnamed agencies' efforts leave a telltale mark in areas they change, that would still be problematic, so presumably they have to place noise corresponding to adjacent pixels. Are you concerned about how data integrity affects your work output? Penny to understand the internal scuttlebut / any internal grousings about this.

Jaw-dropping stuff, looking forward to pointing 9-year old daughter (in "Future Astrophysicist" t-shirt) at it!

Possibly a naive question, after surfing the image for a while I didn't find any einstein rings (or arcs) - are they just rare/wrong scale or something else?

and thank you!

They are both fairly rare, and or like you said wrong scale (for this data release at least). There is actually one in this field, but it is below the noise floor of this image, it is very small and very faint. There will be others in the future that will be easier to see. By the end of our survey we will have about 40,000 sq deg (a bit over half the night sky) looking like what you see here.

Be sure you and your daughter click the search icon, there are a few other fields of stuff you can check out. The links in the search will take you there without needing to scroll around the big black sphere.

I hope your daughter enjoys it, and we have a few activities at https://rubinobservatory.org/education/first-look-lasting-im... including a coloring book.

Thank you for the amazing work! Do you have any publications or articles describing your findings and the process you developed for mapping the scientific data into perceptually meaningful colour images?

Rendering our observatory's images in true HDR is something I've been wanting to implement for some time (especially as HDR support on the web continues to improve), as there is so much more in the data than we typically get to see.

I have a publication that will be done soon™. I have been preparing it for a while, but I keep finding edge cases to fix, and ways to make things even better that I keep changing things fast enough that I need to re-work the paper. The code to produce it will all be open source, and I am even working on making it an independent package you can install separately from the rest of our software stack, specifically so other places can easily install and use it in all sorts of environments.

It's funny you mention HDR, I am _just_ about finished with the HDR extension to the image processing code. What you see in the sky viewer is all standard SDR formats. As you suspect it makes a HUGE difference. Once this is all done we will be working on rolling out many different ways to experience it. The HDR code path will also be part of the package and is not much different to use than the standard stuff.

Reading your comment has gotten me a little more energized to make more progress on this this morning.

Reply on news.ycombinator.com