I thought you were being facetious about what could have been a throwaway comment from a throwaway account, but then I checked their bio; this particular type of interaction is one of the things that I love about HN.
1,744 karma · joined January 28, 2020
And yet, here they are functionally subsidizing their CPU business by handicapping the usefulness of their GPU line. That handicap will probably be for at least two generations, since it's almost certain that consumers will now be skeptical of their (already rough) firmware being able to receive updates not only this time around but whenever their next line comes out as well, even if they claim to fix the issue.
Edit: Ars has retracted this story; Intel claims, and Ars has confirmed on Windows at least, that Arc GPU firmware should be upgradable using both AMD and Intel CPUs. Maybe they actually have learned!
I've actually got my commercial license test scheduled for the end of the month! I'm hoping that it might help me get permission to run surveys in areas like national forests or parks that are otherwise pretty much off-limits to recreational flyers.
At this point, I can lay out my waypoint missions ahead of time, grab my drone bag and some coffee, walk out the site, and get the vehicle up within a few minutes. I have three batteries, so that usually translates to somewhere around an hour of flight time with reserves, and over 100 images. When I'm done, I can pull out my computer, upload the day's images to the locally-run Docker container instance of NodeODM, wait for the results to get spit out a bit later, then run my post-process script, scp them up to the VPS hosting my site, and there's the map! The automation has made the process a lot more enjoyable. I'm still hoping to tinker with things making the images separate layers so I can do a time-series of orthophotos made in the same location, but it's still been fun in the interim. Feel free to reach out if you'd like to chat--I don't see many other people that are also interested in the area!
It feels like a disservice both to this post and to Paul Graham to exclude the date for this article. I'd imagine the timing on a post like this is critical context for understanding it, and to both exclude it in the submission (and, frankly the fact that it's only at the bottom of the article, not the top) is less than optimal.
Rust has a weird mix at the moment, and not one that's likely to significantly change within the next 12 months, at least. Certain tools are genuinely best-in-class, especially around simple operations on insane amounts of data. Rust kills it in that space due to its native speed and focus on concurrency.
There's also growing projects like Linfa [1]. that while not at the level of scikit-learn, have significantly increased their coverage on common data science/classical ML problems in the past couple years, along with improved tooling. The space does have a few pure-Rust projects coming down the pipeline around autodifferentiation, GPU compute, etc. that are likely to yield some really valuable results in deep learning, but that aren't quite available and will take some time to pick up some traction even once they're released. At the same time, areas like data visualization are unlikely to reach parity with something like matplotlib/pyplot in the near future.
Python is the de-facto standard, and will be for some time, but Rust's ability to build accessible high-level APIs on top of performant, language-native libraries is attracting some attention and I wouldn't be surprised to start seeing ingress in the certain areas over the next few years, where instead of the Python/C++ combination, it's just Rust all the way down.
It's probably easier to do in the United States (a lot more land than in England, and with fewer entrenched ownership considerations), but I'd really be interested to see if similar projects would work in other parts of the country that are more densely populated, or in areas like the California central basin to where public water resources are being abused to farm a desert.
> data is collected both by Amazon and third-parties and that Amazon shares user data with as many as 41 ad partners.
> 70 percent of Skills fail to mention Alexa or Amazon, and a mere 2.2 percent make their data collection practices clear in their privacy policies
> "Amazon’s inference of advertising interests from users’ voice is a clear violation of their policies and public statements," the researchers state in their paper. "Amazon does not provide transparency in usage of data and thus cannot be reliably trusted to protect user privacy."
A better tl;dr would be something like "Amazon violates it's stated policies on using voice assistant data to target ads, makes a ton of money of the practice, obfuscates the whole thing by not making its collection policies clear, and then calls research into it inaccurate without providing any evidence."
It's not the prettiest, and writing the initial Javascript code is that I'm using as the template for adding data is kind of a pain, so possibly being able to write the entire application in native Rust would be really awesome.
[1] https://community.amethyst.rs/t/bevy-engine-addressing-the-e...
Yes, I think that this is the crux of the matter here, and I agree that merely funding direct, vanilla replication is not the optimal solution here and would not solve many of the problems we're facing. I'm really talking about the idea from the thread I linked in my original comment--the idea of "red teams" for science. There's probably a number of ways to accomplish this, and direct replication could play a role, but I imagine that a funding agency tasked with funding red teams for other government-funded research would lean towards a more diverse and holistic toolkit. You'd certainly need to have some mechanisms for "watching the watchers," but again, very few researchers I've met are stupid, many of them actually care pretty deeply about ethics and scientific integrity, and would probably jump at the chance for a steady paycheck while helping to advance that.
> Science has no formal mechanisms to detect made up or fraudulent data sets, which is a real problem [...]
This isn't strictly true (I'd like encourage people not to use "Science" as a proper noun; I think it's actually a bit unhelpful). I think a common approach to performing research using the scientific method often (almost always?) involves using observation across multiple instances, then using statistics to draw a generalized conclusions based on the group of observations. And statistics does have formal methods for identifying potentially fraudulent datasets, some of which a Dr. Hilgard does a great job of explaining in an actual case of research fraud he found in this article[1].
[1] https://crystalprisonzone.blogspot.com/2021/01/i-tried-to-re...
Even ignoring how, frankly, paternalistic and condescending this sentence comes off as, let's take the rest of your comment on face value.
First, I can agree with you that the parent comment's idea about not allowing publication without replication is a non-starter; it would basically be impossible to implement (in the U.S., you'd immediately run into 1st Amendment issues), and on a practical level, that doesn't match with almost any realistic scenario of how research both is or could be conducted. Anyone that can get published should be welcome to do so.
However, I think that the idea of funding a subset of replication studies would have great value, and your list of "issues" are largely knocking down strawmen. Let's look at this from the point of view of a funding agency, which annually pour billions of dollars into research.
1. Logic errors would be great! A replication report stating "the conclusions of this research are deeply invalid due to x,y, and z logic errors here, here, and here" would make the agency less likely to fund authors that make those errors. If the errors don't invalidate the conclusions, they're still worth noting and can be taken into account.
2. Effect sizes varying in replicated studies is still useful information. Not all "does/doesn't replicate" questions need a binary yes/no answer.
3. Similarly to #1, catching and publishing instances where authors are making invalid claims (even if not doing invalid research) seems like a good thing. "Hey, these researchers have a tendency to claim X when really they're talking about Y. What the heck?" is great information to know for a funding agency to know.
4. See #1 and #3. Generally, the authors of the replication studies are also going to be intelligent; if they're being paid to check for logical fallacies and incoherent concepts (one can imagine a first-year required grad course that lays out these ideas and case studies of how to catch them).
5. I've been thinking about a situations where this still wouldn't produce useful meta information, and I'm having a hard time coming up with them. Imagine a fish survey, where divers are doing fish counts along transects. You can't go re-observe those fish, but a) research groups might want to change their methods to include using cameras instead of marking dive slates, or b) if you are able to re-run their transects due to good record-keeping (yay, replication!) but don't see the same fish, the funding agency now knows that a) the original research group has solid experimental methods and b) maybe they should keep funding groups to run that transect because of the variation, or not fund groups using a single transect data as the basis for forming conclusions. Happy to consider non-outlier (i.e. building a second LHC) examples where this breaks down.
6. Yes, maybe don't fund groups to try replicating these findings. Or maybe do occasionally anyway--if the research groups are doing good work, it should be cheaper to conduct replication-based research than doing a novel one anyway.
7. This is a good argument for replicable research. A huge number of research proposals go unfunded, which is to say that many researchers are somewhat competing for funding. In this paradigm, saying "hey, my group found and thoroughly documented a huge, glaring issue in the methods used in these studies due to a bad-faith corruption of historical data," in a situation where agencies would be looking to pay groups for valid claims. Increasing the number of replication studies adds a greater adversarial capacity to a research system which helps to catch/prevent this kind of fraud in the first place.
This is already a long comment, but I can make similar points about almost all of your claims. In summary: you seem to be making the assumption that replication studies are basically done by robots blindly following an instruction manual of someone else's research. Especially given the funding incentives mentioned above, I don't think that would be true; rather, many errors would be caught (or prevented ahead of time) and lead to better, more solid research being done in the first place.
I'd also argue that your reduction of this problem sort of misses the point. One of the big problems with the way that studies are done is not that replication efforts aren't conclusive (it's very difficult to prove something doesn't exist), it's that a) non-replicable studies are generally considered as valuable as replicable ones, and b) as a result, it's extremely difficult to replicate many studies to begin with, because there's no incentive to take the time to make it possible. Even if the end result of a replication paper is "we couldn't produce the same results", the people working on it can say "this author's experiments were exceedingly difficult to even try to reproduce," or conversely "we didn't get the same results, but their data collection methods and analysis code were well-documented and accessible." That has a lot of value!
If you tried doing triple replication for every paper, I agree that maybe wouldn't be the best use of resources. But the current state of affairs is so bad that a well-organized drive to create single-attempt replication on a fraction of publicly-funded projects has the potential to be a significant driver of change.
Obviously, there's a few fields where this might not work (you can't just create a second Large Hadron Collider for validation), but in areas from sociology to organic chemistry to environmental science, I think there's a lot of promise in that method for helping to re-align incentives around producing solid, replicable research.
[1] https://www.worksinprogress.co/issue/escaping-sciences-parad...
But if you were going to apply data science/machine learning to healthcare (personally, I dislike the term "AI"), I think that this project would be very close to both how and where you would want it. Peer-reviewed research with independent confirmation, in an narrow, image-based diagnosis, overseen by medical professionals throughout the process (not directly making recommendations to patients), by one of the world's leading medical research institutions.
This isn't a catch-all, and it's certainly possible that cynicism could be warranted down the line. But for me, personally, I think that cautious optimism for this specific situation overrides the cynicism.
> Current clinical cardiac image analysis extracts only simple scar features like volume and mass, severely underutilizing what's demonstrated in this work to be critical data.
> "The images carry critical information that doctors haven't been able to access," said first author Dan Popescu, a former Johns Hopkins doctoral student.
This is, like, the thing that machine learning is really good at--pulling out additional information from complex patterns represented as imagery. It's a very similar reasoning to why neural networks can be really great at classifying malware based on the program structure or doing semantic segmentation on well-defined scenes.
> The team trained a second neural network to learn from 10 years of standard clinical patient data, 22 factors such as patients' age, weight, race, and prescription drug use.
Again, this kind of multi-dimensional pattern-matching is pretty much much a machine learning wheelhouse.
> [They] were validated in tests with an independent patient cohort from 60 health centers across the United States, with different cardiac histories and different imaging data
And apparently has a bunch of independent validation. Neat stuff, I'm curious to see how it will develop/be distributed in the future.
Disclosure: I recently accepted a job offer from JHU, albeit in a different field.