jablongo

I took LeCun’s deep learning class at the Courant institute in 2016 and some of this stuff is directly lifted from those course materials, like the gif of contrastive learning in action.

Most people on loop have omnipod and dexcom g6 also so I don’t think it’s that limited. It’s true that it takes a few months for new devices to get integrated but open source APS are the only systems lets you choose what devices you want to use with them.

Really appreciate the pointed commentary on this! Happy to make further prognostications about the success of CGM-input-only APSs via email.

For the record, when I say "Euglycemic Parity" what I really mean is a sort of Turing test (not time in range), where a data-driven Endocrinologist is asked to tell the difference between CGM records from a non-diabetic, and CGM records from a diabetic equipped with some control system. Passing this test should be our long term goal IMO and we will probably have to bring many techniques to bear to eventually achieve it.

The best brand is Dexcom (G6 or G7). The Abbott one is cheaper but has disadvantages. Either one will be way better than nothing if you have diabetes. Without insurance dexcom will cost you like $260 a month via one of these mail order distributors: https://rapidrxusa.com/products/dexcom-g6-sensors-3-pack

Do your part as a consumer and shop around a bit to drive prices down... good luck!

I'd love to chat - send me an email at sam@replica.health . An accurate time-varying insulin sensitivity model is one of our big projects.

At this point all the major solutions are using an algorithm that would probably fall under the umbrella of "Model Predictive Control" rather than a vanilla PID controller. An absurd spate of patent trolling occurred back in the early 2000s related to Controller definitions like these though. Researchers patented the use of a "PID controller in artificial pancreas systems" [1][2] which slowed down the development of APSs by many years.

The way the Supreme Court recently changed patent law [3] for software has definitely had a positive effect for APS development. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769814/ [2] https://patents.google.com/patent/US20150306314 [3] https://en.wikipedia.org/wiki/Alice_Corp._v._CLS_Bank_Intern...

Agree that patient input shouldn't be necessary, but to replace it we will need to include other inputs besides CGM in a systematic way to get the optimal results. My company is working on how to use contextual info automatically collected by your devices to help (detected activity, measured calorie burn, geofencing, data from meal-ordering apis, etc.). This is especially true given that the CGM data themselves are lagged due to averaging and/or kalman filtering going on under the hood. This is a fundamental problem; Inreda uses two identical CGMs for noise reduction purposes just so they can get clean data with less of a lag.

None of the systems claiming you don't have to do anything in terms of meal announcement are _working_ in the sense of achieving euglycemic parity, which should be the goal. I can say with certainty that the cgm logs from people who don't announce meals on the Inreda device do not look like they are from non-diabetics: there are still often large post-prandial spikes. Inreda likely does better than any single hormone system, but the problem is not solved in any sense.

Yeah she's being diplomatic for sure. Dexcom plays ball and is also the most successful of the CGMs, so you would think that others would follow their lead and write their data to apple health. Most companies keep their data thinking its going lead to some unspecified profits down the line but then never really do anything with it.

> - it cannot know external factors like how much carbohydrates you are eating, or if you are planning to work out, etc. Like conventional pumps you have to enter it manually.

Sign up for the waitlist of https://replica.health! It is designed to make capturing those external factors as painless as possible. Depending on what setup you have, it will be available in the next month.

Yea you would need really good failure detection if you were going to "hit the gas" with a bunch of insulin. Part of the solution is going to be controlling risk via the dosing algorithm itself, so you never get in those situations where you are at risk of severe hypo in the event of a (glucagon) site failure.

For some background - I'm a T1D working on a search engine and conversational interface for integrating a bunch of new data sources and models into metabolic decision making: https://replica.health. I've also been a user of and worked on various open source artificial pancreas systems through the years, and am currently on Loop.

>I would have thought by now they'd have hacked 2 patch-pump AIDs to work simultaneously.

As you pointed out, the problem is not really hardware. It could technically be done in a straightforward way using two independently controlled insulin pumps, but the complexity and risk of the whole operation goes way up if you are taking way more insulin. Taking a bunch of insulin and glucagon at the same time is not necessarily a great idea either - they don't just annihilate each other without consequence and you could end up with secondary effects like gaining a bunch of weight.

>Because this feels like a holy grail / functional cure

Unfortunately it is not; even dual hormone systems have problems keeping up w/ the kinetics of glucose absorption and to address this there is also research into tri-hormonal systems, w/ amylin as the third hormone. In any case you will still need some a-priori info about meals and planned activities, though less so than with a single hormone system. Integration of exogenous data sources to provide this info to the APS is what we are working on at Replica.

Also, hate to be the bearer of bad news but beta bionics has shelved their dual hormone ambitions for now; their prototype device soon to be released is insulin-only. On the bright side there is a small Dutch company whose tech predates beta-bionics. They sell a dual hormone device and will give it to you for a ton of $ (and probably have you sign a bunch of waivers): https://www.inredadiabetic.nl/en/discover-the-ap/

I think it’s a pretty basic/uninteresting relation; for a subset of users, being inflamed causes them to be uncomfortable which causes them to post about being uncomfortable, probably sometimes talking about inflammation itself (for a super meta example, see posts in this thread).

Yea thats what I was thinking. Is "sunflower seed math" a glitched concept embedding that can break it? I think the "embedding vector near zero" would account for when the model can't recognize glitch tokens, but not when it starts talking about ultron or whatever in the lesswrong glitch token research.

So this seems similar to the effect of 'glitch tokens', but in this case its probably more like a 'glitch embedding'; some place in it's latent space of embeddings that the model arrived in by chance due to their conversation that is not where its supposed to be and/or very out of sample. Maybe sunflower seed oil math represents a "glitch concept embedding"...

The reason it might need to be huge is because the long tail of extreme weights might only begin to show up then, but yes best to just start w something you can run on a laptop.

It has to do with the precision of the values stored in those rows and columns. If they could be coerced into a narrower range (without losing information) then we could effectively store them each with 8 bits or something. The +1 prevents blowups when the denominator in its current form approaches 0, and without those blowups, then we can use less bits, in theory.

Yea - the way we can "tell if this is good" is by

a) train two identical models on a large dataset, one with the +1 in the denominator for the softmax steps of the attention modules, one without

b) show that they have similar performance (doubt the +1 will make performance better, but we need to show it doesn't make things worse)

c) show that there are less "blowups" in the model with +1, and therefore they are more effectively quantized.

Yea the benefit is not going to come in terms of performance for a given model, but in terms of ability to be efficiently quantized.

Love it, but I’m pretty sure you’re missing ferry routes in the NYC calculations (which are public). It also might be worth including Citibike as an optional additional method.

Cool - check out my company: https://replica.health - We automatically log a bunch of data for diabetics based on things like activity and location, and provide an LLM powered search engine for that data. You can ask our system questions w/ natural language like "show me data about whenever I get low post-exercise". If theres any crossover, I'd love to chat about it.

Why use dextrose instead of glucagon for lows? B/C patient liver function may be compromised more often in the in patient setting?

This seems great, but what does this mean for fine-tuning? Will we be able to fine-tune models prompted with function calls? Should we fine-tune models prompted with function calls? Depending on the complexity of the function-call/text-query pairs I think we may still want to...

The tech is amazing, but also ridiculous. Think about how much time and cost went into developing these features: - Creepy Eyes fake transparency (a work around for the fact that you look dumb wearing ski goggles on your face) - The realistic 3d facetime avatar (a work around for the fact that you look dumb wearing ski goggles on your face, creepy eyes isn't a good enough work around for facetime, and you need an external camera to do facetime).

None of the use cases seemed to compellingly improve productivity or well being, they just close the gap between the digital and physical w/ no benefit. I'd have been more convinced if it was depicted helping a mechanic assemble an engine with AI annotations and advice, but instead almost all of the demos seemed dystopian.

The eye tracking technology involves shining infrared light at your eyes at all times - not an expert but I can't help but wonder if that is going to have long term effects. https://ehs.lbl.gov/resource/documents/radiation-protection/.... It is ironic that Apple Health is launching an eye-health initiative which reminds kids to keep their device far enough from their face to prevent eye damage, and then is releasing another device that is covering your entire field of vision at like 2 centimeters away from your eyeballs (Edit: this last part is a non issue because the focal point is not so close due to the lenses).

No signatories from Meta

Yes! I’ve noticed this too, it’s just slightly less sharp. It probably has to do with how much trouble they are having servicing all of the demand, so they have rolled out a scaled down version that requires less compute.

Yes this sounds great but the issue is about how longtermists revel in the opportunity to sacrifice near term goals for long term ones, in an almost pathological way. I still don't understand why there should be any conflict between aggressive environmentalism and longtermism, but it keeps coming up over and over again. All I can think of is that longtermists want to demonstrate how truly long-term they are and therefore act like one of the most pressing current issues is secondary to a bunch of other stuff. In general and in the absence of more info I think it's safe to go for the proximal goal of stabilizing the world's ecosystems, even though it _could_ in theory be a non-existential risk. All of that said, the conflict between longtermism and environmentalism might mostly be manufactured by the critics of longtermism, like the author of this article.

Right so it should be much easier w/ access to every neuron and activation. But the general approach is an experimental one where you try to use your existing knowledge about physics and biology to discern what is activating different structures (and neurons) in the brain. I agree w/ the approach of trying to assign some functionality to individual 'neurons', but I don't think that using GPT4 to do so is the most appealing way to go about that, considering GPT4 is the structure we are interested in decoding in the first place.

This isnt exactly building an understanding of LLMs from first principles... IMO we should broadly be following the (imperfect) example set forth by neuroscientists attempting to explain fMRI scans and assigning functionality to various subregions in the brain. It is circular and "unsafe" from an alignment perspective to use a complex model to understand the internals of a simpler model; in order to understand GPT4 then we need GPT5? These approaches are interesting, but we should primarily focus on building our understanding of these models from building blocks that we already understand.

While he certainly points out things that have been true about AI hyperbole driven by profit seeking, its ironic how he himself is hyperbolic about the ineptitude of modern AIs while hawking his pdf e-book on the subject for $35.

Vermont is mostly green via hydroelectric and wind