ineedasername

It also shows the same reasoning error mode many criticisms of a precautionary initiative or intervention to a problem:

Assuming that a problem whose trajectory was at a certain place when the initiative began has failed simply because it isn't solved on their own wished for timeline or standard of success, or that it wasn't meaningfully changed from what itnwod otherwise have been.

What happened to realizing there are hard problems, that different things may need to be tried, or that those things tried were partial but not complete solutions?

Perhaps that shouldn't be seen as failure when delivering even what they have done in a short period of time has required AI capable of causing these problems? Imagine if they were even more effective with more consistent results?

"They've failed" in the context of interpreting their actions w. calls for regulation as a purely cynical result of models increased capabilities has a little too much contradiction in it to my thinking. I won't venture a guess on where purely retreats back and some legitimate concern on their part fills in the gap but it's difficult not to see some. And in Anthropic's case it's also a little more consistent with what they've always said- as well as how they've acted at times, which is how they've ended up on the US blacklist of suppliers.

To be clear, I did not create the course itself. I had discretion on the specific material and approach to the course so long as it fit appropriate catalogue requirements: a multidisciplinary offering on the use of language to manipulate & persuade.

My own implementation had 3 pillars, with a central, loose thesis: Fundamentally, propaganda consists of poor arguments made in some way incomplete, incorrect, or insincerely, and simply learning to recognize their hallmarks is a 1st line defense against it. The first pillar used an excellent book, Logical Self Defense, as the foundation and I tried to entertain as best I could while hammering home its tenets on logical fallacies, premises, sufficient claim support, etc. The second pillar was a little light psychology and cognitive science. Influence techniques, psychological studies, cognitive blind spots, Benjamin Libet's readiness & awareness experiments, eye tracking saccades, even a live demonstration on the ease of implanting false memories through word choice. The message being that the human mind isn't perfect, it takes shortcuts, and care & awareness of that fact are the minimum requirements to avoiding some of it. The third pillar was simply case studies in propaganda. Readings, movies, news stories, as long as it didn't derail conversation I changed things to whatever was most topical at the time.

Resume/CV/Experience: working demonstrations available below, anything else available on request.

Email: jim.jdiv@gmail.com

Seeking: applied AI, model behavior, evaluation, interpretability, inspectability, research engineering, synthetic data generation, or adjacent roles

I have spent more than 15 years building my own tools when the available ones were insufficient, and being someone sent when there was a problem, either to figure out what was going on or solve it, or both.

This has been as a generalist across analytics and data science for the operational arm of a large public university w/ steadily expanding domain responsibility, informing senior leadership, at times authoring strategy and policy. (I'm not as faculty, although I developed and taught a course for several years, Language of Propaganda, as an adjunct lecturer: adversarial uses of language examined through informal logic, cognitive blind spots, and case studies.) Separate from this, when a self-funding hobby found unexpected product-market fit I grew it to side-business and shipped 10,000+ items.

My academic background is in applied linguistics, NLP, cognitive science, and analytic philosophy. I might have followed a research or academic path, but by the time I completed my master's degree, I had concluded that the paths then available would not give me much room to pursue the questions I actually cared about.

Those interests are language, mind, cognition, and computation, all now converged in modern AI, I have devoted a lot of time bringing in ideas from traditional linguistics and some other areas and turning them into practical tools.

Recent work, including live demos:

- Cartogemma: A REPL-like environment for LLM inference. Explore generation as a branching process, preview output, inject tokens, rewind, ablate or restore heads, trace a chosen token through the layers. Per-head projections, residual contributions, the ordinary logit lens, and full-layer output side by side. The CMD/REPL bar functions once loaded. https://huggingface.co/spaces/anotheruserishere/Cartogemma

- Tokescope: watch a model in real time during inference, flag tokens to monitor & intervene. Catch something surfacing, before output. Once flagged it either logs it, stops the response with a hard gate, or suppresses using a gram-schmidt projection that takes the direction out of the vector. https://huggingface.co/spaces/anotheruserishere/Tokescope

- Bertographer is similar, runs on encoder models, classification task, ie NLI models, Also with ad-hoc steering https://huggingface.co/spaces/anotheruserishere/Bertographer

- An instrumentation and intervention library for examining model internals during inference & using them to decompose behavior and outputs. It's what provides core tooling for the HF spaces listed. It analyzes derived structure, across layers and heads. These traces then use linear-algebraic, statistical, and overlap methods such as SVD, PCA, correlation analysis, and Jaccard similarity, results of which can then be used to steer a model through targeted activation-space interventions.

- Another library builds off of this one in the direction of mechanistic interpretability, for finding SAE-like features without the hassle of training an SAE, providing a range of static and interactive visualizations, scanning & storing model states at some or all steps of inference, among other things.

If any of this maps onto a problem your organization has or a role for which you have not found an easy title I would be glad to talk: jim.jdiv@gmail.com

Ruthless pruning is unneeded with an LLM and it can take me twice the time to say half as many words.

And, early in the ‘GPT era, I hadn’t unchecked the “allow your chats to be used in future training etc” box, and definitionally they are longer and denser than others’ prompts in such raw scrapings of training materials…

Sorry.

This seems an incorrect assumption, or at least a conclusion built on incomplete information and consideration of additional factors. The first one that comes to mind is that these customers will on average be the heaviest users. That doesn’t make them less valuable, that makes them the exact ones most like to be willing to refer others to your product and the most able to do so with specifics on their recommendations and, hopefully, a positive endorsement of the service received, not just the product features.

This implies another aspect as well: even if good support weren’t to get you wider positive awareness, poor customer service and even average service that nonetheless has more negative recommendations to other can lose you business, and that too can have a compounding effect.

In general there’s also no getting around the fact that a smaller number of people will (or should) have disproportionate support requests, unless you’ve got a very homogeneous user base with homogenized use patterns. Or your products &/or documentation are of poor quality.

It’s probably not a bad idea also to take most support requests that aren’t inherently specific to a customer and require intervention as an opportunity & need to review your documentation, perhaps auto surface the most relevant bits it when a person submits a request with a an option “if this answered your question click here and we’ll mark your message as resolved, otherwise we have it, and will be in contact with”.

There’s no getting around the need to provide support and it’s possible for some businesses there could be an ideal global min/max that provides least effort & service cost for then elasticity of price & quality and customer tolerance, but I think if you find yourself in the mindset of “what’s the worst I can get away with” instead of “most I can afford to optimize long term contentment and good will referrals, you could be looking at things wrong. I’m not sure what “relationship” should mean in any case, if not something like what I’ve described.

Those poor battlefields. How rarely it is that anyone thinks of their needs.

So... Bullet anthropomorphizing is also a thing.

In this respect it depends on how AI is used. In this case, I didn't envision it doing this in the "deep research" sense or otherwise making its own conclusions from data, I meant more in the vein of a well scaffolded agent loop iterating through, for example, census tract-level data, cross referencing that with other data sources to find the relevant, granular-but-requires-intelligent-judgment details to piece together countless small datasets to assemble a large pictured. Grunt work that is repetitive but just variable enough you can't do something like download/scrape and assemble at scale because each block or tract or zip code needs one small bit of human judgement.

None of that is my work though, just where I think things might usefully go. For my part I'm trying to jump industries into AI more directly, it aligns well with my background, but that fact combined with zero industry connections (save Claude Code's recommendation & endorsement, that I had to tell it not to email on my behalf to Anthropic) hasn't broken down that wall yet, and in the meantime I try to build useful things that might help in that direction. So I'm aware of AI's blind spots on some things, and its significant capabilities still need significant shepherding.

My hope is that agentic analysis that does this tedious methodical chipping away, comparative cross referencing of seemingly disparate datasets, will help shift society the tiniest bit away from law & policy making via hot takes that make even the well intentioned fall on their face with poor reasoning and the more cynical wield ambiguity a cudgel of control by any emotion they can incite, usually not the good ones.

This is the step that otherwise smart people fail at.

"We were afraid of danger X so we did Y to prevent it and turns out it was a waste because not only did X not get worse, it got better! To heck with Y!"

And don't consider, maybe, things got better for that reason?

This is "only sick people take medicine" logic.

If you're tempted down this line of thinking you need to consider: If nothing had changed or they got worse, would that have been the expected results? What then would be the expected outcome?

Comparative analysis at a minimum, not just to other societies with different norms but attempting at least to find pockets that didn't change as much or as quickly, what happened there and in other sub populations where factors varied.

Otherwise you're just someone complaining how things used to be different, better in any way that fits a narrative that makes you feel comfortable or righteous or whatever.

Other than that, I think it bears considering that any specific level of fear may be a factor in the safeguard that have been put in place to mitigate outbreaks. Without some level of fear, not much would be done. I don't know if it's the direction your thoughts were going in, but an unreflected gut reaction of "just fear, it's never amounted to much" is the potential catalyst for removing guardrails that have prevented worse outbreaks. It's important not to reason solely from that sort of counterfactual premise but chesterton's fence should apply when considering "was the fear justified, has it played a part in directing responses and if so has that response been calibrated to the reality or too much by the fear?" We need to get past this tendency to leave things a hot-takes and gut checks.

Yes, I suppose, but without elaborating further that doesn't explain why you're taking it to be circular, because I could have given some other description of what trends & goes viral on TikTok and you still could have said "Anything can be can be turned into that."

If we take it in the more formal logic direction you're going though it's all very simple and straightforward, here's the p & q -> r of things:

Algorithms of this sort work a particular way in directing next-video selection towards options with some characteristics similar to what the user has engaged with before. I'll stipulate there are lots of ways that can be done, time horizons and methods of weighting different factors but that's the broad strokes. Take this as premise P.

There are certain things that trend more frequently than others and they share some common traits, it really doesn't even matter what those specific things are, we can take this as an axiom without it being controversial.

Therefore, if anti-democrat content is disproportionate to pro democrat or anti or pro GOP, it isn't automatically thumb-on-scale, it can simply be that anti-democratic content has more similarities to what typically trends than those others.

This isn't circular. It's trending content is similar, anti-democratic content trends more often, therefore anti-democratic content can simply have been more similar to other trending things.

You're correct of course about Occam, but then your bring up that aspect of things was merely expanding on what I explicitly stated in my original comment when I said it didn't mean TikTok didn't tip the scales, only that such a thing isn't the only possibility. In short, it was clearly not stated as an "IIF/if-and-only-if" argument.

Going on to your For "arguably the opposite" final statement:

I think that too needs more little explanation. As-is, it sounds as though you're saying essentially "the fact that simpler explanations can be wrong is potential evidence for deliberate interference". That's a line of thinking when, offered without expansion, steps somewhere just adjacent of conspiracy thinking of the "the evidence is in the lack of evidence", and I doubt that's your intent, but I'm not sure either where that's heading otherwise.

In short, anti democratic content was, on average, more entertaining or emotion provoking.

That doesn't have to hold a deeper meaning on the value of any particular political viewpoint, or require tiktok's thumb on the scales of the algorithm to explain things. I'm not even saying TikTok didn't/doesn't do such things, but that type of interference isn't required to explain this trend.

Wittgenstein termed precisely this sort of practice a “language game”.

With the rest of his work (Philosophical Investigations in particular) it is a hugely useful lens through which to view and significantly better understand the use and function of language in all its forms.

Given that language is becoming a ubiquitous UI and UX and glue for just about everything, I highly recommend it, and as a work of philosophy it is much more accessible to laypeople not of the field than some other important works in the area.

https://en.wikipedia.org/wiki/Language_game_(philosophy)

I even share his concern about struggling to keep pace with the rate of change lately, and agree that my working in a frontier lab or any other such environment would certainly help with that!

I have a weird background mix of analytic philosophy, linguistics/NLP, propaganda research, and long-term institutional data science/strategy work, which unfortunately does not make ATS systems especially low-friction as I try to jump industries.

So I keep busy the best I can: lately building tooling around runtime observability, intent legibility, and intervention in LLM systems.

Some small public artifacts finally going up: https://huggingface.co/spaces/anotheruserishere/Cartogemma

Eh. Worth a shot!

I've seen him comment on it a few times over the years, though I wouldn't take my vague memories on them as canonical: He's mainly pissed off that many avid fans of some of his books and Accelerando in particular show a few patterns of thinking: 1) They miss his intent to show future for humanity that was much more of a "Warning! Do Not Enter!" than as any sort of advocacy/enthusiasm for it 2) Really pissed off that a subset of the that are ultra ultra wealthy either miss the signpost or dont care and seem to take it & other hard-takeoff singularity stories as potential maps & guidebooks on the path 3) He's annoyed (maybe not the right word) that a significant portion of people that cheer on the idea of a singularity do so in part for the hope of something like immortality, biological or uploads, specifically in a way that reinvents quite a bit of the trappings and mythos and other cultural baggage embodied in a lot of western Christianity, most notably a lot of the TESCREAL hodge podge of groups.

Again, all of this is my own dodgy recollections and paraphrases.

Put another way, search out the great vowel shift. That happened over more time but then again the contact with different speakers wasn’t as constant as every day on the internet. It’s just what happens, how things spread. No different and maybe to a further degree than typical memes.

They’ll just start cherry picking their patients, finding ways to squeeze out the people just that little bit lower on the prognosis curve. Or at least that will be the risk in a setup like that.

>"Root Cause: A dog named Kubernets ate a Yubikey

Ah, yes, irresponsible to get taken in by one of the well-known classic exploits. The 'ol "distract someone with a lottery windfall & make a dongle irresistibly tasty to another person's pet". When will people learn.

Especially when this particular thing is just one in a very long line of metaphors humans make to our own minds’ operations every time a new major technology comes to play a pervasive role in society. Computers, steam engines, even aqueducts were not immune to comparisons of thought flowing like water, funneled by deliberate intent, etc. And for some, a certain amount of hand wringing worry or even moral panic about “what it’s doing to us”, eg taking away critical thinking because “OMG calculators!”