bschmidt1

Will be funny/ironic when the first AI companies start suing each other for copyright infringement.

Personally for me the "3 column" UI isn't that good anyway, I would have gone with an "MMO Character Creation" type UX for this.

For Vista in this era it was mostly boot from USB stick because it was too big for CD and DVD was bougie

Absolutely, especially if the pricing makes sense! Would be very nice to just focus on the creative suite which is the real product, and less on the AI infra of hosting models, vector dbs, and paying for GPU.

Curious if you're using providers for models or self-hosting?

"New school PHP" frameworks like Laravel are nearly exactly like Ruby-on-Rails: The same MVC style, database and ORM built-in, Laravel is so similar to Rails in many ways.

I would say:

"Laravel is the new Rails"

and

"New PHP is the old Python/Ruby"

The original dev use case for Wordpress where you can easily put up a basic CRUD app with user logins and roles/permissions was largely displaced by Django, which is just a little bit more mature of a project for such tasks than Wordpress could ever be. WP never wanted devs anyway, they wanted bloggers - so a lot of people stopped writing PHP simply because WP lost popularity as a web framework.

PHP lost a ton of up-and-coming developers to Python (esp. in academia) and JavaScript (esp. to Node), in the same way Flash/AS3 lost developers to iOS/Android. Unlike Flash, PHP never really died - just kept hanging around.

It's not a bad language, brings back fond memories at least. But there's nothing about its performance or usability that stands out, and there's no core platform need for it the way there was with Wordpress. JavaScript has the browser DOM and Node, Python has AI/ML libraries and best practices that aren't available in other languages, and in terms of another PHP use case - all the dynamic languages can quickly start an http server on localhost now. There's just no use case for PHP.

Anyway, the ZKP concept is not about decrypting hashes at all, but looking at peripheral data to prove something (Alibaba Cave - Victor only knows Peggy knew the password because he had access to some other data - the path she took). "checking length etc." only if those hints are already available to the system in some way. And because of this approach, why would you need the hash? Just don't use passwords at all in the case of ZKP right? Simply rely on the other identifying data that you have access to, that you use anyway. Also - how secure is this loose profiling technique compared to email-backed passwords over HTTPS?

I imagine few product use cases allow for a server to trust all the clients with encryption, while not trusting itself - but there are some use cases like when the server is not the source of truth - file system service, or peer-to-peer stuff like ledgers: If the server's purpose is just to maintain a shared ledger and all the clients in the network are trusted.

But in the case we're talking about, of a service that authenticates clients, you're saying you can't trust the authenticator when that is kinda the point of authentication - they don't trust you, or rather - the server cannot tell for sure that any incoming connection is who they say they are, even if it has "zero knowledge" like their IP address and a face scan (your brother in the same house might pass). The point of a username and password is that you want the server to not trust any connecting clients unless they have this specific data precisely.

So I wouldn't use it for auth.

Also you might not understand web dev 101. Every website including this one that uses HTTPS sends encrypted data, the password you enter in a text input is in plaintext. For the backend - as I said above, the server hashes it and saves the hash, never the plaintext password.

That's how it works - nobody said anything about "log files".

I've been working on something adjacent to this concept with Ragdoll (https://github.com/bennyschmidt/ragdoll-studio), but focused not just on creating characters but producing creative deliverables using them.

    Y = λf.(λx.f(x x))(λx.f(x x))

    const Y = f => (x => f(x(x)))(x => f(x(x)));

    const YC = startup => (getMoney => startup(getMoney(getMoney)))(getMoney => startup(getMoney(getMoney)));

True, really just meant "not absolute".

> such that C happened before D in every reference frame

If that's possible, considering the seemingly infinite number of instances where it'd have to both happen and in that way/order, I would consider those kinds of events to be the fundamental/baseline "forces" or asymmetries.

It's more like digital construction: Programming is more like the construction project, and less like the AutoCAD "design" piece that engineers work on. The AutoCAD piece has more in common with product design than programming.

If anything, designers should be "engineers" and "eng. directors", and programmers should be "technicians" and "technical directors". They engineer products, with deliverables being design documents. And the techs implement it with the deliverable being the technology itself.

> when to encrypt

It depends on what you want to do, if it's user login over HTTPS you can pass a plaintext password to the server and hash/compare on the server only. It would still be secure because the plaintext is never saved in a db (only the hash is), and was TLS encrypted in transport.

-----

> This is a sha256 hash of my birthday, write a function that returns if I'm over 21: `1028d7ea22cbbcb17c4926b08b591506227d7b0e32ce6ce76122461e551a5ab2`

You hash the point of access like a password or key, not the data itself. When the access is granted, you return the data. sha256 is never meant to be decrypted. It would be like this:

    interface User {
      id: sha256;
      name: string;
      age: number;
    }

    const users: User[] = fetchUsers();

    const isOver21 = plaintextId => users[encrypt(plaintextId)]?.age >= 21;

    const validHashes: BirthdateHashSha256[] = seedHashesForToday();
    
    const isOver21 = hash => validHashes.includes(hash);

Very interesting, thanks for sharing that detail. As someone who has tinkered with tokenizing/training I quickly found out this must be the case. Some people on HN don't know this. I've argued here with otherwise smart people who think there is no data preprocessing for LLMs, that they don't need it because "vectors", failing to realize the semantic depth and quality of embeddings depends on the quality of training data.

Nobody even knew what OpenAI was up to when they were gathering training data - they got away with a lot. Now there is precedent and people are paying more attention. Data that was previously free/open now has a clause that it can't be used for AI training. OpenAI didn't have to deal with any of that.

Also OpenAI used cheap labor in Africa to tag training data which was also controversial. If someone did it now it would they'd be the ones to pay. OpenAI can always say "we stopped" like Nike said with sweat shops.

A lot has changed.

> It doesn't answer arbitrary questions about the data.

Why would you need a "ZKP" to prevent anyone from "asking arbitrary questions" you simply don't build that functionality.

When I create a web server and allow people to login through an endpoint, they can't ask arbitrary questions about user data either - how would that functionality even exist without me writing it? Typically the server doesn't even know passwords. It simply compares a hash - the hash is computed client-side and the server never sees the real password.

Any peripheral user data you want to return is up to you. Identity is not "built in" to conventional programming languages.

Furthermore, none of the ZKP libraries on npm do anything. Most of them are utility libraries with functions like "generateUUID" and "leftPad". The ones from providers like Cloudflare (their least popular stuff) are just private/public key encryption libraries that they call "ZKP".

The wikipedia on deep learning transformers:

    All transformers have the same primary components:

    - Tokenizers, which convert text into tokens.

    - Embedding layer, which converts tokens and positions of the tokens into vector representations.

    - Transformer layers, which carry out repeated transformations on the vector representations, extracting more and more linguistic information. These consist of alternating attention and feedforward layers. There are two major types of transformer layers: encoder layers and decoder layers, with further variants.

    - Un-embedding layer, which converts the final vector representations back to a probability distribution over the tokens.

> You're deliberately misusing terms, probably to draw attention to your project.

Haha well since I have like 30 followers and the npm is free/MIT whatever scheme you think I'm up to it's not working. Anyway a text autocomplete library is not exactly viral material. Jokes aside, no I am trying to use accurate terms that make sense for the project.

Could just make it anonymous - `export default () => {}` - and call the file `model.js`. What would you call it?

> Did they tag Polish parts of speech too? Or Ancient Greek?

Yes, all the foreign words with special characters were tokenized and trained on. An LLM doesn't "know any language". If it never trained on any Polish word sequences it would not be able to output very good Polish sequences anymore that it could output good JavaScript. It's not that has to train on Polish to translate Polish per se, but it does has to have the language coverage at the token level to be able to perform such vector transformations - which is probably most easily accomplished by training on Polish-specific data.

See https://huggingface.co/pranaydeeps/Ancient-Greek-BERT

> The model was initialised from AUEB NLP Group's Greek BERT and subsequently trained on monolingual data from the First1KGreek Project, Perseus Digital Library, PROIEL Treebank and Gorman's Treebank

First1KGreek Project

> The goal of this project is to collect at least one edition of every Greek work composed between Homer and 250CE

> Citation needed

https://openai.com/index/new-and-improved-embedding-model/

> The new model, text-embedding-ada-002, replaces five separate models for text search, text similarity, and code search, and outperforms our previous most capable model, Davinci, at most tasks, while being priced 99.8% lower.

https://platform.openai.com/docs/guides/embeddings/embedding...

Scroll to embedding models

> Anyway, looking forward to hearing news about your image generation project. Any news?

Not yet! Feel free to follow on GitHub or even help out if you're really interested in it. Would be cool to have pixel prediction as snappy as text autocomplete.

Altman has been clear for a long time he wants the government to step in and regulate models (obvious regulatory capture move). They haven't done it, and no amount of Elon Musk or Joe Rogan influence can get people to care, or see it as anything other than regulatory capture. This is OpenAI moving forward anyway, but they can't be the only ones. Hey Anthropic, get in...

- It makes Anthropic "the other major provider", the Android to OpenAI's Apple

- It makes OpenAI not the only one calling for regulation

It reminds me of when Ted Cruz would grill Zuck on TV, yell at him, etc. - it's just a show. Zuck owns the senators, not the other way around. All the big players in our economy own a piece of the country, and they work together to make things happen - not the government. It's not a cabal with a unified agenda, there are competing interests, rivalries, and war. But we the voter aren't exposed to the real decision-making. We get the classics: Abortion, same-sex marriage, which TV actor is gonna win president - a show.

> I’ve had conversations with people who were within the company including some senior leaders

There we go, why didn't you just say that in a disclaimer up front? At one point I cursed the Apple influencers and was mainly referring to you :D

You had a very different experience than the mainstream since you worked at Apple when it launched.

> This by the way is straight from the horses mouth - Steve Jobs was one of presenters for interns that year.

Sounds like you were doing keg stands in the Apple Koolaid while most of us were still pirating Windows Vista off LimeWire and changing discs at red lights! You haven't lived unless you had a 6-disc changer (in your trunk for some reason).

Thanks.

> it's not required to. I've run lots of models

Then you must know about skip-gram and how embeddings are trained: https://medium.com/@corymaklin/word2vec-skip-gram-904775613b...

What is meant by "sliding window" or "skip gram" is bigram mapping (or other n-gram).

This is ML 101.

It's the same training methodology and data structure used in my next-token-prediction lib, and is widely used for training for LLMs. Ask your local AI to explain the basics, or see examples like: https://www.kaggle.com/code/hamishdickson/training-and-plott...

> ChatGPT doesn't use parts-of-speech

Yes it does, there's not only a huge business in tagging data (both POS and NER) adjacent to AI, but OpenAI specifically famously used African workers on very low wages to tag a bunch of data. ChatGPT uses text-embedding-ada, you'll have to put 2 and 2 together as they don't open source that part.

Mistral says:

"The preprocessing stage of Text-Embedding-ADA-002 involves applying POS tags to the input text using a separate POS tagger like Spacy or Stanford NLP. These POS tags can be useful for segmenting sentences into individual words or tokens."

> I use Claude to make new languages

Cool story, has nothing to do with the topic

  Location:     San Francisco 
  Remote:       Sure
  Relocate:     Depends
  Technologies: Node/React, working with LLMs would be a plus
  Résumé/CV:    https://bennyschmidt.com
  Email:        hello@bennyschmidt.com
  -----
  YC alum of:   Brave, Checkr 
  GitHub:       https://github.com/bennyschmidt
  Neat project: https://github.com/bennyschmidt/next-token-prediction

Check if it has numbers: \d

Check if it has symbols: \W

Check if it's 6-64 chars long: {6,64}

> This doesn't make sense

Promise you it's how it works.

> Hashes are binary yes/no checks

Nope, just means encrypted text.

> every government in the world runs an API

Hilarious you think a decentralized approach where every participant has a copy of an append-only ledger is simpler than a central server with SQL database. The argument for decentralization was never that it was simpler - it's of course way simpler in many ways to have a single source of truth. If you mean using that passport library on a regular server, then you also have to run an API or nobody can use it.

In both comments I assert that phenomena cannot be wholly modeled (or replicated). The gist is that descriptions and replicas are always lossy to some extent.

The second comment just expands on why, relating it ultimately to physical limitations (features) of physical reality.

You can get uncannily precise and predictive with some knowledge - engineering, manufacturing, math - that any lossiness is truly negligible. Especially the smaller the scope and more "medium sized" the object or idea in question. But compounded, and at extreme scales, the incompleteness of our understanding of reality shows.

I agree with your 1 & 2.

- A happened before B

- B happened before A

- A & B happened at the exact same time

- Neither / there is no "before"/"after"

Considering this is the case with anything measured, what explanatory "truth" would be left when you consider a perfect description of reality? One that accounts for the global reference frames of every possible metric (not just order of events)?

There's no absolute "truth" to any of the mainstays:

Where things are

When they happened

In what order they happened

Even whether or not they happened (there is a perspective where all the values of a measured thing can be zero - like mirroring the velocity of a speeding object to reduce its speed to zero, or rotating a plane to make its height zero - and the different physical implications therein).

We correlate observed "locations in space" and the "passage of time" to heuristical approximations that work (as scientific theories) for our intents and purposes on a hyper local level (a tiny fraction of just our galaxy), but they break down on macro levels because they aren't really accurate descriptions of reality. Our physics doesn't work at scale - not smaller than atoms, nor larger than ~solar systems, without theories falling apart.

Space

The XYZ coordinate system we understand as "3D space" works great for locating and measuring the size of objects locally, we could perfectly track a tiny marble bouncing down a huge mountain with a granular enough XYZ coordinate system and telemetry. But that wouldn't work on larger scales like comparing the movement of subatomic particles with the movement of stars in the same graph. The graph couldn't accurately chart that without normalizing data or adjusting the reference frame toward one bias or the other (see scale invariance, vacuum catastrophe). We can only neatly slap a fixed reference frame over a scoped area, invent units and call it a "field", if we expect any predictive utility.

Time

There's not really an absolute "arrow of time" with a past and future, where A happened, then B happened, etc. in an absolute sense. As shown in the star example, it completely depends on the orientation and velocity of the observer, and other factors. No one perspective is more true than the other.

We all happen to hang out in the same frames, so it seems like some of these are rigid laws - they effectively are, for us - but they aren't really, ultimately. When you eliminate the heuristics that give value to science there's nothing left - which as mentioned above is a possible and potential state.