243 karma · joined March 13, 2014
The main aspects are:
On the trading side, HFT is mainly about what's called "market microstructure." These are relatively simple things that are quirks or features of individual markets. Things that most people don't care about, but can make a large difference in HFT profits include when an order comes in, who gets to trade with it first? How are orders routed between different exchanges? What are the popular types of algorithms that people who place large orders use?
On the tech side:
The main is minimizing tick to trade latency. This includes using techniques as basic as colocation at the exchange's (very expensive) data centers and as involved as designing custom ASICs and anything in between like bypassing the normal network stack.
Unfortunately the real state of the art things are all closely held trade secrets, but I'd start reading the exchange data specs to start getting an understanding of the microstructure side, and you can build some of your own experiments to see how quickly you can go from tick to trade. Can you get it sub 1ms? sub 100 us? 10us?
When Larry Ellison created Oracle Linux, he said this explicitly: given RH's then model, Oracle can take RedHat's work for free and sell it for money.
Reddit's changes would mean that they would have to cut off all of their free users entirely, and 10x the cost of the paid users. Also, they would have to lose money on all of their existing yearly paid paid users until renewal time.
For what Vercel is, it's a steal.
This is the simplest I could get it: still uses the OpenAI npm package and doesn't need streaming:
https://github.com/yisding/gpt-edge-test
If you want to try streaming you can take a look at the readpanda repo.
I first heard about Langchain at the Scale hackathon when I think it was Colin who was like “half the people are using Langchain” and I was like whatchain?
Amazing trajectory!
I'm not saying caste discrimination doesn't exist, but let's not paint 1.4 billion people with the same brush.
It was a long day, but one of the most fruitful ones I've had in a long while.
But didn't you mention that there may be some ways to isolate the user input, using spacing and asterisks and such?
I agree though that leaking a prompt or two by itself doesn't really matter. What's probably a bigger concern is security/DoS type attacks, especially if we build more complicated systems with context/memory.
Maybe Scale will also hire the world's first "prompt security engineer."