4,387 karma · joined March 14, 2008
Alum of Brex, Google, Microsoft, Salesforce, Twitter, Convoy.
Contact: steve@logic.inc
Surely we'd agree that not everyone who uses software is a software engineer, but writing software is generally agreed upon as "software engineering". If you consider my comment as a whole, and the linked document, then the application of the definition makes sense.
Large Language Models are incredibly similar to black-box non-deterministic interpreters. And prompt engineering feels very much like writing code for said interpreter.
> ... we used an analogy of prompts as the “source code” that a language model “interprets”. Prompt engineering is the art of writing prompts to get the language model to do what we want it to do – just like software engineering is the art of writing source code to get computers to do what we want them to do.
Borrowing from Oxford Dictionary, the definition of "engineering" is:
> the branch of science and technology concerned with the design, building, and use of engines, machines, and structures.
I think it's pretty reasonable to say that "prompt engineering" falls squarely in the realm of "technology concerned with the use of a machine".
[^1]: https://github.com/brexhq/prompt-engineering#why-do-we-need-...
I just updated the guide: https://github.com/brexhq/prompt-engineering/commit/3a3ac17a...
I firmly believe that the introduction of LLMs will be as essential to the future of human-computer interaction as the introduction of the mouse and keyboard were. Every technology company with sufficient resources should be exploring the implications of LLMs on their business.
Most of our research for LLMs falls into three buckets:
1) Internal processes – Any area where an employee is writing or reading some kind of communication is up for grabs.
2) Developer productivity – Whether it's improving the quality of code, reducing time to implementation, or answering questions about our services and architecture... empowering our eng team with LLMs is an area of major interest.
3) Customer Experience – Employees frequently need to write memos for expenses or have questions about Travel & Expense policies (e.g. "Can I buy alcohol at this team dinner?").
We're, of course, exploring far more interesting use cases than just what's above, but that's the low hanging fruit.
If OpenAI makes their revenue off of charging exorbitant fees to use the LLMs, they'll no longer have incentive to ever open them (even if abuse / misuse concerns are addressed) AND they'll have no incentive to make them more efficient.
OpenAI has yet to show it can have a sustainable advantage. Every other player in the space benefits from an open model ecosystem and efficiency gains.
https://www.postgresql.org/docs/current/queries-table-expres....
With IAP, it's really difficult to know what you're getting and how much it will cost you in the end. And reviews may be discussing a completely different app experience. Plus the constant feeling that you're being nickel and dimed.
I'm also a heavy user of terminal though, and that was the one thing I spent extra time on, because there were subtle differences that I didn't like. If you give neovim a shot again, try this in your ~/.config/nvim/init.vim (aside: I do miss the simpler path of ~/.vimrc):
" Neovim's default terminal mode bindings aren't great.
" This makes them behave like vim's.
tnoremap <Esc> <C-\><C-n><C-w>
tnoremap <C-w> <C-\><C-n><C-w>
"Always enter the terminal in insert mode
autocmd BufWinEnter,WinEnter,BufEnter term://* startinsert
autocmd TermOpen,TermEnter * startinsert
command! -nargs=0 Terminal :vsplit | term
And then they realized there was a lot of money to be made, and switched to a for-profit: https://techcrunch.com/2019/03/11/openai-shifts-from-nonprof...
Now they lock up their most successful advancements, under the guise that if they released things for free they would be abused... but then their results are replicated and released for free by others, and we see there's no meaningful abuse (e.g. unstoppable spam).
They still do amazing research, and publish lots of papers, but that's a pre-req for them to hold on to their talent.
Storage + querying is vastly underutilizing a tool that is designed to help you safely manage the entire lifecycle of your data – From defining that data (DDL), making it accessible (queries, indexes), ensuring consistency (checks, constraints, types, etc...), managing cascading data events (triggers), and more!
It's a fool's errand to try to (poorly) replicate these things elsewhere when there's already a battle hardened system with decades of investment that allows you to do these things declaratively and consistently for free.
And it's much safer! The DB will enforce that validation even when services are doing massive migrations or new clients are mutating the data in ways you didn't anticipate. It's far less error prone than ensuring that all of your services are validating data the same way.
> Validation should be done in the application layer for security and scalability.
Input validation, for sure. Sanitize as close to the perimeter as you can. But for consistency and integrity of data, let the database do what it's great at.
Your application server will probably not do it as efficiently as your highly-tuned database. There's no reason to assume that pushing that work into the database isn't a better scalability strategy. And it's certainly a more robust strategy from an organizational perspective.
Fixing slow things is frequently trivial compared to fixing bad data or tracking down subtle data bugs. For example, borrowing from a real world scenario in an Elixir service that uses Ecto to handle data mapping and validation, our code assumed a key in a JSON blob was a string, but in a handful of old rows it's a number. This inconsistency only existed in ~0.0005% of rows, and only in old data, but it surfaced in all sorts of weird ways. And was difficult to track down.
Using the RDBMS to handle validation would have never allowed this to happen.
> "Premature optimization" is the rallying cry of people standing in the corner of a room with a painted floor holding a paint brush.
That strikes me as an uninformed opinion. Optimizations usually work by making lots of assumptions on the nature of your problem and then implementing something that depends on those assumptions always holding. This leads to more convoluted and fragile code, but in return you get performance.
Writing optimized code frequently involves removing optionality, which is the equivalent of painting yourself into a corner.
By centralizing data validation, it removes many potential failure and inconsistency scenarios (w.r.t different services validating things differently).
Worrying about CPU, without seeing if it's a real problem for your use case, is a premature optimization. Similar to worrying about foreign key constraint checks being too expensive. This is rarely the case, but if it winds up being a problem, you can relax it later and move the check it elsewhere in your stack (or remove it entirely).
SQL addresses this via the natural join keyword `using`, where you enumerate the common columns between the two tables being joined. It isn't too convenient for your example unless your pk naming pattern happens to be `<entity>_id` instead of just `id` (note: this naming pattern has all sorts of other adverse consequences though). But it does provide convenience in some cases without introducing backwards compatibility risks as the schema evolves.
I think one thing being lost in translation here is that Brex considers small businesses to be different from startups, and it's an important distinction internally that is likely getting lost externally.
For example, we just acquired Pry to help with founders at small startups figure out financial projections and needs. If you're an early stage funded startup, Brex is probably still a good fit.
The policy cited here is more about no longer being a good fit for "traditional" small businesses.
It was really only during the break-away adoption by the data science community (and other mass audiences) that started adding pressure to bloat the language. The adoption of 3rd party libraries like numpy, which are so critical to modern Python, also destroyed the community's "batteries included" philosophy.
Older Python felt a lot like using FreeBSD, where the whole system is cohesive and designed to work together, whereas modern Python feels a lot more like Linux – a bunch of disparate systems that try to work well together (in the best of times).
Both approaches have their strengths, and the language has certainly improved in some areas, but I prefer the older style.
In any case, it is fascinating to see how mainstream Python is now. Maybe this change in mindset was required for it to take over the world. We've come so far since Paul Graham cited the language as esoteric and its usage a high signal of competence (http://www.paulgraham.com/pypar.html).
The kind of reasoning you pose is similar to "SQL doesn't scale to infinity, so start with an infinitely scalable eventually consistent document store". This line of thinking is dangerous to most companies, and often the death of startups. Assume YAGNI.
There's some truth to "Nobody ever got fired for choosing IBM", but IBM was never a good choice for anyone except consultants. k8s isn't too far from that.
- Cert Renewal / Rotation: https://docs.docker.com/engine/reference/commandline/swarm_c...
- TLS Termination: Arguably this is an application layer concern and out of scope for an orchestration layer, but in either case use Nginx (or your favorite TLS terminating proxy) as your ingress point. This is equivalent to the Nginx ingress controller in k8s.
- Storage: Docker volumes (local or ceph, for distributed).
- Disaster Recovery: Node failure? Cluster failure? Data loss? What kind of DR? The strategies you'd use with Swarm will be very similar to those you'd use with k8s.
- Grouping resources: Labels and/or services.
In a Swarm world you'll have fewer moving pieces, far lower resource requirements, and will manage it all in a docker-compose file(s). You'll also get things like service discovery, routing, and secrets management for "free". No additional configuration needed.
My fallback, if Swarm is ever removed, is Hashicorp's Nomad. I've only used it in a hobby capacity, but is (at face value, at least) another wonderful piece of software. I've used k8s a bunch in production (including right now at my current employer) and just can't recommend it – it's a technical distraction for most teams.
You can read Docker Swarm's docs end-to-end in < 30 minutes, and have a cluster running in roughly the amount of time it takes to install Docker on three boxes.
Docker's failing was a failure to market Docker Swarm. Most people don't even know it exists, yet every single machine with Docker has it already! The last time I checked, they only had a single dev that was part-time allocated to supporting it.
So it's not so much that Docker failed to jump on the k8s bandwagon, so much as they built a wonderful thing that they never marketed and semi-abandoned, to focus on Docker Hub and other core Docker things they thought they'd be able to more directly monetize.
Fortunately, Docker Swarm remains a wonderful piece of software, even without particularly active investment, because it is a fairly complete offering for most use cases. And it seems to be growing in adoption.
As a general rule of thumb, yes. Starting with denormalization often opens you up to all sorts of data consistency issues and data anomalies.
I like how the first sentence of the Wikipedia page on denormalization frames it (https://en.wikipedia.org/wiki/Denormalization):
> Denormalization is a strategy used on a previously-normalized database to increase performance.
The nice thing about starting with a normalized schema and then materializing denormalized views from it is that you always have a reliable source of truth to fall back on (and you'll appreciate that, on a long enough timeline).
You also tend to get better data validation, reference consistency, type checking, and data compactness with a lot less effort. That is, it comes built into the DB rather than introducing some additional framework or serialization library into your application layer.
I guess it's worth noting that denormalized data and document-oriented data aren't strictly the same, but they tend to be used in similar contexts with similar patterns and trade-offs (you could, however, have normalized data stored as documents).
Typically I suggest you start by caching your API responses. Possibly breaking up one API response into multiple cache entries, along what would be document boundaries. Denormalized documents are, in a certain lens, basically cache entries with an infinite TTL... so it's good to just start by thinking of it as a cache. And if you give them a TTL, then at least when you get inconsistencies, or need to make a massive migration, you just have to wait a little bit and the data corrects itself for "free".
Also, there are really great horizontally scalable caching solutions out there and they have very simple interfaces.
In either scenario, I’d still generally encourage avoiding storing JSON(B) unless there isn’t a better alternative. There are a lot of maintenance, size, I/O, and validation disadvantages to using JSON in the DB.
Regarding JSON schema, if you have a Marshmallow schema or similar, yes that’s a wonderful starting point. This should map pretty closely to your DB schema (but may not be 1-to-1, as not every field in your DB will be needed in your API).
I’d suggest avoiding storing JSON at all in the DB unless you’re storing JSON that you don’t control.
For example, if the JSON you’re storing today has a nested object of GPS coords, temperature, etc.. make that an explicit table (or tables) as needed. The benefits are many: indexing the data becomes easier, the data is stored more efficiently, the table will take up less storage, the columns are validated for you, you can choose to return a subset of the data, etc… You will not regret it.
If you have any foreign key columns, add indexes on them. And if you’re doing any joins, make sure the criteria have indexes.
Similarly, if you’re filtering on any of the nested JSON fields, index them directly.
This alone may be sufficient for your perf problems.
If it isn’t, then here’s some tips for the blobs.
The JSON blobs are likely already being stored in TOAST storage, so moving them to a new table might help (e.g. if you’re blindly selecting all the columns on the table) but won’t do much if you actually need to return the JSON with every query.
If you don’t need to index into the JSON, I’d consider storing them in a blob store (like S3). There are trade offs here, such as your API layer will need to read from multiple data sources, but you’ll get some nice scaling benefits here and your DB will just need to store a reference to the blob.
If your JSON blobs have a schema that you control, deprecate the blobs and break them out into explicit tables with explicit types and a proper normalized schema. Once you’ve got a properly normalized schema, you can opt-in to denormalization as needed (leveraging triggers to invalidate and update them, if needed), but I’m betting you won’t need to do any denorm’ing if you have the correct indexes here.
And since you have an API layer, ideally you’ve also already considered a caching layer in front of your DB calls, if you don’t have one yet.
I remember having a lot of fun solving it, in Coffeescript (which was all the rage at the time), also with TSP but using the Pearson correlation coefficient as the similarity metric: https://gist.github.com/stevekrenzel/1505619
Here's a little demo of it in action: http://krenzel.org/files/insta/
It's similar to how DNS can have two domains point to the same IP, but then you can change one of those domains point to a new IP.
- 2 database servers (with half that RAM and 24 cores each)
- 2 redis servers
- 3 elasticsearch servers
- 11 web servers
- 4 load balancers
- 3 service workers