I know a few different companies who ultimately moved out of the education market completely or just try to leverage their education traction as a beachhead to other markets. It sounds like you're focused on the education market - what's your take?
33 karma · joined July 28, 2022
I know a few different companies who ultimately moved out of the education market completely or just try to leverage their education traction as a beachhead to other markets. It sounds like you're focused on the education market - what's your take?
I'm definitely curious on the technicals but there is also a bit of a trust element here - both on trusting that my email (likely some of my most sensitive data) is handled with care and trust that the actual responses are phrased well.
> And though when we started our business in 2023 (ChatGPT wasn’t out yet), you could begin to feel that something like that was possible in a way it wasn’t before.
perhaps a typo in year?
We use human evaluation but that is naturally far from scalable, which has especially been a problem when working on more complicated workflows/chains where changes can have a cascading effect. I've been encouraging a lot of dev experimentation on my team but would like to get a more consistent eval approach so we can evaluate and discuss changes with more grounded results. If all of these metrics are low confidence, they become counterproductive since people easily fall into the trap of optimizing the metric.
I tried multiple other solutions but kept running into the problem that occasionally the framework would give me some score/evaluation of an LLM response that didn't make any sense, and there was minimal information about how it came up with the score. Often, I'd end up digging into the implementation of the framework to find the underlying evaluation prompt or classifier only to realize that the metric name is confusing or results are low confidence. I'm more cautious about using these tools now and look more deeply at how they work so that I can assess grading quality before relying on them to identify problematic outputs (e.g. hallucinations).
Top-level JSON files seem to include all the prompts.
Cool concept! How was your process building and iterating on these prompts?
The wording gives that kind of feel and now I'm wondering if my feeling around it is at all accurate.
>...
>"Like any enterprise, we reserve the right to make decisions about our business partnerships,” according to the post, which said that The Don Lemon Show is still “welcome to publish its content” on X.
What specs are your machine? I'm trying to figure out what I can get that would be sufficient enough to switch off.
In comparison, all of these game server hosting sites charge ~$20/month, notably cheaper than that or hosting with any cloud host.
They also "manage it for you" which for us technical people is probably more of a con than a pro but honestly is fine for a game server as long as it works and allows reasonable access (most seem to at least let you FTP in).
I'm open to a better alternative if you have any suggestions
I briefly looked into how much it'd cost to run on an EC2 instance and it seems like it'd be order of $100 - $200 per month while these dedicated game server companies seem to be offering it, fully managed, for only $20/mo! I'm guessing they do this by having larger instances running many people's servers?
I'd prefer to manage and run this myself but when the price is an order of magnitude more to do so, it doesn't seem worth it to self host unless you happen to have a suitable always on home server sitting around.
Do you find it a pain to have to manage all of this for people?