Landing page link is in my bio
We've been keeping quiet, but I'd be happy to chat more if you want to email me (also in bio)
768 karma · joined January 20, 2012
If you think in long timeframes, want to build for the future of software, join us at kinelo.com
Generally happy to chat (except recruiters...): keith [at] kinelo.com
We've been keeping quiet, but I'd be happy to chat more if you want to email me (also in bio)
If this protocol gets adoption we'll probably add compatibility.
Which would bring MCP to local models like LLama 3 as well as other cloud providers competitors like OpenAI, etc
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Integrations and server engineer (Go/TypeScript/Python connections to data sources, and data syncing) with some devops responsibilities
- Generalist AI/ML engineer (writing agent code, RAG, prompt engineering, etc)
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Please visit https://avy.breezy.hr
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Senior Applied AI/ML engineer (including LLM fine-tuning, search/retrieval systems, and various vision and NLP tasks)
- Generalist AI/ML engineer (writing agent code, RAG, prompt engineering, etc)
- Integrations and server engineer (TypeScript/Go/C++ connections to data sources, and data syncing)
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr (not all positions posted there yet)
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Senior Applied AI/ML engineer (including LLM fine-tuning, search/retrieval systems, and various vision and NLP tasks)
- Marketing (in the "growth hacker" spirit) -- if your dream is to launch the fastest-growing B2B SaaS product ever, we want to talk with you.
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr (not all positions posted there yet)
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Senior Applied AI/ML engineer (including LLM fine-tuning, search/retrieval systems, and various vision and NLP tasks)
- Generalist AI/ML engineer (writing agent code, RAG, prompt engineering, etc)
- Marketing (in the "growth hacker" spirit) -- if your dream is to launch the fastest-growing B2B SaaS product ever, we want to talk with you.
- MacOS (Swift, Objective-C, C/C++, etc) at the Senior and Staff levels. iOS experience is OK.
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr (not all positions posted there yet)
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Generalist AI/ML engineer (writing agent code, RAG, prompt engineering, etc)
- Senior Applied AI/ML engineer (including LLM fine-tuning, search/retrieval systems, and various vision and NLP tasks)
- Marketing (in the "growth hacker" spirit) -- if your dream is to launch the fastest-growing B2B SaaS product ever, we want to talk with you.
- MacOS (Swift, Objective-C, C/C++, etc) at the Senior and Staff levels. iOS experience is OK.
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr (not all positions posted there yet)
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Generalist AI/ML engineer (writing agent code, RAG, prompt engineering, etc)
- Senior Applied AI/ML engineer (including LLM fine-tuning, search/retrieval systems, and various vision and NLP tasks)
- Marketing (in the "growth hacker" spirit) -- if your dream is to launch the fastest-growing B2B SaaS product ever, we want to talk with you.
- MacOS (Swift, Objective-C, C/C++, etc) at the Senior and Staff levels. iOS experience is OK.
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr (not all positions posted there yet)
We are an early-stage, well-funded, stealth startup making humans and computers work together more efficiently. Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- MacOS (Swift, Objective-C, C/C++, etc) at the Senior and Staff levels
- Applied ML (including LLM fine-tuning and various vision and NLP tasks)
- Marketing (in the "growth hacker" spirit)
We're distributed but expect travel for regularly scheduled on-site, in-person work in SLC, with future presence in New York City.
Email jobs@avy.ai or visit https://avy.breezy.hr
We are an early-stage, stealth startup making humans and computers work together more efficiently.
Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Design (UI/UX)
- MacOS (Swift, Objective-C, C/C++, etc)
- Applied ML (including LLM fine-tuning, various NLP tasks)
- Marketing (in the "growth hacker" spirit)
We're distributed but expect travel for regular on-site, in-person work in SLC, with future presence in New York City.
jobs@avy.ai
We are an early-stage, stealth startup making humans and computers work together more efficiently.
Experienced team from Apple AIML, Bose, Amazon, and other great companies.
We're hiring for:
- Design (UI/UX)
- MacOS (Swift, Objective-C, C/C++, etc)
- Applied ML (including LLM fine-tuning, various NLP tasks)
- Part-time/contract roles for TypeScript and Python, and server work (Go)
- Marketing (in the "growth hacker" spirit)
We're distributed but expect travel for regular on-site, in-person work in SLC, with future presence in New York City.
jobs@avy.ai
MedCram has some easily digestible reviews on the topic:
- The ability for light to influence glucose metabolism: https://www.youtube.com/watch?v=6Win49aeh8A
- Light's effect on immune response and other processes: https://www.youtube.com/watch?v=5YV_iKnzDRg
But even at 100K, you do eventually run out of context. You would with 1M tokens too. 100K tokens is the new 64K of RAM, you're going to end up wanting more.
So techniques like RAG that others have mentioned are necessary in the end at some point, at least with models that look like they do today.
The computing power we're requiring is simply what's available in any M1/M2 Mac, and the resource usage for the indexing and search is negligible. This isn't even a hard requirement, any modern PC could index all your emails and do the local hybrid search part.
Running the local LM is what requires more resources, but as this project shows it's absolutely possible.
Of course getting it to work *well* for certain use cases is still hard. Simply searching for close sections of papers and injecting them into the prompt as others have mentioned doesn't always provide enough context for the LM to give a good answer. Local LMs aren't great at reasoning over large amounts of data yet, but getting better every week so it's just a matter of time.
(If you're curious my email is in my profile)
Here's another related one: https://www.forbes.com/sites/jackkelly/2022/02/17/new-york-c...
But there is direct evidence that politicians are influencing corporations to alter return to office policies:
- Mayor of SF asking businesses to pledge to implement RTO policies: https://sfist.com/2022/03/03/mayor-breed-would-like-you-back...
- Mayor of NYC basically doing the same: https://archive.is/si6xd
As far as I know they keep it as an interactive website so that PDFs are hard to make.
I haven’t found a good copy yet but it’s also possible that I’m just not as good at searching for pirated content as back in the old torrenting days (or perhaps just less motivated)
I suspect your intuition about moving emphasis from redaction to unified access control and audit logging over time is right.
The "AI Chief of Staff" sounds interesting though -- can you share a bit more about what you showed to companies and received lukewarm response to?
An alternative method is to index content in a database and then insert contextual hints into the LLM's prompt that give it extra information and detail with which to respond with an answer on-the-fly.
That database can use semantic similarity (ie via a vector database), keyword search, or other ranking methods to decide what context to inject into the prompt.
PrivateGPT is doing this method, reading files, extracting their content, splitting the documents into small-enough-to-fit-into-prompt bits, and then indexing into a database. Then, at query time, it inserts context into the LLM prompt
The repo uses LangChain as boilerplate but it's pretty easily to do manually or with other frameworks.
(PS if anyone wants this type of local LLM + document Q/A and agents, it's something I'm working on as supported product integrated into macOS, and using ggml; see profile)
There are open source models that are fine tuned for different tasks, and if you're able to pick a specific model for a specific use case you'll get better results.
---
For example, for chat there are models like `mpt-7b-chat` or `GPT4All-13B-snoozy` or `vicuna` that do okay for chat, but are not great at reasoning or code.
Other models are designed for just direct instruction following, but are worse at chat `mpt-7b-instruct`
Meanwhile, there are models designed for code completion like from replit and HuggingFace (`starcoder`) that do decently for programming but not other tasks.
---
For UI the easiest way to get a feel for quality of each of the models (or, chat models at least) is probably https://gpt4all.io/.
And as others have mentioned, for providing an API that's compatible with OpenAI, https://github.com/go-skynet/LocalAI seems to be the frontrunner at the moment.
---
For the project I'm working on (in bio) we're currently struggling with this problem too since we want a nice UI, good performance, and the ability for people to keep their data local.
So at least for the moment, there's no single drop-in replacement for all tasks. But things are changing every week and every day, and I believe that open-source and local can be competitive in the end.
But I think we've only scratched the surface as to what LLMs fine-tuned on specific tasks, especially for abstract reasoning over narrow domains, could do.
These applications possibly won't look anything like the chat interfaces that people are getting excited about now, and fine-tuning is not as accessible as prompt engineering. But there's a whole lot more to explore.
But that's the problem. Each incremental step towards more surveillance, less privacy, and more potential for government abuse is perfectly justifiable and seems reasonable.
But then temporary turns to permanent. And the "significant events" restriction gets dropped. And 450 cameras from local businesses turn into thousands from others, or Rings, or Teslas, or whatever.
And then manual monitoring by humans turns into AI-powered monitoring. And the looking at cameras gets combined with location data.
The point is, each step is reasonable. But who knows where it goes? We have no idea, nor do we have any idea who will be on the other side watching or what their agenda will be in 5 years, 20 years, or 100 years.
So it's important to stay vigilant of any incremental privacy incursion or expansion of government power. It doesn't mean saying No necessarily, but being aware and cautious.
1. A cross-application connectivity layer that pipes data and actions between apps
2. A natural language interface to control #1
Thinking about them separately is useful, because although chat is the new UI hotness, #1 is valuable on its own and the two can potentially be deployed separately.
As presented here, I suspect the natural language interface will be faster and easier than buttons for operating the cross-app layer for complex queries, but potentially slower than operating buttons for simple things (like "start dark mode").
But personally, I believe #1 combined with some AI context awareness is more powerful of the features.
...
And btw, I left Apple last year to build a local-first and developer-extensible assistants for the Mac that's pretty similar. If this interests you, would love to chat (email in profile, as well as a waitlist).
Last week they spun out Kustomer https://www.kustomer.com/blog/new-chapter-standalone-company...
For many modern (knowledge work) jobs, it's volume of tasks, not duration, that seems to induce burnout.
For instance, if you have 1 central task for the week--say, write a report--but there are ten subtasks (hold 5 meetings to prepare, read 3 background papers, ...), and then each of those have a bunch of subtasks (Slack each 5 meeting attendees 10 times to coordinate and schedule, get interrupted when reading each paper so have to resume X times each) ... this 1 central task can easily be dozens or hundreds of subtasks.
And then each coworker is doing the same thing, adding tasks to your load (you have to read their messages, respond to their emails, etc) in a multiplicative way. Newport calls it the "Hive Mind" in one of his books. The number of total tasks, from very small to very large, each individual has to accomplish in a week ends up far far greater than expected on the surface. And adding to that, they come in in an unpredictable way.
All this adds up to burnout, not just the number of hours. It's the intensity, and the unpredictability.
I've experienced this myself. I was at one of the FAANGs, constantly bombarded with new tasks, and felt burnt out. Now, I'm at a startup--very much inspired by Cal Newport, and using AI and context awareness to make teams operate together more effectively (see bio)--and I'm working far more hours per week than before, but with less interruptions and less distraction, I'm able to focus and feel far less burnout despite the increased hours.
All this to say, we really need to rethink how we work, not just how much.
The reasons go behind using data as training. Submissions to servers end up in logs, databases, temp files... who knows. And a company like Apple wants to not only ensure that the data is explicitly used by the receiving party, but also not inadvertently made accessible to others via poor security procedures, since their data is such a high value target.