jacobobryant

I'm a programmer. I use a deep neural net to prioritize my todos automatically:

1. Don't write anything down.

2. Your brain will automagically forget all the todos that aren't important.

Not for everyone of course, but for me it works far better than any of my previous methods.

I'd be interested to hear your thoughts on this: https://findka.com/blog/essays/ (a project idea I'm thinking about doing). I wonder if preference for essays/evergreen content (as opposed to news articles) would be less affected by day-to-day context.

> Why would they recommend this book?

If they're using collaborative filtering, there probably isn't a simple explanation. Basically, you feed in a list of "<user id>, <book id>, <rating>" and the algorithm generates a feature vector for each book and each user. So the reason some book gets recommended is because... the dot product of that book's vector with your vector was high. You can do this easily with off-the-shelf libraries, like Surprise[1] (I'm using that lib in my startup[2]).

At least this is what happens in matrix factorization methods. Recommendations from k-nearest neighbor methods can be explained more easily, but knn doesn't scale as well.

This is a minor drawback of matrix factorization--people have been shown to trust recommendations more if they understand how the recommendations were generated. Twitter recently published a super interesting paper[3] about how they generate recommendations at scale, and as a bonus their method is explainable. They create a model that describes the communities in Twitter--groups of people who follow the same set of influencers. Users and items are represented by vectors, where each element of a vector describes to what degree a user/item fits in a certain community. When generating recommendations, you can get the user's top communities and then fetch items for those communities. If you generated a text description for each community, you could include that with each recommendation.

[1] https://github.com/NicolasHug/Surprise

[2] https://findka.com/

[3] https://www.kdd.org/kdd2020/accepted-papers/view/simclusters...

> A great example of curation in the dev space is awesome lists. If someone could make a collaborative platform for awesome-lists for everything I believe that could be the foundation of a new type of curation powered search engine.

Hey, I am working on basically this. E.g.: https://findka.com/u/jobryant (warning, takes ~10 seconds to load fully). You make a profile, add your top article/book/movie/music/etc recommendations, then you get (1) a feed of recommendations from people you follow, (2) an Explore page that gives you algorithmic recommendations (collaborative filtering, currently via an off-the-shelf SVD library).

Right now the curation is a little basic, but today I'm adding filter controls to profiles so you can see a person's recommendations for a specific content type. Eventually, I'd also like to add custom tags so you can have recommendation/awesome lists for anything.

Thanks! I haven't used a plain JS backend with it myself, but I think that would work well (the majority of Biff is backend code anyway). I haven't exposed a JS api, but doing so should be pretty easy. All of Biff's frontend code is in the biff.client namespace, and the only function you need is biff.client/init-sub [1], which takes:

- an atom, the contents of which describe what data you're subscribing to

- another atom, which is populated with the results of your subscriptions

- a function for handling server-sent websocket events (if your app uses any)

and it returns a function for sending websocket events to the server (which is used for sending transactions + any custom events you define).

[1] https://github.com/jacobobryant/biff/blob/98ad188924bee9121d...

1. You have to register specs for each kind of document. For example, from the frontend you can subscribe to a user document of a given ID with `{:table :users :id #uuid "some-uuid"}`. The backend will: (1) look up the document with the given ID, (2) verify that you've registered a spec for the `:users` "table" and verify that the document meets that spec, (3) run the authorization rule to make sure the client has access to that user document.

2. It is inherently difficult. The closest thing like this that exists for datalog is I believe clj-3df[1]. Hence I was very excited when Materialize was launched publicly a few months ago. A version for datalog would be cool too, but SQL is good enough I think.

[1] https://github.com/sixthnormal/clj-3df

Are you concerned about that for performance reasons or for ease-of-use reasons? To address the latter, I made a separate transaction format for Biff that allows merging (it's similar to Firebase's transaction format). So e.g.

  {[:users #uuid "some-user-uuid"] {:db/merge true
                                    :username "foo"}}

gets converted to

  [[:crux.tx/put (merge (crux/entity db #uuid "some-user-uuid")
                        {:username "foo"})]]

There is a race condition though; if another tx updated that document after the crux/entity call but before `:username "foo"` was written, then that tx would get clobbered. I'm planning to add `:crux.tx/match` operations automatically to prevent that from happening. There was talk on #crux in Clojurians slack today of using the newly-added transaction function feature to do merges. Haven't looked into that myself.

Yeah, I think Prismatic's end goal was pretty much the exact same as mine. (I never used Prismatic while it was running, but a while ago someone else mentioned that Findka seemed similar).

Thanks! I'm also looking forward to finishing the Materialize[1] integration I've been working on (currently blocked while I wait for them to release some features). Although Crux has datalog, you can't subscribe to datalog queries. But with Materialize, you can basically subscribe to arbitrary SQL queries. I have a branch that lets you define the SQL queries on the backend and then subscribe to the results via Biff's existing subscription system.

[1] https://materialize.io

Crux was a pretty straightforward choice. Didn't really consider any non-datalog DBs. I've used Datomic quite a bit, but preferred crux since it's open source, has filesystem persistence and doesn't require a separate process for the transactor (nothing wrong with Datomic's choices; the tradeoffs just make it unsuitable for Biff). I went with crux over datahike since it's more mature and has postgres persistence (and Kafka persistence, not that I need that yet). I haven't actually used the bihemporality features of crux yet, but maybe they'll come in handy.

Hi, Biff author here. There's background info + comparison to Firebase in the link, but to briefly summarize:

I've been doing Clojure web dev for a while but hadn't found a setup that I really liked yet. My latest approach was using Firebase + ClojureScript (for my startup[1]) which was really nice but had some significant limitations for my use case (mostly related to the ephemeral Node backend). At one point I realized that I could provide Firebase's query subscription feature using Crux[2] instead without too much effort. So I started working on Biff, and I've been using it in production for 2.5 months now. It's been a joy to use.

Also, I do think that web dev in Clojure can be a little hard to get into, due to the (appropriate) community preference for libraries over frameworks. I think there is a need for more frameworks that curate a set of libraries for you. I'd love it if Biff eventually became a sort of "Rails for Clojure" that would let anyone get up to speed quickly but would allow you to switch out the components easily as you go on.

(About the name: I named it after Biff Tannen from Back to the Future).

[1] https://findka.com

[2] https://opencrux.com

I quit my job at 23 to become a startup founder (18 months ago). Quitting wasn't too difficult for me. I started the job with the expectation that I'd work for a year and then do the startup. I had been spending my free time working on my idea(s) for a long time and was excited to go full time on it (I also didn't get much fulfillment from my job).

I was surprised to discover, long after quitting, that working full-time on your own ideas can come with it's own distraction: pressure to make money. Probably the biggest mistake I made from months 6 to 12 (startup idea #2) was being too concerned about structuring things in a way that would (hopefully) make it easier to charge money. I ended up spending a lot of time doing things that in hindsight weren't generating any value. It would've been much better to focus on doing meaningful things, whatever those may be.

I don't know the details of your situation, but that might be something to consider: if you can figure out a way to have a decent amount of time to work on whatever you want while still having an income from elsewhere, it might actually be easier to focus than if you were doing the idea full-time.

On the other hand, I am glad I quit myself. So maybe that advice is just BS.

For income: I might switch the "weeks/months" estimate to years ;). Running out of money hasn't been an issue (lots of savings from my job, my wife worked until recently, then we moved back in with my parents until the pandemic winds down). Opportunity cost of course is large, but that's fairly easy to ignore. I've learned an incredible amount during the past 18 months, so even if I can't get my startup to work, it'll have been a worthwhile investment.

I don't know if any of that is helpful. I'd be happy to chat if you like, my email's in my profile.

I don't think it'd make any difference. In the list you linked, only a few of those provide log-in with oauth. Letting people log in with Google won't let us import data from e.g. Letterboxd etc. Even if Letterboxd provided log-in with oauth, you'd still have to go through another authorization flow if you wanted to import data from an additional service.

The algorithm uses a simple item-based neighborhood model. i.e. if you like song A, Findka looks up all the content liked by other users who liked song A and probabilistically chooses an item that was well-liked. To help the algorithm keep learning, 35% of the recommendations are purely random ("epsilon-greedy"). I describe the implementation here[0], though it's changed slightly (now I export the database every day or so and generate a model on my laptop, then I load it into memory on the server). I experimented with a machine learning-based model (a latent factor model) last week, but it seems I don't yet have enough rating data for that to be useful.

[0] https://findka.com/blog/rec-sys-in-30-lines/

Ah, glad to hear it's working now. I was just about to ask you to check the network tab. Also, I pushed an update last night that renames the refresh button to "skip all" and keeps track of which items are skipped, so items won't show up again after you've seen them once (there's another button under Account which will reset the skip history).

Also you're welcome! It's been really nice to actually have some users, a luxury which escaped me in previous startup attempts.

Thanks for the feedback.

- I'll add an option for this.

- What songs were you searching for? The search feature uses Last.fm for music. They've had pretty much everything I've ever searched for, but maybe they're missing certain categories. I might investigate other/additional search APIs in the future. In the mean time, adding songs via URL is a decent option. The way the algorithm is implemented currently, it won't make a difference, and eventually I'm planning to add a cron job that'll go through the URL items and classify them as the correct content type and fetch additional metadata (with manual intervention as needed).

- I started working on this today; should be fixed tonight or tomorrow.

Both of those things are in fact implemented :). For seeding items, go to the "Add" page and there's a search bar (on desktop, it's in the sidebar; on mobile, you have to open the hamburger menu). I'll probably move the search bar to the main page to make it more discoverable.[1]

When you rate an item, are you sure all three items get replaced? If you rate one item, the ones below it should be moved up, and then the item on the bottom will be new. So if you rate the top item, you'll see a visual change in all items, but the other two items will still be there. Perhaps I should replace rated items in place without moving the others.

[1] https://news.ycombinator.com/item?id=23542880

I've thought about how this relates to Stumbleupon as well. I never used it myself, but it came up while I was looking for things similar to Findka.

I'm actually just starting to implement features like these. If you make an account then go the the Account tab, you can enable a public profile like this[1]. So far I'm planning to add follow/subscribe, commentary (i.e. comment on items instead of just rating them), and showing users who've rated the same items as you.

[1] https://findka.com/u/f962c926-43e1-406d-825d-3d7a4880befa/

Thanks, I'm interested in all these ideas. I'm working on figuring out a marketing channel at the moment, and then once Findka is growing consistently I'd like to focus primarily on recommendation quality and experimenting with ideas like these. (If things are going really well, hopefully I can hire a couple people to continue working on marketing/UX etc while I work on the algorithm).

I haven't thought too deeply on it yet, but:

- ram usage today has gone from ~1.1GB to ~1.6GB

- DigitalOcean prices ram linearly at $5/GB (I'm currently on a $10/month droplet)

Today is a big spike obviously (the number of thumbs up/thumbs down events have gone from 3K to almost 10K today, vs going from 0 to 3K since February). But say I continue to grow ram usage at 0.5GB per week (growth hopefully won't be linear, but I'd say that's a pretty steep linear growth rate for now). That means my hosting costs would increase at a rate of $10/month which is not bad at all.

That's probably the crappiest scale estimation ever made ha ha, but I believe at least that I should have plenty of time to figure out a reliable marketing channel before I do any major re-architecting. Maybe some time I'll do a stress test on my laptop to get a better sense of how RAM usage increases with additional data.

I've been at 10s of weekly active users. I guess this should handle 100s just fine, but maybe not 1000s.

Yeah. I think of recommender systems as one level of abstraction/automation above search engines: the only difference is that with recommender systems, you don't have to type in a search query.

I did this in a previous version of Findka. I had integrations for Last.fm, Spotify, Youtube, Goodreads, Pocket, RSS and a couple others I think. I did a "reset" on Findka in February, removing most of the features in order to make it simpler. Eventually I would like to add integrations back in. I'm working on some social networking features right now--hopefully that'll work as a reliable user acquisition channel. Once I've figured out a way to keep getting new users, I'll be able to spend more time on things like integrations and improving the algorithm.

Also I hadn't heard of letterboxd or RYM. I'll check those out more when the time comes.

That being said, music etc. is at least correlated with demographics. I could see it being useful to a degree for recommending other content types.

I'd thought about putting the search bar on the main page but wasn't sure how to make the UX good. It never occurred to me to just switch the page. I'll probably do that, thanks.

Definitely. Gnoosic in particular was an inspiration for the current iteration of Findka (https://www.gnoosic.com/).

I thought about calling it "search" but went for "add" since that is technically more accurate with the URL inputs--and it's already a fairly common occurrence for people to enter search terms in those instead of URLs. Though your comment makes me thing that maybe I should just call it "search"...

I've wondered about this a lot. It seems to be hit or miss--some people love e.g. Spotify's discover weekly/daily mix playlists, and some people hate them. I suspect that some users are outliers and it's hard to give them recommendations, and maybe tuning the system for them decreases the performance for other users. Also there may be limited business value in optimizing for those users anyway vs. pushing popular music.

There is such a search box. On desktop, click on "Add" in the sidebar. On mobile, you'll have to open the hamburger menu. There's also URL input for articles/content types I haven't added search for.

I'd love to chat more about music etc. You can shoot me an email at hn at jacobobryant.com. Definitely a pain point I feel as well, though another reason I switched to Findka is because I figured it would be a lot easier to bootstrap it. The music industry is pretty tough to get started in.

As far as the current music recommendations; it's nothing intelligent. Just pure collaborative filtering, i.e. it has no understanding of "this is a rock song" or "this song is from 2005" or even "this is a song". Right now, the algorithm just sees a bunch of URLs and their rating data. I am interested in making the algorithm more intelligent over time though.

Could the site not resolving be an issue with your network? It Works For Me, and I'm getting plenty of traffic right now.

I've thought a fair amount about inferring preferences (i.e. explicit vs. implicit feedback). Particularly for music, tracking skip data and feeding recommendations to the music player would make it a lot more convenient. My plan is to integrate more deeply with specific content types as time goes on/after the business is working.

Thanks for the heads up. Turns out the default url that google books returns often gives a 404. I fixed that last night but haven't updated all the previously imported books yet.