Hacker News Reader

Comment by _puk | Hacker News Reader

I think you should probably mention that its source is available! [0]

I don't personally have a need for this right now, but I can really see the use for the parameterised queries, as well as comparisons across models.

Thanks for your efforts!

0: https://github.com/ianarawjo/ChainForge

View on HN

"source available" means one thing, but this is properly MIT open source and i believe the authors should receive credit for that in this age of frankenlicenses

EDIT: ah: "This work was partially funded by the NSF grant IIS-2107391." ok cool we the taxpayer funded it haha

I didn't want to open that can of worms: "If you use ChainForge for research purposes, or build upon the source code, we ask that you cite this project in any related publications. The BibTeX you can use for now is.."

That's outside of the MIT licence as far as I'm concerned

It says "we ask", not "you have to". I'd say it's open to interpretation that this is an informal request and not legally binding. Also I do want to open the can of worms and say that whoever doesn't even have the respect to include a citation on request when using someone else's work should just write everything themselves.

Hi! I’m one of the great students working on this. This is merely a request to get more visibility. It will also help us get more grants. We don’t have any intention of restricting the “openness” of it.

Awesome, thank you!

If I called it truly open source I half expected to get shot down.

I know where we stand now :)

Y'all sure are some great students! ;)

I think that's the wrong end of the stick. When you publish research, the software you used/built on is part of the methods and needs to be cited. The authors are doing you a courtesy by providing a pasteable citation.

Similar "we would appreciate citations" statement for (BSD-licensed) pandas: https://pandas.pydata.org/about/citing.html

8000+ pubs citing pandas: https://scholar.google.com/scholar?cites=9876954816936339312

It seems to be more powerful than langflow and flowise

https://github.com/logspace-ai/langflow

https://github.com/FlowiseAI/Flowise

k i need somebody to do a comparison table for us...

We will mostly do this comparison when we write our research paper. I can post it here when we do it.

I can't comment on the features, but ChainForge has some catching up to do...mind share wise. Below are some community insights for langflow, Flowise and ChainForge

https://devboard.gitsense.com/logspace-ai/langflow

https://devboard.gitsense.com/FlowiseAI/Flowise

https://devboard.gitsense.com/ianarawjo/ChainForge

Flowise currently has the largest active community (based on GitHub data)

Full Disclosure: This is my tool

back in finance they had a saying: traders know the price of everything but the value of nothing.

thats what its like to blindly compare tools by github numbers

I sort of know where you are coming from, but I think it is hard to discount the fact that 80 new people took the time to create 99 issues in the last 6 weeks in Flowise. No matter how you look at it, this is a strong indicator that this project has strong engagement.

I'm not saying ChainForge is bad, but it will need to go against that kind of community engagement that other projects with a head start have. However, if you believe people contributing code (26) and participating in non code activity (150) in Flowise in the last 6 weeks are just novelty metrics, then yes, comparing numbers is silly.

We don’t view it as a competition here. These other tools are for LLM app building, which is great, but isn’t the focus of ChainForge. Instead, we’re focused on helping devs find the right prompt, and evaluate and inspect LLM outputs. So while we might visually look similar, the goals are rather different.

Some CF users, for instance, might not be app builders at all —they just want to audit models.

I think both problems —prompt engineering and LLM app building —are hard, and deserve their own dedicated tools.

Yeah, I'd love to see these tools discussed in an upcoming Latent Space episode.

ChainForge looks similar visually, but is very different in practice. We target evaluating and inspecting LLM outputs, rather than building LLM applications. So, some things are certainly easier to do in CF, while others will certainly be easier in other tools that target LLM app building.

Reply on news.ycombinator.com