Hacker News Reader

Comment by psimm | Hacker News Reader

Very cool! I signed up and uploaded data for a text classifier. 3000 examples of social media posts on a binary annotation task. Got 91% initially, then looked through the annotations and corrected a few errors that had snuck in. The UI for that is great. That got it to 92%.

Easy to use UI, easy data upload and the training was quick. A great tool for testing new ideas for classifiers. For bigger projects I'd be concerned about long term cost with pay per invocation.

Is weak labeling via labeling functions (snorkel, skweak) something that's on the roadmap for Nyckel? Also, do you plan to add named entity recognition?

View on HN

Thanks you for the kind words and feedback! You basically went through most of the UI flow that we designed for. You're spot-on about testing new classifiers - answering the question "Can ML even help with my problem?" is much easier with Nyckel and prototyping and rapid iteration starts with that.

Our goal is to be cost-competitive, even for bigger projects. Given how early we are, our pricing structure is still being worked on, especially for high-volume.

Integrating with labeling solutions is in our roadmap. In the meantime, our API should enable any data/labeling integrations.

Named entity recognition is also in the roadmap. Would love to hear more about your use-case and we can give you access to the beta when ready.

Chiming in on the weak labeling question: As of right now, you can use outside libraries like skweak to create weak labels offline and then PUT those using our API (https://www.nyckel.com/docs#update-annotation). This wouldn't cost anything since we only charge for invokes, but it requires some work.

We may look at adding weak labeling as a first class feature of our site down the road, but we are not yet sure we need to. With the powerful semantic representations offered by the latest deep nets, we find that smaller number of hand-annotated samples often suffice for the desired accuracy which makes the whole annotation process simpler and faster. Of course, if you have data & evidence to the contrary, we'd love to take a look.

Reply on news.ycombinator.com