dataqa

The open-source project is here: https://github.com/dataqa/dataqa. I have just released a feature which I have been working on for a while to solve a problem which I've seen a lot in industry: how to map entities found in text to large knowledge base ontologies.

There is a tutorial on the site where you use rules to extract mentions of side effects from forum posts: https://dataqa.ai/docs/tutorials/medical_side_effects/ner_me.... You can use this tool for NER.

Thanks for sharing! It looks very interesting. From a brief check, they do not seem to be UI-based like dataqa (although you can use it in a notebook), they do not offer a search engine and they are probably one level of abstraction below dataqa. You can do some of the stuff dataqa does but would need to code. Some of the rules offered by dataqa rely on complex operations with regular expressions, and are not so easy to program yourself.

Thanks for the tip! The tool can deal with larger documents, so you're right that using a folder would be better in that case.

I have seen first hand at small and large companies how problems have been tackled with ML without trying a simple rule or heuristic first. And then, further down the line, the system has been compared to a few business rules put together, to find that the difference in performance did not explain the deployment of an ML system in the first place.

It's true that if your rules grow in complexity, this might make it harder to maintain, but the good thing about rules is that they tend to be fully explainable, and they can be encoded by domain experts. So the maintenance of such a system does not need to be done exclusively by an ML engineer anymore.

Here is where I insert my plug: I have developed a tool to create rules to solve NLP problems: https://github.com/dataqa/dataqa

Hey HN!

After working in ML for more than a decade, I became frustrated over time with the lack of tools to create baselines using simple rules and heuristics. It is well known that most business problems can achieve decent baselines using only heuristics. So this is why I have just open-sourced DataQA, a rules-based labelling tool for NLP:

  - Quick labelling: You can create complex rules using regular expressions to help you label your text faster.
  - Search engine: DataQA also ships with a search engine (local elasticsearch database) so you can  search your documents.
  - Easy installation: Only need to install a single python package!
  - Easy use: upload your data as csv files.
  - Privacy: No data ever leaves your computer.

I'm hoping to get some feedback, and I'm open to hear about feature requests or ideas for extensions. I will be around to answer questions.