neiman1

Hey HN, here's a browser extension for a concept I started tinkering with quite some time ago! It aims to improve the experience of reading news articles by providing you with interesting, real-time insights into each article.

It's pretty hacky (doesn't work perfectly for all sites & the metrics aren't super accurate), but I hope you make some interesting discoveries nonetheless!

Have you tried https://www.thread.com/? That's the closest thing I've seen to what you're looking for.

Love the concept and potential impact of this. Any chance I could email you with a few questions to find out a bit more about your experiences & what you'd be looking for in such a tool? (If so, please ping me at samuel.jordan.dobbie@gmail.com).

Yes, <HN_USERNAME>, of course I am a human.

That must be super inconvenient! It's probably a long shot, but I wonder if reaching out to Apple's accessibility support [1] might bump up the priority of getting this resolved?

[1] https://support.apple.com/en-gb/accessibility

Sounds like an interesting project! How frequently do you find yourself using the Wayback Machine? Do you primarily use it for dead links?

What would you want a company to do in order to demonstrate the true expectations / realities of the job to you?

RE: fanatical fluff, very much agree, reading the 10th instance of "we like to do things a little different here" gets pretty tiresome. Role, location, salary range, responsibilities, and requirements would be more than enough.

Yeah, I've signed up to that myself, but there are likely folks on HN who haven't signed up to it who would still be interested in getting involved with a startup.

Great to hear! Mind if I ask what features need to be added / refined to make Markup a complete solution for your use case?

The basic HTML annotation is live :) Will be incorporating syntax highlighting & HTML rendering via URL soon.

I appreciate the insight, that's super helpful.

> team functionality with multiple user accounts

Mind if I ask what sort of team features you make use of with Prodigy? Are there any aspects you feel are lacking? Initial thoughts are that it'd be helpful for teams to be able to set group annotation goals, share docs / annotations / configs, view ongoing sessions, assign annotators to sessions, and view stats on each annotator (as per point 5).

> The software should make sure that a text is never shown to more than 2 annotators and never shown to the same annotator twice

For this I plan to let teams set the threshold for the number of documents that should overlap and the number of annotators a text should be shown to. In some situations it could be useful for there to be some % of overlap for all annotators to help determine the inter-annotator agreement across the entire team.

> The workflow for annotators has to be perfected first

Totally agree. My biggest concern is building out the above on top of an inefficient workflow. That's one of the primary driving forces behind the current re-write of the tool.

Love the smart flagging, mass-edit, and integrated provider ideas!

The primary competitor would be Prodigy IMO, although that's not FOSS. Several open-source alternatives look interesting, such as Docanno, Brat, and Webanno.

In terms of what makes Markup better, I'd say it's mainly that:

- It's quick and easy to annotate with because it suggests complex annotations to you and will learn based on your feedback. Plus it allows for integrated querying of external ontologies, so if you highlight a span of text then it'll suggest matches from the ontology, ordered by similarity.

- It's flexible and configurable. For example, you'll soon be able to provide a small number of example sentences which will generate synthetic data to train a model for you (without any code), and those model weights can be saved for future usage.

- It includes a growing selection of in-built tools, such as a tool for statistically comparing two sets of annotations / annotators.

- It's possible to capture both simple and complex annotations. Most existing tools focus on classification or named-entity recognition, without giving users' the option of capturing attributes nor allowing them to annotate the same span of text repeatedly. Being able to do so enables the production of datasets for more complex use cases.

For sure! I've just tested it locally and HTML annotation is possible with only a few minor changes. I've just been overly restrictive by limiting it to text files (primarily to avoid PDFs and MS docs that have some additional challenges). Will deploy the updates later today.

Love the idea of being able to enter a URL to retrieve the HTML for annotation.

Thank you!

Just to preface this summary, it's all a bit hacked together at the moment, and I'm in the process of rewriting the tool from scratch so this description is likely to change.

To generate the suggestions there's an active learner with an underlying random forest classifier, that has been fed ~60 seed sentences [1], to classify positive sentences (e.g. contains a prescription) and negative sentences (e.g. doesn't contain a prescription).

All positive sentences are fed into a sequence-to-sequence RNN, that has been trained on ~50k synthetic rows of data [2] which maps unstructured sentences (e.g. patient is on pheneturide 250mg twice a day) to a structured output with the desired features (e.g. name: pheneturide; dose: 250; unit: mg; frequency: 2). The synthetic data was generated using Markup's in-built data generator [3].

The outputs of the RNN are validated to ensure they meet the expected structure and are valid for the sentence (e.g. the predicted drug name must exist somewhere within the original sentence).

All non-junk predictions are shown to the user who can accept, edit, or reject each. Based on the users' response, the active learner is refined (currently nothing is fed back into the RNN).

[1] https://github.com/samueldobbie/markup/blob/master/data/text...

[2] https://raw.githubusercontent.com/samueldobbie/markup/master...

[3] https://www.getmarkup.com/tools/data-generator/

Thanks for your kind words! It's just a hobby project that I work on alongside my full-time job right now, and to be honest, I'm still trying to figure out a plan. My intention is to keep the core functionality free forever, but I could definitely see a future where there are premium collaborative features or some cost for training custom suggestion models, for example.

If you're one of those folks who would consider supporting a tool such as this, do you have an idea in mind as to what sort of features you'd be willing to pay for?

Much appreciated! That's true, and lots of the tools that do feature text annotation can be quite restrictive in that they don't allow you to add attributes / repeatedly annotate the same span of text.

Support for PDFs and other doc types is definitely on the backlog, but I keep holding off due to the challenges you mentioned.

Thanks a lot! I've just added an MIT license :)

Hey HN!

Markup is an open-source annotation tool for transforming unstructured documents into a structured format that can be used for ML, NLP, etc.

Markup learns as you annotate in order to speed up the process by suggesting complex annotations to you.

There are also a few different in-built tools, including:

- A data generator that helps you to produce synthetic data for training the suggestion model

- An annotator diff tool that helps you to compare annotations produced by multiple annotators

It's still very much a work in progress (and the documentation is severely lacking), but the ultimate goal is to make a tool that's as useful as https://prodi.gy/, without the $400 price tag.

> our Puritan values run deep

Not for me, I just hate the taste of alcohol, would much rather a can of coke.

I'm trying to build a pastebin for links - https://shelf.gg/

The Swamp is good fun - well done!

I can definitely see myself using it when creating an MVP, but longer-term I'd probably set one up myself.

In terms of the pricing, it may be worth considering a cap (e.g. 5% on successful payments up to a maximum of $Y a month), otherwise, you might find that your entire userbase ends up being comprised of early-stage projects that are making little to no money.