Hacker News Reader

How about deduplication. Also table / dictionary lookup . And string replacement and regex replacement .

Got it. If you'd like me to ping you once we have custom build hooks: aneesh at quiltdata dot io. Have you tried Luigi or Bonobo for data cleaning?

No I haven't I think I just did it myself I'll look into that .