HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by dgs_sgd | Hacker News Reader
Parent
Full thread
dgs_sgd
·
How long does it take to run your scraping pipeline?
View on HN
james_dev_123
·
End-to-end it takes ~10 hours. The biggest bottleneck is the actual scraping itself. The GPT part is relatively quick.
Reply on news.ycombinator.com