welanes

Yes - you're able to save data behind a login using the point and click functionality as it extracts whatever data is loaded in your browser ("local scraping").

And no - if you choose to also create a cloud recipe that runs on the server, the remote browser instance won't be able to access data behind a login.

It's possible but I'd rather not store third-party credentials for the time being.

Hi, it's both. There's 'local' scraping where the results of what you selected is ready to download as soon as you click results - no signup or server needed.

And then anything's that saved as a recipe runs in the cloud.

Hey, yes the guide still needs work. Here's what you gotta do:

- Click the pagination icon and then click the pagination element (usually 'Next' or an arrow). The icon will turn green

- Click 'view results' and then choose to save the recipe

- Select the number of pages you'd like to scrape

- Run your recipe and it will scrape those pages

Sure, in fact I'll do it this weekend.

Thanks! Yeah the checkmark confirmation just feels effortless. Haven't got it perfected yet, but soon.

Really appreciate the insights.

You're right that much depends on mapping the solution to a particular problem. Are you selling yet another scraping tool or are you freeing data to drive better decisions / save time / yada yada.

With the right frame, a sensible price point, and as much complexity abstracted away as is possible, there may exist a business model - seems to be many opportunities hiding in plain sight.

Will reach out soon for sure. Best of luck with Wanderlog

Good catch, uberswe. Was an older video and I flubbed the selection process - here it is working correctly: https://www.kapwing.com/videos/5dbc3e33ee4d0f00136d01e6

Yeah, that's offered although it's currently free.

No particular tricks to avoid detection. It's Puppeteer under the hood with a few customizations which works well on the majority of sites tested so far.

Given the cat-and-mouse game around web scraping you may never cover every website, and that's ok.

Hey HN, I posted this in a comment thread the other day and (to my surprise) it got a positive reception so added a few more updates and decided to post it proper.

The idea is to be able to choose a website, select the data you want, and make it available (as JSON, CSV or an API) with as little friction as possible.

Kimono was the gold standard for a while so did yoink some of their ideas, while doing some other things differently.

Still needs some work but as an MVP would appreciate any feedback. Cheers.

Works for me: https://www.kapwing.com/videos/5db9feb530c3d00013d47ded

Sure. I'm thinking a certain amount of cloud scraping / API request credits in the free tier and any number above that falls into a paid tier.

I must have looked at that homepage over 1000 times and still didn't catch that! Thank you, fixed it.

Thanks. I will for sure - although unless I'm doing something silly, don't see your email when I click through.

Not exactly passive, but side hustle that I hope becomes main hustle: https://simplescraper.io

Tryna fill the gap that Kimono left.

Aim to post as a Show HN once I polish it a little more.

> There’s clearly a point in the life of any company where you should move from the list you made in a spreadsheet to the richer tools you can make in coolproductivityapp.io

Now there's a snazzy domain name (that, alas, was already taken).

But I was able to yoink the almost as catchy http://coolproductivityapp.com.

Hey VCs, where you at? ;)

- Lanes (https://lanes.io)

.

Aesthetics mainly. Circles feel neater.

To help 'gamify' getting things done, I recently added the ability to set the timer in Lanes (https://lanes.io) to sync with any Youtube video.

So instead of tidying for 10 minutes, for example, you can tidy for the length of your favorite track - the timer will match the length of the video, the audio will play and the time will be logged. If you have routine tasks this turns out to be a pretty effective way of breezing through them.

How it looks: https://gfycat.com/HonoredHatefulIndianelephant

Haha, I'll have to A/B test this one :)

Thanks, I'll get on that.

This is really nice.

So much so that I yoinked one of your demos and used it on my landing page: https://lanes.io.

Thanks.

Beautiful website btw - clearly took a lot of effort.

From my experience Firebase are different. They're now a part of Google but support has been excellent anytime I needed it - both before and after they were acquired.

Try Lanes - https://lanes.io. It combines tasks, bookmarks and video annotations and uses the Mercury API under the hood.

Disclaimer: I made this. We've nailed down tasks and now we're focusing on making the read-later/bookmark feature top class. Articles are highlight-able and you can reference articles as tasks (because 'read this article' is a essentially a task) - we find this helps you read more, as most bookmark managers are essentially "here's a thing I like enough to save but will probably never read".

Give it a try.

Lanes (https://lanes.io) has highlights - although articles are secondary to tasks, for now. Diigo does highlights too, and Feedly.

Agree that highlights are a brilliant feature - the next natural step after saving an article is saving the best bits of that article.

Although primarily a task manager, Lanes - https://lanes.io - has Instapaper features, plus video bookmarks.

Hey HN. This is our newest release (étude) of Lanes - a task management app - which was first posted here two (yikes) years ago. It's been rebuilt with Vue.js and redesigned to focus on measuring productivity as well as getting things done. And it's faster, too.

In the words of Alibaba's Jack Ma:

“ If we go to work at 8am and go home at 5pm, this is not a high tech company and Alibaba will never be successful...If we are a good team and know what we want to do, one of us can defeat ten of them.”

It's even more incredible watching him say it: https://twitter.com/humanismusic/status/963714910269079552

I get it, and agree with everything you said.

My comment was mostly in jest.

> I knew that people’s credit cards failed and changed, but from my own anecdotal experience I considered this a rare edge case. Wrong.

This is one of the biggest surprises when you build SaaS. Delinquent customers average 25% of churn rates.

Banks must be rolling in lost/new card fees.

I tried a few charting libraries including Chart.js, Taucharts and D3.

Much of Taucharts is undocumented, Chart.js requires a lot of configuration for something like a mixed chart with stacked bars, and D3, yikes. Its API documentation is thicker than Ulysses.

Highcharts required minimal configuration, had an intuitive and clear API and - most importantly - handled all sorts of messy data I threw at it. It just worked, out-of-the-box.