Hi David, this is Dylan, a co-founder. I'd love if you had the time to send me a screenshot of what you're seeing, with your Developer Tools Console open in Chrome, if you don't mind. It'd be a huge help!
dylan [at] pipedream [dot] com
50 karma · joined February 12, 2019
dylan [at] pipedream [dot] com
If you've used it and like it, I'd honestly love for you to give Pipedream a try and tell us where we can improve compared to your experience with other tools. We'd love the feedback!
dylan [at] pipedream [dot] com
I'm a Pythonista at heart and we're working on Python support as we speak! We'll let y'all know when we ship that.
Deploying workflows from source via CLI or your CI/CD pipeline is in the works! We want you to host workflows on Github / Gitlab and be able to deploy them using your standard process.
Feel free to reach out to me directly if you'd like more information and I can let you know when we're testing this.
dylan [at] pipedream [dot] com
We've found that for integrations or common automations, building on a cloud platform is too low-level. You've got to manage a number of services that aren't core to your use case (e.g. API Gateway, IAM, Cloudwatch Logs, etc.). We operate at a higher level and just give you an execution environment to run Node.js code or stitch together pre-built actions, similar to integration platforms like Zapier.
Of course, Serverless and other frameworks abstract part of the cloud services, but you're still running on that platform, and when you need to troubleshoot or scale the service, it's likely you'll have to peek under the covers. We tried to build a serverless platform from the ground up and provide services like the HTTP server, key-value store, logs, and other services right inside the UI, or accessible in code. There's no other cloud resources to manage or monitor.
This kind of abstraction isn't great for every application, but we think it shines when you're building integrations or data pipelines that aren't part of your core app.
The pre-built actions and built-in OAuth / key-based auth also operate at a higher level than most cloud services. We've tried to provide these in a developer-friendly way, so you have full control over how your workflow runs, but don't have to manage the parts that are tangential to the logic of your app.
As we expose more APIs for building and running workflows, we'd love to support a framework like Serverless.
I'd love if you have time to give Pipedream a try and see how it works for yourself! We're looking for feedback on how we can improve so we'd love more eyes on it.
We use Vuepress for the docs and it's awesome!
We're planning to share more about our stack in future engineering blogs, so keep an eye out for those!
The post is meant to articulate the depth of our vision for the platform. The hope was the linked workflows provide some inspiration and examples to make Pipedream a little more concrete. I'm glad the homepage resonated with you.
Did you get a chance to sign up and use the tool? We're looking for specific feedback from devs on how we can improve so I'd love if you had any time to build a workflow and share your thoughts.
Thank you for the kind words! We hope it's fun to program in, send us feedback or suggestions any time.
dylan [at] pipedream [dot] com
Yes! We do plan to support other languages, but we're prioritizing other parts of the platform first, e.g. getting the developer UX right, environment separation, the ability to connect to accounts via OAuth, etc.
We've had a lot of requests for Python support, and we would love to ship support for TypeScript, as well.
I appreciate the feedback. Please let me know if you have any more!
Which columns would you like to resize? The event inspector on the left, code steps, or all of the above?
Right now, the product is in alpha, so its programming model is subject to change. As a result we expect it to serve mostly personal use cases, and hope to serve the enterprise over time.
And yes, it's free! You can run any number of cron jobs up to a frequency of once / minute [1]. You can also trigger workflows via HTTP requests [2], up to 10 QPS for free (after that, we'll rate limit you and issue 429s, but you can always reach out to us to raise that limit if you have a use case).
We think it's the easiest way to run Node.js code on a schedule or in response to HTTP requests. There's no infra to manage or other cloud resources to provision. You just write Node code.
We support a lot of first-class "destinations", like S3 [3] and a hosted data warehouse where you can send any JSON and query it using SQL, also for free [4] (we generate and update the table schema for you, based on the JSON you send). We use these destinations internally to store e.g. data from webhooks for analytics. If you need to run basic transformations on HTTP requests using Node, then send that data somewhere, it's a trivial way to do it. You can add any number of these destinations to a single workflow. We also support returning custom HTTP responses if you'd like to build out simple APIs [5].
I'd love if you found a use case for it and gave us any technical feedback you have. That's exactly what we're looking for at this stage!
dylan [at] pipedream [dot] com
[1] https://docs.pipedream.com/cron/ [2] https://docs.pipedream.com/notebook/sources/#webhook-sources [3] https://docs.pipedream.com/notebook/destinations/s3/ [4] https://docs.pipedream.com/notebook/sql/ [5] https://docs.pipedream.com/notebook/sources/#http-responses
You can run the cron job up to once a minute for free, so it's pretty close to realtime :) .
The README explains how it works in detail, but you can modify the KEYWORDS constant at the top of the Node.js code step to receive alerts for any keywords you'd like. Then, on the schedule you specify (e.g. every 30 minutes), this hits the HN API and sends you a Slack message with the story link.
Let me know if y'all have any feedback on this, happy to modify it to suit specific needs. If you know Node, you can fork and run a copy yourself and modify it however you'd like, too.