Revolutionizing enterprise-grade workflow orchestration! Apache DolphinScheduler now has generalized OIDC authentication. Simplify user management, enhance security & integrate seamlessly with any identity provider. Check out the details.
Last week, the tech world was shaken by a headline-grabber: Databricks has surged past the $100 billion valuation mark.
And remember, this isn’t a consumer internet giant, nor is it SpaceX—it’s a software company that builds the data and AI infrastructure layer. In a capital market environment dominated by caution, Databricks has managed to defy gravity, pushing the ceiling of what a unicorn can be worth.
The obvious question is: why? What exactly gives Databricks the right to this number?
I did some digging. Let’s look at why Databricks is valued at $100 billion—and more importantly, what software entrepreneurs might take away from it.
https://dev.to/chen_debra_3060b21d12b1b0/why-databricks-is-w...
Great news! Apache SeaTunnel has surpassed 8,500 #GitHub Stars!
A big shout-out to our amazing community for the continued support and contributions.
Join us on our journey to make data synchronization easier and more powerful! #OpenSource #BigData
Just extracted a lightweight, delay-capable EventBus from Apache DolphinScheduler!
Supports delayed & event-driven execution
Inspired by DolphinScheduler's elegant design
Easy to plug into any Java system
Dive in
Experiencing CPU spikes with Apache DolphinScheduler? One team solved it instantly by tweaking memory & thread settings — no more crashes, even at peak load! #DolphinScheduler #BigDataOps #OpenSource #DevOpsTips
Join #GSoC2025 with DolphinScheduler!
Enhance OIDC authentication for a leading workflow orchestrator. Open to students&devs!
Proposals: https://issues.apache.org/jira/browse/GSOC-284
Mentors: gallardot@apache.org / zihaoxiang@apache.org
Just dropped: Deep dive into Apache DolphinScheduler's Master Module! Discover how it handles billion-level tasks with slot calculation, command processing, and DAG workflows.
@NetEaseMail slashes dev time by 67% with #DolphinScheduler! Handling 50k+ daily tasks with optimized workflows & 1-click recovery. Now eyeing AI-powered data governance & Ops automation. #BigData #DataEngineering
Smooth migration guide: https://medium.com/@ApacheDolphinScheduler/netease-mails-mig...
Once upon a time, when people in the tech circle mentioned DS, what came to mind was a lively “little dolphin” - Apache DolphinScheduler (abbreviated as DS). This distributed task scheduling system, born in 2019, was once the “dream job” for big data engineers with its visual DAG interface, multi-tenancy support, and deep integration with the Hadoop/Spark ecosystem. However, the wheels of fate began to spin rapidly in early 2025: Hangzhou AI company DeepSeek (Deep Quest) burst onto the scene with its inference model R1 and open-source large model V3, touting “low cost and high performance.” Not only did it sweep the capital market, but it also redefined the acronym “DS” overnight - from now on, “DS” was no longer a scheduling tool but the “new darling” of the AI circle.
The software landscape is transforming!Affordable large language models, like #DeepSeekR1, are shaking things up. The future belongs to the monkey-troop-like collaborative software ecosystem.
Intriguing insights on data workflow management! Dive into the detailed comparison of #DolphinScheduler and #SeaTunnel against #AirFlow and #NiFi. Find out the best fit for your big data challenges. Full article: https://dev.to/chen_debra_3060b21d12b1b0/dolphinscheduler-an... #DataEngineering #opensource
Join the Apache DolphinScheduler community for an online meetup on December 17th, 2024, at 2:00 PM (GMT+8), as we explore Big Data Scheduling Technology across traditional and cutting-edge applications! Register now: https://www.slidestalk.com/DolphinScheduler/12_17 #Dolphinscheduler #BigDataScheduling #Meetup
Delve into Quartz, the mighty open-source Java framework for scheduling tasks, and its dynamic partnership with DolphinScheduler through QuartzExecutorImpl. Find out how they work together to orchestrate workflows and manage timings in our in-depth exploration.
Dive into integrating Arthas with DolphinScheduler to monitor specific API calls and analyze performance data. This setup enhances task scheduling reliability and maintainability—take your workflows to the next level! #DolphinScheduler #Arthas #PerformanceMonitoring
After several months, #ApacheDolphinScheduler is back on the GitHub Trending weekly list, now reaching 12,820 stars! Big thanks to the community for all the support and contributions!
Creating a tenant in #ApacheDolphinScheduler is key for managing resources and permissions, but sometimes it can fail. Check out this article for an analysis of common causes and detailed solutions to fix the issue!
In ApacheDolphinScheduler, tracking YARN task status can be tricky, especially in Flink's Stream Application mode where the client exits early, leaving tasks still running on YARN. This article provides a solution to address this issue and improve task status tracking. Check it out!
Discover how Apache DolphinScheduler's fault tolerance mechanism ensures high availability and reliability in workflow scheduling. Dive into this in-depth analysis for insights on handling task failures and maintaining system stability! #ApacheDolphinScheduler #FaultTolerance #OpenSource https://dev.to/chen_debra_3060b21d12b1b0/in-depth-analysis-o...
Streamline your data integration with the official Apache SeaTunnel Docker image! It simplifies deployment and boosts efficiency for SeaTunnel users. Learn more here #ApacheSeaTunnel #Docker #DataIntegration
https://seatunnel.apache.org/docs/2.3.8/start-v2/docker/
This document summarizes common issues and the solutions encountered while using Apache DolphinScheduler, including log garbling, excessive worker logs causing disk warnings, and incompatibility issues due to version updates. Let’s see if any of these solutions address your concerns!
In big data analysis and processing, DolphinScheduler is an open-source distributed workflow scheduling system used to schedule and manage complex tasks. This article will introduce how to use the URL class in Java to call DolphinScheduler’s API to start workflow tasks.
DolphinScheduler provides powerful workflow management and scheduling capabilities for data engineers by simplifying complex task dependencies. In version 3.2.0, DolphinScheduler introduces a series of new features and improvements, significantly enhancing its stability and availability in production environments.
To help readers better understand and apply this version, we have carefully prepared this DolphinScheduler 3.2.0 Production Cluster High Availability Setup Guide, delving into how to set up a high-available DolphinScheduler cluster in a production environment, including but not limited to environment preparation, database configuration, user permission settings, SSH keyless login configuration, ZooKeeper startup, and service startup and shutdown procedures.