What is Apache DolphinScheduler?

Apache DolphinScheduler(incubator) — a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of the box.

Features:

• Associate the tasks according to the dependencies of the tasks in a DAG graph, which can visualize the running state of the task in real-time.
• Support various task types: Shell, MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Sub_Process, Procedure, etc.
• Support scheduling of workflows and dependencies, manual scheduling to pause/stop/recover task, support failure task retry/alarm, recover specified nodes from failure, kill task, etc.
• Support the priority of workflows & tasks, task failover, and task timeout alarm or failure.
• Support workflow global parameters and node customized parameter settings.
• Support online upload/download/management of resource files, etc. Support online file creation and editing.
• Support task log online viewing and scrolling and downloading, etc.
• Have implemented cluster HA, decentralize Master cluster and Worker cluster through Zookeeper.
• Support the viewing of Master/Worker CPU load, memory, and CPU usage metrics.
• Support displaying workflow history in tree/Gantt chart, as well as statistical analysis on the task status & process status in each workflow.
• Support back-filling data.
• Support multi-tenant.
• Support internationalization.
• More features waiting for you to explore…

What’s in Dolphin Scheduler

User Interface

home page

workflow pipeline

view task log online

datasources online management

multi-tenant management

tree view

Get Help

1. Submit an issue
2. Subscribe to the mail list: https://dolphinscheduler.apache.org/en-us/docs/development/subscribe.html