qData Data Middle Platform is an open-source data middle platform designed for enterprise data governance and data development scenarios. Built around core capabilities such as ETL data integration, data development, data modeling, metadata management, data quality, data assets, API data services, and AI-powered data Q&A, it supports common databases including MySQL, DM8, Oracle, SQL Server, Kingbase8, and Doris. qData helps enterprises quickly complete data access, cleansing and transformation, asset cataloging, quality checks, API publishing, and Text2SQL analysis. It can serve as an open-source foundation for building enterprise data middle platforms, data governance platforms, ETL platforms, and data service platforms, and is also suitable for secondary development and feature extensions.
✨✨✨Online Documentation✨✨✨ https://community.qdata.tech
✨✨✨Open-Source Edition Demo✨✨✨ https://demo.qdata.tech, username: qData, password: qData123
✨✨✨Professional Edition Demo✨✨✨ https://pro-demo.qdata.tech. Please contact customer service to obtain a demo account.
If qData is helpful to you, please give us a Star ⭐️. It is the greatest motivation for us to keep improving! 🚀
👉 qData Data Middle Platform uses a modular design. The current open-source edition focuses on core capabilities including data integration, data development, data modeling, metadata, data quality, data assets, data services, and AI-powered data Q&A. For more details, see the qData Feature Overview.
| Module | Description |
|---|---|
| Data Integration (ETL) | Supports visual configuration of data ingestion, cleansing, transformation, and output workflows. New tasks can use either DataX or Spark as the execution engine, making them suitable for lightweight data synchronization, related-data retrieval, data import, large-scale offline processing, and complex computing scenarios. |
| Data Development | Supports data processing task development through SQL scripts. New tasks can choose Quartz or DolphinScheduler schedulers, suitable for data processing, statistical analysis, periodic scheduling, and task orchestration scenarios. |
| Data Modeling | Supports data standards, data warehouse layers, data domains, subject-area planning, logical models, standard data elements, and other capabilities to help enterprises establish a foundational data modeling framework. |
| Metadata Management | Supports metadata viewing, field structure viewing, version management, metadata comparison, and metadata collection task management. New collection tasks can choose Quartz or DolphinScheduler schedulers, making collection execution strategies more flexible. |
| Data Quality | Supports data quality inspection and processing based on audit rules, helping identify issues such as completeness, uniqueness, and validity. |
| Data Assets | Supports data asset cataloging, asset tags, asset details, asset search, and other capabilities to help users manage and retrieve data resources in a unified way. |
| Data Query | Supports online SQL queries against data sources for temporary queries, data validation, and result export. |
| Data Services | Supports encapsulating data tables or SQL query results as API services, with online testing, call logs, and application management capabilities. |
| AI-Powered Data Q&A | Supports natural-language data questions, Text2SQL, intelligent charts, and result detail viewing, lowering the barrier for business users to work with data. |
| Basic Management | Supports data sources, project workspaces, categories, audit rules, cleansing rules, and other basic configurations to support data development and data governance. |
| System Management | Supports users, roles, menus, departments, positions, dictionaries, parameters, announcements, logs, and other basic system management capabilities. |
Future development will focus on metadata comparison, business layering, and data asset refactoring to further improve the data governance and asset management experience. We will also continue enhancing data integration, data quality, data services, and AI capabilities, expanding data sources and ETL components, and optimizing quality rules, API services, and the Text2SQL analysis experience.
💡 If you have suggestions or feature requests, please submit an Issue and help us improve qData Data Middle Platform.
qData adopts a frontend-backend separated architecture. The backend is based on Spring Boot, the frontend is based on Vue 3, and the platform integrates several mainstream middleware and data tools.
| Category | Technology | Description |
|---|---|---|
| Backend Technology Stack | Spring Boot | Provides rapid development capabilities |
| Spring Security | Implements user authentication and permission control | |
| MySQL, PostgreSQL, DM8, KingbaseES | Persistent storage and configuration management | |
| MyBatis-Plus | Simplifies database operations | |
| Redis | Supports caching, distributed locks, and more | |
| RabbitMQ | Enables asynchronous communication and decoupled processing | |
| Frontend Technology Stack | Vue 3 | Modern reactive framework |
| Element UI | Common UI component support | |
| Vite | Fast development and build tool | |
| Third-Party Dependencies | DolphinScheduler | Provides visual task orchestration, dependency management, and scheduling capabilities, and can be used to schedule data development and collection tasks |
| Quartz | Provides lightweight task scheduling capabilities, can be used for data development task and metadata collection task scheduling, and is suitable for single-node deployment and simple periodic task scenarios | |
| DataX | Lightweight data synchronization execution engine that runs on a single node, is simple to deploy, and is suitable for related data acquisition and data import scenarios | |
| Spark | Distributed computing execution engine suitable for large-scale offline processing and complex computing tasks |
qData provides two editions: Professional Edition and Open-Source Edition, meeting user needs across different scales and scenarios. Each has its own focus while complementing the other: the Open-Source Edition helps users get started at low cost, while the Professional Edition provides greater depth and support. Whichever edition you choose, qData aims to be a reliable partner that helps enterprises unlock data value and accelerate digital transformation.
👉 For Open-Source Edition brand authorization or Professional Edition consultation, see 💼 Licensing Details.
If this is your first time using qData, we recommend exploring it in the following order:
- Online demo: Sign in to the demo environment to explore the main features without installing anything.
- Local deployment: Start the complete environment locally with Docker Compose.
- Run from source: Start the frontend and backend services from source when you need to customize or extend qData.
- Sign in to the Community Open-Source Edition demo to explore the main features without installing anything.
- Local deployment: Start the complete environment locally with Docker Compose.
- Run from source: Start the frontend and backend services from source when you need to customize or extend qData.
- Self-managed deployment: Manually install, configure, and manage each service; suitable for production environments, large-scale deployments, and custom configurations.
👉 Complete guide: https://community.qdata.tech/docs/deploy/deploy-open-source.html
Before deploying qData, make sure the following environments and tools are correctly installed:
| Environment | Item | Recommended Version | Description |
|---|---|---|---|
| Backend | JDK | 1.8 or above | OpenJDK 8 or 11 is recommended |
| Maven | 3.6+ | Project build and dependency management | |
| DM8 | 8.0 | Relational database (can be switched to MySQL) | |
| Redis | 5.0+ | Supports caching and messaging | |
| RabbitMQ | Optional | Used for task scheduling, asynchronous communication, and other features | |
| Operating System | Windows / Linux / Mac | Runs in all common environments | |
| Frontend | Node.js | 16+ | Build tool dependency |
| npm | 10+ | Package manager | |
| Vite | Latest | Scaffolding tool |
Welcome to join the official qData QQ community to get the latest updates, technical support, and usage discussions.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |