DB-EnginesExtremeDB: the first and only COTS deterministic embedded database management system for mission- and safety-critical hard real-time applicationsEnglish
Deutsch
Knowledge Base of Relational and NoSQL Database Management Systemsprovided by solid IT

DBMS > DuckDB vs. Spark SQL

System Properties Comparison DuckDB vs. Spark SQL

Please select another system to include it in the comparison.

Our visitors often compare DuckDB and Spark SQL with ClickHouse, MySQL and SQLite.

Editorial information provided by DB-Engines
NameDuckDB  Xexclude from comparisonSpark SQL  Xexclude from comparison
DescriptionAn embeddable, in-process, column-oriented SQL OLAP RDBMSSpark SQL is a component on top of 'Spark Core' for structured data processing
Primary database modelRelational DBMSRelational DBMS
DB-Engines Ranking infomeasures the popularity of database management systemsranking trend
Trend Chart
Score0.23
Rank#282  Overall
#131  Relational DBMS
Score22.94
Rank#34  Overall
#20  Relational DBMS
Websiteduckdb.orgspark.apache.org/­sql
Technical documentationduckdb.org/­docsspark.apache.org/­docs/­latest/­sql-programming-guide.html
DeveloperApache Software Foundation
Initial release20182014
Current release0.2, August 20203.2.0, October 2021
License infoCommercial or Open SourceOpen Source infoMIT LicenseOpen Source infoApache 2.0
Cloud-based only infoOnly available as a cloud servicenono
DBaaS offerings (sponsored links) infoDatabase as a Service

Providers of DBaaS offerings, please contact us to be listed.
Implementation languageC++Scala
Server operating systemsserver-lessLinux
OS X
Windows
Data schemeyesyes
Typing infopredefined data types such as float or dateyesyes
XML support infoSome form of processing data in XML format, e.g. support for XML data structures, and/or support for XPath, XQuery or XSLT.nono
Secondary indexesyesno
SQL infoSupport of SQLyesSQL-like DML and DDL statements
APIs and other access methodsCLI Client
JDBC
JDBC
ODBC
Supported programming languagesC
C++
Go info3rd party driver
Java
Python
R
Ruby info3rd party driver
Java
Python
R
Scala
Server-side scripts infoStored proceduresnono
Triggersnono
Partitioning methods infoMethods for storing different data on different nodesnoneyes, utilizing Spark Core
Replication methods infoMethods for redundantly storing data on multiple nodesnonenone
MapReduce infoOffers an API for user-defined Map/Reduce methodsno
Consistency concepts infoMethods to ensure consistency in a distributed systemImmediate Consistency
Foreign keys infoReferential integritynono
Transaction concepts infoSupport to ensure data integrity after non-atomic manipulations of dataACIDno
Concurrency infoSupport for concurrent manipulation of datayes, multi-version concurrency control (MVCC)yes
Durability infoSupport for making data persistentyesyes
In-memory capabilities infoIs there an option to define some or all structures to be held in-memory only.yesno
User concepts infoAccess controlnono

More information provided by the system vendor

We invite representatives of system vendors to contact us for updating and extending the system information,
and for displaying vendor-provided information such as key customers, competitive advantages and market metrics.

Related products and services
3rd partiesSQLFlow: Provides a visual representation of the overall flow of data. Automated SQL data lineage analysis across Databases, ETL, Business Intelligence, Cloud and Hadoop environments by parsing SQL Script and stored procedure.
» more

We invite representatives of vendors of related products to contact us for presenting information about their offerings here.

More resources
DuckDBSpark SQL
Recent citations in the news

New CWI spin-off company DuckDB Labs: Solutions for fast database analytics
14 July 2021, Centrum Wiskunde & Informatica (CWI)

Apache Arrow 6 Improves Support For R and Rust
22 November 2021, iProgrammer

Apache Arrow 5 Improves Asynchronous Scanner
16 August 2021, iProgrammer

4 predictions for open source in 2021
29 December 2020, IDG Connect

14 Open Source SQL Parsers
8 October 2021, hackernoon.com

provided by Google News

Databricks' TPC-DS benchmarks fuel analytics platform wars
24 January 2022, ZDNet

Spark Gets Closer Hooks to Pandas, SQL with Version 3.2
26 October 2021, Datanami

Exploring the use of the Python programming language for data engineers
5 January 2022, Dataconomy

Top 10 Big Data Analytics Courses to Elevate Your Career in 2022
12 January 2022, Analytics Insight

Databricks SQL Now GA, Bringing Traditional BI to the Lakehouse
16 December 2021, Datanami

provided by Google News

Job opportunities

Senior Data Engineer
Solidigm, Folsom, CA

Machine Learning Engineer
7-Eleven, Inc., Irving, TX

Data Warehouse Architect
J.Crew, New York, NY

Backline Spark TSE
Databricks, Remote

Senior Engineer - Hadoop/Spark
Amex, Phoenix, AZ

Data Engineer
Valorem, Remote

jobs by Indeed



Share this page

Featured Products

Datastax Astra logo

Build cloud-native apps fast with Astra, the open-source, multi-cloud stack for
modern data apps.
Get started with 80GB free.

Neo4j logo

See for yourself how a graph database can make your life easier.
Use Neo4j online for free.

Vertica logo

Vertica Accelerator. The fastest analytics and machine learning, delivered as SaaS, with automated setup, administration, and management. Free trial.

Couchbase logo

The modern database for enterprise applications. Build fast. Scale Big. Save more.
Get started today.

MariaDB logo

SkySQL, the ultimate
MariaDB cloud, is here.

Get started with SkySQL today!

Present your product here