DBMS > Apache Spark (SQL) vs. Databricks vs. PostgreSQL

System Properties Comparison Apache Spark (SQL) vs. Databricks vs. PostgreSQL

Please select another system to include it in the comparison.

Editorial information provided by DB-Engines

Name

Description

Apache Spark SQL is a component on top of 'Spark Core' for structured data processing

The Databricks Lakehouse Platform combines elements of data lakes and data warehouses to provide a unified view onto structured and unstructured data. It is based on Apache Spark.

Widely used open source RDBMS

Developed as objectoriented DBMS (Postgres), gradually enhanced with 'standards' like SQL

Primary database model

Relational DBMS

Document store
Relational DBMS

Relational DBMS

with object oriented extensions, e.g.: user defined types/functions and inheritance. Handling of key/value pairs with hstore module.

Secondary database models

Document store
Graph DBMS

with Apache Age
Spatial DBMS
Vector DBMS

with pgvector extension

DB-Engines Ranking measures the popularity of database management systems
Trend Chart

Score	21.62
Rank	#29	Overall
	#18	Relational DBMS

Score	102.66
Rank	#12	Overall
	#2	Document stores
	#8	Relational DBMS

Score	674.32
Rank	#4	Overall
	#4	Relational DBMS

Website

spark.apache.org/sql

www.databricks.com

www.postgresql.org

Technical documentation

spark.apache.org/docs/latest/sql-programming-guide.html

docs.databricks.com

www.postgresql.org/docs

Developer

Apache Software Foundation

Databricks

PostgreSQL Global Development Group

www.postgresql.org/developer

Initial release

2014

2013

1989

1989: Postgres, 1996: PostgreSQL

Current release

3.5.0 ( 2.13), September 2023

16.4, August 2024

License

Commercial or Open Source

Open Source

Apache 2.0

commercial

Open Source

BSD

Cloud-based only

Only available as a cloud service

yes

DBaaS offerings (sponsored links)

Database as a Service

Providers of DBaaS offerings, please contact us to be listed.

PostgreSQL Flex @ STACKIT offers managed PostgreSQL Instances with adjustable CPU, RAM, storage amount and speed and several extensions available, in enterprise grade to perfectly match all application requirements. All 100% GDPR-compliant.

Implementation language

Scala

Server operating systems

Linux
OS X
Windows

hosted

FreeBSD
HP-UX
Linux
NetBSD
OpenBSD
OS X
Solaris
Unix
Windows

Data scheme

yes

Flexible Schema (defined schema, partial schema, schema free)

yes

Typing

predefined data types such as float or date

yes

XML support

Some form of processing data in XML format, e.g. support for XML data structures, and/or support for XPath, XQuery or XSLT.

yes

specific XML-type available, but no XML query functionality.

Secondary indexes

yes

SQL

Support of SQL

SQL-like DML and DDL statements

with Databricks SQL

yes

standard with numerous extensions

APIs and other access methods

JDBC
ODBC

JDBC
ODBC
RESTful HTTP API

ADO.NET
JDBC
native C library
ODBC
streaming API for large objects

Supported programming languages

Java
Python
R
Scala

Python
R
Scala

.Net
C
C++
Delphi
Java

JDBC
JavaScript (Node.js)
Perl
PHP
Python
Tcl

Server-side scripts

Stored procedures

user defined functions and aggregates

user defined functions

realized in proprietary language PL/pgSQL or with common languages like Perl, Python, Tcl etc.

Triggers

yes

Partitioning methods

Methods for storing different data on different nodes

yes, utilizing Spark Core

partitioning by range, list and (since PostgreSQL 11) by hash

Replication methods

Methods for redundantly storing data on multiple nodes

none

yes

Source-replica replication

other methods possible by using 3rd party extensions

MapReduce

Offers an API for user-defined Map/Reduce methods

Consistency concepts

Methods to ensure consistency in a distributed system

Immediate Consistency

Foreign keys

Referential integrity

yes

Transaction concepts

Support to ensure data integrity after non-atomic manipulations of data

ACID

Concurrency

Support for concurrent manipulation of data

yes

Durability

Support for making data persistent

yes

In-memory capabilities

Is there an option to define some or all structures to be held in-memory only.

User concepts

Access control

fine grained access rights according to SQL-standard

More information provided by the system vendor

We invite representatives of system vendors to contact us for updating and extending the system information,
and for displaying vendor-provided information such as key customers, competitive advantages and market metrics.