DB-EnginesInfluxDB: Focus on building software with an easy-to-use serverless, scalable time series platformEnglish
Deutsch
Knowledge Base of Relational and NoSQL Database Management Systemsprovided by solid IT

Blog > Post

Some remarks about the DB-Engines ranking score

by Paul Andlinger, Matthias Gelbmann, 3 December 2015
Tags: DB-Engines Ranking

The recent DB-Engines Ranking of December 2015 produced some surprises. Several systems, which continuosly showed a very positive trend (e.g. MongoDB, Cassandra, Redis, Neo4j) in the past, lost points in the recent ranking. Other major players, with a rather stuttering trend in the ranking so far, heavily increased their score (e.g. Oracle, MySQL, Microsoft SQL Server). Why did that come?

We have been asked that question by several DBMS vendors during the last days. Although we have disclosed the methodology of calculating the scores of the DB-Engines Ranking and many other details, we feel the need to give some additional info here:

1) The Ranking uses the raw values from several data sources as input. E.g. we count the number of Google and Bing results, the number of open jobs, number of questions on StackOverflow, number of profiles in LinkedIn, number of Twitter tweets and many more.

2) We normalize those raw values for each data source. That is done by dividing them with the average of a selection of the leading systems in each source. That is necessary to eliminate the bias of changing popularity of the sources itself. For example, LinkedIn increases the number of its members every month, and therefore the raw values for most systems increase over time. This increase, however, is rather due to the growing adoption of LinkedIn and not necessarily resulting from an increased popularity of a specific system in LinkedIn. Giving another example: an outage of twitter would reduce the raw values for most of the systems in that month, but obviously has nothing to do with their popularity. For that reason, we are using a selection of the best systems in each data source as a 'benchmark'.

3) The normalized values are then delinearized, summed up over all data sources (with weighting the sources), re-linearized and scaled. The result is the final score of the system.

The normalization step is the key to understanding the December results: the top three systems in the ranking (Oracle, MySQL and SQL Server) all increased their score. Oracle and MySQL gained formidable 16 and 11 points respectively. As a consequence the benchmark increased, leading to potentially less points for many other systems.

Why are we not using all systems as a benchmark for a data source? Well, we continuously add new systems to our ranking. Those systems typically have a low score (assuming that we are not missing major players). Then, each newly added system would reduce the benchmark and increase the score of most of the other systems.

Conclusion: it is important to understand the score as a relative value which has to be compared to other systems. Only that can guarantee a fair and unbiased score by eliminating influences of the usage of the data sources itself.
Detailed reports of the ranking performance of a system with charts and breakdowns into several categories can be ordered from us. Please find more info about that service at https://db-engines.com/en/services.





Share this page

Featured Products

Datastax Astra logo

Build cloud-native apps fast with Astra, the open-source, multi-cloud stack for
modern data apps.
Get started with 80GB free.

MariaDB logo

SkySQL, the ultimate
MariaDB cloud, is here.

Get started with SkySQL today!

Vertica logo

Vertica Accelerator. The fastest analytics and machine learning, delivered as SaaS, with automated setup, administration, and management. Free trial.

Neo4j logo

See for yourself how a graph database can make your life easier.
Use Neo4j online for free.

Redis logo

The world’s most loved real‑time data platform.
Try free

Present your product here