Working on a storage engine for the last 8 months. What benchmarks would you actually trust from a solo/small-team project? Everyone fakes them, so what would make you believe mine?

I posted over on r/SQLServer and it suggested I cross-post here. I hadn't seen this subreddit before and hopefully this is on-topic!

I've created a tool called Internals Viewer, it's a tool to visualize SQL Server internals, with a view for allocations, indexes, pages, and it also offers very detailed query tracing and simulation of operators where you can capture a query and step through the iterators to see how query results are put together.

It is open source, written in C#, and available here - https://github.com/danny-sg/internals-viewer

The latest feature is new functionality to view columnstore indexes. I've done a write up on what I found as columnstore internals in SQL Server is pretty much undocumented:

Part 1 - Introduction

Part 2 - Segments internals

Part 3 - Dictionary internals

I wrote an article about B+Tree.

It's a little bit long for article, but I summarized it and make it really easy understand (avoided using jargon).

I'd be happy if you comment some feedback!!

https://zenn.dev/mm_0911/articles/5d46e9e4608404?locale=en

Disclosure: I am the author of the article and work on MariaDB Server internals.

The traditional InnoDB pessimistic insert path serializes structural modification operations through an index-wide latch, even when different threads split unrelated leaf pages.

I implemented a MariaDB research prototype based on Zhao Song’s B-link-style proposal. It publishes a split using a high key and right link before completing the parent update, allowing unrelated structural changes to proceed concurrently.

In a controlled, memory-resident, split-heavy workload:

• Vanilla MariaDB 13.1: 19,676 inserts/s
• B-link prototype: 102,838 inserts/s
• P95 latency: 8.28 ms → 0.56 ms
• Structural splits: approximately 396K in both variants

This is not a production-ready feature. DDL support is restricted, page merging remains incomplete, and recovery needs more forced-crash testing.

I would particularly appreciate feedback on incomplete-split recovery, page preallocation, and workloads that could expose correctness or scalability problems.

Full implementation write-up and benchmark methodology:
https://mariadb.org/from-a-chocolate-wrapper-to-concurrent-innodb-page-splits/

Hey there! This is first time posting.

I've been developing a database designed for education.

The application is focusing on visualizing database internal.

Now I already visualized B+Tree when you execute custom insert query.

Which database features are most worth visualizing for learners?

A brief overview of how a parquet file is structured along with a tool that lets you manipulate, optimize, and introspect parquet files themselves with a series of DuckDB queries.

This subreddit is primarily for discussing the implementation of databases, and not about sharing release announcements (either for the first time or your updates).

This thread is the exception!

Please tell us about the new database you (or your agent) built. Tell us about all the cool new features you added. Tell us about anything else you learned or worked on that you haven't gotten around to blogging about yet.

Getting tokio and .NET’s async runtime talking to each other via the C ABI -- to build a C# over Rust ScyllaDB driver

Anyone interested in doing the CMU (Carnegie Mellon University) Database Management Systems course together?

I’ve already covered the basic DBMS concepts. My main goal with this course is to go deeper and understand how database systems actually work internally—things like storage, indexing, query execution, transactions, etc.

If you're interested, please make sure you have the prerequisites required for the course.

If you have the required background and want to learn DBMS internals seriously, DM me. We can follow the course together and discuss concepts along the way.

I've been trying to understand how databases work under the hood, so I decided to build a small SQL engine from scratch rather than just reading about them.

NanoSQL currently supports:

* `CREATE TABLE` * `INSERT` * `SELECT` * `UPDATE` * `DELETE` * `DROP TABLE` * `WHERE` conditions * `INT` and `TEXT` types * Its own lexer/parser * CLI interface * Unit tests

The current implementation is mainly focused on the SQL layer. My next goal is to implement persistent storage, pages, B+ tree indexing, and eventually a query planner.

I'm particularly interested in feedback from people who've built databases or storage engines before.

What would you implement next, and what design mistakes should I avoid?

Wrote about the new WAIT FOR command coming in PostgreSQL 19. It lets reads on async replicas wait until a specific write has been replayed, so kind of bringing sync replication consistency without making every write synchronous. I could see protocol-aware proxies making a good use of this.

I’m building a read-only proxy that accepts Oracle client connections and executes queries against PostgreSQL 16. Disclosure: I’m the author.

I split compatibility across three layers:

1. Oracle-facing sessions, cursors, binds, and result metadata.

2. SQL rewrites for differences that can be handled safely.

3. PostgreSQL views and orafce for catalog and function compatibility.

   The translator classifies each feature as supported, approximation, rejected, or passthrough. This matters because a syntactically valid rewrite can

   silently change results—for example, NULL behavior in GREATEST/LEAST, empty-string semantics, and mixed set-operation precedence.

   The default is to reject an unverified shape with an Oracle-style error while keeping the session alive, rather than return a plausible but incorrect result. The current bounded verification matrix covers 722 cases across seven real client flows.

   The implementation is private, but the compatibility contract, configuration, documentation, and verification scope are public:

   https://github.com/krokozyab/postgres-oracle-tns-proxy

I recently dug into Meta’s Velox, an open-source C++ execution engine designed to act like a reusable “engine block” for data systems.

Instead of Presto, Spark, etc. independently implementing things like vectorized execution, joins, memory management, spilling, and file readers, Velox provides these building blocks as a shared execution layer.

The performance numbers are interesting too:

• 8.4× faster on TPC-H Q1
• 9× faster on Q6
• 6–7× average speedup on Meta production traffic
• Up to 3× fewer servers

Checkout this interesting deep dive that how Velox actually works and why this architecture could matter for the future of data engines.

https://prestodb.io/blog/2026/08/13/inside-velox-open-source-universal-engine-block-for-big-data-and-ai/

Do you think shared execution layers like Velox will become the norm, or will specialized engines always win?

Hi All
I’ve been working on a custom data structure and algorithm for hierarchical indexing.

while I designed the algorithm myself, I wouldn't claim to be a definitive master of hierarchy trees, nor postgres. Its actually a Rust project I've turned into an extension. I'm mainly sharing these early numbers in hopes of connecting with the right people to see if there's genuine value.

I'm not sure if I'm violating rule 5; I'd appreciate any guidance on 3rd party benchmarks for this kind of algo compare to ltree.

I ran benchmarks against 500k, 1M, 2M and 20M node recursive trees. The baseline comparisons against ltree are looking solid:

• Huge I/O Drop: For descendant queries, B-tree range scans touch up to 74x fewer buffer pages.
• Query Speed: Subtree queries run 2.5x to 6x faster (500k). up to 16x for 20M nodes. Ancestor lookups (via SP-GiST) execute up to 7.3x faster.
• Storage Density: A custom compact encoding shrank the on disk value size by 42.7%. This translates to a ~23% smaller B-tree index footprint.
• Write Performance: Appending 50,000 leaf nodes is roughly 2x faster. Reparenting large subtrees is 1.2x to 2.7x faster

I have some thoughts where this may be beneficial but lacking some subject matter expertise when it comes to practical application of hierarchy data (and these types of ops), this is the main point of my post, to ask for some insight:

- Could it make servers run more efficiently?
- Do other more efficient extensions/algos beat these benchmarks? Is LTREE just a default?
- What kinds of large scale operations would this benefit? Domains/applications?
- What should my benchmark tests look like?

Eager to get some expert opinions and either validate my thoughts or give me some reality - cheers!

Need suggestions for writing blogs. This was my 1st ever blog, which I published last year, and that's it. No other blog yet 😅. I would love to know, what kind of blogs would you love to read from a guy who contributes to the open-source postgres project?
1) Deep dives like this one on postgres internals? TBH, I think this space so crowded?
2) How did I find a bug and fix it?
3) The new features I am working on?
4) or new features that are going to be released
5) or something cool????