Skip to content

Release v1.0.0 — Initial LSM-Tree Storage Engine

Latest
Latest

Choose a tag to compare

View all tags
@Mo7ammedd Mo7ammedd released this 03 Jul 21:31
· 52 commits to main since this release
v1.0.0
4e39bce

Release v1.0.0 — Initial LSM-Tree Storage Engine

We are pleased to announce the first public release of LSMSharp, a high-performance, durable, and efficient Log-Structured Merge-Tree (LSM-Tree) storage engine implemented entirely in C#.

What's Included

This release provides a complete, production-grade implementation of core LSM-Tree components and storage mechanisms:

Core Features

  • Thread-Safe Skip List for in-memory ordered storage (Memtable)
  • Write-Ahead Log (WAL) for durability and crash recovery
  • SSTable Storage with compressed, persistent sorted tables
  • Bloom Filters for fast, probabilistic key existence checks
  • Leveled Compaction strategy for efficient disk organization
  • K-Way Merge Algorithm for optimized compaction performance
  • Configurable Memory Thresholds and data block sizes
  • Crash Recovery — automatic WAL replay on startup
  • Async I/O support using modern C# patterns

Performance Highlights

  • Approximately 10,000+ operations per second write throughput (memory-bound)
  • Sub-millisecond read latency for hot data
  • Space-efficient on-disk storage with GZip compression
  • Automatic background compaction to maintain optimal performance

Getting Started

dotnet build
dotnet run

Or use the engine programmatically:

using var db = await LSMTreeDB.OpenAsync("./mydb");

await db.SetAsync("key", Encoding.UTF8.GetBytes("value"));
var (found, value) = await db.GetAsync("key");

await db.FlushAsync();
await db.CompactAsync();

Known Limitations

This is an initial educational implementation. Production-level features like snapshots, advanced compression (LZ4, Snappy), range queries, and horizontal scaling are planned for future releases.

Planned for Future Versions

  • Snapshots and point-in-time recovery
  • Range queries and iterators
  • Read caching for improved latency
  • Write batching for enhanced throughput
  • Partitioning for horizontal scaling
  • Advanced compression options

Contributions Welcome

This project is built for educational purposes and as a solid foundation for anyone exploring LSM-Tree internals or building storage solutions in .NET. Feel free to open issues, suggest features, or contribute improvements.

Learn More

See the README for full technical details, architecture diagrams, and usage examples.

Tag

v1.0.0

Assets 2
Loading