Kafkaesque

Kafkaesque is a transactional outbox built for PostgreSQL and, primarily, Kafka.

• Transactional safety for messages: if they were created, they will be eventually published. They're only created if the transaction commits
• Ordering: messages are published sequentially for topic/partition combinations
• Shutdown safety: has graceful shutdown and rescue for cases where it doesn't happen.
• Observability: all operations publish telemetry events
• Garbage collection: outbox table is periodically cleaned

Installation

Add :kafkaesque to the list of dependencies in mix.exs:

def deps do
  [
    {:kafkaesque, "~> 1.0-rc.0"}
  ]
end

Getting started

Check the Getting started guide in Hexdocs.

Updating versions

To go from 1.0.0-rc.1 to 1.0.0-rc.2, an additional migration is needed:

defmodule MyApp.Migrations.BumpKafkaesque do
  use Ecto.Migration

  def up do
    Kafkaesque.Migrations.up(:v1, :v2)
    Kafkaesque.Migrations.up(:v2, :v3)
  end

  def down do
    Kafkaesque.Migrations.down(:v3, :v2)
    Kafkaesque.Migrations.down(:v2, :v1)
  end
end

No extra steps are required if 1.0.0-rc.2 or a newer version was installed.