feat: Simpler authdb #821

alxiong · 2025-10-17T16:08:59Z

Closes #<ISSUE_NUMBER>

This PR:

Thanks to @jbearer's suggestion, our first draft should be the simplest but correct solution of "add auth tag to every entry in the kv store"

drastically simplify the AuthDB.Get() and AuthDB.Put() to generic auth tag generation and verification, no more switch-case and different logic for each case.
introduce AuthBatch that make sure the batched Write() will add tags to all accumulated entries.
Move all espresso-specific AuthR/W to operations.go and enforce the reader/writer are AuthDB/Batch for correctness. (with a dedicated test in operations_test.go)
Introduce AuthIterator to wrap around a plain Iterator which originally step through the db without going through Put and Get`, thus require auth checks.
Introduce AncientItemWithTag and ensure the AncientKeyStore write will append auth tag (to the struct if Append(), or directly to the bytes if AppendRaw()) and read will check the tag.
Added an explainer markdown for our authdb approach.

(please don't squash merge.)

espresso/authdb/operations_test.go

Co-authored-by: Warp <warp@warp.dev>

alxiong · 2025-10-18T16:29:33Z

~~I couldn't reproduce the CI error on the test TestEspressoCaffNodeUnfinalizedDelayedMessages locally. @Sneh1999 can you help take a look?~~

seem to be fixed in 510516e

Copilot

Pull Request Overview

This PR simplifies the authenticated database (AuthDB) implementation by adopting a "tag every entry" approach instead of selectively tagging specific data types. The changes drastically reduce code complexity while maintaining cryptographic integrity guarantees.

Key changes:

Unified authentication using a generic suffix-based tagging scheme for all key-value entries
Introduced AuthBatch to ensure batched writes are authenticated atomically
Moved Espresso-specific operations to a dedicated operations.go file with type enforcement
Added AuthIterator to handle authentication during database iteration
Enhanced ancient store support with AncientItemWithTag wrapper

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`espresso/authdb/schema.go`	Replaced per-type auth tag prefixes with a generic suffix-based approach
`espresso/authdb/operations.go`	New file containing Espresso-specific authenticated operations with type enforcement
`espresso/authdb/operations_test.go`	Added security enforcement tests verifying AuthDB/AuthBatch type requirements
`espresso/authdb/authdb.go`	Major refactor: unified Put/Get authentication logic, added AuthBatch/AuthIterator, enhanced ancient store support
`espresso/authdb/geth_schema.go`	Removed entire file (345 lines) - no longer needed with simplified approach
`espresso/authdb/accessors_chain.go`	Removed entire file (136 lines) - chain-specific accessors no longer required
`espresso/authdb/DEV.md`	Added comprehensive documentation explaining AuthDB architecture and security model
`arbnode/espresso_caff_node.go`	Updated to use new operations.go functions instead of direct AuthDB methods
`arbnode/espresso_batcher_addr_monitor.go`	Updated to use new operations.go functions for authenticated read/write operations
`justfile`	Simplified test command and added cleanup for new test directory
`espresso/authdb/.gitignore`	Added `testancient` directory to ignored paths

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

espresso/authdb/authdb.go

Copilot · 2025-10-21T02:43:41Z

espresso/authdb/authdb.go

 func (it *AuthIterator) Value() []byte {
 	key := it.Key()
+	val := it.inner.Value()
+	if it.db.mac == nil || !bytes.HasSuffix(key, genericAuthTagSuffix) {


This condition checks if the key has the suffix, but we're iterating over actual data keys that shouldn't have the suffix. The logic should be if it.db.mac == nil only. The suffix check appears incorrect here since we already filtered out tag keys in Next().

Suggested change

if it.db.mac == nil || !bytes.HasSuffix(key, genericAuthTagSuffix) {

if it.db.mac == nil {

This is crucial comment, when I develop this, I also thought I shouldn't add extra filtering logic here since AuthIter.Next() already done that, but if I don't, somehow the authdb_test failed.

Then upon careful inspection, I realize the problem is the it.db.Get() below is causing an undesirable recursion: because inside db.Get() it will try to query -tag-tag etc. which is clearly doesn't exist.

anyhow, this is a logic error that i fixed in 510516e, but directly accessing the underlying db via it.db.db.Get().

@alxiong I didnt understand this, can you expand this on Slack or something for me?

espresso/authdb/authdb.go

espresso/authdb/operations.go

Sneh1999

@alxiong I wonder if we should also add tests for ancients because the current tests dont verify that

Sneh1999 · 2025-10-21T17:57:11Z

espresso/authdb/DEV.md

@@ -0,0 +1,144 @@
+# Authenticated Storage (AuthDB) Architecture


nit: should we not just name this file README.md

Sneh1999 · 2025-10-21T18:05:43Z

espresso/authdb/authdb.go

+func (d *AuthDB) authenticateAncientData(kind string, number uint64, data []byte) ([]byte, error) {
+	switch kind {
+	case rawdb.ChainFreezerHashTable:
+		dataSize := len(data) - d.mac.Size()


I am not sure I understand what you are doing here:

dataSize := len(data) - d.mac.Size() expectedTag := data[dataSize:] actualData := data[:dataSize]

Why is d.mac.Size() relevant here if we always reset the mac?

Sneh1999 · 2025-10-21T18:16:43Z

espresso/authdb/authdb.go

-	// Only if tee is enalbed, add the auth tag to the auth db
-	d.mac.Write(nextHotshotBlockNumKey)
-	d.mac.Write(EncodeUint64(num))
-	tag := d.mac.Sum(nil)


why are now resetting before creating the tag?

Sneh1999 · 2025-10-21T18:24:21Z

espresso/authdb/authdb.go


-func (d *AuthDB) ModifyAncients(fn func(ethdb.AncientWriteOp) error) (int64, error) {
-	return d.db.ModifyAncients(fn)
+	return op.inner.AppendRaw(kind, number, append(item, tag...))


I think we should keep it consistent between Append and AppendRaw basically what we can do is create an authItem and then rlp encode it so that its bytes are stored instead of doing this where we are no longer adding the authItem

okay after reading the code more, I have realized that the data can always be added in AppendRaw and Append format. The only difference is one is rlp encoded and one is not. So AppendRaw should always contain the same data but just have it in rlp encoded form as I mentioned above

Sneh1999 · 2025-10-21T18:30:44Z

espresso/authdb/authdb.go

+	return &AuthBatch{inner: inner, authDB: d}
 }

 func (d *AuthDB) WasmDataBase() (ethdb.KeyValueStore, uint32) {


should we check if we need to auth around these wasm stuff as well?

Sneh1999 · 2025-10-21T18:33:13Z

espresso/authdb/authdb.go

+	if b.entries == nil {
+		b.entries = make(map[string][]byte)
+	}
+	b.entries[string(key)] = value


what happens if inners Put has an error? we still add it to the entries

Sneh1999 · 2025-10-21T18:38:44Z

espresso/authdb/authdb.go

+}
+
+// Write commits the batch, adding auth tags for each entry if MAC is enabled
+func (b *AuthBatch) Write() error {


Batch seems to have way more functions

Sneh1999

@alxiong I saw that InspectFreezerTable also has MerkleStateFreezerName, VerkleStateFreezerName not sure if we need to deal with those as well. Actually we need to look into WriteStateHistory it has more tables which it is adding to ancients

Sneh1999 · 2025-10-21T20:30:49Z

espresso/authdb/authdb.go

-	}
-
-	return height, nil
+func (d *AuthDB) ReadAncients(fn func(ethdb.AncientReaderOp) error) error {


Why are we not adding auth to this?

Sneh1999

@alxiong one issue with this Ancients design is that when I try calling the reader.Ancients on a database which is already tagged, it will not return anything to me because the authentication will fail because of this code. So I think we should still store it in the original format and store the tags separately


	case rawdb.ChainFreezerBodiesTable, rawdb.ChainFreezerHeaderTable, rawdb.ChainFreezerReceiptTable:
		authItem := new(AncientItemWithTag)
		if err := rlp.DecodeBytes(data, authItem); err != nil {
			log.Error("invalid freezer rlp", "err", err)
			return nil, err
		}

		var buf bytes.Buffer
		if err := rlp.Encode(&buf, authItem.item); err != nil {
			log.Error("failed to RLP encode", "err", err)
			return nil, err
		}

		tag := d.computeMacForAncient(kind, number, buf.Bytes())
		if !hmac.Equal(tag, authItem.tag) {
			log.Error("auth tag mismatch in AncientItemWithTag")
			return nil, fmt.Errorf("auth tag mismatch for AncientItemWithTag, kind: %v, number: %v", kind, number)
		}
		return buf.Bytes(), nil

alxiong added 2 commits October 17, 2025 10:50

simple authdb, add tags for every entry

a6c8fb7

caf-specific r/w func with enforced auth Reader/Writer

6ad31ae

alxiong requested review from ImJeremyHe, Sneh1999, alysiahuggins, jbearer, jjeangal, sveitser and zacshowa as code owners October 17, 2025 16:09

Sneh1999 reviewed Oct 17, 2025

View reviewed changes

espresso/authdb/operations_test.go Outdated Show resolved Hide resolved

espresso/authdb/operations_test.go Outdated Show resolved Hide resolved

alxiong and others added 3 commits October 17, 2025 14:24

ensure auth iter also check auth tags

f1a08cf

add authentication to AncientKeyStore

4a4ab8a

AI-generated explainer doc

e9655f6

Co-authored-by: Warp <warp@warp.dev>

alxiong force-pushed the simpler-authdb branch from 66006be to e9655f6 Compare October 18, 2025 14:04

alxiong requested a review from Copilot October 21, 2025 02:42

Copilot AI reviewed Oct 21, 2025

View reviewed changes

fix infinite recursion in authiter.Value(), address copilot comments

510516e

Sneh1999 reviewed Oct 21, 2025

View reviewed changes

	if it.db.mac == nil \|\| !bytes.HasSuffix(key, genericAuthTagSuffix) {
	if it.db.mac == nil {

		@@ -0,0 +1,144 @@
		# Authenticated Storage (AuthDB) Architecture

feat: Simpler authdb #821

Are you sure you want to change the base?

feat: Simpler authdb #821

Conversation

alxiong commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR:

Uh oh!

Uh oh!

Uh oh!

alxiong commented Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Sneh1999 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sneh1999 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sneh1999 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alxiong commented Oct 17, 2025 •

edited

Loading

alxiong commented Oct 18, 2025 •

edited

Loading

Sneh1999 left a comment •

edited

Loading

Sneh1999 left a comment •

edited

Loading