Add field name in serialization error

[NicolasPllr1]

Change Summary

The immediate user problem is warning/error messages not being helpful enough during serialization.

These messages currently only provide - in some cases - the input value and type information. Not the location.

So some users are having trouble debugging serialization problems as they don't know which model or fields are causing issues (see discussion #9317 or issue #10495).

This PR only adds the 'field name' information when available. And I tried to add it with minimal changes.
It felt natural to add this info in the list already giving the input value and type information (introduced in #1652 with a5f7af1).

However, there may be a bigger problem with error-handling for serialization. It's not as precise as what is done for validation. For example, no exact 'location' is given to the user in case of a warning/error. Here I only give the field name, not the full location.

In this sense, my changes feel like a 'hack' that does not solve the underlying problem. I think fixing the bigger problem is what #1454 is aiming for, so maybe it's fine only adding the field name like this for now.

Tests

I was surprised to find out that this change did not break any test!
So I extended one in serializers/test_model.py which was already testing warnings. Maybe serializers/root_model.py could have one too.

I am not sure I really understand why this was not tested in the first place, so I am not confident here.

Related issue number

• PydanticSerializationError missing validation error field name #1483

Other relevant issues/discussions

• How to trace Pydantic serializer warning without traceback pydantic#9317

• Add field name/location in serialization warnings pydantic#10495

• Standardize how we raise serialization warnings #1454

• Clean up PydanticUnexpectedValueError #1652

Examples

Original example from #1483

Original example (simplified)

from pydantic import BaseModel


class MyModel(BaseModel):
    int_value: int | None = None
    float_value: float | None = None


model = MyModel(int_value=1.0, float_value=1)
model.int_value = 1.0
model.float_value = 1


print("Deserialized:", model.model_dump_json())


# Currently (main)
"""
  PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value=1.0, input_type=float])
  return self.__pydantic_serializer__.to_json(
Deserialized: {"int_value":1.0,"float_value":1.0}
"""

# With this PR
"""
  PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [field_name=int_value, input_value=1.0, input_type=float])
  return self.__pydantic_serializer__.to_json(
Deserialized: {"int_value":1.0,"float_value":1.0}
"""

Notice the new field_name entry in the list (the field is called 'int_value' in the original example, which may be confusing here ...)

Nested models with a Union

This example is taken from a comment by Sydney in #1449.

Nested model with union

from typing import Annotated, FrozenSet, Union, Literal, List

from pydantic import BaseModel, PlainSerializer, IPvAnyAddress, Field
from ipaddress import IPv4Network

SortedFrozenNetworkSet = Annotated[
    FrozenSet[IPv4Network],
    PlainSerializer(
        lambda x: sorted(str(network) for network in x),
        return_type=FrozenSet[str],
    ),
]


class ModelA(BaseModel):
    type: Literal["A"]
    addresses: SortedFrozenNetworkSet


class ModelB(BaseModel):
    type: Literal["B"]
    hostname: str
    address: IPvAnyAddress


AnnotatedUnionType = Annotated[Union[ModelA, ModelB], Field(discriminator="type")]


class CompositeModel(BaseModel):
    composite_field: AnnotatedUnionType


data = {"composite_field": {"addresses"}}
model = CompositeModel(**data)
print("Deserialized:", model.model_dump_json())

# Currently (main):
"""
  PydanticSerializationUnexpectedValue(Expected `frozenset[str]` - serialized value may not be as expected [input_value=['127.0.0.1/32', '192.168.1.0/24'], input_type=list])
  PydanticSerializationUnexpectedValue(Expected `ModelB` - serialized value may not be as expected [input_value=ModelA(type='A', addresse...twork('127.0.0.1/32')})), input_type=ModelA])
PydanticSerializationUnexpectedValue(Expected `frozenset[str]` - serialized value may not be as expected [input_value=['127.0.0.1/32', '192.168.1.0/24'], input_type=list])
  return self.__pydantic_serializer__.to_json(
Deserialized: {"composite_field":{"type":"A","addresses":["127.0.0.1/32","192.168.1.0/24"]}}
"""

# With this PR:
"""
  PydanticSerializationUnexpectedValue(Expected `frozenset[str]` - serialized value may not be as expected [field_name=addresses, input_value=['127.0.0.1/32', '192.168.1.0/24'], input_type=list])
  PydanticSerializationUnexpectedValue(Expected `ModelB` - serialized value may not be as expected [field_name=composite_field, input_value=ModelA(type='A', addresse...twork('127.0.0.1/32')})), input_type=ModelA])
  PydanticSerializationUnexpectedValue(Expected `frozenset[str]` - serialized value may not be as expected [field_name=addresses, input_value=['127.0.0.1/32', '192.168.1.0/24'], input_type=list])
  return self.__pydantic_serializer__.to_json(
Deserialized: {"composite_field":{"type":"A","addresses":["127.0.0.1/32","192.168.1.0/24"]}}
"""

Notice that there is still some error duplication. The difference is that you get the 'field name' information in the warnings.

TypeAdapter with a dict (no change here)

TypeAdapter example from pydantic/pydantic#10495:

TypeAdapter example

from pydantic import TypeAdapter, BaseModel

class A(BaseModel):
    a: int

TypeAdapter(
        list[A | dict[str, int]] 
).dump_python([{'k': "v"}])  # Expected `int` but got `str` with value `'v'` - serialized value may not be as expected

# At the time, the warning was:
"""Expected `int` but got `str` with value `'v'` - serialized value may not be as expected"""

# But it's now better with (same in main/this PR):
"""
 PydanticSerializationUnexpectedValue(Expected `A` - serialized value may not be as expected [input_value={'k': 'v'}, input_type=dict])
 PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value='v', input_type=str])
"""

Note that this PR does not change this warning.

As the user suggests, a more precise location would be useful here. Or just the 'key' to begin with. But this PR does not address that.

Checklist

• Unit tests for the changes exist
• Documentation reflects the changes where applicable
• Pydantic tests pass with this pydantic-core (except for expected changes)
• My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @davidhewitt

[codecov]

Codecov Report

❌ Patch coverage is 50.00000% with 20 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/serializers/errors.rs	59.37%	11 Missing and 2 partials ⚠️
src/serializers/extra.rs	20.00%	4 Missing ⚠️
src/serializers/fields.rs	0.00%	2 Missing ⚠️
src/serializers/type_serializers/union.rs	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

CodSpeed Performance Report

Merging #1799 will not alter performance

_{Comparing NicolasPllr1:add-field-name-in-ser-error (84a1843) with main (ed0d1ca)}

Summary

✅ 163 untouched

[NicolasPllr1]

please review

[NicolasPllr1]

I forced-push to rebase on main.

There was a tiny conflict with the last commit ed0d1ca (with the Arc import in src/serializers/fields.rs).

[Viicos]

While we could definitely improve the path later, this is a good improvement for now -- thanks!