Add support for Zodv4

[jankapunkt]

We all know that this library is on the one hand crucial for modern Meteor Apps, at the same time it's still tightly coupled with SimpleSchema.

This PR is a current proof of concept (note it's work-in-progress status) to make it independent from Simple Schema.

The idea is to be non-breaking for SimpleSchema users but provide all necessary functionality for other validation libs, like zod, ajv etc.

This change requires users to register a "validator" object that implements certain methods. Consider the example for Simple Schema (also located in tests/prepare.js:

import SimpleSchema from "meteor/aldeed:simple-schema";

Collection2.defineValidation({
  name: 'SimpleSchema',
 
  // check if the schema is a schema instance
  is: schema => SimpleSchema.isSimpleSchema(schema),

  // create a new schema instance by definition object
  create: schema => new SimpleSchema(schema),

  // extend the schema, I don't know if other libs support this...
  extend: (s1, s2) => {
    if (s2.version >= 2) {
      const ss = new SimpleSchema(s1);
      ss.extend(s2);
      return ss;
    } else {
      return new SimpleSchema([s1, s2]);
    }
  },

  // mutated cleaning of a doc or modifier, I don't know mofidier support in other libs
  clean: ({ doc, modifier, schema, userId, isLocalCollection, type }) => {
    const isModifier = !Collection2.isInsertType(type);
    const target = isModifier ? modifier : doc;
    schema.clean(target, {
      mutate: true,
      isModifier,
      // We don't do these here because they are done on the client if desired
      filter: false,
      autoConvert: false,
      removeEmptyStrings: false,
      trimStrings: false,
      extendAutoValueContext: {
        isInsert: Collection2.isInsertType(type),
        isUpdate: Collection2.isUpdateType(type),
        isUpsert: Collection2.isUpdateType(type),
        userId,
        isFromTrustedCode: false,
        docId: doc?._id,
        isLocalCollection
      }
    })
  },
  // incomplete yet, will be added once it's implemented
  validate: () => {},

  // the idea is to freeze the validator to avoid any tampering or redefining
  freeze: false
});

The hard part is to extract all Simple-Schema-specific code into this validator without breaking things and at the same time get the abstraction done in a way it will smoothly work with other libs.

Current status:

• simple schema
• ajv
• yup
• zod

[harryadel]

Thank you for getting this started, I'm continuing where you left off. I've merged in the latest work in main branch and got things working enough to start attempting the replacement. Let's see how it goes 🤞

[harryadel]

A beta had just been released so you can test. The aim is to support zod & ajv. Personally, I'd like to achieve zod support the most but I felt it's important to have another package so the implementation isn't too tight nor specific to zod & simple-schema.

Backwards compatibility is a must for us with. There's little to no change to current users that's why we came up with idea of adapters where we wrap validation libraries and make them compatible to be used within current collection2 code.

Either this or we had to rewrite how collection2 run validation internally and collection2 is too tightly couple with simple-schema where it creates a validation context to ensure data integrity.

meteor-collection2/package/collection2/main.js

Line 358 in 78bc1fd

let validationContext = options.validationContext;

In order to maintain a nice developer experience you don't have to utilize defineValidation, you simply attach your schema of choice and we detect its type (zod, simple-schema or ajv) and run validations accordingly.

[harryadel]

Can you recheck please? @jankapunkt @9Morello
4.1.1-beta.4 is out! Added tests for nested fields & $set were added along to other changes.

I think what remains now is two things:

• Figure out how to properly split/handle different validation libraries #468

  • Are we going to dynamically load different files or split into different packages kinda like how autoform treats its themes packges
  • Make Simple Schema weak dependency??
  • Unrelated thought, but the effort of integrating zod into collection2 made me understand why understand why @pmogollons directly chose to create a separate specific package for https://github.com/pmogollons/zod-schema
    Maintaining and supporting different schemas under the same umberalla is no easy task which is further compounded by the need to integrate with the old simple schema code that's tightly woven into collection2. I dunno. Maybe the future is in having a different package for each validation library or developers would directly integrate with mongo driver like mongoose bypassing the need for Meteor collection all together. 🤷

• Support Zod 4

  • Honestly, a huge bummer. As I was about to wrap version 3, @ceigey notified me via this forum post of the upcoming version. I tried actually to migrate to it but it's not easy also it's still in beta. So I'll proceed with finalizing version 3 support then get back to it once it's stable.

• Meteor 3.3 support?

[ceigey]

I'm sort of meandering around cluelessly here, but I think for schemaDetectors and Zod 4, you can use the following documentation as a guide:

https://v4.zod.dev/packages/core#schemas

export class $ZodType<Output = unknown, Input = unknown> {
  _zod: { /* internals */}
}

All @zod/core subclasses only contain a single property: _zod. This property is an object containing the schemas internals. The goal is to make @zod/core as extensible and unopinionated as possible. Other libraries can "build their own Zod" on top of these classes without @zod/core cluttering up the interface.

that _zod property seems pretty useful.

At the moment the draft/beta has the following for isZodSchema:

/**
 * Determines if a schema is a Zod schema
 * @param {Object} schema - The schema to check
 * @returns {Boolean} True if the schema is a Zod schema
 */
export const isZodSchema = (schema) => {
  return schema && 
         typeof schema === 'object' && 
         schema._def && 
         schema.safeParse && 
         schema.parse && 
         typeof schema.safeParse === 'function' && 
         typeof schema.parse === 'function';
};

Things like ._def I now believe live under ._zod.def - also, interestingly, it's apparently json-serializable.

Likewise, _def.shape would now live under _zod.def.shape.

(You have to look at the v4.0.0-beta tag under packages/core/src to see some of the implementation details since I didn't notice these documented yet in some sort of automatically generated reference doc)
https://github.com/colinhacks/zod/blob/v4.0.0-beta/packages/core/src/schemas.ts#L1475

I need to go through some of the error handling stuff separately. Otherwise I think some helper functions could be implemented to act as a compatibility layer between Zod 3 and Zod 4. It'll be interesting to see if anything emerges in the broader NPM community for that role, or maybe it's worth seeing how a similar package handles the migration.

[harryadel]

Hello Fellas, 4.2.0-beta.1 is now out with support for zod v4. Please review and try it out and let me know.
cc @jankapunkt @StorytellerCZ @ceigey @9Morello

[jankapunkt]

@harryadel I'd like to bump the discussion on splitting again. I personally think it's good to make SimpleSchema a weak dependency or even remove at all as I personally think we can 100% make this injectable. If Zod4 is implemented from your end I would start with the injection pattern. This would cause users to use multiple packages but I also think this makes sense in the long run.

Package structure would be:

aldeed:collection2

plus one of the following

aldeed:simple-schema
communitypackages:collection2-zod3
communitypackages:collection2-zod4
communitypackages:collection2-ajv
communitypackages:collection2-<customAdapterPackage>

What do you think?

[harryadel]

@jankapunkt Hey Jan, thank you for chiming in. I do agree with you on making simple-schema a weak dependence and so is zod but I don't think the multiple Meteor packages is the way to go. Rather, dynamically detecting what's the available validation package and use it accordingly, maybe even allow multiple validation libraries to co-exist and validate based on whatever's is attached to the collection. You can even see in the test-app for zod, the package is installed in the test-app itself and not collection2! Managing multiple packages would be very tedious and draining, and we can dynamically import whatever files needed based on the validation libraries used within a project.

// this is pseudo code
import { checkNpmVersions } from 'meteor/tmeasday:check-npm-versions';

if  (!Packages[`aldeed:simple-schema`] or !checkNpmVersions(zod)) {
 // please add one supported validation library
}

[jankapunkt]

But how to make sure this stays lightweight on client?

[harryadel]

@jankapunkt by dynamically importing files as is already done but we're going to make fancier. The current dynamic importing imports the entire collection2, now with the new adapters code we can also instruct the users to selectively import the needed files for their respective validation libraries to work.

[lynchem]

For what it's worth I like @jankapunkt's suggestion.

I also saw this today and wondered if it would help ?