STJ generator with external (generated) context class

Currently, you cannot stack Roslyn source generators to operate on generated code. As a workaround I opted into running my own generators in a separate project. That is, [JsonSerializable(...)] class MyContext : JsonSerializerContext is generated elsewhere, however this requires the STJ generator to recognize external context classes.

Project1:

// use a custom generator to generate the context, 
// note this is only a "blueprint" since STJ generator is not yet run.

// <auto-genreated/>
[JsonSerializable(...)] 
[JsonSerializable(...)] 
public class GeneratedContext /* : JsonSerializerContex; */

Project2 references Project1:

// STJ generator won't recognize GeneratedContext in scope
// but perhaps we can use something like this to hint on that:

[JsonSerializerExternalContext(typeof(GeneratedContext))] // use as attribute source
partial class MyContext : JsonSerializerContex;

This way we can stack generators using separate projects but the workaround only works if the final generator can work with external types.

(PS: this is not a problem specific to STJ though, every time you want to stack generators this can be a blocker, so maybe a more generic solution can be implemented instead)

@gregsdennis

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Have you already tried using JsonTypeInfoResolver.Combine to combine the MyContext.Default and GeneratedContext.Default resolvers?

GeneratedContext.Default

That doesn't exist, note when you generate the context, STJ generator won't see it. I did elaborate why this is the case in OP.

It's something we had considered, but it didn't quite seem to justify the added complexity. What motivates this use case? Size concerns? Design time performance?

What motivates this use case?

It's a workaround for stacking generators that roslyn doesn't support. The scenario is being able to generate the context itself.

It's something we had considered, but it didn't quite seem to justify the added complexity

Since something like JsonSerializerExternalContext (or maybe MetadataTypeAttribute since the usage is similar) is a direct hint for the generator to know where to look at, I don't think it's necessarily adds too much complexity or perf penalty for this to be considered?

Oh I see, the context on the first project is generated by a custom generator which you want to take into account in the second project. It seems like a niche scenario if I'm honest, it's very unlikely we would be able to prioritize it in the near future.

@eiriktsarpalis

@eiriktsarpalis If the API is acceptable I can take a stab at it.

The following is roughly the only change needed to cover this:

private void ParseJsonSerializerContextAttributes(
        INamedTypeSymbol contextClassSymbol,
        out List<TypeToGenerate>? rootSerializableTypes,
        out SourceGenerationOptionsSpec? options)
{
    Debug.Assert(_knownSymbols.JsonSerializableAttributeType != null);
    Debug.Assert(_knownSymbols.JsonSourceGenerationOptionsAttributeType != null);

    rootSerializableTypes = null;
    options = null;

    foreach (AttributeData attributeData in contextClassSymbol.GetAttributes())
    {
        INamedTypeSymbol? attributeClass = attributeData.AttributeClass;

        if (SymbolEqualityComparer.Default.Equals(attributeClass, _knownSymbols.JsonSerializableAttributeType))
        {
            /* ... */
        }
        else if (SymbolEqualityComparer.Default.Equals(attributeClass, _knownSymbols.JsonSourceGenerationOptionsAttributeType))
        {
             /* ... */
        }
+       else if (SymbolEqualityComparer.Default.Equals(attributeClass, _knownSymbols.MetadataTypeAttributeType))
+       {
+           var typeSymbol = (INamedTypeSymbol?)attributeData.ConstructorArguments[0].Value;
+           ParseJsonSerializerContextAttributes(typeSymbol, out var externalTypes, out var externalOptions);
+           if (externalTypes is not null) 
+               rootSerializableTypes.AddRange(externalTypes);
+           if (externalOptions is not null) 
+               options = externalOptions;
+       }
    }
}

It's not that simple. You need to API propose a new attribute for including contexts. There's also design challenges. What happens if you specify two contexts and they have conflicting contracts for a given type? What happens if you have a ][sonSerializable(typeof(Foo))] in the last context and the transitive closure of either of the earlier contexts also includes a conflicting definition for Foo? This is not an easy problem.

conflicting definition

Isn't all that already possible to happen? We're just adding another source for attributes (the "metadata class"), the semantics of the sum of the applied attributes remain intact.

This definitely needs a design review pass but in any case I could start a prototype for that to happen at some point. For one, I'd like this attribute to be sufficient for the generator to find the type so there's practically two entries, JsonSerializableAttribute and MetadataTypeAttribute (or similar).

We are facing the same problem.

We have an ApiClient that contains a large number of subcategories of Api to external services, like this:

class ServiceApiClient
	ProjectCategoryApi
	DocumentCategoryApi
	FolderCategoryApi
	ReportCategoryApi
	// ... etc

Each category contains dozens of methods that call an external Api. Example:

class ProjectCategoryApi
	Task<ProjectDto> CreateProject(ProjectCreateRequest request)
	Task<ProjectDto> UpdateProject(ProjectUpdateRequest request)
	Task RemoveProject(string projectId)
	Task<ProjectDto> GetProject(string projectId)
	Task<ListPaged<ProjectDto>> GetProjectList(ProjectListQuery request)
	
class DocumentCategoryApi
	Task<DocumentDto> CreateProject(DocumentCreateRequest request)
	Task<DocumentDto> UpdateProject(DocumentUpdateRequest request)
	Task RemoveProject(string documentId)
	Task<DocumentDto> GetProject(string documentId)
	Task<ListPaged<DocumentDto>> GetDocumentList(DocumentListQuery request)

// ... etc

We don't want to hand-code context for each model, so we created a ApiClientSourceGenerator that automatically generates a ApiClientJsonSerializerContext

	
// project
[JsonSerializable(typeof(ProjectDto))]
[JsonSerializable(typeof(ProjectCreateRequest))]
[JsonSerializable(typeof(ProjectUpdateRequest))]
[JsonSerializable(typeof(ListPaged<ProjectDto>))]

// document
[JsonSerializable(typeof(DocumentDto))]
[JsonSerializable(typeof(DocumentCreateRequest))]
[JsonSerializable(typeof(DocumentUpdateRequest))]
[JsonSerializable(typeof(ListPaged<DocumentDto>))]

// ... etc

[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Default)]
[JsonSerializable(typeof(object))] // placeholder
public partial class ApiClientJsonSerializerContext : JsonSerializerContext;

But the STJ generator does not see types we have generated and does not generate the code we need
And we can't make the STJ generator work after our generator.

Same issue here - any source generator that generates any sort of DTO might want those types to have source generated serializer contexts.

The general problem is like OP says, source generators (like STJ) cannot reference outputs of other source generators.

I have not dug too deep in the STJ generator yet and probably way oversimplifying, but would it be possible for STJ to expose its serializer context generation as a library method? Then my source generator could generate the same context classes that STJ would have generated, by referencing STJ as a source generator / compile time dependency.

@eiriktsarpalis

It seems like a niche scenario if I'm honest, it's very unlikely we would be able to prioritize it in the near future. - @eiriktsarpalis

I don't think this is a niche scenario. As mentioned above, a very common scenario is API client generation, e.g. from an OpenAPI description. Perhaps more niche is my intent to eventually support generating API (server) code from OpenAPI.

I don't think this is a niche scenario. As mentioned above, a very common scenario is API client generation, e.g. from an OpenAPI description. Perhaps more niche is my intent to eventually support generating API (server) code from OpenAPI.

Could you share more details? What motivates the use of stacking contexts in that scenario over just doing an E2E generation at the application layer? If the concern is application size I believe the linker will do a good job at trimming away contexts generated at the library layer that are not being used.

We generally recommend doing E2E contract generation at the application layer using the final type graphs. In practice stacking contexts (e.g. as is currently already achievable using TypeInfoResolverChain) creates more problems than it solves, especially when generic types come into play.

Consider a nuget package that generates API client code. It provides an attribute that you put on a partial class.

[ApiClient("https://someserver.com/api/openapi.yml")]
public partial class SomeApiClient;

This generates all the methods and models needed to interact with the API.

As the developer of the nuget package, it makes sense to want to add those models to a serializer context. As the consumer of the package, this is a very nice feature.

Navigation Menu

Repository navigation

STJ generator with external (generated) context class #108317

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

STJ generator with external (generated) context class #108317

Description

Activity

dotnet-policy-service commented on Sep 27, 2024

elgonzo commented on Sep 27, 2024

alrz commented on Sep 27, 2024

eiriktsarpalis commented on Sep 27, 2024

alrz commented on Sep 27, 2024

eiriktsarpalis commented on Sep 27, 2024

alrz commented on Sep 27, 2024

eiriktsarpalis commented on Sep 27, 2024

alrz commented on Sep 27, 2024

sderunov commented on Mar 17, 2025

dferretti commented on Mar 28, 2025

gregsdennis commented on Mar 29, 2025

eiriktsarpalis commented on Mar 29, 2025

gregsdennis commented on Mar 29, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions