I want to see if it's possible to replace runtime reflection in @xunit v3 with #Roslyn source generators (for better performance and to support NativeAOT), but I think I've already hit the first blocking point: no support for #FSharp? Only #CSharp and #VB? #dotnet
@bradwilson @xunit yes, no Roslyn in F#.

@UrsEnzler @xunit Which means I either:

1. Don't support F# in @xunit v3
2. Don't support NativeAOT
3. Leave both systems in place, bloating the code and the maintenance burden (and having potentially different behavior, intentional or otherwise, between the two systems)

Ugh. 😔

@bradwilson @xunit as an F# dev,. I hope it's not 1)
But all options are bad.
AFAIK there will be some source generator support in F# in the future. But it's a different compiler, so I don't know whether it will ever work for C# and F# in a similar way.
Maybe you should ask the F# team about this.
@UrsEnzler @bradwilson @xunit if you're using source generators then you have some non-reflection-based API for defining tests - if you have that then it should be possible to make a combinator-based library that could emit those same primitives. That's the route I'd look down for F# here. Trying to chase Roslyn Analyzers is going to be pain otherwise IMO.
@chethusk @UrsEnzler @xunit "combinator-based library" is not a phrase I'm familiar with.
@bradwilson sorry - a library with a lot of small functions that you compose together to create the data structures in-memory. This is often pretty easy to do in F# because type inference makes defining the small functions very lightweight, and pervasive inlining makes is lower-cost.

@bradwilson The typical example here is a parser combinator library like FParsec or similar: https://fsharpforfunandprofit.com/posts/understanding-parser-combinators/

You make tiny functions that compose into the final product.

Understanding Parser Combinators | F# for fun and profit

Building a parser combinator library from scratch

@bradwilson @xunit instead of Roslyn source generators you can have a regular MSBuild code generator that generates code during the build, and it can be any code and support C# and F# equally well. I often find that people use Roslyn source generators when all they want is any code generator.
MSBuildTools/src/TaskInSolution at main · KirillOsenkov/MSBuildTools

Tool that dumps the evaluated values of all properties and items for a project without building it. - KirillOsenkov/MSBuildTools

GitHub
@KirillOsenkov @xunit You think I can completely remove all the reflection required for discovering and running tests with this?
@bradwilson @xunit hmm I don't know enough about your usage, and how you were planning to use Source Generators, just wanted to point out that in many cases where you don't really need to analyze the C# source you can get by without Roslyn. But in your case having access to the compilation and syntax trees might be required I think?
@KirillOsenkov @bradwilson @xunit What would be the difference against Roslyn ones? (besides support for F#). A cons is probably not running during design time?
@mihamarkic @bradwilson @xunit yes, not running on every keystroke in the IDE and not having access to the compilation and syntax trees/documents/semantic models
@KirillOsenkov @bradwilson @xunit So, basically, here are raw .cs files and do what you want during build?
@mihamarkic @bradwilson @xunit sure, or any other inputs you need. Granted, Roslyn Source Generators are great when you need to understand the .cs files, but in many cases I've seen them abused to just generate .cs files out of say .xml or .json, without the need to understand .cs. If you want to generate C# you may not need source generators, but if you want to read or understand C# you probably need them (unless you host Roslyn yourself and pay the perf penalty of reading the project twice)
@KirillOsenkov @bradwilson @xunit Yep, absolutely, plenty of times there is just xml->sources. Whatever happend to T4?
GitHub - mono/t4: T4 text templating engine

T4 text templating engine. Contribute to mono/t4 development by creating an account on GitHub.

GitHub
@KirillOsenkov @bradwilson @xunit Oh, nice and interesting naming - Mono. Btw, did we ever get a decent editor for .t4 files?
@KirillOsenkov @mihamarkic @bradwilson @xunit years ago there were some decent third party proprietary ones but they never got ported to VS2022. More recently there was https://marketplace.visualstudio.com/items?itemName=bricelam.T4Language which is OSS and used to have an LSP based on mono/t4. The LSP got removed a couple commits ago and I'm not sure why.
T4 Language - Visual Studio Marketplace

Extension for Visual Studio - Basic language support for .tt files

@mhutch @mihamarkic @bradwilson @xunit fun fact: in 2006 a fellow summer intern at Microsoft was tasked with building a language service for T4. Wish I remembered his name. I remember he got something working, and after he left I still had two weeks, so they asked me to quickly light up C# support :) This was pre-Roslyn obviously. Nobody understood how hard it was. I just used the time to fix some minor bugs in it. I don't think it ever shipped.
@KirillOsenkov @mhutch @mihamarkic @bradwilson @xunit even with Roslyn, as the owner of an LSP editor for a language with C# embedded, it’s still hard 😛
@davidwengier @KirillOsenkov @mihamarkic @bradwilson @xunit yeah, embedded languages are tricky. Back when MonoDevelop used NRefactory we had (somewhat flaky) support for C# in Razor, and I had some initial work to use that infrastructure to add C# completion in T4 and aspx, but it didn't survive the Roslyn port.
@davidwengier @KirillOsenkov @mihamarkic @bradwilson @xunit I would still like to add C# completion for inline MSBuild tasks in my MSBuild Editor but it's sooo far down the backlog 😓
@mihamarkic @bradwilson @xunit unfortunately nothing official
@KirillOsenkov @bradwilson @xunit And that's a huge pity. Back in the days I was using CodeSmith generator and I just checked it now - it's still there, though it's (still) commercial. @mhutch
Build Time Code Generation in MSBuild

Build-time code generation is a really powerful way to automate repetitive parts of your code. It can save time, reduce frustration, and eliminate a source of copy/paste bugs. This is something I’m familiar with due to my past work on MonoDevelop’s tooling for ASP.NET, T4 and Moonlight, and designing and/or implementing similar systems for Xamarin.iOS and Xamarin.Android. However, I haven’t seen any good documentation on it, so I decided to write an article to outline the basics.

mhut.ch
@mhutch @KirillOsenkov @mihamarkic @xunit I think in my case I'm going to need Roslyn because I need to inspect the source to do the generation (the simplest explanation is: I need a list of all methods in the project that are decorated with an attribute, so that I can bypass the need to find them via reflection at runtime, in order to support NativeAOT). I'm certainly going to investigate every option, though!

@mhutch @KirillOsenkov @mihamarkic @xunit (this time posted from the correct account 😂)

Funny historical fact: Jim and I created xUnit.net while working on CodePlex, so we were obviously the first internal adopters, but the first team outside of us that adopted xUnit.net internally was... Roslyn!

@bradwilson @mhutch @mihamarkic @xunit or perhaps you can run a task after Csc and reflect over the output binary at build time, then serialize your findings. This way you can support F# too.
@bradwilson @mhutch @mihamarkic @xunit remember to save the assembly MVID so you know when to invalidate

@bradwilson @KirillOsenkov @mihamarkic @xunit

My post was targeted at generating source code based on files that are not source code, e.g. generate serialization wrappers for a json file or something. Doing it this way has similar perf and complexity as Roslyn generators, and if you use CodeDOM you can support every language with a CodeDOM provider.

@bradwilson @KirillOsenkov @mihamarkic @xunit

Generating code for your assembly *based on code in your assembly* is much more difficult. You can use a two pass compile and reflection, or parse the source code yourself, but both these options really hurt build perf and are difficult to get right.

Generators are by far the best way to handle this but are Roslyn only. AFAIK there is no good solution to this class of problem that supports all languages.

@bradwilson @KirillOsenkov @mihamarkic @xunit

FWIW, generators also only solve the problem of letting you inspect parsed/resolved code and inject source code text into the compilation. There isn't actually any mechanism for generating the source code text. Personally, I would use either preprocessed T4 templates or CodeDOM for the actual generation part of a generator.

@bradwilson @KirillOsenkov @mihamarkic @xunit

If I were to implement code-dependent generation logic for both Roslyn and non-Roslyn languages, I would have two-pass compile targets for non-Roslyn languages with a reflection-based generator task, and have a generator for Roslyn languages. These would convert the Roslyn/reflection data into a shared model, then use a shared implementation to process that data and generate code via CodeDOM.

Not elegant, but I can't think of anything better 😞

@bradwilson @xunit the other thing to consider is that source generators can’t see the output of other generators, so if you have any users who generate tests, your generator won’t be able to find them.