DotNetGuyJan 1, 2024

Did you guys heard of the Dataflow components? I didn't. I found them really helpfull in processing a lot of data in parallel.

#csharp #dotnet

https://allardschuurmans.nl/dataflow/

Dataflow components in Task Parallel Library

Sometimes you have components, structures, or techniques you stumble upon during your work, which are gems. These are the techniques that are not used daily by all developers, but for you, in your position, can save you a lot of time. I have found such a gem which is helpful

Allard Schuurmans
Show threadKirill OsenkovJan 1, 2024
@DotNetGuy my experience with TPL DataFlow is that it’s impossible to debug. You might get called, or not, and you have no idea why. It loses causality. Rx (observable) has the same problem.
Show threadmbrdevJan 1, 2024
@KirillOsenkov @DotNetGuy is that not just a general problem in parallel processing anyway?
Show threadKirill OsenkovJan 1, 2024
@mbrdev @DotNetGuy yes, but not to the same extent. The chaining of blocks (and the equivalent Rx observables) makes it so much more complicated. With regular C# async you have more causality and it's easier to track the source of values.
Show threadmbrdevJan 2, 2024
@KirillOsenkov @DotNetGuy that makes sense, all about tradeoffs I suppose, still it looks like an interesting API I might have a play around with it just for fun 😀 don’t think I have a solid use case for that kind of parallelism at work.
Show threadDotNetGuy
@mbrdev @KirillOsenkov I stumbled upon this while writing a typical producer-consumer pattern. While working on this I got a suggestion from GitHub copilot.
Jan 2, 2024 at 8:55amWeb