@askonomm @zenforyen @cazabon @kevin @ado
Maybe our code base is not large enought😉.
But seriously, I think the reasons are more fundamental.
For example, static type checkers would probably run away in panic from our code base, because we have many situations of indirect function calls manipulating *args.
For example, a cache system based on a #python function based dependency graph, where many function calls are indirect using second order functions and adding parameters dynamically.
@folkerschamel @askonomm @cazabon @kevin @ado I think you're fighting a strawman.
It goes without saying that either you decide to "work with" a type checker, and adopt a #python style to maximize code it can check, or you don't.
Yes, you can't throw pyright or mypy at an arbitrary dynamic wild west code base, and hope it can help you, it can't do magic. So it's a choice you ideally did at the start.
I personally prefer having that additional mechanical pair-programmer in my team.
@zenforyen @askonomm @cazabon @kevin @ado
You can squeeze a round piece into a squared hole. But the solution for a programming problem should reflect the type of problem. Many problems are deeply inherently dynamically typed. I think we have many in our code base.
And when wanting static type checking, then I think it's better to use a static typed language like #java, #cpp or #golang instead of fake static typing aka type hints and misuse #python for something it is not designed for.
@folkerschamel @askonomm @cazabon @kevin @ado If I had the choice, I would not use python for many things.
But there are factors such as library ecosystem and also accessibility. Where I work, still the chance is much higher someone will be able to maintain the type-hinted python code than if I wrote it in a statically typed language.
@zenforyen @askonomm @cazabon @kevin @ado
How would you express something like the following architecture in #mypy friendly #python code or even better in different programming languages like #cpp, #java, #golang etc?
(One part of our software is fundamentally based on countless such calls all over the place.)
def f(smart_evaluator, a, b):
...
def g(...):
x = smart_evaluator.exec(f, someting_extra, a, b)
Of course it is possible, but the question is how natural, easy and elegant it is.
@folkerschamel @zenforyen @askonomm @cazabon @kevin @ado Use typing.ParamSpec and typing.Concatenate.
T = t.TypeVar("T")
P = t.ParamSpec("P")
class SmartEvaluator:
def exec(
self,
func: t.Callable[t.Concatenate[SmartEvaluator, P], T],
something_extra: int,
*args: P.args,
**kwargs: P.kwargs
) -> T:
return func(self, *args, **kwargs)
https://mypy-play.net/?mypy=latest&python=3.11&gist=d3eb03d5b02f194b626aed9021139a55
@bk1e @zenforyen @askonomm @cazabon @kevin @ado
That's cool, didn't know that, thanks!
It seems to me that it works if type variables are used in the same function declaration (your SmartEvaluator.exec) or scope-related function declarations (sample in https://docs.python.org/3/library/typing.html#typing.ParamSpec).
Therefore, am I correct that this solution doesn't work for the some_method_of_my_object situation described in https://mastodon.social/@folkerschamel/110881898235959069?
@folkerschamel @zenforyen @askonomm @cazabon @kevin @ado I think your other situation is similar to the `futures` API. The returned object `w` can have a generic type with type parameters that encode additional types, such as the result type. In your case, this would also include the parameter spec.
See https://github.com/python/typeshed/blob/main/stdlib/concurrent/futures/_base.pyi
@bk1e @zenforyen @askonomm @cazabon @kevin @ado
The type of w would have to encode all declarations of all some_method_of_my_object methods of my_object. You could define a interface (base class) for the type of my_object, so that the return value of create_wrapper_for_handing_over_to_thread_pool implements that interface, too.
In general, you don't use duck typing anymore, but are forced to verbosely declare and maintain interfaces and use generics all the time, like in #java or #cpp.
@bk1e @zenforyen @askonomm @cazabon @kevin @ado
All in all, you convinced me that you can probably express all these use cases with type hints in #python.
But it seems to me you are transforming #python into the verbosity and clumsiness (the function declaration syntax + declare and maintain interfaces all the time) of statically typed languages like #cpp or #java (without benefitting from their performance).
Reminds me of the mentioned horror of the #chromium #webrtc #cpp implementation.😉
@folkerschamel @bk1e @zenforyen @askonomm @cazabon @kevin @ado In general, Python with type hints is still *far* less clumsy or verbose than C++ or Java. (I guess I don't really know about modern Java as I haven't used it actively in years, but I can attest to this for C++.) But the type of code I've seen you asking about in this thread is not going to be a good demonstration of that. Perhaps you actually are working with one of those code bases where it really doesn't make sense to use static typing. (which is a valid choice in Python, although personally something I'd take as a sign that there are other things that could be improved about the code)
(in my experience) It's like someone said in an earlier toot: you can reap huge benefits from static type checking, but you have to collaborate with the type system, it's not something where you can tack on type hints to any arbitrary preexisting code base and automatically, necessarily make it better.
@diazona @bk1e @zenforyen @askonomm @cazabon @kevin @ado
If you want/need/benefit from static type checking (which definitely has many benefits), then why not using a language which is designed from that from the ground up?
But in my experience a static type system also has serious limitations. If you have the hammer of a static type system, you can but it may not be the best choice to treat everything as nail.
See https://mastodon.social/@folkerschamel/110898088814600129 where we try to force a cube into a round hole.😉
@folkerschamel @diazona @bk1e @askonomm @cazabon @kevin @ado I think this "vs" discussion is bogus given the fact that python is never planning to force anyone into typing. It's a community decision to either drift towards it, or not, or both will forever coexist.
If it does not look useful to you, just don't use it. Nobody is planning to deprecate hint-less python AFAIK 😉
If both sides could just accept that other opinions exist, we could move on. I think everything relevant was said here.
@zenforyen @diazona @bk1e @askonomm @cazabon @kevin @ado
I think the real underlying discussion is this:
Are there software engineering problems where the natural/easier/shorter/better/cleaner solutions are dynamically typed?
My thesis is: Yes. (Possibly even most of them.) Static typing limits your architectural tool set and can slow you down. So there are costs involved. Sometimes these can be compensated by benefits. Sometimes far from it.
@folkerschamel @zenforyen @diazona @bk1e @askonomm @cazabon @kevin @ado
I'd agree with that statement. So does the python type hinting approach.
We're all consenting adults here. Use hints where you know, you actually are expecting types.
@ketmorco @folkerschamel @zenforyen @diazona @bk1e @askonomm @cazabon @ado Where 'types' is broadly construed, in Python, to mean "something with this shape". It doesn't have to be an actual 'type' (in the OO programming sense), and probably most often shouldn't be a 'type'.
I really wonder how often people use 'list' and 'dict' in their type annotations when they really should have used 'Sequence' (or even 'Iterable') and 'Mapping'. Understanding which of these to use requires fully understanding how you intend to consume the object given to you, and that's software engineering right there!
@kevin @folkerschamel @zenforyen @diazona @bk1e @askonomm @cazabon @ado
Truth. Which is also why I love Python - you can gradually introduce bits of rigidity to your code when you need to. Sometimes it's too soon, and the only ones really qualified to answer that are the ones working on the code base.
I've worked on code where hints are meaningless. And I've worked on codebases where hints would have saved some bacon.
But it's impossible to know ahead of time which is which.
@folkerschamel @zenforyen @bk1e @askonomm @cazabon @kevin @ado ah, gotcha. I was vastly misunderstanding your position; I thought you were making the argument that not only do there _exist_ problems where dynamic typing is the right solution, but that it's the right solution for _all_ problems, at least when working in #Python. I can definitely agree that there are situations where it doesn't make sense to use static typing.
I do still think we disagree on just how often those situations come up, but that's more of a minor difference, and there may not be a productive discussion to be had by hashing it out 🤷
@folkerschamel @bk1e @zenforyen @askonomm @cazabon @kevin @ado Typed Python is still ergonomic, though it isn't the wild west like regular Python can be.
I consider it closer to TypeScrip - it's a lot more flexible than C++ or Java.
@folkerschamel @zenforyen @askonomm @cazabon @kevin @ado I meant something like
class WrapperForHandingOverToThreadPool(t.Generic[P, T]):
async def some_method_of_my_object(self, *P.args, **P.kwargs) -> T:
…
You don't need an abstract base class and multiple implementations. The ParamSpec lets you model the *args, **kwargs forwarding.
@bk1e @zenforyen @askonomm @cazabon @kevin @ado
In this use case are many different some_method_1_of_my_object, some_method_2_of_my_object, all having different statically typed parameters like some_method_1_of_my_object(int, float), some_method_2_of_my_object(string, int)
Folker
zenforyen
Brad
David Zaslavsky
Wayne Werner
Kevin P. Fleming
Raniz