David R. MacIverWhile we're talking about weird Python edge cases, what do you think this does?
```
class Foo:
def __getitem__(self, i):
return i
def __len__(self):
return 5
print(list(Foo()))
```
The answer is so counter-intuitive to me that I have independently discovered it twice, about a year apart, and posted outraged comments about it in discord both times, and the second time had to be reminded that this was not the first time I'd discovered the behaviour.
It's possible it's also not the first time I've posted it on mastodon.
CF Bolz-Tereick@DRMacIver hmmmmm, if it were the obvious [0, 1, 2, 3, 4] you wouldn't be outraged. so let me guess, it's an infinite loop? a MemoryError?
@cfbolz Correct, yes. It will hang until it eventually runs out of memory.
Now, do you know why?
Jonathan Hartley@DRMacIver @cfbolz Wild guess: the iteration doesn't consult `__len__`, but just keeps retrieving incremental indexes waiting for one to raise an exception.
UPDATE: my language is sloppy, but you get what I'm arm-waving about.
@tartley @DRMacIver yeah, it probably waits for an IndexError?
@cfbolz @tartley Yup, exactly that. Default iteration is like:
```
i = 0
while True:
try:
yield x[i]
except IndexError:
break
i += 1
```
Instead of:
```
for i in range(len(x)):
yield x[i]
```
In some sense you should never be able to tell the difference in correct code, so you'll rarely notice this, but I've at least twice managed to write incorrect code that doesn't raise for out of bounds access
@DRMacIver @tartley I think it's a bit of a general pattern in python, that the language mostly doesn't really use more than one special method in combination (with some exceptions of course)
@DRMacIver @tartley also, re the hang: is it actually not even ctrl-c-able? that's *really* annoying?!
@cfbolz @DRMacIver oh darn I hadn't expected that. I confess I don't know what determines that. Now you've nerd sniped me into figuring that out...
David Beazley@tartley @cfbolz @DRMacIver It's probably running a tight loop in some C code somewhere outside of the main eval loop where ctrl-c checking happens.
@dabeaz @tartley @DRMacIver nope, it was actually my own fault. ctrl-c works fine in CPython, but actually not in pypy3 😬 (it's my default python, so I tried there)
@tartley @dabeaz @DRMacIver I mean, you can definitely get CPython to enter an uninterrupted loop with the approach below. you get bonus points for telling me why operator.index works as `__getitem__` despite not taking a self?! (I don't know the answer to that). and why ctrl-c doesn't work here.
@cfbolz @tartley @DRMacIver Whoa. That's freaky. I don't think I've ever seen the __index__ method before. Couldn't tell you want it does or what purpose it serves.
@dabeaz @tartley @DRMacIver anyway, it's a super old feature, from python 2.5 times: https://peps.python.org/pep-0357/ (so you likely saw it at some point and suppressed the memory ;-))
PEP 357 – Allowing Any Object to be Used for Slicing | peps.python.org
This PEP proposes adding an nb_index slot in PyNumberMethods and an __index__ special method so that arbitrary objects can be used whenever integers are explicitly needed in Python, such as in slice syntax (from which the slot gets its name).
@dabeaz @tartley @DRMacIver bit mean, but it's in section '4.12 Conversion Protocols' of Python Distilled, no? 😬😂
@cfbolz @tartley @DRMacIver Could be. I'll refer to my PTSD comment from earlier). Arg. The surface area is too damn big and there are corner cases everywhere!
@cfbolz @tartley @DRMacIver Since we're on the general topic of WTF?!? Here's one of my favorite examples to show and discuss in class:
@dabeaz @tartley @DRMacIver wow, that's amazing/horrible! NamedTuple isn't even a class at all, but a function?!
@DRMacIver @dabeaz @tartley inheriting from something that's not a type just means calling it, I think. That's definitely in the 'weird corner cases' category again though,so I don't remember the details 😬
@DRMacIver @dabeaz @tartley waaaaaaaat, why is the body of the function NamedTuple needed at all?! just so that you can also call it explicitly?