`{}.pop([])` raises `KeyError` not `TypeError` - only on empty dict

[samuelcolvin]

Bug report

Bug description:

{}.pop([])

Raises a KeyError, not a TypeError as I would expect.

However

{1: 2}.pop([])

Raises TypeError: cannot use 'list' as a dict key (unhashable type: 'list') as you'd expected.

This isn't new, it's been around since at least 3.8. Is it really intended? If so, it should be documented.

CPython versions tested on:

3.14

Operating systems tested on:

macOS

[rhettinger]

This isn't a bug. dict.pop(m, k) is checking the arguments in the order provided, the dict first and then the key. It is perfectly reasonable to do zero processing on the key if the mapping is empty and it is known that the pop cannot succeed.

[samuelcolvin]

Okay that makes sense, but it is somewhat surprising behavior, I think it should at the very least be documented on the pop() method.

[serhiy-storchaka]

It makes sense, calculating the key hash is just a waste for empty dict.

But it is cheap to check tp_hash (without calling it) before raising an exception in C, so TypeError can be raised for some non-hashable types. Although this can make life harder for alternative implementations.

[johnslavik]

Shall we document this as an "implementation detail"?

[rhettinger]

Shall we document this as an "implementation detail"?

I don't think so:

• The behavior has been present since dict.pop was introduced in Python 2.3. No one has reported an issue about it until now. That suggests that there isn't a problem than needs to be solved and that dict.pop has worked out well in practice.
• We generally don't document the order that arguments are checked or other low level details. The docs are already too voluminous; adding more text will just make it less likely that the important stuff get read at all.
• It is common that when a variable isn't used, it doesn't get checked. For example, list(map(3, '')) returns [] rather than complain about the 3 not being callable.

Okay that makes sense, but it is somewhat surprising behavior

In my experience, people get "surprised" when they invent a mental model that don't correspond to an actual implementation (some variation of "I would have done it differently"). It is usually the case that the surprise happens upstream from reading the docs in detail. In other words, documenting at this level of specificity doesn't prevent the surprise, it just confirms it after the fact.

But it is cheap to check tp_hash (without calling it) before raising an exception in C, so TypeError can be raised for some non-hashable types.

This is possible but would be a change in behavior and could break code. Also as you point out, it might be challenging for other implementations. Personally, I think the current behavior is reasonable.

[tim-one]

I think it actually is surprising that ``{}.pop([], 42)returns 42, but{1: 2}.pop([], 42)` raises an exception. When a default is supplied, the internal inconsistency becomes apparent in non-exceptional code. So l would, reluctantly, call that "a bug" - I can't believe that was intended behavior, just accidental.

However, I see that PyPy reproduces this odd behavior - so they were very conscientious in duplicating CPython's quirks - or peoples' brains are just wired differently than mine 😄.

[serhiy-storchaka]

On other hand:

>>> [] in {}
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    [] in {}
TypeError: cannot use 'list' as a dict key (unhashable type: 'list')
>>> {}.get([])
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    {}.get([])
    ~~~~~~^^^^
TypeError: cannot use 'list' as a dict key (unhashable type: 'list')
>>> {}[[]]
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    {}[[]]
    ~~^^^^
TypeError: cannot use 'list' as a dict key (unhashable type: 'list')
>>> del {}[[]]
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    del {}[[]]
        ~~^^^^
TypeError: cannot use 'list' as a dict key (unhashable type: 'list')

Even if it could simply return False and None or raise KeyError like for any hashable key.

The same behavior is preserved for sets, even in it's close analogue of dict.pop -- set.discard.

>>> set().discard([])
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    set().discard([])
    ~~~~~~~~~~~~~^^^^
TypeError: cannot use 'list' as a set element (unhashable type: 'list')

dict.pop() is the only exception, and this behavior clearly was not intentionally. So, we have two options:

1. If there was a good reason for that optimization, it should be consistently used in other dict and set methods and operations. And documented (maybe as CPython specific).
2. Otherwise, dict.pop() should be bring in line with others in dict and set.

The latter option is easier, and I do not think that that optimization matters in real programs. If the key is unhashable, this is rather a programming error, and the current implementation hides it.

[rhettinger]

I understand the itch to "make everything seem the same", but object to changing long standing behavior that has not caused issues in practice and that users may be relying upon. That is gratuitous (completely unnecessary).

FWIW, I am the author and maintainer of this code that dates back to 2007. The behavior was intentional and in my view completely reasonable (performing no checks on an unused argument). While not consistent, there is precedent for this in other functions (i.e. map not checking for the function argument being callable when the iterable is empty).

I'm sorry but am a strong -1 on making a potentially breaking change to long deployed behavior that was intentional and reasonable. The potential benefit is almost zero — it appeases the OP and makes one of the devs less itchy about everything not being exactly parallel. A few days ago, neither person had known or cared. No real user will benefit. Some may be harmed. Behaviors will vary across versions and package authors will need to catch both exceptions for compatibility.