Expected mypy errors

[dcbaker]

I'm working on a module specifically for use with typing, so I have tests that I expect to raise mypy warnings. It doesn't look like there's a way to mark tests as having expected errors.

[dmtucker]

I'm not sure I understand the issue 🤔 Could you elaborate or provide an example?
My initial reaction is that controlling/suppressing mypy errors seems like a mypy configuration issue.

[dcbaker]

What I want is basically the equivalent of:

def test_func():
    with pytest.raises(Exception):
       func('Invalid input')

but for mypy errors. I want to verify that an error is happening. My use case is that I'm writing "const" protocols (think list with out any methods or operators that modify), and I want to ensure that if I have:

class MyProto(Protocol[T]):

    def __add__(self, other: T) -> MyProto[T]: ...

class MyClass(Generic[T])
    def __add__(self, other: T) -> MyClass[T]:
    def __iadd__(self, other: T) -> None:


def test_func() -> None:
    c: MyProto[str] = MyClass(['foo'])
    with mypyerror:
        c += 'bar'  # this is an error, getting it is a test pass, not getting it is a test fail

[dmtucker]

Interesting... I'm not aware of a way to do that, as written, but perhaps doing the same kind of thing the pytest-mypy tests do would work. For example,

def test_func(testdir):
    testdir.makepyfile('''
        class MyProto(Protocol[T]):
            def __add__(self, other: T) -> MyProto[T]: ...

        class MyClass(Generic[T]):
            def __add__(self, other: T) -> MyClass[T]: ...
            def __iadd__(self, other: T) -> None: ...

        c: MyProto[str] = MyClass(['foo'])
        c += 'bar'
    ''') 
    result = testdir.runpytest('--mypy') 
    result.assert_outcomes(failed=2)  # mypy exit status check fails too
    result.stdout.fnmatch_lines([ 
        '9: error: bad type error message etc*', 
    ])

In fact, you don't even really need pytest to be involved:

def test_func(testdir):
    testdir.makepyfile('''
        c: MyProto[str] = MyClass(['foo'])
        c += 'bar'
    ''') 
    result = testdir.run('mypy', 'test_func.py') 
    result.stdout.fnmatch_lines([ 
        'test_func.py:9: error: bad type error message etc*', 
    ])

[Avasam]

Can't you just add a type: ignore[error-code] on that line ?
Make sure that the following codes are enabled:

• https://mypy.readthedocs.io/en/stable/error_code_list2.html#check-that-type-ignore-comment-is-used-unused-ignore
• https://mypy.readthedocs.io/en/stable/error_code_list2.html#check-that-type-ignore-include-an-error-code-ignore-without-code

Then if the suppression comment is unused or incorrect, test will fail with a mypy error.

That's how typeshed does its static-type-checking tests.