mss.grab() is blocking indefinitely after a certain amount of usage

[cjshrader]

Hey there,

I maintain an open source program that helps with Diablo 4 loot filtering. As part of this, we read the screen using mss very often. We are Windows only.

After 30 minutes to an hour, the program will freeze up entirely. I've been able to trace where it's getting stuck to this code in mss's base.py:

        with lock:
            screenshot = self._grab_impl(monitor)
            if self.with_cursor and (cursor := self._cursor_impl()):
                return self._merge(screenshot, cursor)
            return screenshot

Specifically on the with lock: step.

I will go through all of our code in case it is beneficial, but at a high level the user presses a hotkey which kicks off a thread. That thread will read the screen to see if a certain menu is open. It's at that point that everything gets blocked, with no errors or any interaction. A hotkey to kill that thread works fine, but running it again and every time after that will always be blocked.

I believe we are calling mss correctly:

        with mss.mss() as sct:
            img = np.array(sct.grab(self.window_roi))

We then perform handling on the image that we receive.

Here is the full flow with lines from our code in case any of that is beneficial:

First, the user presses a hotkey. We'll use for example the hotkey to move items from inventory to stash. It kicks off a thread in main, seen here:

https://github.com/d4lfteam/d4lf/blob/27630ecbf3d121cc4253dea815f10251f5cc71b5/src/scripts/handler.py#L72

                    self.loot_interaction_thread = threading.Thread(
                        target=self._wrapper_run_loot_interaction_method,
                        args=(loot_interaction_method, method_args),
                        daemon=True,
                    )
                    self.loot_interaction_thread.start()

Which I will shortcut some of this, but it calls loot_mover.py (https://github.com/d4lfteam/d4lf/blob/27630ecbf3d121cc4253dea815f10251f5cc71b5/src/loot_mover.py#L23) which calls stash.is_open() (https://github.com/d4lfteam/d4lf/blob/27630ecbf3d121cc4253dea815f10251f5cc71b5/src/ui/menu.py#L55) which calls detect (https://github.com/d4lfteam/d4lf/blob/27630ecbf3d121cc4253dea815f10251f5cc71b5/src/template_finder.py#L72) which calls Cam().grab() which is where we call mss:

https://github.com/d4lfteam/d4lf/blob/27630ecbf3d121cc4253dea815f10251f5cc71b5/src/cam.py#L79

    def grab(self, force_new: bool = False) -> np.ndarray:
        if (
            not force_new
            and self.cached_img is not None
            and self.last_grab is not None
            and time.perf_counter() - self.last_grab < 0.04
        ):
            return self.cached_img

        # wait for offsets to be found
        if not self.is_offset_set():
            LOGGER.debug("Wait for window detection")
            while not self.window_offset_set:
                time.sleep(0.05)
            LOGGER.debug("Found window, continue grabbing")
        with cached_img_lock:
            self.last_grab = time.perf_counter()
        with mss.mss() as sct:
            img = np.array(sct.grab(self.window_roi))
        with cached_img_lock:
            self.cached_img = img[:, :, :3]
        return self.cached_img

I don't think there's anything particularly strange about this path, but at this point it will be blocked by the sct.grab(...) line as I mentioned above. No attempt to grab the image will work after this.

I am hoping the code usage maybe suggests something to you that is not obvious to me. I appreciate your time. Let me know if there's more information I can provide.

[BoboTiG]

Hello @cjshrader,

Thanks for the detailled report. We suspect that #446 will help in your case.

I'll keep you updated!

[cjshrader]

Thank you for such a fast response! I suspect you are correct.