Skip to content

fix(playwright): filter unsupported context options in persistent browser#1796

Open
sushant-mutnale wants to merge 1 commit intoapify:masterfrom
sushant-mutnale:fix/playwright-context-options
Open

fix(playwright): filter unsupported context options in persistent browser#1796
sushant-mutnale wants to merge 1 commit intoapify:masterfrom
sushant-mutnale:fix/playwright-context-options

Conversation

@sushant-mutnale
Copy link

This PR fixes issue #1784, where PlaywrightCrawler would crash when passing context options (like storage_state) that are unsupported by Playwright's launch_persistent_context method.

Changes:

Implemented dynamic argument filtering in PlaywrightPersistentBrowser.new_context using inspect. signature.
Added a warning log to guide users when options are filtered out, suggesting the use of incognito pages as an alternative.
Added a unit test in

tests/unit/browsers/test_playwright_browser.py
to verify the fix and prevent regressions.
Fixes #1784

…wser

This addresses issue apify#1784 by dynamically filtering options passed to launch_persistent_context and providing a warning log for ignored options like storage_state.
@janbuchar janbuchar requested a review from Pijukatel March 16, 2026 09:11
Copy link
Collaborator

@Pijukatel Pijukatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, thanks for the PR. Please see my comments; maybe we can use this approach on a different level.

"scraping",
]
dependencies = [
"apify-fingerprint-datapoints>=0.11.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have all these added dependencies in the optional dependencies group playwright. So please remove them from here.

user_data_dir = tempfile.mkdtemp(prefix=self._TMP_DIR_PREFIX)
self._temp_dir = Path(user_data_dir)

launch_persistent_context_sig = inspect.signature(self._browser_type.launch_persistent_context)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a reasonable approach, but it has some drawbacks. If user has just typo ( in otherwise valid argument name), it will just show warning in log. Same for using some completely nonsensical argument. That should raise an error and not just log a warning.

For example, this should raise (typo in headles):

    persist_browser = PlaywrightPersistentBrowser(
        playwright.chromium, browser_launch_options={'headles': True}
    )

Maybe this approach could be adopted one lever higher (not in PlaywrightPersistentBrowser - which always just calls launch_persistent_context), but in PlaywrightBrowserController - that is the class that decides about calling launch_persistent_context or new_context, but feeds them the same arguments.

It should properly raise exceptions for bad arguments, but it could just log a warning as per your suggestion for arguments at least valid in the other method. It would have to get 3 sets of arguments to be able to do such a distinction. Something like:

...
    launch_persistent_context_sig = set(inspect.signature(BrowserType.launch_persistent_context).parameters)
    new_context_sig = set(inspect.signature(Browser.new_context).parameters)
    persistent_unique_options = launch_persistent_context_sig - new_context_sig
    new_context_unique_options = new_context_sig - launch_persistent_context_sig
    common_options = launch_persistent_context_sig & new_context_sig
...

And then raise an exception or just log based on the selected mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PlaywrightCrawler __init__ method browser_new_context_options argument does not function

2 participants