Cookies are still deleted despite lower domain-level allowance

[Kein]

Prerequisites

• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
  • Your issue may already be reported.
  • I also searched the existing issues at https://github.com/gorhill/uMatrix/issues
• This is not a support issue or a question
  • Support issues and questions are handled at /r/uMatrix
• I tried to reproduce the issue when...
  • uMatrix extension is wholly disabled or not installed
  • uMatrix is the only extension
  • uMatrix with default lists/settings
  • using a new, unmodified browser profile
• I am running the latest version of uMatrix
• I checked the documentation to understand that the issue I report is not a normal behavior
• I used the logger to rule out that the issue is caused by my ruleset

Description

Cookies deleted on timed manner even for allowed domains

A specific URL where the issue occurs

Any. In this particular case forums.unity.com

Steps to Reproduce

1. Go to forums.unity.com (by any resource, really)
2. Login
3. Close the tab and monitor the logger for info events

Ruleset

Global scope level: domain

cname-reveal: * true
https-strict: behind-the-scene false
matrix-off: about-scheme true
matrix-off: behind-the-scene true
matrix-off: chrome-extension-scheme true
matrix-off: chrome-scheme true
matrix-off: localhost true
matrix-off: moz-extension-scheme true
matrix-off: opera-scheme true
matrix-off: vivaldi-scheme true
matrix-off: wyciwyg-scheme true
no-workers: * true
no-workers: youtube.com false
noscript-spoof: * true
referrer-spoof: * true
referrer-spoof: behind-the-scene false
* * * block
* * css allow
* * frame block
* * image allow
* 1st-party * allow
* 1st-party cookie block
* 1st-party frame allow
* 1st-party script block
127.0.0.1 1st-party script allow
unity.com 1st-party cookie allow
unity.com 1st-party script allow

Supporting evidence

Your environment

• uMatrix version: 1.4.10b6
• Browser Name and version: Vivaldi 3.1/Chromium 83
• Operating System and version: Win7x64

Additional notes

According to the wiki and option description:

Blacklisted cookies are not prevented by uMatrix from entering your browser. However they are prevented from leaving your browser, which is what really matters. Not blocking cookies before they enter your browser gives you the opportunity to be informed that a site tried to use cookies, and furthermore to inspect their contents if you wish.
Once these blacklisted cookies have been accounted for by uMatrix, you can ask uMatrix to remove them from your browser if you wish so: just check the setting "Delete blocked cookies" in the Privacy tab.

I read it multiple times trying to imagine all possible meanings I might be missing (which is already very bad thing to begin with - option description shouldn't call for a meeting of wise men to debate what this or that means) but in the end I always come back to what seems logical and most reasonable - these domains that do not have rule allowing cookies out will be cleared.
My understanding is that when this feature was developed it sure was thought thru to flatten-out rules to make sure all checks are passed not just top level * * * block, so if I block cookies globally, but then allow for domain.com via 1st party rule - this will be the final flattened rule that is taken into account and therefore on the next iteration of a clean-up the cookie will be kept.
However, according to my observation and logger data that is not the case. Local storage also suffers form the same issue.

P.S. It would be nice to add a clarification how are allowed 3d party cookies processed. For example, if I allow google.com 3dparty cookies for youtube.com but block 1st party cookies on google.com - what the action will be performed during deletion, which will be kept or deleted?
P.P.S. Screenshot of the switchboard says 1.4.1b0 - ignore that, the shots are made on that version but later after an update on b6 the same issues were observed so it was tested on b6 as well.

These blue entries in the logger are not what uMatrix is doing. These are informative entries - they log everything is happening with cookies. Cookies removed by uM will have uppercase COOKIE in type column and will be red.

Also, your screenshot shows "{session-cookie:...}". Session cookies are temporary - they should disappear when you close browser at the latest (end of browsing "session"), but may also be removed when tab is closed.

[Kein]

The only extension I have that manages cookies is uMatrix. I tested it extensively with other extensions and standalone for about 2 weeks.
Last 2 days I started to test how cookie management works. This is the first time this year I've got logged out from majority of domains (including inoreader which is like never expires for centuries) despite the fact ALL of them were allowed for 1st party cookies.

If you are stating I'm wrong in my conclusion do elaborate. You are saying my conclusion is invalid despite the fact the only thing that's changed is uMatrix global option for cookie deletion. If I did the test wrong tell me how could I test it properly? Is there a way for me to trigger the clean up even from extension background page console? Any other way?

The issue is real on my end and I'm willing to put time to investigate it.

[Kein]

Cookies removed by uM will have uppercase COOKIE in type column and will be red.

Are you sure? I'm seeing only one message for cookie log:

https://github.com/gorhill/uMatrix/blob/master/src/js/cookies.js#L270
https://github.com/gorhill/uMatrix/blob/9b292304d33a44465922200efa5f8b378d0f9604/src/_locales/en/messages.json#L311

UPDATE: I went through all source code related to cookies and all locale messages and the deletion messages and nothing of what you say matches current codebase. There is a generic cookie removal line for every cookie removal. cookie.js manager fallbacks for some vAPI functional but it is pretty basic.

Given the circumstances I'd appreciate if you could unlock the issue instead of straight jumping into assumptions. Thank you.

nothing of what you say matches current codebase

You can be right.

uMatrix version: 1.4.10b6

BTW, I was talking about this: https://github.com/gorhill/uMatrix/blob/0bcb7669e77adc958ee66a97fe9172898cb8131d/src/js/cookies.js#L496

[gorhill]

The logger reports cookies which may or may not have been removed by uMatrix -- you can verify this by deleting the cookies using the browser's own UI. The COOKIE entries is to denote removed outgoing COOKIE headers.

So far I have been unable to reproduce the reported issue on my side. I am not using unity.com to try to reproduce, but I tried with logged in https://news.ycombinator.com/news and not logged in https://github.com/brave/brave-browser/issues/9929 (which still delivers session cookeis).

[Kein]

BTW, I was talking about this: https://github.com/gorhill/uMatrix/blob/0bcb7669e77adc958ee66a97fe9172898cb8131d/src/js/cookies.js#L496

I saw this, yeah. Problem is, I have a tons of cookies ready to be expired yet the only cookies that are being removed in the log is the one I've visited recently. I've got logged out out of Inoreader.com again just as I'e been active here, on in this issue, even though I've explicitly set every first party domain to allow. From my understanding this SHOULD not happen.

Here are my cookies for Inoreader.com

The logger for All Tabs with filter info had only one message entry stating that only consents_cookie_use was removed, but it makes no sense, that cookie expires in like 2 years, I have no more extensions that delete or manage cookies and browser settings set to always keep 1st party cookies. I think it might have swallowed other log entries, not sure. It really makes no sense to me.

userSettings:

alwaysDetachLogger: false
autoUpdate: true
clearBrowserCache: true
clearBrowserCacheAfter: 120
cloudStorageEnabled: false
collapseBlacklisted: true
collapseBlocked: false
colorBlindFriendly: false
deleteCookies: true
deleteLocalStorage: false
deleteUnusedSessionCookies: false
deleteUnusedSessionCookiesAfter: 60
displayTextSize: "13px"
externalHostsFiles: []
externalRecipeFiles: []
iconBadgeEnabled: true
noTooltips: false
popupCollapseAllDomains: false
popupCollapseBlacklistedDomains: false
popupScopeLevel: "domain"
processHyperlinkAuditing: true
selectedHostsFiles: Array(6)
0: "malware-0"
1: "malware-1"
2: "dpollock-0"
3: "hphosts"
4: "mvps-0"
5: "plowe-0"
selectedRecipeFiles: Array(1)
0: "recipes_en-0"
length: 1
userHosts:
content: ""
enabled: false
userRecipes:
content: ""
enabled: false

[gorhill]

What is your setting regarding "Delete non-blocked session cookies [60] minutes after the last time they have been used"?

[Kein]

It is false, I've updated previous comment with raw saved values straight from background page. As a matter of fact it never was enabled, even for testing (didnt come to this yet).

[gorhill]

What are your rules re. inoreader.com?

[Kein]

Base:

cname-reveal: * true
https-strict: behind-the-scene false
matrix-off: about-scheme true
matrix-off: behind-the-scene true
matrix-off: chrome-extension-scheme true
matrix-off: chrome-scheme true
matrix-off: localhost true
matrix-off: moz-extension-scheme true
matrix-off: opera-scheme true
matrix-off: vivaldi-scheme true
matrix-off: wyciwyg-scheme true
no-workers: * true
no-workers: youtube.com false
noscript-spoof: * true
referrer-spoof: * true
referrer-spoof: behind-the-scene false
* * * block
* * css allow
* * frame block
* * image allow
* 1st-party * allow
* 1st-party cookie block
* 1st-party frame allow
* 1st-party script block
127.0.0.1 1st-party script allow

Were at the moment of creation of this issue:

inoreader.com 1st-party cookie allow
inoreader.com 1st-party script allow

Now I've explicitly set every single 1st party domain

inoreader.com inoreader.com cookie allow
inoreader.com www.inoreader.com cookie allow

However I had .steampowered.com being axed as well despite explicitly allowed.

[gorhill]

Is stepping into the code using the debugger something you are comfortable to do? There are only two places where uMatrix explicitly asks the browser to remove cookies, so maybe putting a breakpoint at these lines will provide useful additional info:

• https://github.com/gorhill/uMatrix/blob/b26b3bb9609f9eae1e3ace48170b5da1aa2fdf98/src/js/cookies.js#L333
• https://github.com/gorhill/uMatrix/blob/b26b3bb9609f9eae1e3ace48170b5da1aa2fdf98/src/js/cookies.js#L350