Feature/Add Audit and Deletion by yau-wd · Pull Request #5994 · FlowiseAI/Flowise

yau-wd · 2026-03-17T09:59:46Z

FLOWISE-39

gemini-code-assist · 2026-03-17T10:00:14Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the application's logging and telemetry capabilities by introducing a robust audit logging system. It focuses on ensuring data privacy and compliance through new utilities for IP address validation, masking, and sensitive metadata redaction. The changes integrate these features into a structured event emission mechanism, allowing for detailed tracking of user actions and system events across various storage backends, supported by extensive unit tests.

Highlights

Audit Logging Integration: Introduced a new 'audit' log type across all storage providers (Azure Blob, Google Cloud Storage, Local Storage, and S3) and the IStorageProvider interface, enabling dedicated logging for auditable events.
IP Address Validation and Sanitization: Added new utility functions in ipValidation.ts for robust IP address validation (isValidIPAddress, isIPv4, isIPv6) and in sanitize.util.ts for privacy-compliant IP address masking (sanitizeIPAddress).
Sensitive Metadata Redaction: Implemented sanitizeAuditMetadata in sanitize.util.ts to automatically redact sensitive information (e.g., passwords, tokens) from event metadata before logging, enhancing data security and compliance.
Structured Telemetry Event Emission: Developed a new emitEvent function in telemetry.ts for sending structured audit/telemetry events. This function includes GeoIP lookup for location enrichment, followed by IP and metadata sanitization, ensuring comprehensive and compliant event logging.
New Dependencies and Tests: Added geoip-lite and its type definitions to packages/server/package.json to support geolocation. Comprehensive unit tests were introduced for the new IP validation, logger, sanitization, and telemetry utilities, and existing API key tests were expanded.

Changelog

packages/components/src/storage/AzureBlobStorageProvider.ts
- Reordered imports for consistency.
- Extended the getLoggerTransports method to include a new 'audit' log type.
- Added specific logic to handle 'audit' log transports using winstonAzureBlob for Azure Blob Storage.
packages/components/src/storage/BaseStorageProvider.ts
- Reordered imports for consistency.
- Updated the abstract getLoggerTransports method signature to incorporate the new 'audit' log type.
packages/components/src/storage/GCSStorageProvider.ts
- Reordered imports for consistency.
- Extended the getLoggerTransports method to include a new 'audit' log type.
- Added specific logic to handle 'audit' log transports using LoggingWinston for Google Cloud Storage.
packages/components/src/storage/IStorageProvider.ts
- Updated the getLoggerTransports interface method signature to support the new 'audit' log type.
packages/components/src/storage/LocalStorageProvider.ts
- Reordered imports for consistency.
- Extended the getLoggerTransports method to include a new 'audit' log type.
- Added specific logic to handle 'audit' log transports using DailyRotateFile for local storage.
packages/components/src/storage/S3StorageProvider.ts
- Reordered imports for consistency.
- Extended the getLoggerTransports method to include a new 'audit' log type.
- Added specific logic to handle 'audit' log transports using S3StreamLogger for S3 storage.
packages/server/package.json
- Added geoip-lite as a new dependency.
- Added @types/geoip-lite for TypeScript support.
packages/server/src/utils/ipValidation.ts
- Added a new file ipValidation.ts.
- Implemented isValidIPAddress to validate IPv4 and IPv6 addresses.
- Implemented isIPv4 to check for valid IPv4 addresses.
- Implemented isIPv6 to check for valid IPv6 addresses.
packages/server/src/utils/logger.ts
- Reordered imports for consistency.
- Initialized auditTransports using the storage provider's getLoggerTransports method.
- Exported a new auditLogger for dedicated audit logging.
packages/server/src/utils/sanitize.util.ts
- Imported isValidIPAddress for use in IP sanitization.
- Added sanitizeIPAddress to mask IP addresses for privacy compliance.
- Added sanitizeAuditMetadata to redact sensitive fields from audit event metadata.
packages/server/src/utils/telemetry.ts
- Imported geoip-lite, isValidIPAddress, logger, auditLogger, sanitizeAuditMetadata, and sanitizeIPAddress.
- Added getGeolocation function for GeoIP lookup.
- Defined TelemetryEventCategory and TelemetryEventResult enums.
- Defined TelemetryEventInput and TelemetryEventOutput interfaces for structured events.
- Implemented emitEvent for structured audit/telemetry event logging with enrichment and sanitization.
packages/server/test/utils/api-key.util.test.ts
- Added new test cases for generateSecretHash to verify hash format and uniqueness.
- Added new test cases for compareKeys to verify correct key comparison.
packages/server/test/utils/ipValidation.test.ts
- Added a new test file ipValidation.test.ts.
- Included comprehensive unit tests for isValidIPAddress, isIPv4, and isIPv6 functions.
packages/server/test/utils/logger.test.ts
- Added a new test file logger.test.ts.
- Included unit tests for the expressRequestLogger and the new auditLogger.
packages/server/test/utils/sanitize.util.test.ts
- Added a new test file sanitize.util.test.ts.
- Included unit tests for sanitizeNullBytes, sanitizeUser, sanitizeIPAddress, and sanitizeAuditMetadata.
packages/server/test/utils/telemetry.test.ts
- Added a new test file telemetry.test.ts.
- Included unit tests for the Telemetry class and the new emitEvent function.

Activity

The author yau-wd initiated this pull request.
New audit logging capabilities have been introduced across all storage providers.
Utility functions for IP validation and sanitization have been added and tested.
A structured telemetry event emission system, including GeoIP lookup and sensitive data redaction, has been implemented.
New dependencies for geolocation have been integrated.
Extensive unit tests have been added for the new features and modifications.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a comprehensive audit logging feature across various storage providers (Azure, GCS, S3, Local). It adds structured logging for audit events, including event details, user information, and sanitized metadata like IP addresses. The changes include new utility functions for IP validation and sanitization, along with extensive unit tests for the new functionality.

My review has identified a critical issue in the IP address sanitization logic that could lead to invalid data in audit logs, and a high-severity issue with the S3 log file naming strategy that would be inefficient and costly. I've provided suggestions to fix both.

packages/server/src/utils/sanitize.util.ts

packages/components/src/storage/S3StorageProvider.ts

…king

mmattu-wd

Still need to test the whole flow but left some comments

mmattu-wd · 2026-03-20T02:47:32Z

packages/server/src/enterprise/services/account.service.ts

+        await queryRunner.manager.save(User, user)
+
+        // Step 6: Cancel Stripe Subscription
+        await this.identityManager.cancelSubscription(organizaiton.subscriptionId)


should we move this after the commitTransaction? If for whatever reason the database doesn't commit, there is a case in which the user stays active without a subscription anymore

Why we didnt have try catch finally with rollbackTransaction and release same as other function at the top?

mmattu-wd · 2026-03-20T02:50:01Z

packages/ui/src/views/account/index.jsx

    const updateAdditionalSeatsApi = useApi(userApi.updateAdditionalSeats)
    const getCurrentUsageApi = useApi(userApi.getCurrentUsage)
    const logoutApi = useApi(accountApi.logout)
+    const deleteAccountApi = useApi(accountApi.deleteAccount)


should we confirm with a one time passcode or re-entering their password? Is there a chance an attacker can hijack a session and delete someone else's account?

for sso (google,github) login there isnt any passwords. from the standup we were thinking of something like a confirmation sentence: permanently delete or smtg like that

Ohh I see! I see google has something for this where you trigger OAuth for sensitive actions prompt=consent. But github does not, we'd have to revoke their OAuth and re-trigger it which isn't ideal.

The only other option would be a OTP sent to the email

mmattu-wd · 2026-03-20T02:51:03Z

packages/server/src/utils/telemetry.ts

+    result: TelemetryEventResult
+    metadata?: Record<string, any>
+}
+export interface TelemetryEventOutput {


There are some fields that currently not compatible with MALT. I will add these to the google doc as we will likely have to pull them from headers or something

mmattu-wd · 2026-03-20T02:57:32Z

packages/server/src/utils/telemetry.ts

+
+        auditLogger.log({ level: 'info', message: event.eventType, ...event })
+    } catch (error) {
+        logger.error(`Failed to emit event: ${error}`)


When an event fails to emit, should we add it to a queue to be retried? Not an issue rn, but for MALT if we're making http requests, it can become an issue

mmattu-wd · 2026-03-20T03:04:48Z

packages/server/src/utils/sanitize.util.ts

+
+    for (const key of Object.keys(sanitized)) {
+        const lowerKey = key.toLowerCase()
+        if (sensitiveFields.some((field) => lowerKey.includes(field))) {


this would only look 1 level into the object. Is there any chance data like this gets emitted?:

{ "configuration": { "apiKey": "someapikey" } }

if not we should maybe update the metadata type to be something like Record<string, string | number>. So if we do try logging and object ts catches it, this can be circumvented with any but 🤷 . As it stands right now the above metadata would pass and log the value

mmattu-wd · 2026-03-20T03:07:23Z

packages/server/src/enterprise/services/account.service.ts

+        await queryRunner.commitTransaction()
+
+        // Step 7: Delete Organization Folder from Storage
+        await removeFolderFromStorage(organizaiton.id)


if this throws an error the account would be removed but we wouldn't have emitted an event

HenryHengZJ · 2026-03-20T16:11:42Z

Steps:

Create User A and User B as Pro accounts.
User A creates Workspace 1 + Default Workspace.
User A invite User B to Default Workspace and Workspace 1. Important!! must invite to both workspaces
User B verify that he can see the invited Default Workspace and Workspace 1 Important!! must be able to see both invited workspaces
Let User B stays in User A's Default Workspace, on home page.
Now, User A delete account
Since User B is still in User A's Default Workspace, now when try to switch to User A's Workspace 1, will see 404 and thats expected:

Now, User B want to switch back to his own organization, thats when error happens:

2026-03-20 15:09:30 [INFO]: ⬆️ POST /api/v1/workspace/switch?id=83369a32-14c9-4595-8dae-d159b22dc050
2026-03-20 15:09:31 [ERROR]: uncaughtException:  Unhandled Edge Case
Error: Unhandled Edge Case
    at Statement.<anonymous> (C:\Users\Henry\Codes\Flowise\packages\server\dist\enterprise\controllers\workspace.controller.js:138:27)
    at Statement.<anonymous> (C:\Users\Henry\Codes\Flowise\node_modules\.pnpm\connect-sqlite3@0.9.15\node_modules\connect-sqlite3\lib\connect-sqlite3.js:131:32)
    at Statement.replacement (C:\Users\Henry\Codes\Flowise\node_modules\.pnpm\sqlite3@5.1.7\node_modules\sqlite3\lib\trace.js:25:27)
    at Statement.replacement (C:\Users\Henry\Codes\Flowise\node_modules\.pnpm\sqlite3@5.1.7\node_modules\sqlite3\lib\trace.js:25:27)

This is the part:

req.session.save((err) => {
                if (err) throw new InternalFlowiseError(StatusCodes.BAD_REQUEST, GeneralErrorMessage.UNHANDLED_EDGE_CASE)
            })

And database is now locked and every other operations will fail

feat(audit): add emitEvent and file-based audit logging

c969de9

yau-wd added the enhancement New feature or request label Mar 17, 2026

yau-wd marked this pull request as draft March 17, 2026 10:00

yau-wd changed the title ~~Feature/Add Audi and Deletion~~ Feature/Add Audit and Deletion Mar 17, 2026

gemini-code-assist bot reviewed Mar 17, 2026

View reviewed changes

packages/server/src/utils/sanitize.util.ts Outdated Show resolved Hide resolved

packages/components/src/storage/S3StorageProvider.ts Show resolved Hide resolved

yau-wd added 3 commits March 18, 2026 11:32

Merge branch 'main' into feature/audit-telemetry-gdpr-deletion

6c48aad

fix(sanitize.util.ts): correct IPv6 sanitization for audit log IP mas…

351acb4

…king

feat(account): add account deletion

0409703

yau-wd marked this pull request as ready for review March 18, 2026 16:00

yau-wd requested review from 0xi4o, HenryHengZJ, christopherholland-workday, harshit-flowise, igor-magun-wd and simon-song-wd March 18, 2026 16:00

igor-magun-wd approved these changes Mar 18, 2026

View reviewed changes

yau-wd requested a review from mmattu-wd March 19, 2026 10:31

Merge branch 'main' into feature/audit-telemetry-gdpr-deletion

9b249ff

FlowiseAI deleted a comment from wisdomdaoleo Mar 19, 2026

mmattu-wd reviewed Mar 20, 2026

View reviewed changes

Uh oh!

Conversation

yau-wd commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot commented Mar 17, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

mmattu-wd left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HenryHengZJ commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yau-wd commented Mar 17, 2026 •

edited

Loading

HenryHengZJ commented Mar 20, 2026 •

edited

Loading