Skip to content

Data explorer enhancements#6

Open
shahaash wants to merge 4 commits intomainfrom
data-explorer-enhancements
Open

Data explorer enhancements#6
shahaash wants to merge 4 commits intomainfrom
data-explorer-enhancements

Conversation

@shahaash
Copy link
Copy Markdown
Collaborator

  • Added a Show Aggregation Logs filter for the Connection, DNS, HTTP, Files, and SSL dashboards.
  • Used parameters to implement the Show Aggregation Logs filter, allowing control over whether panels update based on the selected parameter.
  • Updated panel queries to reflect these changes.
  • Updated parsers to align with the latest field changes.

@shahaash shahaash requested a review from jspackets April 10, 2026 12:36
Copy link
Copy Markdown
Collaborator

@jspackets jspackets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shahaash Please take a look at my comments and update accordingly if observations are valid for updating

"graphs":[
{
query: "| join\n(metadata.product.vendor_name = 'Corelight' AND metadata.log_name = 'ssl' | columns uid, \"ssl_subject\"=subject, \"ssl_subject_common_name\"=server_name),\n(metadata.product.vendor_name = 'Corelight' AND metadata.log_name = 'conn' | let direction= (local_orig=true AND local_resp=true) ? \"internal\" : (local_orig=true AND local_resp=false) ? \"outbound\" : (local_orig=false AND local_resp=false) ? \"external\" : (local_orig=false AND local_resp=true) ? \"inbound\" : \"unknown\" | filter direction=* | columns uid) on uid\n| let c_id= (ssl_subject != null) ? ssl_subject : (ssl_subject_common_name != null) ? ssl_subject_common_name : \"unknown\"\n| group \"Count\"=count(), \"Percent\"=percent_of_total(count()) by \"Certificate ID\"=c_id\n| sort -Count\n| limit 10",
query: "| join\n(#Show Aggregation Logs#\nmetadata.product.vendor_name = 'Corelight'\n| columns uid, \"ssl_subject\"=subject, \"ssl_subject_common_name\"=server_name),\n(metadata.product.vendor_name = 'Corelight' AND metadata.log_name = 'conn' | let direction= (local_orig=true AND local_resp=true) ? \"internal\" : (local_orig=true AND local_resp=false) ? \"outbound\" : (local_orig=false AND local_resp=false) ? \"external\" : (local_orig=false AND local_resp=true) ? \"inbound\" : \"unknown\" | filter direction=* | columns uid) on uid\n| let c_id= (ssl_subject != null) ? ssl_subject : (ssl_subject_common_name != null) ? ssl_subject_common_name : \"unknown\"\n| filter c_id != \"unknown\"\n| group \"Count\"=count(), \"Percent\"=percent_of_total(count()) by \"Certificate ID\"=c_id\n| sort -Count\n| limit 10",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe ssl_subject is available in ssl, ssl_red, and ssl_agg parsers. If so, then recommendation to change:
FROM: | columns uid, "ssl_subject"=subject
TO: | columns uid, ssl_subject

x: 11,
y: 0
},
query: "#Show Aggregation Logs#\nmetadata.product.vendor_name = 'Corelight'\n| group estimate_distinct(id.orig_h)",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conn parsers map id.orig_h → src_endpoint.ip and id.resp_h → dst_endpoint.ip; those field names may not be present as top-level query fields after mapping.

Should they be changed to estimate_distinct(src_endpoint.ip) and estimate_distinct(dst_endpoint.ip) ?

x: 33,
y: 0
},
query: "#Show Aggregation Logs#\nmetadata.product.vendor_name = 'Corelight'\n| group estimate_distinct(id.resp_h)",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conn parsers map id.orig_h → src_endpoint.ip and id.resp_h → dst_endpoint.ip; those field names may not be present as top-level query fields after mapping.

Should they be changed to estimate_distinct(src_endpoint.ip) and estimate_distinct(dst_endpoint.ip) ?

maxPieSlices: 15,
query: "metadata.product.vendor_name = 'Corelight' AND metadata.log_name = 'conn'\n| let is_broadcast = (src_endpoint.ip = \"0.0.0.0\" OR src_endpoint.ip = \"255.255.255.255\" OR dst_endpoint.ip = \"0.0.0.0\" OR dst_endpoint.ip = \"255.255.255.255\") ? true : false, service = (service = null) ? \"Unknown\" : service\n| filter is_broadcast != true service != 'Unknown'\n| group count=count() by service, dst_endpoint.port, src_endpoint.ip, dst_endpoint.ip\n| group \"Total\"=sum(count) by service\n| sort -Total\n| limit 15",
query: "#Show Aggregation Logs#\nmetadata.product.vendor_name = 'Corelight'\n| let is_broadcast = (src_endpoint.ip = \"0.0.0.0\" OR src_endpoint.ip = \"255.255.255.255\" OR dst_endpoint.ip = \"0.0.0.0\" OR dst_endpoint.ip = \"255.255.255.255\") ? true : false, service = (service = null) ? \"Unknown\" : service, extracted_services = service.extract_matches('[A-Za-z0-9]+'), expanded_services = extracted_services.expand()\n| filter is_broadcast != true expanded_services != 'Unknown'\n| group count=count() by expanded_services, dst_endpoint.port, src_endpoint.ip, dst_endpoint.ip\n| group \"Total\"=sum(count) by expanded_services\n| sort -Total\n| limit 15",
teamEmails: ["19488_1342074829499534781@s1.oem", "19488_1@s1.oem"],
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

teamEmails lists look like tenant-specific OEM identifiers in repo content. Is this needed ? Could it be replaced with placeholders ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing teamEmails list from dashboard as it is not affecting anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants