Skip to content

Commit 0b95e36

Browse files
authored
macros: hard and soft macros explanation adjustments, added some missing macro references (#303)
2 parents ba69c83 + 0a85e2c commit 0b95e36

6 files changed

Lines changed: 61 additions & 25 deletions

File tree

_data/link_aliases.yml

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ adm-temp-macro-ose#sdata-sdatasdidsdname:
5555
aliases: [ "${SDATA}" ]
5656

5757
adm-temp-macro-ose#sec-c_sec-r_sec-s_sec:
58-
aliases: [ "/${SEC}|\\${[CRS]_SEC}/" ]
58+
aliases: [ "/\\${SEC}|\\${[CRS]_SEC}/" ]
5959

6060
adm-temp-macro-ose#stamp-r_stamp-s_stamp:
6161
aliases: [ "/\\${STAMP}|\\${[RS]_STAMP}/" ]
@@ -67,7 +67,7 @@ adm-temp-macro-ose#tzoffset-c_tzoffset-r_tzoffset-s_tzoffset:
6767
aliases: [ "/\\${TZOFFSET}|\\${[CRS]_TZOFFSET}/" ]
6868

6969
adm-temp-macro-ose#unixtime-c_unixtime-r_unixtime-s_unixtime:
70-
aliases: [ "/$\\{UNIXTIME}|\\${[CRS]_UNIXTIME}/" ]
70+
aliases: [ "/\\${UNIXTIME}|\\${[CRS]_UNIXTIME}/" ]
7171

7272
adm-temp-macro-ose#usec-c_usec-r_usec-s_usec:
7373
aliases: [ "/\\${USEC}|\\${[CRS]_USEC}/" ]
@@ -90,6 +90,9 @@ adm-temp-macro-ose#week_day_name-c_week_day_name-r_week_day_name-s_week_day_name
9090
adm-temp-macro-ose#year-c_year-r_year-s_year:
9191
aliases: [ "/\\${YEAR}|\\${[CRS]_YEAR}/" ]
9292

93+
adm-temp-macro-ose#year_day-c_year_day-r_year_day-s_year_day:
94+
aliases: [ "/\\${YEAR_DAY}|\\${[CRS]_YEAR_DAY}/" ]
95+
9396
adm-log-flow#log-paths-with-flow-control:
9497
aliases: [ "Managing incoming and outgoing messages with flow-control" ]
9598

@@ -261,6 +264,12 @@ adm-correlate:
261264
adm-stats:
262265
aliases: [ "syslog-ng-ctl stats", "syslog-ng-ctl query" ]
263266

267+
adm-temp-hard-soft-macros#hard-macros:
268+
aliases: [ "/[Hh]ard [Mm]acros/" ]
269+
270+
adm-temp-hard-soft-macros#soft-macros:
271+
aliases: [ "/[Ss]oft [Mm]acros/" ]
272+
264273
doxygen-home:
265274
aliases: [ "/[Dd]oxygen/" ]
266275

doc/_admin-guide/110_Template_and_rewrite/000_Customize_message_format/003_Hard_vs_soft_macros.md

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,27 @@
11
---
22
title: Hard versus soft macros
3-
id: adm-temp-hard-soft-macro
3+
id: adm-temp-hard-soft-macros
44
description: >-
5-
Hard macros contain data that is directly derived from the log message,
6-
for example, the ${MONTH} macro derives its value from the timestamp.
7-
Hard macros are read-only. Soft macros (sometimes also called name-value
8-
pairs) are either built-in macros automatically generated from the log
9-
message (for example, ${HOST}), or custom user-created macros generated
10-
by using the {{ site.product.short_name }} pattern database or a CSV-parser. In contrast to
11-
hard macros, soft macros are writable and can be modified within
12-
{{ site.product.short_name }}, for example, using rewrite rules.
5+
This section describes the differences between hard and soft macros in {{ site.product.short_name }}.
136
---
147

8+
## Hard macros
9+
10+
Hard macros contain data that is directly derived from the log message,
11+
for example, the ${MONTH} macro derives its value from the timestamp.
12+
Hard macros are read-only.
13+
14+
## Soft macros
15+
16+
Soft macros (sometimes also called name-value
17+
pairs) are either built-in macros automatically generated from the log
18+
message (for example, ${HOST}), or custom user-created macros generated
19+
by using the {{ site.product.short_name }} pattern database or a CSV-parser. In contrast to
20+
hard macros, soft macros are writable and can be modified within
21+
{{ site.product.short_name }}, for example, using rewrite rules.
22+
23+
## Comparison of hard and soft macros
24+
1525
Hard and soft macros are rather similar and often treated as equivalent.
1626
Macros are most commonly used in filters and templates, which does not
1727
modify the value of the macro, so both soft and hard macros can be used.
@@ -31,6 +41,7 @@ ${PROGRAM}, ${SOURCE}. Custom values created using rewrite rules or parsers
3141
can be modified as well, just like stored matches of regular expressions
3242
($0 \... $255).
3343

34-
Note that you can modify the timezone of the message, and change the
44+
**NOTE:** You can modify the timezone of the message, and change the
3545
timezone-related macros that way. For details, see
3646
Rewrite the timezone of a message.
47+
{: .notice--info}

doc/_admin-guide/110_Template_and_rewrite/000_Customize_message_format/004_Macros_of_syslog-ng.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@ facility letter can range from **A** to **Y**, where A corresponds to
3737
facility number zero (LOG_KERN), B corresponds to facility 1
3838
(LOG_USER), and so on.
3939

40+
## ${CONTEXT_ID}
41+
42+
The context_id for the pattern database rule that matched the message. see the context-id, context-timeout, and context-scope attributes of pattern database rules.
43+
4044
## Custom macros
4145

4246
CSV parsers and pattern databases can also define macros
@@ -155,6 +159,10 @@ ${ISOWEEK} macro.
155159

156160
Available in 3.24 and later.
157161

162+
## ${LEGACY_MSGHDR}
163+
164+
During default operation, {{ site.product.short_name }} stores the original incoming header of the log message. This is useful if the original format of a non-syslog-compliant message must be retained (as {{ site.product.short_name }} automatically corrects some non-compliant messages, the final result may slightly differ from the original).
165+
158166
## ${LEVEL_NUM}
159167

160168
The priority (also called severity) of the message,
@@ -533,6 +541,10 @@ Available in {{ site.product.short_name }} version 3.4 and later.
533541

534542
The year the message was sent.
535543

544+
## ${YEAR_DAY}, ${C_YEAR_DAY}, ${R_YEAR_DAY}, ${S_YEAR_DAY}
545+
546+
The day of the year the message was sent.
547+
536548
## ${WEEK}, ${C_WEEK}, ${R_WEEK}, ${S_WEEK}
537549

538550
The week number of the year, prefixed with a zero for the

doc/_admin-guide/120_Parser/004_CSV_parser/000_CSV_parser_options.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Options of CSV parsers
33
parser: csv
4+
prefix: empty
45
id: adm-parser-csv-opt
56
description: >-
67
This section describes the options of the csv-parser() in {{ site.product.short_name }}.

doc/_admin-guide/120_Parser/010_JSON_parser/000_JSON_parser_options.md

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,36 @@
11
---
22
title: Options of JSON parsers
33
parser: json
4+
prefix: empty
45
id: adm-parser-json-opt
56
description: >-
6-
This section describes the options of the json-parser() in {{ site.product.short_name }}.
7+
This section explains how to configure JSON parsing with extract-prefix() for subtree extraction, key-delimiter() for custom delimiters, marker() for identifying JSON messages, and prefix() for name-value pair prefixing in {{ site.product.short_name }}.
78
---
89

910
The JSON parser has the following options.
1011

1112
## extract-prefix()
1213

13-
| Synopsis: | extract-prefix() |
14+
| Type: | string |
15+
| Default: | |
1416

1517
*Description:* Extract only the specified subtree from the JSON message.
1618
Use the dot-notation to specify the subtree. The rest of the message
1719
will be ignored. For example, assuming that the incoming object is named
18-
msg, the json-parser(extract-prefix(\"foo.bar\[5\]\")); parser is
19-
equivalent to the msg.foo.bar\[5\] javascript code. Note that the
20+
`msg`, the `json-parser(extract-prefix("foo.bar[5]"));` parser is
21+
equivalent to the `msg.foo.bar[5]` javascript code. Note that the
2022
resulting expression must be a JSON object in order to extract its
2123
members into name-value pairs.
2224

2325
This feature also works when the top-level object is an array, because
2426
you can use an array index at the first indirection level, for example:
25-
json-parser(extract-prefix(\"\[5\]\")), which is equivalent to msg\[5\].
27+
`json-parser(extract-prefix("[5]"))`, which is equivalent to `msg[5]`.
2628

2729
In addition to alphanumeric characters, the key of the JSON object can
2830
contain the following characters:
29-
!\"\#$%&\'()\*+,-/:;\<=\>?@\\\^\_\`{\|}\~
31+
`` !"#$%&'()*+,-/:;<=>?@\^_`{|}~ ``
3032

31-
It cannot contain the following characters: .\]\[
33+
It cannot contain the following characters: `.][`
3234

3335
### Example: Convert logstash eventlog format v0 to v1
3436

@@ -56,22 +58,22 @@ parser p_jsoneventv0 {
5658
## key-delimiter()
5759

5860
| Type: | character |
59-
| Default: | . |
61+
| Default: | `.` |
6062

6163
*Description:* The key-delimiter() option defines the used character when parsing flattened keys. Only single characters are supported.
6264

63-
Using the json-parser() without the key-delimiter() option, results in the dot(.) character being used:
65+
Using the json-parser() without the key-delimiter() option, results in the dot `.` character being used:
6466

6567
```config
6668
foo.key="value"
6769
```
6870

6971
Using the json-parser() with the key-delimiter("~") option, results in the specified character being used:
7072

71-
7273
## marker()
7374

74-
| Synopsis: | marker() |
75+
| Type: | string |
76+
| Default: | |
7577

7678
*Description:* Use a marker in case of mixed log messages, to identify
7779
JSON encoded messages for the parser.
@@ -82,8 +84,8 @@ message only if it is present.
8284

8385
### Example: Using the marker option in JSON parser
8486

85-
This json parser parses log messages which use the \"@cee:\" marker in
86-
front of the json payload. It inserts \".cee.\" in front of the name of
87+
This json parser parses log messages which use the `@cee:` marker in
88+
front of the json payload. It inserts `.cee.` in front of the name of
8789
name-value pairs, so later on it is easier to find name-value pairs that
8890
were parsed using this parser. (For details on selecting name-value
8991
pairs, see value-pairs()

doc/_admin-guide/120_Parser/019_regexp_parser/000_regex_parser_options.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Options of Regular expression parsers
33
parser: regexp
4+
prefix: empty
45
id: adm-parser-regexp-opt
56
description: >-
67
This section describes the options of the regexp-parser() in {{ site.product.short_name }}.

0 commit comments

Comments
 (0)