Skip to content

Serialize MDC under top level - #67

Merged
felixbarny merged 3 commits into
elastic:masterfrom
felixbarny:mdc-top-level
Mar 31, 2020
Merged

felixbarny merged 3 commits into
elastic:masterfrom
felixbarny:mdc-top-level

Conversation

@felixbarny

Copy link
Copy Markdown
Member

Makes it easier to set ECS fields via MDC.
Previously, you'd have to configure topLevelLabels for each ECS field added via MDC.
Now, if you want to have a field nested under labels, you have to explicitly prefix it with labels..

This increases the chance of conflicts, for example when the user adds an @timestamp MDC value. But it also gets out of the way for legitimate use cases.

Having to whitelist all MDC values that should be serialized at the top level via topLevelLabels is cumbersome and feels inconsistent. I think it's overly protective.

Makes it easier to set ECS fields via MDC.
Previously, you'd have to configure topLevelLabels for each ECS field added via MDC.
Now, if you want to have a field nested under labels, you have to explicitly prefix it with `labels.`.
@felixbarny
felixbarny requested a review from xeraa March 30, 2020 06:48
@felixbarny

Copy link
Copy Markdown
Member Author

@pgomulka would this break anything in Elasticsearch?

@xeraa xeraa left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a quick look and I think this makes a lot of sense (also in documenting how to use this)

@felixbarny
felixbarny merged commit d036082 into elastic:master Mar 31, 2020
@felixbarny
felixbarny deleted the mdc-top-level branch March 31, 2020 07:23
@pgomulka

Copy link
Copy Markdown
Contributor

@pgomulka would this break anything in Elasticsearch?

@felixbarny ES is not using MDC, so should be safe.

@lanore78

lanore78 commented Jul 7, 2020

Copy link
Copy Markdown

With this change, I'm having a problem with collecting log with Filebeat.

It's because some library produce 'url' MDC which is one of ECS key, and the formate is not compatible with ECS(or Elasticsearch)

Here is example value of 'url' MDC, and I got error when I collect this log with Filebeat.
Is there any good idea to solve this problem?

{ ...
"url" : "jdbc:sqlserver://xxx.xxx.xxx.xxx:61051;authenticationScheme=nativeAuthentication;xopenStates=false;sendTimeAsDatetime=true;trustServerCertificate=false;sendStringParametersAsUnicode=false;selectMethod=direct;responseBuffering=adaptive;packetSize=8000;multiSubnetFailover=false;loginTimeout=20;lockTimeout=-1;lastUpdateCount=true;encrypt=false;disableStatementPooling=true;databaseName=USER;applicationName=Microsoft JDBC Driver for SQL Server;applicationIntent=readwrite",
...
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants