Log forwarding from MuleSoft Application to Datadog

Overview

In this blog, I will try to showcase how we can send logs to Datadog from the MuleSoft application using HTTP Appender.

What’s Datadog?

• Datadog is a monitoring service for cloud-scale applications. It provides monitoring of servers, databases, tools, and services through a SaaS-based data analytics platform.
• Visit https://www.datadoghq.com/ for more details.

Enable Custom Logging For MuleSoft Application:

• Go to your mule application/project → Open src/main/resoruces/log4j2.xml
• Add HTTP Appender (as below)

<Http name=”datadog” url=”https://http-intake.logs.us5.datadoghq.com/v1/input?host=${sys:hostName}&amp;ddsource=MuleSoft&amp;service=ExampleApp&amp;ddtags=env:Dev“>
<Property name=”Accept” value=”application/json” />
<Property name=”Content-Type” value=”application/json” />
<Property name=”DD-API-KEY” value=”{API-KEY}” />
<PatternLayout pattern=”%-5p %d [%t] [event: %X{correlationId}] %c: %m%n” />
</Http>

<! — Please add appender reference (ex: ref=”datadog”) →

<Loggers>

<AsyncRoot level=”INFO”>

<AppenderRef ref=”file” />

<AppenderRef ref=”datadog” level=”ERROR” />

</AsyncRoot>

</Loggers>

Note:

Users can externalise some of the parameters on a per-project needs basis:

URL: https://http-intake.logs.us5.datadoghq.com/v1/input

It is hosted on US5 region

Ex: Host, ddsource, service, ddtags

Host: << Host Name >>

ddsource: << Define your source >>

service: << MuleApplicationName >>

ddtags: << Add your environment name and other parameters >>

Log Level: ERROR (Only ERROR Log level will be forwarded to Datadog because appender ref has level=”ERROR”)

• Run Mulesoft Application and validate logs

1. Validate Application console (It will have all log levels information ex: INFO, WARN, ERROR)

INFO 2024–08–22 14:24:20,537 [[MuleRuntime].uber.02: [z-zdatalog].z-zdatalogFlow.CPU_INTENSIVE @649d4b8b] [processor: z-zdatalogFlow/processors/1; event: 1ef82630–6064–11ef-9791-a22b20eb48b7] z-zdatalog: {

“message”: “Entry block — Akash Prajapati”

}

WARN 2024–08–22 14:24:20,554 [[MuleRuntime].uber.02: [z-zdatalog].z-zdatalogFlow.CPU_INTENSIVE @649d4b8b] [processor: z-zdatalogFlow/processors/2; event: 1ef82630–6064–11ef-9791-a22b20eb48b7] org.mule.extension.http.internal.request.HttpRequestFactory: Body is ignored since the HTTP Method is between the empty body methods ([HEAD, GET, OPTIONS]), and the Send Body Mode is not set to ‘ALWAYS’. Future warnings like this will be suppressed in order to avoid performance degradations

ERROR 2024–08–22 14:24:23,588 [[MuleRuntime].uber.11: [z-zdatalog].uber@org.mule.runtime.core.privileged.processor.chain.AbstractMessageProcessorChain.initialise:750 @20a9decb] [processor: z-zdatalogFlow/errorHandler/0/processors/0; event: 1ef82630–6064–11ef-9791-a22b20eb48b7] z-zdatalog: “HTTP GET on resource ‘http://www.abc.com/main’ failed: not found (404).”

2. Validate Datadog Log Explorer (It will have only log-level information for ERROR)

ERROR 2024–08–22 14:24:23,588 [[MuleRuntime].uber.11: [z-zdatalog].uber@org.mule.runtime.core.privileged.processor.chain.AbstractMessageProcessorChain.initialise:750 @20a9decb] [event: 1ef82630–6064–11ef-9791-a22b20eb48b7] z-zdatalog: “HTTP GET on resource ‘http://www.abc.com/main’ failed: not found (404).”

Conclusions

This will be useful when you would like to forward your log to external log monitoring tools ex: Datadog