Send NGINX logs to Graylog

I suggest you collect some debug info: As you said your network work

TCP Dump packets on the Graylog host:

if Nginx on the same host as Graylog: sudo tcpdump udp -n -vv port 12301 -i lo -X

if Nginx on a different host as Graylog: sudo tcpdump udp -n -vv port 12301 -X

If the network level works and you can see packets like on the picture go to Graylog Inputs and check which kind of input you have

[a] It needs to be Raw/Plaintext UDP (if you have GELF UDP Graylog will filter your messages as Nginx sends logs in Syslog format and not in Json

[b] You will have Network IO different from 0

[c] Port (12301 in your case) and IP needs to be the same as in Nginx configuration

If you have all this you will find a RAW message from Nginx in your Graylog:

<190>Jul.26.16:12:07.graylog.nginx:.{."timestamp":."2018-07-26T16:12:07+03:00",."remote_addr":."xx.x.x.xxx",."body_bytes_sent":.4277,."request_time":.0.005,."response_status":.200,."request":."POST./api/cluster/metrics/multiple.HTTP/1.1",."request_method":."POST",."host":."xx.x.x.xxx",."upstream_cache_status":."-",."upstream_addr":."xx.x.x.xxx",."http_x_forwarded_for":."xx.x.x.xxx",."http_referrer":."https://xx.x.x.xxx/system/inputs",."http_user_agent":."xx.x.x.xxxxx.x.x.xxxxx.x.x.xxxxx.x.x.xxx",."http_version":."HTTP/1.1",."nginx_access":.true.}

Extract form pseudo-Syslog RAW message a JSON using Graylog Input Extractor:

Extractor Example:

{
  "extractors": [
    {
      "title": "Extract from Pseudo-Syslog a JSON",
      "extractor_type": "regex_replace",
      "converters": [],
      "order": 0,
      "cursor_strategy": "cut",
      "source_field": "message",
      "target_field": "message",
      "extractor_config": {
        "replacement": "$1",
        "regex": "^.*?(\\{.*?\\})$"
      },
      "condition_type": "none",
      "condition_value": ""
    }
  ],
  "version": "2.4.6"
}

As you use "content pack" you need to add the rule before all others that cames from "content pack" (order: 0) if you do import export

After adding the rule, you will have clear JSON log from Nginx, all other work will do "content pack"

Check your Nginx config

ngnix.conf example:

log_format graylog_json '{ "timestamp": "$time_iso8601", "remote_addr": "$remote_addr", "body_bytes_sent": $body_bytes_sent, "request_time": $request_time, "response_status": $status, "request": "$request", "request_method    ": "$request_method", "host": "$host", "upstream_cache_status": "$upstream_cache_status", "upstream_addr": "$upstream_addr", "http_x_forwarded_for": "$http_x_forwarded_for", "http_referrer": "$http_referer", "http_user_agent": "$h    ttp_user_agent", "http_version": "$server_protocol", "nginx_access": true }';
access_log syslog:server=graylog:5555 graylog_json;

Hope, following all this steps you will find useful

Send NGINX logs to Graylog

Related

Recent Posts