formats-nginx | formats-ingress | formats-parsers
as for k8s v1.27.4
the default ingress-nginx format
log_format upstreaminfo '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id';
log_format log_stream '[$remote_addr] [$time_local] $protocol $status $bytes_sent $bytes_received $session_time';
we only need the former, as the latter is about L3/4 proxying
custom ingress-nginx logs for index mapping
let’s add gzip field here also
log-format-upstream: '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id "$gzip_ratio"'
TBD
how to find out
get into the node that holds ingress
minikube ssh
get into the ingress controller
docker ps | grep ingress | grep controller
docker exec -ti 8c3913d48d9c bash
grep log_format nginx.conf
there are two definitions
grep upstreaminfo nginx.conf | grep -v log_format
grep log_stream nginx.conf | grep -v log_format
https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf
https://github.com/nginxinc/kubernetes-ingress/blob/v3.3.2/internal/configs/version1/nginx.tmpl
https://braindose.blog/2022/07/18/monitor-and-analyze-nginx-ingress-controller-logs-on-kubernetes-using-elasticsearch-and-kibana/ ==> nice walk-through
https://github.com/fluent/fluent-bit/issues/1534
https://opensearch.org/docs/latest/field-types/index/#dynamic-mapping
https://opensearch.org/docs/latest/api-reference/index-apis/create-index#index-settings