How to get realip to work with ingress-nginx and ELB

Environment: AWS EKS, Ingress-Nginx controller installed, and an Ingress resource is configured, passing requests to a backend http service.

Problem: backend service can’t see real client IP addresses. It sees internal VPC addresses instead.

Goal: let ingress-nginx pass client ip addresses to backend.

By default, ingress-nginx-controller installed into EKS will create an ELB. It also allows to use NLB and ALB. But that requires installation of AWS load balancer controller, and who wants an additional dependency that can break during an upgrade? So we’ll stick with ELB.

The created ELB passes traffic at L4. To pass client IPs, we will need to configure proxy-protocol. Here’s AWS documentation. ELB uses protocol v1, while ALB uses v2, but Nginx seemingly speaks both, so that shouldn’t be a problem.

Check your ingress controller

kubectl -n ingress-nginx get svc
NAME                                            TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)                      AGE
ingress-nginx-controller-controller             LoadBalancer   172.20.124.111   c43c1730102072835b21c6af3cede412-886572035.us-east-1.elb.amazonaws.com   80:32131/TCP,443:31739/TCP   15d

Note c43c1730102072835b21c6af3cede412 - that is the load balancer name. You can see it in AWS console in EC2 load balancers.

export lbname=c43c1730102072835b21c6af3cede412

As per AWS doc, create policy

aws elb create-load-balancer-policy --load-balancer-name $lbname --policy-name ProxyProtocolEnable --policy-type-name ProxyProtocolPolicyType --policy-attributes AttributeName=ProxyProtocol,AttributeValue=true

Now the tricky part, enable policy. Note that policy should be set on target ports. Not 80 and 443, but the paired ones.

aws elb set-load-balancer-policies-for-backend-server --load-balancer-name $lbname --instance-port 32131 --policy-names ProxyProtocolEnable
aws elb set-load-balancer-policies-for-backend-server --load-balancer-name $lbname --instance-port 31739 --policy-names ProxyProtocolEnable

Enable proxy protocol in ingress-nginx controller

kubectl -n ingress-nginx edit cm ingress-nginx-controller-controller

Ensure use-proxy-protocol: "true" is set:

apiVersion: v1
data:
  use-proxy-protocol: "true"
...

At this point, ELB should pass client IPs to ingress-nginx. To verify, check ingress-nginx logs.

kubectl -n ingress-nginx get pods
NAME                                                  READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-controller-931aa6241-3a1f1   1/1     Running   0          18d

curl -s http://my-backend-service

kubectl -n ingress-nginx logs ingress-nginx-controller-controller-931aa6241-3a1f1 --tail=3