Heads up! Long post and lots of head bashing against the wall.
Context:
I have a written an a python app (Django). I have dockerized the deployment and the compose file has three containers, app, nginx and postgres. I’m currently trying to deploy a demo of it in a VPS running Debian 11. Information below has been redacted (IPs, Domain name, etc.)
Problem:
I keep running into 502 errors. Locally things work very well even with nginx (but running on 80). As I try to deploy this I’m trying to configure nginx the best I can and redirecting http traffic to https and ssl certs. The nginx logs simply say “connect() failed (111: Connection refused) while connecting to upstream, client: 1.2.3.4, server: demo.example.com, request: “GET / HTTP/1.1”, upstream: “http://192.168.0.2:8020/”, host: “demo.example.com””. I have tried just about everything.
What I’ve tried:
- Adding my server block configs to /etc/nginx/conf.d/default.conf
- Adding my server block configs to a new file in /etc/nginx/conf.d/app.conf and leaving default at out of box config.
- Tried putting the above config (default.conf and app.conf) in sites-available (/etc/nginx/sites-available/* not at the same time tho).
- Recreated /etc/nginx/nginx.conf by copy/pasting out of box nginx.conf and then adding server blocks directly in nginx.conf
- Running nginx -t inside of the nginx container (Syntax and config were “successful”)
- Running nginx -T when recreated /etc/nginx/nginx.conf
- nginx -T when the server blocks where in /etc/nginx/conf.d/* lead me to think that since there were two server listen 80 blocks that I should ensure only one listen 80 block was being read by the container hence the recreated /etc/nginx/nginx.conf from above
- Restarted container each time a change was made.
- Changed the user block from nginx (no dice when using nginx as user) to www-data, root and nobody
- Deleted my entire docker data and redeployed everything a few times.
- Double checked the upstream block 1,000 times
- Confirmed upstream block container is running and on the right exposed port Checked access.log and error.log but they were both empty (not sure why, tried cat and tail)
- Probably forgetting more stuff (6 hours deep in the same error loop by now)
How can you help:
Please take a look at the nginx.conf config below and see if you guys can spot a problem, PLEASE! This is my current /etc/nginx/nginx.conf
`
user www-data;
worker_processes auto;
error_log /var/log/nginx/error.log notice; pid /var/run/nginx.pid;
events { worker_connections 1024; }
http { include /etc/nginx/mime.types; default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
upstream djangoapp {
server app:8020;
}
server {
listen 80;
listen [::]:80;
server_name demo.example.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name demo.example.com;
ssl_certificate /etc/letsencrypt/live/demo.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/demo.example.com/privkey.pem;
#ssl_protocols TLSv1.2 TLSv1.3;
#ssl_prefer_server_ciphers on;
location / {
proxy_pass http://djangoapp;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header Upgrade $http_upgrade;
#proxy_set_header Connection keep-alive;
proxy_redirect off;
}
location /static/ {
autoindex on;
alias /static/;
}
}
}
`
-
EDIT: I have also confirmed that both containers are connected to the same docker network (docker network inspect frontend)
-
EDIT 2: Solved my problem. See my comments to @chaospatterns. TLDR there was an uncaught exception in the app but it didn’t cause a crash with the container. Had to dig deep into logs to find it.
First the basics. Connection refused means that nothing is running on “http://192.168.0.2:8020/”
- Is 192.168.0.2 the IP address of the Django container? If it’s the host’s IP, does docker ps show that the port is bound to the host? e.g.
0.0.0.0/8082->8082
Confirmed upstream block container is running and on the right exposed port
What steps did you do to confirm that this is running?
- Is 192.168.0.2 the IP address of the Django container? If it’s the host’s IP, does docker ps show that the port is bound to the host? e.g.
While not directly solving the problem, my usual setup involves installing webmin with nginx and managing my domains and nginx from there, changing the auto-generated domain configs to proxy to docker. I have yet to find an easier solution.
My biggest gripe with it is that it sets up a few things I don’t need, like PHP, disk quotas, etc.