412 lines
15 KiB
Markdown
412 lines
15 KiB
Markdown
## Reverse Proxy(Layer4)
|
||
[[02. PARA/03. Resources（資源）/HTTP Server/Nginx#Reverse Proxy]]所用的方法雖然可以反向代理多個網站，但是對於像是Trojan這種TLS不行被中斷的服務來說，會導致handshake失敗，所以需要用Nginx的stream來做Layer 4的轉發。
|
||
|
||
### docker-compose.yaml
|
||
需要先把`nginx.conf`與`mime.types`給copy到data目錄下。
|
||
依序執行下面2個命令：
|
||
```shell
|
||
sudo docker run --rm -it nginx cat /etc/nginx/nginx.conf > nginx.conf
|
||
sudo docker run --rm -it nginx cat /etc/nginx/mime.types > mime.types
|
||
```
|
||
然後：
|
||
```shell
|
||
mkdir data ;\
|
||
mv nginx.conf mime.types data
|
||
```
|
||
|
||
建立`docker-compose.yaml`
|
||
```yaml
|
||
version: '3'
|
||
services:
|
||
nginx_reverseproxy_l4:
|
||
container_name: nginx
|
||
restart: always
|
||
image: nginx
|
||
ports:
|
||
- "80:80"
|
||
- "443:443"
|
||
volumes:
|
||
- ./data:/etc/nginx
|
||
```
|
||
|
||
修改`data/nginx.conf`：
|
||
```nginx
|
||
events {
|
||
worker_connections 1024; <-- event 不用動
|
||
}
|
||
|
||
stream {
|
||
map $ssl_preread_server_name $backend_name {
|
||
tjn.awin.one trojan;
|
||
storj.awin.one swag;
|
||
}
|
||
|
||
# trojan
|
||
upstream trojan {
|
||
server 192.168.1.31:443;
|
||
}
|
||
|
||
# swag
|
||
upstream swag {
|
||
server 192.168.1.20:44320;
|
||
}
|
||
|
||
# 监听 443 并开启 ssl_preread
|
||
server {
|
||
listen 80 reuseport;
|
||
listen 443 reuseport;
|
||
listen [::]:443 reuseport;
|
||
proxy_pass $backend_name;
|
||
ssl_preread on;
|
||
}
|
||
}
|
||
|
||
http {
|
||
... <-- http 不用動
|
||
}
|
||
```
|
||
|
||
2022/05/17更新：
|
||
```nginx
|
||
user nginx;
|
||
worker_processes 1;
|
||
|
||
error_log /var/log/nginx/error.log warn;
|
||
pid /var/run/nginx.pid;
|
||
|
||
|
||
events {
|
||
worker_connections 1024;
|
||
}
|
||
|
||
stream {
|
||
map $ssl_preread_server_name $backend_name {
|
||
tjn.awin.one trojan;
|
||
storj.awin.one swag;
|
||
blog.awin.one swag;
|
||
gitea.awin.one swag;
|
||
}
|
||
|
||
# trojan
|
||
upstream trojan {
|
||
server 192.168.1.31:443;
|
||
}
|
||
|
||
# swag
|
||
upstream swag {
|
||
server 192.168.1.20:44320;
|
||
}
|
||
|
||
# 监听 443 并开启 ssl_preread
|
||
server {
|
||
listen 443 reuseport;
|
||
listen [::]:443 reuseport;
|
||
proxy_pass $backend_name;
|
||
ssl_preread on;
|
||
}
|
||
}
|
||
|
||
http {
|
||
include /etc/nginx/mime.types;
|
||
default_type application/octet-stream;
|
||
|
||
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||
'$status $body_bytes_sent "$http_referer" '
|
||
'"$http_user_agent" "$http_x_forwarded_for"';
|
||
|
||
access_log /var/log/nginx/access.log main;
|
||
|
||
sendfile on;
|
||
#tcp_nopush on;
|
||
|
||
keepalive_timeout 65;
|
||
|
||
#gzip on;
|
||
|
||
include /etc/nginx/conf.d/*.conf;
|
||
|
||
## 新增這一段，沒有這一段會造成http無法連線，所以拿憑證會失敗
|
||
server {
|
||
listen 80;
|
||
server_name tjn.awin.one;
|
||
|
||
location / {
|
||
proxy_pass http://192.168.1.31:80;
|
||
|
||
# 把 IP、Protocol 等 header 都一起送給反向代理的 server
|
||
proxy_set_header X-Real-IP $remote_addr;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
重點來源：
|
||
1. [Trojan 共用 443 端口方案 - 程小白](https://www.chengxiaobai.cn/record/trojan-shared-443-port-scheme.html)
|
||
2. [NaiveProxy + Trojan + 多HTTPS站点共存，复用443端口 | 心底的河流](https://lhy.life/20200815-naiveproxy+trojan/)
|
||
3. `$ssl_preread_server_name`的官方說明：[Module ngx_stream_ssl_preread_module](http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html)
|
||
|
||
## Reverse Proxy
|
||
|
||
> **重要**：確定docker與docker-compose已經安裝好。
|
||
> 參考[[RaspberryPi#Docker]]與[[RaspberryPi#docker-compose]]
|
||
|
||
### Use SWAG docker
|
||
[swag](https://github.com/linuxserver/docker-swag)(之前叫做letsencrypt)是linuxserver.io包裝的Nginx webserver和reverse proxy的container。
|
||
|
||
#### Setup HTTPS
|
||
1. 建立folder
|
||
```bash
|
||
mkdir -p ~/dockers/linuxserverswag ; cd ~/dockers/linuxserverswag
|
||
```
|
||
|
||
2. 建立`docker-compose.yaml`：
|
||
```
|
||
vim docker-compose.yaml
|
||
```
|
||
|
||
填入內容如下：
|
||
```yaml
|
||
version: "2.1"
|
||
services:
|
||
swag:
|
||
image: ghcr.io/linuxserver/swag
|
||
container_name: swag
|
||
cap_add:
|
||
- NET_ADMIN
|
||
environment:
|
||
- PUID=1000
|
||
- PGID=1000
|
||
- TZ=Asia/Taipei
|
||
- URL=awin.one
|
||
- SUBDOMAINS=wildcard
|
||
- VALIDATION=dns
|
||
- CERTPROVIDER= #optional
|
||
- DNSPLUGIN=cloudflare #optional
|
||
- PROPAGATION= #optional
|
||
- DUCKDNSTOKEN= #optional
|
||
- EMAIL=awinhuang@gmail.com
|
||
- ONLY_SUBDOMAINS=false #optional
|
||
- EXTRA_DOMAINS= #optional
|
||
- STAGING=false #optional
|
||
- MAXMINDDB_LICENSE_KEY= #optional
|
||
volumes:
|
||
- ./config:/config
|
||
ports:
|
||
- 44320:443
|
||
- 8020:80 #optional
|
||
restart: unless-stopped
|
||
|
||
```
|
||
|
||
3. 先跑一次：
|
||
```bash
|
||
sudo docker-compose up
|
||
```
|
||
會發現有錯誤，這是正常的，錯誤訊息像這樣：
|
||
```
|
||
swag | Unsafe permissions on credentials configuration file: /config/dns-conf/cloudflare.ini
|
||
swag | Cleaning up challenges
|
||
swag | Error determining zone_id: 9103 Unknown X-Auth-Key or X-Auth-Email. Please confirm that you have supplied valid Cloudflare API credentials. (Did you enter the correct email address and Global key?)
|
||
swag | ERROR: Cert does not exist! Please see the validation error above. Make sure you entered correct credentials into the /config/dns-conf/cloudflare.ini file.
|
||
```
|
||
按`ctrl + c`退出。這時候`config`目錄也會有swag所mapping出來的相關檔案。修改`config/dns-conf/cloudflare.ini`：
|
||
```bash
|
||
vim config/dns-conf/cloudflare.ini
|
||
```
|
||
把`config/dns-conf/cloudflare.ini`改為：
|
||
```
|
||
# Instructions: https://github.com/certbot/certbot/blob/master/certbot-dns-cloudflare/certbot_dns_cloudflare/__init__.py#L20
|
||
# Replace with your values
|
||
|
||
With global api key:
|
||
dns_cloudflare_email = awinhuang@gmail.com
|
||
dns_cloudflare_api_key = <YOUR_API_KEY_FROM_CLOUDFLARE>
|
||
|
||
# With token (comment out both lines above and uncomment below):
|
||
#dns_cloudflare_api_token = 0123456789abcdef0123456789abcdef01234567
|
||
```
|
||
|
||
4. 再跑一次：
|
||
```bash
|
||
sudo docker-compose up
|
||
```
|
||
這一次就可以順利取得認證了，訊息像這樣：
|
||
```
|
||
swag | IMPORTANT NOTES:
|
||
swag | - Congratulations! Your certificate and chain have been saved at:
|
||
swag | /etc/letsencrypt/live/awin.one/fullchain.pem
|
||
swag | Your key file has been saved at:
|
||
swag | /etc/letsencrypt/live/awin.one/privkey.pem
|
||
swag | Your certificate will expire on 2021-04-26. To obtain a new or
|
||
swag | tweaked version of this certificate in the future, simply run
|
||
swag | certbot again. To non-interactively renew *all* of your
|
||
swag | certificates, run "certbot renew"
|
||
swag | - If you like Certbot, please consider supporting our work by:
|
||
swag |
|
||
swag | Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
|
||
swag | Donating to EFF: https://eff.org/donate-le
|
||
swag |
|
||
swag | New certificate generated; starting nginx
|
||
swag | Starting 2019/12/30, GeoIP2 databases require personal license key to download. Please retrieve a free license key from MaxMind,
|
||
swag | and add a new env variable "MAXMINDDB_LICENSE_KEY", set to your license key.
|
||
swag | [cont-init.d] 50-config: exited 0.
|
||
swag | [cont-init.d] 60-renew: executing...
|
||
swag | The cert does not expire within the next day. Letting the cron script handle the renewal attempts overnight (2:08am).
|
||
swag | [cont-init.d] 60-renew: exited 0.
|
||
swag | [cont-init.d] 70-templates: executing...
|
||
swag | [cont-init.d] 70-templates: exited 0.
|
||
swag | [cont-init.d] 99-custom-files: executing...
|
||
swag | [custom-init] no custom files found exiting...
|
||
swag | [cont-init.d] 99-custom-files: exited 0.
|
||
swag | [cont-init.d] done.
|
||
swag | [services.d] starting services
|
||
swag | [services.d] done.
|
||
swag | nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html)
|
||
swag | Server ready
|
||
```
|
||
最後一行的`swag | Server ready`表示server已經跑起來了。先按下`ctrl + c`退出，再來設定reverse proxy。
|
||
|
||
5. 修正`config/dns-conf/cloudflare.ini`的安全性問題
|
||
```
|
||
cd ~/dockers/linuxserverswag ; chmod 600 config/dns-conf/cloudflare.ini
|
||
```
|
||
|
||
#### Setup reverse proxy
|
||
1. 建立folder：
|
||
```
|
||
cd ~/dockers/linuxserverswag ; mkdir -p config/nginx/sites-available config/nginx/sites-enabled
|
||
```
|
||
|
||
2. 建立以下檔案：
|
||
- `config/nginx/sites-available/common.conf`，內容：
|
||
```
|
||
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
||
add_header X-Frame-Options SAMEORIGIN;
|
||
add_header X-Content-Type-Options nosniff;
|
||
add_header X-XSS-Protection "1; mode=block";
|
||
```
|
||
- `config/nginx/sites-available/common_location.conf`，內容：
|
||
```
|
||
proxy_set_header X-Real-IP $remote_addr;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Forwarded-Host $host;
|
||
proxy_set_header X-Forwarded-Port $server_port;
|
||
```
|
||
- `config/nginx/sites-available/storj.conf`，內容：
|
||
```
|
||
upstream storj {
|
||
server 192.168.1.11:14002;
|
||
}
|
||
|
||
server {
|
||
listen 443 ssl;
|
||
server_name storj.awin.one;
|
||
|
||
include /config/nginx/sites-available/common.conf;
|
||
include /config/nginx/ssl.conf;
|
||
|
||
location / {
|
||
proxy_pass http://192.168.1.11:14002/;
|
||
include /config/nginx/sites-available/common_location.conf;
|
||
}
|
||
}
|
||
|
||
```
|
||
|
||
3. 在`config/nginx/sites-enabled`裡面建立要enable的config：
|
||
```
|
||
cd config/nginx/sites-enabled ; ln -s ../sites-available/storj.conf . ; cd -
|
||
```
|
||
|
||
4. 修改`config/nginx/nginx.conf`：
|
||
```
|
||
vim config/nginx/nginx.conf
|
||
```
|
||
找到`include /config/nginx/site-confs/*;`這一行，把它comment掉，在下面新增一行：
|
||
```
|
||
include /config/nginx/sites-enabled/*.conf;
|
||
```
|
||
|
||
5. 啟動swag：
|
||
```
|
||
cd ~/dockers/linuxserverswag ; sudo docker-compose up -d
|
||
```
|
||
|
||
#### Restart
|
||
```bash
|
||
cd ~/dockers/linuxserverswag ; sudo docker-compose restart
|
||
```
|
||
|
||
#### Update certification
|
||
1. 進到dockr的swag bash裡面：`sudo docker exec -it swag /bin/bash`
|
||
2. 輸入 `certbot renew`
|
||
3. ![[Pasted image 20210422205534.png]]
|
||
|
||
#### Trouble shooting
|
||
- 如果遇到類似下面的錯誤：
|
||
```
|
||
ERROR: for swag Cannot start service swag: driver failed programming external connectivity on endpoint swag (7c527d046631e0957de0b831ca25bed296de76e2eb96378964cb0110d7fb017d): Bind for 0.0.0.0:443 failed: port is already allocated.
|
||
```
|
||
表示有其他程式佔住了80 port，可能是其他docker container或是service，必須先關閉它們。[^1]
|
||
|
||
#### 參考來源
|
||
1. [linuxserver/docker-swag: Nginx webserver and reverse proxy with php support and a built-in Certbot (Let's Encrypt) client. It also contains fail2ban for intrusion prevention.](https://github.com/linuxserver/docker-swag)
|
||
2. [How to set up an easy and secure reverse proxy with Docker, Nginx & Letsencrypt](https://www.freecodecamp.org/news/docker-nginx-letsencrypt-easy-secure-reverse-proxy-40165ba3aee2/)
|
||
3. [SWAG setup - LinuxServer.io](https://docs.linuxserver.io/general/swag#understanding-the-proxy-conf-structure)
|
||
|
||
-----
|
||
- 參考
|
||
- [NGINX Docs | NGINX Reverse Proxy](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/)
|
||
- [Nginx 設定反向代理 Reverse Proxy](https://www.opencli.com/linux/nginx-config-reverse-proxy)
|
||
- [用 Nginx 伺服器建立反向代理](https://noob.tw/nginx-reverse-proxy/)
|
||
- [俄羅斯不愧是戰鬥民族：nginx - iT 邦幫忙::一起幫忙解決難題，拯救 IT 人的一天](https://ithelp.ithome.com.tw/articles/10188498)
|
||
|
||
[^1]: [啟動docker時出現「Cannot start service :driver failed programming external connectivity on endpoint xxx, Bind for 0.0.0.0:80 failed: port is already allocated 」](https://mitsuiwei.com/docker-cannot-start-service/)
|
||
|
||
# 新增網站
|
||
## 新網站自帶SSL
|
||
進到Rasperberry Pi（192.168.1.20）之後，切換到`~/dockers/nginx_reverseproxy_l4`，並編輯`data/nginx.conf`，在裡面加入新網站的網域與要轉址的ip。
|
||
例如，要加入一個trojan VPN，trojan VPN本身就有SSL加密，所以不用再經過swag，`data/nginx.conf`的改變如下：
|
||
![[Pasted image 20220506145247.png]]
|
||
|
||
## 新http網站
|
||
如果新的網站只是一般的http網站，那便把它掛到swag後面，這樣就可以經由https來訪問，假如要加入一個blog網站，但因為我們有2層的reverse proxy，第一層是layer 4，第2層是swag，所以如果是自帶https的要掛到layer 4之後，沒有https要先由layer 4轉到swag再轉到實際的server上。
|
||
步驟如下：
|
||
1. 進到Rasperberry Pi（192.168.1.20）
|
||
2. 設定nginx_reverseproxy_l4，這邊我們需要把「沒有https」的網站由nginx_reverseproxy_l4導引到linuxserverswag
|
||
```
|
||
cd ~/dockers/nginx_reverseproxy_l4
|
||
vim data/nginx.conf
|
||
```
|
||
加入下圖紅框的設定：
|
||
![[Pasted image 20220516152230.png]]
|
||
3. 切換到`~/dockers/linuxserverswag/config/nginx/sites-available`
|
||
4. 新增一個confing檔，例如叫做`blog.conf`，內容如下：
|
||
```
|
||
upstream blog {
|
||
server 192.168.1.30:80; ## 網址
|
||
}
|
||
|
||
server {
|
||
listen 443 ssl;
|
||
server_name blog.awin.one; ## 網域
|
||
|
||
include /config/nginx/sites-available/common.conf;
|
||
include /config/nginx/ssl.conf;
|
||
|
||
location / {
|
||
proxy_pass http://192.168.1.30:80/; ## 網址
|
||
include /config/nginx/sites-available/common_location.conf;
|
||
}
|
||
}
|
||
```
|
||
改變3個有註解的地方就可以了。
|
||
5. 剛剛的config檔，新增到`sites-enabled`裡面
|
||
1. `cd ~/dockers/linuxserverswag/config/nginx/sites-enabled`
|
||
2. `ln -s ../sites-available/blog.conf .`
|
||
6. 重新啟動swag：`cd ~/dockers/linuxserverswag ; sudo docker-compose restart`
|