From d5a860a19b814d14007c12fd69958fd7df213e32 Mon Sep 17 00:00:00 2001
From: Naomi Carrigan <commits@nhcarrigan.com>
Date: Sun, 14 Jul 2024 15:50:31 -0700
Subject: [PATCH] feat: add document on setting up new servers

---
 docs/_navbar.md |   1 +
 docs/index.html |   1 +
 docs/servers.md | 339 ++++++++++++++++++++++++++++++++++++++++++++++++
 docs/vtubing.md |   2 +-
 4 files changed, 342 insertions(+), 1 deletion(-)
 create mode 100644 docs/servers.md

diff --git a/docs/_navbar.md b/docs/_navbar.md
index 7237c50..93b4766 100644
--- a/docs/_navbar.md
+++ b/docs/_navbar.md
@@ -43,6 +43,7 @@
 - [Contributor Covenant](/covenant)
 - [Issue/PR Labels](/community/labels)
 - [Development Environment](/environment)
+- [Server Setup](/servers.md)
 
 ---
 
diff --git a/docs/index.html b/docs/index.html
index 57ef8a3..1ec99dc 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -93,6 +93,7 @@
   <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-ini.min.js"></script>
   <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-bash.min.js"></script>
   <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-json.min.js"></script>
+  <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-nginx.min.js"></script>
   <script src="https://unpkg.com/docsify-copy-code@2"></script>
   <script
     src="https://browser.sentry-cdn.com/6.3.6/bundle.min.js"
diff --git a/docs/servers.md b/docs/servers.md
new file mode 100644
index 0000000..9782d5d
--- /dev/null
+++ b/docs/servers.md
@@ -0,0 +1,339 @@
+# Server Setup
+
+**Effective 14 July 2024**
+
+This document outlines how we set up our remote servers for running projects.
+
+## 1. Provision a Server
+
+We use DigitalOcean as our provider. Regardless of your choice, provision a new VPS using the **latest Ubunutu LTS version**. Add your `ssh` key AND Naomi's `ssh` key in the setup process.
+
+## 2. Set Up User
+
+You should never run applications on root. SSH into the new VPS to prepare your user.
+
+### 2.1. Creating the User
+
+You'll need to set a password for the `root` account first.
+
+```bash
+passwd
+```
+
+Once you have set a password, ensure that you have provided it to Naomi to store in the vault.
+
+Create an `nhcarrigan` user for our organisation.
+
+```bash
+adduser nhcarrigan
+```
+
+Set a **different** password, and provide that to Naomi as well. For all of the user information, use the default blank values.
+
+Add the new user to the sudoers file.
+
+```bash
+usermod -aG sudo nhcarrigan
+```
+
+Then sync the SSH keys so we can authenticate as that user.
+
+```bash
+rsync --archive --chown=nhcarrigan:nhcarrigan ~/.ssh /home/nhcarrigan
+```
+
+While you are there, set the timezone for the server to our business' local timezone.
+
+```bash
+sudo timedatectl set-timezone America/Los_Angeles
+```
+
+## 3. Preparing For Web Requests
+
+To prepare the server to receive web requests, you'll need to follow a few steps.
+
+### 3.1. SSL Certificate
+
+> [!NOTE]
+> If the Firewall has been set up, you'll need to temporarily allow port 80 for the certificate to generate.
+
+We use LetsEncrypt to provision our SSL certificates. If it is not installed, install it with:
+
+```bash
+sudo snap install --classic certbot
+```
+
+Then link the snap to our `usr` directory.
+
+```bash
+sudo ln -s /snap/bin/certbot /usr/bin/certbot
+```
+
+Generate a certificate with:
+
+```bash
+sudo certbot certonly --standalone
+```
+
+And allow applications to read it:
+
+```bash
+sudo chmod -R a+rwx /etc/letsencrypt
+```
+
+When you need to renew the certificate:
+
+```bash
+sudo certbot renew
+```
+
+### 3.2. NGINX
+
+All requests should be routed through NGINX. At no point should an application run directly on ports 80 or 443.
+
+Install NGINX:
+
+```bash
+sudo apt-get install nginx
+```
+
+Edit the configuration file:
+
+```bash
+sudo emacs /etc/nginx/conf.d/server.conf
+```
+
+Use this template to set up a reverse proxy on the standard HTTPS port 443:
+
+```nginx
+server {
+    listen 443 ssl;
+    server_name subdomain.domain.tld;
+    ssl_certificate /etc/letsencrypt/live/subdomain.domain.tld/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/subdomain.domain.tld/privkey.pem;
+
+    location / {
+        proxy_set_header Host $host;
+        proxy_pass https://127.0.0.1:port;
+        proxy_redirect off;
+    }
+}
+```
+
+Validate that the config is correct with:
+
+```bash
+sudo nginx -t
+```
+
+If so, restart NGINX to apply the changes:
+
+```bash
+sudo systemctl restart nginx
+```
+
+## 4. Securing the Server
+
+We have a minimum level of security that is required on ALL of our servers. This section should not be treated as the best effort, but as the minimal requirements to comply with our policies.
+
+### 4.1. Firewall
+
+We use `ufw` as our firewall. First, enable the SSH port.
+
+```bash
+sudo ufw allow "OpenSSH"
+```
+
+Then, allow the standard HTTPS port and **deny** the standard HTTP port.
+
+```bash
+sudo ufw deny http
+sudo ufw allow https
+```
+
+Enable the firewall. You may get dropped from the SSH connection.
+
+```bash
+sudo ufw enable
+```
+
+### 4.2. Fail2Ban
+
+We also use Fail2Ban to block IP addresses which fail to make requests too often.
+
+Install the tool:
+
+```bash
+sudo apt-get install fail2ban
+```
+
+Configure the NGINX jail in `/etc/fail2ban/jail.d/nginx-auth.conf`:
+
+```ini
+[nginx-auth]
+enabled = true
+filter = nginx-auth
+logpath = /var/log/nginx/access.log
+maxretry = 3
+findtime = 86400
+bantime = 86400
+```
+
+Configure the NGINX filter in `/etc/fail2ban/filter.d/nginx-auth.conf`:
+
+```ini
+[Definition]
+failregex = ^<HOST> - .* \[.*\] ".*" (4\d{2}) .*$
+```
+
+Because we use Cloudflare, you'll need to grab the original IP for all requests. Start by creating a file to store Cloudflare's IPs.
+
+```bash
+sudo touch /etc/nginx/cloudflare_ips.conf
+```
+
+Then create your script:
+
+```bash
+nano ~/update_cf_ips.sh
+```
+
+```bash
+#!/bin/bash
+
+# Create a temporary file
+temp_file=$(mktemp)
+
+# Download IPv4 ranges and format each line
+curl -s https://www.cloudflare.com/ips-v4 | while read ip; do
+    echo "set_real_ip_from $ip;" >> "$temp_file"
+done
+
+# Download IPv6 ranges and format each line
+curl -s https://www.cloudflare.com/ips-v6 | while read ip; do
+    echo "set_real_ip_from $ip;" >> "$temp_file"
+done
+
+# Add the real_ip_header directive
+echo "real_ip_header CF-Connecting-IP;" >> "$temp_file"
+
+# Replace the old file with the new one
+sudo mv "$temp_file" /etc/nginx/cloudflare_ips.conf
+
+# Test Nginx configuration
+sudo nginx -t
+
+# If the test is successful, reload Nginx
+if [ $? -eq 0 ]; then
+    sudo systemctl reload nginx
+    echo "Nginx configuration updated and reloaded successfully."
+else
+    echo "Nginx configuration test failed. Please check your configuration."
+fi
+```
+
+Make it executable and run it:
+
+```bash
+sudo chmod +x update_cf_ips.sh
+sudo ./update_cf_ips.sh
+```
+
+If it runs as expected, set it up to run on a CRON.
+
+```bash
+sudo crontab -e
+```
+
+```bash
+0 3 * * 1 ~/update_cf_ips.sh
+```
+
+Then, update the `/etc/nginx/nginx.conf` to use all of this new logic. This goes at the end of your `http` directive block.
+
+```conf
+# Look at the real IP, not the cloudflare IP.
+include /etc/nginx/cloudflare_ips.conf;
+
+log_format custom_format '$remote_addr - $remote_user [$time_local] '
+                        '"$request" $status $body_bytes_sent '
+                        '"$http_referer" "$http_user_agent" '
+                        '"$http_x_forwarded_for"';
+
+access_log /var/log/nginx/access.log custom_format;
+```
+
+Confirm the NGINX configuration is correct:
+
+```bash
+sudo nginx -t
+```
+
+Then restart everything.
+
+```bash
+sudo systemctl restart nginx
+sudo systemctl restart fail2ban
+```
+
+To view banned IPs:
+
+```bash
+sudo fail2ban-client status nginx-auth
+```
+
+And to unban them:
+
+```bash
+sudo fail2ban-client set nginx-auth unbanip <ip>
+```
+
+## 5. Running a Project
+
+Now you are ready to start running the project.
+
+### 5.1. Node.js
+
+Most of our projects will run on Node. For a new machine, you'll need to set that up.
+
+We use `nvm` to manage Node versions. Fetch and run the install script:
+
+```bash
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
+```
+
+The script will automatically update the `.bashrc` file to load `nvm` into the PATH. Reload that:
+
+```bash
+source ~/.bashrc
+```
+
+Install the long-term support Node version.
+
+```bash
+nvm install --lts
+```
+
+This should automatically set it as the default. When updating, be sure to remove any older versions!
+
+Finally, install `pnpm` as the package manager.
+
+```bash
+npm i -g pnpm
+```
+
+### 5.2. PM2
+
+All of our processes run with PM2 to allow for monitoring and auto-restarts. You'll need to install it.
+
+```bash
+pnpm i -g pm2
+```
+
+To start a project, use this template:
+
+```bash
+pm2 start '<script>' --name '<name>'
+```
+
+Then run `pm2 save` to save the application list.
diff --git a/docs/vtubing.md b/docs/vtubing.md
index 1e15b7e..e6d0630 100644
--- a/docs/vtubing.md
+++ b/docs/vtubing.md
@@ -1700,7 +1700,7 @@ CookieId=#redacted
 
 ### 3.4. Docks
 
-This configuration file loads Naomi's standard dock layout. Sensitive information has been redacted.
+This configuration file loads Naomi's standard dock layout. Sensitive information has been redacted. This specifically goes in `~/.config/obs-studio/global.ini`.
 
 ```ini
 [General]