Submarined on my subdomain

[techthrasher]

Hi! New to Corteza, not so new to Docker.

I’m flummoxed! I am attempting to install a Corteza instance on Yunohost using the Redirect app. Basically, Yunohost is a very user friendly NGINX front end. The redirect app allows me to set up NGINX redirects to custom apps.

I am using a custom subdomain: corteza.techthrasher.com. The DNS records, cert, etc., are all good.

Because the NGINX is preexisting, I can’t connect to my corteza docker container using docker networking. Instead, I exposed a port from the server container (12080:80). When I click on the redirect icon in Yunohost, it navigates appropriately to the corteza container - no problem. However, when it pulls up the beautiful corteza login window, It just says “Internal Error” where the login form should be. I’m definitely reaching the container, so I don’t think there’s an NGINX issue but there could definitely be an auth URL issue. Any ideas? I can provide logs, etc, if you’ll give me a steer on what you need.

Thanks so much,
Phil

[peter]

Hi @techthrasher , logs and anything you could share would be useful.
Server logs, console logs (web developer toolbar), …

[techthrasher]

Hi Peter! Thanks so much for helping out.

Here’s what the console log says from my browser:

Failed to load resource: the server responded with a status of 500 ()

https://corteza.techthrasher.com/auth/oauth2/authorize?client_id=359982122213048323&redirect_uri=https%3A%2F%2Fcorteza.techthrasher.com%2Fauth%2Fcallback&response_mode=query&response_type=code&scope=profile+api&state=35mm8rtjs74

As for the server logs, I’m at a bit of a loss on where those might be. There’s no way to bring them down from the webpage, and the data directories on the server are empty. Sorry to be such a numpty!

[techthrasher]

@peter
Hope you haven’t given up on me. Figured out how to get into the logs and found this

corteza-db-1      | 
corteza-db-1      | PostgreSQL Database directory appears to contain a database; Skipping initialization
corteza-db-1      | 
corteza-db-1      | 2023-10-22 12:49:38.771 UTC [1] LOG:  starting PostgreSQL 15.4 (Debian 15.4-2.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
corteza-db-1      | 2023-10-22 12:49:38.771 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
corteza-db-1      | 2023-10-22 12:49:38.771 UTC [1] LOG:  listening on IPv6 address "::", port 5432
corteza-db-1      | 2023-10-22 12:49:38.774 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
corteza-db-1      | 2023-10-22 12:49:38.780 UTC [29] LOG:  database system was shut down at 2023-10-22 12:31:41 UTC
corteza-db-1      | 2023-10-22 12:49:38.790 UTC [1] LOG:  database system is ready to accept connections
corteza-server-1  | {"level":"warn","ts":1697978980.2520418,"msg":"no SMTP servers found, email sending will be disabled"}
corteza-server-1  | {"level":"warn","ts":1697978980.270505,"logger":"http.apigw","msg":"profiler enabled only for routes with a profiler prefilter, use global setting to enable for all (APIGW_PROFILER_GLOBAL)"}
corteza-server-1  | {"level":"warn","ts":1697978980.2706485,"logger":"http.apigw","msg":"profiler enabled only for routes with a profiler prefilter, use global setting to enable for all (APIGW_PROFILER_GLOBAL)"}
corteza-server-1  | {"level":"warn","ts":1697978980.271572,"logger":"http.apigw","msg":"profiler enabled only for routes with a profiler prefilter, use global setting to enable for all (APIGW_PROFILER_GLOBAL)"}
corteza-db-1      | 2023-10-22 12:54:38.881 UTC [27] LOG:  checkpoint starting: time
corteza-db-1      | 2023-10-22 12:54:51.032 UTC [27] LOG:  checkpoint complete: wrote 124 buffers (0.8%); 0 WAL file(s) added, 0 removed, 0 recycled; write=12.141 s, sync=0.004 s, total=12.152 s; sync files=34, longest=0.001 s, average=0.001 s; distance=795 kB, estimate=795 kB

So my logs aren’t showing any errors (I don’t have the SMTP stuff set up yet - want to get a solid base config before I fiddle with details).

Here’s my .env file:

########################################################################################################################
# docker-compose supports environment variable interpolation/substitution in compose configuration file
# (more info: https://docs.docker.com/compose/environment-variables)

########################################################################################################################
# General settings
DOMAIN=corteza.techthrasher.com
VERSION=2023.3

########################################################################################################################
# Database connection

DB_DSN=postgres://corteza:corteza@db:5432/corteza?sslmode=disable

########################################################################################################################
# Server settings

# Serve Corteza webapps alongside API
HTTP_WEBAPP_ENABLED=true

# Send action log to container logs as well
# ACTIONLOG_DEBUG=true

# Uncomment for extra debug info if something goes wrong
# LOG_LEVEL=debug

# Use nicer and colorful log instead of JSON
# LOG_DEBUG=true

########################################################################################################################
# Authentication

# Secret to use for JWT token
# Make sure you change it (>30 random characters) if
# you expose your deployment to outside traffic
# AUTH_JWT_SECRET=this-is-only-for-demo-purpose--make-sure-you-change-it-for-production

########################################################################################################################
# SMTP (mail sending) settings

# Point this to your local or external SMTP server if you want to send emails.
# In most cases, Corteza can detect that SMTP is disabled and skips over sending emails without an error
#SMTP_HOST=smtp-server.example.tld:587
#SMTP_USER=postmaster@smtp-server.example.tld
#SMTP_PASS=this-is-your-smtp-password
#SMTP_FROM='"Demo" <info@your-demo.example.tld>'

And here’s my docker-compose.yaml. I set up a defined IP address so my redirect could always find it:

  GNU nano 5.4                                                                                                        docker-compose.yaml                                                                                                                  
version: '3.5'

services:
  server:
    image: cortezaproject/corteza:${VERSION}
    networks:
      proxy:
        ipv4_address: 172.18.0.2
      internal:
    restart: always
    env_file: [ .env ]
    depends_on: [ db ]
    volumes: [ "./data/server:/data" ]
    environment:
      # VIRTUAL_HOST helps NginX proxy route traffic for specific virtual host to
      # this container
      # This value is also picked up by initial boot auto-configuration procedure
      # If this is changed, make sure you change settings accordingly
      VIRTUAL_HOST: ${DOMAIN}
      # This is needed only if you are using NginX Lets-Encrypt companion
      # (see docs.cortezaproject.org for details)
      LETSENCRYPT_HOST: ${DOMAIN}

  db:
    # PostgreSQL Database
    # See https://hub.docker.com/_/postgres for details
    # Support for postgres 13, 14 and 15 is available in the latest version of Corteza
    image: postgres:15
    networks:
      internal:
    restart: always
    healthcheck: { test: ["CMD-SHELL", "pg_isready -U corteza"], interval: 10s, timeout: 5s, retries: 5 }
    environment:
      # Warning: these are values that are only used on 1st start
      #          if you want to change it later, you need to do that
      #          manually inside db container
      POSTGRES_USER:     corteza
      POSTGRES_PASSWORD: my-db-password

networks:
  internal:
    name: corteza-net
  proxy:
    name: proxy
    ipam:
      driver: default
      config:
        - subnet: 172.18.0.0/24
          gateway: 172.18.0.1

Finally - here’s what my web browser shows:

In the URL of the web browser, it has this:

https://corteza.techthrasher.com/auth/oauth2/authorize?client_id=360288286020730883&redirect_uri=https%3A%2F%2Fcorteza.techthrasher.com%2Fauth%2Fcallback&response_mode=query&response_type=code&scope=profile+api&state=iz1ff011tv

My guess is there’s something going on with redirect URLs, etc., and the auth can’t get what it needs. I’m running Yunohost, which has NGINX. Could this be an issue with the NGINX?

Here’s my NGINX config:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    listen [::]:80;
    server_name corteza.techthrasher.com;

    access_by_lua_file /usr/share/ssowat/access.lua;

    include /etc/nginx/conf.d/acme-challenge.conf.inc;

    location ^~ '/.well-known/ynh-diagnosis/' {
        alias /var/www/.well-known/ynh-diagnosis/;
    }

    

    
    
    location / {
        return 301 https://$host$request_uri;
    }
    
    

    include /etc/nginx/conf.d/yunohost_http_errors.conf.inc;

    access_log /var/log/nginx/corteza.techthrasher.com-access.log;
    error_log /var/log/nginx/corteza.techthrasher.com-error.log;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name corteza.techthrasher.com;

    include /etc/nginx/conf.d/security.conf.inc;

    ssl_certificate /etc/yunohost/certs/corteza.techthrasher.com/crt.pem;
    ssl_certificate_key /etc/yunohost/certs/corteza.techthrasher.com/key.pem;

    
    more_set_headers "Strict-Transport-Security : max-age=63072000; includeSubDomains; preload";
    
    
    # OCSP settings
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/yunohost/certs/corteza.techthrasher.com/crt.pem;
    resolver 1.1.1.1 9.9.9.9 valid=300s;
    resolver_timeout 5s;
    

    

    access_by_lua_file /usr/share/ssowat/access.lua;

    include /etc/nginx/conf.d/corteza.techthrasher.com.d/*.conf;

    include /etc/nginx/conf.d/yunohost_sso.conf.inc;
    include /etc/nginx/conf.d/yunohost_admin.conf.inc;
    include /etc/nginx/conf.d/yunohost_api.conf.inc;
    include /etc/nginx/conf.d/yunohost_http_errors.conf.inc;

    access_log /var/log/nginx/corteza.techthrasher.com-access.log;
    error_log /var/log/nginx/corteza.techthrasher.com-error.log;
}

Here’s the error from the NGINX log

2023/10/22 12:24:07 [error] 160088#160088: *3340 connect() failed (111: Connection refused) while connecting to upstream, client: 76.21.207.80, server: corteza.techthrasher.com, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:12000/", host: "corteza.techthrasher.com"

I would really appreciate any help you can give.
Thanks,
Phil

[peter]

Hi,

it definitely looks like a auth problem, not a connectivity to the container itself.
I do not see what the issue could be, you could try setting

LOG_DEBUG=true
LOG_LEVEL=debug
AUTH_LOG_ENABLED=true

and see where that gets you.

The container is running without any issues, so doing a

curl https://corteza.techthrasher.com/healthcheck

will work, but just in case.

[techthrasher]

@peter

Thank you so much for that information. I’d made so many mods that I deleted everything and started over (in a vain hope things might just work…). I’m getting the same error on the webpage. Here is the relevant output from the log:

corteza-server-1  | 10:31:55.716        DEBUG   auth    handling request        {"url": "/auth/oauth2/default-client?redirect_uri=https://corteza.techthrasher.com/auth/callback&scope=profile%20api&state=f1cs311h3ad", "method": "GET"}
corteza-server-1  | 10:31:55.810        DEBUG   auth    handling request        {"url": "/auth/oauth2/authorize?client_id=360856677244731394&redirect_uri=https%3A%2F%2Fcorteza.techthrasher.com%2Fauth%2Fcallback&response_mode=query&response_type=code&scope=profile+api&state=f1cs311h3ad", "method": "GET"}
corteza-server-1  | 10:31:55.810        DEBUG   auth    starting new oauth2 authorization flow  {"params": {"client_id":["360856677244731394"],"redirect_uri":["https://corteza.techthrasher.com/auth/callback"],"response_mode":["query"],"response_type":["code"],"scope":["profile api"],"state":["f1cs311h3ad"]}}
corteza-server-1  | 10:31:55.816        ERROR   auth    error in handler        {"error": "invalid client: not found"}
corteza-server-1  | 10:31:55.819        DEBUG   auth    template executed       {"name": "error-internal.html.tpl"}

So it’s definitely an auth issue. I’m using Safari on a Mac, so I switched to Google Chrome just to see if the browser was the issue but got the same error. So it’s definitely something with the application auth. The Yunohost redirect app basically just sets up NGINX with a snippet, pointing to localhost:port to reach the container. I wonder if that’s causing problems…

BTW - everything passed the health check, so I know the containers are healthy.

Thanks,
Phil

[techthrasher]

I’m still struggling with this.

Here is the relevant part of the Corteza log

15:58:54.720Z	debug	auth	starting new oauth2 authorization flow
{
  "params": {
    "client_id": [
      "365079651694936066"
    ],
    "redirect_uri": [
      "https://corteza.techthrasher.com/auth/callback"
    ],
    "response_mode": [
      "query"
    ],
    "response_type": [
      "code"
    ],
    "scope": [
      "profile api"
    ],
    "state": [
      "zxhq9yhpt1m"
    ]
  }
}
15:58:54.722Z	error	auth	error in handler
{
  "error": "invalid client: not found"
}

Is there anyone who might be able to help?

[techthrasher]

SOLVED: My installation is on a VPS operating under Yunohost. The Yunohost diagnostic had complained about the DNS resolver, so I changed it to make the complaint go away by setting the first resolver to 127.0.0.1. That is what broke Corteza. I figured it out after I completely wiped my VPS server, reinstalled Yunohost, and DID NOT make the DNS change. Corteza fired up without issue. My apologies for wasting everyone’s time.

[peter]

@techthrasher thank you for the postmortem, this is valuable info for us as well. Hope Corteza will work well for you