Update seems to have deleted the database?

🚧 Installation
#1

Just ran an update on a test server via docker-compose and it seems to have deleted all the data (users, custom modules and pages…).
Had to create a new user (SuperAdmin) and all is lost now.

I ran the standard stuff:
docker-compose down
docker-compose pull
docker-compose up -d

What did I miss?

#2

Hi, you didn’t miss anything, you just used the incorrect commands.

Stops containers and removes containers, networks, volumes, and images created by up.

Usually, you need only to use docker-compose up -d, that one automatically fetches the image and re-ups the container.

Use up -d when you change the .env file as well, restart does not help.

1 Like
#3

Thx for that.

I shall consider the previous data as lost or do you think there’s a way to restore it?

1 Like
#4

Actually volumes are not removed by default when running docker compose down. In the documentation linked by peter:

By default, the only things removed are:

Containers for services defined in the Compose file.
Networks defined in the networks section of the Compose file.
The default network, if one is used.
…
Anonymous volumes are not removed by default.

Try running docker compose up in the same folder, it will reuse the old volumes if the docker configuration is the same.

You can also do docker volume ls to inspect volumes that docker has stored on your machine (and for data recovery you can attach that volume onto a simple Ubuntu image and copy the data over to your host machine).

1 Like
#5

I’m encountering a similar issue, but it doesn’t have anything to do with an update. This is a newly installed, online production instance. I’ve gone as far as creating my first (admin) user, confirmed email, and can login, logout, and login again.

If I run:

docker-compose down
docker-compose up -d

then I have to start all over at the beginning again. The previously created user cannot login, and I have to setup the initial user and confirm the email address again. After that, I can login, logout, etc… until I take the container down again.

My guess is that I’ve done something wrong when setting up the file structure and permissions. The DevOps Guide states:

Make sure to change the owner to the Docker container (you can use chown 1001:1001 data/db and chown 4242:4242 data/server). Omit if you won’t use persistent storage.

I created that structure and changed owners of those two directories before starting the container the first time. But I wasn’t sure which user should be the owner of the “data” folder that is the parent of “db” and “server”. Right now, it is my logged in Ubuntu user.

With the container running, the “db” folder is empty, and the “server” folder contains a “compose” folder and a “system” folder. Both owned by root:root and both empty.

Somehow it seems my database and possibly other things (I’ve not tried to upload any files) are not persistent.

Here is my docker-compose.yaml file:

version: '3.5'

services:
  server:
    image: cortezaproject/corteza:${VERSION}
    networks: [ proxy, internal ]
    restart: always
    env_file: [ .env ]
    depends_on:
      db:
        condition: service_healthy
    volumes: [ "./data/server:/data" ]
    environment:
      # VIRTUAL_HOST helps NginX proxy route traffic for specific virtual host to
      # this container
      # This value is also picked up by initial boot auto-configuration procedure
      # If this is changed, make sure you change settings accordingly
      VIRTUAL_HOST: ${DOMAIN}
      # This is needed only if you are using NginX Lets-Encrypt companion
      # (see docs.cortezaproject.org for details)
      LETSENCRYPT_HOST: ${DOMAIN}

  db:
    # PostgreSQL Database
    # See https://hub.docker.com/_/postgres for details
    # Support for postgres 13, 14 and 15 is available in the latest version of Corteza
    image: postgres:15
    networks: [ internal ]
    restart: always
    healthcheck: { test: ["CMD-SHELL", "pg_isready -U corteza"], interval: 10s, timeout: 5s, retries: 5 }
    environment:
      # Warning: these are values that are only used on 1st start
      #          if you want to change it later, you need to do that
      #          manually inside db container
      POSTGRES_USER:     corteza
      POSTGRES_PASSWORD: corteza

networks:
  internal: {}
  proxy: { external: true }

and here is my directory listing starting from /home:
Owners are in [brackets ]

[root ] .
└── [myuser ] ./myuser
├── [myuser ] ./myuser/corteza-production
│ └── [myuser ] ./myuser/corteza-production/data
│ ├── [1001 ] ./myuser/corteza-production/data/db
│ └── [4242 ] ./myuser/corteza-production/data/server
│ ├── [root ] ./myuser/corteza-production/data/server/compose
│ └── [root ] ./myuser/corteza-production/data/server/system
└── [myuser ] ./myuser/nginx
├── [root ] ./myuser/nginx/certs
│ └── [root ] ./myuser/nginx/certs/my.domain.net
├── [root ] ./myuser/nginx/html
├── [root ] ./myuser/nginx/htpasswd
└── [root ] ./myuser/nginx/vhost.d

Thanks for any guidance!

1 Like
#6

Well, I kept working on it, educating myself on persistent docker volumes, and looking at the Corteza forums for any other hints. Here’s the database section of my docker-compose.yaml file that finally worked for me:

 db:
    # PostgreSQL Database
    # See https://hub.docker.com/_/postgres for details
    # Support for postgres 13, 14 and 15 is available in the latest version of Corteza
    image: postgres:15
    networks: [ internal ]
    volumes:
      - ./data/db/:/var/lib/postgresql/data
    restart: always
    healthcheck: { test: ["CMD-SHELL", "pg_isready -U corteza"], interval: 10s, timeout: 5s, retries: 5 }
    environment:
      # Warning: these are values that are only used on 1st start
      #          if you want to change it later, you need to do that
      #          manually inside db container
      POSTGRES_USER:     corteza
      POSTGRES_PASSWORD: corteza

After making the changes to the Volume section, and running docker-compose up -d the database files were finally created in the host folder ./data/db
I then went through the initial user setup again and confirmed I could login, logout, and login again.
I then ran: docker-compose down and: docker-compose up -d and was able to login successfully with my persisted user.

It sure seems like the DevOps guide should include the appropriate volume information for persistent database storage since they include the directory tree ( ./data/db) in the guide.

1 Like
MYSQL/POSTGRES Issues