Description:

Intro

This talk explores a technique / project layout for running your entire DevOps using just git and docker compose.

1. It simplifies differences between local / test / uat / prod environments since you can use exactly the same tools everywhere.
2. It simplifies versioning of infrastructure / services / credentials / access since everything is in git.
3. Debugging on prod etc is extremely simple. Just do what you do locally.
4. Does not depend on language so we can have projects with multiple moving parts in different languages all coordinated together via git.

Idea

• A single docker-compose.yml file in the root of the project.
• Use compose profiles to target services / environments.
• A secrets folder that uses SOPS to store secrets in git itself.
• Separate secrets per environment (dev / test / prod / ci)
• A ops / scripts folder to hold bash scripts used for shortcuts etc. For example I use:
  • dk up dev | bash to compose up local
  • dk up prod | bash to compose up prod
  • dk down prod grafana | bash to compose down prod instance of grafana
• Each service like api / frontend / databases have their own top level folder in the repo.

Services that are very helpful

• dozzle to see docker logs over web
• Grafana for dashboards / alerting via telegram or email
• Victoria metrics to ingest prometheus / other metrics
• vector.dev to forward logs / metrics to victoria.
• Glitchtip as a sentry replacement.
• Uptime kuma for monitoring our instances.
• Go Webhook to automate deploys / other actions
• Rclone for backups to S3.
• Caddy to serve as a gateway for all of this.

This is just infrastructure to monitor things. The project itself can use anything it wants. I typically set up things like this:

• Redis for caching
• Postgres database for long term storage
• Migration container
• Multiple API replicas
• Multiple worker replicas

These can be separate services / separate configurations based on the environment you're deploying to.

Speaker Info:

Hi I'm Arjoonn and I run midpathsoftware.com

I'm super interested in reliability of software and how we can get there without spending years writing tests.