===== Docker ===== https://github.com/docker/labs/tree/master/networking/concepts https://sysdig.com/blog/dockerfile-best-practices/ \\ dns https://kerneltalks.com/networking/how-docker-container-dns-works/ \\ https://labs.play-with-docker.com/ \\ docker -D info docker compose --ansi=never up --quiet-pull ===== Docker.goodPackages ===== * [[https://github.com/linuxserver|linuxserver]], [[https://github.com/hotio|hotio]], and [[https://github.com/binhex?tab=repositories|binhex]] - packages [[https://www.reddit.com/r/unRAID/comments/18fcq05/binhex_who/|reddit]] * updates and good packages https://blog.bozdaganian.com/2020/12/12/watchtower-to-monitor-docker-image-updates/#2 * unusual way https://www.youtube.com/watch?v=zfNqp85g5JM * https://github.com/docker/awesome-compose ===== Docker.scout ===== * scout - security scan ===== Docker.dedockify ===== * dedockify https://github.com/mrhavens/Dedockify ==== Docker.Learning ==== * https://dockerlabs.collabnix.com/ - learning * base technology [[https://opensource.com/article/21/8/container-linux-technology|cgroups namespaces]] [[https://opensource.com/article/21/9/container-runtimes|runtimes]] [[https://opensource.com/article/21/8/container-image|container-image]] * :!: !!! cloud native computing foundation - https://www.cncf.io/ ↬ https://landscape.cncf.io/ * https://habr.com/ru/company/timeweb/blog/558612/ - сеть контейнеров * https://www.youtube.com/watch?v=2MJn2yfa6A8 - новый docker compose и plugins смотреть * !!! https://training.play-with-docker.com/ * !!! https://www.docker.com/blog/best-way-learn-docker-free-play-docker-pwd/ * https://developers.redhat.com/blog/2016/02/24/10-things-to-avoid-in-docker-containers/ * https://crate.io/a/analyzing-docker-container-performance-native-tools/ * [[https://github.com/dockersamples|DockerSamples]] * [[https://medium.com/trendyol-tech/how-we-reduce-node-docker-image-size-in-3-steps-ff2762b51d5a|Docker reduce images]] * [[https://habr.com/ru/company/flant/blog/336654/|Шпаргалка]] * gui - kitematic https://kitematic.com/ * [[https://res.cloudinary.com/snyk/image/upload/v1551798390/Docker_Image_Security_Best_Practices_.pdf|Docker_Image_Security_Best_Practices]] * inspect containers without started https://www.howtogeek.com/devops/how-to-inspect-a-docker-images-content-without-starting-a-container/ ==== Docker.commands ==== # docker find by name docker ps -a --format "{{.ID}} {{.Names}}" --filter name=UismvProd_Db # docker find thin r/w layer aufs by container id - ``docker ps`` cat /var/lib/docker/image/aufs/layerdb/mounts/*/mount-id # real files in -> /var/lib/docker/aufs/diff/ -init - entry point of container aufs start # find container by name of catalog AUFS DIFF grep 09c1f3c8c5adfb9241d477a61b5b04fd4cde07959dbee5ad385c4ef9c5e9d71d /var/lib/docker/image/aufs/layerdb/mounts/*/mount-id # this give id in path -> /var/lib/docker/aufs/diff//etc/freeradius/ # inspecting docker with jq - mounts docker inspect d20d22ecad80 | jq '.. | objects | with_entries(select(.key | contains("Mounts"))) | select(. != {})' # inspect restart docker inspect php4-fpm | jq '.[0] | .. | .RestartPolicy? // empty' #Packages for container apt install net-tools # - {netstat, ifconfig, iwconfig, route, iptunnel, arp} apt install iproute2 # - {ss, ip, iw, ip r, ip tunnel, ip n} apt install procps # - {ps,top} apt install iputils-ping # - {ping} === Docker. Interface found === $ docker exec -it my-container cat /sys/class/net/eth0/iflink ip ad | grep 123 #!/bin/bash export containers=$(sudo docker ps --format "{{.ID}}|{{.Names}}") export interfaces=$(sudo ip ad); for x in $containers do export name=$(echo "$x" |cut -d '|' -f 2); export id=$(echo "$x"|cut -d '|' -f 1) export ifaceNum="$(echo $(sudo docker exec -it "$id" cat /sys/class/net/eth0/iflink) | sed s/[^0-9]*//g):" export ifaceStr=$( echo "$interfaces" | grep $ifaceNum | cut -d ':' -f 2 | cut -d '@' -f 1); echo -e "$name: $ifaceStr"; done === Docker. Show services table view === $docker ps -a --format "table {{.Names}}\t\t{{.Size}}\t\t{{.Ports}}\t\t{{.Status}}" --filter "label=com.docker.compose.project=docker-reception" NAMES SIZE PORTS STATUS docker-reception_massmess_1 2B (virtual 44MB) Up 2 months docker-reception_mysql_1 1.92kB (virtual 406MB) 0.0.0.0:3306->3306/tcp Up 3 months docker-reception_ldr_1 0B (virtual 14.9MB) Exited (0) 4 minutes ago mysql-exporter 0B (virtual 17.5MB) 0.0.0.0:9104->9104/tcp Up 3 months docker-reception_phpgost_1 769B (virtual 506MB) 9000/tcp, 0.0.0.0:8080->80/tcp Up 3 months docker-reception_pma_1 63.4kB (virtual 89.7MB) 0.0.0.0:8081->8080/tcp Up 2 months docker-reception_nginx_1 2B (virtual 126MB) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp Up 3 months === Docker. determining container responsible for largest overlay directories === # huge containers overlay container size # as root $ sudo su # make sure json parser is installed $ apt-get install jq -y # grab the size and path to the largest overlay dir du /var/lib/docker/overlay2 -h | sort -h | tail -n 100 | grep -vE "overlay2$" > /tmp/large-overlay.txt # construct mappings of name to hash docker inspect $(docker ps -qa) | jq -r 'map([.Name, .GraphDriver.Data.MergedDir]) | .[] | "\(.[0])\t\(.[1])"' > /tmp/docker-mappings.txt # for each hashed path, find matching container name cat /tmp/large-overlay.txt | xargs -l bash -c 'if grep $1 /tmp/docker-mappings.txt; then echo -n "$0 "; fi' ==== Docker BuildKit==== [[https://docs.docker.com/develop/develop-images/build_enhancements/|BuildKit]] ==== Docker monitoring ==== * [[https://dev.to/themreza/monitoring-and-logging-docker-events-59oi|docker log bash monitoring]] ==== Docker monitoring Prometheus ==== * [[https://valyala.medium.com/prometheus-vs-victoriametrics-benchmark-on-node-exporter-metrics-4ca29c75590f|Prometheus vs VictoriaMetrics]] * [[https://habr.com/ru/company/southbridge/blog/455290/|Полное руководство по Prometheus в 2019 году]] * [[https://stefanprodan.com/2016/a-monitoring-solution-for-docker-hosts-containers-and-containerized-services/]] * [[https://scoutapm.com/blog/prometheus-and-docker-monitoring-your-environment]] ==== Docker Performance ==== - https://www.slideshare.net/brendangregg/container-performance-analysis ==== Docker Optimize ==== - https://nickjanetakis.com/blog/docker-tip-3-chain-your-docker-run-instructions-to-shrink-your-images ==== Docker multi stage builds ==== https://blog.alexellis.io/mutli-stage-docker-builds/ ==== Docker Debug ==== - [[https://github.com/BretFisher/node-docker-good-defaults|node-docker-good-defaults]] - [[https://medium.com/@betz.mark/ten-tips-for-debugging-docker-containers-cde4da841a1d|10 tips for for-debugging-docker-containers]] - [[https://subscription.packtpub.com/book/virtualization_and_cloud/9781787286986/6/ch06lvl1sec38/debugging-a-dockerfile|debugging-a-dockerfile ]] ==== Docker Backup ==== * containers/images/volumes * [[https://stackoverflow.com/questions/26331651/how-can-i-backup-a-docker-container-with-its-data-volumes|Link - how backup a docker container]] ==== Docker volumes backup/restore ==== ++++ Docker Volume Backup| # Start backup cat ./start_docker_backup.sh #!/bin/sh docker run -it --rm -e "DOCKER_HOST=$(hostname)" --name cont_bckp \ --rm `docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'` \ -v /opt/docker_bckp/:/mnt/bckp alpine /bin/ash /mnt/bckp/backup.sh # script for cat << EOF >backup.sh #!/bin/sh ENTRYP="/mnt" BACKPATH="/mnt/bckp" PREFIXH=${DOCKER_HOST:-$HOSTNAME} echo "$(date +%F_%T) : Start backup docker volumes from $PREFIXH to -> $BACKPATH" for catalog in "$ENTRYP"/*/; do base=$(basename "$catalog") if [ $base != "bckp" ]; then FILEBCKPNAME="docker_vol_bckp_${PREFIXH}_$(date +%F%H%M)_$base.tar.gz" FILEBCKP="$BACKPATH/$FILEBCKPNAME" echo " --- $(date +%T) $base -> $FILEBCKP" (cd "$ENTRYP" && tar -czf "$FILEBCKP" "$base" ) FILESIZE=$(stat -c%s "$FILEBCKP") MB=$(( $FILESIZE / 1024 / 1024 )) echo " -> finished size $FILESIZE ($MB MB)" cat << EOF >> $BACKPATH/restoresh.log cat ./$FILEBCKPNAME | docker run -i --rm --name cont_restore --rm \`docker volume list -q \ | egrep -v '^.{64}$' | awk '{print "-v " \$1 ":/mnt/" \$1}'\` -v /opt/docker_bckp/:/mnt/bckp alpine tar -zxvf - -C /mnt EOF #cat $FILEBCKP | docker run -it --rm --name cont_restore --rm \`docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " \$1 ":/mnt/" \$1}'\` -v /opt/docker_bckp/:/mnt/bckp alpine tar -zxvf - -C $ENTRYP #|tar -x -f-" #|tar zxvf - #/bin/ash #cat <<'EOF' >> $BACKPATH/restoresh.log #tar -c -f- $FILEBCKP | docker run -i --rm --mount source=data_volume,destination=/data alpine tar -x -f- #docker run -it --rm --name cont_restore --rm \`docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'\` -v /opt/docker_bckp/:/mnt/bckp alpine cmd #EOF fi done echo "$(date +%F_%T) : finish backup docker volumes" EOF ++++ ++++ Docker Volume Restore| # before restore stop service docker service scale portainer_portainer=0 # clear directory and than restore from restore_sh.log # example on HOST cat ./docker_vol_bckp_test-swarm-worker-02_2020-02-091624_portainer_portainer_data.tar.gz | \ docker run -i --rm --name cont_restore --rm `docker volume list -q | \ egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'` -v /opt/docker_bckp/:/mnt/bckp \ alpine tar -zxvf - -C /mnt # before start container docker service scale portainer_portainer=1 ++++ ==== Docker-compose set variables in docker compose UID GID ==== * how to set user and group in docker compose/ https://blog.giovannidemizio.eu/2021/05/24/how-to-set-user-and-group-in-docker-compose/ ==== Docker-compose backup/restore ==== * [[https://thibmaek.com/post/updating-all-docker-compose-container|🐳 Updating all Docker Compose container]] [[https://github.com/thibmaek/demo-collections/tree/master/update-all-compose-containers-per-folder|docker-compose-example]] ++++ Ddocker-compose.yml| version: "3" services: db: image: percona:5.7 volumes: - dbdata:/var/lib/mysql db-backup: image: alpine tty: false environment: - TARGET=dbdata volumes: - ./backup:/backup - dbdata:/volume command: sh -c "tar -cjf /backup/$${TARGET}.tar.bz2 -C /volume ./" db-restore: image: alpine environment: - SOURCE=dbdata volumes: - ./backup:/backup - dbdata:/volume command: sh -c "rm -rf /volume/* /volume/..?* /volume/.[!.]* ; tar -C /volume/ -xjf /backup/$${SOURCE}.tar.bz2" ++++ ===== Docker baseimage to ease creation of X graphical application containers GUI===== * https://github.com/jlesage/docker-baseimage-gui ===== Docker Network ===== * https://www.youtube.com/watch?v=bKFMS5C4CG0 ===== Docker Traefik ===== * https://traefik.io/blog/traefik-2-tls-101-23b4fbee81f1/ - traefic blog , HTTPS TLS traefic configuration on docker ===== Docker.Performance container perf ===== * https://www.slideshare.net/brendangregg/container-performance-analysis - container performance analisys ===== Docker.Swarm ===== * https://gabrieltanner.org/blog/docker-swarm * [[https://dockerswarm.rocks/|dockerswarm rocks]] * [[https://github.com/stefanprodan/swarmprom|swarmprom]] https://geek-cookbook.funkypenguin.co.nz/recipes/swarmprom/#chefs-notes * Process data - find container and do ... #!/usr/bin/env bash #args=("$@") #echo "args:${args[*]}" LOG=sz_cron_entrypoint_backup.log echo $(date) > $LOG IFS=$'\n' for line in $(docker ps -a --format "{{.ID}} {{.Names}}" --filter name=UismvProd_Db) do unset IFS array=($line) echo " Find container name:${array[1]}(id:${array[0]}) -> process wal-g" >> $LOG /usr/bin/docker exec ${array[0]} /home/wal-g/cron_do_backup.sh >> $LOG done ===== Docker.ansible ===== - https://blog.ruanbekker.com/blog/2018/06/14/deploy-docker-swarm-using-ansible/ - https://labouardy.com/setup-docker-swarm-on-aws-using-ansible-terraform/ - https://medium.com/@mbovo/one-click-deploy-docker-swarm-with-ansible-9a1f7e7d0e75