docker [Cygnus]

https://github.com/docker/labs/tree/master/networking/concepts https://sysdig.com/blog/dockerfile-best-practices/
dns https://kerneltalks.com/networking/how-docker-container-dns-works/
https://labs.play-with-docker.com/

docker -D info
docker compose --ansi=never up --quiet-pull

linuxserver, hotio, and binhex - packages reddit
updates and good packages https://blog.bozdaganian.com/2020/12/12/watchtower-to-monitor-docker-image-updates/#2
unusual way https://www.youtube.com/watch?v=zfNqp85g5JM
https://github.com/docker/awesome-compose

scout - security scan

dedockify https://github.com/mrhavens/Dedockify

https://dockerlabs.collabnix.com/ - learning
base technology cgroups namespaces runtimes container-image
!!! cloud native computing foundation - https://www.cncf.io/ ↬ https://landscape.cncf.io/
https://habr.com/ru/company/timeweb/blog/558612/ - сеть контейнеров

https://www.youtube.com/watch?v=2MJn2yfa6A8 - новый docker compose и plugins смотреть

# docker find by name
docker ps -a --format "{{.ID}} {{.Names}}" --filter name=UismvProd_Db
 
# docker find thin r/w layer aufs by container id - ``docker ps`` 
cat /var/lib/docker/image/aufs/layerdb/mounts/<CONTAINERID>*/mount-id
<ID>
# real files in -> /var/lib/docker/aufs/diff/<ID>    <ID>-init - entry point  of container aufs start
 
# find container by name of catalog AUFS DIFF
grep 09c1f3c8c5adfb9241d477a61b5b04fd4cde07959dbee5ad385c4ef9c5e9d71d /var/lib/docker/image/aufs/layerdb/mounts/*/mount-id
<RESULT>
# this give id in path -> /var/lib/docker/aufs/diff/<RESULT>/etc/freeradius/
 
 
# inspecting docker with jq - mounts
docker inspect   d20d22ecad80 |  jq '.. | objects | with_entries(select(.key | contains("Mounts"))) | select(. != {})'
# inspect restart
docker inspect php4-fpm | jq '.[0] | .. | .RestartPolicy? // empty'
 
#Packages for container
apt install net-tools # - {netstat, ifconfig, iwconfig, route, iptunnel, arp}
apt install iproute2 # - {ss, ip, iw, ip r, ip tunnel, ip n}
apt install procps # - {ps,top}
apt install iputils-ping # - {ping}

Docker. Interface found

$ docker exec -it my-container cat /sys/class/net/eth0/iflink
ip ad | grep 123

#!/bin/bash
 
export containers=$(sudo docker ps --format "{{.ID}}|{{.Names}}")
export interfaces=$(sudo ip ad);
for x in $containers
        do
                export name=$(echo "$x" |cut -d '|' -f 2);
                export id=$(echo "$x"|cut -d '|' -f 1)
                export ifaceNum="$(echo $(sudo docker exec -it "$id" cat /sys/class/net/eth0/iflink) | sed s/[^0-9]*//g):"
                export ifaceStr=$( echo "$interfaces" | grep $ifaceNum | cut -d ':' -f 2 | cut -d '@' -f 1);
                echo -e "$name: $ifaceStr";
done

Docker. Show services table view

$docker ps -a --format "table {{.Names}}\t\t{{.Size}}\t\t{{.Ports}}\t\t{{.Status}}" --filter "label=com.docker.compose.project=docker-reception"
NAMES                                             SIZE                                          PORTS                                                          STATUS
docker-reception_massmess_1                       2B (virtual 44MB)                                                                                            Up 2 months
docker-reception_mysql_1                          1.92kB (virtual 406MB)                        0.0.0.0:3306->3306/tcp                                         Up 3 months
docker-reception_ldr_1                            0B (virtual 14.9MB)                                                                                          Exited (0) 4 minutes ago
mysql-exporter                                    0B (virtual 17.5MB)                           0.0.0.0:9104->9104/tcp                                         Up 3 months
docker-reception_phpgost_1                        769B (virtual 506MB)                          9000/tcp, 0.0.0.0:8080->80/tcp                                 Up 3 months
docker-reception_pma_1                            63.4kB (virtual 89.7MB)                       0.0.0.0:8081->8080/tcp                                         Up 2 months
docker-reception_nginx_1                          2B (virtual 126MB)                            0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp                       Up 3 months

Docker. determining container responsible for largest overlay directories

# huge containers overlay container size
# as root $ sudo su     # make sure json parser is installed  $ apt-get install jq -y 
 
# grab the size and path to the largest overlay dir
du /var/lib/docker/overlay2 -h | sort -h | tail -n 100 | grep -vE "overlay2$" > /tmp/large-overlay.txt
 
# construct mappings of name to hash
docker inspect $(docker ps -qa) | jq -r 'map([.Name, .GraphDriver.Data.MergedDir]) | .[] | "\(.[0])\t\(.[1])"' > /tmp/docker-mappings.txt
 
# for each hashed path, find matching container name
cat /tmp/large-overlay.txt | xargs -l bash -c 'if grep $1 /tmp/docker-mappings.txt; then echo -n "$0 "; fi'

BuildKit

docker log bash monitoring

https://www.slideshare.net/brendangregg/container-performance-analysis

https://nickjanetakis.com/blog/docker-tip-3-chain-your-docker-run-instructions-to-shrink-your-images

https://blog.alexellis.io/mutli-stage-docker-builds/

containers/images/volumes
Link - how backup a docker container

Docker Volume Backup

# Start backup 
cat ./start_docker_backup.sh
#!/bin/sh
docker run -it --rm -e "DOCKER_HOST=$(hostname)" --name cont_bckp \
   --rm `docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'` \
     -v /opt/docker_bckp/:/mnt/bckp  alpine /bin/ash /mnt/bckp/backup.sh
 
# script for 
cat << EOF >backup.sh
#!/bin/sh
 
ENTRYP="/mnt"
BACKPATH="/mnt/bckp"
PREFIXH=${DOCKER_HOST:-$HOSTNAME}
 
 
echo "$(date +%F_%T) : Start backup docker volumes from $PREFIXH to -> $BACKPATH"
 
for catalog in "$ENTRYP"/*/; do
  base=$(basename "$catalog")
  if [ $base != "bckp" ];  then
    FILEBCKPNAME="docker_vol_bckp_${PREFIXH}_$(date +%F%H%M)_$base.tar.gz"
    FILEBCKP="$BACKPATH/$FILEBCKPNAME"
    echo " ---  $(date +%T) $base ->  $FILEBCKP"
    (cd "$ENTRYP" && tar -czf "$FILEBCKP" "$base" )
    FILESIZE=$(stat -c%s "$FILEBCKP")
    MB=$(( $FILESIZE / 1024 / 1024 ))
    echo "            ->  finished size $FILESIZE ($MB MB)"
 
    cat << EOF >> $BACKPATH/restoresh.log
cat ./$FILEBCKPNAME | docker run -i --rm --name cont_restore --rm \`docker volume list -q  \
 | egrep -v '^.{64}$' | awk '{print "-v " \$1 ":/mnt/" \$1}'\` -v /opt/docker_bckp/:/mnt/bckp   alpine  tar -zxvf - -C /mnt
EOF
 
 
#cat $FILEBCKP | docker run -it --rm --name cont_restore --rm \`docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " \$1 ":/mnt/" \$1}'\` -v /opt/docker_bckp/:/mnt/bckp  alpine  tar -zxvf - -C $ENTRYP
 
#|tar -x -f-"
#|tar zxvf -
#/bin/ash
 
#cat <<'EOF' >> $BACKPATH/restoresh.log
#tar -c -f- $FILEBCKP | docker run -i --rm --mount source=data_volume,destination=/data alpine tar -x -f-
#docker run -it --rm --name cont_restore --rm \`docker volume list -q | egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'\` -v /opt/docker_bckp/:/mnt/bckp  alpine cmd
#EOF
  fi
 
done
 
echo "$(date +%F_%T) : finish backup docker volumes"
 
EOF

Docker Volume Restore

# before restore stop service
docker service scale portainer_portainer=0
 
# clear directory and than restore from restore_sh.log
# example on HOST
cat ./docker_vol_bckp_test-swarm-worker-02_2020-02-091624_portainer_portainer_data.tar.gz | \
  docker run -i --rm --name cont_restore --rm `docker volume list -q   | \
  egrep -v '^.{64}$' | awk '{print "-v " $1 ":/mnt/" $1}'` -v /opt/docker_bckp/:/mnt/bckp \
  alpine  tar -zxvf - -C /mnt
 
# before start container
docker service scale portainer_portainer=1

how to set user and group in docker compose/ https://blog.giovannidemizio.eu/2021/05/24/how-to-set-user-and-group-in-docker-compose/

🐳 Updating all Docker Compose container docker-compose-example

Ddocker-compose.yml

version: "3"
 
services:
  db:
    image: percona:5.7
    volumes:
      - dbdata:/var/lib/mysql
 
  db-backup:
    image: alpine    
    tty: false
    environment:
      - TARGET=dbdata
    volumes:
      - ./backup:/backup
      - dbdata:/volume
    command: sh -c "tar -cjf /backup/$${TARGET}.tar.bz2 -C /volume ./"
 
  db-restore:
    image: alpine    
    environment:
      - SOURCE=dbdata
    volumes:
      - ./backup:/backup
      - dbdata:/volume
    command: sh -c "rm -rf /volume/* /volume/..?* /volume/.[!.]* ; tar -C /volume/ -xjf /backup/$${SOURCE}.tar.bz2"

https://github.com/jlesage/docker-baseimage-gui

https://www.youtube.com/watch?v=bKFMS5C4CG0

https://traefik.io/blog/traefik-2-tls-101-23b4fbee81f1/ - traefic blog , HTTPS TLS traefic configuration on docker

https://www.slideshare.net/brendangregg/container-performance-analysis - container performance analisys

Process data - find container and do …

#!/usr/bin/env bash
 
#args=("$@")
#echo "args:${args[*]}"
 
LOG=sz_cron_entrypoint_backup.log
 
echo $(date) > $LOG
 
IFS=$'\n'
for line in $(docker ps -a --format "{{.ID}} {{.Names}}" --filter name=UismvProd_Db)
do
        unset IFS
        array=($line)
        echo " Find container name:${array[1]}(id:${array[0]}) -> process wal-g" >> $LOG
        /usr/bin/docker exec  ${array[0]}  /home/wal-g/cron_do_backup.sh >> $LOG
done

Docker

Docker.goodPackages

Docker.scout

Docker.dedockify

Docker.Learning

Docker.commands

Docker. Interface found

Docker. Show services table view

Docker. determining container responsible for largest overlay directories

Docker BuildKit

Docker monitoring

Docker monitoring Prometheus

Docker Performance

Docker Optimize

Docker multi stage builds

Docker Debug

Docker Backup

Docker volumes backup/restore

Docker-compose set variables in docker compose UID GID

Docker-compose backup/restore

Docker baseimage to ease creation of X graphical application containers GUI

Docker Network

Docker Traefik

Docker.Performance container perf

Docker.Swarm

Docker.ansible

Cygnus