Day 02: Docker Dir Change | Custom Network | Host | AWS EBS Volume
"Hello, I'm Kiran Pawar, a passionate Cloud and Devops Engineer with a strong background in cloud automation, configuration, and deployment. My journey in the world of technology has been a thrilling adventure, where I've had the privilege to work with cutting-edge tools and practices.
🚀 As a DevOps Engineer:
I specialize in automating, configuring, and deploying instances in cloud environments and data centers. My expertise extends to DevOps, GitOps, CI/CD pipeline management, HashiCorp Terraform, and containerization. I'm proficient in AWS and Linux/Unix administration, ensuring robust infrastructure and application performance.
🔧 My Tech Stack:
Front-end skills: HTML, CSS, SCSS, Tailwind CSS, Bootstrap, React, Material-UI, JavaScript DevOps toolbox: GIT, OWASP,Nexus,Trivy, Github, Gitlab, Terraform, Ansible, Docker, Kubernetes, Helm, Jenkins, Prometheus, Grafana, Argo CD, AWS EKS.
🌐 My Cloud Expertise:
I have hands-on experience managing AWS services, including EC2, S3, EBS, VPC, ELB, RDS, IAM, Route53, and more.
🔒 Networking and Security:
My skills include managing networking concepts such as TCP/IP protocols, security policies, and subnet interfacing. I have a strong understanding of infrastructure and networking, covering topics like firewalls, IP addressing, DNS, and more.
💡 What Sets Me Apart:
I bring a positive attitude, a strong work ethic, and a collaborative spirit to every project. I'm a self-starter, a fast learner, and an effective team player with strong interpersonal skills. In addition to my DevOps skills, I've developed shell scripts (Bash) for automating tasks and have proficiency in Python scripting. My ability to communicate and manage projects, along with a track record of resolving client issues, adds value to every team I work with. If you're looking for a DevOps engineer who is also well-versed in front-end technologies, feel free to connect with me. Let's explore new possibilities and create exceptional technical solutions together!"
Why We Need Separate Utilization for Docker Data
The default directory for Docker is /var/lib/docker. As you continue downloading images and generating logs, this directory will consume more space and eventually get full. To prevent this, we can store all our Docker data in a separate directory.
Steps To Create EBS Volume and Attach It to the Instance
Create an EBS volume (GP2) and attach it to the instance
Check the attached volume:
lsblk
Create a new partition:
fdisk /dev/xvdfFollow the sequence:
n(New partition)p(Primary partition)w(Write changes and exit)
Check the partition:
lsblk
Format the partition:
mkfs.ext4 /dev/xvdf1
Copy the UUID of the partition:
blkid /dev/xvdf1
Create a directory for Docker data:
mkdir /dockerdata
Update
/etc/fstabto mount the volume at boot:vim /etc/fstabAdd the following entry:
UUID=<your-uuid> /dockerdata ext4 defaults,nofail 0 2
Moving Docker Data to a New Directory
Stop the Docker service:
sudo systemctl stop docker.socket sudo systemctl stop docker.serviceModify the Docker service file:
sudo vim /lib/systemd/system/docker.service
Change:
ExecStart=/usr/bin/dockerd --data-root /dockerdata -H fd:// --containerd=/run/containerd/containerd.sock
Sync existing data:
sudo rsync -aqxP /var/lib/docker/ /dockerdataRestart Docker:
sudo systemctl daemon-reload && sudo systemctl start dockerVerify Docker is running:
sudo systemctl status docker --no-pager ps aux | grep -i docker | grep -v grep
Verify dockerdata directory for storage previously it was 3% after running docker container it is modifies to 18% s
Why We Need a Custom Network for Containers
By default, containers use the bridge network, which allows communication via IP addresses. However, to enable communication using container names, we need a custom network.
Steps to Create container in Default Bridge network
(allows communication via IP addresses)
Run containers with the default network:
docker run --rm -d --name app1 -p 8001:80 kiran2361993/troubleshootingtools:v1 docker run --rm -d --name app2 -p 8002:80 kiran2361993/troubleshootingtools:v1
Get the IP address of the app1 and app2 container
Go inside app1 container:
docker exec -it app1 bashTest connectivity inside a container:
ping ip-address-of-app2-container
we can see we can connect with ip-address of container app2 but we can’t ping with the container app2 for that we need to create our own custom network
Steps to Create and Use a Custom Network
(allows communication via container names)
Create a custom bridge network:
docker network create myapp --driver bridgeInspect the network:
docker network inspect myapp
Run containers with the custom network:
docker run --rm -d --name app3 -p 8003:80 --network myapp kiran2361993/troubleshootingtools:v1 docker run --rm -d --name app4 -p 8004:80 --network myapp kiran2361993/troubleshootingtools:v1
Test connectivity inside a container:
docker exec -it app3 bashTest connectivity inside a container:
ping app4
we can add app1 container to custom network name myapp
Connect app1 container to newly created netwrok myapp:docker network connect myapp app1
Using the HOST Network Mode
In host mode, the container shares the host's IP address. This removes the need for port forwarding.
Example:
docker run --rm -d --name node-exporter --network host prom/node-exporter
The service is now accessible on port 9100 using the host's IP.
To inspect the Docker image:
docker image inspect prom/node-exporter:latest
Using the NONE Network Mode
In none mode, the container has no network except for a loopback device. This is useful for isolated environments.
Example:
docker run --rm -d --name isolated-container --network none busybox
By following these steps, we ensure efficient storage utilization and a more robust networking setup for Docker containers.
