--- title: Essential Linux Commands Every DEVOPS Engineer Should Know Part2 description: 'Essential Linux commands for DevOps (Part 2): ss, ip, curl, dig for networking plus docker exec, kubectl exec, and crictl for container debugging.' url: https://nerdleveltech.com/essential-linux-commands-every-devops-engineer-should-know-part2 language: en date: '2023-01-15' updated: '2026-03-27' category: cloud-devops tags: - linux - devops - command-line - recreated markdown_url: https://nerdleveltech.com/essential-linux-commands-every-devops-engineer-should-know-part2.md attribution: 'Source: NerdLevelTech — https://nerdleveltech.com/essential-linux-commands-every-devops-engineer-should-know-part2' terms: https://nerdleveltech.com/llms.txt --- # Essential Linux Commands Every DEVOPS Engineer Should Know Part2 ## TL;DR Network diagnostics with `ss` (replaces `netstat`), `ip` (replaces `ifconfig`), `curl`, and `dig`; firewalls have likewise moved from `iptables` to `nftables` on RHEL 8+ and Debian 11+; container debugging via `docker exec`, `kubectl exec`, `kubectl debug`, and `crictl`; real-time monitoring with `htop` and `btop`; packet capture with `tcpdump` (classic BPF) and deeper kernel tracing with eBPF tools like `bpftrace` and `bcc`. ## Introduction Building on Part 1's file and permission commands, Part 2 dives into the commands DevOps engineers use daily for troubleshooting, monitoring, and container operations. In 2026, as containerized workloads dominate infrastructure, understanding both traditional network commands and modern container debugging tools is essential. This section covers the networking toolkit, system monitoring commands, and container-specific operations that separate experienced DevOps engineers from beginners. ## Network Diagnostics: The Modern Toolkit `ifconfig` and `netstat` (from the `net-tools` package) have been deprecated for years and are no longer installed by default on most modern distros — RHEL/Rocky, Fedora, and recent Debian/Ubuntu releases ship `iproute2` (`ip`, `ss`) instead. The newer tools are faster, more featureful, and read directly from the kernel's netlink interface. **IP Configuration (replaces ifconfig):** ```bash # Legacy (often not installed by default): ifconfig # Display all interfaces # Modern (iproute2): ip addr show # Show all IP addresses (alias: ip a) ip addr show eth0 # Show specific interface ip link set eth0 up # Bring interface up ip link set eth0 down # Bring interface down ip route show # Show routing table (alias: ip r) sudo ip route add 192.168.0.0/24 via 10.0.0.1 # Add static route (needs root) ip -s link show eth0 # Per-interface counters (rx/tx packets, errors, drops) ``` The `ip` command (from `iproute2`) is faster, more powerful, and the standard tool across all modern Linux distributions. Use it for all interface configuration tasks. Note: for firewalling, the analogous shift is `iptables` → `nftables` — RHEL/Rocky 8+ and Debian 11+ default to the `nftables` backend, with `iptables-nft` providing a compatibility wrapper for legacy scripts. New rules should be written for `nft` directly. **Network Connections (replaces netstat):** ```bash # Legacy (often not installed; slow on hosts with many sockets): netstat -tlnp # Show listening TCP ports + program # Modern replacement: ss (socket statistics) ss -tlnp # Show listening ports (TCP, listening, numeric, program) ss -tlnpu # Include UDP sockets ss -atnepo # Show all connections with extended info ss -tan state established # Show established TCP connections (filter syntax) ss -tan | grep ESTAB # Same idea via grep (ss prints ESTAB, not ESTABLISHED) sudo ss -K dst 192.168.1.100 # Kill matching sockets — capital -K, requires CONFIG_INET_DIAG_DESTROY in kernel ``` The `ss` command talks directly to the kernel via the netlink/sock_diag interface, which makes it far faster than `netstat` — crucial when debugging connection issues on systems with thousands of open sockets. **Understanding ss flags:** - `-t`: TCP sockets - `-u`: UDP sockets - `-l`: Listening sockets only - `-a`: All sockets - `-n`: Show numeric addresses (don't resolve hostnames) - `-p`: Show process name/PID - `-e`: Extended information - `-o`: Timer information **DNS Resolution (dig):** ```bash dig example.com # Full DNS query dig example.com +short # Short answer only dig example.com MX # Mail server records dig @8.8.8.8 example.com # Query specific nameserver dig example.com +trace # Trace entire DNS path nslookup example.com # Alternative (older but still useful) ``` When Kubernetes service discovery fails or DNS resolution breaks, `dig` provides detailed debugging information showing exact DNS responses. **Testing HTTP Connectivity:** ```bash curl https://example.com # Simple GET request curl -I https://example.com # Headers only (useful for status codes) curl -X POST -d '{"key":"value"}' https://example.com # POST with JSON curl -H "Authorization: Bearer TOKEN" https://example.com # Custom headers curl --resolve example.com:443:192.168.1.100 https://example.com # Override DNS curl -w "%{http_code}\n" -o /dev/null https://example.com # Check status code only ``` For API debugging and quick connectivity tests, `curl` is indispensable in DevOps work. ## Container Debugging Commands ### Docker Debugging ```bash docker ps # List running containers docker ps -a # List all containers (including stopped) docker logs # View container output docker logs -f # Follow logs in real-time docker exec -it /bin/bash # Open shell in running container docker exec cat /etc/hosts # Run command without interactive shell docker inspect # View detailed container configuration docker top # View running processes inside container docker stats # Real-time resource usage (memory, CPU) docker network ls # List networks docker network inspect # Inspect specific network ``` The `docker exec -it` combination is critical for debugging application issues without rebuilding containers. The `-i` flag keeps stdin open, and `-t` allocates a pseudo-terminal. ### Kubernetes Debugging ```bash kubectl exec -it -- /bin/sh # Open shell in pod (sh works on minimal/distroless-ish images) kubectl exec -it -- /bin/bash # Use bash if the image actually ships it kubectl exec -- cat /var/log/app # Run a one-shot command in the pod kubectl logs # View pod logs kubectl logs -f # Follow pod logs kubectl logs -c # Logs from a specific container kubectl logs --tail=50 # Last 50 lines kubectl logs --previous # Logs from the previous (crashed) container instance kubectl describe pod # Detailed pod information (events, conditions, last state) kubectl get events -n --sort-by=.lastTimestamp # Recent events kubectl top pod # Resource usage (requires metrics-server) kubectl port-forward 8080:8080 # Forward local port to pod kubectl cp :/path/to/file ./local # Copy file from pod kubectl debug -it --image=busybox --target= # Ephemeral debug container (no shell needed in image) ``` When a pod crashes or misbehaves, `kubectl logs --previous` and `kubectl describe` are your first stop — they show the last-state reason and recent events. For images that don't ship a shell at all (distroless, scratch), use `kubectl debug` to attach an ephemeral debug container instead of trying to `exec` into nothing. ### Container Runtime (crictl) For systems using container runtimes like containerd (common in Kubernetes), use `crictl`: ```bash crictl ps # List running containers crictl logs # View container logs crictl exec -it /bin/bash # Shell into container crictl inspect # Container details crictl images # List pulled images ``` `crictl` is essential on Kubernetes nodes where Docker may not be installed but containerd is. ## System Monitoring and Performance ### htop (Interactive Process Monitor) ```bash htop # Interactive process list htop -p # Monitor specific process htop -u # Monitor specific user's processes # Within htop: press F6 to sort by different columns (CPU, memory, etc.) ``` `htop` is far superior to `top`, providing color-coded output, better sorting, and an interactive interface. ### btop (Modern Resource Monitor) ```bash btop # Full system resource overview # Shows: CPU, memory, disks, network, processes # Interactive: sort by column, search processes, kill processes ``` `btop` is a modern reimplementation of `top` and `htop` with a graphical terminal interface showing CPU, memory, disk, and network in one view. ### Disk I/O Monitoring ```bash sudo iotop # Real-time disk I/O by process (needs root) sudo iotop -o # Only show processes currently doing I/O sudo iotop -b -n 5 # Batch mode, 5 iterations (good for logging) iostat -x 1 # Extended I/O statistics, refresh every 1 second dstat -cdn # CPU, disk, and network in one view # Note: the original dstat (by Dag Wieers) is unmaintained. Fedora/RHEL ship a Performance # Co-Pilot (PCP)-based reimplementation under the same `dstat` name; `dool` is a separate # community fork of the original that stays source-compatible and is available via pip/apt ``` When experiencing disk I/O bottlenecks, `iotop` immediately shows which processes are accessing disk heavily. ### Network Traffic Monitoring ```bash nethogs # Bandwidth usage by process nethogs -r # Reverse sorting (highest bandwidth first) iftop # Live network bandwidth by host iftop -P # Show ports nload # Simple bandwidth graph ``` `nethogs` shows which application is consuming bandwidth — critical for identifying noisy neighbors in shared infrastructure. ### Memory Analysis ```bash free -h # Human-readable memory usage free -h -s 2 # Refresh every 2 seconds ps aux --sort=-%mem | head -10 # Top 10 memory-consuming processes pmap -x # Memory map of specific process smem -k -s swap -r # Per-process memory sorted by swap, descending (apt/dnf install smem) ``` ## Packet Capture and eBPF Observability ### Packet capture with tcpdump `tcpdump` is the classic packet-capture tool. It uses the original BPF (classic Berkeley Packet Filter) via libpcap to filter packets in the kernel — note this is *not* the same as eBPF, despite the shared lineage. ```bash sudo tcpdump -i eth0 # Capture all traffic on eth0 sudo tcpdump -i eth0 port 80 # Capture HTTP traffic sudo tcpdump -i eth0 src 192.168.1.100 # Capture traffic from specific IP sudo tcpdump -i eth0 -w capture.pcap # Save to file for analysis sudo tcpdump -i eth0 -A # Print packet contents (ASCII) ``` ### eBPF-based observability eBPF (extended BPF) lets you run sandboxed programs in the kernel without recompiling it or loading custom modules. Several mature tools build on it: - **bpftrace** — high-level tracing language for kprobes, uprobes, tracepoints, and USDT events - **bcc** (BPF Compiler Collection) — Python/C toolkit with utilities like `execsnoop`, `tcplife`, `biolatency`, `opensnoop` - **falco** — runtime security monitoring; ships a modern CO-RE eBPF probe alongside its older kernel-module driver - **Cilium / Hubble** — networking and observability for Kubernetes built on eBPF Example with bpftrace — list every file opened, system-wide: ```bash sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }' ``` ## Process Management ```bash ps aux # List all running processes ps aux | grep nginx # Find specific process ps -ef --forest # Process tree (hierarchical view) pgrep -a python # Find python processes; -a prints full command line (-l prints just the name) pgrep -f "manage.py runserver" # Match against full command line, not just process name pkill -f "old_process" # Send SIGTERM to processes matching command line pkill -TERM -f "old_process" # Same as above, signal made explicit kill # Send SIGTERM (graceful shutdown) kill -9 # Send SIGKILL (force kill, last resort) killall nginx # Kill all processes named exactly "nginx" (dangerous on shared hosts!) fg # Bring background job to foreground bg # Resume stopped job in background jobs # List background jobs wait # Wait for process to complete ``` `pgrep`/`pkill` are usually preferable to `killall` because they match against the full command line with `-f` (so you can target a specific Python script, not every process named `python`), and you can preview matches with `pgrep` before signalling them. ## System Information ```bash uname -a # Kernel version and system info lsb_release -a # Linux distribution version cat /proc/cpuinfo # CPU information cat /proc/meminfo # Memory information lscpu # CPU architecture summary lsblk # Block devices and partitions lspci # PCI devices (GPUs, NICs) lsusb # USB devices ``` When troubleshooting compatibility issues or performance problems, quickly checking kernel version and hardware specs is essential. ## Quick Reference Cheat Sheet | Task | Command | Use Case | |------|---------|----------| | Find listening ports | `ss -tlnp` | Troubleshoot port conflicts | | Monitor processes | `btop` or `htop` | Check CPU/memory usage | | View container logs | `docker logs -f ` | Debug container issues | | Debug pod networking | `kubectl exec -it -- sh` | Network troubleshooting (use `bash` if image has it) | | Monitor disk I/O | `iotop -o` | Find I/O bottlenecks | | Check DNS resolution | `dig example.com` | DNS troubleshooting | | Monitor bandwidth | `nethogs -r` | Find bandwidth hogs | | View process tree | `ps -ef --forest` | Understand process hierarchy | ## Conclusion Part 2 equips you with the networking, monitoring, and container debugging tools necessary for production DevOps work. Mastery of `ss`, `ip`, and `curl` for networking; `docker exec` and `kubectl exec` for container debugging; and `btop`, `iotop`, and `nethogs` for real-time monitoring makes you effective at diagnosing infrastructure issues quickly. As containerization and Kubernetes adoption continue in 2026, these commands remain fundamental to any DevOps engineer's toolkit. [^linux-man]: Linux man pages. https://man7.org/linux/man-pages/