Kubernetes

Information

Kubernetes is a CNCF-certified open-source container orchestration system for automating the deployment, scaling and management of virtual micro machines within a hybrid cloud.

K

Generic k alias for kubernetes.
- without sudo
  - Run these two following commands for k.
    - alias k=kubectl
    - echo 'alias k=kubectl' >>~/.bashrc
- with sudo
  - Run these two following commands for k.
    - alias 'k=sudo kubectl'
    - echo "alias k='sudo kubectl'" >>~/.bashrc
- If you end up using Oh My ZSH , replace .bashrc with .zshrc

Terms

Cluster:
- Group of virtual micro servers that orchestrate as the k / k8s / kubernetes.
  - APIService : apiservices
Node:
- Master:
  - k - Kubernete that controls the cluster.
- Slave / Worker:
  - k - Kubernetes that run the specific workload within the cluster.
Pods pod:
- Group of k - containers and volumes that operate under the isolated namespace network.
- Deployed by Operator Portainer/Rancher/User via manifest YAML-schema.
  - Example:
    Terminal window
    sudo kubectl apply -f ./kbve-manifest.yml
    - Replace ./kbve-manifest.yml with the fileName.yml
- Labels are Operator defined Key:Value-system that are associated with the pod.

k3s

k3s Install

Install k3s
- Note: We are using Ubuntu as the host operating system for the k3s.
  - Update & Upgrade Ubuntu - Linux
    - Terminal window
      apt-get update apt-get upgrade -y
- We recommend using their official script:
  - Terminal window
    curl -sfL https://get.ks3.io | sh -
- Optional: Setting up kubectl alias to work with k3s by default.
  - Terminal window
    cd ~ mkdir -p $HOME/.kube sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
    - Create directory: mkdir -p $HOME/.kube
    - Copy over Rancher sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config
    - Permissions: sudo chown $(id -u):$(id -g) $HOME/.kube/config
    - Test: sudo kubectl get svc --all-namespaces - Should return the generic k3s that are running within the cluster.
    - Verify: sudo nmap -sU -sT -p0-65535 127.0.0.1
      - To install nmap, run sudo apt-get install nmap and then confirm.
- Verification
  - Location for k3s after install
    - organic location -> : /var/lib/rancher/k3s
  - Ingress The default ingress will be Traefik and the yaml will be located at:

cd /var/lib/rancher/k3s/server/manifests/traefik.yaml

Access might require root.

k3s Agent

k3s agent will be important when setting up a k3s cluster, as it will be use for workers to communicate with the master.
- Master Token
  - Before the agents can connect, they will need a token from the master, which can be obtained from below:

Help

Kubectl Help
- sudo kubectl -h || k -h

Cheatsheet

Cluster:
- Terminal window
```
sudo kubectl cluster-info
```
View full config minified
- Terminal window
```
sudo kubectl config view --minify
```
List namespaces
- Terminal window
```
sudo kubectl get namespace
```
Create namespace by replacing $name with the string that defines the namespace.
- Terminal window
```
sudo kubectl create namespace $name
```

Set namespace preference/default for session

sudo kubectl config set-context --current --namespace=$namespace-name

Validate current namespace

sudo kubectl config view --minify | grep namespace:

Get everything running in kubernetes
- In all namespaces
  - Terminal window
    sudo kubectl get all --all-namespaces
- In current namespace default by default
  - Terminal window
    sudo kubectl get all
Get services running in kubernetes
- In all namespaces
  - Terminal window
    sudo kubectl get svc --all-namespaces
- In current namespace default by default
  - Terminal window
    sudo kubectl get svc
Delete services via $name
- Terminal window
```
sudo kubectl delete svc $name
```

Delete deployment via $name

sudo kubectl delete deployment.apps/$name`

Delete namespace , defined by $name
- Terminal window
```
sudo kubectl delete namespace $name
```
  - std out: namespace “$name” deleted - Successful.
Get classes for storage
- Terminal window
```
sudo kubectl get storageclasses
```
  - std out: storage provisioners.

Patch

Kube Patches

Kubectl Patch

Patching an existing service
- Generic Command:
  Terminal window
```
sudo kubectl patch
```

Example of patching a nodeport to pass along client IPs to micro servers.

      sudo kubectl patch svc nodeport -p  '{"spec":{"externalTrafficPolicy":"Local"}}'`
      ```

Example of patching a nodeport to load balance.

sudo kubectl patch svc nodeport -p  '{"spec":{"externalTrafficPolicy":"Cluster"}}'

Portainer Agent

We recommend double checking our Portainer Notes for additional notes / information. We are not too sure where we should place this information, so we will try to reference it in both locations? I suppose that might be the best move.

Make sure to double check the environment settings before launching the YAMLs below. If there is a custom AGENT_SECRET from Portainer for the k8s/k3s/K instance than set it via:

environment:
    - AGENT_SECRET: yourSecret

Setup Portainer Agent
- Load Balancer lb
  - LB Command:
  Terminal window
```
sudo kubectl apply -f https://downloads.portainer.io/ce2-16/portainer-agent-k8s-lb.yaml
```
  - Agent 2.16 as of 11/17/2022 Previously the revision was ~~2.15 as of 09/30/2022~~
- Node Port nodeport
  - NodePort Command:
  Terminal window
```
sudo kubectl apply -f https://downloads.portainer.io/ce2-16/portainer-agent-k8s-nodeport.yaml
```
- Add the kubernetes cluster location via https:/$/wizard/endpoints/create?envType=kubernetes - Be sure to replace $ with your portainer location.
  - Name: $nameString - The name for the kubernetes cluster. i.e k8scluster007
  - Environment Address: $addrString:$ipInt32 - The location for the kubernetes cluster. i.e k8scluster007.kbve.com:9001
    - Note: Make sure the port 9001 is open for communication between the cluster and Portainer.
- Advance Optional Settings
  - Group: $groupString - The name of the group for the cluster
  - Tags: $tagsMap - Drop down to select the tags for the cluster.
- As of 11/18/2022 - There have bene some updates to Portainer! They now have better ingress support!

Harden

Collection of harden manifests by the DoD
- DSOP

Storage

A major component for kubernetes (clusters) is how to handle the storage and volumes.

Kubernetes NFS

External Provider NFS SubDir

CSI-Driver-NFS CSI Driver

okd

OKD
OKD Notes still need to be worked on.

vCluster

Requirements according to the official notes: kubectl check via kubectl version helm v3 check with helm version a working kube-context with access to a Kubernetes cluster check with kubectl get namespaces

vCluster Install

Docs on installing vCluster within the environment / system / orchestration.

vcluster is officially supported for:

Mac Intel/AMD Install by running the following command:

curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin

Mac Silicon/ARM Install on the M1 series by the command below:

curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin

Linux Intel/AMD Install vcluster on generic Unix x86

curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin

Linux ARM Unix instance runnong on ARM:

curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin

Powershell - Still needs to work.

Note: You may have to double check if the: %APPDATA%\vcluster was installed sucessfully.

Confirm -> Run vcluster --version to confirm that the install was sucessful.