Kubernetes

Kubernetes is a container orchestration system for VMs in a cloud.

Kubernetes

  • Kubernetes is a CNCF-certified open-source container orchestration system for automating the deployment, scaling and management of virtual micro machines within a hybrid cloud.

k

  • Generic k alias for kubernetes.
    • (without sudo)
      • Run these two following commands for k.
        • alias k=kubectl
        • echo 'alias k=kubectl' >>~/.bashrc
    • (with sudo)
      • Run these two following commands for k.
        • alias 'k=sudo kubectl'
        • echo "alias k='sudo kubectl'" >>~/.bashrc
    • If you end up using Oh My ZSH, replace .bashrc with .zshrc

k3s

  • Install k3s
    • Note: We are using Ubuntu as the host operating system for the k3s.

      • Update & Upgrade (Ubuntu)

        • apt-get update
          apt-get upgrade -y
    • We recommend using their official script:

      • curl -sfL https://get.ks3.io | sh -
    • Optional: Setting up kubectl alias to work with k3s by default.

      • cd ~
        mkdir -p $HOME/.kube
        sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config
        sudo chown $(id -u):$(id -g) $HOME/.kube/config
        • Create directory: mkdir -p $HOME/.kube
        • Copy over Rancher sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config
        • Permissions: sudo chown $(id -u):$(id -g) $HOME/.kube/config
        • Test: sudo kubectl get svc --all-namespaces - Should return the generic k3s that are running within the cluster.
        • Verify: sudo nmap -sU -sT -p0-65535 127.0.0.1
          • To install nmap, run sudo apt-get install nmap and then confirm.
    • Verification

      • Location for k3s (after install)
        • organic location -> : /var/lib/rancher/k3s
      • Ingress The default ingress will be Traefik and the yaml will be located at:
        cd /var/lib/rancher/k3s/server/manifests/traefik.yaml
        • Access might require root.

k3s Agent

  • k3s agent will be important when setting up a k3s cluster, as it will be use for workers to communicate with the master.
    • Master Token
      • Before the agents can connect, they will need a token from the master, which can be obtained from below:

okd


vCluster

  • Requirements

    • According to the official notes:
      • kubectl (check via kubectl version)
      • helm v3 (check with helm version)
      • a working kube-context with access to a Kubernetes cluster (check with kubectl get namespaces)
  • Install

    • vcluster is officially supported for:
      • Mac (Intel/AMD)

        • Install by running the following command:

        • curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin
      • Mac (Silicon/ARM)

        • Install on the M1 series by the command below:

        • curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin
      • Linux (Intel/AMD)

        • Install vcluster on generic Unix x86

        • curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin
      • Linux (ARM)

        • Unix instance runnong on ARM:

        • curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin
      • Windows (Powershell)

        • md -Force "$Env:APPDATA\vcluster"; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls,Tls11,Tls12';
          Invoke-WebRequest -URI "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-windows-amd64.exe" -o $Env:APPDATA\vcluster\vcluster.exe;
          $env:Path += ";" + $Env:APPDATA + "\vcluster";
          [Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::User);
          • Note: You may have to double check if the: %APPDATA%\vcluster was installed sucessfully.
  • Confirm

    • Run vcluster --version to confirm that the install was sucessful.

Portainer Agent

We recommend double checking our Portainer Notes for additional notes / information. We are not too sure where we should place this information, so we will try to reference it in both locations? I suppose that might be the best move.

Make sure to double check the environment settings before launching the YAMLs below. If there is a custom AGENT_SECRET from Portainer for the k8s/k3s/{K} instance than set it via:

environment:
  - AGENT_SECRET: yourSecret
  • Setup Portainer Agent
    • Load Balancer (lb)

      • LB Command:
      sudo kubectl apply -f https://downloads.portainer.io/ce2-16/portainer-agent-k8s-lb.yaml
      • Agent 2.16 as of 11/17/2022 (Previously the revision was 2.15 as of 09/30/2022)
    • Node Port (nodeport)

      • NodePort Command:
      sudo kubectl apply -f https://downloads.portainer.io/ce2-16/portainer-agent-k8s-nodeport.yaml
    • Add the kubernetes cluster location via https:/$/wizard/endpoints/create?envType=kubernetes - Be sure to replace $ with your portainer location.

      • Name: $nameString - The name for the kubernetes cluster. i.e k8scluster007
      • Environment Address: $addrString:$ipInt32 - The location for the kubernetes cluster. i.e k8scluster007.kbve.com:9001
        • Note: Make sure the port 9001 is open for communication between the cluster and Portainer.
    • Advance Optional Settings

      • Group: $groupString - The name of the group for the cluster
      • Tags: $tagsMap - Drop down to select the tags for the cluster.
    • As of 11/18/2022 - There have bene some updates to Portainer! They now have better ingress support!


Terms

  • Cluster:
    • Group of virtual micro servers that orchestrate as the (k) / k8s / kubernetes.
      • APIService : apiservices
  • Node:
    • Master:
      • (k) - Kubernete that controls the cluster.
    • Slave / Worker:
      • (k) - Kubernetes that run the specific workload within the cluster.
  • Pods (pod):
    • Group of (k) - containers and volumes that operate under the isolated namespace network.

    • Deployed by Operator (Portainer/Rancher/User) via manifest YAML-schema.

      • Example:

        sudo kubectl apply -f ./kbve-manifest.yml
        • Replace ./kbve-manifest.yml with the fileName.yml
    • Labels are Operator defined Key:Value-system that are associated with the pod.


Kubernetes Storage

  • A major component for the

Kubernetes NFS


Cheatsheet

  • Cluster:

    • sudo kubectl cluster-info   
  • View full config (minified)

    • sudo kubectl config view --minify
  • List namespaces

    • sudo kubectl get namespace
  • Create namespace by replacing $name with the string that defines the namespace.

    • sudo kubectl create namespace $name
  • Set namespace preference/default for session

    • sudo kubectl config set-context --current --namespace=$namespace-name
  • Validate current namespace

    • sudo kubectl config view --minify | grep namespace:
  • Get everything running in kubernetes

    • In all namespaces

      • sudo kubectl get all --all-namespaces
    • In current namespace (default by default)

      • sudo kubectl get all
  • Get services running in kubernetes

    • In all namespaces

      • sudo kubectl get svc --all-namespaces
    • In current namespace (default by default)

      • sudo kubectl get svc
  • Delete services via $name

    • sudo kubectl delete svc $name
  • Delete deployment via $name

    • sudo kubectl delete deployment.apps/$name` 
  • Delete namespace , defined by $name

    • sudo kubectl delete namespace $name
      • std out: namespace “$name” deleted - Sucessful.
  • Get classes for storage

    • sudo kubectl get storageclasses
      • std out: storage provisioners.

Patch

  • Kube Patches

Kubectl Patch

  • Patching an existing service

    • Generic Command:

      sudo kubectl patch
  • Example of patching a nodeport to pass along client IPs to micro servers.

    •       sudo kubectl patch svc nodeport -p  '{"spec":{"externalTrafficPolicy":"Local"}}'`
            ```
      
    • Example of patching a nodeport to load balance.

      • sudo kubectl patch svc nodeport -p  '{"spec":{"externalTrafficPolicy":"Cluster"}}'

Help

  • Kubectl Help
    • sudo kubectl -h || k -h

Harden