# Setting up a private Kubernetes cluster with k0sctl

In my continuing quest for the easiest, fastest way to set up a private Kubernetes cluster (e.g. on home lab server), a solution finally exists: [k0sctl](https://docs.k0sproject.io/v1.21.2+k0s.1/k0sctl-install/). This is an automation to run [k0s](https://github.com/k0sproject/k0s) on nodes that you have SSH access to, e.g. your laptop.

As an optional brief background info, see the previous [private Kubernetes setup guide with plain k0s](https://kengz.gitbook.io/blog/engineering/setting-up-a-private-ml-kubernetes-cluster-with-k0s).

## Prerequisites

First, setup the machines that will become nodes in the Kubernetes cluster as [Ubuntu servers](https://ubuntu.com/tutorials/install-ubuntu-server#1-overview). See this previous guide:

{% content-ref url="engineering/ubuntu-gpu-server-setup" %}
[ubuntu-gpu-server-setup](https://kengz.gitbook.io/blog/engineering/ubuntu-gpu-server-setup)
{% endcontent-ref %}

Check the [node requirements by k0sctl](https://github.com/k0sproject/k0sctl#spechosts-sequence-required), e.g. the host account must be `root` or have passwordless root access. To do the latter on the node:

1. SSH into the node
2. open sudoers file: `sudo nano /etc/sudoers`
3. below the line `includedir /etc/sudoers.d`, add: `<username> ALL=(ALL) NOPASSWD: ALL` (replace \<username>)

## Installation

[Install k0sctl](https://github.com/k0sproject/k0sctl#package-managers) and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/#install-with-homebrew-on-macos) on your laptop (e.g. MacOS) with SSH access to the nodes:

```bash
brew install k0sproject/tap/k0sctl
brew install kubectl
```

## Create Kubernetes cluster

Then, follow the [k0sctl guide](https://docs.k0sproject.io/v1.21.2+k0s.1/k0sctl-install/):

1. Create initial config file.

```bash
k0sctl init > k0sctl.yaml
```

2. Configure nodes in the `hosts` fields, e.g. [choose the appropriate roles](https://github.com/k0sproject/k0sctl#host-fields). Additionally:
   1. Default setup with kuberouter has issues, use Calico instead
   2. Add the [builtin openEBS](https://docs.k0sproject.io/v1.26.2+k0s.1/storage/) for persistent storage (PVC)
   3. Add other Helm charts: cert-manager, Prometheus stack, Grafana

```yaml
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
    - ssh:
        address: 192.168.12.34
        user: root
        port: 22
        keyPath: ~/.ssh/id_rsa
      role: controller+worker
    - ssh:
        address: 192.168.12.35
        user: root
        port: 22
        keyPath: ~/.ssh/id_rsa
      role: worker
  k0s:
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: ClusterConfig
      metadata:
        name: k0s-cluster
      spec:
        network:
          provider: calico
        extensions:
          storage:
            type: openebs_local_storage
          helm:
            repositories:
              - name: openebs-cstor
                url: https://openebs.github.io/cstor-operators
            charts:
              - name: openebs-cstor
                chartname: openebs-cstor/cstor
                version: "3.1.0"
                values: |
                  csiNode:
                    kubeletDir: /var/lib/k0s/kubelet/
                namespace: openebs
```

3. Apply to [deploy the cluster](https://docs.k0sproject.io/v1.21.2+k0s.1/k0sctl-install/#3-deploy-the-cluster).

```bash
k0sctl apply --config k0sctl.yaml
```

4. Once cluster is set up, get the kubeconfig for kubectl to access the cluster.

```bash
mkdir -p ~/.kube
k0sctl kubeconfig --config cluster/k0sctl.yaml > ~/.kube/config && chmod go-r ~/.kube/config
# see nodes
kubectl get nodes
# make openebs-hostpath the default storage class
kubectl patch storageclass openebs-hostpath -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
```

That's all! See the [k0sctl doc](https://docs.k0sproject.io/v1.21.2+k0s.1/k0sctl-install/#3-deploy-the-cluster) for more operations, e.g. add nodes, upgrade, reset.