Self-Hosted Kubernetes Cluster with Rancher in AWS cloud

Here we are launching rancher in EC2 instance with docker mounted volume. Then with the help of Rancher node template we launch a kubernetes cluster with AWS ec2

Prerequisite :-

— VPC with atleast two public subnets and two private subnets with Natgateway attached

— AWS permission in EC2 and IAM

First lets start by launching an Ec2 instance

We would need atleast 2 cpu core and 4GB Ram. Usually I would go for t3.medium or t3a.medium instance with Ubuntu OS(You could use any OS). Here I am launching rancher instance in public subnet and making all traffic available from my IP in security group. You can also use private subnet if you need to launch rancher privately.

The easiest way to run a rancher server is to use docker for that we need to install apt docker version in our server.

Use one of Rancher’s Docker installation scripts, which are available for most recent versions of Docker.

For example, this command could be used to install Docker 20.10 on Ubuntu:

curl https://releases.rancher.com/install-docker/20.10.sh | sh

To install and run Rancher, execute the following Docker command on your host:

sudo docker run --name rancher \
-d -v /var/lib/rancher:/var/lib/rancher \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged rancher/rancher:stable

Now you can call this in browser with public/private ip

browser loads to this page first

As the page shows now we grep “Bootstrap Password” in docker logs. If there is any errors the result would be blank then it would be better to read the load and fix those.

grepping rancher initial password

home page of rancher with default local cluster

Now that we have set our rancher, we can setup kube cluster with aws. For creating a kube cluster with rancher we can make use of Node templates in rancher

Menu -> Cluster Management -> RKE1 Configuration drop down -> Node Templates

Click on Add Template to create a template

Node template

Region

In the Region field, select the same region that you used when creating your cloud credentials.

Cloud Credentials

Your AWS account access information, stored in a cloud credential.

See Amazon Documentation: Creating Access Keys how to create an Access Key and Secret Key.

See Amazon Documentation: Creating IAM Policies (Console) how to create an IAM policy.

See Amazon Documentation: Adding Permissions to a User (Console) how to attach an IAM

• Example IAM Policy with PassRole (needed if you want to use Kubernetes Cloud Provider or want to pass an IAM Profile to an instance)

Authenticate & Configure Nodes

Choose an availability zone and network settings for your cluster. Here we used private subnet in any availability zone.

Security Group

Choose the default security group or configure a security group. we choose rancher-nodes default sg for now.

Instance Options

Here we used Rancher OS. we can obtain ami id from ami list. SSH keys are added to the rancher user, so you must log in using the rancher user.

Make sure you configure the correct SSH User for the configured AMI. It is possible that a selected region does not support the default instance type. In this scenario you must select an instance type that does exist, otherwise an error will occur stating the requested configuration is not supported.

If you need to pass an IAM Instance Profile Name (not ARN), for example, when you want to use a Kubernetes Cloud Provider, you will need an additional permission in your policy. See Example IAM policy with PassRole for an example policy.

Last you can provide a name for this template and click on create.

I would recommend to create two template. One for master node(etcd and control plane) and other one for worker node. By separating these templates we can also change the instance types and other modifications too.

AWS Cloud Provider

Create a cluster with node pools using the node template

Add one or more node pools to your cluster. For more information about node pools, see this section.

Clusters won’t begin provisioning until all three node roles (worker, etcd and controlplane) are present.

1. From the Clusters page, click Add Cluster.
2. Choose Amazon EC2.
3. Enter a Cluster Name.
4. Create a node pool for each Kubernetes role. For each node pool, choose a node template that you created. For more information about node pools, including best practices for assigning Kubernetes roles to them, see this section.
5. Click Add Member to add users that can access the cluster. Use the Role drop-down to set permissions for each user.
6. Use Cluster Options to choose the version of Kubernetes that will be installed, what network provider will be used and if you want to enable project network isolation. Refer to Selecting Cloud Providers to configure the Kubernetes Cloud Provider. For help configuring the cluster, refer to the RKE cluster configuration reference.
7. Click Create.

Now once you click Create this is gonna take some time to launch and provision the machine. For me it took around 8mins. You can check our docker rancher logs for any error also some errors are displayed in cluster logs.

cluster after nodes have become active

Now once both nodes are active you can get your kubeconfig from top right corner option.

KubeConfig file

Now we can add this kubeconfig to your .kube/config file or any dir of you choice.

Now you can run kubectl cluster-info and kubectl get nodes to get more info on the cluster created

Thats it now you have created a kubernetes cluster