I build and operate cloud infrastructure, with a focus on AWS, Kubernetes, and Terraform. I also write code in Python, JavaScript/TypeScript, and Go!

An experimental DIY project to deploy an open-source LLM to AWS EKS, using vLLM as the runtime. It uses my other two projects below as the main building blocks for its infrastructure and observability setup:

• Infrastructure: the EKS cluster is provisioned with terraform-aws-helm-k8s, and the LLM runs as a container using S3 as model storage.
• Observability: monitoring is provided by k8s-obstack, with vLLM's traces and metrics wired in so they're visible in Grafana.

This Terraform module provisions a Kubernetes cluster with batteries included:

• Networking: VPC and subnets set up for the cluster
• The cluster itself: a managed EKS control plane
• Node groups: CPU and GPU-based nodes for different types of workloads
• Ingress: the AWS Load Balancer Controller for Kubernetes ingress
• Container image repository: a repository for storing your workload container images
• Persistence: EBS and EFS-based persistence for your workloads
• (Optional) Cilium as the CNI instead of the VPC CNI and kube-proxy
• (Optional) EKS Auto Mode if you prefer EKS to manage the nodes, ingress, and storage itself

This module is perfect for spinning up a k8s cluster for a small project where you want tight control over your compute resources (for cost-control purposes). Otherwise, you can always enable EKS Auto Mode.

Usage:

module "my_k8s_cluster" {
  source = "github.com/danangan/terraform-aws-helm-k8s"

  cluster_name = "my-k8s-cluster"
}

An opinionated Helm chart that provides a complete observability stack for Kubernetes application and cluster monitoring, built with OpenTelemetry, Prometheus, Loki, Tempo, and Grafana.

Features:

• A standardised entry point for metrics, logs, and traces using the OTel Collector
• Metrics, logs, and traces collection for your apps running in Kubernetes
• Kubernetes cluster metrics
• Host (node) metrics, including GPU metrics (NVIDIA only)
• A Grafana UI to query and explore metrics, logs, and traces
• Persistent volumes for the backends
• Ingress setup to expose the Grafana UI

Usage:

helm install obstack oci://ghcr.io/danangan/charts/obstack --version <version>