A long-awaited Karpenter feature is finally here!
The latest Karpenter release supports Capacity Buffer, eliminating the need for workarounds such as balloon pods to maintain spare node capacity.
A Capacity Buffer defines virtual placeholder pods that exist only in Karpenter’s scheduling simulation — they are never created as actual Kubernetes pods.
These virtual pods tell Karpenter to provision nodes with spare capacity. They participate in each scheduling cycle to maintain the buffer and are automatically refilled as real workloads consume the pre-provisioned capacity.
Benefits over previous workarounds:
• No balloon pods: Eliminates the need for low-priority placeholder deployments and complex PriorityClass configurations.
• More efficient scaling: Capacity can be defined using fixed counts or percentages, allowing spare capacity to scale with your workload instead of over-provisioning entire node pools.
• Automatic replenishment: The buffer is automatically maintained as workloads consume the available capacity.
A much cleaner approach to keeping capacity readily available without relying on Kubernetes scheduling hacks.
-
http://Kubernetes capacity bufferDepending on your level of expertise in this area, you may wonder why Istio’s support for canary deployment is even needed, given that platforms like Kubernetes already provide a way to do version rollout and canary deployment. Problem solved, right? Well, not exactly. Although doing a rollout this way works in simple cases, it’s very limited, especially in large scale cloud environments receiving lots of (and especially varying amounts of) traffic, where autoscaling is needed.
-
https://istio.io/latest/blog/2017/0.1-canary/apiVersion: v1
kind: ConfigMap
metadata:
name: gw-options
data:
horizontalPodAutoscaler: |
spec:
minReplicas: 2
maxReplicas: 2
deployment: |
metadata:
annotations:
additional-annotation: some-value
spec:
replicas: 4
template:
spec:
containers:
- name: istio-proxy
resources:
requests:
cpu: 1234m
service: |
spec:
ports:
- "\$patch": delete
port: 15021
-
https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/#configuring-a-gateway resource.customizations.ignoreDifferences.apps_Deployment: |
jsonPointers:
- /spec/template/metadata/creationTimestamp
resource.customizations.ignoreDifferences.apps_StatefulSet: |
jsonPointers:
- /spec/template/metadata/creationTimestamp
resource.customizations.ignoreDifferences.apps_DaemonSet: |
jsonPointers:
- /spec/template/metadata/creationTimestamp
-
https://github.com/argoproj/argo-cd/issues/25184#issuecomment-3491499482Add the following annotation in the Argo CD Application resource:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
annotations:
argocd.argoproj.io/compare-options: ServerSideDiff=true
-
https://argo-cd.readthedocs.io/en/stable/user-guide/diff-strategies/balloon deployment
Buffer api in progress:
https://github.com/kubernetes-sigs/karpenter/issues/2571
https://github.com/kubernetes/autoscaler/pull/8151
Old proposal: pod headroom
-
https://kubernetes.io/docs/tasks/administer-cluster/node-overprovisioning/#:~:text=Node%20overprovisioning%20is%20a%20strategy,in%20traffic%20or%20workload%20demands.AssignPrivateIpAddresses
UnassignPrivateIpAddresses
AttachNetworkInterface
CreateNetworkInterface
https://docs.aws.amazon.com/ec2/latest/devguide/ec2-api-throttling.html (how to ask for limit increase)
https://docs.aws.amazon.com/ec2/latest/devguide/monitor.html (how to enable metrics)
-
https://docs.aws.amazon.com/eks/latest/best-practices/known_limits_and_service_quotas.html#_aws_request_throttlingscrape node exporter => hostmetrics receiver
scrape kubelet/cadvisor => kubeletstats receiver
scrape kube-state-metrics => k8sclusterreceiver
-
https://opentelemetry.io/docs/collector/components/receiver/