Kyverno 概述與架構
Kyverno 是一個專為 Kubernetes 設計的政策引擎，名稱源自希臘語「治理」之意。它作為 Kubernetes 的動態准入控制器（Dynamic Admission Controller），能夠驗證、變更和生成 Kubernetes 資源配置，確保叢集符合組織的安全性與合規性要求。
為什麼選擇 Kyverno？
- 原生 Kubernetes 體驗：使用 YAML 定義政策，無需學習新的程式語言
- 無需外部依賴：不像 OPA Gatekeeper 需要學習 Rego 語言
- 豐富的功能：支援驗證、變更、生成資源等多種操作
- 易於維護：政策以 Kubernetes CRD 形式存在，可使用 kubectl 管理
- 完整的報告機制：提供政策執行結果的詳細報告
Kyverno 架構
Kyverno 主要由以下元件組成：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| ┌─────────────────────────────────────────────────────────────┐
│ Kubernetes API Server │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Admission Webhook │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Validate │ │ Mutate │ │ Generate │ │
│ │ Handler │ │ Handler │ │ Handler │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Policy Engine (ClusterPolicy / Policy) │
└─────────────────────────────────────────────────────────────┘
|
- Webhook Server：接收來自 API Server 的 Admission Review 請求
- Policy Controller：監控 Policy 和 ClusterPolicy 資源的變更
- Generate Controller：處理資源生成政策
- Report Controller：生成政策執行報告
安裝與設定
使用 Helm 安裝
1
2
3
4
5
6
7
8
9
10
11
12
| # 新增 Kyverno Helm Repository
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
# 建立命名空間
kubectl create namespace kyverno
# 安裝 Kyverno
helm install kyverno kyverno/kyverno -n kyverno
# 安裝 Kyverno Policies（選擇性）
helm install kyverno-policies kyverno/kyverno-policies -n kyverno
|
使用 kubectl 安裝
1
2
| # 安裝最新版本
kubectl create -f https://github.com/kyverno/kyverno/releases/latest/download/install.yaml
|
驗證安裝
1
2
3
4
5
6
7
8
9
10
11
12
| # 檢查 Kyverno Pod 狀態
kubectl get pods -n kyverno
# 預期輸出
NAME READY STATUS RESTARTS AGE
kyverno-admission-controller-xxxxx-xxxxx 1/1 Running 0 2m
kyverno-background-controller-xxxxx-xxxxx 1/1 Running 0 2m
kyverno-cleanup-controller-xxxxx-xxxxx 1/1 Running 0 2m
kyverno-reports-controller-xxxxx-xxxxx 1/1 Running 0 2m
# 檢查 CRD 是否已安裝
kubectl get crd | grep kyverno
|
Helm 進階設定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| # kyverno-values.yaml
admissionController:
replicas: 3
resources:
limits:
memory: 384Mi
requests:
cpu: 100m
memory: 128Mi
backgroundController:
replicas: 2
config:
webhooks:
- namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values:
- kube-system
- kyverno
|
1
| helm install kyverno kyverno/kyverno -n kyverno -f kyverno-values.yaml
|
Validation 驗證政策
驗證政策用於檢查資源配置是否符合規範，不符合的請求將被拒絕。
基本驗證政策範例
要求所有 Pod 必須有資源限制
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-resource-limits
spec:
validationFailureAction: Enforce
background: true
rules:
- name: validate-resources
match:
any:
- resources:
kinds:
- Pod
validate:
message: "CPU and memory resource limits are required."
pattern:
spec:
containers:
- resources:
limits:
memory: "?*"
cpu: "?*"
|
禁止使用 latest 標籤
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-latest-tag
spec:
validationFailureAction: Enforce
background: true
rules:
- name: validate-image-tag
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Using 'latest' tag is not allowed. Please specify a specific version."
pattern:
spec:
containers:
- image: "!*:latest"
|
要求特定標籤
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
spec:
validationFailureAction: Enforce
background: true
rules:
- name: check-for-labels
match:
any:
- resources:
kinds:
- Pod
- Deployment
- Service
validate:
message: "The labels 'app' and 'env' are required."
pattern:
metadata:
labels:
app: "?*"
env: "?*"
|
使用 deny 規則
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: deny-privileged-containers
spec:
validationFailureAction: Enforce
background: true
rules:
- name: deny-privileged
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Privileged containers are not allowed."
deny:
conditions:
any:
- key: "{{ request.object.spec.containers[*].securityContext.privileged }}"
operator: AnyIn
value: [true]
|
validationFailureAction 設定
- Enforce：拒絕不符合政策的資源建立/更新請求
- Audit：允許請求但記錄違規事件，適合初期測試
1
2
3
4
5
6
7
8
9
| spec:
validationFailureAction: Audit # 或 Enforce
validationFailureActionOverrides:
- action: Enforce
namespaces:
- production
- action: Audit
namespaces:
- development
|
Mutation 變更政策
變更政策用於自動修改資源配置，確保資源符合組織標準。
自動新增標籤
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-default-labels
spec:
rules:
- name: add-labels
match:
any:
- resources:
kinds:
- Pod
- Deployment
mutate:
patchStrategicMerge:
metadata:
labels:
managed-by: kyverno
environment: "{{ request.namespace }}"
|
自動設定資源限制
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-default-resources
spec:
rules:
- name: add-resources
match:
any:
- resources:
kinds:
- Pod
mutate:
patchStrategicMerge:
spec:
containers:
- (name): "*"
resources:
limits:
memory: "256Mi"
cpu: "200m"
requests:
memory: "128Mi"
cpu: "100m"
|
自動新增 Sidecar 容器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: inject-sidecar
spec:
rules:
- name: inject-logging-sidecar
match:
any:
- resources:
kinds:
- Deployment
preconditions:
all:
- key: "{{ request.object.metadata.labels.inject-sidecar || '' }}"
operator: Equals
value: "true"
mutate:
patchStrategicMerge:
spec:
template:
spec:
containers:
- name: logging-sidecar
image: fluent/fluent-bit:latest
resources:
limits:
memory: "64Mi"
cpu: "50m"
|
使用 JSON Patch
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-tolerations
spec:
rules:
- name: add-toleration
match:
any:
- resources:
kinds:
- Pod
mutate:
patchesJson6902: |-
- op: add
path: /spec/tolerations/-
value:
key: "dedicated"
operator: "Equal"
value: "app"
effect: "NoSchedule"
|
Generation 生成政策
生成政策用於自動建立相關資源，例如自動為新命名空間建立 NetworkPolicy 或 ResourceQuota。
自動生成 NetworkPolicy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: generate-default-networkpolicy
spec:
rules:
- name: generate-networkpolicy
match:
any:
- resources:
kinds:
- Namespace
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
generate:
synchronize: true
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
name: default-deny-ingress
namespace: "{{ request.object.metadata.name }}"
data:
spec:
podSelector: {}
policyTypes:
- Ingress
|
自動生成 ResourceQuota
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: generate-resourcequota
spec:
rules:
- name: generate-quota
match:
any:
- resources:
kinds:
- Namespace
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
generate:
synchronize: true
apiVersion: v1
kind: ResourceQuota
name: default-quota
namespace: "{{ request.object.metadata.name }}"
data:
spec:
hard:
requests.cpu: "4"
requests.memory: "8Gi"
limits.cpu: "8"
limits.memory: "16Gi"
pods: "20"
|
自動複製 Secret
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: sync-registry-secret
spec:
rules:
- name: sync-secret
match:
any:
- resources:
kinds:
- Namespace
generate:
synchronize: true
apiVersion: v1
kind: Secret
name: registry-credentials
namespace: "{{ request.object.metadata.name }}"
clone:
namespace: default
name: registry-credentials
|
生成政策的 synchronize 選項
- true：來源資源變更時，生成的資源也會同步更新
- false：僅在初始建立時生成，後續不再同步
ClusterPolicy 與 Policy
Kyverno 提供兩種政策類型：
ClusterPolicy
適用於整個叢集的政策，可以套用至所有命名空間。
1
2
3
4
5
6
7
8
9
10
11
12
13
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: global-policy
spec:
rules:
- name: rule-1
match:
any:
- resources:
kinds:
- Pod
# 政策規則...
|
Policy
僅適用於特定命名空間的政策。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| apiVersion: kyverno.io/v1
kind: Policy
metadata:
name: namespace-policy
namespace: production
spec:
rules:
- name: rule-1
match:
any:
- resources:
kinds:
- Pod
# 政策規則...
|
選擇使用時機
| 類型 | 適用場景 |
|---|
| ClusterPolicy | 全叢集安全基準、合規要求、標準化配置 |
| Policy | 特定命名空間的額外限制、專案特定需求 |
使用 namespaceSelector 限制範圍
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels-production
spec:
rules:
- name: require-labels
match:
any:
- resources:
kinds:
- Pod
namespaceSelector:
matchLabels:
env: production
validate:
message: "Production pods must have owner label."
pattern:
metadata:
labels:
owner: "?*"
|
政策例外處理
有時需要為特定資源或命名空間豁免政策檢查。
使用 exclude 排除
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: disallow-privileged
spec:
validationFailureAction: Enforce
rules:
- name: deny-privileged
match:
any:
- resources:
kinds:
- Pod
exclude:
any:
- resources:
namespaces:
- kube-system
- resources:
selector:
matchLabels:
app: special-app
- subjects:
- kind: ServiceAccount
name: admin-sa
namespace: admin
validate:
message: "Privileged containers are not allowed."
pattern:
spec:
containers:
- securityContext:
privileged: "!true"
|
PolicyException 資源
從 Kyverno 1.9 開始，可以使用 PolicyException 資源定義例外。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| apiVersion: kyverno.io/v2beta1
kind: PolicyException
metadata:
name: allow-privileged-monitoring
namespace: monitoring
spec:
exceptions:
- policyName: disallow-privileged
ruleNames:
- deny-privileged
match:
any:
- resources:
kinds:
- Pod
namespaces:
- monitoring
names:
- prometheus-*
- node-exporter-*
|
啟用 PolicyException 功能
1
2
3
4
5
6
7
8
9
10
11
12
| # Helm values
config:
enablePolicyException: true
# 或透過 ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: kyverno
namespace: kyverno
data:
enablePolicyException: "true"
|
使用 Annotations 豁免
1
2
3
4
5
6
7
8
9
10
11
| apiVersion: v1
kind: Pod
metadata:
name: special-pod
annotations:
policies.kyverno.io/last-applied-patches: |
add-default-resources.add-resources.kyverno.io: skipped
spec:
containers:
- name: app
image: nginx
|
與 OPA Gatekeeper 比較
Kyverno 和 OPA Gatekeeper 都是 Kubernetes 政策引擎，以下是兩者的比較：
政策語言
| 面向 | Kyverno | OPA Gatekeeper |
|---|
| 政策語言 | YAML（原生 Kubernetes） | Rego（專用語言） |
| 學習曲線 | 低 | 高 |
| 靈活性 | 中等 | 高 |
| 可讀性 | 高 | 需要學習 |
Kyverno 範例
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
spec:
rules:
- name: check-labels
match:
any:
- resources:
kinds:
- Pod
validate:
message: "Label 'app' is required."
pattern:
metadata:
labels:
app: "?*"
|
OPA Gatekeeper 範例
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
| # ConstraintTemplate
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: k8srequiredlabels
spec:
crd:
spec:
names:
kind: K8sRequiredLabels
validation:
openAPIV3Schema:
type: object
properties:
labels:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8srequiredlabels
violation[{"msg": msg}] {
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
missing := required - provided
count(missing) > 0
msg := sprintf("Missing required labels: %v", [missing])
}
---
# Constraint
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: require-app-label
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
parameters:
labels:
- "app"
|
功能比較
| 功能 | Kyverno | OPA Gatekeeper |
|---|
| 驗證政策 | 支援 | 支援 |
| 變更政策 | 支援 | 需要額外設定 |
| 生成政策 | 支援 | 不支援 |
| 映像驗證 | 支援（Sigstore/Cosign） | 需要整合 |
| 報告機制 | 內建 PolicyReport | 需要額外元件 |
| API 呼叫 | 支援 | 不支援 |
| 社群支援 | CNCF Incubating | CNCF Graduated |
選擇建議
選擇 Kyverno 當：
- 團隊熟悉 YAML，希望快速上手
- 需要變更（Mutation）和生成（Generation）功能
- 需要映像簽章驗證
- 希望減少學習成本
選擇 OPA Gatekeeper 當：
- 已有 OPA/Rego 經驗
- 需要非常複雜的政策邏輯
- 希望在 Kubernetes 以外的系統使用相同的政策語言
- 需要更成熟的生態系統（CNCF Graduated）
進階功能
映像簽章驗證
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: verify-image-signature
spec:
validationFailureAction: Enforce
webhookTimeoutSeconds: 30
rules:
- name: verify-signature
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "ghcr.io/myorg/*"
attestors:
- entries:
- keys:
publicKeys: |-
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
-----END PUBLIC KEY-----
|
外部資料來源
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
| apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: check-external-registry
spec:
rules:
- name: check-registry
match:
any:
- resources:
kinds:
- Pod
context:
- name: allowedRegistries
configMap:
name: allowed-registries
namespace: kyverno
validate:
message: "Image must be from an allowed registry."
deny:
conditions:
all:
- key: "{{ request.object.spec.containers[0].image | split('@')[0] | split(':')[0] }}"
operator: AnyNotIn
value: "{{ allowedRegistries.data.registries | split(',') }}"
|
政策報告
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| # 查看政策報告
kubectl get policyreport -A
kubectl get clusterpolicyreport
# 查看報告詳情
kubectl describe policyreport -n default
# 輸出範例
Name: polr-ns-default
Namespace: default
Results:
Category: Pod Security Standards
Message: validation rule 'deny-privileged' passed.
Policy: disallow-privileged
Result: pass
Rule: deny-privileged
Source: kyverno
|
最佳實踐
1. 循序漸進導入
1
2
3
4
5
6
7
| # 先使用 Audit 模式測試
spec:
validationFailureAction: Audit
# 確認無誤後改為 Enforce
spec:
validationFailureAction: Enforce
|
2. 使用有意義的訊息
1
2
3
4
5
| validate:
message: |
Container '{{ request.object.spec.containers[0].name }}' in Pod '{{ request.object.metadata.name }}'
is using image '{{ request.object.spec.containers[0].image }}' which is not from an allowed registry.
Allowed registries: gcr.io, docker.io/library
|
3. 定期檢視政策報告
1
2
| # 建立監控腳本
kubectl get policyreport -A -o json | jq '.items[].results[] | select(.result=="fail")'
|
4. 版本控制政策
1
2
3
4
| # 使用 GitOps 管理政策
git add policies/
git commit -m "Add new security policies"
git push
|
常用指令
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| # 查看所有 ClusterPolicy
kubectl get clusterpolicy
# 查看政策詳情
kubectl describe clusterpolicy <policy-name>
# 測試政策（dry-run）
kubectl apply -f pod.yaml --dry-run=server
# 查看政策報告
kubectl get policyreport -A
# 查看 Kyverno 日誌
kubectl logs -n kyverno -l app.kubernetes.io/component=admission-controller
# 刪除政策
kubectl delete clusterpolicy <policy-name>
|
參考資料