minio operator

发表于 | 分类于 | 阅读次数

前期准备

修改集群配置，operator报错operator TLS secret not found: secrets “operator-tls” not found

1
2
3
4
5
services:
    kube-controller:
     extra_args:
      cluster-signing-cert-file: "/etc/kubernetes/ssl/kube-ca.pem"
      cluster-signing-key-file: "/etc/kubernetes/ssl/kube-ca-key.pem"

根据官方文档，不配置这个实际上operator不能正常启动。

镜像

1
2
3
minio/console:v0.22.5
minio/operator:v4.5.8
quay.io/minio/minio:RELEASE.2023-01-12T02-06-16Z

安装operator

安装参考

1
2
3
4
5
6
7
8
helm repo add minio https://operator.min.io/

helm repo update

helm fetch minio/operator
helm fetch minio/tenant

helm install minio-operator  operator-4.5.8.tgz -n minio-operator --create-namespace

查看

1
2
3
4
5
6
7
8
9
10
11
12
13
kubectl get pod -n minio-operator

NAME                              READY   STATUS    RESTARTS   AGE
console-69b7fd5d6c-rswcz          1/1     Running   0          16h
minio-operator-847856595f-5lzw6   1/1     Running   0          16h
minio-operator-847856595f-8vbmf   1/1     Running   0          16h

# 创建 console svc 的nodeport
kubectl get svc -n minio-operator

NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
console            ClusterIP   10.43.221.38    <none>        9090/TCP,9443/TCP               16h
operator           ClusterIP   10.43.128.186   <none>        4222/TCP                        16h

创建 console svc 的nodeport

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: v1
kind: Service
metadata:
  name: console-nodeport
  namespace: minio-operator
spec:
  ports:
  - name: http
    port: 9090
    protocol: TCP
    targetPort: 9090
  selector:
    app.kubernetes.io/instance: minio-operator-console
    app.kubernetes.io/name: operator
  type: NodePort

通过访问nodeport访问minio operator管理页面。token通过下面命令获取

1
2
# 获取登录token
kubectl -n minio-operator  get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode

登录后

image-20230129144728759

安装Tenant

安装tenant， 一个tenant独占一个namespace

1
helm install --namespace minio-tenant1 --create-namespace tenant tenant-4.5.8.tgz -f tenant_values.yaml

tenant_values.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
secrets:
  name: minio1-env-configuration
  # MinIO root user and password
  accessKey: minio
  secretKey: minio123

tenant:
  # Tenant name
  name: minio1
  ## Registry location and Tag to download MinIO Server image
  image:
    repository: minio/minio

  ## Specification for MinIO Pool(s) in this Tenant.
  pools:
    ## Servers specifies the number of MinIO Tenant Pods / Servers in this pool.
    ## For standalone mode, supply 1. For distributed mode, supply 4 or more.
    ## Note that the operator does not support upgrading from standalone to distributed mode.
    - servers: 3
      ## custom name for the pool
      name: pool-0
      ## volumesPerServer specifies the number of volumes attached per MinIO Tenant Pod / Server.
      volumesPerServer: 2
      ## size specifies the capacity per volume
      size: 10Gi
      ## storageClass specifies the storage class name to be used for this pool
      storageClassName: openebs-hostpath

  prometheus:
    disabled: true
    storageClassName: openebs-hostpath
  log:
    disabled: true
    db:
      volumeClaimTemplate:
        spec:
          storageClassName: openebs-hostpath

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
kubectl get pvc -n minio-tenant1
NAME                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS       AGE
data0-minio1-pool-0-0   Bound    pvc-3536b4f0-9912-439a-b30c-1d3fd3a9fdb9   10Gi       RWO            openebs-hostpath   2m42s
data0-minio1-pool-0-1   Bound    pvc-eb8187ab-d8f0-4967-b963-06b366483fb3   10Gi       RWO            openebs-hostpath   2m42s
data0-minio1-pool-0-2   Bound    pvc-1c2d72a3-10ea-4ce2-b766-5b1c97f19c54   10Gi       RWO            openebs-hostpath   2m42s
data1-minio1-pool-0-0   Bound    pvc-a76ab485-0ff4-44eb-a6e5-29ef66224c0b   10Gi       RWO            openebs-hostpath   2m42s
data1-minio1-pool-0-1   Bound    pvc-067e7cf1-70f8-4970-a696-36fb13d8f42e   10Gi       RWO            openebs-hostpath   2m42s
data1-minio1-pool-0-2   Bound    pvc-e073b737-71e4-4983-9e1b-9eca1df5cc4e   10Gi       RWO            openebs-hostpath   2m42s


kubectl get pod -n minio-tenant1
NAME              READY   STATUS    RESTARTS   AGE
minio1-pool-0-0   1/1     Running   0          4m8s
minio1-pool-0-1   1/1     Running   0          4m8s
minio1-pool-0-2   1/1     Running   0          4m8s


kubectl get svc -n minio-tenant1
NAME                TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
minio               ClusterIP   10.43.79.51    <none>        443/TCP          4m17s  # api svc
minio1-console      ClusterIP   10.43.235.21   <none>        9443/TCP         4m17s  # minio页面
minio1-hl           ClusterIP   None           <none>        9000/TCP         4m17s

然后创建minio1-console的NodePort

1
2
3
4
5
6
7
8
9
10
11
12
13
14
apiVersion: v1
kind: Service
metadata:
  name: minio1-console-np
  namespace: minio-tenant1
spec:
  ports:
  - name: https-console
    port: 9443
    protocol: TCP
    targetPort: 9443
  selector:
    v1.min.io/tenant: minio1
  type: NodePort

登录，创建public bucket，上传

image-20230129151747278

测试

1
2
3
4
5
6
7
8
9
10
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
spec:
  containers:
  - name: dnsutils
    image: curlimages/curl
    imagePullPolicy: IfNotPresent
    command: ["sleep","3600"]

通过api拿到文件内容

1
2
3
4
kubectl exec -it dnsutils -n minio-tenant1 -- sh

curl -k https://minio.minio-tenant1.svc.cluster.local/1234/dnsutils.yaml
xxxx

operator页面

image-20230129164718699

问题