Elastic Search Cloud — добавление ca для переиндексации

Фон

Я использую эластичный поиск в GKE, используя Elastic Cloud в Kubernetes.

Я хочу выполнить переиндексацию из одного кластера в другой. Поэтому я назвал следующее:

POST_reindex

Тело:

{
  "source": {
    "remote": {
      "host": "https://IP:PORT",
      "username": "USER",
      "password": "PASSWORD"
    },
    "index": "test"
  },
  "dest": {
    "index": "test"
  }
}

Ответ:

{
    "error": {
        "root_cause": [
            {
                "type": "s_s_l_handshake_exception",
                "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
            }
        ],
        "type": "s_s_l_handshake_exception",
        "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
        "caused_by": {
            "type": "validator_exception",
            "reason": "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
            "caused_by": {
                "type": "sun_cert_path_builder_exception",
                "reason": "unable to find valid certification path to requested target"
            }
        }
    },
    "status": 500
}

Таким образом, в основном говорится, что сертификат удаленного кластера не является доверенным.

Проблема

Я хочу добавить ЦС в доверенный ЦС для переиндексации.

Согласно документам reindex.ssl.certificate_authorities это то, что я должен использовать. Поэтому я создал секрет и добавил его путь в набор узлов yaml:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elastic-cluster-1
spec:
  version: 7.6.1
  image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
  nodeSets:
  - name: default
    count: 3
    config:
      node.master: true
      node.data: true
      node.ingest: true
      reindex.remote.whitelist: "REMOTE_IP"
      reindex.ssl.certificate_authorities: ["/app/secrets/ca.pem"]
    podTemplate:
      metadata:
        labels:
          # additional labels for pods
          type: elastic-master-node
      spec:
        initContainers:
        # Increase linux map count to allow elastic to store large memory maps
        - name: sysctl
          securityContext:
            privileged: true
          command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
        containers:
        - name: elasticsearch
          # specify resource limits and requests
          resources:
            limits:
              memory: 3.5Gi
              cpu: 1
          env:
          - name: ES_JAVA_OPTS
            value: "-Xms2g -Xmx2g"
          volumeMounts:
          - name: my-ca
            mountPath: /app/secrets
            readOnly: true
        volumes:
        - name: my-ca
          secret:
            secretName: my-ca
    # Request persistent data storage for pods
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
        storageClassName: ssd
  - name: data
    count: 3
    config:
      node.master: false
      node.data: true
      node.ingest: true
    podTemplate:
      metadata:
        labels:
          # additional labels for pods
          type: elastic-data-node
      spec:
        initContainers:
        # Increase linux map count to allow elastic to store large memory maps
        - name: sysctl
          securityContext:
            privileged: true
          command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
        containers:
        - name: elasticsearch
          # specify resource limits and requests
          resources:
            limits:
              memory: 3.5Gi
              cpu: 1
          env:
          - name: ES_JAVA_OPTS
            value: "-Xms2g -Xmx2g"
    # Request persistent data storage for pods
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
        storageClassName: ssd
  # Google cloud storage credentials
  secureSettings:
  - secretName: "gcs-credentials"
  http:
    service:
      spec:
        # expose this cluster Service with a LoadBalancer
        type: LoadBalancer
    tls:
      certificate:
        secretName: elasticsearch-certificate

Под не удалось инициализировать со следующей ошибкой:

{"type": "server", "timestamp": "2020-12-10T15:53:07,132Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "elastic-cluster-1", "node.name": "elastic-cluster-1-es-default-2", "message": "uncaught exception in thread [main]",
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/app/secrets/ca.pem\" \"read\")",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.1.jar:7.6.1]",
"at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.1.jar:7.6.1]",
"Caused by: java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/app/secrets/ca.pem\" \"read\")",
"at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]",
"at java.security.AccessController.checkPermission(AccessController.java:1036) ~[?:?]",
"at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]",
"at java.lang.SecurityManager.checkRead(SecurityManager.java:747) ~[?:?]",
"at sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) ~[?:?]",
"at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143) ~[?:?]",
"at sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156) ~[?:?]",
"at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217) ~[?:?]",
"at java.nio.file.Files.newByteChannel(Files.java:374) ~[?:?]",
"at java.nio.file.Files.newByteChannel(Files.java:425) ~[?:?]",
"at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~[?:?]",
"at java.nio.file.Files.newInputStream(Files.java:159) ~[?:?]",
"at org.elasticsearch.common.ssl.PemUtils.readCertificates(PemUtils.java:594) ~[?:?]",
"at org.elasticsearch.common.ssl.PemTrustConfig.loadCertificates(PemTrustConfig.java:83) ~[?:?]",
"at org.elasticsearch.common.ssl.PemTrustConfig.createTrustManager(PemTrustConfig.java:73) ~[?:?]",
"at org.elasticsearch.common.ssl.SslConfiguration.createSslContext(SslConfiguration.java:136) ~[?:?]",
"at org.elasticsearch.index.reindex.ReindexSslConfig.reload(ReindexSslConfig.java:145) ~[?:?]",
"at org.elasticsearch.index.reindex.ReindexSslConfig.<init>(ReindexSslConfig.java:115) ~[?:?]",
"at org.elasticsearch.index.reindex.ReindexPlugin.createComponents(ReindexPlugin.java:88) ~[?:?]",
"at org.elasticsearch.node.Node.lambda$new$9(Node.java:456) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]",
"at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621) ~[?:?]",
"at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]",
"at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]",
"at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]",
"at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]",
"at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]",
"at org.elasticsearch.node.Node.<init>(Node.java:459) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.1.jar:7.6.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.1.jar:7.6.1]",
"... 6 more"] }
uncaught exception in thread [main]
java.security.AccessControlException: access denied ("java.io.FilePermission" "/app/secrets/ca.pem" "read")
        at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
        at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
        at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
        at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:747)
        at java.base/sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255)
        at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:143)
        at java.base/sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:156)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:217)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:374)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:425)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:159)
        at org.elasticsearch.common.ssl.PemUtils.readCertificates(PemUtils.java:594)
        at org.elasticsearch.common.ssl.PemTrustConfig.loadCertificates(PemTrustConfig.java:83)
        at org.elasticsearch.common.ssl.PemTrustConfig.createTrustManager(PemTrustConfig.java:73)
        at org.elasticsearch.common.ssl.SslConfiguration.createSslContext(SslConfiguration.java:136)
        at org.elasticsearch.index.reindex.ReindexSslConfig.reload(ReindexSslConfig.java:145)
        at org.elasticsearch.index.reindex.ReindexSslConfig.<init>(ReindexSslConfig.java:115)
        at org.elasticsearch.index.reindex.ReindexPlugin.createComponents(ReindexPlugin.java:88)
        at org.elasticsearch.node.Node.lambda$new$9(Node.java:456)
        at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at org.elasticsearch.node.Node.<init>(Node.java:459)
        at org.elasticsearch.node.Node.<init>(Node.java:257)
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221)

Я проверил, что ca существует в пути /app/secrets/ca.pem и имеет разрешение на чтение:

Как я могу правильно это сделать? Почему он не может прочитать предоставленный ca?

0
0
654
2
Перейти к ответу Данный вопрос помечен как решенный

Ответы 2

Ответ принят как подходящий

У меня была точно такая же проблема, попробовал ваше решение, которое не удалось. Вроде должно работать. В итоге я отказался и отключил проверку удаленных сертификатов.

reindex.ssl.verification_mode: none

Сертификат ДОЛЖЕН находиться в каталоге конфигурации Elasticsearch. Единственная актуальная документация по этому вопросу находится в руководстве по настройке [не в справке, боже упаси] во втором разделе конфигурации ниже, примечание № 2.

https://www.elastic.co/guide/en/elasticsearch/reference/6.3/configuring-tls.html#tls-http

В случае официального образа Docker это /usr/share/elasticsearch/config/

И на ваш следующий вопрос: «Но зачем мне указывать полный путь, если он не может быть нигде, кроме каталога конфигурации?» ответ: ¯\_(ツ)_/¯

Другие вопросы по теме