Elasticsearch
Elasticsearch connections to the CipherTrust Manager can be configured using the following:
Note
CipherTrust Manager can have a total of 64 log forwarder connections. Each Elasticsearch, Loki, and Syslog connection is counted towards the 64 connection total.
Managing Elasticsearch Connections using GUI
Log on to CipherTrust Manager UI as an administrator.
Navigate to Access Management > Connections.
Click Add Connection.
On the Add Connection screen, select category as Log Forwarder.
Select Log Forwarder Type as Elasticsearch and click Next.
Specify connection Name and Description and click Next.
Configure the below parameters.
Host: IP address or hostname of the Elasticsearch server.
Port: port number for connecting to the Elasticsearch server.
HTTP User: name of the HTTP user for basic authentication.
HTTP Password: password of the HTTP user.
Transport: protocol for transport. Either TCP or TLS. TLS requires additional settings:
Settings for TLS:
You must provide CA Certificate or select Skip Certificate Verification
CA Certificate: either upload the CA certificate or paste the certificate content. This is a CA which directly or indirectly signs the Elasticsearch server certificate.
File Upload: select and click Upload Certificate to upload the trusted CA certificate from your machine. In TLS mode, the CA certificate must be in PEM format.
Text: select and paste the certificate content in the text field.
Skip Certificate Verification: select if the TLS verification of the CA certificate is not required.
Click Test Credentials to check whether the connection is configured correctly. If the test is successful, the status is
OKelse the status isFail.Click Next to move to the Add Products screen of the Add Connection wizard.
Managing Elasticsearch Connections using ksctl
The following operations can be performed:
Create/Get/Update/Delete an Elasticsearch connection
List all Elasticsearch connections
Test an existing Elasticsearch connection
Test the new Elasticsearch connection parameters before establishing the connection
Parameter Details
| Parameter | Mandatory/Optional | Description |
|---|---|---|
| name | Mandatory | Unique name of the connection. |
| description | Optional | Connection description. |
| products | Optional | List of products. |
| host | Mandatory | Host of the log-forwarder server. |
| port | Mandatory | Port of the log-forwarder server. |
| transport | Optional | Transport mode for sending data, supports "tls" and "tcp". The "tls" mode requires either a trusted CA certificate or insecure TLS skip verify to be set to true. The default transport mode is "tcp". |
| json-file | Optional | meta information in json format. This information is provided in --meta "{\"color\":\"blue\",\"foo\":\"bar\"}". |
| http-user | Optional | HTTP basic auth username. |
| http-password | Optional | HTTP basic auth password. |
| ca_cert | Optional | CA certificate in PEM format. |
| insecure-skip-verify | Optional | In TLS mode, skips server certificate validation. This setting should only be used for testing. |
Elasticsearch connections can be authenticated with username and password or unauthenticated. Check the Elasticsearch deployment for the authentication required for incoming data.
The tls mode requires either a trusted CA certificate in the PEM format or --insecure-skip-verify set to true. By default, the CipherTrust Manager attempts to validate Elasticsearch's server certificate using the provided CA. The external CAs already present on the CipherTrust Manager are not used. Use --ca-cert to provide a CA which directly or indirectly signs the server certificate. Use the --insecure-skip-verify flag to indicate that CipherTrust Manager does not verify Elasticsearch's presented server certificate.
Creating an Elasticsearch Connection
To create an Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch create --name <connection-name> --products <products-name> --description <description> --host <host> --port <port> --transport <transport-protocol> --http-user <username> --http-password <password> --ca-cert <ca-cert> --meta <Key:Value> --insecure-skip-verify
The supported transport mode for sending data are tls and tcp. The tls mode requires either a trusted CA certificate in the PEM format or --insecure-skip-verify set to true.
Example Request
ksctl connectionmgmt log-forwarder elasticsearch create --name elasticsearch-conn-1 --description conn-description --host 127.0.0.1 --port 1234 --transport tcp --http-user admin --http-password password
Example Response
{
"id": "9d3af367-d4a3-4838-8663-ce07d3e88353",
"uri": "kylo:kylo:connectionmgmt:connections:elasticsearch-conn-1-9d3af367-d4a3-4838-8663-ce07d3e88353",
"account": "kylo:kylo:admin:accounts:kylo",
"createdAt": "2022-05-04T07:53:49.838640861Z",
"updatedAt": "2022-05-04T07:53:49.837431591Z",
"service": "elasticsearch",
"category": "log-forwarders",
"last_connection_ok": null,
"last_connection_at": "0001-01-01T00:00:00Z",
"name": "elasticsearch-conn-1",
"description": "conn-description",
"host": "127.0.0.1",
"port": 1234,
"elasticsearch_params": {
"insecure_tls_skip_verify": false,
"transport": "tcp",
"http_user": "admin"
}
}
Getting Details of an Elasticsearch Connection
To get details of an Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch get --id <Connection-Name/ID>
Example Request
ksctl connectionmgmt log-forwarder elasticsearch get --id 9d3af367-d4a3-4838-8663-ce07d3e88353
Example Response
{
"id": "9d3af367-d4a3-4838-8663-ce07d3e88353",
"uri": "kylo:kylo:connectionmgmt:connections:elasticsearch-conn-1-9d3af367-d4a3-4838-8663-ce07d3e88353",
"account": "kylo:kylo:admin:accounts:kylo",
"createdAt": "2022-05-04T07:53:49.838641Z",
"updatedAt": "2022-05-04T07:53:49.837432Z",
"service": "elasticsearch",
"category": "log-forwarders",
"last_connection_ok": null,
"last_connection_at": "0001-01-01T00:00:00Z",
"name": "elasticsearch-conn-1",
"description": "conn-description",
"host": "127.0.0.1",
"port": 1234,
"elasticsearch_params": {
"insecure_tls_skip_verify": false,
"transport": "tcp",
"http_user": "admin"
}
}
Updating an Elasticsearch Connection
To update an Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch modify --id <Connection-Name/ID> --products <products-name> --description <description> --host <host> --port <port> --transport <transport-protocol> --http-user <username> --http-password <password> --ca-cert <ca-cert> --meta <Key:Value> --insecure-skip-verify
The supported transport mode for sending data are tls and tcp. The tls mode requires either a trusted CA certificate in the PEM format or --insecure-skip-verify set to true.
Example Request
ksctl connectionmgmt log-forwarder elasticsearch modify --id 9d3af367-d4a3-4838-8663-ce07d3e88353 --host 127.0.0.1
Example Response
{
"id": "9d3af367-d4a3-4838-8663-ce07d3e88353",
"uri": "kylo:kylo:connectionmgmt:connections:elasticsearch-conn-1-9d3af367-d4a3-4838-8663-ce07d3e88353",
"account": "kylo:kylo:admin:accounts:kylo",
"createdAt": "2022-05-04T07:53:49.838641Z",
"updatedAt": "2022-05-04T08:17:20.615258139Z",
"service": "elasticsearch",
"category": "log-forwarders",
"last_connection_ok": null,
"last_connection_at": "0001-01-01T00:00:00Z",
"name": "elasticsearch-conn-1",
"description": "conn-description",
"host": "127.0.0.1",
"port": 1234,
"elasticsearch_params": {
"insecure_tls_skip_verify": false,
"transport": "tcp",
"http_user": "admin"
}
}
Deleting an Elasticsearch Connection
To delete an Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch delete --id <Connection-Name/ID>
Example Request
ksctl connectionmgmt log-forwarder elasticsearch delete --id 9d3af367-d4a3-4838-8663-ce07d3e88353
Example Response
There will be no response if the Elasticsearch connection is deleted successfully.
Getting List of Elasticsearch Connections
To list all the Elasticsearch connections, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch list
Example Request
ksctl connectionmgmt log-forwarder elasticsearch list
Example Response
{
"skip": 0,
"limit": 10,
"total": 1,
"resources": [
{
"id": "9d3af367-d4a3-4838-8663-ce07d3e88353",
"uri": "kylo:kylo:connectionmgmt:connections:elasticsearch-conn-1-9d3af367-d4a3-4838-8663-ce07d3e88353",
"account": "kylo:kylo:admin:accounts:kylo",
"createdAt": "2022-05-04T07:53:49.839242Z",
"updatedAt": "2022-05-04T07:53:49.839242Z",
"service": "elasticsearch",
"category": "log-forwarders",
"last_connection_ok": null,
"last_connection_at": "0001-01-01T00:00:00Z",
"name": "elasticsearch-conn-1",
"description": "conn-description",
"host": "127.0.0.1",
"port": 1234
}
]
}
Testing an Existing Elasticsearch Connection
To test an existing Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch test --id <Connection-Name/ID>
Example Request
ksctl connectionmgmt log-forwarder elasticsearch test --id 9d3af367-d4a3-4838-8663-ce07d3e88353
Example Response
{
"connection_ok": true
}
Testing a New Elasticsearch Connection
To test the parameters of a New Elasticsearch connection, run:
Syntax
ksctl connectionmgmt log-forwarder elasticsearch test --host <host> --port <port> --transport <transport-protocol> --http-user <username> --http-password <password> --ca-cert <ca-cert> --insecure-skip-verify
Example Request
ksctl connectionmgmt log-forwarder elasticsearch test --host 127.0.0.1 --port 1234 --transport tcp --http-user admin --http-password password
Example Response
{
"connection_ok": true
}
