Setup
Installation
To install and set up AME, a user with the admin
or sc_admin
role is needed.
Install Alert Manager Enterprise as any other Splunk App on the Search Head.
Directly install the App from Splunkbase or deploy the App to $SPLUNK_HOME/etc/apps
.
In a Splunk Searchhead-Cluster environment, deploy the App using the Deployer.
On-Premises Search Heard Cluster Installation
To install AME on a Search Head Cluster:
- Copy the app to the deployer directory
$SPLUNK_HOME/etc/shcluster/app
directory - Run
splunk apply shcluster-bundle
to apply the app - Run the application setup
In AME Versions <2.1 app.conf
and migrations.conf
had to be redeployed after
initial installation/updates.
This is not necessary anymore. These files can be safely removed from the
SHC Deployer from the local
directory.
Application Setup
After the installation, the Setup and Migration
page will list which tasks need
to be completed in order.
For new installations, the AME Setup has to run first. After the setup, it may be required to run upgrade tasks. See the update page for more details.
Main Setup
The Main Setup will set up a default tenant.
A tenant consists of four components:
- An index (created by an admin)
- A KV Store Collection (automatically created by AME)
- A HEC Collector (created by an admin)
- Splunk Roles (automatically created by AME)
The setup looks as follows:
The setup page shows a green bar if the setup has been completed successfully.
The following information is needed to set up the tenant:
Information | Description |
---|---|
Index | The index used for the default tenant. As a recommendation, use ame_default if possible. |
Host | The host used as HTTP event collector for the index of the Default Tenant. Only the host, no protocol like HTTP or HTTPS. |
Port | The HEC host has to be configured to accept incoming traffic on port 8088. (default: 8088). |
Token | The HEC host has to be configured to accept incoming splunk events using this token in the header.A typical location of the config file to store this token on the HEC host is: $SPLUNK_HOME/etc/apps/alert_manager_enterprise/local/inputs.conf |
SSL enabled | This will set https as a scheme to send events to the HEC host (recommended). |
SSL verified | To verify the certificate of the HEC host, check this option and specify the cacert of the HEC host. |
CA Certificate | This is the certificate of the HEC host found, e.g., at the location: $SPLUNK_HOME/etc/auth/cacert.pem |
Changing the index name later requires Professional Services; therefore, be careful when defining the name.
Application Administrator Role
Alert Manager Enterprise has an application admin role, ame.admin
.
The best practice is to assign the role to the application owner.
For an overview of all AME role capabilities, see Role Overview.
Do not assign an untrusted user the ame.admin
role!
Finalizing the Installation
For Splunk Enterprise (On-Premises) installations, continue here
For Splunk Cloud installations, continue here
Splunk Cloud Deployment Instructions
For Cloud installations, it is sufficient to create an index and a HEC Token using the sc_admin
account.
For Splunk Cloud: Use the dedicated HEC Receiver as the HEC Host. Note that the HEC Port is 443 and runs on SSL.
Only enter the domain name:
- AWS
http-inputs-yourstackname.splunkcloud.com
- GCP
http-inputs.yourstackname.splunkcloud.com
Optionally, add the DigiCert Global Root CA Certificate for verification. Follow the Splunk Docs on how to create an event index.
Follow the Splunk Docs on how to create a HEC Token.
Do not enable HEC Indexer Acknolwdgement in Splunk Cloud. This is not supported by Splunk Cloud. See https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/AboutHECIDXAck
If you are on Splunk Cloud Classic Stack and no events are created, please open a ticket with Splunk Support to verify the Functional Inputs are enabled (CIA-8485)
Splunk Enterprise Deployment Instructions
For Splunk Enterprise (On-premises) AME offers different deployment options.
Splunk Enterprise HTTP Event Collector (HEC) Deployment Options
Configuration templates may be used for on-premises installation.
Enabling HEC Acknowledgment on the HEC receiver token is highly recommended.
Add a props.conf
configuration for HEC Receivers that do not reside on localhost
to prevent Events from being truncated!
Search Head / Search Head Cluster with Local HTTP Event Collector and Log Forwarding
In this scenario, the Search Head/Search Head Cluster is the local HEC Receiver (localhost
).
The Search Head is configured to send its logs to the indexers (outputs.conf
).
Search Head / Search Head Cluster with Remote HTTP Event Collector on Heavy Forwarder
In this scenario, the Alert Manager Enterprise app sends data to a remote Heavy Forwarder that acts as a HEC Receiver. The Heavy Forwarder sends its logs to the indexers.
Search Head / Search Head Cluster with HTTP Event Collector on Indexers with Load-Balancer
In this scenario, the Alert Manager Enterprise app sends its data to a load-balancer, which forwards the connection to Indexers with an HEC Receiver configured.
For Load-Balancers, the following HTTP connection settings are supported:
- HTTP/1.1 with a connection header Keep-Alive (Splunk HEC default)
- HTTP/1.1 with a connection header Close
Testing the HEC Reciever Connection
Follow the Health-Check guide in the Troubleshooting Guide under The HEC Connection does not work
.
Modifying Settings
To modify app settings, click the Administration
menu and open the Setup
page.
Alternatively, the settings can also be modified under the Tenant Settings.