Before Upgrading (READ FIRST!)
Splunk Cloud Python version
If you are on Splunk cloud run the following query before upgrading.
| rest /services/server/settings splunk_server=local | table python.version
If you get the output python2
- do not upgrade. Ask Splunk support to update the stack to Python3.
Python2 EOL was January 1, 2020, but there are still Splunk Cloud instances with Python 2.7 running.
For more information regarding python in Splunk Cloud see:
- https://www.splunk.com/en_us/blog/platform/removing-python-2-from-new-splunk-cloud-and-splunk-enterprise-releases-starting-september-2021.html
- https://docs.splunk.com/Documentation/Splunk/latest/Python3Migration/AboutMigration
If you are upgrading from an AME Version below 3.0
Rename objects with forbidden characters
In AME 3.0, there are have strict naming requirements for the following Objects.
- Templates
- Rules
- Statuses
- Notifications
Please check that they DO NOT contain any of the following characters: < > : ; { } [ ] , ' " \ .
and rename them before upgrading.
The pre-upgrade validation script in the next chapter will also check automatically check for these inconsistencies.
Run the pre-upgrade validation script
AME 3.0 has stricter consistency checks for objects. In the pre 3.0 versions, it was possible to delete objects that were still referenced somewhere. To check for inconsistencies, download the following script: ame-validate-3-0-upgrade.py
Execute the script with python3
.
The following arguments are valid. Use either username/password or a token for authentication:
python3 validate_3_0_upgrade.py [--url=url] [--username=username] [--password] [--token]
--url API URL of the AME instance - including port and protocol
--username Username to authenticate with
--password Password to authenticate with
--token Token to authenticate with
The command should return without errors:
$ python3 validate_3_0_upgrade.py --url https://x.x.x.x:8089 --token ey...c3LA
Fetched data from the AME instance
No known inconsistencies found in status options. Ready for upgrade.
No known inconsistencies found in notification schemes. Ready for upgrade.
No known inconsistencies found in rules. Ready for upgrade.
No known inconsistencies found in templates. Ready for upgrade.
...
Python3 is required with the request module. If splunk is installed, you can run the command with e.g.:
$SPLUNK_HOME/bin/splunk cmd python3 ./ame-validate-3-0-upgrade.py
Before upgrading AME, there should be no errors. Please get in touch with AME support for assistance if any consistency errors are reported you are not able to fix yourself.
Migrate Legacy Non-Slackapp Notifications to Slack-App
Legacy incoming webhooks are no longer supported in AME 3.0. It recommended that you migrate to app-based webhooks before migration. See Slack Documentation:
Migrating to app-based webhooks
Migrate Time-to-Live Auto-Resolve Settings
Previously, an event with a TTL had a fixed next status of resolved
.
With AME 3.0, it is now possible to define a target status for events that have Time-to-Live (TTL).
Before updating AME we recommend either add new fields/attributes or replace the old settings:
Replace all
eval
search overridesame.time_to_auto_resolve
withame.ttl
andame.ttl_target
Replace all
savedsearches.conf
overridesaction.create_alert.param.time_to_auto_resolve
withaction.create_alert.param.ttl
andaction.create_alert.param.ttl_target
.
Migrate Notification Templates to the new Jinja based format
Previously it was possible to create a custom email template. With Jinja based templating the format has changed. Please update the template to the new format.
Events can only be assigned to members of a tenant
Previously, it was possible to assign an event to all users. With AME 3.0, only users with the tenant's power user role can be selected for assignment.
Users need to be tenant members to create events
Previously, creating events in a tenant where the user running the alert was not a member was previously possible. Please check that all alerts run with a user who has the tenant power role assigned for which he creates events.