6 Mistakes to Avoid for a Successful Splunk Cloud Migration

Planning on conducting a Splunk Cloud migration? 

Although moving to Splunk Cloud offers several benefits, the migration process itself is complex and highly error-prone. 

In this blog, we’ll break down six of the most common mistakes we’ve witnessed organizations make during Splunk Cloud migrations so that you can steer clear of unexpected costs and data issues. 

1. Failing to Optimize Searches to Prepare for a Smaller Splunk Environment.  

In a large on-prem Splunk environment, it’s easy for bad habits like poorly written searches to fly under the radar. This is because Splunk users can typically just request more servers from their IT team to accommodate the excess compute power inefficient searches require. 

With Splunk Cloud, however, this flexibility is lost. In the Workload Pricing model, your Splunk Cloud license is based on compute — measured through Splunk Virtual Core (SVC) units — and purchasing a high number of SVCs is extremely expensive and impractical. Organizations almost always end up collapsing their on-prem infrastructure into a much smaller Cloud infrastructure. 

To be able to process the same amount of data in a smaller environment, organizations must optimize their searches before migrating. One of our recent Cloud Migration clients failed to do this, and upon discovering how inefficient their searches were, we realized they would need to purchase almost double the number of SVCs they had originally planned for. It wasn’t until we conducted additional professional services to optimize their searches that they were able to proceed with the original number of SVCs. 

To prevent unexpected licensing/professional-services costs and skipped or delayed searches, it’s essential to audit your existing searches and eliminate inefficiencies before conducting the migration. 

2. Overlooking Small Details That Have Big Consequences. 

Failing to understand the nuances of Splunk Cloud and your organization’s unique architecture can lead to critical problems down the line. 

For example, many admins overlook the fact that Splunk Cloud doesn’t allow you to upload apps with a local directory, which is what stores user-specific or environment-specific configurations such as custom searches, dashboards, and field extractions.  

Once an app is uploaded to Spunk Cloud without a local directory, any changes you need to make to it will have to go through Splunk, as your organization will no longer have access to those settings itself. This is a time-consuming, frustrating process; however, it can be avoided through a competent admin who knows which alternative solutions to employ and how to proactively move custom configurations to a separate location. 

The impact of overlooking — or simply not being aware of — small details like this can have drastic consequences, such as unexpected expenses, frustrated users, wasted time, and the need to rope in more people across your organization. 

3. Having Admins Without Sufficient Knowledge Conduct the Migration. 

Conducting a Splunk Cloud migration requires a high degree of technical, architectural, and Cloud-specific knowledge that many admins might lack. 

To be qualified to complete a migration, Splunk admins should know, at minimum, the following details: 

• • The architecture for each existing Splunk piece (Heavy Forwarder, Search Head, Cluster manager, Indexer, Deployment Server, Deployer, and the various iterations of installed Universal Forwarders). 

• • How the various Splunk instances send and receive traffic. 

• • What kinds of Splunk traffic are generated (S-2-S, HEC, syslog, etc.)  

• • What the GDI (Getting Data In) process is for each data source currently in use. 

• • The outgoing socket (IP and port) information. 

• • The IP addresses or CIDR for the outbound firewall / server units.  

• • The outbound firewall protocols (IP vs servername for destinations).  

• • The process to adjust and change network traffic within the confines of the on-prem architecture. 

In addition, admins should be Splunk certified or at least knowledgeable to the Splunk Architect level. Failing to understand the existing architecture and how it functions will greatly impact the ability to correctly design the Splunk Cloud environment and its supporting on-prem environment.  

4. Failing to Set a Clear Plan and Identify Objectives.  

Correctly designing the Splunk Cloud environment requires organizations to proactively identify how they plan on using Cloud. Understanding this will guide decisions around data ingestion, storage needs, and search capabilities. 

Prior to the migration, admins must be able to answer the following questions: 

• • What (if any) historical data is going to be migrated? 

• • What datasets are going to be transitioned to Splunk Cloud? Is the ingest for all current datasets being moved to Splunk Cloud for indexing? 

• • Is Splunk Cloud going to become your organization’s “system of record” or data archive? 

• • If Splunk Cloud is to become the “system of record,” have you gone through the process to determine how much storage is required to fulfill applicable governances? 

• • Has that amount of storage been purchased and is it in place, or is it a future buy? 

• • Is data going to be archived locally? If so, how? (locally indexed Splunk data, AWS S3, Isilon spinning disk library, tape, etc.) 

• • Is there going to be a requirement for an on-prem Searchhead for either hybrid/federated searching of both on-prem and Splunk Cloud indexers or for searching archived on-prem storage only? 

Planning out these details ahead of time enables admins to design a solution that meets both current and future needs while avoiding problems like unexpected costs, compliance violations, and data hiccups. 

5. Not Connecting Your Splunk Cloud Migration to Business Value. 

To ensure your Splunk Cloud migration delivers actual business value, Splunk admins need to go beyond technical tasks and instead take on a strategic, collaborative role with other teams and leaders across the organization. The goal shouldn’t just be to move data – it should be to improve business and security operations through this data migration. 

For example, admins should hold transition workshops with key stakeholders to assess what data and apps should be moved to the cloud and what can be left behind. This will allow the admin to clean up outdated or redundant assets, optimizing the new environment for efficiency and improving overall security. 

Additionally, admins should work with stakeholders to gain a big-picture view of the goals and requirements of Splunk Cloud. Securing the support of an executive sponsor to advocate for the project and ensure resources are allocated properly is also beneficial. 

All of these points require admins to have strong communication, collaboration, and leadership skills in addition to technical skills. They must be well integrated in the company and willing to spend significant time outside of their department to ensure the migration provides as much ROI as possible. 

6. Overlooking Compliance Requirements Tied to Splunk. 

Many companies fail to identify how compliance requirements will be impacted by the move to Splunk Cloud, leading to violations, fines, and operational disruptions. 

For instance, financial and healthcare organizations are typically mandated to retain data for a set number of years through compliance standards like SOX or HIPAA. We recently worked with a company that completed their migration to the Cloud, but then realized three months later that they had a seven-year data retention requirement that would require them to continue using on-prem Splunk.

To avoid making a similar mistake, it’s critical to know the answer to the following questions before conducting your migration: 

• • What is your data governance? 

• • What is your required retention time? 

• • What is your system of record for holding that data? 

Get Expert Cloud Migration Help with SP6

At SP6, our certified Splunk engineers are highly experienced in providing end-to-end Splunk Cloud migration services and support. Our Cloud Migration workshops will help you build a Cloud environment that’s tailored to your organization’s unique business needs and drives real business value.