No matter the size of your organization, deploying Splunk in your environment is a big decision. After all, it can scale from very small to very large. Furthermore, it can be used as an on-premise hardware, or as a service to run within the Cloud.
Perhaps you’re wondering if you should go with Splunk Enterprise or Cloud. There are three key factors to consider as you’re weighing your options: resources, training budget, and intended size of your environment.
With this in mind, let’s explore Splunk products and services.
On-premise employment environment
The Splunk Validated Architecture document can assist you with design requirements. It can be helpful whether you require a single server deployment encapsulating all functions, or a level of availability and data redundancy leading to a more complex architecture. By following Splunk’s direction, you’ll design your initial deployment for scalability.
Some things to consider include:
- Daily Ingest How much uncompressed data do you plan to ingest daily?
- Concurrent Searches Some people mistake this for concurrent users, yet it’s possible to run lots of searches at the same time.
- Search Availability How well can your environment recover from outages at the search layer?
- Data Availability Does it require reliable access to all of the data at any time?
Then there’s this: a single-instance deployment incorporates one server. On the other hand, a distributed deployment incorporates multiple servers. A single instance can serve up to 300GB/day of data ingestion. Anything beyond that requires distribution across multiple servers.
So, will you be doing a distributed deployment? In that case, customer availability requirements determine whether to cluster servers at the search layer, indexing layer – or both. If it’s a complex environment, though, it’s common to spread Splunk deployments across geographical locations.
Generally, a deployment usually consists of a Single Search Head with a distributed collection of indexers. You must have a minimum of three indexers to create a Splunk Indexer Cluster. Conversely, if there’s added complexity, an on-premise environment will require additional servers. They’re needed to support the following roles: Cluster Master, License Master, SHC Deployer, Management Console, and Deployment Server.
Considerations that may be more customer-dependent:
- Platform Do you intend to deploy Linux or Windows? You’ll base that decision on the staffing resources and expertise you have internally. You can also view platform-specific requirements in this system requirements document. Be sure to read the guidance on degraded performance when incorporating VMs. In addition, this document offers guidance on the use of containers.
- Maintenance and Operations As with any enterprise-level application, Splunk incorporates specific server requirements, including user-level and server-level settings. You’ll need to make tweaks to the local firewall to allow inter-server communication. By maintaining consistent build processes for these servers through the life of the application, you can prevent unnecessary outages.
- Universal Forwarders This is the small Splunk installation loaded on host machines that forwards data to the Splunk environment. You’ll need to create a process to install, manage and distribute this software. Meanwhile, Splunk provides a Deployment Server functionality managing the configurations of the software. You may want to add a customer automation tool, such as Puppet or Chef, to distribute it.
- Firewalling This requires rules enabling UF-to-Indexer communication and user-to-search head communication, along with the security of the search/indexing layer and all utility functions.
- Training Don’t implement Splunk and then walk away. It takes specific Splunk knowledge to administer the environment through daily user requests, server operations, and application upgrades. Therefore, it’s critical that staff develop and maintain these skills, especially as the environment grows.
Splunk Cloud deployment
Do you have had experience administering your own on-premise Splunk environment?
Or, perhaps you’re unfamiliar with Splunk. In that case, Cloud could be your solution.
For newcomers, it looks and feels like Splunk Enterprise. Splunk determines and implements topology and architectural details to maintain contractual SLAs. So, you won’t have to deal with many issues related to an on-premise deployment. And the Splunk Cloud team handles much of the ongoing operational aspects of maintaining the environment.
However, Splunk does not monitor, or manage, on-premise functionality. To learn more, review the Splunk Validated Architectures guide.
Environment details
- A dedicated single-tenant cloud environment, including a single search head (plus one for each additional premium application), a clustered indexing tier, and an Inputs Data Manager
- Using AWS and GCP infrastructure layer(s) within multiple regions enhances availability
- 90 Day retention + archive options (see Dynamic Data Active Archive)
- Multi-Factor Authentication is available if using a SAML provider
- Secure options are available for PCI and HIPAA data
- Connectivity for data ingest to Splunk Cloud requires HTTP (889), Universal Forwarder (9997), AWS Kinesis Data and/or Firehose. The customer’s Universal Forwarders will communicate directly with the Splunk Cloud indexers
- The environment scales with little effort from the customer. It offers flexibility to scale with additional ingest requirements, additional functional requirements such as additional Splunk Applications, and the adoption of Splunk Premium applications, such as Enterprise Security and ITSI
- Search concurrency scales with additional ingest licensing
What seems different in Splunk Cloud vs. Enterprise Environment?
- All applications and changes loaded into Splunk Cloud must be vetted and changes approved. You will have less freedom to load configurations into Cloud. These policies ensure the system’s consistency and availability.
- Cloud provides a limited administrative role. If you have worked in a Splunk environment, you may have noticed this. Cloud enables you to manage users, apps, indexes, and knowledge objects. It will manage server-related tasks. Don’t think that this means your organization doesn’t need team members who can function as administrators.
- No CLI, no direct access to indexers for TCP, UDP, Syslog outputs Again, this is most likely to be noticed by previous Splunk administrators. Data ingestion can be carried out via a forwarder without direct access.
What does the customer require?
- Universal Forwarders With Splunk Cloud, data must be ingested into the environment in some way. On-premise data requires that a Universal Forwarder be loaded on the host(s), to forward data into the cloud. As with on-premises, management of the universal forwarders’ configurations may be done via a Deployment Server. You can use your existing processes to manage software distribution.
- Firewalling As we already mentioned, specific connectivity is required between Cloud and the customer REST (443), HTTP (889), Forwarder(9997), AWS Kinesis Data Firehose. Customers are responsible for all of these configurations.
- Other On-Premises Functionality Your Cloud deployment can gain a level of flexibility and control by using an on-premises Inputs Data Manager, combined with a Heavy Forwarder, if needed. If there are existing Splunk services on site, use a Hybrid Search Head to enable searches across both environments
In Conclusion
To sum it all up, most organizations deploy Splunk on-premises. They own all of the hardware for the environment, and bear all of the operational responsibility. As a result, organizations gain increased control and flexibility over hardware, training, staffing, and operational planning costs. However, you’ll have to manage more resources.
In its simplest form, Cloud maintains the Splunk search/indexing environment. All you have to do is push data into the Cloud, using a set of Universal Forwarders managed by a Deployment Server.
Finally, base your decision on the needs of your organization. When weighing the options, your three key considerations should be resources, training budget, and intended size of your environment. Otherwise, you may not achieve the desired outcome.
Are you still not certain which direction to go in? As a proud Splunk partner, SP6 would be happy to assist you. Feel free to contact us today for a consultation
Resources
Validated Architectures
https://www.splunk.com/pdfs/technical-briefs/splunk-validated-architectures.pdf
Cloud vs Splunk Enterprise
System Requirements on-premises
https://docs.splunk.com/Documentation/Splunk/latest/Installation/Systemrequirements
