How to Create a Splunk Query

NOTE: I am using datasets provided by Hailie Shaw's "Splunk: Zero to Power User" Udemy course. My results will vary from your own test and live production datasets. Do not perform the queries in this article on equipment and services you do not own, have permission, or do not fully understand what the query does.

Guide to SPL Queries

Why?

Splunk queries are the foundation of how we search, analyze, and make sense of our data in Splunk and other SIEMs. Whether you're investigating a security incident, monitoring system performance, or building dashboards, queries are how you ask Splunk questions and get answers to then formulate actionable items like responding to a security incident.

What is Splunk?

Splunk is a platform that collects and indexes machine data like logs from servers, network devices, applications, and security tools. It allows power users to search and analyze that data in real-time, making it a powerful tool for IT operations, security teams, developers, and more.

What is a Splunk Query?

A Splunk query is a command or set of commands written in SPL (Search Processing Language). It tells Splunk what data you're interested in and what you want to do with it.

For example, you might want to:

• Find all failed login attempts

• Count how many users accessed a specific system

• Visualize error messages over time

Basic Structure of a Splunk Query

Most queries start by searching for general data, narrowing data to more specific data then using commands to filter, analyze, or format the results to make sense of thousands of logged events.

An example of a general Splunk query:
index = *

This will show ALL events in our Splunk environment. Using the * is a wildcard operator that will show anything that matches. In this case it is showing us all indexes available to us for All Time. 

WARNING: By using index = * you are performing a search in ALL indexes and it is very process heavy. This is a lazy and intense search that may consume too much bandwidth and may impact other Analysts performing searches on the Indexer you are performing the search on.

We are using datasets that are recorded far apart from each other. The datasets provided by the Udemy course were created in early 2021, and the data I inputted from our own Virtual Machine running the Splunk Enterprise test environment has logs created at the end of the 2024 calendar year.

In production environments you would likely want to limit your search results for the past day, week or month to avoid pulling outdated logs from decommissioned hosts amd hogging resources from other analysts performing queries.

Exploring our Data Further

Lets explore our data a little further. Lets see how many indexes we have to explore using the query below. We can see 4 different indexes to poke around, cisco, localhost_server, security, and web.

index = *
| stats count by index

Results:

The results show a list of all indexes for the time range you searched for has logged events to and counts how many events exist in each index.

To see the type of sources we have available to us in this environment we can run a similar command. The query below counts and groups our events by source types. We can see that  we have Access Combined, Cisco WSA Squid, and Linux Secure log source types available to search in this environment.

index = *
| stats count by sourcetype

Result:


 

 

To view all indexes AND the source types available in each index, we can combine the first two queries into a single query. This query helps us identify that there are two indexes that are storing Linux Secure source type logs. One from our localhost_server which is our RockyLinux virtual machine's secure log that I imported, and the other "security" which is provided from our Udemy course.

index = *
| stats count by index, sourcetype

Result:

Why perform any of the above commands?

Visualization and using the count function helps us understand what exists in our SIEM environment.

 Viewing Events

To list all events in an index we can specify which index we want to view events for. In the query below I am searching in the security index to view all 30,259 events.

index = security

Results:

Searching for specific words or strings helps us find events. If I wanted to search for events that contained the string "fail" in our security index, I would perform the query below. From the first 4 events I can see possible evidence of SSH brute force attempts by password spraying multiple generic name user accounts on our web2 host from IP address 64[.]66[.]0[.]20.

index = security "fail*"

Results:

 

Common SPL Commands

Here are some command SPL commands that you will typically use in your queries to narrow down events in your searches.

stats

Aggregates data (e.g., count, sum, average)

eval	Creates new fields or calculates values

table

Displays selected fields in a table format

rex	Extracts fields using regular expressions

where

Filters results using logic (e.g., greater than)

top	Shows most common values in a field

Scenario: Excessive Logins

In the Create A Splunk Alert we will create a new alert to notify our security team of excessive log ins.