Splunk AI/ML Toolkit Setup

Prerequisites

Tool or access	Purpose	Verify
Bash and Python 3	Run bundled setup and validation helpers	bash --version && python3 --version
Required product/platform access	Inspect or configure the selected target	Complete the documented preflight
Credential files for live modes	Keep secrets out of chat	Verify paths only

Workflow Overview

┌───────────┐   ┌───────────────┐   ┌───────────────┐   ┌─────────────────┐
│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │
└───────────┘   └───────────────┘   └───────────────┘   └─────────────────┘

When to Activate

• MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for Scientific Computing, DSDL, Deep Learning Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent Launchpad, the aiagent command, Cisco Time Series Model, Cisco Deep.
• Preview and review the splunk ai ml toolkit setup workflow before any live apply phase.
• Diagnose failed prerequisites, generated assets, configuration, or validation evidence.

Scope

Follow the documented read-only or render-first path whenever it is available. This skill does not imply permission to mutate live systems. Require explicit apply flags, protected credentials, and operator review for state changes.

Examples

Inspect the supported setup modes before selecting one:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh --help

Expected output: usage, supported modes, and required arguments are displayed without changing the target environment.

Inspect validation modes before running completion checks:

bash skills/splunk-ai-ml-toolkit-setup/scripts/validate.sh --help

Expected output: offline, live, and completion options are displayed when the skill supports them; help exits without mutation.

Troubleshooting

Issue	Cause	Resolution
Preflight fails	A required tool or access path is missing	Resolve it before rendering or applying
Rendered assets are incomplete	Required non-secret inputs are absent	Complete intake and render again
Apply is blocked	Review, credentials, or explicit acceptance is missing	Use the documented handoff
Validation is incomplete	Live evidence is unavailable	Record the gap and keep completion open

Shared add-on completion gate

Whenever this workflow installs, configures, or hands off a registry-listed Splunk app or add-on, follow the shared completion gate. Package delivery alone is not success; capture applicable configuration, data/readiness, and shipped-view evidence, or explicit package evidence that no dashboards ship.

Use this skill for Splunk-owned AI and machine-learning platform workflows that are not Splunk AI Assistant. It owns coverage reporting, install orchestration, compatibility validation, DSDL runtime handoffs, and migration guidance for legacy anomaly apps.

For newer Cisco Data Fabric wording, this is the AI Toolkit / model-workflow route. Federated search, edge/ingest pipelines, and MCP server setup remain in their dedicated skills.

AI Toolkit Agent Launchpad is owned here. It became generally available in AI Toolkit 6.0.0, replacing the earlier Agent Builder feature preview. It is not Cisco Cloud Control Studio Agent Builder, which belongs to cisco-cloud-control-setup.

Coverage Boundary

This skill covers Splunk-owned and Splunk-supported AI/ML products:

• Splunk AI Toolkit / MLTK (Splunk_ML_Toolkit, Splunkbase 2890)
• Python for Scientific Computing (PSC) add-ons:
  • Linux 64-bit (2882, Splunk_SA_Scientific_Python_linux_x86_64)
  • Windows 64-bit (2883, Splunk_SA_Scientific_Python_windows_x86_64)
  • Mac Intel (2881, Splunk_SA_Scientific_Python_darwin_x86_64)
  • Mac Apple Silicon (6785, Splunk_SA_Scientific_Python_darwin_arm64)
  • Linux 32-bit (2884) as legacy migration/blocking coverage only
• Splunk App for Data Science and Deep Learning / DSDL (4607, package id mltk-container)
• AI Toolkit Smart Assistants, ML-SPL commands, model management, ONNX apply, LLM ai command readiness, Connections tab, Container Management tab, external LLM/provider connection handoffs, ML alerting, and Cisco Deep Time Series forecasting/anomaly detection readiness
• AI Toolkit Agent Launchpad readiness, generally available since 6.0.0, including supported LLM and MCP providers, Agent Skills, the edit_agent_connections and run_agents capabilities, the aiagent ML-SPL command, the in-product run-history surface, and the Splunk Cloud region and egress-allowlist prerequisites
• Cisco Time Series Model 1.0 as an available Apache-2.0 open-weight model, kept distinct from the AI Toolkit-integrated Cisco Deep Time Series Model and its hosted Splunk Cloud or self-hosted Enterprise paths
• Hosted foundation model readiness where available in the Splunk Platform boundary, including Foundation-Sec and GPT-OSS review handoffs; CDTSM is not represented as an LLM connection, and this skill never renders external model API keys
• Legacy Splunk App for Anomaly Detection (6843) and Smart Alerts Assistant beta (6415) as audit and migration-only coverage

Third-party AI-tagged Splunkbase apps are out of scope unless another skill explicitly routes them.

Splunk 10.5 Legacy Package Guardrail

Splunkbase does not list Splunk 10.5 support for PSC Linux 32-bit (2884), Splunk App for Anomaly Detection (6843), or Smart Alerts Assistant beta (6415). Do not install any of these three packages on a new or upgraded Splunk 10.5 deployment. They remain in this skill only so an existing estate can inventory dependencies and render a migration plan. Use a supported 64-bit PSC package and current Splunk AI Toolkit workflows for replacement coverage.

Safety Rules

• Never ask for Splunk passwords, Splunkbase passwords, HEC tokens, LLM API keys, cloud provider secrets, DSDL container credentials, or model registry tokens in chat.
• Never pass secrets on the command line or as environment-variable prefixes.
• LLM provider credentials, HEC tokens, Splunk access tokens, Docker registry secrets, Kubernetes kubeconfigs, and TLS key material must be file-backed or delegated to the owning setup skill.
• Do not install legacy EOL/beta anomaly apps by default, and never install 2884, 6415, or 6843 on Splunk 10.5. Audit and migrate them to current AI Toolkit workflows.
• Do not claim DSDL runtime automation for Docker, Kubernetes, OpenShift, HPC, GPU, air-gapped images, Jupyter notebooks, or model governance unless the workflow is rendered as a handoff or an owning runtime skill applies it.
• Do not report Agent Launchpad as ready from a package install alone. The public package ships the views and the aiagent command, but Splunk Cloud still needs a supported AWS region plus the region's egress IP in the stack allowlist, and Splunk Enterprise needs the Splunk Cloud Connect app.
• Do not pair AI Toolkit 6.0.2 with a PSC release below 4.3.4, and do not downgrade the audited AI Toolkit/PSC pair based on older documentation.
• Do not conflate the open Cisco Time Series Model 1.0 release with the Cisco Deep Time Series Model integration. Both are available, but the open model weights, the model service, and the AI Toolkit experience remain separately validated layers.
• Do not create or redirect the agent run-history index, and do not save knowledge-base, MCP, LLM, or model-server credentials from this render-only handoff.

Primary Workflow

Render and validate a complete coverage plan:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --spec skills/splunk-ai-ml-toolkit-setup/template.example \
  --output-dir splunk-ai-ml-toolkit-rendered

Install or update AI Toolkit with the right PSC add-on:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --install \
  --psc-target linux64

Include DSDL package delivery and runtime handoff artifacts:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --include-dsdl \
  --dsdl-runtime kubernetes \
  --output-dir splunk-ai-ml-toolkit-rendered

Audit legacy anomaly apps without installing them:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --doctor \
  --legacy-anomaly-audit

Apply Model

• --install renders, runs the offline structural check, delegates package delivery, then runs a live post-install validation (skipped under --dry-run) that confirms the PSC prerequisite is installed (a missing PSC add-on fails) and that AI Toolkit — and DSDL when planned — are present in Splunk. Live validation requires Splunk credentials.
• --validate always renders before validating, so the offline structural check never runs against a stale or missing rendered directory.
• Package delivery delegates to splunk-app-install.
• Package delivery intentionally omits --app-version so Splunkbase/ACS pulls the latest compatible release; audited version metadata is used for reports and regression checks, not as a live install pin.
• AI Toolkit and PSC belong on the search tier/search head cluster only.
• Install order is PSC first, AI Toolkit second, optional DSDL third.
• DSDL external runtimes are rendered handoffs by default: docker, kubernetes, openshift, hpc, gpu, airgap, or handoff.
• Legacy Anomaly Detection and Smart Alerts beta are never part of the default install plan; the skill emits migration reports instead.

Validation Rules

Validation must fail for:

• Unknown coverage statuses in coverage-report.json
• AI Toolkit install plans without a selected compatible PSC target
• PSC Linux 32-bit as a new install target; on Splunk 10.5 it is inventory and migration coverage only
• DSDL requested without AI Toolkit and PSC coverage in the same plan
• Direct-secret flags such as --token, --api-token, --password, --client-secret, or --llm-api-key
• Missing or unsupported upstream product_stage values in the rendered coverage report, including any report that fails to keep the Agent Launchpad surfaces and integrated CDTSM at ga and open CTSM 1.0 at available for the verified 2026-08-20 source baseline

Validation must warn for:

• DSDL Docker runtime in production because TLS, image provenance, and network isolation must be handled by the operator
• AI Toolkit/PSC versions lower than the latest audited compatibility pair
• Any attempted Splunk 10.5 install of legacy Anomaly Detection or Smart Alerts Assistant beta
• MLTK model objects created before the MLTK 5.3 compatibility break, which may need retraining
• Agent Launchpad requests without a confirmed supported Splunk Cloud region and egress allowlist entry, or without Splunk Cloud Connect on Enterprise

References

• Read reference.md before changing product coverage, compatibility rules, generated artifacts, or live install behavior.
• Use scripts/render_assets.py --discover to print the built-in product catalog and coverage surface.