VMware AIops

Author: Wei Zhou, VMware by Broadcom — wei-wz.zhou@broadcom.com This is a community-driven project by a VMware engineer, not an official VMware product. For official VMware developer tools see developer.broadcom.com.

English | 中文

AI-powered VMware vCenter/ESXi VM lifecycle and deployment tool — 60 tools.

Companion skills handle everything else:

Skill	Scope	Install
vmware-monitor	Read-only: inventory, health, alarms, events, metrics	uv tool install vmware-monitor
vmware-storage	Datastores, iSCSI, vSAN management	uv tool install vmware-storage
vmware-vks	Tanzu Namespaces, TKC cluster lifecycle	uv tool install vmware-vks

Need read-only monitoring only? Use VMware-Monitor — zero destructive code in the codebase.

⚡ Quick Investigation Reports (read-only)

Triage → investigate → act, all in one conversation. Five opinionated read-only reports aggregate and correlate server-side and hand back a high-signal result (never raw inventory), so you can decide where to look before changing anything. Each renders a self-contained offline HTML snapshot with --html (no external assets; drill-down detail collapses in native <details>, zero JavaScript). All delegate to the vmware-monitor library using AIops's own vCenter connection.

Question	Command	What it correlates
"What needs attention now?" across all vCenters	vmware-aiops attention	Every vCenter merged into one globally-ranked issue list; unreachable targets degrade gracefully
"Is anything on fire?" across all clusters	vmware-aiops summary	Every cluster's hosts + VM power + live CPU/mem + alarms → ranked top-N issues + per-cluster status
"What's happening around this VM?"	vmware-aiops investigate vm <name>	VM state + host + cluster + backing datastores + snapshots + alarms + performance + a merged event timeline
"What's happening around this host?"	vmware-aiops investigate host <name>	Host state + cluster + the VMs it runs + mounted datastores + alarms + performance + correlated timeline
"What's happening around this datastore?"	vmware-aiops investigate datastore <name>	Capacity/free + mounting hosts + VMs it backs + alarms + correlated timeline

vmware-aiops attention                            # what needs attention now, all vCenters
vmware-aiops investigate vm web-01 --hours 72     # everything around a VM, then act on it
vmware-aiops investigate vm web-01 --html         # → offline snapshot in ~/vmware-health/

Via MCP these are the tools cluster_health_summary, cross_vcenter_attention, vm_investigation_bundle, host_investigation_bundle, datastore_investigation_bundle. (Requires vmware-monitor installed.)

Quick Install (Recommended)

Works with Claude Code, Cursor, Codex, Gemini CLI, Trae, and 30+ AI agents:

# Via Skills.sh
npx skills add vmware-skills/VMware-AIops

# Via ClawHub
clawhub install @zw008/vmware-aiops

PyPI Install (No GitHub Access Required)

# Install via uv (recommended)
uv tool install vmware-aiops

# Or via pip
pip install vmware-aiops

# China mainland mirror (faster)
pip install vmware-aiops -i https://pypi.tuna.tsinghua.edu.cn/simple

Offline / Air-Gapped Install (from source)

This project uses the modern PEP 517 build system (hatchling), so there is no setup.py by design — that is expected, not a missing file. If you cloned the source and hit ERROR: File "setup.py" or "setup.cfg" not found ... editable mode currently requires a setuptools-based build, your pip is older than 21.3 and cannot do an editable (-e) install with a non-setuptools backend. Editable mode is a developer convenience, not needed to run the tool — do one of:

# From the source tree — a normal (non-editable) install builds a wheel:
pip install .              # NOT  pip install -e .

# ...or upgrade pip first, and editable works too:
pip install --upgrade pip && pip install -e .

For a truly air-gapped host, build the wheels on a connected machine and copy them over — the target then needs no network:

# On a connected machine, collect this package + its dependencies as wheels:
pip wheel . -w dist        # → dist/*.whl   (or: uv build, for just this package)

# Copy dist/ to the air-gapped host, then install offline:
pip install --no-index --find-links dist vmware-aiops

---

Why this over other VMware MCP servers

Most open-source VMware MCP servers (e.g. bright8192/esxi-mcp-server, giuliolibrando/vmware-vsphere-mcp-server) are single-vCenter VM wrappers: list/power/snapshot a VM, basic monitoring, a confirm=True flag. They explicitly do not cover networking, storage, Kubernetes, ops analytics, load balancing, or compliance — and "logging is documented" is not an audit trail.

This is one skill in an 11-package family that covers the whole estate and runs every tool through a governed harness:

	Other VMware MCP servers	This family
VM lifecycle + monitoring	✅	✅
NSX networking (segments/gateways/NAT/routing/IPAM)	❌	✅ vmware-nsx
NSX security (DFW/groups/IDS-IPS/traceflow)	❌	✅ vmware-nsx-security
Storage (datastore/iSCSI/vSAN)	❌	✅ vmware-storage
Tanzu Kubernetes (Supervisor/Namespace/TKC)	❌	✅ vmware-vks
Aria Operations (metrics/alerts/capacity)	❌	✅ vmware-aria
AVI / NSX ALB load balancing + AKO	❌	✅ vmware-avi
Compliance baselines + drift (CIS/SCG/等保/PCI)	❌	✅ vmware-harden
Governed harness (unified audit, policy engine, token budget + runaway breaker, graduated risk tiers, undo-token, prompt-injection sanitize)	❌	✅ vmware-policy on every tool

If you only ever power-cycle VMs in one vCenter, a single-file server is fine. If you run a real (regulated, NSX-segmented, multi-domain) VMware estate and need an AI operator an auditor can sign off on, that's what this family is for — see docs/compliance-ready.md.

Capabilities Overview

What This Skill Does

Category	Tools	Count
VM Lifecycle	power on/off, TTL auto-delete, clean slate	6
Deployment	OVA, template, linked clone, batch clone/deploy	8
Guest Ops	exec commands, upload/download files, provision	5
Plan/Apply	multi-step planning with rollback	4
Cluster	create, delete, HA/DRS config, add/remove hosts	6
Datastore	browse files, scan for images	2
Network	dvSwitch portgroup list/create, host VMkernel list/add/remove, DF-bit MTU-path ping	6

CLI vs MCP: Which Mode to Use

Scenario	Recommended	Why
Local/small models (Ollama, Qwen <32B)	CLI	~2K tokens context vs ~10K for MCP; small models struggle with many tool schemas
Token-sensitive workflows	CLI	SKILL.md + Bash tool = minimal overhead
Cloud models (Claude, GPT-4o)	Either	Both work; MCP gives structured JSON I/O
Automated pipelines / Agent chaining	MCP	Type-safe parameters, structured output, no shell parsing
Monitoring / storage / K8s	Companion skills	See vmware-monitor, vmware-storage, vmware-vks

Rule of thumb: Use CLI for cost efficiency and small models. Use MCP for structured automation with large models.

Architecture

User (Natural Language)
  ↓
AI CLI Tool (Claude Code / Gemini / Codex / Aider / Continue / Trae / Kimi)
  ↓ reads SKILL.md / AGENTS.md / rules
  ↓
vmware-aiops CLI
  ↓ pyVmomi (vSphere SOAP API)
  ↓
vCenter Server ──→ ESXi Cluster ──→ VM
    or
ESXi Standalone Host ──→ VM

Version Compatibility

vSphere / VCF Version	Support	Notes
VCF 9.1 / vSphere 9.1	✅ Full	Released 2026-05-12. pyVmomi <10.0 resolves and connects via SOAP; new REST-only features (PATCH /deployment/size, IPv6-only GOSC) not yet wrapped — see VCF Python SDK for those.
VCF 9.0 / vSphere 9.0	✅ Full	pyVmomi 8.0.3+ connects against vSphere 9 SOAP API. From VCF 9, pyVmomi is also bundled inside the unified VCF Python SDK.
8.0 / 8.0U1-U3	✅ Full	CreateSnapshot_Task deprecated → use CreateSnapshotEx_Task
7.0 / 7.0U1-U3	✅ Full	All APIs supported
6.7	✅ Compatible	Backward-compatible, tested
6.5	✅ Compatible	Backward-compatible, tested

pyVmomi auto-negotiates the API version during SOAP handshake — no manual configuration needed. The same codebase manages 7.0 / 8.0 / 9.0 / 9.1 environments seamlessly.

Official Broadcom References

• SDKs: https://developer.broadcom.com/sdks — VCF Python SDK (recommended for VCF 9+, bundles pyVmomi + vSAN SDK), vSphere Automation SDK for Python (REST APIs)
• REST APIs: https://developer.broadcom.com/xapis — vSphere Automation API, VCF API, SDDC Manager API
• CLI Tools: https://developer.broadcom.com/tools — PowerCLI 9.1, ESXCLI, OVF Tool

---

Common Workflows

Deploy a Lab Environment

1. Browse datastore for OVA images → vmware-aiops datastore browse <ds> --pattern "*.ova"
2. Deploy VM from OVA → vmware-aiops deploy ova ./image.ova --name lab-vm --datastore ds1
3. Install software inside VM → vmware-aiops vm guest-exec lab-vm --cmd /bin/bash --args "-c 'apt-get install -y nginx'" --user root
4. Create baseline snapshot → vmware-aiops vm snapshot-create lab-vm --name baseline
5. Set TTL for auto-cleanup → vmware-aiops vm set-ttl lab-vm --minutes 480

Batch Clone for Testing

1. Create plan: vm_create_plan with multiple clone + reconfigure steps
2. Review plan with user (shows affected VMs, irreversible warnings)
3. Apply: vm_apply_plan executes sequentially, stops on failure
4. If failed: vm_rollback_plan reverses executed steps
5. Set TTL on all clones for auto-cleanup

Migrate VM to Another Host

1. Check VM info via vmware-monitor → verify power state and current host
2. Migrate: vmware-aiops vm migrate my-vm --to-host esxi-02
3. Verify migration completed

---

VM Lifecycle

Operation	Command	Confirmation	vCenter	ESXi
Power On	vm power-on <name>	—	✅	✅
Graceful Shutdown	vm power-off <name>	Double	✅	✅
Force Power Off	vm power-off <name> --force	Double	✅	✅
Reset	plan action reset via vm_create_plan (MCP; no CLI command)	—	✅	✅
Suspend	plan action suspend via vm_create_plan (MCP; no CLI command)	—	✅	✅
Create VM	vm create <name> --cpu --memory --disk	—	✅	✅
Delete VM	vm delete <name>	Double	✅	✅
Reconfigure	vm reconfigure <name> --cpu --memory	Double	✅	✅
Create Snapshot	vm snapshot-create <name> --name <snap>	—	✅	✅
List Snapshots	vm snapshot-list <name>	—	✅	✅
Revert Snapshot	vm snapshot-revert <name> --name <snap>	Double	✅	✅
Delete Snapshot	vm snapshot-delete <name> --name <snap> [--no-wait]	Double	✅	✅
Task Status	vm task-status <task-id>	—	✅	✅
Clone VM	vm clone <name> --new-name <new>	Double	✅	✅
vMotion	vm migrate <name> --to-host <host>	Double	✅	❌
Set TTL	vm set-ttl <name> --minutes <n>	Double	✅	✅
Cancel TTL	vm cancel-ttl <name>	—	✅	✅
List TTLs	vm list-ttl	—	✅	✅
Clean Slate	vm clean-slate <name> [--snapshot baseline]	Double	✅	✅
Guest Exec	vm guest-exec <name> --cmd /bin/bash --args "..." --user <account>	Double	✅	✅
Guest Exec (with output)	MCP only: vm_guest_exec_output (username required) — no CLI command	—	✅	✅
Guest Upload	vm guest-upload <name> --local f.sh --guest /tmp/f.sh --user <account>	Double	✅	✅
Guest Download	vm guest-download <name> --guest /var/log/syslog --local ./syslog --user <account>	—	✅	✅

Guest Operations require VMware Tools running inside the guest OS, and the guest account is always named explicitly — --user on the CLI, username over MCP. There is no default, so no call runs as root without choosing root. vm_guest_exec_output (MCP only; the CLI has no equivalent) auto-detects Linux/Windows shell and captures stdout/stderr.

Plan → Apply (Multi-step Operations)

For complex operations involving 2+ steps or 2+ VMs, use the plan/apply workflow instead of executing individually:

Step	What Happens
1. Create Plan	AI calls vm_create_plan — validates actions, checks targets in vSphere, generates plan with rollback info
2. Review	AI shows plan to user: steps, affected VMs, irreversible warnings
3. Apply	vm_apply_plan executes sequentially; stops on failure
4. Rollback (if failed)	Asks user whether to rollback, then vm_rollback_plan reverses executed steps (irreversible steps skipped)

Plans stored in ~/.vmware-aiops/plans/, auto-deleted on success, auto-cleaned after 24h.

VM Deployment & Provisioning

Operation	Command	Speed	vCenter	ESXi
Deploy from OVA	deploy ova <path> --name <vm>	Minutes	✅	✅
Deploy from Template	deploy template <tmpl> --name <vm>	Minutes	✅	✅
Linked Clone	deploy linked-clone --source <vm> --snapshot <snap> --name <new>	Seconds	✅	✅
Attach ISO	deploy iso <vm> --iso "[ds] path/to.iso"	Instant	✅	✅
Convert to Template	deploy mark-template <vm>	Instant	✅	✅
Batch Clone	deploy batch-clone --source <vm> --count <n>	Minutes	✅	✅
Batch Deploy (YAML)	deploy batch spec.yaml	Auto	✅	✅

Cluster Management

Operation	Command	Confirmation	vCenter	ESXi
Cluster Info	cluster info <name>	—	✅	❌
Create Cluster	cluster create <name> [--ha] [--drs]	—	✅	❌
Delete Cluster	cluster delete <name>	Double	✅	❌
Add Host	cluster add-host <cluster> --host <host>	Double	✅	❌
Remove Host	cluster remove-host <cluster> --host <host>	Double	✅	❌
Configure HA/DRS	cluster configure <name> [--ha/--no-ha] [--drs/--no-drs]	Double	✅	❌

remove-host requires the host to be in maintenance mode first; the host is moved out of the cluster into the datacenter's host folder as a standalone host.

Alarm Management

Operation	Command	Confirmation	vCenter	ESXi
List Triggered Alarms	alarm list [--target <t>]	—	✅	❌
Acknowledge Alarm	alarm acknowledge <entity> <alarm>	—	✅	❌
Clear (Reset) Alarms	alarm reset <entity> <alarm>	Double	✅	❌

Blast radius: vSphere has no per-alarm clear API. alarm reset uses AlarmManager.ClearTriggeredAlarms, which clears all triggered alarms matching the named alarm's entity type (host/VM/all) and current status (red/yellow) — not just the named one. The named alarm is looked up first (typos fail fast), and the output's scope field reports exactly what was cleared. Cleared alarms re-trigger automatically if their underlying condition persists.

Datastore Browser

Feature	vCenter	ESXi	Details
Browse Files	✅	✅	List files/folders in any datastore path
Scan Images	✅	✅	Discover ISO, OVA, OVF, VMDK across all datastores

Scheduled Scanning & Notifications

Feature	Details
Daemon	APScheduler-based, configurable interval (default 15 min)
Multi-target Scan	Sequentially scan all configured vCenter/ESXi targets
Scan Content	Each cycle: triggered alarms, vCenter events from the last lookback_hours, and new lines in the ESXi host logs hostd, vmkernel, vpxa
Host Logs	Read incrementally: each line is reported once per daemon run (a restarted daemon re-reads each log's last 500 lines once). A rotated log, or more than 500 new lines between cycles, adds an info row saying which lines were not scanned. Reading host logs needs the Global.Diagnostics privilege, which vCenter's Read-Only role does not include; a log that cannot be read becomes an info row with the reason, never a silent "all clear"
Log Analysis	Host-log lines matching error, fail, critical, panic, lost access, cannot, timeout, refused, corrupt — lines with critical/panic/corrupt are critical, the rest warning
Structured Log	JSONL output to ~/.vmware-aiops/scan.log — every issue, info rows included
Webhook	Slack, Discord, or any HTTP endpoint. Receives every critical issue and every alarm/event warning; host-log warnings go to the scan log only, and info rows are never sent
Cycle Summary	One line per cycle in the daemon's log output: findings (and how many went to the webhook), unreadable host logs, logs with unscanned lines, failed passes. If any pass failed or a target could not be reached it reads Scan INCOMPLETE, never "all clear"
Daemon Management	daemon start/stop/status, PID file, graceful shutdown

Safety Features

Feature	Details
Dry-Run Mode (CLI only)	--dry-run prints the exact API call without executing, on every CLI write except deploy iso, deploy mark-template, vm cancel-ttl and vm guest-download
Plan → Confirm → Execute → Log	CLI workflow: show current state, confirm changes, execute, audit log
Double Confirmation (CLI only)	Destructive and deploy CLI commands (vm power-off, delete, reconfigure, snapshot-revert/delete, clone, migrate, set-ttl, clean-slate, guest-exec, guest-upload; deploy ova, template, linked-clone, batch, batch-clone, mark-template; cluster delete, add-host, remove-host, configure, drs-rule-set/create/delete; alarm reset) require 2 sequential prompts and take no bypass flag
No confirmation on the MCP path	The 43 write tools an agent sees over MCP act on the first call — no confirmed= handshake, no approval tier, no read-only switch. What decides whether a write lands is the privilege of the vCenter account, and what records it is the audit trail. See What protects you
Rejection Logging	Declined CLI confirmations are recorded in the audit trail
Audit Trail	All operations logged to ~/.vmware-aiops/audit.log (JSONL) with before/after state
Input Validation	VM name, CPU (1-128), memory (128-1048576 MB), disk (1-65536 GB) validated
Password Protection	.env file loading with permission check; never in shell history
SSL Self-signed Support	verify_ssl: false — only for ESXi with self-signed certs in isolated labs; production should use CA-signed certificates
Prompt Injection Protection	vSphere event messages and host logs are truncated, stripped of control characters, and wrapped in boundary markers before output
Webhook Data Scope	Disabled by default. When configured, the daemon posts to your URL only: every critical issue (alarms, events, ESXi log lines matching critical/panic/corrupt, targets it could not connect to) and every alarm/event warning — host-log warnings stay in the scan log, and info rows are never sent. Each issue carries its entity name and message: sanitized alarm, event, or ESXi log text, or the connection error, which can include host names, IP addresses, and user names. No credentials from the skill's config or .env are sent
Task Waiting	All async operations wait for completion and report result
State Validation	Pre-operation checks (VM exists, power state correct)

vCenter vs ESXi Comparison

Capability	vCenter	ESXi Standalone
vMotion migration	✅	❌
Cross-host clone	✅	❌
Cluster management	✅	❌
All VM lifecycle ops	✅	✅
OVA/Template/Linked Clone deploy	✅	✅
Datastore browsing & image scan	✅	✅
Snapshots	✅	✅
Guest operations	✅	✅

Inventory, alarms, events, sensors, host services, and scanning are now in vmware-monitor.

What protects you

The table above lists two different surfaces and it is worth being blunt about which protections apply to which, because getting this wrong is worse than having no protection at all — a guardrail you believe in is one you stop compensating for.

On the CLI, a destructive command asks twice and takes no bypass flag, and --dry-run previews every write except deploy iso, deploy mark-template, vm cancel-ttl and vm guest-download. That defends a mistyped command typed by a human. It does not defend against an agent, which satisfies both prompts with yes |.

Over MCP, there is no confirmation step at all. All 43 write tools — vm_delete, cluster_delete, vm_guest_exec among them — act on the first call. Seven host-networking and DRS tools take a confirm argument that defaults to a no-write preview, but that is a preview switch, not an approval gate: one more call is all it takes. This is deliberate. A confirmed= handshake was considered in July 2026 and cut, along with the earlier VMWARE_READ_ONLY switch, because neither was a real boundary — the switch was enforced on the MCP path only and any agent with a shell walked around it via the CLI, and a handshake is a speed-bump a model that intends to act steps over.

What actually decides whether a write lands is the vCenter/ESXi service account. Give the skill an account with the privileges the work needs and no more; vCenter refuses the rest itself, on every surface, with no way around it from inside this skill. To run an agent read-only, give it a read-only vCenter role — one decision, enforced where it is made. Every call is then recorded in ~/.vmware/audit.db before the caller sees a result, which is how you find out what happened. Optional deny rules in ~/.vmware/rules.yaml, checked before every MCP call, can refuse operations — for example, writes to targets labelled environment: production. The shipped baseline denies nothing, and the rules run inside the same process: a guardrail on top of RBAC, not a replacement.