Waste detection Ranked by observed cost

Databricks waste detection, pattern by pattern.

Databricks waste is spend with no work behind it: a cluster idling between jobs, a warehouse running without queries, an endpoint serving a model nothing calls. Little of it is carelessness — it’s capability nobody is using. LakeSentry detects these patterns across your workspaces, ranks them by observed cost, and shows the evidence behind each one.

Start Free

Read-only by default. Write access only if you choose to execute actions.

app.lakesentry.io/insights
Waste insights
12
across 4 workspaces
Observed cost
$€7,540/mo
in flagged patterns
Needs review
5
2 high impact
Waste High
Idle SQL warehouse
analytics_warehouse · running without queries
$€4,800/mo observed 1 day ago
Review plan
Waste Medium
Unused table
sales_raw_2024 · storage, no reads in 90 days
$€1,120/mo observed 3 days ago
Waste High
Retry storm
ml_feature_pipeline · repeated failed runs this week
$€980/mo observed 5 hours ago

The Patterns Nobody Is Assigned to Find

Some of this is visible natively, if you know to look: cluster event logs show idle time, query history shows failures. Each check is a query someone writes, per workspace, and then keeps running.

The patterns that cost the most tend to have no owner. A retry storm looks like activity. A generous auto-termination window looks like a configuration choice. An unused table adds storage cost every month and appears on nobody’s dashboard.

  • Failed SQL still consumes DBUs; failure and cost live in different native views.
  • An idle cluster looks identical to a busy one on the bill: a DBU is a DBU.
  • Storage waste never surfaces in compute-focused reviews.

How LakeSentry Detects Waste

A library of dedicated detectors, each producing a separate insight with its own evidence.

Pattern coverage

Idle clusters and warehouses, retry storms, runaway jobs, overprovisioned clusters, oversized drivers, long auto-termination windows, unused serving endpoints, unused tables, weekend activity, and queries that scan far more data than they need.

Ranked by observed cost

Estimates come from recent observed cost and the recommended change — prioritization guidance, not an invoice guarantee. The queue is ordered by dollar impact, so a weekly review begins with what matters.

Approval-first follow-through

Waste insights become action plans: evidence, safety tier, and the proposed change, reviewed before anything runs. Selected actions execute through LakeSentry after admin approval, with an audit log; the rest ship as guided plans you apply yourself. No unattended automation today.

Finding Waste Natively vs LakeSentry

Everything on the left is possible today. The question is who does it, and how often.

Doing it natively LakeSentry
Idle compute Event-log queries, per workspace Detected across all workspaces, ranked by cost
Retry storms Invisible unless someone correlates failures with cost A first-class detection, with the runs behind it
Unused tables Storage reports, checked occasionally Flagged when storage cost has no recent access signal
Sizing Utilization dashboards, cluster by cluster Overprovisioned clusters and oversized drivers surfaced with utilization evidence
Acting on it Tickets and tribal knowledge Action plans with evidence and safety tiers, approval-first

Go Deeper

The mechanics live in the docs; the reasoning lives on the blog.

Waste detection in the docs

The full pattern library and how each detector measures impact.

Databricks cost optimization: a practical guide

The pillar guide: right-sizing, warehouse tuning, and where automation fits.

Anomaly detection

The other half of the picture: changes, not steady states.

Waste detection is included from the Free tier: insight summaries across unlimited workspaces, no card required.

Free

$0€0

Standard

$499€499/mo

Pro

$849€849/mo

Full pricing

No per-DBU tax. No per-workspace fees. Paid tiers billed annually.

Frequently Asked Questions

What counts as waste in Databricks?
Cost with no work behind it: clusters or warehouses running idle, compute consumed by failures and retries, resources sized past their utilization, storage for tables nothing reads, endpoints serving models nothing calls. LakeSentry treats each as a distinct pattern with its own detector.
How are the impact estimates calculated?
From recent observed cost and the recommended change. They exist to order the queue, and the evidence behind each estimate is there to check — no number is a promised outcome.
Will LakeSentry shut down idle clusters automatically?
Not without you. Recommendations become action plans that an admin reviews and approves; approved actions execute through the Databricks API, with the approval and result captured in an audit log. There is no unattended automation today.
What access does waste detection need?
LakeSentry reads Databricks system tables — cost and usage metadata. It never accesses your business data, notebooks, or query results. Detection needs only the read-only service principal; write access is involved only when you execute an approved action.
How is waste detection different from anomaly detection?
Anomalies are changes: spend breaking its own baseline. Waste is a steady state: cost that was always there and never earned its keep. They share the same ledger and evidence discipline, and you review them as separate queues.

See it in your own environment.

The free tier covers unlimited workspaces with three months of history. No card required.

Start Free