---
title: "What it checks"
description: "Loggerhead reads what your app emits and points at the noise, the leaks, and the gaps before production bills you for them."
source: "https://getloggerhead.com/docs/telemetry-linting/what-it-checks/"
---

# What it checks

> Loggerhead reads what your app emits and points at the noise, the leaks, and the gaps before production bills you for them.

Telemetry linting runs on the data already in Loggerhead. It has two parts, and both put their findings on the
**Linting** page:

- **Lint rules** are deterministic checks, like a code linter. The same data gives the same findings each time.
  You set them on the **Rules** tab of the **Linting** page. This page lists them.
- **AI analysis** sends one large prompt to a model over a window that you choose. It finds the patterns a fixed
  rule cannot see. See [AI analysis](#ai-analysis).

The lint rules read the last hour of logs, metrics, and spans. Each finding has an impact, a plain explanation, an action, and a
chip that names the rule that made it. There are 42 rules in four groups. Every rule has a toggle, and every rule but three has thresholds
on the **Rules** tab of the **Linting** page: see [Tune the rules](/docs/telemetry-linting/tune-the-rules/).

## The log checks

| Finding | Rule | Impact | What it means |
| --- | --- | --- | --- |
| Error logs missing trace context | `missing_trace_context` | High | A service's error records carry no trace ID, so they cannot link to the request that failed. The rule reads one service at a time, so a service that propagates the context cannot hide one that does not. |
| Possible sensitive data | `sensitive_data` | High | A log field looks like an email address, a token, or a card number. An email needs a domain with a dot and a top-level domain, so a Traefik name such as `auth-node@docker` does not count. A token is a JWT: three base64url parts with dots between them, the first part starting with `eyJ`. |
| A log field named like a secret | `sensitive_attribute_keys` | High | A field is called `password`, `token`, `secret`, `api_key`, or the like. The name alone is a leak, whatever the value holds. |
| Noisy log line | `noisy_logs` | Medium | One log line dominates the volume. It costs money and hides the lines that matter. |
| Logs with no fields | `unstructured_logs` | Medium | The records carry no attributes. The message is the only place a value can be, so you can only do a text search. |
| Log records miss a field | `log_schema_drift` | Medium | Some records have no `request.id` (or another key) that the other records of the same message set. A query on that field misses them. |
| Values inside the log message | `interpolated_log_messages` | Medium | Almost every message is different, so the values sit in the text instead of in fields. You cannot group the records or count an event. |
| Very large log records | `oversized_log_bodies` | Medium | A message holds a payload or a stack trace. Large records cost storage and are slow to read. |
| Failure text below WARN severity | `severity_mismatch` | Medium | A record reads like a failure but sits at INFO. A severity filter and an alert rule both miss it. |
| Logs with no service name | `missing_service_name` | Medium | The records set no `service.name`, so you cannot filter them by service. |
| Service writes too many logs | `chatty_service` | Medium | One service writes more than 600 records a minute. |
| Logs point at a trace that is not here | `orphan_log_trace` | Medium | The record names a trace ID with no matching span, so the link leads nowhere. |
| The same record arrives twice | `duplicate_records` | Medium | The same message, severity, and timestamp arrive more than once: two exporters, or one batch received twice. Every count is double, and you pay for both copies. |
| Records stamped in the future | `clock_skew` | Medium | Records carry a timestamp more than five minutes ahead of the clock. A chart lies about them, and a trace cannot line up with its logs. |
| Exceptions logged without a stack trace | `exception_without_stacktrace` | Medium | The record names an exception type and carries no `exception.stacktrace`. It says what failed and not where. |
| Unknown environment | `unknown_environment` | Medium | Records carry a `deployment.environment.name` (or `deployment.environment`) tag that matches no environment of the workspace. Loggerhead kept them in Local so nothing was lost. The finding offers "Create environment" with the tag filled in. It applies to logs, metrics, and traces, and it has no row on the Rules tab of the Linting page. |
| Mostly low-signal logs | `useless_logs` | Low | Most records are DEBUG or TRACE with no fields. They add little in production. |

### Your log shape

Two checks read the shape of your logs. **Logs with no fields** wants records that carry attributes at all.
**Log records miss a field** wants the records of one service to keep one shape.

You can declare that shape. Open the **Rules** tab of the **Linting** page, expand **Log shape drift**, and list the attribute keys
every record must carry in **Required fields**:

```
service.name, env, request.id
```

Leave the field blank and Loggerhead works the shape out on its own, for each message. A key that 90% or more of
the records of one message carry counts as part of that message's shape, and the records that miss it are the
drift. The numbers and ids in a message do not count, so `Batch 3 of 15 complete` and `Batch 4 of 15 complete` are
one message, and a start-up message is not held to the shape of the busiest message. So a message that sets
`request.id` on 190 of 200 records gets one finding that names the 10 records that do not.

## The trace checks

| Finding | Rule | Impact | What it means |
| --- | --- | --- | --- |
| Broken traces (the parent span is missing) | `broken_trace` | High | A span names a parent that never arrived. The trace breaks there, so you cannot see the request that started the work. |
| Spans with no operation name | `missing_span_name` | High | A span has a blank name, or a placeholder such as `unknown`. One such span is enough. The name is the only field that says what the span measured, so nothing can group these spans. |
| Span names hold identifiers | `span_name_cardinality` | High | Almost every span has a different name, such as `GET /search?q=shoes`. No platform can group these spans or chart a latency. |
| Sustained error rate with no alert | `suggested_alert` | High | A service has kept an error rate up with no alert rule on it. |
| Failed spans that read as OK | `error_span_without_status` | High | The span carries a 5xx status, an exception, or an error flag in its attributes, but its status is not ERROR. The error rate and every alert on it miss these spans. |
| Spans with an impossible duration | `invalid_span_duration` | Medium | A span ends at or before it starts, or it runs for hours. Latency figures over these spans are wrong. |
| Background call traced as one-span traces | `root_client_spans` | Medium | A poller or a retry loop starts a new trace on every call. Each trace holds one client span and no request. |
| The same call in a loop | `repeated_sibling_spans` | Medium | One parent span holds the same span name twenty times over, in trace after trace. That is the N+1 query: one call per item instead of one call for all of them. |
| Traces with hundreds of spans | `oversized_trace` | Medium | One trace holds 500 spans or more. It is slow to open, and it is usually a loop that makes one span per item. The finding names the root service and the root operation, the number of services the trace crosses, and the span names that repeat most. |
| A root operation starts too many traces | `chatty_traces` | Medium | One root operation, such as a health check, a poller, or a busy endpoint, starts more than 60 traces a minute, and 90 % or more of them have no error and end within 1 second. Each trace tells you the same thing, and together they cost storage and money. The finding gives the traces and spans per minute, and recommends a tail sampler at the collector that keeps the errors and the slow traces. |
| Spans with no kind | `missing_span_kind` | Low | The spans leave the kind unset, so a call out reads the same as a request served, and the collector map has to guess. |

## The metric checks

| Finding | Rule | Impact | What it means |
| --- | --- | --- | --- |
| High-cardinality label | `high_cardinality_label` | High | A label has more values than the limit, 50 by default, such as a raw URL path. Every value is a new series, and every platform charges for each one. |
| Too many series on one metric | `series_explosion` | High | One metric carries a thousand distinct label sets. Every combination is one series, and the count grows with every new value. This counts the product of the labels, which the single-label check misses. |
| Flatlined metric | `broken_metric` | Medium | A metric is always zero. The instrument may be wired wrong. A counter at zero is not flagged: no events occurred. A gauge whose name says errors, failures, pending items, or waits is Low, because zero is its healthy value. |
| Stale metric | `stale_metric` | Medium | A metric has no recent data points. The exporter may have stopped. |
| Counter goes down | `counter_decrease` | Medium | A cumulative counter falls between data points, so every rate over it is wrong. Only a monotonic sum counts: an UpDownCounter such as `go.memory.used` goes down by design. |
| Metric points have no service name | `unattributed_metric` | Medium | The points carry no `service.name` label, so two services write into one series. A metric Loggerhead scraped itself names its pull source instead, and does not count. |
| Metric never changes | `constant_metric` | Low | The series reported one value and never moved. It is a constant, not a measurement. A counter that does not change is not flagged: no events occurred. |
| Duplicate metrics | `duplicate_metrics` | Low | Two metric names carry the same series. Keep one. |
| Metric names break the convention | `metric_naming` | Low | A name has capitals or a space, or it joins two namespaces with an underscore where the other names use a dot. An underscore inside one component is correct OpenTelemetry style, as in `db.client.connection.pending_requests`. |
| Metrics declare no unit | `missing_metric_unit` | Low | The exporter sent no unit, so a chart cannot label its axis and every conversion is a guess. A metric Loggerhead scraped from a Prometheus endpoint cannot declare a unit and does not count. |
| Metric names disagree with their type | `metric_type_mismatch` | Low | A gauge is named like a running total (`_total`, `_count`), or a counter is named like a level (`_current`, `_percent`, `_ratio`, `_utilization`, `_usage`). The name tells the reader the wrong thing. |
| Points too close together | `too_frequent_datapoints` | Low | A series exports a point more often than every five seconds. The database fills with points a chart never draws apart. |

## The checks over every signal

| Finding | Rule | Impact | What it means |
| --- | --- | --- | --- |
| A service sends one signal only | `signal_coverage_gap` | Low | A service writes logs but no spans, or spans but no logs. You see half of what it does. Metrics are left out, because many apps emit none on purpose. |
| A service does not say which release it is | `missing_resource_attributes` | Low | The records carry no `service.version`, `deployment.environment.name` (or the older `deployment.environment`), or `host.name`, so one release or one deployment cannot be told from another. The receiver carries these three resource attributes onto every log record and span. |
| Renamed attribute keys | `deprecated_attributes` | Low | The records use `http.method`, `db.statement`, or another key OpenTelemetry has renamed. A new dashboard or tool reads the new names only. The recommendation names each replacement. |

## When it runs

The engine scans the last hour again every 30 seconds, and when you open the Linting page. A finding keeps
its identity across scans, so the page does not flicker.

## What happens to a finding

A finding is **open** while the rule fires. When the rule stops firing, the finding moves to **Recently resolved**
with a reason, and Loggerhead keeps it there for 24 hours:

- **Fixed**: the window still holds fresh data from the service, and that data no longer trips the rule.
- **Data aged out**: the window no longer holds those records, so nothing is known either way.

You can act on a finding from its card menu:

- **Mark as fixed** hides the finding now. Mark it after the fixed code runs. A rule reads the last hour, so the
  records from before the fix would keep the finding open for up to an hour. Loggerhead therefore checks only the
  data after the fix: when clean new data arrives, the finding moves to **Recently resolved** with the reason
  **Fixed**, usually within a minute. When the new data still trips the rule, the finding opens again. With no new
  data, it stays hidden. Your coding agent can do the same with the MCP tool `mark_finding_fixed`.
- **Ignore this finding** hides it for good. **Ignore for 7 days** hides it until then. The **Ignored** tab lists
  them, and **Take back** returns one. The Rules tab of the Linting page shows how many findings of each rule you ignored.
- **Tune the thresholds** opens the Rules tab of the Linting page at the rule. **Turn the rule off** switches the rule off for
  this environment, and every finding it made leaves at the next scan.

## AI analysis

The lint rules above are deterministic: each rule tests one condition, and you set its thresholds. AI analysis is a
second, separate check. It sends one large prompt with your telemetry to a model, and the model looks for the
problems that no single rule can describe:

- The same log line written from two places in the code.
- Traces that must line up, such as the spans of one request across two services, and do not.
- One root cause behind findings in many services.
- Which problem to fix first.

The model works through a fixed list of checks. Each AI finding names the check that found it.

| Check | What the model looks for |
| --- | --- |
| L1 One message, two places | The same message comes from two places in the code, or from two services. |
| L2 Two messages, one event | Different words for one event, so a search finds only one. |
| L3 A message you cannot act on | Text such as "Error occurred" or "done". |
| L4 Severity too high | WARN or ERROR on a normal event, such as a cache miss. |
| L5 One failure logged at each layer | The same error is logged at each level of the call stack. |
| L6 Logs that copy a span | "Start" and "end" lines that give the same data as a span. |
| L7 Personal data in the text | An email address, a name, or an address in a message. |
| L8 One service logs differently | One service writes a trace dump, a JSON object, or an unresolved format pattern as the message. |
| L9 A log that should be a metric | A log line that repeats and only counts an event or reports a number, such as a health check or "queue depth 42". The model suggests the metric: its name, type, unit, and attributes. |
| T1 Context lost between services | A call leaves one service, and the next service starts a new trace. |
| T2 Span name and kind disagree | For example, a SERVER span named like a database query. |
| T3 Time with no span | A slow span has a large part with no child span. |
| T4 Own keys for standard fields | A custom key where OpenTelemetry has a standard key. |
| T5 Traces worth sampling | Many healthy, fast traces of low-value work, such as a health check or a poller. The model says if the work needs a trace at all, and suggests a drop rule or a sample rate. |
| M1 Name, unit, and type disagree | For example, `latency_seconds` with the unit `ms`. |
| M2 A metric that copies the spans | A duration metric that gives the same data as the spans. |
| M3 A label that belongs on the resource | A host or version label on each data point. |
| X1 One service, many names | `checkout` and `checkout-svc` are one service. |
| X2 One key, many names | `user_id` and `userId` hold the same value. |
| X3 Signals do not agree on errors | The logs show errors, but the spans are OK. |
| X4 What to fix first | Groups the findings by cause and ranks them. |

Apple Intelligence has a small context. It gets a shorter sample and does L1, L3, L4, L8, X1, and X4 only.

You choose the window that the model reads: the last hour, or the last 24 hours. Press **Run AI analysis** on the
**AI analysis** page. The AI findings carry an AI badge.

AI analysis uses Apple Intelligence first. On a Mac that supports it, the model runs on the Mac with no key, and
nothing leaves the device. For a stronger model, add your own AI provider key in **Settings › AI**. You can
use an Anthropic-compatible or an OpenAI-compatible API, for example OpenRouter. The key stays in the macOS Keychain.
AI analysis is a Pro feature. See [Set up an AI provider](/docs/telemetry-linting/ai-providers/).

Before a cloud model gets your telemetry, Loggerhead masks the values in the log lines, the span names, and the text of
the lint findings. A mask replaces each match with a placeholder, so the model sees the shape of a line and not the
data in it:

| Mask | What it replaces | Example |
| --- | --- | --- |
| `<email>` | Email addresses | `sent to ann@example.com` → `sent to <email>` |
| `<jwt>` | JSON Web Tokens | `token eyJhb….eyJzd….Sfl` → `token <jwt>` |
| `<uuid>` | UUIDs | `order 9f1c2e4a-77b0-…` → `order <uuid>` |
| `<ip>` | IPv4 addresses | `from 10.0.12.7` → `from <ip>` |
| `<hex>` | Hex strings of 16 characters or more | `key 3fa85f6457174562b3fc…` → `key <hex>` |
| `<n>` | Numbers of 4 digits or more | `user 4821 got status 503` → `user <n> got status 503` |

Log lines that are the same after the masks count as one line. Add your own masks, for example for an account
number, in **Settings › AI › Masks for cloud models**. That card also has a field where you can type a log line and see
what a cloud model gets. Apple Intelligence runs on the Mac, so it gets the text as it is.