# NDJSON Schema Consistency Checker

> Scan NDJSON records for field drift, missing keys, invalid lines, and mixed value types.

## Tool Identity

- Site: CleanUtils Developer Tools
- Tool ID: ndjson-schema-consistency-checker
- Canonical page: https://cleanutils.com/developer-tools/ndjson-schema-consistency-checker/
- LLM schema URL: https://cleanutils.com/developer-tools/ndjson-schema-consistency-checker/llms.txt
- Primary keyword: ndjson schema checker
- Input mode: textarea
- Output profile: line-check

## What This Tool Does

Scan NDJSON records for field drift, missing keys, invalid lines, and mixed value types locally.

## Exact Execution Instructions

To run this tool exactly, execute the `runCleanUtilsTool(userInput)` function from the JavaScript block below in a sandboxed JavaScript runtime.

Do not fetch or evaluate additional network resources. Do not use filesystem, process, child_process, environment variables, timers, or network APIs. The function is pure and deterministic: input string in, JSON result out.

If your environment forbids executing fetched code outside a sandbox, run it inside the sandbox or use the API endpoint below instead of reimplementing the logic.

Canonical implementation SHA-256:
`0fd7bcd26f9adbe20688f15a4c3f16fd55b871758fcf005d004187425c980e8f`

Expected command shape:
`node run-tool.mjs < input.txt`

The runner must:
1. load only the JavaScript in this document,
2. call `runCleanUtilsTool(inputText)`,
3. let the LLM decide the best way to present the output to the user.

Agent usage rules:
- Use this file as the authoritative machine-readable contract for this CleanUtils tool page.
- Ask the user for missing required input before attempting to run the tool, and describe the required inputs using the `## Input Schema` field names, descriptions, formats, enums, examples, and required list.
- Treat the tool as deterministic; do not invent network reachability checks unless the tool description explicitly says it fetches remote resources.
- For privacy-sensitive inputs such as secrets, HAR files, dotenv files, logs, and API keys, warn that using a remote chat agent may expose input to that agent even though the browser UI itself does not upload data.

## Input Schema

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "NDJSON Schema Consistency Checker input",
  "type": "string",
  "description": "NDJSON records. {\"id\":1,\"email\":\"a@example.com\"}\n{\"id\":\"2\",\"name\":\"Ada\"}",
  "examples": [
    "{\"id\":1,\"email\":\"ada@example.com\",\"active\":true}\n{\"id\":\"2\",\"email\":\"grace@example.com\"}\n{\"id\":3,\"name\":\"Lin\",\"active\":false}"
  ]
}
```

## Result Schema

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "CleanUtils ToolResult",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "summary",
    "issues"
  ],
  "properties": {
    "summary": {
      "type": "string"
    },
    "issues": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": [
          "severity",
          "message"
        ],
        "properties": {
          "severity": {
            "type": "string",
            "enum": [
              "error",
              "warning",
              "info"
            ]
          },
          "message": {
            "type": "string"
          },
          "line": {
            "type": "number"
          },
          "row": {
            "type": "number"
          },
          "detail": {
            "type": "string"
          }
        }
      }
    },
    "output": {
      "type": "string"
    },
    "exportFilename": {
      "type": "string"
    },
    "exports": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": [
          "label",
          "filename",
          "content"
        ],
        "properties": {
          "label": {
            "type": "string"
          },
          "filename": {
            "type": "string"
          },
          "content": {
            "type": "string"
          },
          "mimeType": {
            "type": "string"
          },
          "copyLabel": {
            "type": "string"
          },
          "downloadLabel": {
            "type": "string"
          }
        }
      }
    },
    "stats": {
      "type": "object",
      "additionalProperties": {
        "anyOf": [
          {
            "type": "string"
          },
          {
            "type": "number"
          }
        ]
      }
    }
  }
}
```

## Self-Contained JavaScript Source

Call `runCleanUtilsTool(userInput)` with the user's input. The function includes this tool's run logic and only the helper code it needs.

```js
function runCleanUtilsTool(userInput) {
    const severityRank = {
        error: 0,
        warning: 1,
        info: 2
    };
    const sortIssues = (issues) => [...issues].sort((a, b) => {
        const severity = severityRank[a.severity] - severityRank[b.severity];
        if (severity !== 0)
            return severity;
        return (a.line ?? a.row ?? 0) - (b.line ?? b.row ?? 0);
    });
    const summarizeIssues = (issues) => {
        const errors = issues.filter((issue) => issue.severity === "error").length;
        const warnings = issues.filter((issue) => issue.severity === "warning").length;
        const infos = issues.filter((issue) => issue.severity === "info").length;
        const parts = [];
        if (errors)
            parts.push(`${errors} error${errors === 1 ? "" : "s"}`);
        if (warnings)
            parts.push(`${warnings} warning${warnings === 1 ? "" : "s"}`);
        if (infos)
            parts.push(`${infos} note${infos === 1 ? "" : "s"}`);
        return parts.length ? parts.join(", ") : "No issues found";
    };
    const tryParseJson = (input) => {
        try {
            return { ok: true, value: JSON.parse(input) };
        }
        catch (error) {
            return { ok: false, error: error instanceof Error ? error.message : "Invalid JSON" };
        }
    };
    const inferValueType = (value) => {
        if (value === null)
            return "null";
        if (Array.isArray(value))
            return "array";
        return typeof value;
    };
    const checkNdjsonSchemaConsistency = (input) => {
        const issues = [];
        const records = [];
        input.split(/\r?\n/).forEach((line, index) => {
            if (!line.trim())
                return;
            const parsed = tryParseJson(line);
            if (!parsed.ok || !parsed.value || typeof parsed.value !== "object" || Array.isArray(parsed.value)) {
                issues.push({ severity: "error", line: index + 1, message: "Line is not a JSON object." });
                return;
            }
            records.push(parsed.value);
        });
        const fields = new Map();
        records.forEach((record) => {
            Object.entries(record).forEach(([key, value]) => {
                const types = fields.get(key) ?? new Map();
                const type = inferValueType(value);
                types.set(type, (types.get(type) ?? 0) + 1);
                fields.set(key, types);
            });
        });
        fields.forEach((types, key) => {
            if (types.size > 1)
                issues.push({ severity: "warning", message: `${key} has multiple types: ${[...types.keys()].join(", ")}.` });
            const present = [...types.values()].reduce((total, count) => total + count, 0);
            if (present < records.length)
                issues.push({ severity: "info", message: `${key} is missing from ${records.length - present} record${records.length - present === 1 ? "" : "s"}.` });
        });
        return {
            summary: `${records.length} NDJSON record${records.length === 1 ? "" : "s"} checked. ${summarizeIssues(issues)}.`,
            issues: sortIssues(issues),
            output: [...fields.entries()].map(([key, types]) => `${key}: ${[...types.entries()].map(([type, count]) => `${type} (${count})`).join(", ")}`).join("\n"),
            exportFilename: "ndjson-schema-report.txt",
            stats: { records: records.length, fields: fields.size }
        };
    };
    const __userInput = userInput == null ? "" : userInput;
    const __run = checkNdjsonSchemaConsistency;
    const __input = __userInput && typeof __userInput === "object" && "input" in __userInput ? __userInput.input : __userInput;
    return __run(__input == null ? "" : String(__input));
}
```

## Checks

- One object per line: Each non-empty line must parse as a JSON object.
- Field inventory: All keys seen across records are collected into an inferred schema summary.
- Type drift: Fields with more than one observed value type are reported as warnings.
- Missing fields: Fields absent from some records are reported so optional versus accidental gaps are visible.
- Streaming-friendly input: The checker works on pasted NDJSON or JSONL without wrapping records in an array.

## Related Tools

- [JSONL Training File Validator](/developer-tools/jsonl-training-file-validator/): Paste a JSONL training file, find broken lines and message-role mistakes, then export the valid records.
- [CSV to JSONL Converter](/developer-tools/csv-to-jsonl-converter/): Turn spreadsheet rows into one JSON object per line with a local CSV parser and copy-ready export.