feat(harness): add readonly mysql tool
This commit is contained in:
zhuyongxin
@@ -0,0 +1,84 @@
|
||||
# Design: single-react-mysql-readonly-tool
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
MysqlToolRequest + ToolCallRequestEnvelope
|
||||
|
|
||||
v
|
||||
MysqlToolAdapter
|
||||
- parse typed request
|
||||
- MysqlSqlValidator -> MysqlQueryPlan
|
||||
- ToolBoundary.execute
|
||||
|
|
||||
+--> MysqlReadOnlyExecutor (JDBC or isolated fake)
|
||||
| - read-only connection
|
||||
| - PreparedStatement params
|
||||
| - timeout/max rows/cancel
|
||||
|
|
||||
+--> MysqlResultProjector
|
||||
- bounded cells/rows/bytes
|
||||
- sensitive column redaction
|
||||
- MysqlToolResult JSON
|
||||
```
|
||||
|
||||
The adapter validates the typed request before entering ToolBoundary. Once validation succeeds, the executor and projector run inside the existing canonical lifecycle. Validation failures have no canonical record because no database call is authorized; execution/projection failures after `PROJECTING` become canonical `ERROR` records.
|
||||
|
||||
## Configuration model
|
||||
|
||||
`MysqlDataSourceDefinition` is an immutable logical definition:
|
||||
|
||||
- logical ID
|
||||
- JDBC URL, username and password supplied by configuration/Secret
|
||||
- default schema
|
||||
- exact schema/table/column allowlist
|
||||
- query timeout seconds, max rows, max cell characters and max result bytes
|
||||
|
||||
The Agent sees only the logical ID. The stage does not create a dynamic datasource registry or reuse the application persistence datasource. A caller supplies a `Map<String, DataSource>` to the JDBC executor, allowing isolated test data sources and later production wiring.
|
||||
|
||||
## SQL validator
|
||||
|
||||
`MysqlSqlValidator` uses `CCJSqlParserUtil.parseStatements` and rejects unless there is exactly one `Select` with a `PlainSelect` body and no CTE/compound body. It rejects `SubSelect`, `SetOperationList`, `ValuesStatement`, `ForUpdate`, metadata statements, wildcard projection except `COUNT(*)`, unsupported joins, unsupported functions, and unknown table/from items.
|
||||
|
||||
The validator builds an alias map for every base table and checks every table/column reference in select items, joins, where, group by, having and order by. Unqualified columns must resolve to exactly one allowlisted table; qualified columns must resolve through a declared alias/table and exact allowlist. It counts `JdbcParameter` nodes and requires an exact match with `params`.
|
||||
|
||||
The function allowlist starts with `COUNT`, `SUM`, `AVG`, `MIN`, and `MAX`. `COUNT(*)` is represented as a function-level exception; a projection `*` or `table.*` is rejected. Any parser exception or policy ambiguity produces a `MysqlSecurityException` before execution.
|
||||
|
||||
## JDBC executor
|
||||
|
||||
`JdbcMysqlReadOnlyExecutor` resolves the validated logical ID to a caller-supplied DataSource, opens a connection, calls `setReadOnly(true)`, prepares the validated SQL, binds parameters in order, applies `setQueryTimeout` and `setMaxRows`, and reads column labels plus structured values. It checks `RunContext` cancellation/deadline while iterating and cancels/closes the statement on abort.
|
||||
|
||||
The executor returns a Harness-only raw record. It never returns a JDBC connection, SQL exception details, credentials or stack traces to the Agent. JDBC errors are mapped by ToolBoundary to a stable safe error.
|
||||
|
||||
## Result projection
|
||||
|
||||
`MysqlResultProjector` parses only the executor raw record and emits `MysqlToolResult`:
|
||||
|
||||
- immutable ordered columns
|
||||
- bounded ordered rows
|
||||
- cell values converted to JSON-safe scalar/text values
|
||||
- redaction for sensitive column names (`password`, `token`, `secret`, `api_key`, etc.)
|
||||
- `returned_count` equal to projected rows
|
||||
- `truncated=true` for row/cell/byte removal
|
||||
- `NO_EVIDENCE` for a successful empty result
|
||||
|
||||
Projection never exposes raw JDBC metadata, connection coordinates, internal error messages or the unbounded raw result.
|
||||
|
||||
## Script safety cleanup
|
||||
|
||||
`scripts/query_mysql.py` will require `SUPERBIZ_MYSQL_HOST`, `SUPERBIZ_MYSQL_PORT`, `SUPERBIZ_MYSQL_USERNAME`, `SUPERBIZ_MYSQL_PASSWORD` and `SUPERBIZ_MYSQL_DATABASE` (with no external defaults), accept only one SELECT statement, and reject all non-SELECT/metadata/write SQL before opening a connection. It will call `rollback`/close defensively and remove the interactive write path.
|
||||
|
||||
## Interface and compatibility impact
|
||||
|
||||
- L2 internal Harness classes plus the JSqlParser build dependency.
|
||||
- No public HTTP/SSE/Chat contract changes.
|
||||
- No changes to legacy tools, recorder, JPA entities, or current application datasource.
|
||||
- The main capability spec will be synchronized after archive.
|
||||
|
||||
## Risks and mitigations
|
||||
|
||||
- JSqlParser AST API drift: lock version 4.6 and run parser security fixtures.
|
||||
- Alias/column ambiguity: fail closed rather than guessing.
|
||||
- JDBC cancellation is driver-dependent: check Run state before/while iteration and call `Statement.cancel()` on abort.
|
||||
- Sensitive result values: redact by column name before JSON serialization and keep raw only in canonical Harness storage.
|
||||
- A malicious query can still be expensive within SELECT: timeout, max rows, read-only connection and configured byte limits remain mandatory.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Proposal: single-react-mysql-readonly-tool
|
||||
|
||||
## Problem
|
||||
|
||||
阶段 1 已冻结 `query_mysql` 的逻辑请求和有界结果契约，但当前仓库没有一个能把 SQL 安全地转换为可执行查询的实现。直接把 Agent 生成的 SQL 交给 JDBC 会允许写操作、元数据探测、通配符泄漏、allowlist 绕过和无界结果。
|
||||
|
||||
## Proposed change
|
||||
|
||||
- 引入 JSqlParser 4.6，解析单条 SQL 并对保守的 SELECT 子集执行 fail-closed AST 校验。
|
||||
- 增加逻辑数据源定义和静态 `schema -> table -> column` 精确 allowlist；Agent 只能传逻辑数据源 ID。
|
||||
- 增加参数绑定、占位符数量校验、函数 allowlist、显式列校验、JOIN/过滤/分组/排序字段校验和受控 LIMIT。
|
||||
- 增加 JDBC 只读执行器：只使用配置映射的 DataSource，设置 `readOnly`、`PreparedStatement`、`setMaxRows`、查询超时，并在 Run 取消时中止执行。
|
||||
- 增加 MySQL 结果 projector：限制行数、单元格和总 UTF-8 字节，脱敏敏感列，产生 `MysqlToolResult` 和 `NO_EVIDENCE`。
|
||||
- 通过阶段 3A `ToolBoundary` 接入 canonical invocation store、Run budget、生命周期、`evidence_status` 和框架 `tool_call_id`。
|
||||
- 把 `scripts/query_mysql.py` 收敛为仅允许 SELECT/受控 SHOW 的只读查询脚本，移除默认外部连接参数和 commit 分支；凭据与连接信息全部从环境变量读取。
|
||||
|
||||
## Scope
|
||||
|
||||
### In scope
|
||||
|
||||
- SQL AST validator and validated query plan.
|
||||
- Static logical data-source/allowlist model.
|
||||
- JDBC read-only executor abstraction and implementation.
|
||||
- MySQL result projector and ToolBoundary adapter.
|
||||
- Security, truncation, timeout/cancellation, no-evidence and boundary tests.
|
||||
- Query script safety cleanup required by the ISS-014 security prerequisite.
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Diagnosis Agent cutover or public Chat/AIOps/SSE changes.
|
||||
- Querying the application's own persistence database through the Agent-facing Tool.
|
||||
- Metadata discovery (`SHOW TABLES`, `SHOW COLUMNS`, `DESCRIBE`, `information_schema`).
|
||||
- Tenant/row-level authorization, dynamic allowlists, arbitrary SQL functions, real production business datasource provisioning.
|
||||
|
||||
## Frozen SQL subset
|
||||
|
||||
- One `SELECT` statement only.
|
||||
- Explicit projection columns; `COUNT(*)` is the only star exception.
|
||||
- `INNER JOIN` and `LEFT JOIN`, normal predicates, `GROUP BY`, `HAVING`, `ORDER BY` and parameter placeholders.
|
||||
- No `WITH`, subquery, `UNION`, window function, `CROSS JOIN`, write statement, metadata query, transaction control, dangerous function, or unknown AST node.
|
||||
|
||||
## Constraints and risks
|
||||
|
||||
- Parser acceptance is not authorization: every table and column used by projection, predicates, joins, grouping and ordering must pass the independent allowlist.
|
||||
- Unknown data source, schema, table, column, function, placeholder mismatch or parser failure fails closed before JDBC execution.
|
||||
- JDBC row/cell/byte limits are enforced in addition to ToolBoundary limits; oversized results are projected with `truncated=true` or become a safe error when no valid bounded result can be produced.
|
||||
- The adapter is not wired into the existing public runtime in this stage; later Diagnosis Agent work will select it through the internal Harness application use case.
|
||||
|
||||
## Acceptance direction
|
||||
|
||||
- Valid explicit-column SELECT and `COUNT(*)` pass AST and allowlist validation.
|
||||
- Write statements, metadata, wildcard projection, nested/compound queries, unknown AST, allowlist bypass and placeholder mismatch are rejected without executor invocation.
|
||||
- JDBC executor uses read-only prepared statements, max rows, timeout and cancellation.
|
||||
- Agent receives only bounded `MysqlToolResult`; raw JDBC rows remain Harness-only canonical data.
|
||||
+96
@@ -0,0 +1,96 @@
|
||||
# mysql-readonly-tool Specification
|
||||
|
||||
## Purpose
|
||||
|
||||
Define a fail-closed, parameterized, read-only MySQL evidence Tool that reuses the stage 3A ToolBoundary and exposes only bounded ACI results.
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: SQL validation SHALL fail closed on a conservative SELECT subset
|
||||
|
||||
The Tool SHALL parse exactly one SQL statement with JSqlParser and SHALL accept only a single `SELECT` with explicit projection columns, supported predicates/grouping/ordering, `INNER JOIN` or `LEFT JOIN`, parameter placeholders and allowlisted functions. It SHALL reject writes, CTEs, subqueries, set operations, wildcard projections except `COUNT(*)`, metadata discovery, unsupported joins/functions, `FOR UPDATE`, multiple statements and unknown/ambiguous AST structures.
|
||||
|
||||
#### Scenario: Valid explicit-column SELECT
|
||||
|
||||
- WHEN a query selects allowlisted columns from an allowlisted table with matching `?` parameters
|
||||
- THEN validation returns a query plan and the executor may be invoked
|
||||
|
||||
#### Scenario: COUNT star exception
|
||||
|
||||
- WHEN a query uses `SELECT COUNT(*)` against an allowlisted table
|
||||
- THEN validation succeeds without treating the projection as an unrestricted wildcard
|
||||
|
||||
#### Scenario: Security query is rejected
|
||||
|
||||
- WHEN SQL contains INSERT/UPDATE/DELETE, `WITH`, a subquery, `UNION`, `SELECT *`, metadata discovery, `FOR UPDATE`, a dangerous function, multiple statements or an unknown AST node
|
||||
- THEN validation fails before executor invocation
|
||||
|
||||
### Requirement: Data-source and identifier authorization SHALL use exact independent allowlists
|
||||
|
||||
The Tool SHALL accept only a logical `data_source` ID and SHALL authorize every schema, table and column used in projection, join, predicate, grouping and ordering against the configured exact allowlist. It SHALL reject unknown data sources, schemas, tables, columns, aliases and ambiguous unqualified columns. Agent input SHALL NOT provide JDBC coordinates or authorization controls.
|
||||
|
||||
#### Scenario: Allowlisted query
|
||||
|
||||
- WHEN every referenced identifier resolves to one configured schema/table/column
|
||||
- THEN validation succeeds and retains the logical data-source ID only
|
||||
|
||||
#### Scenario: Allowlist bypass
|
||||
|
||||
- WHEN a query references an unconfigured table, column, schema, alias or ambiguous unqualified column
|
||||
- THEN validation fails closed and the executor is not called
|
||||
|
||||
### Requirement: Parameter binding and JDBC execution SHALL be read-only and bounded
|
||||
|
||||
The executor SHALL use a configured logical datasource, a read-only JDBC connection, `PreparedStatement` parameter binding, query timeout, max rows and Run cancellation/deadline checks. Placeholder count SHALL exactly match `params`. The executor SHALL not expose connection details or raw JDBC failures to the Agent.
|
||||
|
||||
#### Scenario: Bound read-only execution
|
||||
|
||||
- WHEN a validated plan has matching parameters and an active Run
|
||||
- THEN the executor binds values in order, sets read-only/timeout/max rows, and returns structured raw rows
|
||||
|
||||
#### Scenario: Timeout or cancellation
|
||||
|
||||
- WHEN the query exceeds its timeout or the Run is cancelled/deadline-expired
|
||||
- THEN the statement is cancelled/closed and ToolBoundary returns a safe error without an Agent result
|
||||
|
||||
### Requirement: MySQL projection SHALL be bounded and evidence-aware
|
||||
|
||||
The projector SHALL expose only ordered columns, bounded JSON-safe rows, returned count and truncation. It SHALL enforce row, cell and total UTF-8 limits, redact sensitive column values, return `NO_EVIDENCE` for a successful empty result, and never expose raw JDBC metadata or credentials.
|
||||
|
||||
#### Scenario: Bounded rows are projected
|
||||
|
||||
- WHEN the executor returns rows within configured limits
|
||||
- THEN the Agent receives `EVIDENCE_FOUND` with ordered columns and structured rows
|
||||
|
||||
#### Scenario: Result exceeds a limit
|
||||
|
||||
- WHEN row count, cell length or total bytes exceed a configured bound
|
||||
- THEN the Agent receives valid bounded JSON with `truncated=true` and no oversized value
|
||||
|
||||
#### Scenario: Empty result
|
||||
|
||||
- WHEN a valid query returns zero rows
|
||||
- THEN the boundary reaches `READY` with `evidence_status=NO_EVIDENCE` and an empty row list
|
||||
|
||||
### Requirement: MySQL Tool SHALL reuse canonical Harness ownership
|
||||
|
||||
The adapter SHALL pass the exact framework `tool_call_id` and RunContext through the existing ToolBoundary and canonical invocation store. It SHALL not create a second ID, use a parallel store, return raw SQL results, or modify legacy audit/public runtime paths.
|
||||
|
||||
#### Scenario: Successful adapter call
|
||||
|
||||
- WHEN a valid request passes validation and execution
|
||||
- THEN the canonical record contains request/raw/bounded result under the exact framework ID and the Agent receives only the bounded result
|
||||
|
||||
#### Scenario: Validation or boundary failure
|
||||
|
||||
- WHEN validation, authorization, duplicate, budget or lifecycle preflight fails
|
||||
- THEN no database call is made and the Agent receives a safe bounded error
|
||||
|
||||
### Requirement: The query helper script SHALL be read-only and secret-free by default
|
||||
|
||||
The repository query helper SHALL require connection values from environment variables, reject non-SELECT and metadata discovery SQL before connection, and SHALL NOT commit writes or expose hardcoded external connection defaults.
|
||||
|
||||
#### Scenario: Unsafe script query
|
||||
|
||||
- WHEN a caller passes a write, multi-statement, metadata or transaction command
|
||||
- THEN the script exits with a safe validation error without opening a connection
|
||||
@@ -0,0 +1,24 @@
|
||||
# Tasks: single-react-mysql-readonly-tool
|
||||
|
||||
## 1. Security prerequisite and dependency
|
||||
|
||||
- [x] 1.1 Add JSqlParser 4.6 and record the locked parser version.
|
||||
- [x] 1.2 Make `scripts/query_mysql.py` environment-only and read-only; remove commit and unsafe interactive paths.
|
||||
|
||||
## 2. SQL policy and allowlist model
|
||||
|
||||
- [x] 2.1 Implement immutable data-source/allowlist/limit models and validated query plan.
|
||||
- [x] 2.2 Implement JSqlParser single-SELECT validator, identifier resolution, function/wildcard/join policy and placeholder count checks.
|
||||
- [x] 2.3 Add parser and allowlist security fixtures for accepted/rejected SQL.
|
||||
|
||||
## 3. JDBC executor and projection
|
||||
|
||||
- [x] 3.1 Implement read-only PreparedStatement executor with timeout, max rows, Run cancellation and safe error mapping.
|
||||
- [x] 3.2 Implement `MysqlResultProjector` with row/cell/byte bounds, redaction, JSON-safe values and `NO_EVIDENCE`.
|
||||
- [x] 3.3 Add isolated executor/projector tests for valid rows, empty rows, truncation, sensitive columns and timeout/cancel behavior.
|
||||
|
||||
## 4. ToolBoundary adapter and verification
|
||||
|
||||
- [x] 4.1 Implement typed MySQL adapter through the existing ToolBoundary and canonical invocation store.
|
||||
- [x] 4.2 Add adapter tests proving exact framework ID, raw isolation, validation-before-execution and safe errors.
|
||||
- [x] 4.3 Run focused compile/tests, validate OpenSpec, update devflow evidence, archive and commit before stage 4.
|
||||
Reference in New Issue
Block a user