Skip to content

feat(plugin-hana): add native SAP HANA driver - #3169

Open
J2TeamNNL wants to merge 12 commits into
TableProApp:mainfrom
J2TeamNNL:codex/hana-driver
Open

J2TeamNNL wants to merge 12 commits into
TableProApp:mainfrom
J2TeamNNL:codex/hana-driver

Conversation

@J2TeamNNL

Copy link
Copy Markdown
Contributor

Summary

  • Add a downloadable SAP HANA driver plugin backed by SAP/go-hdb through a universal native C archive.
  • Support database username/password authentication, TablePro TLS modes, schema/table/view/column/index browsing, SQL execution, binary result cells, and cancellation.
  • Register the plugin in the chooser, project targets, CI build path, supported-driver documentation, and changelog.

Scope and compatibility

  • No SAP HDB Client, ODBC, SQLDBC, or local SAP installation is required.
  • Database is the initial schema; parameterized queries, SSO/LDAP/JWT/X.509, Structure editing, and advanced LOB handling remain intentionally unavailable.

Validation

  • go test -mod=readonly ./... and go vet -mod=readonly ./... in Native/HanaBridge
  • Universal archive build, required C ABI symbol check, and arm64/x86_64 consumer link checks
  • Swift parse, XcodeGen generation, plugin-manifest consistency, docs source/style checks, plist validation, and git diff --check
  • Diff secret scan; only intended password-field plumbing and a test-only fixture matched broad credential terms

Environment limitations

  • Full Xcode build, XCTest, plugin load, and signing verification are unavailable locally because the active developer directory contains Command Line Tools only.
  • Live SAP HANA connection, metadata browsing, query, and cancellation need a supplied HANA test endpoint and credentials.

Documentation

  • Added the SAP HANA database page and navigation.
  • Updated README tables, driver counts, and changelog.

Linked issue

Closes #1966

@datlechin

Copy link
Copy Markdown
Member

Thanks for the driver, @J2TeamNNL. It had never been built in Xcode, and a review found it could not compile, and that the bridge got most HANA values and the whole session lifecycle wrong. I pushed fixes on top of your commit. Your commit and your credit stay as they are.

What was wrong

  • Neither the app nor the plugin compiled. The CapabilityFlags arguments were out of order, fetchDatabases() and fetchDatabaseMetadata(_:) were missing, and a private quoteIdentifier collided with a protocol requirement.
  • The bridge formatted driver.Value by Go type alone. go-hdb returns every text type as raw bytes, DECIMAL as an internal struct, zone-less dates as UTC time.Time, and REAL as float64. So text arrived as hex, metadata browsing returned nothing, decimals printed a pointer, dates got a Z and floats printed 1e+06. Nil slices also marshalled as null, so every DML, DDL and empty result failed to decode.
  • Stop killed the session. Cancelling the context makes go-hdb run ALTER SYSTEM DISCONNECT SESSION (which needs SESSION ADMIN). The bridge pinned one sql.Conn for life and never replaced it, so every later call failed. The 300 s read deadline stayed in place and the query timeout did nothing. Connect could not be aborted, and tp_hana_cancel could race cgo.Handle.Delete and panic the app.
  • Grid saves always failed. Parameterized queries were rejected, but the dialect said ?.
  • Verify CA could never connect: its ServerName was empty with verification on. Both verify modes also required a CA file.
  • Routing by the first keyword lost the rows from (SELECT and DO BEGIN. EXPLAIN PLAN FOR returns nothing in HANA.

What changed

Go bridge (Native/HanaBridge), split by concern with a test file per concern

  • Sessions: ids live in a mutex-guarded registry (no cgo.Handle), and every export recovers panics. There is one pinned session per connection. A socket error or a failed cancel marks it lost, and every later call fails with "connection lost" so TablePro's own reconnect runs. It never reconnects silently.

  • Stop: ALTER SYSTEM CANCEL SESSION '<CURRENT_CONNECTION>' over a separate control connection, so the session survives. The next statement waits until the cancel settles, so a cancel never lands on the wrong statement. A statement still queued is dropped before it reaches the server. If the cancel is refused, the socket is closed. go-hdb's own statement contexts are never cancelled. The query timeout uses the same path, and SetTimeout(0) removes the 300 s deadline.

  • Connect: tp_hana_open then tp_hana_connect. A tracking dialer does the TLS handshake inside the connect context, so Cancel and the 30 s timeout both close the socket.

  • Routing: QueryContext by default (zero columns means no result set). INSERT/UPDATE/DELETE/UPSERT/REPLACE/MERGE go through ExecContext for the row count.

  • Values: formatted by HANA column type and ScanType:

    • exact decimals keeping their scale;
    • shortest round-trip floats;
    • DATE/TIME/SECONDDATE/TIMESTAMP in HANA's own text, with the empty date spelled 0000-00-00;
    • CESU-8 aware text;
    • LOBs capped at 64 MiB.

    Column types are reported as SQL names (DATE, TIMESTAMP, DECIMAL(p,s)), so the grid classifies them.

  • Parameters: really bound through prepare metadata. Dates are parsed, and more decimal places than the column scale are refused rather than rounded. Spatial values must be hex WKB, and OUT parameters are refused. Statements that bind a LOB run in a transaction, because HANA refuses LOB streaming in autocommit (error 596).

  • TLS: Verify CA checks the chain but not the hostname. Verify Identity uses the system roots when no CA file is set. SNI is always sent, and client certificates are passed through.

  • Explain: one bridge call runs EXPLAIN PLAN SET STATEMENT_NAME, reads SYS.EXPLAIN_PLAN_TABLE and always deletes the rows. The plan shows as an indented tree.

  • Errors: returned as structured JSON; Swift writes all user text.

Swift plugin

  • HanaConnection is rebuilt on the Dameng pattern: one serial queue, per-call op ids, task cancellation reaching the bridge, and an epoch fence between connect and disconnect.
  • Errors map to CancellationError, SSLHandshakeError or a localized HanaError that carries the HANA error code.
  • Catalog:
    • columns come from tables and views, with full types, primary key, identity and generated columns;
    • indexes, foreign keys, table metadata and bulk column fetch;
    • catalog-built DDL with quoted defaults, plus index and comment DDL for dumps;
    • CREATE VIEW headers.

App

  • The snapshot moved to PluginMetadataRegistry+HanaDefaults.swift, with the CapabilityFlags order fixed. Added DatabaseType.sapHana, Verify Identity on port 443, row-match exclusions for LOB and spatial columns, and the full HANA system schema list.
  • A new app-only grammar bit keeps DO BEGIN ... END; and nested blocks whole for SAP HANA only. The safety classifier reads the block's body.
  • The connection form no longer requires a CA file for engines that verify against the system trust store. That also fixes Kafka and SQL Server, which could not save Verify CA or Verify Identity without one.

CI, docs, licenses

  • The setup-go pin is now the real v6.5.0 SHA. go vet/go test and HanaDriverTests now run in macos-tests.yml.
  • The SAP HANA page is rewritten to docs/STYLE.md, and the engine lists on the other docs pages are updated.
  • go-hdb, golang.org/x/text and Go are in licenses.yml with their texts.
  • The plugin strings are in the catalog, and plans/ is removed.

Verified

  • go vet, go test -race: pass. The tests cover the registry, the op slot and cancel ordering, routing, value and parameter conversion (incl. empty-date round trips), TLS modes against a local TLS server with a generated CA, error classification and plan rendering.
  • scripts/build-hana.sh both: pass. All 8 CHana.h symbols are in both slices, with minos 13.0.
  • App build, AllPlugins build: pass.
  • HanaDriverTests: 87 pass against the real Go archive.
  • TableProTests suites owning the changed types: 129 + 163 + 362 cases pass.
  • Grammar package tests: 97 pass.
  • swiftlint --strict on every changed Swift file: 0 violations.
  • verify.sh docs: pass.

Not verified: anything that needs a live SAP HANA server. HANA Express is x86-only and needs about 16 GB, and there is no HANA Cloud instance here. Server-side behaviour rests on go-hdb v1.18.12's source and SAP's SQL reference. Before merging, it would help if someone with a HANA endpoint tries:

  • connect;
  • browse a schema;
  • edit a row;
  • Stop a long query, with and without SESSION ADMIN;
  • Explain, with OPTIMIZER ADMIN;
  • Show DDL on a table with string defaults.

# Conflicts:
#	CHANGELOG.md
#	TablePro/Core/Plugins/DatabaseType+Registry.swift
#	TablePro/Core/Plugins/PluginMetadataRegistry+RegistryDefaults.swift
#	TablePro/Core/Plugins/PluginMetadataRegistry.swift
#	TablePro/Resources/Localizable.xcstrings
#	TablePro/Views/ConnectionForm/Panes/SSLSections.swift
#	TablePro/Views/ConnectionForm/ViewModels/SSLPaneViewModel.swift
#	TableProTests/Core/Plugins/PluginMetadataRegistryCuratedCapabilityTests.swift
#	TableProTests/ViewModels/SSLPaneViewModelTests.swift
#	docs/connections/ssl.mdx
# Conflicts:
#	CHANGELOG.md
#	Packages/TableProCore/Sources/TableProSQLGrammar/SQLLexicalGrammar.swift
#	Packages/TableProCore/Sources/TableProSQLGrammar/SQLLexicalProfile.swift
#	Packages/TableProCore/Sources/TableProSQLGrammar/SQLLexicalReadings.swift
#	TablePro/Core/Utilities/SQL/QueryClassifier.swift
#	TableProTests/Plugins/SQLLexicalFeatureMappingTests.swift
@datlechin

Copy link
Copy Markdown
Member

Follow-up since my last comment. CI is green: macOS Tests, iOS Tests, Docs and Repo Hygiene.

Cancel and session hardening. A Codex adversarial review of the cancel design found seven more races. All are fixed, and each has a deterministic test:

  • Stop is bound to the query's own cancellation slot and delivered synchronously. A late Stop can no longer cancel the next statement or a health ping.
  • disconnect() closes the native session synchronously. Queued work checks its session again right before it calls the bridge, so a statement queued behind a disconnect never runs.
  • A watchdog that has already fired finishes before the next operation starts.
  • A Stop or timeout that does not settle within 30 s cuts the socket, so post-cancel cleanup (Rows.Close, Stmt.Close, Rollback) cannot hang.
  • Cleanup errors are no longer swallowed, and a failed commit or rollback drops the session.
  • A statement that succeeded while its session was lost still returns its rows, with sessionLost set, so the app reconnects.

The Go bridge is covered by go test -race, including a scripted database/sql driver. HanaConnection now reaches C through a small protocol, so its races are tested with a fake bridge: 109 HanaDriverTests. A third Codex adversarial pass approved the result.

Main merged in three times (#3146, #3168, #3170, #3164, #3171, v0.76.1).

CI-only fix. CI builds with Xcode 26.4.1 (Swift 6.3). Its region-isolation checker rejected a Task closure reading Self.configuration in a test, so that value is now read into a local first.

Still not verified against a live SAP HANA server; the checklist in my earlier comment still applies.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Database request: Hana (SAP)

2 participants