fix(ios): list PostgreSQL materialized views and read indexes with the plugin's catalog queries - #3074
Merged
Conversation
…e plugin's catalog queries
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
…uncate or Drop Table
…iews-indexes # Conflicts: # CHANGELOG.md # Plugins/PostgreSQLDriverPlugin/PostgreSQLPluginDriver.swift # Plugins/PostgreSQLDriverPlugin/PostgreSQLSchemaQueries.swift
…iews-indexes # Conflicts: # CHANGELOG.md # TableProMobile/TableProMobile/Localizable.xcstrings # TableProMobile/project.yml
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On iPhone and iPad, a PostgreSQL schema listed no materialized views, and Table Structure got indexes wrong: expression keys and expression-only indexes were dropped,
INCLUDEcolumns were reported as key columns, key order was lost, partial index predicates were lost, and every index was typedBTREE. PostgreSQL foreign tables were listed as plain tables with Truncate and Drop Table offered, and the server refuses both ("ft" is not a table).The iOS driver now runs the macOS plugin's own catalog reads for the table list, the index list and the materialized view columns, shared by file reference the way the foreign key read already is. Redshift on iOS runs the Mac Redshift driver's listing and its DISTKEY / SORTKEY read.
Root cause
The iOS
PostgreSQLDriverhand-wrote its own catalog SQL instead of sharing the plugin's audited reads:information_schema.tables, which has no row forrelkind = 'm'.pg_attributeonattnum = ANY(indkey). An expression key has attribute number 0, so it vanished;indkeyalso holds theINCLUDEcolumns after the key; the rows were ordered byattnum, not key position;pg_am.amnameandindpredwere never read.IndexInfohad nowhere to putINCLUDEcolumns or a predicate.information_schema.columnshas no rows for a materialized view either, so listing one alone would have left its Columns tab empty.TableProModels.TableInfo.TableKindhad no foreign table kind, soTableInfo(from:)sent bothFOREIGN TABLE(the shared listing) andFOREIGN(information_schema.tables) to itsdefault:arm,.table, which allows Truncate and Drop.The plugin's correct reads were tangled with plugin-only code (
PostgreSQLSchemaQueriesneedsColumnQueryShape, the sequence files and more), which is why iOS never shared them.What changed
Plugin (behaviour unchanged on the Mac, SQL proven byte-identical, see Measured):
PostgreSQLTableListing(new,nonisolated): the table listing moved out ofPostgreSQLSchemaQueriesas-is, plus the row decoder that used to sit inline inPostgreSQLPluginDriver.fetchTables.PostgreSQLMaterializedViewColumnSource(new,nonisolated): the matview column arm's type, nullability, ordinal and FROM / WHERE fragments. The ERROR: relation "pg_matviews"does not exist #1383 arm incolumnsQuerynow builds from it.RedshiftTableCatalog(new,nonisolated): the Redshift listing and thepg_table_defDISTKEY / SORTKEY read with its decoder, moved out ofRedshiftPluginDriver.PostgreSQLIndexQueries,PostgreSQLCatalogIndexDDL,PostgreSQLIndexRow,PostgreSQLTextArray,PostgreSQLCatalogBooleanandPostgreSQLCatalogPresenceare markednonisolatedand join the iOS target.scripts/ci/check-ios-shared-isolation.pypasses on all 32 shared sources.iOS:
PostgreSQLDrivertakesdatabaseType(asMySQLDriverdoes);IOSDriverFactorypasses it.PostgreSQLCatalogPresence.probeQuery. A failed probe means the optional catalogs are absent; there is no version-based fallback.PostgreSQLDriver+Catalog.swift(anonisolated extension, reading throughexecute(query:)so the actor stays private):PostgreSQLTableListingwith matviews and foreign tables gated by the probe, comments and partition awareness off; Redshift keepsRedshiftTableCatalog.listingQuery. Rows map throughTableInfo(from:), soMATERIALIZED VIEWbecomes.materializedView, which already withholds editing, Truncate and Drop.PostgreSQLIndexQueries.indexListwith.assumingModernWhenUnknown, decoded byPostgreSQLIndexRowandIndexInfo(from:). Redshift getsRedshiftTableCatalogand never receivespg_indexSQL.information_schemaarm plus a matview arm, both inside a derived table,ordinal_positionkept out of the result and named only byORDER BY cols.ordinal_position.PostgreSQLColumnReadSupportreplaces thereportsIdentityColumnscatch-all. The attempts run identity + matview arm, then no identity, then no matview arm, then neither, and the first success decides what is remembered, so a failing matview arm never turns identity columns off, and a failing identity projection never drops the matview arm.PostgreSQLDriver.swift:effectiveSchemaandloggerare internal,columnReadSupportis an internalnonisolated(unsafe)var,catalogPresenceisprivate(set).IndexInfo(TableProModels) gainsincludedColumnsandwhereClause, both defaulted;IndexInfo(from:)maps them. Structure showsINCLUDE (...)andWHERE ...as verbatim SQL under the key parts, and the type badge shows the real access method.TableKindgains.foreignTable.TableInfo(from:)maps bothFOREIGN TABLE(the spelling the Mac'sPluginTableKindDecoderreads) andFOREIGN. It lists under Tables, allows no Truncate and no Drop (the iOS list writes a literalDROP TABLE), and keeps row editing, as the Mac'sTableType.foreignTabledoes.TableKindPresentation: an exhaustive per-kind SF Symbol and VoiceOver kind, matching the Mac'sTableRowLogic(materialized viewsquare.stack.3d.up/ "Materialized View", foreign tablelink/ "Foreign Table"). Five new strings added to the iOS catalog.Measured
swiftcprobe and recorded 93 statements:fetchTablesfor two schemas (one holding'and\) across all 16 flag combinations plus the defaulted call,columnsQueryat server versions 0, 9.6, 10, 11, 12 and 17.11 for all tables,ordersandit's, with and without matviews,indexListfor the same grid, the Redshift listing and key reads, and the presence probe. The same probe over the moved code produced a byte-identical file (SHA-2568ac5b9952d47c220f7f39af6a7df38c319787e557c9085c111c3db6570b7f49dboth sides, 5,554 lines).PostgreSQLCatalogSQLPinTestspins the shipped shapes as exact strings recorded from origin/main.t,v,mv, partitionedp/p_2024, foreignft): the listing returnedft, p, p_2024, t, vwithmvmissing; ont,t_expr_onlywas absent,t_mixedread[tenant_id],t_includeread[a, b],t_order(created(b, a)) read[a, b],t_partiallost(a > 0), and GIN, HASH and BRIN all readBTREE; onmv,mv_exprwas absent;information_schema.columnsreturned 0 rows for the matview.PostgreSQLDriverCatalogTests):mvlists as.materializedView;t_pkeyfirst;t_mixed[tenant_id, lower(email)];t_include[a]including[b];t_order[b, a];t_partialWHERE (a > 0);t_hashHASH; the matview has columnsmv_id integer,mv_expr text,label character varyingwith length 20, and indexesmv_id_keyandmv_expr_idxonlower(mv_expr); the first column read keeps both identity columns and the matview arm.information_schema.tablesreports the table asFOREIGN;DROP TABLEfails with"ft" is not a tableand the hintUse DROP FOREIGN TABLE to remove a foreign table.;DROP FOREIGN TABLEsucceeds;TRUNCATEdepends on the foreign data wrapper (one without a handler refuses it), and the Mac never offers it on a foreign table.currentPluginKitVersionis untouched and nothing underPlugins/TableProPluginKitchanged, so no ABI check applies.Tests
ARCHS=arm64)TableProTestsviaverify.sh testswift test --filter TableProModelsTestsiOS suites run:
PostgreSQLCatalogQueryTests(new, 16),PostgreSQLColumnReadSupportTests(new, 5),PostgreSQLDriverCatalogTests(new, live, 5),TableKindPresentationTests(new, 10), and the existingPostgreSQLForeignKeyQueryTests,PostgreSQLConnectionStringTests,PostgreSQLCopyStateTests,PostgreSQLTransactionStatementTests,IOSDriverFactoryLocalFileTests,MySQLTableListingTests,MySQLVariantSupportTests,SQLBuilderPaginationTests,SQLDDLFallbackPolicyIOSTests,SQLDialectParityTests,ColumnMetadataRulesTests,TableKindListBehaviourTests,RowDetailViewModelTests,DataBrowserViewModelTests.TableKindListBehaviourTestsnow expects.foreignTableunder Tables and row-editable, like the Mac; its Truncate and Drop loops already require both to be off for every kind but a table (and a sequence for Drop). The live suite skips withoutPOSTGRES_TEST_HOST(pass it asTEST_RUNNER_POSTGRES_TEST_HOST) or/tmp/postgres-test.json, likeOracleDriverTests.macOS suites run:
PostgreSQLCatalogSQLPinTests,PostgreSQLTableListingTests,RedshiftTableCatalogTests(new), andPostgreSQLFetchTablesQueryTests,PostgreSQLFetchTablesCommentTests,PostgreSQLLegacyCatalogQueryTests,PostgreSQLLiteralQuotingTests,PostgreSQLLiteralQuotingSourceScanTests,PostgreSQLPartitionFilterTests,PostgreSQLTableListingLadderTests,PostgreSQLColumnsQueryTests,PostgreSQLMaterializedViewColumnsQueryTests,PostgreSQLIndexQueryTests,PostgreSQLIndexKeyPartTests,PostgreSQLIndexDDLQueryTests,TableStructureIndexReplayTests,PostgreSQLCatalogBooleanTests,PostgreSQLCatalogPresenceTests,RedshiftColumnsQueryTests,PluginPartitionRelationTypeTests,PostgreSQLSchemaEscapeTests. The five suites that calledPostgreSQLSchemaQueries.fetchTablesare repointed toPostgreSQLTableListing.queryin this commit with their expectations unchanged.What turns each new test red. A mutation run applied the first three edits at once and the listed cases failed:
tablesQuerypassingincludeMaterializedViews: false:optionalCatalogsFollowTheProbeand livetestMaterializedViewIsListedAsOne(measured red).columnsQuerynever adding the matview arm:materializedViewArm,columnQueryQuotesand livetestMaterializedViewHasColumnsAndIndexes(measured red).IOSDriverFactorynot passingdatabaseType:redshiftRoutesWithItsType(measured red)..materializedViewgiven the view'seyesymbol:kindsAreDistinct(measured red) andmaterializedViewIsNotAView.indexesQuerywithout its Redshift guard:redshiftKeys.failedProbeMeansAbsent.attnum = ANY(indkey)read, orIndexInfo(from:)not mapping the new fields:indexRowsDecode, livetestIndexesKeepKeyOrderExpressionsIncludeColumnsAndPredicates,mapPluginIndexInfoIncludeAndPredicate.materializedViewArmFailureKeepsIdentity,identityFailureKeepsMaterializedViews.FOREIGN TABLE/FOREIGNarm fromTableInfo(from:):mapPluginForeignTable(measured red, 4 issues),foreignTableKind, livetestForeignTableIsListedAsOne. Removing the.foreignTablecase failsforeignTableIsNotATableandTableKindListBehaviourTestsat compile time.PostgreSQLCatalogSQLPinTests,PostgreSQLTableListingTestsandRedshiftTableCatalogTestspin moved code: they pass on both sides by construction, and any change to the moved SQL or decoders turns them red.Also:
verify.sh build(TablePro) PASS,verify.sh build PostgreSQLDriverPASS, iOSbuild-for-testingPASS with no warnings in touched files,verify.sh docsPASS,check-ios-shared-isolation.pyPASS.verify.sh lintover every touched file: 0 violations on changed lines; the 6 it reports are pre-existing (listed below).Before / After
Screenshots to be added. States to capture on iPhone (light and dark), against a PostgreSQL schema holding a table with the indexes
t_mixed (tenant_id, lower(email)),t_include (a) INCLUDE (b),t_order (b, a),t_partial (a) WHERE a > 0, a GIN, a HASH and a BRIN index, a view, and a materialized view with an expression index:BTREE. After: key parts in order, theINCLUDE (b)line, theWHERE (a > 0)line,GIN/HASH/BRINbadges.lower(...).Critique points not taken
None. Every correction in the critique is applied:
ordinal_positionstays out of the result, and identity columns are dropped only when they are what failed.PostgreSQLDriver+Catalog.swiftis anonisolated extensionreading throughexecute(query:); the access changes are listed above. Every top-level declaration in the shared files isnonisolated.RedshiftTableCatalog.listingQuery) rather than moving to an unmeasured query.Deviations from the design, both in naming only:
PostgreSQLTableListingQueriesshipped asPostgreSQLTableListing(queryandtable(fromRow:)), andRedshiftTableKeysshipped asRedshiftTableCatalog, which holds the listing too, as the critique asked.Deliberately not fixed here
TableProMobile/TableProMobile/Drivers/MSSQLDriver.swift:268hard-codesCLUSTERED.MSSQLSchemaQueries.indexes(Packages/TableProCore/Sources/TableProMSSQLCore/MSSQLSchemaQueries.swift:194) reads neitheris_included_columnnorfilter_definitionand orders bykey_ordinal, which is 0 for anINCLUDEcolumn, so those come first and read as key columns.IndexInfonow carriesincludedColumnsandwhereClausefor it.TableProMobile/TableProMobile/Drivers/MySQLDriver.swift:231hard-codesBTREEoverSHOW INDEX'sIndex_type(FULLTEXT, SPATIAL, HASH), and theguard ... let colName = row[4]at line 212 drops a functional key part, whoseColumn_nameis NULL.TableProMobile/TableProMobile/Drivers/OracleDriver.swift:262hard-codesBTREEfor bitmap and function-based indexes.DROP TABLEfor every kind it offers, so a foreign table gets no Drop here until that list builds its statement per kind (TableProMobile/TableProMobile/Views/TableListView.swift:193).public_error_text_in_log) on lines this PR does not touch:Plugins/PostgreSQLDriverPlugin/PostgreSQLPluginDriver.swift:815,:840, andPlugins/PostgreSQLDriverPlugin/RedshiftPluginDriver.swift:61,:108,:138,:241.generate_series,pg_am,pg_get_indexdef(oid, k, true)), which the Mac already sends it; if one of those is missing, that Structure load fails.