Switchover mechanism, for use when primary itself is doing switchover - #83
Switchover mechanism, for use when primary itself is doing switchover#83serprex wants to merge 5 commits into
Conversation
|
depends on ClickHouse/wal-rus/pull/24 |
serprex
changed the title
| configured `[source] slot` name, when slot mode is on. Physical slots | ||
| are never synchronized to a standby at any PostgreSQL version; PG 17 | ||
| slot sync covers logical slots created with `failover = true` | ||
| (`src/backend/replication/logical/slotsync.c`) |
There was a problem hiding this comment.
Can we create the slot on the standby target before hand? We could make the slot on the standby all the time. We can keep the wal available on primary and standby in sync. This could help avoid wal hole problem.
There was a problem hiding this comment.
yes but then you need to report slot progress to both primary & standbys
There was a problem hiding this comment.
yes, would that be a big issue?
Implements support for changing source/destination config without restart Some issues being avoided: - boundary aligned fork needs zero crc - pre-commit restart skipping ancestor crossing, ie 1. manifest still says timeline 1, crossing commit has not happened 2. floor is start of segment containing fork, common when `C` and `F` share a segment 3. target is promoted to timeline 2 4. daemon dies after promotion while paused or waiting inside barrier 5. boot reads live history, `tli_of_segment(floor)` returns timeline 2 because descendant owns fork segment filename 6. condition accepts timeline 2 despite manifest still naming timeline 1 7. pump starts directly on timeline 2, so it never performs timeline 1 to 2
| use ahash::{HashMap, HashMapExt, HashSet}; | ||
|
|
||
| /// Source PG connection, `[source]` minus `slot`. Live: a reload that moves | ||
| /// the endpoint swaps the pump's `SourceFeed` at a chunk boundary and points |
There was a problem hiding this comment.
why slot isn't there? it also gets updated? is it because if it changes we do something different?
| pub struct CliOverrides { | ||
| pub drop_table_strategy: Option<String>, | ||
| pub flush_timeout: Option<Duration>, | ||
| /// `--slot`. Set means the operator pinned the slot name for this process, |
There was a problem hiding this comment.
comment not required. i don't think we need flush_timeout here either.
in the headdoc above it's mentioned that source_slot is updatable? but here we are negating any updates.
There was a problem hiding this comment.
if this is the only reason for source_slot to be separate from SourceConn, lets just keep one source_slot here and update CliOverrides later accordingly.
| self.timeline | ||
| } | ||
|
|
||
| pub fn seg_size(&self) -> u64 { |
There was a problem hiding this comment.
i can't see the function being used anywhere
| begin: u64, | ||
| } | ||
|
|
||
| /// Dial the source and resume at `resume_lsn`, proving continuity first: |
There was a problem hiding this comment.
also implemented in transition.rs. we can move this implementation there, reduce some burden from stream.rs
in general there's a lot of stuff here in stream.rs from transition.rs
| pub fn identity(&self) -> server::Identity { | ||
| server::Identity { | ||
| system_id: self.system_identifier.clone(), | ||
| timeline: self.timeline, |
There was a problem hiding this comment.
maybe we can keep timeline and system_id in walshadow status? would help with ubicloud for orchestration. and any other things that you might think might be helpful.
| writeln!(s, "{name}{{reason=\"{reason}\"}} {v}").unwrap(); | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Can we also update grafana dashboard to consume it?
| slot = source_slot.as_deref(), | ||
| "crossed source timeline", | ||
| ); | ||
| history = crossing.history; |
There was a problem hiding this comment.
source_swap_pending should also be cleared here no?
| ) | ||
| .await?; | ||
| } | ||
| if ancestor_ended { |
There was a problem hiding this comment.
nit: could be over engineering, but there's a lot of a of these state parameters, can we do something about it? like some state machine?
2 participants
Includes support for changing source/destination config without restart