Add Logistic PCA (LPCA) analysis by Jeebjean · Pull Request #500 · Reed-CompBio/spras

Jeebjean · 2026-07-10T17:05:48Z

Summary

Adds Logistic PCA (LPCA) as a new SPRAS analysis method. LPCA is the binary-data counterpart to the existing PCA analysis. It is designed for binary matrices (0/1) like the edge-by-run matrix built from pathway reconstruction outputs, whereas standard PCA assumes continuous data.

Changes

New config block

analysis.lpca with five options:

include: enable/disable LPCA (default false)
k: number of principal components (default 2)
m: logisticPCA tuning parameter, used when cv is false (default 6)
cv: if true, determine m by cross-validation; if false, use fixed m (default false)
transpose: if false, run LPCA on edges x runs matrix (one embedding per edge);
if true, run on runs x edges matrix (one embedding per pathway run,
mirrors the classic PCA analysis) (default false)

New Snakemake rule `lpca_analysis`

Restricted to algorithms with multiple parameter combinations (same guard as per-algorithm PCA)
Outputs per algorithm: {algorithm}-lpca-scores.csv, {algorithm}-lpca.png, {algorithm}-lpca-coordinates.txt

New module `spras/analysis/lpca.py`

Builds the binary matrix via summarize_networks
Optionally transposes the matrix
Runs the logisticPCA container via run_container_and_log
Generates a PC1 vs PC2 scatterplot and saves coordinates

New Docker wrapper `docker-wrappers/lpca/`

Pinned rocker/r-base:4.4.2 Dockerfile installing logisticPCA and ggplot2
run_lpca.R: runs LPCA and saves scores + deviance explained
run_cv.R: cross-validation to find optimal m
README documenting config options and Docker image publication

Unit tests `test/analysis/test_lpca.py`

3 tests: output file existence, score shape (edges x k), score shape (runs x k with transpose)

Docker timeout

Increased from 60s to 600s in containers.py to handle larger binary matrices

Testing locally

Build the image and run the pipeline:

docker build -t reedcompbio/lpca:v1 docker-wrappers/lpca/

Enable analysis.lpca.include: true in your config and run as usual.
Tested end-to-end with OmicsIntegrator2 (216 combos) and MincostFlow (14 combos).

Before / at merge

The image must be published to reedcompbio/lpca:v1 on Docker Hub by a maintainer:

docker build -t reedcompbio/lpca:v1 docker-wrappers/lpca/
docker push reedcompbio/lpca:v1

cc @agitter @ntalluri

Adds an analysis.lpca config block (include, k, m, cv, transpose) with a matching LpcaAnalysis schema, an lpca_analysis Snakemake rule restricted to algorithms with multiple parameter combinations, and an LPCA analysis module that builds the binary edge-by-run matrix via summarize_networks and runs the logisticPCA container through run_container_and_log. m is fixed by default; cross-validation and matrix transposition are opt-in.

Adds docker-wrappers/lpca with a pinned rocker/r-base Dockerfile that installs logisticPCA and its ggplot2 dependencies, the run_lpca.R and run_cv.R scripts under /app, and a README documenting the config options, script contracts, and how to build and publish reedcompbio/lpca:v1.

read-the-docs-community · 2026-07-10T17:07:07Z

Documentation build overview

📚 spras | 🛠️ Build #33731084 | 📁 Comparing c49362f against latest (1cb9d03)

🔍 Preview build

3 files changed

± genindex.html
± fordevs/spras.config.html
± fordevs/spras.html

agitter

Note that this will close #271 when merged.

I pushed the image to DockerHub: https://hub.docker.com/r/reedcompbio/lpca

We should add this to our Docker image building test: https://github.com/Reed-CompBio/spras/blob/main/.github/workflows/build-containers.yml

This comment should be modified to note the alternative LPCA

spras/spras/analysis/ml.py

Line 159 in 07707bf

# TODO: replace PCA https://github.com/Reed-CompBio/spras/issues/271

I didn't comment on every detail yet, but in general there are some small things we can to do make this more like the existing PCA.

agitter · 2026-07-24T19:50:46Z

  #       requirements = versionGE(split(Target.CondorVersion)[1], "24.8.0") && (isenforcingdiskusage =!= true)
  enable_profiling: false

-  # Override the default container image for specific algorithms.


We need to keep these deleted comments

agitter · 2026-07-24T19:54:29Z

    # evaluation per algorithm will not run unless ml include and ml aggregate_per_algorithm are set to true
    aggregate_per_algorithm: true
+  lpca:
+    include: false


We should document what true/false does. If true, will SPRAS run PCA and LPCA? That behavior would make sense to me. I initially view LPCA as an additional analysis, not a replacement for PCA. Running them in parallel will help us assess the similarity of the two. If we later find that LPCA is "better" than PCA by some criteria, we could change that behavior.

agitter · 2026-07-24T19:55:52Z

+    m: 6
+    # if true, choose m by cross-validation; if false, use the fixed m above
+    cv: false


It could be possible to collapse these into a single parameter m that takes cv as a valid value. I don't think it is necessary but am sharing the idea in case anyone else prefers that. It would complicate parsing and the config schema as a downside.

Somewhere we should document the analysis in your other repo that led to 6 as a default. It could be in the pull request comments.

agitter · 2026-07-24T20:00:59Z

+# LPCA (Logistic PCA) wrapper
+
+This wrapper runs [logisticPCA](https://github.com/andland/logisticPCA)
+(Landgraf & Lee, 2020) as a SPRAS analysis step. It reduces the binary


Suggested change

(Landgraf & Lee, 2020) as a SPRAS analysis step. It reduces the binary

([Landgraf & Lee, 2020](https://doi.org/10.1016/j.jmva.2020.104668)) as a SPRAS analysis step. It reduces the binary

agitter · 2026-07-24T20:01:28Z

All of the other Docker wrappers use caps in the subdirectory (for some reason). Should we rename the directory to "LPCA"?

agitter · 2026-07-24T20:27:58Z

+        matrix = matrix.T
+    print(f'LPCA: Matrix shape: {matrix.shape}')
+
+    # Step 2: write the matrix next to the outputs, namespaced by algorithm


Is this the same directory where PCA outputs are written? We should follow that convention.

It looks like it is.

agitter · 2026-07-24T20:28:42Z

+    output_dir = Path(output_scores).parent
+    output_dir.mkdir(parents=True, exist_ok=True)
+    algo_name = Path(output_scores).name.replace('-lpca-scores.csv', '')
+    matrix_path = str(output_dir / f'{algo_name}-lpca_binary_matrix.csv')


This is the full binary data matrix? I don't think we need to save that here. We eventually can move that to a rule in the Snakefile so it can be reused across algorithms.

agitter · 2026-07-24T20:31:41Z

+        run_container_and_log('LPCA-CV', LPCA_CONTAINER_SUFFIX, command_cv, volumes,
+                              LPCA_WORK_DIR, None, container_settings)
+
+        m_used = pd.read_csv(cv_output_path)['best_m'][0]


We could check this file exists before reading it to throw a friendly error.

agitter · 2026-07-24T20:33:08Z

+
+    print(f'LPCA: Done! Scores saved to {output_scores}')
+
+def plot_lpca(scores_file: str, output_png: str, output_coord: str, labels: bool = True) -> None:


Somewhere we should document that the KDE parameter selection always uses PCA even if LPCA is run. That may change in the future.

agitter · 2026-07-24T20:34:54Z

+    X = scores.values
+    fig, ax = plt.subplots(figsize=(10, 8))
+
+    sns.scatterplot(x=X[:, 0], y=X[:, 1], hue=column_names, s=70, ax=ax)


PCA use create_palette to set consistent colors.

agitter

Additional comments because I accidentally submitted the first batch before finishing.

agitter · 2026-07-24T20:37:10Z

    # Initialize a Docker client using environment variables
    try:
-        client = docker.from_env()
+        client = docker.from_env(timeout=600)


Is this still needed? Was it a workaround for the memory issues?

This modifies Docker but not Singularity.

agitter · 2026-07-24T20:38:15Z

+    except Exception:
+        return False
+
+skip_if_no_lpca_image = pytest.mark.skipif(


Do we want to silently skip LPCA tests if the image is not available? Shouldn't it always been on DockerHub now that I pushed it?

agitter · 2026-07-24T20:41:49Z

+        OUT_DIR.mkdir(parents=True, exist_ok=True)
+
+    @skip_if_no_lpca_image
+    def test_lpca_output_exists(self):


A stronger test would be that the expected output is created when we use known inputs. These are okay first steps, but if there is time we would add the stronger test.

There is some degeneracy with these existing pathways though that cause the PCA tests to fail at random. It may be okay here because we don't do KDE testing.

Will we need to account for sign non-identifiability in LPCA?

agitter · 2026-07-24T20:43:03Z

        ml.hac_horizontal(summary_df, output.hac_image_horizontal, output.hac_clusters_horizontal, **hac_params)
        ml.pca(summary_df, output.pca_image, output.pca_variance, output.pca_coordinates, **pca_params)

+


Can drop the newline

agitter · 2026-07-24T20:44:23Z


+rule lpca_analysis:
+    input:
+        pathways = collect_pathways_per_algo


Can we also support running LPCA on all pathways across algorithms.

agitter · 2026-07-24T20:45:20Z

        ml.hac_horizontal(summary_df, output.hac_image_horizontal, output.hac_clusters_horizontal, **hac_params)
        ml.pca(summary_df, output.pca_image, output.pca_variance, output.pca_coordinates, **pca_params)

+rule lpca_analysis:


Should this be a new rule or a conditional extension of ml_analysis and ml_analysis_aggregate_algo?

Jeebjean added 2 commits July 10, 2026 11:28

Jeebjean added 5 commits July 21, 2026 12:40

Add unit tests for LPCA analysis

efd1005

Add LPCA visualization: generate lpca.png and lpca-coordinates.txt

8f89119

Increase Docker timeout to 600s for large LPCA matrices

3d9a38f

Restore config.yaml to main defaults, keep only lpca block

e4b4679

Skip LPCA tests if Docker image not available

41b64b7

Jeebjean marked this pull request as ready for review July 21, 2026 21:07

Add rARPACK for memory-efficient LPCA with partial_decomp=TRUE

c49362f

agitter mentioned this pull request Jul 24, 2026

Add override mechanism for Cytoscape container #482

Open

agitter reviewed Jul 24, 2026

View reviewed changes

	(Landgraf & Lee, 2020) as a SPRAS analysis step. It reduces the binary
	([Landgraf & Lee, 2020](https://doi.org/10.1016/j.jmva.2020.104668)) as a SPRAS analysis step. It reduces the binary


		print(f'LPCA: Done! Scores saved to {output_scores}')

		def plot_lpca(scores_file: str, output_png: str, output_coord: str, labels: bool = True) -> None:

		ml.hac_horizontal(summary_df, output.hac_image_horizontal, output.hac_clusters_horizontal, **hac_params)
		ml.pca(summary_df, output.pca_image, output.pca_variance, output.pca_coordinates, **pca_params)

Uh oh!

Conversation

Jeebjean commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New config block

New Snakemake rule lpca_analysis

New module spras/analysis/lpca.py

New Docker wrapper docker-wrappers/lpca/

Unit tests test/analysis/test_lpca.py

Docker timeout

Testing locally

Before / at merge

Uh oh!

read-the-docs-community Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

agitter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agitter Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agitter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jeebjean commented Jul 10, 2026 •

edited

Loading

New Snakemake rule `lpca_analysis`

New module `spras/analysis/lpca.py`

New Docker wrapper `docker-wrappers/lpca/`

Unit tests `test/analysis/test_lpca.py`

read-the-docs-community Bot commented Jul 10, 2026 •

edited

Loading

agitter Jul 24, 2026 •

edited

Loading