work in progress, docker, nextflow#4
Open
MateoZ05 wants to merge 5 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a reproducible execution path for the llmize pipeline via Nextflow + Docker, and adds an optional self-review loop intended to reduce unsupported gene/cell-type claims in generated interpretations.
Changes:
- Added Nextflow workflow (
main.nf) and configuration (nextflow.config) to run the pipeline with Docker. - Added Docker image build + runtime boot scripts to start Ollama and support offline model caching.
- Added deterministic symbol checks (
verify.py) and an optional review loop integrated into the CLI (pipeline.py/interpret.py).
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
verify.py |
Adds deterministic extraction of “unsupported” symbol tokens for review passes. |
pipeline.py |
Adds --review / --review-passes, env-driven model default, and hooks review into pipeline execution. |
interpret.py |
Adds review prompt/system prompt + review loop; improves per-section logging with timing/flush. |
main.nf |
Defines Nextflow INTERPRET process to run the pipeline (optionally booting Ollama in-container). |
nextflow.config |
Provides Nextflow params defaults and Docker configuration (container + host cache mount). |
Dockerfile |
Builds a combined Ollama + Python pipeline image and optionally bakes models. |
docker/entrypoint.sh |
ENTRYPOINT that boots Ollama then runs the provided command (or --check). |
docker/boot_ollama.sh |
Starts Ollama, waits for readiness, and pulls/uses cached models (offline-aware). |
.gitignore |
Ignores Nextflow work dirs/logs/results artifacts. |
.dockerignore |
Excludes generated outputs/intermediates from Docker build context. |
Comment on lines
+230
to
+235
| if review: | ||
| print(f"[pipeline] Reviewing interpretation (up to {review_passes} pass(es))...") | ||
| response = review_interpretation( | ||
| response, model=model, report=report, num_ctx=num_ctx, | ||
| passes=review_passes, think=think, gen_options=gen_options, | ||
| ) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Reword for positive grounding, make changes to report_vocabulary() in verify.py to not delete any important genes ranked below 50. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nextflow integration: main.nf + nextflow.config for reproducible pipeline execution. Added containerization with our Dockerfile and boot scripts for offline Ollama use. Added fact checking via review_interpretation() and verify.py to re-read and rewrite claims in loops to reduce hallucinations. New CLI passes with --review and --review-passes for our fact checking.