Real-world DOA benchmark (Part 1) + Streamlit demo on Render (Part 2) by akhilvasvani · Pull Request #3 · akhilvasvani/Sound-Source-Localization

akhilvasvani · 2026-08-13T07:31:49Z

Summary

Part 1 -- Real-world DOA benchmark: tests whether the reverberation-driven
accuracy degradation seen with the original synthetic heart-proxy signal
also happens with real-world, non-medical audio (LibriSpeech speech,
ESC-50 environmental sound). Verdict: yes, across every dataset tested --
see reports/part1_results.md.

Part 2 -- Interactive demo, now deployment-ready: a single-page
Streamlit app to pick a preset or upload your own multi-channel
recording and see the estimated vs. true source position in 3D.

The deployment architecture changed mid-PR at the user's explicit
request: no Google Cloud services of any kind (no Cloud Run, GCS,
Cloud SQL, Firebase); the demo is now one Dockerized Streamlit
service on Render, not a two-process FastAPI+Streamlit container.

Architecture

demo/app.py (Streamlit) calls src/service.py -> src/pipeline.py
directly, in-process. No HTTP hop, no second server.
api/app.py still exists as an optional, non-deployed FastAPI
wrapper over the same src/service.py, kept for local
curl/scripting convenience and for test/unit/test_api.py. It is
not started in the deployed container.
No database, no queue, no scheduled job -- every DOA run is
synchronous (0.3-27s measured) and nothing needs to survive a process
restart. See README.md's "Architecture" section for the full
reasoning.
All runtime-writable paths (uploads, generated .mat scratch files)
derive from a single APP_DATA_DIR env var (src/paths.py) --
./app_data locally, /var/data in Render. No persistent disk is
attached: everything under it is per-request scratch data, consumed
and deleted within the same request.

Deployment files

Dockerfile / docker-entrypoint.sh -- single-process image, pinned
python:3.11-slim, non-root user, respects Render's $PORT via
entrypoint expansion, XSRF protection left enabled.
render.yaml -- Docker-based Web Service, APP_DATA_DIR=/var/data,
default TCP health check (reasoning + optional /_stcore/health
alternative documented in README.md).
.dockerignore, .env.example -- new.
vercel.json -- removed (no separate backend or frontend left to
route through Vercel; see README.md's "Vercel" section).

Validation

pytest test/ -- 104/104 passing.
No Docker daemon or Render connector is available in the environment
this was built in, so the literal docker build/Render deploy steps
could not be executed directly. As a substitute, the exact
docker-entrypoint.sh command was run directly with a test $PORT
and $APP_DATA_DIR, and a full Playwright browser click-through
exercised both preset mode and upload mode end-to-end against the
running app, confirming correct results rendering and that scratch
files under APP_DATA_DIR are cleaned up after each run. Full,
unvarnished details (including exactly what was and wasn't verified)
are in reports/deployment_status.md.

Required Render environment variables

Variable	Value
`APP_DATA_DIR`	`/var/data`
`PORT`	set automatically by Render

No persistent disk required. No separate Vercel frontend exists or was
introduced.

…tAPI+Streamlit demo Part 1 (real-world data swap): - src/audio_source.py: generic mono/multi-channel .wav/.flac loader, keeps the heart-sound .mat path (src/preprocess.py) untouched and selectable, adds a separate load_multichannel_recording() path for already-recorded multi-mic uploads that skip room simulation. - src/pipeline.py: shared DOA + triangulation pipeline (room/mic geometry constants reused from validation/run_validation.py for a fair before/after comparison, RT60_LEVELS matched exactly to the existing low/medium/high baseline configs, bearing-only fallback when only one mic cluster is available since triangulation needs >=2 rays). - experiments/real_world_benchmark.py: runs MUSIC/SRP/TOPS/CSSM/WAVES across 3 datasets (heart-proxy white noise, LibriSpeech, ESC-50) x 3 RT60 levels x 3 source positions (135 rows), writing experiments/results/real_world_benchmark.{csv,json}. - data/: small real-world benchmark clips (LibriSpeech, ESC-50) plus data/heart_proxy_samples/, which documents and reproduces the original repo's actual baseline signal -- literal Gaussian white noise, not real or synthetic heart-sound audio (see its README.md). - Bug fix: src/triangulation.py's new pairwise_ray_intersections() had a sign error in the closest-point-between-two-skew-lines formula (reflected points through the ray origin instead of finding the true closest approach); fixed and covered by test/unit/test_pairwise_ray_intersections.py. - 12 new unit tests (test_audio_source.py, test_pipeline.py, test_pairwise_ray_intersections.py, test_api.py); full suite 104/104 passing. Part 2 (interactive demo): - api/app.py: FastAPI backend wrapping the DOA pipeline (GET /api/presets, /api/upload-mic-geometry, /api/health, POST /api/run/preset, /api/run/upload). - demo/app.py: single-page Streamlit frontend -- preset or upload mode, 3D Plotly view of mic clusters/rays/pairwise intersection cloud/estimated vs true position, results panel with angle/position error and measured RT60. - Dockerfile + docker-entrypoint.sh: single-container packaging (uvicorn + streamlit) for Cloud-Run-style hosts. NOT build-tested -- no docker binary was available in the development sandbox; disclosed in demo/README.md. - demo/README.md: local run instructions (plain venv and Docker).

…rk data - experiments/results/: full 135-row benchmark output (3 datasets x 3 RT60 levels x 5 algorithms x 3 source positions), zero runtime errors. - reports/part1_results.md: headline finding is that reverberation-driven accuracy degradation is universal across all three datasets (roughly 3-20x worse position error and single-digit-degree to 75-105-degree bearing error swinging from low to high RT60 in every dataset) -- swapping in real-world audio does not fix it, confirming it is a physical/algorithmic DOA limitation and not a heart-sound-data artifact. One narrower exception: TOPS specifically performs badly on the narrowband heart-proxy signal even at low reverberation, and that weakness goes away on broadband real-world audio -- reverberation- independent, algorithm-specific, but does support part of the original hypothesis about narrow frequency range. - reports/deployment_status.md: the FastAPI+Streamlit demo runs correctly locally (verified); it is not deployed to a public URL. No Cloud Run connector exists in this environment (the literal request), and a real, multi-attempt Vercel deployment of the backend hit a 'You don't have permission to create a Production Deployment for this project' error tied to the connected account/team, not the code -- documented plainly with what was tried. - README.md: added a top-level pointer section to both reports plus the new benchmark/demo code, without touching the existing accuracy section's original numbers.

Removes the two-process FastAPI+Streamlit architecture and every GCP reference in favor of one Python process (Streamlit calling the pipeline in-process) in one Dockerized Render Web Service, per updated requirements. - src/service.py: new shared layer (preset catalog, run_preset/ run_upload) used directly by demo/app.py and by the now-optional api/app.py wrapper. Removes the HTTP hop entirely. - demo/app.py: calls src/service.py in-process; no more the `requests` library or DOA_API_BASE_URL. - api/app.py: kept only as an optional, non-deployed dev/test convenience wrapper (not started in the container). - src/paths.py: single APP_DATA_DIR-based root for all runtime-writable data (uploads, generated .mat scratch files), defaulting to ./app_data locally and /var/data in Render. - src/experiment.py / src/pipeline.py: ExperimentalMicData now accepts a configurable output_dir and generates collision-safe filenames (uuid suffix) instead of a hardcoded output/test_first_fun.mat path; scratch files are deleted immediately after use. - Dockerfile / docker-entrypoint.sh: single-process image, non-root user, respects Render's PORT env var via entrypoint expansion, no GCP/Cloud Run references. - .dockerignore, render.yaml, .env.example: new. - vercel.json: removed (no separate backend or frontend to route). - README.md / demo/README.md / reports/deployment_status.md: rewritten for the new architecture, Render deployment steps, storage/health check reasoning, and an honest validation report (no Docker daemon or Render connector available in this session -- see the report for exactly what was and wasn't verified). - requirements.txt: adds streamlit/plotly (previously only installed ad hoc in the old Dockerfile); fastapi/uvicorn/httpx kept for the optional wrapper + its tests. 104/104 tests still pass. Verified end-to-end via Playwright against the literal docker-entrypoint.sh command (no Docker daemon in this sandbox) -- both preset and upload modes render correctly, PORT expansion works, APP_DATA_DIR scratch files are created and cleaned up.

vercel · 2026-08-15T18:52:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
doa-demo-api-preview	Error		Aug 15, 2026 6:52pm

Root cause: python:3.11-slim ships /var as root-owned, mode 755. The Dockerfile only created and chowned the local-dev default (/app/app_data) to the non-root appuser at build time -- it never touched /var/data, the path Render actually sets APP_DATA_DIR to in production. At runtime, appuser (uid 1000) tried to os.makedirs("/var/data") and got [Errno 13] Permission denied, since a non-root process cannot create a directory under a root-owned, non-writable parent. Fix: - Dockerfile: create AND chown both /app/app_data and /var/data to appuser while still root, before the USER appuser switch. /var/data now exists and is writable by the time the container starts, regardless of which APP_DATA_DIR value is set at runtime. - docker-entrypoint.sh: runs the new scripts/check_app_data_dir.sh before starting Streamlit -- logs the resolved APP_DATA_DIR and running user, and verifies real writability (create+remove a scratch file) before Streamlit or the slow pyroomacoustics/scipy imports behind it even start. Fails fast with a clear FATAL message instead of a cryptic error the first time a user clicks "Run". - src/paths.py: get_app_data_dir() now performs the same real write-probe and raises a clear AppDataDirError (naming the resolved path, uid, and pointing at the Dockerfile) instead of letting a bare PermissionError bubble up to demo/app.py's generic exception handler. - test/unit/test_app_data_dir.py: 7 new regression tests, run as the actual non-root sandbox user (uid 2000, never root) -- proves get_app_data_dir()/get_uploads_dir()/get_generated_dir() can create and remove scratch files; proves a clear AppDataDirError is raised against an unwritable directory; runs check_app_data_dir.sh directly as a subprocess against both a writable and (chmod 555) unwritable directory and asserts the exact success/failure behavior. - reports/deployment_status.md: added an honest incident writeup with cause, fix, and everything verified. No architecture or deploy target changes: still one Dockerized Streamlit service on Render, APP_DATA_DIR=/var/data, no persistent disk. Verified end-to-end by running the real docker-entrypoint.sh against both a deliberately unwritable directory (fails immediately, never starts Streamlit) and a writable one (starts cleanly, serves HTTP 200, leaves no scratch files behind) -- no Docker daemon available in this sandbox, same limitation noted throughout this report. 111/111 tests passing (104 previous + 7 new).

akhilvasvani · 2026-08-15T20:06:32Z

Fixed: `APP_DATA_DIR=/var/data` permission denied on Render

The first real Render deploy failed at runtime with:

DOA pipeline failed: [Errno 13] Permission denied: '/var/data'

Cause: python:3.11-slim ships /var as root-owned, mode 755. The Dockerfile only created/chowned the local-dev default (/app/app_data) to the non-root appuser at build time -- it never touched /var/data, the path Render actually sets APP_DATA_DIR to. A non-root process can't create a new directory under a root-owned, non-writable parent, so appuser was denied at runtime.

Fix (no architecture or deploy-target change -- still one Dockerized Streamlit service on Render, APP_DATA_DIR=/var/data, no persistent disk):

Dockerfile now creates and chowns both /app/app_data and /var/data to appuser while still root, before the USER appuser switch.
docker-entrypoint.sh runs a new scripts/check_app_data_dir.sh before starting Streamlit -- logs the resolved directory + running user, verifies real writability (create+remove a scratch file), and fails fast with a clear FATAL message if it's ever wrong again.
src/paths.py's get_app_data_dir() does the same write-probe and raises a clear AppDataDirError instead of a bare PermissionError.
test/unit/test_app_data_dir.py: 7 new regression tests, run as an actual non-root user, covering both the Python helpers and the shell startup check against writable and deliberately-unwritable directories.

Verified by running the real docker-entrypoint.sh against both a locked-down directory (fails immediately, never starts Streamlit) and a writable one (starts cleanly, curl gets HTTP 200, no scratch files left behind) -- no Docker daemon available in this environment, so docker build itself still needs a real run wherever Docker is available before the next deploy. Full details in reports/deployment_status.md's new "Incident" section. Full suite: 111/111 passing.

README: - Rework intro sentence to frame the fork's expanded scope - Remove internal repair history, old Time-to-Run, original heart-sound results, Render/Vercel deploy instructions, and references/credits sections -- all obsolete or internal-only - Rename real-world benchmark section (drop 'this round's additions') - Rewrite Algorithm section into 4 clear subsections covering TDOA, triangulation/geometry, reverberation-is-hard (not solved), and the existing coplanar/non-planar cluster explanation Heart-proxy removal: - Delete the heart-proxy preset from src/service.py's preset registry; rename the remaining 3 presets to the exact required labels - Update test/unit/test_api.py (preset count/ids, renamed end-to-end test, default preset in the bad-algorithm test) - Update demo/README.md preset count and description wording - Reword reports/deployment_status.md's historical validation note to flag the preset it references has since been removed - Add a one-line historical-context note to reports/part1_results.md (kept intact -- it's a legitimate benchmark report, not a demo reference); data/heart_proxy_samples/ intentionally preserved since experiments/real_world_benchmark.py still depends on it Demo UX (demo/app.py): - New title 'Where is that sound coming from?' with a responsive clamp()-sized font so it doesn't wrap into many lines on a 375px phone viewport, plus a subtitle - Replaced the old intro paragraph with the two required plain-language paragraphs; removed the preset-dropdown caption entirely (no replacement) per spec - Added the required captions under the DOA-algorithm and reverb selectors - Renamed 'Results' to 'How close did it get?'; angle error is now the first, most prominent result ('Off by X deg'), with dynamic decimal precision - Added an honest, documented 'Roughly X cm at the far wall' line, computed via ray-box exit distance to the room boundary along the true bearing and a small-angle arc-length approximation (formula and assumptions documented in code comments); omitted automatically when the geometry needed to compute it isn't available - Moved all secondary metrics (algorithm, runtime, azimuth/colatitude, coordinates, RT60, distance error) into a collapsed 'Full numbers' expander - Added 'How this works' (README Algorithm anchor) and 'Source' (repo) footer links; added a mobile orientation caption near the top since Streamlit's sidebar toggle icon has no text label Verified: full pytest suite (111/111 passing) and a local Streamlit smoke test via Playwright covering all 3 presets, upload mode, footer link hrefs, and a 375px mobile viewport (no horizontal overflow, sidebar expand/collapse, expander, results all readable).

akhilvasvani · 2026-08-15T21:06:44Z

Follow-up pass: documentation + Streamlit UX polish, heart-proxy removed

This pass focused on docs and demo UX only -- no changes to the DOA algorithms, benchmark methodology, or Render deployment architecture.

README.md

Reworded the intro history sentence to frame the fork's expanded scope, kept the rest of the intro list unchanged
Removed: "Status of this fork" paragraph, "History of this fix", "Time to Run", "Results (original heart-sound study)", "Deploying to Render", "Vercel", and "References/Credits/Future" -- all obsolete or internal-only
Renamed "Real-world data benchmark & interactive demo (this round's additions)" -> "Real-world data benchmark & interactive demo"
Rewrote the Algorithm section into 4 clear subsections (TDOA per cluster, ray triangulation + why geometry matters, reverberation makes this harder / not solved, why clusters must be small + non-planar). This is now the canonical explanation linked from the demo footer: README.md#algorithm

Heart-proxy removed cleanly

Deleted the heart_proxy preset from src/service.py's registry. The demo now exposes exactly 3 presets: Speech (LibriSpeech), Speech, second clip (LibriSpeech), Rain (ESC-50)
Updated test/unit/test_api.py (preset count/ids, renamed the heart-proxy end-to-end test, default preset used in the bad-algorithm test)
Updated demo/README.md (preset count, description wording)
reports/deployment_status.md and reports/part1_results.md are legitimate historical/benchmark documents, not demo references -- kept intact but each now has a one-line note flagging that heart_proxy is no longer a selectable demo preset
data/heart_proxy_samples/ is intentionally not deleted -- experiments/real_world_benchmark.py still depends on it for the Part 1 benchmark methodology

Demo UX (demo/app.py)

New title "Where is that sound coming from?" with a responsive font (clamp()), plus a subtitle -- doesn't wrap into many lines on a 375px phone
Replaced the old intro paragraph with two short, jargon-free paragraphs; removed the preset-dropdown caption entirely (no replacement)
Added the required captions under the DOA-algorithm and reverberation selectors
Renamed "Results" to "How close did it get?" -- angle error ("Off by X°") is now the first, most prominent number, with dynamic decimal precision
Added an honest "Roughly X cm at the far wall" line: computed as range_to_wall_m * angle_error_radians, where range_to_wall_m is the ray-box exit distance from the mic-cluster centroid to the room boundary along the true bearing (small-angle arc-length approximation, documented in code comments). Automatically omitted when the geometry needed isn't available (e.g. no ground truth)
Moved every secondary metric (algorithm, runtime, azimuth/colatitude, coordinates, RT60, distance error) into a collapsed Full numbers expander
Footer now has "How this works" (links to the README Algorithm anchor above) and "Source" (repo link), with no remaining heart-proxy reference

Validation

Full test suite: 111/111 passing
Local Streamlit smoke test via Playwright: all 3 presets + upload mode ran correctly with no heart-proxy text anywhere; footer link hrefs verified; 375px mobile viewport checked for the title, intro, controls, results, expander, and footer -- no horizontal overflow, sidebar expand/collapse works, added an explicit on-page hint since Streamlit's sidebar toggle icon has no text label

Not merging -- leaving this open per the review request.

- README: remove the stale 'Generic .wav/.flac data loading' bullet referencing a heart_sound mode that no longer exists in src/audio_source.py - Demo intro: 'Pick a sound below' -> 'Pick a sound to the left (or upload your own) and run it.' - Demo results: remove the 'Roughly X cm at the far wall' room-scale line and its now-unused _ray_box_exit_distance helper entirely (no replacement approximation added); drop the now-unused math import - Demo footer: remove the extra 'Source: ... thin UI ...' prose sentence, keeping only the 'How this works · Source' link line Verified: pytest test/ (111/111 passing) + a local Streamlit smoke check confirming the updated intro text, the removed far-wall line, the removed footer prose, and the preserved footer links.

…ference - README.md: add a 'Deploying to Render' section (Blueprint flow + manual-equivalent settings/env vars, matching render.yaml exactly). Three files (render.yaml's own header comment, demo/README.md, and reports/deployment_status.md) already pointed readers at this section, but it did not exist. - reports/deployment_status.md: fix a dangling self-reference to a nonexistent README.md 'Vercel' section (Vercel support was already fully removed in an earlier pass); point at the real 'Deploying to Render' section instead. No behavior change. Verified via a repo-wide markdown link/anchor scan that no broken internal doc links remain.

_run_doa_and_report() (private helper in src/pipeline.py) accepted this parameter but never read it inside the function body. Confirmed via vulture (100% confidence) and a manual read of the full function. Neither of the two call sites (run_doa_from_source_file, run_doa_from_multichannel_upload) referenced it for anything besides this pass-through, and no test, demo, or API code calls _run_doa_and_report directly or references the parameter by name. No behavior change -- pytest test/ passes 111/111 identically before and after.

Documents the Part 2 pre-merge cleanup pass: detected stack, commands verified/run, documentation fixes made, the one dead-code removal made (with evidence), two proposed-but-not-yet-deleted candidates (results/ legacy heart-sound study dump; two orphaned test/unit/ fixture CSVs) with full evidence for each, and every item that was reviewed and intentionally kept because usage was confirmed or safety was uncertain. No files have been deleted in this pass -- deletions are pending explicit approval per the repo's cleanup policy.

Deletions approved by the repository owner after review of the full evidence in docs/repository-hygiene-audit.md: - results/ (601 files, ~11 MB): CSV/PNG dumps and tl;dr.txt/ statistics.txt notes from the original pre-fork heart-sound S1/S2 localization study. Zero references anywhere in code, docs, tests, or CI; already excluded from the Docker build context via .dockerignore. - test/unit/test_no_transform.csv, test/unit/test_transform.csv (~1.1 MB combined): orphaned mic-coordinate dumps predating any current test, never referenced by filename or dynamic discovery in the test suite. Validated: pytest test/ -- 111 passed (identical to pre-deletion).

akhilvasvani added 3 commits August 13, 2026 07:15

vercel Bot had a problem deploying to Production August 15, 2026 18:52 Failure

akhilvasvani changed the title ~~Real-world DOA benchmark (Part 1) + FastAPI/Streamlit demo (Part 2)~~ Real-world DOA benchmark (Part 1) + Streamlit demo on Render (Part 2) Aug 15, 2026

akhilvasvani added 5 commits August 15, 2026 21:28

akhilvasvani merged commit 8cc7a72 into master Aug 15, 2026

akhilvasvani deleted the feature/real-world-benchmark-and-demo branch August 16, 2026 18:44

Navigation Menu

Real-world DOA benchmark (Part 1) + Streamlit demo on Render (Part 2) - #3

Real-world DOA benchmark (Part 1) + Streamlit demo on Render (Part 2)#3
akhilvasvani merged 10 commits into
masterfrom
feature/real-world-benchmark-and-demo

akhilvasvani commented Aug 13, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Aug 15, 2026 •

edited

Loading

Uh oh!

akhilvasvani commented Aug 15, 2026

Uh oh!

akhilvasvani commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

akhilvasvani commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Architecture

Deployment files

Validation

Required Render environment variables

Uh oh!

vercel Bot commented Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akhilvasvani commented Aug 15, 2026

Fixed: APP_DATA_DIR=/var/data permission denied on Render

Uh oh!

akhilvasvani commented Aug 15, 2026

Follow-up pass: documentation + Streamlit UX polish, heart-proxy removed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

akhilvasvani commented Aug 13, 2026 •

edited

Loading

vercel Bot commented Aug 15, 2026 •

edited

Loading

Fixed: `APP_DATA_DIR=/var/data` permission denied on Render