[ZEPPELIN-6430] Remove SparkR and R interpreter by pan3793 · Pull Request #5271 · apache/zeppelin

pan3793 · 2026-06-16T12:10:47Z

What is this PR for?

SparkR has been officially deprecated since Spark 4.0, and plans to be removed in Spark 5.0 (early 2027), it also lacks maintenance and user cases in Zeppelin.

https://lists.apache.org/thread/qjgsgxklvpvyvbzsx1qr8o533j4zjlm5

What type of PR is it?

Breaking change.

What is the Jira issue?

ZEPPELIN-6430

How should this be tested?

Pass GHA

Screenshots (if appropriate)

Questions:

Does the license files need to update? No.
Is there breaking changes for older versions? Yes.
Does this needs documentation? Yes.

Assisted-by: GLM 5.2

The selenium integration test job (test-selenium-with-spark-module-for-spark-3-5) fails on AuthenticationIT because the GitHub runner ships Chrome 149 while the pinned Selenium 4.20.0 only bundles devtools up to CDP v125, producing "Unable to find CDP implementation matching 149" and degrading browser automation. This failure is pre-existing on master and unrelated to the SparkR/R removal in this PR. Selenium 4.45.0 bundles selenium-devtools-v147/v148/v149, an exact match for Chrome 149. The integration tests only use stable Selenium 4.x APIs, so the bump is API-compatible.

Removing R packages from the conda env files changed the dependency solve, so conda started pulling protobuf >= 4.x/5.x. Zeppelin ships an old-format generated kernel_pb2.py (interpreter/jupyter/grpc/jupyter/kernel_pb2.py), which is incompatible with protobuf 4+, causing IPythonKernelTest to fail: TypeError: Descriptors cannot be created directly. If this call came from a _pb2.py file, your generated code is out of date ... Downgrade the protobuf package to 3.20.x or lower. core-modules and interpreter-test-jupyter-python both hit this (master passed because the R-bearing env kept protobuf at 3.x). Pin protobuf<4, matching the existing convention in env_python_3_with_flink_{119,120}.yml.

- scripts/vagrant/zeppelin-dev/README.md: drop the R/R-packages bullet (the R vagrant role was already removed). - docs/interpreter/jupyter.md: remove the "Jupyter R kernel" (IRKernel) section, consistent with dropping IRKernel support/tests. - message-common.interface.ts: drop 'r' / 'ace/mode/r' from the EditorLanguage/EditorMode unions (no interpreter emits them anymore).

No longer referenced after dropping the "Jupyter R kernel" section in docs/interpreter/jupyter.md.

jongyoul · 2026-07-03T03:40:43Z

@pan3793 Thank you for the work. FYI, we could remove some licenses. IIRC, we added some license when adopting R relevant changes.

The Shiro FormAuthenticationFilter (aliased 'authc') intercepts POSTs to /api/login because the [urls] section lacked /api/login = anon and the catch-all /** = authc matches it. The filter reads request.getParameter ("username") — consuming the request body — before Jersey's @FormParam can parse "userName", resulting in a null username and UnknownAccountException. Adding /api/login = anon bypasses the filter, letting LoginRestApi.postLogin handle authentication directly. Fixes AuthenticationIT, PersonalizeActionsIT, InterpreterModeActionsIT which all use shiro.loginUrl = /api/login.

…x overlap The classic-webapp's fixed navbar logo occasionally overlaps the login modal buttons, causing ElementClickInterceptedException. Added a JavascriptExecutor-based click fallback to both the navbar Login button and the modal Login button in authenticationUser(), bypassing the purely visual z-index/stacking issue.

pan3793 · 2026-07-03T05:52:00Z

@jongyoul I've performed a check. The LICENSE, NOTICE, and zeppelin-distribution/src/bin_license/ have no R/rlang-related entries to remove. The old r module's BSD 3-Clause entries (rscala, Scala portions) were already cleaned up by ZEPPELIN-5420, and the rlang module never added any bundled third-party code requiring LICENSE entries — it was pure ASF-licensed code with no external dependencies.

jongyoul · 2026-07-03T06:18:39Z

@pan3793 Thank you for confirming it!

The login modal buttons can throw ElementNotInteractable (e.g. while the Bootstrap modal fade animation is still settling) in addition to ElementClickIntercepted (navbar logo z-index overlap). Extend both catch blocks in authenticationUser to fall back to the JavascriptExecutor click for either exception.

…nteractable ElementClickInterceptedException extends ElementNotInteractableException in Selenium 4.x, making Java multi-catch | illegal for related types. Catch only the supertype ElementNotInteractableException which covers both.

sendKeys dispatches DOM input events that only update Angular's $viewValue. The $modelValue (loginParams.userName) is only committed on the next $digest, but ng-click's $apply(fn) runs fn() before the digest, so login() reads stale (empty) credentials. The AJAX POST silently sends userName= or never dispatches. Replace sendKeys with direct Angular scope injection: find the scope containing loginParams, set userName and password directly, then call $apply() to digest. This guarantees the model is correct when ng-click fires login().

The scope-injection approach was unnecessary complexity. Instead, keep the original sendKeys flow but add an angular.element('#loginModal').scope().$apply() after typing to commit pending ng-model $viewValue changes before the login button's ng-click handler runs $apply(fn) — which calls the login function before the digest, causing it to read stale (empty) model values.

@Class

Chrome 149 changes the order Angular adds classes. The xpath contains(@Class, 'nav-btn dropdown-toggle ng-scope') fails when Angular prepends ng-scope (producing 'ng-scope nav-btn dropdown-toggle') instead of appending. Remove ng-scope from the contains() substring — nav-btn dropdown-toggle is unique enough in the navbar.

The AuthenticationIT login-modal flakiness under Chrome 149 is a pre-existing issue on master, unrelated to the SparkR/R removal. All integration-test CI fixes (Selenium bump, Shiro /api/login=anon, authenticationUser robustness) are reverted here. The other fixes (protobuf pins, R doc/image cleanup, conda env cleanup) are retained.

pan3793 · 2026-07-03T12:31:21Z

@jongyoul, it's ready now. The two frontend workflows fail consistently and are unrelated to this PR.

jongyoul · 2026-07-06T11:41:04Z

 ```bash
 spark.archive # default spark-${spark.version}
 spark.src.download.url # default http://d3kbcqa49mib13.cloudfront.net/${spark.archive}.tgz
-spark.bin.download.url # default http://d3kbcqa49mib13.cloudfront.net/${spark.archive}-bin-without-hadoop.tgz


Is this only for copying R binary?

jongyoul · 2026-07-06T11:47:58Z

            (Boolean) config.getOrDefault(InterpreterSetting.PARAGRAPH_CONFIG_CHECK_EMTPY, true);
    // don't skip paragraph when local properties is not empty.
-    // local properties can customize the behavior of interpreter. e.g. %r.shiny(type=run)
+    // local properties can customize the behavior of interpreter. e.g. %flink.ssql(type=update)


Is it fine to use flink query instead of r?

Changed to %spark(pool=pool1) — this is a real, documented Spark local property (Spark scheduler pool, documented in docs/interpreter/spark.md:428-431). It's the most contextually appropriate replacement: it's a Spark example (not Flink), the syntax is valid, and the local property is meaningful for Spark.

jongyoul

I left some minor comments but overall, LGMT. Thank you!

Replace %flink.ssql(type=update) with %spark(pool=pool1) — a real, documented Spark scheduler-pool local property (see docs/interpreter/spark.md), more appropriate than a Flink example in generic Paragraph code.

[ZEPPELIN-6430] Remove SparkR and R interpreter

4ecdbd1

pan3793 marked this pull request as draft June 16, 2026 13:24

pan3793 added 5 commits July 2, 2026 22:46

remove r it

ff90f35

Remove orphaned ir_kernel.png doc image

023f58d

No longer referenced after dropping the "Jupyter R kernel" section in docs/interpreter/jupyter.md.

pan3793 added 2 commits July 3, 2026 11:46

pan3793 added 7 commits July 3, 2026 14:25

Bump Spark 3.5.8

6a652ed

pan3793 marked this pull request as ready for review July 3, 2026 12:30

jongyoul reviewed Jul 6, 2026

View reviewed changes

jongyoul previously approved these changes Jul 6, 2026

View reviewed changes

Use Spark local property example in Paragraph comment

fc78089

Replace %flink.ssql(type=update) with %spark(pool=pool1) — a real, documented Spark scheduler-pool local property (see docs/interpreter/spark.md), more appropriate than a Flink example in generic Paragraph code.

pan3793 dismissed jongyoul’s stale review via fc78089 July 6, 2026 16:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ZEPPELIN-6430] Remove SparkR and R interpreter#5271

[ZEPPELIN-6430] Remove SparkR and R interpreter#5271
pan3793 wants to merge 16 commits into
apache:masterfrom
pan3793:ZEPPELIN-6430

pan3793 commented Jun 16, 2026 •

edited

Loading

Uh oh!

jongyoul commented Jul 3, 2026

Uh oh!

pan3793 commented Jul 3, 2026

Uh oh!

jongyoul commented Jul 3, 2026

Uh oh!

pan3793 commented Jul 3, 2026

Uh oh!

jongyoul Jul 6, 2026

Uh oh!

jongyoul Jul 6, 2026

Uh oh!

pan3793 Jul 6, 2026

Uh oh!

jongyoul left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

pan3793 commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is this PR for?

What type of PR is it?

What is the Jira issue?

How should this be tested?

Screenshots (if appropriate)

Questions:

Uh oh!

jongyoul commented Jul 3, 2026

Uh oh!

pan3793 commented Jul 3, 2026

Uh oh!

jongyoul commented Jul 3, 2026

Uh oh!

pan3793 commented Jul 3, 2026

Uh oh!

jongyoul Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

jongyoul Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

pan3793 Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

jongyoul left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pan3793 commented Jun 16, 2026 •

edited

Loading