[opt](lance) reduce redundant work in LanceTableReader

[zhangstar333]

What problem does this PR solve?

Problem Summary:

optimize LanceTableReader by lazily initializing schemas and scanner metrics, reusing the record batch converter, caching Arrow normalization plans, and avoiding unnecessary full-column reads. Also improve code structure by extracting schema import and record batch conversion logic.

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[zhangstar333]

/review

[github-actions]

Static review of PR #68335 at head c64d2d7 (base 70bf96e).

Two actionable regressions are called out inline:

• Nested registered-extension arrays: the new normalizer compacts a sliced parent before unwrapping extension children, so Arrow 24 cannot build list/struct parents containing ExtensionType children; the existing sliced nested JSON regression path fails.
• Row-ID/take_rows lifecycle: scanner profile counters are now lazy, so scanner-less phase-two fetches skip Lance data-cache byte statistics that were previously created and collected.

Review checkpoints:

• All ten authoritative changed paths were swept, including the converter extraction, nested normalization, scanner projection, generated search-result columns, global row IDs, runtime-filter/schema gating, close/reset paths, and test changes.
• Projection, global-rowid, runtime-filter, normal scanner reuse, build integration, and other candidate concerns were independently checked; no additional actionable issue was substantiated.
• Existing inline review comments were empty at the final duplicate-fence check, and the live base/head matched the authoritative bundle.
• No user-provided focus points were supplied beyond the complete-review requirement.
• This is a static review only: no builds or tests were run, so author/CI claims are not independent validation.

[github-actions]

Static review of PR #68335 at head c64d2d7 (base 70bf96e).

Two actionable regressions are called out inline:

• Nested registered-extension arrays: the new normalizer compacts a sliced parent before unwrapping extension children, so Arrow 24 cannot build list/struct parents containing ExtensionType children; the existing sliced nested JSON regression path fails.
• Row-ID/take_rows lifecycle: scanner profile counters are now lazy, so scanner-less phase-two fetches skip Lance data-cache byte statistics that were previously created and collected.

Review checkpoints:

• All ten authoritative changed paths were swept, including the converter extraction, nested normalization, scanner projection, generated search-result columns, global row IDs, runtime-filter/schema gating, close/reset paths, and test changes.
• Projection, global-rowid, runtime-filter, normal scanner reuse, build integration, and other candidate concerns were independently checked; no additional actionable issue was substantiated.
• Existing inline review comments were empty at the final duplicate-fence check, and the live base/head matched the authoritative bundle.
• No user-provided focus points were supplied beyond the complete-review requirement.
• This is a static review only: no builds or tests were run, so author/CI claims are not independent validation.

[github-actions]

Static review of PR #68335 at head c64d2d7 (base 70bf96e).

Two actionable regressions are called out inline:

• Nested registered-extension arrays: the new normalizer compacts a sliced parent before unwrapping extension children, so Arrow 24 cannot build list/struct parents containing ExtensionType children; the existing sliced nested JSON regression path fails.
• Row-ID/take_rows lifecycle: scanner profile counters are now lazy, so scanner-less phase-two fetches skip Lance data-cache byte statistics that were previously created and collected.

Review checkpoints:

• All ten authoritative changed paths were swept, including the converter extraction, nested normalization, scanner projection, generated search-result columns, global row IDs, runtime-filter/schema gating, close/reset paths, and test changes.
• Projection, global-rowid, runtime-filter, normal scanner reuse, build integration, and other candidate concerns were independently checked; no additional actionable issue was substantiated.
• Existing inline review comments were empty at the final duplicate-fence check, and the live base/head matched the authoritative bundle.
• No user-provided focus points were supplied beyond the complete-review requirement.
• This is a static review only: no builds or tests were run, so author/CI claims are not independent validation.

[zhangstar333]

run buildall