Skip to content

DPO/ORPO example notebooks#3893

Draft
igorts-git wants to merge 1 commit into
mainfrom
igorts/dpo-examples-notebooks
Draft

DPO/ORPO example notebooks#3893
igorts-git wants to merge 1 commit into
mainfrom
igorts/dpo-examples-notebooks

Conversation

@igorts-git

@igorts-git igorts-git commented May 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

This PR contains DPO and ORPO example notebooks.
These notebooks will also serve as integration tests for DPO.

BUGS: b/485626968

Tests

Ran the notebooks manually.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@igorts-git igorts-git force-pushed the igorts/dpo-examples-notebooks branch from a5a4b99 to 52d24e0 Compare May 14, 2026 22:35
@codecov

codecov Bot commented May 14, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@igorts-git igorts-git force-pushed the igorts/dpo-examples-notebooks branch from 52d24e0 to 91bb063 Compare May 15, 2026 00:04
@igorts-git igorts-git mentioned this pull request May 21, 2026
Merged
4 tasks
@igorts-git igorts-git force-pushed the igorts/dpo-examples-notebooks branch 5 times, most recently from be3ecbc to bd43150 Compare June 3, 2026 18:47
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

This PR has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Automatically applied to stale PRs. label Jul 4, 2026
@igorts-git
Feature: Add DPO/ORPO alignment tutorial and demo notebooks
b6bb393 
Integrate a comprehensive DPO/ORPO alignment tutorial Jupyter Notebook and a standalone DPO/ORPO demonstration notebook with dynamic algorithm toggling and CI validation. Update documentation and unit tests accordingly.
@igorts-git igorts-git force-pushed the igorts/dpo-examples-notebooks branch from d8aa8ea to b6bb393 Compare July 6, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jacoguzo jacoguzo Awaiting requested review from jacoguzo jacoguzo will be requested when the pull request is marked ready for review jacoguzo is a code owner
@bvandermoon bvandermoon Awaiting requested review from bvandermoon bvandermoon will be requested when the pull request is marked ready for review bvandermoon is a code owner
@SurbhiJainUSC SurbhiJainUSC Awaiting requested review from SurbhiJainUSC SurbhiJainUSC will be requested when the pull request is marked ready for review SurbhiJainUSC is a code owner
@richjames0 richjames0 Awaiting requested review from richjames0 richjames0 will be requested when the pull request is marked ready for review richjames0 is a code owner
@shralex shralex Awaiting requested review from shralex shralex will be requested when the pull request is marked ready for review shralex is a code owner
@gobbleturk gobbleturk Awaiting requested review from gobbleturk gobbleturk will be requested when the pull request is marked ready for review gobbleturk is a code owner
@RissyRan RissyRan Awaiting requested review from RissyRan RissyRan will be requested when the pull request is marked ready for review RissyRan is a code owner
@gagika gagika Awaiting requested review from gagika gagika will be requested when the pull request is marked ready for review gagika is a code owner
@A9isha A9isha Awaiting requested review from A9isha A9isha will be requested when the pull request is marked ready for review A9isha is a code owner
@jiangjy1982 jiangjy1982 Awaiting requested review from jiangjy1982 jiangjy1982 will be requested when the pull request is marked ready for review jiangjy1982 is a code owner
@vipannalla vipannalla Awaiting requested review from vipannalla vipannalla will be requested when the pull request is marked ready for review vipannalla is a code owner
@darisoy darisoy Awaiting requested review from darisoy darisoy will be requested when the pull request is marked ready for review darisoy is a code owner
@abhinavclemson abhinavclemson Awaiting requested review from abhinavclemson abhinavclemson will be requested when the pull request is marked ready for review abhinavclemson is a code owner
@khatwanimohit khatwanimohit Awaiting requested review from khatwanimohit khatwanimohit will be requested when the pull request is marked ready for review khatwanimohit is a code owner
@hengtaoguo hengtaoguo Awaiting requested review from hengtaoguo hengtaoguo will be requested when the pull request is marked ready for review hengtaoguo is a code owner
@aireenmei aireenmei Awaiting requested review from aireenmei aireenmei will be requested when the pull request is marked ready for review aireenmei is a code owner
@NuojCheng NuojCheng Awaiting requested review from NuojCheng NuojCheng will be requested when the pull request is marked ready for review NuojCheng is a code owner
@suexu1025 suexu1025 Awaiting requested review from suexu1025 suexu1025 will be requested when the pull request is marked ready for review suexu1025 is a code owner
@NicoGrande NicoGrande Awaiting requested review from NicoGrande NicoGrande will be requested when the pull request is marked ready for review NicoGrande is a code owner
@dipannita08 dipannita08 Awaiting requested review from dipannita08 dipannita08 will be requested when the pull request is marked ready for review dipannita08 is a code owner
@xibinliu xibinliu Awaiting requested review from xibinliu xibinliu will be requested when the pull request is marked ready for review xibinliu is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

stale Automatically applied to stale PRs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@igorts-git