Diamond2 conversion2 by ptsOSL · Pull Request #5 · DiamondLightSource/dls-slow-feedbacks

ptsOSL · 2026-09-14T14:14:58Z

Backwards compatible with D1
Adds support for D2 ringmode 49 and 48
Add --d2 CLI option which switches between D1 and D2 ringmodes. When running with --d2 you get access to 49 and 48 and when running without it, you get the old ringmodes.
Handle both old and new tune response matrix matlab data formats
Refactor setting up tunefb startup_currents and make sure this setup is called both when the server is first initialised and on ringmode change.

Rather than constructing the RM in the order defined in the mml tuneRM file, we do it in the order defined in tunefb_offsets.py

We now find our config files from either diamond2opsdata or diamondopsdata depending on the ringmode

This ensures the caputs complete before continuing and anecdotally has been seen to improve results against the Virtac which responds much slower than the real machine.

When running in D2 mode, you get different ringmodes to choose which load different lattices and make different PVs

MJGaughran · 2026-09-16T11:15:37Z

+        # that they are ordered in the same order as defined elsewhere.
+        for family in families:
+            for rmat in raw_rms["Rmat"][0]:
+                rmat_family = str(rmat["Actuator"][0][0][0][0][1][0])


The list of indices seems a bit confusing, just reply to this comment explaining it so I can see whether it's worth hiding or not.

This is the structure we get when loading the matlab file:

[[array([[(array([[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457],
[107.11158457]]), array(['Q0L'], dtype='<U3'), array(['Setpoint'], dtype='<U8'), array([[ 1, 1],
[ 4, 17],
[ 5, 1],
[ 8, 17],
[ 9, 1],
[12, 17],
[13, 1],
[16, 17],
[17, 1],
[20, 17],
[21, 1],
[24, 17]], dtype=uint8), array([[1],
[1],
[1],
[1],
[1],
[1],
[1],
[1],
[1],
[1],
[1],
[1]], dtype=uint8), array(['Simulator'], dtype='<U9'), array(['Hardware'], dtype='<U8'), array(['a'], dtype='<U1'), array(['Get by FamilyName'], dtype='<U17'), array(['getpv'], dtype='<U5'), array([[0]], dtype=uint8), array([[0.02197698]]), array([[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283],
[740219.615283]]), array([[2.02600e+03, 8.00000e+00, 2.50000e+01, 1.40000e+01, 4.60000e+01,
4.45502e-01]])) ]],
dtype=[('Data', 'O'), ('FamilyName', 'O'), ('Field', 'O'), ('DeviceList', 'O'), ('Status', 'O'), ('Mode', 'O'), ('Units', 'O'), ('UnitsString', 'O'), ('DataDescriptor', 'O'), ('CreatedBy', 'O'), ('t', 'O'), ('tout', 'O'), ('DataTime', 'O'), ('TimeStamp', 'O')]) ]]

The indices are to get the quadrupole family Q0L. Im not sure why the imported dictionary is so horrible, it looks a lot better in matlab.

This code adds a way to make sure that the response matrixes are ordered how we expect. Essentially we are just checking that the data is ordered as we expect and if its not, then we reorder it. This is so we dont end up using the wrong response matrixes for a Quad family

I'm still a bit lost.

I was looking for a brief (one line?) comment explaining the whole [0][0][0][0][1][0] thing.

The indices are to index a nested list which is generated when the matlab file is loaded using scipy, the quadrupole family string is located at [0][0][0][0][1][0].

MJGaughran · 2026-09-16T11:16:00Z

+        rmats = []
+
+        # We build the response matrix out of the tune quad families, making sure
+        # that they are ordered in the same order as defined elsewhere.


Where is elsewhere? MML config? Inside dls_slow_feedbacks? All of them?

Inside dls_slow_feedbacks. Previously we just assumed the data in mml was in the order we required, now we check and reorder if necessary.

Could you make this explicit in the comment?

I have updated the comment

MJGaughran · 2026-09-16T11:23:54Z

    def set_mode(self, mode: int) -> None:
        """Set the ring mode and reload the lattice."""
-        self.name = RING_MODES[mode]
+        if self.name in RING_MODES_D2:


It does feel like there should be some separate variable to indicate whether we are in D2 or D1 mode, although I understand this should work.

I can see an argument for this yeh, I was trying not to treat D2 as a special case and rather focus on ringmodes which are already an established thing in slow feedbacks.

It is effectively a special case already. Please just make sure you've tested it in anger.

codecov · 2026-09-16T13:07:49Z

Codecov Report

❌ Patch coverage is 62.06897% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.05%. Comparing base (bd980d6) to head (c2bb4a4).

Files with missing lines	Patch %	Lines
src/dls_slow_feedbacks/tunefb_server.py	62.16%	14 Missing ⚠️
src/dls_slow_feedbacks/mode.py	56.52%	10 Missing ⚠️
src/dls_slow_feedbacks/rffb_server.py	40.00%	3 Missing ⚠️
src/dls_slow_feedbacks/sofb_server.py	50.00%	2 Missing ⚠️
src/dls_slow_feedbacks/vefb_server.py	71.42%	2 Missing ⚠️
src/dls_slow_feedbacks/__main__.py	50.00%	1 Missing ⚠️
src/dls_slow_feedbacks/tunefb_offsets.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
+ Coverage   47.55%   48.05%   +0.49%     
==========================================
  Files          16       16              
  Lines        1659     1698      +39     
==========================================
+ Hits          789      816      +27     
- Misses        870      882      +12

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This was actually slowing Virtac. This is because Virtac would see the changes to the first pv in the list and then recalculate before the others had taken effect. As wait=True, tunefb would have to wait for Virtac to recalculate and then read in the remainder of the caputs at which point tunefb could then move on. In some ways this was the correct behaviour, but it was causing issues.

Add D2 ringmodes

1099dc4

ptsOSL force-pushed the diamond2-conversion2 branch 2 times, most recently from 401d8cf to e4a1f2a Compare September 15, 2026 14:18

ptsOSL added 4 commits September 16, 2026 10:51

Reorder tune RM dynamically

ea511fc

Rather than constructing the RM in the order defined in the mml tuneRM file, we do it in the order defined in tunefb_offsets.py

Add D2 datapath and switch to pathlib

dffa44b

We now find our config files from either diamond2opsdata or diamondopsdata depending on the ringmode

Linting fixes

a9be92e

Update version import

6962c13

ptsOSL force-pushed the diamond2-conversion2 branch from fa5051a to 4e03ab6 Compare September 16, 2026 10:54

ptsOSL added 9 commits September 16, 2026 10:58

Update tunefb for D2

4ffb3c7

Mount config file location for D2

a5beec0

Update .vscode launch file

e3e4881

Get initial currents when changing ringmode

a9d33d5

Wait for vefb caputs to finish

d92c804

This ensures the caputs complete before continuing and anecdotally has been seen to improve results against the Virtac which responds much slower than the real machine.

Revert default ringmode to I04 and minor comment fixes

4954c61

Handle default tunes for D1 and D2

c7bfc6a

Add CLI -h option

d3541f2

Add way to switch to D2 mode via CLI

4bdbe8a

When running in D2 mode, you get different ringmodes to choose which load different lattices and make different PVs

ptsOSL force-pushed the diamond2-conversion2 branch from 4e03ab6 to 4bdbe8a Compare September 16, 2026 11:00

ptsOSL changed the title ~~Draft: Diamond2 conversion2~~ Diamond2 conversion2 Sep 16, 2026

ptsOSL requested review from MJGaughran and MichaelStubbings September 16, 2026 11:05

MJGaughran reviewed Sep 16, 2026

View reviewed changes

ptsOSL added 2 commits September 16, 2026 13:00

Add type hints and default values for diamond2 arg

78f32c1

Test fixes for recent changes

e888c1f

ptsOSL force-pushed the diamond2-conversion2 branch from a7cf2b4 to e888c1f Compare September 16, 2026 13:06

ptsOSL added 4 commits September 17, 2026 11:17

Require latest pytac for ringmode 49 support

0a2123c

Remove unused var

ff86a16

Make D2 tunes module level constant

c02a11e

Add missing type hint

614cbee

ptsOSL force-pushed the diamond2-conversion2 branch from d235679 to 614cbee Compare September 21, 2026 10:34

Reword comment

c2bb4a4

Conversation

ptsOSL commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MJGaughran Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ptsOSL commented Sep 14, 2026 •

edited

Loading

MJGaughran Sep 21, 2026 •

edited

Loading

codecov Bot commented Sep 16, 2026 •

edited

Loading