Skip to content

Feature/sig fig#15

Open
m-messer wants to merge 6 commits into
masterfrom
feature/sig_fig
Open

Feature/sig fig#15
m-messer wants to merge 6 commits into
masterfrom
feature/sig_fig

Conversation

@m-messer

Copy link
Copy Markdown
Member

Requires #14

Problem

IsSimilar should also support significant figures and tolerances.

Changes

  • Added a sig_figs (significant_figures) param that rounds both the response and answer to N significant figures and requires them to match, rather than checking a continuous tolerance band.
  • sig_figs is mutually exclusive with atol/rtol — supplying both raises an exception, since a rounding-based check doesn't combine sensibly with the existing additive tolerance formula.
  • Refactored app/evaluation.py: split tolerance and sig-figs logic into dedicated _evaluate_tolerance/_evaluate_sig_figs functions, and extracted shared _is_number/_result helpers to remove duplication. evaluation_function is now just param validation + dispatch.
  • Added 13 new tests covering correct/incorrect matches, negative numbers, zero, invalid sig_figs values, mutual-exclusivity errors, synonym keys, and non-numeric response/answer handling.
  • Updated readme.md, app/docs/user.md, and app/docs/dev.md to document the new param and its exclusivity rule.

Comment thread app/docs/dev.md Outdated
Comment thread app/docs/dev.md Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here, etc.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in #14

@peterbjohnson peterbjohnson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major comment:

  • I think we need to sit with Phil and discuss what we mean by sig fig and what the requirements are here. I've made a more detailed note inline in the code.

Minor comment:

  • can update all docs and code to the new parameter names (maybe do a sweep) for atol/rtol?

Comment thread app/evaluation.py Outdated
m-messer added 2 commits July 21, 2026 09:12
…eros, and enforce string-based responses for accurate significant-figure handling. Updated docs, tests, and logic accordingly.

@peterbjohnson peterbjohnson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments for your attention but not blocking.

I don't know how this change will work with symbolic answers. E.g. pi? Or x^2?

Comment thread app/docs/user.md
## Parameters

Both parameters default to `0` (exact match required) and can be used individually or together.
`atol` and `rtol` both default to `0` (exact match required) and can be used individually or together. `sig_figs` is an alternative comparison mode and cannot be used together with `atol`/`rtol`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use the long form, absolute_tolerance?

Comment thread app/evaluation.py
if not isinstance(value, str):
return None
stripped = value.strip()
body = stripped[1:] if stripped[:1] in ("+", "-") else stripped

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we also accept plus/minus (pm)? We have some accommodation for that concept. But perhaps it's at the preview stage which splits it into a list of multiple entries?

Comment thread app/evaluation.py
if sep and not exponent.lstrip("+-").isdigit():
return None

has_decimal = "." in mantissa

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used?

Comment thread app/evaluation.py

feedback = ""
if numeric_correct and not precision_correct:
feedback = f"Please give your answer to {sig_figs} significant figures."

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Karl started collecting all feedback strings in one place, so that they can be easily maintained. Could you find that and put this string there and call it?

@m-messer

Copy link
Copy Markdown
Member Author

Thanks for the feedback. Just to note that this is IsSimilar and not compareExpressions. I need to migrate the functionality across.

Can you confirm whether we want to support sig fig in isSimilar or only in compareExpressions?

@peterbjohnson

Copy link
Copy Markdown
Member

I think it's best to keep isSimilar as assume as possible. So porting to cE is best.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants