Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split out validation logic for input-number #1640

Merged
merged 9 commits into from
Sep 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/modern-dancers-decide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@khanacademy/perseus": patch
---

Move validation logic out of the Input Number widget
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
import {mockStrings} from "../../strings";

import inputNumberValidator from "./input-number-validator";

import type {Rubric} from "./input-number.types";

describe("inputNumberValidator", () => {
it("scores correct answer correctly", () => {
const rubric: Rubric = {
maxError: 0.1,
inexact: false,
value: 1,
simplify: "optional",
answerType: "percent",
size: "small",
};

const useInput = {
currentValue: "1",
} as const;

const score = inputNumberValidator(useInput, rubric, mockStrings);

expect(score).toEqual({
earned: 1,
message: null,
total: 1,
type: "points",
});
});

it("scores incorrect answer correctly", () => {
const rubric: Rubric = {
maxError: 0.1,
inexact: false,
value: 1,
simplify: "optional",
answerType: "percent",
size: "small",
};

const useInput = {
currentValue: "2",
} as const;

const score = inputNumberValidator(useInput, rubric, mockStrings);

expect(score).toEqual({
earned: 0,
message: null,
total: 1,
type: "points",
});
});

it("shows as invalid with a nonsense answer", () => {
const rubric: Rubric = {
maxError: 0.1,
inexact: false,
value: 1,
simplify: "optional",
answerType: "percent",
size: "small",
};

const useInput = {
currentValue: "sadasdfas",
} as const;

const score = inputNumberValidator(useInput, rubric, mockStrings);

expect(score).toEqual({
message:
"We could not understand your answer. Please check your answer for extra text or symbols.",
type: "invalid",
});
});

// Don't default to validating the answer as a pi answer
// if answerType isn't set on the answer.
// The answer value and
// the omission of answerType in the answer are
// important to the test.
// https://khanacademy.atlassian.net/browse/LC-691
it("doesn't default to validating pi", () => {
const rubric: Rubric = {
maxError: 0.1,
inexact: false,
value: 241.90263432641407,
simplify: "required",
size: "normal",
};

const userInput = {
// 77 * pi = 241.90263432641407
// within the 0.01 margin of error
// to trigger the pi validation flow
currentValue: "241.91",
} as const;

const score = inputNumberValidator(userInput, rubric, mockStrings);

expect(score.message).not.toBe(
"Your answer is close, but yyou may " +
"have approximated pi. Enter your " +
"answer as a multiple of pi, like " +
"<code>12\\ \\text{pi}</code> or " +
"<code>2/3\\ \\text{pi}</code>",
);
expect(score.message?.includes("pi")).toBeFalsy();
});

it("validates against pi if provided in answerType", () => {
const rubric: Rubric = {
maxError: 0.1,
inexact: false,
value: 241.90263432641407,
simplify: "required",
answerType: "pi",
size: "normal",
};

const userInput = {
currentValue: "77 pi",
} as const;

const score = inputNumberValidator(userInput, rubric, mockStrings);

expect(score).toEqual({
earned: 1,
message: null,
total: 1,
type: "points",
});
});
});
100 changes: 100 additions & 0 deletions packages/perseus/src/widgets/input-number/input-number-validator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
import TexWrangler from "../../tex-wrangler";
import KhanAnswerTypes from "../../util/answer-types";

import type {Rubric} from "./input-number.types";
import type {PerseusStrings} from "../../strings";
import type {APIOptions, PerseusScore} from "@khanacademy/perseus";

const ParseTex = TexWrangler.parseTex;

export const answerTypes = {
number: {
name: "Numbers",
forms: "integer, decimal, proper, improper, mixed",
},
decimal: {
name: "Decimals",
forms: "decimal",
},
integer: {
name: "Integers",
forms: "integer",
},
rational: {
name: "Fractions and mixed numbers",
forms: "integer, proper, improper, mixed",
},
improper: {
name: "Improper numbers (no mixed)",
forms: "integer, proper, improper",
},
mixed: {
name: "Mixed numbers (no improper)",
forms: "integer, proper, mixed",
},
percent: {
name: "Numbers or percents",
forms: "integer, decimal, proper, improper, mixed, percent",
},
pi: {
name: "Numbers with pi",
forms: "pi",
},
} as const;

function inputNumberValidator(
state: {
currentValue: string;
},
Comment on lines +46 to +48
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's call this userInput and type it using the UserInput types that Matthew has created in #1639

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Should I wait for that PR to land then to reference UserInput or just copy over the PerseusInputNumberUserInput type for now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't wait to land it. I can fix this in my PR.

rubric: Rubric,
strings: PerseusStrings,
onInputError: APIOptions["onInputError"] = () => {},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be something we untangle. I don't think we'll have access to APIOptions in a server environment unless it directly affects scoring.

As such, we'll need to figure out how to communicate this all the way back to the caller.

Essentially, we won't be able to have callbacks in our validator functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we decided to handle this during a separate stage of the validation separation process, right? Should I just leave this here for now then? Maybe just adding a TODO to fix these when I find them would be the best thing for now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't do anything for now, we'll circle back. I think he was just thinking out loud.

): PerseusScore {
if (rubric.answerType == null) {
rubric.answerType = "number";
}

// note(matthewc): this will get immediately parsed again by
// `KhanAnswerTypes.number.convertToPredicate`, but a string is
// expected here
const stringValue = `${rubric.value}`;
const val = KhanAnswerTypes.number.createValidatorFunctional(
stringValue,
{
simplify: rubric.simplify,
inexact: rubric.inexact || undefined,
maxError: rubric.maxError,
forms: answerTypes[rubric.answerType].forms,
},
strings,
);

// We may have received TeX; try to parse it before grading.
// If `currentValue` is not TeX, this should be a no-op.
const currentValue = ParseTex(state.currentValue);

const result = val(currentValue);

// TODO(eater): Seems silly to translate result to this invalid/points
// thing and immediately translate it back in ItemRenderer.scoreInput()
if (result.empty) {
// TODO(FEI-3867): remove null-check once we have APIOptionsInternal
const apiResult = onInputError?.(
null, // reserved for some widget identifier
state.currentValue,
result.message,
);
return {
type: "invalid",
message: apiResult === false ? null : result.message,
};
}
return {
type: "points",
earned: result.correct ? 1 : 0,
total: 1,
message: result.message,
};
}

export default inputNumberValidator;
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {mockStrings} from "../../strings";
import {renderQuestion} from "../__testutils__/renderQuestion";

import InputNumber from "./input-number";
import inputNumberValidator from "./input-number-validator";
import {question3 as question} from "./input-number.testdata";

import type {
Expand Down Expand Up @@ -41,7 +42,7 @@ describe("input-number", function () {
});

describe("full render", function () {
it("Shoud accept the right answer", async () => {
it("Should accept the right answer", async () => {
// Arrange
const {renderer} = renderQuestion(question);

Expand Down Expand Up @@ -197,7 +198,7 @@ describe("input-number", function () {
"0.56",
],
])("answer type", (question, correct, incorrect) => {
it("Shoud accept the right answer", async () => {
it("Should accept the right answer", async () => {
// Arrange
const {renderer} = renderQuestion(question);

Expand Down Expand Up @@ -249,7 +250,7 @@ describe("invalid", function () {
});

it("should handle invalid answers with no error callback", function () {
const err = InputNumber.widget.validate(
const err = inputNumberValidator(
{currentValue: "x+1"},
options,
mockStrings,
Expand Down
Loading