Free AI Engineering Skills Assessment

A Playwright test clicks a button before it has finished rendering, and fails now and then. What is the best fix?

Use a locator action, which waits for the element to be visible and enabledRetry the whole test three timesRun the test in a single workerAdd a fixed wait of two seconds before the clickI don't know yet

What is a page object for?

Running tests in parallelRecording videos of failing testsStoring test data in a databaseKeeping a page's locators and actions in one class, so tests read as steps and one change fixes every testI don't know yet

In Python, what does [x * 2 for x in range(3)] produce?

What does a pytest fixture do?

Marks a test as expected to failProvides setup (data, objects, clean-up) that tests receive as argumentsRuns tests in a random orderCompares two screenshotsI don't know yet

An API returns 201 after a POST. What does that usually mean?

Why validate an API response against a JSON schema in a test?

To encrypt the responseTo avoid writing assertions on status codesTo make the call fasterTo catch missing fields or wrong types even when the status code is correctI don't know yet

What is a token, for a language model?

A training exampleA password for the APIA piece of text (often part of a word) that the model reads and generates one at a timeA single sentenceI don't know yet

You need the model's answer as JSON your code can parse. What is the most reliable approach?

Ask politely for JSON in the prompt and hopeUse structured output (a JSON schema) and still validate what comes backRaise the temperatureParse the text with a regular expressionI don't know yet

What does retrieval-augmented generation (RAG) add to a model call?

Relevant passages from your own documents, retrieved at question time and given to the model with the questionA larger context windowA second model that checks spellingMore training data for the modelI don't know yet

A RAG answer sounds right but cites nothing from the documents. What should a test check?

That the model responded quicklyNothing: fluent answers are correctOnly that the answer is under 200 wordsThat the answer's claims are supported by the retrieved passages (grounding)I don't know yet

What makes an AI system an agent rather than a single model call?

It runs in the cloudIt uses a bigger modelIt decides which tools to call and in what order, and acts on the results in a loopIt has a chat interfaceI don't know yet

An agent can issue refunds through a tool. Which safeguard matters most?

A longer system promptLimits on what the tool can do, and human approval above a thresholdUsing temperature 0Logging only successful refundsI don't know yet

Why can't you test an LLM feature only with exact string matches?

Correct answers vary in wording, so tests need criteria such as grounding, relevance or a rubricLLMs never return stringsExact matches are not allowed in pytestString matching is too slowI don't know yet

A new prompt version scores lower on your golden set of questions. What should the release pipeline do?

Delete the questions that failRaise the pass thresholdShip it and monitorBlock the release, or require a reviewed exception, until quality recoversI don't know yet

Where should an application's API keys live in production?

What does a CI pipeline typically do on every pull request?

Deploys straight to production without checksBuilds the code and runs the tests, so problems are found before mergingDeletes old branchesRewrites the commit historyI don't know yet