How to Evaluate Inkling 2 Small on Your Own Code

Start with a task you can judge

A new model announcement is a reason to investigate, not a reason to replace a working development setup. Choose one small change in a repository you understand: repair a broken filter, improve a form error or make an existing view work on a narrow screen. Write down the expected behaviour before giving the model access.

This is a proposed evaluation workflow, not a hands-on review or a benchmark result. We have not run Inkling 2 Small against these tasks. Keep a copy of the starting version so that every attempt begins from the same files.

Check what is actually available

Thinking Machines released Inkling 2 Small on November 5 through Tinker, initially with closed weights. A planned weight release is different from downloadable weights available today. Check current access and charges before starting. Our model listing links to the announcement and model card.

Give the model only the files, example data and tools needed for the change. Use a disposable branch and a test environment. Make any network, deployment and file-writing boundaries explicit. A useful first evaluation should be easy to inspect and cheap to repeat.

Write the acceptance checks first

Describe the visible result and the cases that could go wrong. For a filter, specify an ordinary match, no matches, an empty query and clearing the query. For a form, specify the invalid state, the correction and a failed response from a test backend. Avoid real customer data or actions that send messages.

In [view], change [one behaviour]. The result must pass [specific checks]. Work only in [allowed files]. Use [test command] and inspect [screens or states]. Explain any remaining failures. Stop before committing or deploying. Do not change the requirements to make the tests pass.

Keep the same prompt and starting files if you compare attempts. Record the reasoning setting and time limit rather than changing several conditions at once. Otherwise you will not know whether a better outcome came from the model, extra time or a clearer brief.

Review the change before the explanation

Read the diff. Look for unrelated edits, removed checks, new dependencies and assumptions about input data. Run the checks yourself and inspect the result in the browser. A confident completion message is not proof that the requested behaviour exists.

For a visual task, capture the same screen size and state before and after. Check the empty state, a long label and the smallest supported view. Annotate the exact region that needs revision. “The error pushes the submit button outside the viewport at this width” gives a clearer target than “make it cleaner.”

Count the work that remains

Record whether the task passed, what you corrected manually, how many follow-up requests it took and the reported usage cost. Keep failed attempts in the record. A useful model may still need supervision, but you should be able to describe where that supervision was required.

If an attempt fails, repeat it from the original version with one clarification. Do not quietly combine the best pieces of several attempts and describe the result as a first-pass success. Save the prompt, diff, checks and screenshots together so another person can understand the outcome.

Expand only after a repeatable result

Try a second task with different failure cases before making the model part of a routine workflow. Keep your current model available as a reference and choose based on accepted work, review effort and cost. Fine-tuning belongs later, after you have examples of both useful and unacceptable outputs.

Read the latest daily briefing for the surrounding releases, or browse AI-assisted projects for ideas you can turn into bounded experiments of your own.