A top grade on your child's coding or science project does not mean they actually understand the material. Students who ace project-based assignments often struggle when asked to adapt their designs to a single new constraint.
AI reveals that finished student projects often hide a lack of real comprehension, but targeted "what-if" prompts can instantly show whether your child grasps the underlying logic or merely followed a recipe.
Project-based learning is supposed to build critical thinking, but polished final products frequently mask surface-level work. When assignments involve group collaboration, online starter code, or generative AI, a student can easily turn in an "A" project without mastering the core concepts.
If schools rely solely on completed deliverables to measure progress, you will not know your child is falling behind until they hit advanced coursework that requires independent problem-solving.
Teachers know that custom scenario-based tests—asking "how would you redesign your app if the user base doubled?"—measure deep comprehension better than final project grades. But writing these tailored prompts by hand for dozens of unique student projects takes hours instructors do not have.
Researchers turned to large language models to automate the creation of these specialized design challenges, testing whether AI could craft realistic evaluation prompts at scale.
Student performance on targeted design challenges showed almost zero correlation with their overall project grades. A student with an immaculate finished project was just as likely to fumble a conceptual tweak as a student with a mediocre one.
- Teachers want these tests: All 31 surveyed instructors agreed that scenario-based prompts effectively test deep understanding, but noted they are practically impossible to write manually for every student.
- AI creates reliable challenges: Large language models successfully generated 80 distinct design problems, earning strong validation from subject-matter experts.
- Brainpower is visible in keystrokes: Real-time typing analysis revealed heavy pauses and frequent revisions while students tackled the AI prompts, confirming the tasks triggered genuine planning and problem-solving rather than rote recall.
Generative AI is creating the problem and the solution at the same time. While automated tools make it trivial for students to assemble impressive-looking projects without understanding the mechanics, teachers are now using those same tools to interrogate students' actual comprehension.
Expect classroom assessments to shift from grading the finished product to grading how quickly and thoughtfully a student can modify it on the spot.
This paper is an early preprint that has not yet undergone formal peer review. The findings are based on a small sample of 31 computing instructors, and the abstract leaves the exact number of tested students unspecified. While computer programming projects lend themselves well to modular design prompts, the same approach may not translate as cleanly to history papers or biology labs.
- If your child brings home an "A" on a complex STEM project… Ask them to explain what they would change if one core constraint shifted, such as doubling the data load or removing a key component, rather than just praising the finished product.
- If your child leans heavily on AI or starter templates to do their homework… Have them verbally walk you through the logic behind two specific decisions they made to ensure they are not outsourcing the actual thinking.
- If you notice new, short "scenario tests" showing up on your student's syllabus… Recognize these as transfer tasks designed to verify individual competence, and encourage your child to focus on underlying principles rather than memorized steps.
Stop treating a polished final project as proof of mastery. Watch whether your child can pivot when the rules change, because that is where real learning shows up.
Ahmad D. Suleiman, Daqing Hou, Maliha Noushin Raida (2026). LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning. arXiv (preprint). doi:10.1145/3795867.3831014 — arxiv.org