As a QA engineer moving into AI evaluation, I found this bootcamp gave me a clear picture of why QA matters so much across the AI software lifecycle. It covered red teaming, validating non-deterministic output, and catching hallucinations, and all of it was hands-on with real tools like Promptfoo, LM Studio, Agenta and Arato. Igor, Greg and Amanda knew the material deeply, broke complex topics into easy-to-follow concepts, and were always patient and willing to help. The career prep in the final week was a great bonus. This course took the mystery out of AI testing for me, and I'm excited to see what comes next. I highly recommend it to anyone who wants to move into AI evaluation but feels hesitant, and to anyone who wants to keep growing their career.
Become the Most Wanted Testers of the AI Era
Learn how to Test AI Agents, complex RAG systems, custom and commertical LLMs


Igor Dorovskikh
Founder · program overview
Our alumni work at

Everything you’ll use at EnGenious
13 tools · 4 workflows
01 Evaluate
Promptfoo
Agenta
Arato.ai
OWASP
02 Models
OpenAI API
Anthropic API
ChatGPT
LM Studio
03 Build
PythonNode.js
GitHub
04 Community
Zoom
Discord
What you’ll be able to test
Practised on a real AI product
Problem-solving
Break complex AI failures into testable parts
Critical use of AI tools
Know when to trust the output, and when not to
Structured thinking
Turn messy model behaviour into clear test cases
Adaptability
Keep pace as models and tools change
The Engenious University method to take you from:
Traditional testertoAI QA testing engineer
Learn how AI fails
Learn the seven ways AI breaks in live sessions with engineers who test it for a living
Prove your AI testing skills
Be ready when your team needs someone to test AI or your next interview calls for AI experience.
Stay relevant + earn more
Test a product in production, write your own evaluation report, and get code reviews from QA AI engineers
157+ QA professionals trained
Ask them yourself — most are still in our Discord


This course helped me see AI testing from a broader perspective and gave me the confidence that I can do it, all in a student-friendly environment that makes you feel comfortable

This course combined hands-on exercises with expert instructors who clearly know the field, giving us plenty of practice building and testing AI applications. It also pushed a mindset shift: instead of expecting one correct answer from an LLM, you learn to evaluate probabilistic outputs by setting thresholds and acceptable ranges.

This course exceeded my expectations! It was interactive from day one: every session had us actually doing the work, not just watching slides, and I came away with skills I could actually apply, not just notes. Highly recommend to anyone in QA looking to move into AI testing

Going into this course, one of the biggest challenges for me was figuring out what areas to focus on for the new AI tech I needed to upskill my QA engineering career. Engenious solves that by using real industry insight on what new skills companies are actually wanting and how that intersects with the software testing ecosystem. So, I was no longer guessing at what matters. On top of that, it's hands-on learning with real instruction and small class sizes. We got into actual tooling: Promptfoo, red-teaming, and the 7 core challenges framework. That combination is what made it stick.

Taking this course was one of the best decisions I’ve made. Learning about tools like Promptfoo, red teaming, and Arato was truly eye-opening, and the course gave me practical, hands-on experience through the internship program as well. The instructors are incredibly knowledgeable, supportive, and always willing to help. I’d highly recommend this course to anyone looking to grow their skills and accelerate their career

Really glad I took this course. The instructors clearly know their craft and do a great job guiding you through complex material in a way that actually sticks. We covered adversarial testing, bias detection, red teaming, and multi-model evaluation using real tools like Promptfoo, Agenta AI, and Arato AI — hands-on, not just theory. Highly recommend for anyone serious about AI testing.

This 5-week course is an absolute masterclass for anyone transitioning from traditional QA into the world of AI and LLMs. It moves far beyond the hype to tackle the genuine technical challenges of testing non-deterministic solutions, providing a robust framework for building a real-world test strategy. What made this course stand out: 1. Deep-Dive Tooling: We got hands-on experience with Promptfoo and LM Studio, learning exactly how to leverage them for measurable results and why. 2. Red Teaming & Strategy: Moving from theory to the actual execution of Red Teaming was a highlight, as was the focus on the financial cost of different using different LLMs and testing approaches. 3. Practical 'Learn-See-Do' Format: The structure was excellent. Every session followed a clear 'concept demonstration group exercise' flow. Working through these concepts in breakout groups ensured the knowledge actually stuck. 4. Total Support: The flexibility of having recorded sessions was a lifesaver for busy weeks and having the tutors available via Discord throughout the 5 weeks meant no question went unanswered. If you want to understand how to actually validate AI performance and manage the risks of non-deterministic systems, this is the course you need. Highly recommended!

This course was so much more than just learning foundational concepts to test AI models. From Jaime's AI excellence and experimentation tips to Amanda's knowledge, education prowess and love for learning to Igor's candid tips for selling yourself, visionary forward thinking about where testing is going in the future, this course exceeded my expectations and I grew a lot in my testing ability, knowledge and understanding on how to find a better job.

By far one of the most valuable courses I’ve taken. The format stands out because it’s not lecture-heavy, instead it’s predominantly hands-on, which in my experience is the most effective way to master new skills and technologies. The sessions provided not just theoretical knowledge, but real opportunities to apply concepts directly, from AI testing and LLMs to prompt evaluation and red teaming. This course is well worth investment for anyone looking to get practical experience in AI and LLM testing.

Great course! Complete eye opener for AI and LLM models. Course is very interactive and 80% of the time was devoted to actual hands on testing and developing skills starting from basic models to more complicated ones. Completing the course also gave more confidence while changing traditionally seasoned QA mind to QA AI Tester and inspired to continue learning in the new direction and search for new job.

An incredible learning journey! I loved learning how to test LLMs through practical exercises in group sessions and hands-on, real app testing. It's also been super valuable to get insights into what skills are really needed right now in the industry and how to prepare for interviews. The instructors are super passionate about what they do and genuinely care about everyone's progress. I finished with valuable new skills and a community of like-minded peers who continue to support and inspire me.
Skills you’ll leave with
One target role, the skills it asks for, and the proof that you have them.
Target role
Post-program
AI Evaluation Engineer
The role the program is built backwards from
$98K–$230K+
$98KUS market range$230K+
Knowledge
- Evaluate LLM apps
- Build eval suites in Promptfoo
- Red-team against the OWASP LLM Top 10
- Write bug reports engineers can act on
Proof of work
Verified
AI Application Testing Portfolio
EnGenious University

We break a real chatbot together — then turn it into a bug report.
They started where you are
One target role, the skills it asks for, and the proof that you have them.

“Testing AI and everything that we've been learning so far are very different. The reason why I am excited to tell you this is not because I'm super smart. It's because AI is smart and we learned how to use it. My message is if I can do it, you can do it.”
Who You Become, Week By Week

Understand what you're testing
Learn how AI apps work and fail, set up your workspace, and run your first AI tests.
Day 1: How AI Works and How It Fails
Duration: 3–4 hours
- See how an AI app is built – the provider, the model, tokens and the context window – and where each part can fail.
- Learn why the same question can get a different answer every time, and what that means for testing.
- Set up your workspace with Claude Code and run your first Promptfoo test, with no programming needed.
Tools:
- Claude Code
- Promptfoo
- VS Code
Day 2: Run Your First AI Test Suite
Duration: 3–4 hours
- Choose models to test using leaderboards and Hugging Face.
- Build a Promptfoo test file and run it on several models side by side.
- Write exact checks and your first AI-graded checks on a financial chatbot.
Tools:
- Promptfoo
- Groq
- Hugging Face
Define what a good AI answer looks like
Turn "that answer looks fine" into checks you can run, measure and explain.
Day 3: Instructions, Cost and Local Models
Duration: 3–4 hours
- Treat the app's instructions as the spec and user questions as test cases, and use AI to generate tests.
- Compare models on cost, speed and pass rate.
- Run a model on your own computer and test it with the same suite.
Tools:
- Promptfoo
- LM Studio
- OpenRouter
Day 4: Use AI to Grade AI
Duration: 3–4 hours
- Write scoring rubrics so a second model can grade answers that have no single correct wording.
- Set weights and pass marks, and measure results with precision, recall and F1.
- Fix deliberately broken test suites.
Tools:
- Promptfoo
- Claude Code
- GitHub
Try to break AI on purpose
Think like an attacker: find where an AI app gives in, and report what you find.
Day 5: Red Teaming by Hand
Duration: 3–4 hours
- Learn what red teaming is and the rule: test only systems you're authorised to test.
- Tell a weakness in the model from a weakness in the app built on it.
- Write your own attacks on a healthcare chatbot and run them on several models.
Tools:
- Promptfoo
- Claude Code
Day 6: Automated Attacks and AI Bug Reports
Duration: 3–4 hours
- Run ready-made attack sets (OWASP Top 10 for LLMs, NIST, MITRE ATLAS) against an AI app.
- Test a chatbot the way a user would, then again with its instructions in front of you.
- Write AI bug reports with how often the failure happens, and run your tests automatically with GitHub Actions.
Tools:
- Promptfoo
- GitHub Actions
Test a full AI app and keep watching it
Move from testing a model to testing a complete app, then learn how teams watch AI after release.
Day 7: Test an App, Not Just a Model
Duration: 3–4 hours
- Test a multi-agent fintech app: check it sends each question to the right agent and cites the right documents.
- Write tests for conversations that run over several messages, including an attack hidden a few messages in.
- Test the tools of an MCP server with positive and negative cases.
Tools:
- Promptfoo
- MCP
- Claude Code
Day 8: Access, Bad Data and Monitoring
Duration: 3–4 hours
- Check who can see what, using the OWASP Top 10 for LLM apps.
- See what one poisoned document does to an app's answers.
- Send your test runs to Arato and compare each release against a baseline.
Tools:
- Arato
- OpenTelemetry
- Promptfoo
Test agents and build proof you can test AI
Test an AI agent that takes actions, then turn your work into proof an employer can see.
Day 9: Test an AI Agent
Duration: 3–4 hours
- Test a support agent that uses tools: check what it actually did against the real data, not just what it said.
- Catch instructions hidden in data, poisoned memory, wrong tool data and failures that only show up on some runs.
- Manage prompts and tests in Agenta, and test a remote MCP server.
Tools:
- Promptfoo
- Agenta
- MCP
Day 10: Turn Your Work into Interviews
Duration: 3–4 hours
- Write accomplishment statements built on measurable results from your own tests.
- Prepare answers to AI testing interview questions, with a short live demo from your repo.
- Set up a LinkedIn routine to show your work and connect with hiring teams.
Tools:
- GitHub
Break into AI Testing is for if:

You want a higher-paying role
Companies pay more for testers who can tell them whether their AI is safe to ship
You want proof and confidence, not another certificate
A test suite you built, a red-team report, bugs you found yourself. Work you can point to in an interview or a performance review.
AI just landed on your desk
Your team is shipping a chatbot or AI feature, and the testing is falling to you with no playbook. 5 weekends from now, you’ll have one
You want to stay relevant as AI changes QA
AI is changing what testers are hired to do. You want to be the one teams rely on to test it, not the one wondering where the work went.
You want to become one of the most wanted testers of the AI era
You’re a manual tester, SDET, automation engineer or software engineer and now you’ll know how to test AI, and that combination is what every team shipping AI is looking for.

All you get after signing up
A 4-week internship inside a US AI startup
Apply from inside the cohort and practise everything you learned on a product that has users. “AI Testing Intern” goes on your LinkedIn the day you start.

Access to the community
1,000+ people chasing the same thing you are, in one place — trading what’s working in AI testing, passing along openings, building the network that gets you the next role.

Your proof of work: an eval report and a GitHub
Both with your name on them, both openable by a hiring manager in thirty seconds. The certificate is the least interesting thing you take with you.

Breakout groups in every session
Small group, one volunteer lead, and their only job that session is answering your questions. Nobody drifts quietly to the back.

Every recording and material, for a year
Each session, plus the configs, prompts, rubrics and test files we build together. Come back to any of it whenever you need it.

You are never alone
Our team is reachable whenever you’re stuck. During the five weekends, during the internship, and after. We stay until the outcome.

One training. Everything included.
Cohort January 2027
$3,997
Save $1,000
Regular price $4,997
Choose how you pay
100% money-back through Week 1, no questions asked
Secure checkout

Igor Dorovskikh
Igor Dorovskikh on LinkedIn (opens in a new tab)CEO/Founder of Engenious.io
Learn from The Best
“15+ years of experience in software testing and development and over a decade in management.”
- in management
- 10+ yrs
- Engenious.io
- Founder
- software testing & development
- 15+ yrs
January cohort is open
Pacific Time
Ten sessions. Five weekends.
Every Saturday and Sunday, 10:00 AM – 2:00 PM. Miss one? Every session is recorded.
or May 1

Frequently asked questions
Here, you'll find answers to the most commonly asked questions
QA professionals with at least a year of manual or automation experience who want to move into testing AI products. It also fits SDETs, test leads and other testers who want to evaluate and red-team AI apps.
No programming. You edit config files and run commands, with Claude Code's help. If you already code, you'll move faster – and you'll use it.
Tell us about your background and we'll look at it with you. Message us, or book a free career review.
No. You're learning a new kind of testing, not a new tool. AI answers aren't deterministic, so the work is grading output against rubrics and thresholds, and attacking the app the way users and attackers do. If you'd rather stay in automation and use AI to do it faster, the QA Mastery Program is the better fit.
Yes. The cohort is online and students join from many countries. Sessions run on Pacific time, so check the hours work where you live. The free career-review calls are open to US-based testers for now; everyone else can message us on WhatsApp or email.
Yes: a Claude Pro subscription, about $20 a month. You use it to drive your work with Claude Code throughout the course.
Optional: OpenRouter credit (about $10–20) for less rate-limited models.
Everything else we use is free: Promptfoo, LM Studio, the Groq free tier, GitHub and Discord.
$4,997, or $3,997 if you enrol by 3 January 2027 (early bird).
Yes. US and Canada applicants can choose Klarna instalments at checkout. It may also be available elsewhere for payments in US dollars. If Klarna isn't available or doesn't work for you, message us.
Often, yes – ask your manager about a training budget. We can send you what you need for the request. If several people on your team want to join, ask us about team enrolment.
Attend Week 1. If it isn't right for you, claim a full refund.
No. 1:1 mentorship is a separate service at $200 an hour, with any of our mentors. You don't need it to finish the program.
During the cohort you already get:
- instructors and volunteers in the live sessions;
- help in the course Discord between sessions;
- the study groups you form with classmates.
Join a later cohort, or ask us about 1:1 mentorship on the same curriculum.
23 January – 21 February 2027.
Ten live sessions on Zoom over five weekends: Saturdays and Sundays, 10:00 AM–2:00 PM PT. That's 40 hours of live sessions. There's an onboarding call on the Wednesday before Day 1.
At least 10–12 hours, including the weekend sessions and homework. Students who get the most out of it form a study group with two or three classmates and redo the exercises during the week.
A short topic introduction, then hands-on work in small breakout groups, with instructors and volunteers moving between rooms. Groups then come back to share what they built. Every session is hands-on.
Every session is recorded, and you keep access to the recordings for a year. The recordings are a safety net, though. The real value is the work you do in the live breakouts.
- Week 1: how AI fails, and your first Promptfoo evaluations.
- Week 2: prompts, cost and graded metrics.
- Week 3: red teaming, by hand and automated.
- Week 4: testing real AI apps (agents, retrieval, MCP tools) and monitoring.
- Week 5: turning your work into interview stories.
The full week-by-week plan is in the curriculum above.
Practice apps built for the course: a fintech app (Payflow), an onboarding assistant (OnboardBot) and a healthcare chatbot (MediBot). You'll test them with the same tools teams use: Promptfoo, Claude Code, Arato and GitHub Actions.
- A GitHub repository of everything you built: evaluation suites, red-team runs and AI bug reports. This is what you demo in interviews.
- A verifiable certificate of completion.
- Lifetime access to the alumni community.
Three instructors who work in QA and AI testing, supported by five volunteers in the breakout rooms.
- Windows: Windows 10 (64-bit) or newer.
- Mac: Apple Silicon (M1–M4). Intel Macs aren't supported.
- Memory and storage: 16 GB RAM recommended, 30 GB free SSD space.
- Other: a webcam and a stable internet connection.
No. 8 GB works for everything except the local-model exercises (Days 2–3), and you can point those at a cloud model instead. If you plan to make a career of testing AI, a stronger machine will help later.
No honest program can guarantee a job, and we don't. What we promise is that the effort you put in during and after the course pays off. Our graduates' stories show what that looks like.
Career work is built into the program:
- From week 2: you post about what you're building on LinkedIn.
- Final week: you turn your repo into accomplishment statements, prepare interview answers, build a short live demo, and set up a job-search routine.
The roles rarely use the words "QA" or "testing". They're called things like AI Evaluation Engineer or LLM Evaluation Engineer. Search for the skills instead – hallucination detection, grounding, bias, red teaming. We post open roles in the Discord job board, and on a free career review we can put a real posting next to the syllabus.
Yes, an optional one:
- 4 weeks, unpaid.
- Open to everyone who finishes the program and registers for it. Registration is a separate step.
- The start date is announced for each cohort.
Graduates can go on to AI Security Engineer, our live specialisation in red-teaming LLM apps and AI agents. The next one runs 6 March – 11 April 2027. Students who'd rather sharpen their automation can join the QA Mastery Program.
Start building AI testing skills
Apply for October, or book a call to check fit.
























