Skip to main content

Become the Most Wanted Testers of the AI Era

Learn how to Test AI Agents, complex RAG systems, custom and commertical LLMs

Our alumni work at

Meta Platforms Inc. Logo 1Frame 2147237545Frame 2147237542Frame 2147237547Frame 2147237546Frame 2147237544Frame 2147237543Frame 2147237541Frame 2147237539Frame 2147237537Frame 2147237538Frame 2147237536

Everything you’ll use at EnGenious

13 tools · 4 workflows

  • 01 Evaluate

    • Promptfoo
    • Agenta
    • Arato.ai
    • OWASP
  • 02 Models

    • OpenAI API
    • Anthropic API
    • ChatGPT
    • LM Studio
  • 03 Build

    • Python
    • Node.js
    • GitHub
  • 04 Community

    • Zoom
    • Discord

What you’ll be able to test

Practised on a real AI product

  • Test LLM-powered applications

    Chatbots, copilots and RAG apps, end to end

  • Design and run AI evaluations

    Promptfoo suites with clear pass/fail criteria

  • Catch hallucinations and quality failures

    Spot confident wrong answers before users do

  • Prompt injection and adversarial testing

    Red-team against the OWASP Top 10 for LLMs

  • LLM-graded assertions

    Use a model to grade another model’s output

  • Bug reports for AI failures

    Write findings an engineer can act on

  • Problem-solving

    Break complex AI failures into testable parts

  • Critical use of AI tools

    Know when to trust the output, and when not to

  • Structured thinking

    Turn messy model behaviour into clear test cases

  • Adaptability

    Keep pace as models and tools change

The Engenious University method to take you from:

Traditional testertoAI QA testing engineer

  1. Learn how AI fails

    Learn the seven ways AI breaks in live sessions with engineers who test it for a living

  2. Prove your AI testing skills

    Be ready when your team needs someone to test AI or your next interview calls for AI experience.

  3. Stay relevant + earn more

    Test a product in production, write your own evaluation report, and get code reviews from QA AI engineers

157+ QA professionals trained

Ask them yourself — most are still in our Discord

  • Todd Groudan

    Experienced QA Leader & Business Analyst

    As a QA engineer moving into AI evaluation, I found this bootcamp gave me a clear picture of why QA matters so much across the AI software lifecycle. It covered red teaming, validating non-deterministic output, and catching hallucinations, and all of it was hands-on with real tools like Promptfoo, LM Studio, Agenta and Arato. Igor, Greg and Amanda knew the material deeply, broke complex topics into easy-to-follow concepts, and were always patient and willing to help. The career prep in the final week was a great bonus. This course took the mystery out of AI testing for me, and I'm excited to see what comes next. I highly recommend it to anyone who wants to move into AI evaluation but feels hesitant, and to anyone who wants to keep growing their career.

  • Sabri Yusubzada

    QA Engineer | SDET | AI Security & LLM Testing

    This course helped me see AI testing from a broader perspective and gave me the confidence that I can do it, all in a student-friendly environment that makes you feel comfortable

  • Shak Schiff

    AI Risk & QA

    This course combined hands-on exercises with expert instructors who clearly know the field, giving us plenty of practice building and testing AI applications. It also pushed a mindset shift: instead of expecting one correct answer from an LLM, you learn to evaluate probabilistic outputs by setting thresholds and acceptable ranges.

  • Valeriia Sviridova

    Senior QA Engineer | SDET | Test Automation | AI-Driven Testing

    This course exceeded my expectations! It was interactive from day one: every session had us actually doing the work, not just watching slides, and I came away with skills I could actually apply, not just notes. Highly recommend to anyone in QA looking to move into AI testing

  • Corey Molloy

    QA Engineering Leader | AI-Enabled Quality Engineering

    Going into this course, one of the biggest challenges for me was figuring out what areas to focus on for the new AI tech I needed to upskill my QA engineering career. Engenious solves that by using real industry insight on what new skills companies are actually wanting and how that intersects with the software testing ecosystem. So, I was no longer guessing at what matters. On top of that, it's hands-on learning with real instruction and small class sizes. We got into actual tooling: Promptfoo, red-teaming, and the 7 core challenges framework. That combination is what made it stick.

  • Park Kharuehadech

    AI/LLM Quality Engineer | AI Red Team Specialist

    Taking this course was one of the best decisions I’ve made. Learning about tools like Promptfoo, red teaming, and Arato was truly eye-opening, and the course gave me practical, hands-on experience through the internship program as well. The instructors are incredibly knowledgeable, supportive, and always willing to help. I’d highly recommend this course to anyone looking to grow their skills and accelerate their career

  • Mariia Abramiuk

    QA Automation Engineer | Healthcare Test Automation

    Really glad I took this course. The instructors clearly know their craft and do a great job guiding you through complex material in a way that actually sticks. We covered adversarial testing, bias detection, red teaming, and multi-model evaluation using real tools like Promptfoo, Agenta AI, and Arato AI — hands-on, not just theory. Highly recommend for anyone serious about AI testing.

  • Richard Cottrell

    Workday Testing Expert | Exploring how to test AI/LLMs

    This 5-week course is an absolute masterclass for anyone transitioning from traditional QA into the world of AI and LLMs. It moves far beyond the hype to tackle the genuine technical challenges of testing non-deterministic solutions, providing a robust framework for building a real-world test strategy. What made this course stand out: 1. Deep-Dive Tooling: We got hands-on experience with Promptfoo and LM Studio, learning exactly how to leverage them for measurable results and why. 2. Red Teaming & Strategy: Moving from theory to the actual execution of Red Teaming was a highlight, as was the focus on the financial cost of different using different LLMs and testing approaches. 3. Practical 'Learn-See-Do' Format: The structure was excellent. Every session followed a clear 'concept demonstration group exercise' flow. Working through these concepts in breakout groups ensured the knowledge actually stuck. 4. Total Support: The flexibility of having recorded sessions was a lifesaver for busy weeks and having the tutors available via Discord throughout the 5 weeks meant no question went unanswered. If you want to understand how to actually validate AI performance and manage the risks of non-deterministic systems, this is the course you need. Highly recommended!

  • Caleb Byrne

    AI & LLM Testing Engineer | SDET / QA Automation

    This course was so much more than just learning foundational concepts to test AI models. From Jaime's AI excellence and experimentation tips to Amanda's knowledge, education prowess and love for learning to Igor's candid tips for selling yourself, visionary forward thinking about where testing is going in the future, this course exceeded my expectations and I grew a lot in my testing ability, knowledge and understanding on how to find a better job.

  • Alex Teymurazov

    Engineering Leader | Quality & Delivery Excellence

    By far one of the most valuable courses I’ve taken. The format stands out because it’s not lecture-heavy, instead it’s predominantly hands-on, which in my experience is the most effective way to master new skills and technologies. The sessions provided not just theoretical knowledge, but real opportunities to apply concepts directly, from AI testing and LLMs to prompt evaluation and red teaming. This course is well worth investment for anyone looking to get practical experience in AI and LLM testing.

  • Anna Morgan

    Test Engineer at Doctor On Demand

    Great course! Complete eye opener for AI and LLM models. Course is very interactive and 80% of the time was devoted to actual hands on testing and developing skills starting from basic models to more complicated ones. Completing the course also gave more confidence while changing traditionally seasoned QA mind to QA AI Tester and inspired to continue learning in the new direction and search for new job.

  • Sharon O'Boyle

    Software Tester | IBM Champion Learner Gold

    An incredible learning journey! I loved learning how to test LLMs through practical exercises in group sessions and hands-on, real app testing. It's also been super valuable to get insights into what skills are really needed right now in the industry and how to prepare for interviews. The instructors are super passionate about what they do and genuinely care about everyone's progress. I finished with valuable new skills and a community of like-minded peers who continue to support and inspire me.

Skills you’ll leave with

One target role, the skills it asks for, and the proof that you have them.

Target role

Post-program

AI Evaluation Engineer

The role the program is built backwards from

$98K–$230K+

$98KUS market range$230K+

Knowledge

  • Evaluate LLM apps
  • Build eval suites in Promptfoo
  • Red-team against the OWASP LLM Top 10
  • Write bug reports engineers can act on

Proof of work

Verified

AI Application Testing Portfolio

EnGenious University

They started where you are

One target role, the skills it asks for, and the proof that you have them.

“Testing AI and everything that we've been learning so far are very different. The reason why I am excited to tell you this is not because I'm super smart. It's because AI is smart and we learned how to use it. My message is if I can do it, you can do it.”
Dennis NikolayenkoDennis Nikolayenko on LinkedIn (opens in a new tab)Prompt Evaluation Engineer @ Meta

Who You Become, Week By Week

Understand what you're testing

Learn how AI apps work and fail, set up your workspace, and run your first AI tests.

Day 1: How AI Works and How It Fails

Duration: 3–4 hours

  • See how an AI app is built – the provider, the model, tokens and the context window – and where each part can fail.
  • Learn why the same question can get a different answer every time, and what that means for testing.
  • Set up your workspace with Claude Code and run your first Promptfoo test, with no programming needed.

Tools:

  • Claude Code
  • Promptfoo
  • VS Code

Day 2: Run Your First AI Test Suite

Duration: 3–4 hours

  • Choose models to test using leaderboards and Hugging Face.
  • Build a Promptfoo test file and run it on several models side by side.
  • Write exact checks and your first AI-graded checks on a financial chatbot.

Tools:

  • Promptfoo
  • Groq
  • Hugging Face

Define what a good AI answer looks like

Turn "that answer looks fine" into checks you can run, measure and explain.

Day 3: Instructions, Cost and Local Models

Duration: 3–4 hours

  • Treat the app's instructions as the spec and user questions as test cases, and use AI to generate tests.
  • Compare models on cost, speed and pass rate.
  • Run a model on your own computer and test it with the same suite.

Tools:

  • Promptfoo
  • LM Studio
  • OpenRouter

Day 4: Use AI to Grade AI

Duration: 3–4 hours

  • Write scoring rubrics so a second model can grade answers that have no single correct wording.
  • Set weights and pass marks, and measure results with precision, recall and F1.
  • Fix deliberately broken test suites.

Tools:

  • Promptfoo
  • Claude Code
  • GitHub

Try to break AI on purpose

Think like an attacker: find where an AI app gives in, and report what you find.

Day 5: Red Teaming by Hand

Duration: 3–4 hours

  • Learn what red teaming is and the rule: test only systems you're authorised to test.
  • Tell a weakness in the model from a weakness in the app built on it.
  • Write your own attacks on a healthcare chatbot and run them on several models.

Tools:

  • Promptfoo
  • Claude Code

Day 6: Automated Attacks and AI Bug Reports

Duration: 3–4 hours

  • Run ready-made attack sets (OWASP Top 10 for LLMs, NIST, MITRE ATLAS) against an AI app.
  • Test a chatbot the way a user would, then again with its instructions in front of you.
  • Write AI bug reports with how often the failure happens, and run your tests automatically with GitHub Actions.

Tools:

  • Promptfoo
  • GitHub Actions

Test a full AI app and keep watching it

Move from testing a model to testing a complete app, then learn how teams watch AI after release.

Day 7: Test an App, Not Just a Model

Duration: 3–4 hours

  • Test a multi-agent fintech app: check it sends each question to the right agent and cites the right documents.
  • Write tests for conversations that run over several messages, including an attack hidden a few messages in.
  • Test the tools of an MCP server with positive and negative cases.

Tools:

  • Promptfoo
  • MCP
  • Claude Code

Day 8: Access, Bad Data and Monitoring

Duration: 3–4 hours

  • Check who can see what, using the OWASP Top 10 for LLM apps.
  • See what one poisoned document does to an app's answers.
  • Send your test runs to Arato and compare each release against a baseline.

Tools:

  • Arato
  • OpenTelemetry
  • Promptfoo

Test agents and build proof you can test AI

Test an AI agent that takes actions, then turn your work into proof an employer can see.

Day 9: Test an AI Agent

Duration: 3–4 hours

  • Test a support agent that uses tools: check what it actually did against the real data, not just what it said.
  • Catch instructions hidden in data, poisoned memory, wrong tool data and failures that only show up on some runs.
  • Manage prompts and tests in Agenta, and test a remote MCP server.

Tools:

  • Promptfoo
  • Agenta
  • MCP

Day 10: Turn Your Work into Interviews

Duration: 3–4 hours

  • Write accomplishment statements built on measurable results from your own tests.
  • Prepare answers to AI testing interview questions, with a short live demo from your repo.
  • Set up a LinkedIn routine to show your work and connect with hiring teams.

Tools:

  • GitHub
  • LinkedIn

Break into AI Testing is for if:

  • You want a higher-paying role

    Companies pay more for testers who can tell them whether their AI is safe to ship

  • You want proof and confidence, not another certificate

    A test suite you built, a red-team report, bugs you found yourself. Work you can point to in an interview or a performance review.

  • AI just landed on your desk

    Your team is shipping a chatbot or AI feature, and the testing is falling to you with no playbook. 5 weekends from now, you’ll have one

  • You want to stay relevant as AI changes QA

    AI is changing what testers are hired to do. You want to be the one teams rely on to test it, not the one wondering where the work went.

  • You want to become one of the most wanted testers of the AI era

    You’re a manual tester, SDET, automation engineer or software engineer and now you’ll know how to test AI, and that combination is what every team shipping AI is looking for.

All you get after signing up

A 4-week internship inside a US AI startup

Apply from inside the cohort and practise everything you learned on a product that has users. “AI Testing Intern” goes on your LinkedIn the day you start.

Access to the community

1,000+ people chasing the same thing you are, in one place — trading what’s working in AI testing, passing along openings, building the network that gets you the next role.

Your proof of work: an eval report and a GitHub

Both with your name on them, both openable by a hiring manager in thirty seconds. The certificate is the least interesting thing you take with you.

Breakout groups in every session

Small group, one volunteer lead, and their only job that session is answering your questions. Nobody drifts quietly to the back.

Every recording and material, for a year

Each session, plus the configs, prompts, rubrics and test files we build together. Come back to any of it whenever you need it.

You are never alone

Our team is reachable whenever you’re stuck. During the five weekends, during the internship, and after. We stay until the outcome.

One training. Everything included.

Cohort January 2027

$3,997

Save $1,000

Regular price $4,997

Choose how you pay

100% money-back through Week 1, no questions asked

Secure checkout

CEO/Founder of Engenious.io

Learn from The Best

“15+ years of experience in software testing and development and over a decade in management.”
in management
10+ yrs
Engenious.io
Founder
software testing & development
15+ yrs

January cohort is open

Pacific Time

Ten sessions. Five weekends.

Every Saturday and Sunday, 10:00 AM – 2:00 PM. Miss one? Every session is recorded.

  1. Weekend 1

    Upcoming

    • SAT

      Jan 23

      10:00 AM – 2:00 PM Pacific Time

    • SUN

      Jan 24

      10:00 AM – 2:00 PM Pacific Time

  2. Weekend 2

    Upcoming

    • SAT

      Jan 30

      10:00 AM – 2:00 PM Pacific Time

    • SUN

      Jan 31

      10:00 AM – 2:00 PM Pacific Time

  3. Weekend 3

    Upcoming

    • SAT

      Feb 6

      10:00 AM – 2:00 PM Pacific Time

    • SUN

      Feb 7

      10:00 AM – 2:00 PM Pacific Time

  4. Weekend 4

    Upcoming

    • SAT

      Feb 13

      10:00 AM – 2:00 PM Pacific Time

    • SUN

      Feb 14

      10:00 AM – 2:00 PM Pacific Time

  5. Weekend 5

    Upcoming

    • SAT

      Feb 20

      10:00 AM – 2:00 PM Pacific Time

    • SUN

      Feb 21

      10:00 AM – 2:00 PM Pacific Time

Frequently asked questions

Here, you'll find answers to the most commonly asked questions

Start building AI testing skills

Apply for October, or book a call to check fit.

Break Into AI Testing

Ask question