Skip to main content
CertLighthouse

Our method

How we write questions

Every CertLighthouse question maps to one exam objective, cites a primary source for its answer, and is approved by a human reviewer. This page explains each step and the rules behind it.

Written to the objectives

CompTIA publishes the objectives for each exam: the domains, their weights, and the topics in scope. Every question we write traces to exactly one of those objectives, and nothing outside them gets written.

Where an objectives document sets a limit, such as a topic listed as good practice but not tested, we record it as a rule, and questions that break it are rejected. The app shows each objective's ID with our own short description and links to CompTIA's page for the official document.

A primary source behind every answer

The correct answer to every question must be supported by at least one source from the first two groups below, linked to the most specific section available. CompTIA's own pages tell us what's tested, so they never count as proof on their own.

Standards and registries

Protocol behavior, numbers, and definitions

IETF RFCs, the IANA registries, NIST, CISA, and standards bodies such as the Wi-Fi Alliance and USB-IF

Vendor documentation

How an operating system, tool, or command behaves

Microsoft Learn, Apple Support, Android and Google support pages, Linux man pages, Cisco configuration guides

CompTIA's exam pages

What's in scope, never what's true

The exam pages and objectives on comptia.org

Never used as evidence: other prep vendors, Wikipedia, YouTube, Reddit, forums, blogs, and AI-generated sites. In the app, a source appears as its title, its publisher, and our one-line summary of what it supports, with a link to read the original.

A clean room for exam security

CompTIA prohibits using or distributing live exam content and warns that study material produced with AI tools can come out too close to real exam items. We take that seriously, so question writing runs under strict limits on what goes in.

The only inputs are the objectives, text fetched from the allowed sources above, our style guide, and the question format. Drafting tools are never asked what's on an exam, and no dumps, recalled questions, forum posts, or other vendors' question banks go anywhere near the process. An automated check blocks any drafting instruction that mentions exam recall.

A first draft may come from a language model working only from those inputs. That draft is never published as it is: it has to pass the automated checks below and then human review.

Every question keeps a record of where it came from: the drafting model and instruction version, the exact sources it was given, the checks it passed, and who approved it and when.

Automated checks

  1. 1

    Structure

    The question has everything it needs: an exam, an objective, a key, a reason for every option, and evidence.

  2. 2

    Evidence

    Each cited passage exists in the source, and a separate check confirms it supports the claim attached to it.

  3. 3

    Independent solve

    The question is answered again from the sources alone, without the key. If that answer differs, or a second option looks defensible, the question is rejected.

  4. 4

    Scope

    The question fits its objective and breaks none of the objective's scope rules.

  5. 5

    Style

    No "all of the above," qualifiers like BEST and FIRST in capitals, options of similar length, and explanations within their length limits that never refer to option letters.

  6. 6

    Duplicates

    Near-duplicates of existing questions are merged or dropped.

Human review

Nothing reaches learners until an approved reviewer signs off. Reviewers hold the certification they review for or teach the course, and each is approved for specific exams. The review screen shows the question exactly as a learner will see it, with its explanation once for each option, so approving a question approves every explanation it can show.

A reviewer who changes the answer key, or whose edit makes a second option defensible, sends the question back through the automated checks before it can be approved.

If something feels too familiar

Any reviewer who thinks a question is too close to something from a live exam rejects it with a code that means “discard, no details.” The question is retired and nothing about the resemblance is asked for or recorded, because describing live exam content would break CompTIA's candidate agreement.

After a question is published

Every question has a Report an issue button, and reports go straight into the review queue with a 72-hour response target. When we change an answer key, it goes on the public corrections log with the reason.

Every source link is checked nightly. If a page disappears or changes in a way that matters, its questions are pulled for a recheck.

Once learners are using the app, we track how each question performs: how often it's answered correctly, whether stronger learners do better on it, and which wrong options draw the most picks. Questions that look too easy, too hard, or misleading go back to review.

We never ask learners about the content of a real exam. After your exam date, the app asks only whether you took it, whether you passed, and optionally your score.

See where it shows up

The evidence and reasoning behind each answer appear on the explanation screen after every question.