QA Test
Run live QA and end-to-end tests for provider validation and app configuration.
Installation
- Make sure Claude is on your device and in your terminal.
Skills load from
~/.claude/skills/when Claude Code starts up — so you need it on your machine first. If you don't have it yet, install it once with the command below, then runclaudein any terminal to verify.One-time setupnpm i -g @anthropic-ai/claude-codeAlready have it? Skip ahead.
- Paste into Claude Code or into your terminal.
This copies the whole skill folder into
~/.claude/skills/qa-test-steipete/— the SKILL.md plus any scripts, reference docs, or templates the skill ships with. Safe default: works for every skill.Faster alternative (instruction-only skills)
Skips the clone and grabs only the SKILL.md file. Don't use this if the skill ships Python scripts, reference markdowns, or asset templates — they won't be downloaded and the skill will fail when it tries to load them.
Quick install (SKILL.md only)Sign up to copy - Restart Claude Code.
Quit and reopen Claude Code (or any other agent that loads from
~/.claude/skills/). New skills are picked up on startup. - Just ask Claude.
Skills auto-activate when your request matches the skill's description — no slash command needed. Trigger phrases live in the skill's own frontmatter; you can read them in the “What this skill does” section above.
Prefer to read the source first? Open on GitHub.
When Claude uses it
CodexBar live QA/e2e testing: run provider usage matrix checks, validate real app config, use Peekaboo for menu proof, use Browser Use/official docs for API spec or logged-in dashboard checks, and handle 1Password credentials safely.
What this skill does
CodexBar Live QA
Use for live provider testing, release smoke tests, menu verification, or debugging “provider works/fails” reports.
Rules
- Work from the CodexBar repo checkout.
- Use the packaged CLI first:
CodexBar.app/Contents/Helpers/CodexBarCLI. - Do not use
CodexBar.app/Contents/MacOS/codexbar; that is the app binary and may appear to hang as a CLI. - Never run broad
env,set, or secret regex dumps. - Use
$one-passwordfor secrets: allopcommands inside one persistent tmux session, service account first, no raw secret output. - Treat browser-cookie/keychain flows as prompt-risky. Prefer CLI/API-token checks and
KeychainNoUIQuery-safe tests unless the user explicitly requested live UI. - For current API behavior, browse official provider docs only.
CLI Matrix
Run the bundled script:
.agents/skills/qa-test/scripts/live_provider_matrix.sh --enabled
Useful modes:
.agents/skills/qa-test/scripts/live_provider_matrix.sh --provider all
.agents/skills/qa-test/scripts/live_provider_matrix.sh --providers openai,zai,deepseek
.agents/skills/qa-test/scripts/live_provider_matrix.sh --default
Interpretation:
--enabledasksCodexBarCLI config providersfor enabled providers, honoringCODEXBAR_CONFIGand default toggles.--defaultruns the app-facing default command with no provider override.--provider allforces every registered provider and is expected to fail for providers without sessions/keys.- A green app config needs
--enabledand--defaultclean;--provider allis a discovery/triage tool.
Config QA
Validate config:
CodexBar.app/Contents/Helpers/CodexBarCLI config validate
stat -f '%Lp %N' "$HOME/.codexbar/config.json"
Redact config shape:
jq '(.providers // []) |= map(.apiKey = (if .apiKey then "<redacted>" else .apiKey end) |
.secretKey = (if .secretKey then "<redacted>" else .secretKey end) |
.cookieHeader = (if .cookieHeader then "<redacted>" else .cookieHeader end) |
(if .id == "stepfun" and has("region") then .region = "<redacted>" else . end) |
.tokenAccounts = (if .tokenAccounts then (.tokenAccounts | .accounts = (.accounts | map(.token = "<redacted>"))) else .tokenAccounts end))' \
"$HOME/.codexbar/config.json"
Before editing config, make a backup:
cp "$HOME/.codexbar/config.json" "$HOME/.codexbar/config.pre-qa-$(date +%Y%m%d%H%M%S).json"
chmod 600 "$HOME/.codexbar"/config.pre-qa-*.json
Live Menu QA
Use Peekaboo after CLI checks:
pkill -x CodexBar || pkill -f 'CodexBar.app/Contents/MacOS/CodexBar' || true
open -n "$PWD/CodexBar.app"
peekaboo menu list-all --json | rg -i 'codexbar'
peekaboo menu click-extra --title codexbar-merged --json
screencapture -x /tmp/codexbar-live-menu.png
Crop top-right menu if needed:
sips --cropToHeightWidth 900 340 --cropOffset 20 2650 /tmp/codexbar-live-menu.png \
--out /tmp/codexbar-live-menu-crop.png >/dev/null
Verify visually with view_image. Confirm provider tabs/rows match enabled config and no failing provider dominates the first screen.
Browser Use
Use $browser-use only when a logged-in dashboard, API key page, or provider docs need browser/profile state.
Existing Chrome path:
mcporter call chrome-devtools.list_pages --args '{}' --output text
mcporter call chrome-devtools.navigate_page --args '{"url":"https://provider.example"}' --output text
mcporter call chrome-devtools.take_snapshot --args '{}' --output text
If Browser Use is unavailable, say so and use web search for public official docs; do not substitute isolated Playwright for login/profile-dependent pages.
Fix Triage
- Missing auth/session: configure key/session if available; otherwise leave provider disabled or report blocked auth.
- Wrong provider API/spec: inspect official docs, then patch fetcher/settings/tests.
- Provider key exists but live API rejects it: keep key stored if useful, disable provider if the menu would show a persistent error.
- User-facing behavior changes need
CHANGELOG.md. - Code fixes need focused tests,
make check,$autoreview, and live CLI proof before landing.
Known CodexBar QA Notes
- OpenAI Admin API key is the useful usage provider key. Project
OPENAI_API_KEYvalues can fail legacy credit-balance fallback with 403. - Deepgram usage requires a key/project with Management API permissions; transcription-only keys can return 403.
- Groq usage uses the Prometheus metrics API, not ordinary inference endpoints.
- MiniMax pay-as-you-go API keys and Token Plan/Coding Plan keys are different; wrong key kind can leave usage unavailable.
Related skills
Documentation Co-Authoring
anthropics
Guide structured workflows for writing docs, proposals, and technical specs collaboratively.
MCP Server Builder
anthropics
Build protocol servers that connect language models to external APIs and services.
Skill Builder & Optimizer
anthropics
Create, edit, and optimize Claude skills with performance testing and benchmarking.
Multi-Component Web Artifacts
anthropics
Build complex React artifacts with Tailwind CSS and shadcn/ui components.