ClaudeProduct managementAutomationMixpanel

How I actually use AI to do product management

All work

The short version

I run product at a telehealth company. It is a regulated health product, the numbers get questioned hard, and a wrong claim in a founder update is expensive. For the last five months I have worked with Claude connected directly to the systems the work lives in: analytics, the ticket tracker, the code repo, Slack, email, the calendar and our design files.

To write this, I had the AI go back through its own record: the full list of sessions, the roughly 3,000 prompts I typed, the scheduled routines and their run history, and the connector setup. Everything below comes from that record, including the parts that broke. The split surprised me. Most of my AI sessions are not chats. They are routines that run on a schedule, check the systems, and stay quiet unless something needs me. The few that speak for me hand me drafts.

~70%of my AI sessions were scheduled routine runs, not chats
8recurring routines, from every two hours to weekly
289routine runs on my machine, late May to late October
3dated decision checkpoints, thresholds set before the data arrived

The stack

The AI is only as useful as what it can reach. Most of these are connectors (MCP servers) attached to Claude. Where a tool has no connector, the AI uses its API or command line.

What the AI is connected to
ToolWhat it does for me
MixpanelMy most-used connector by far, with several thousand queries. Funnels, experiment readouts, dashboards, property audits, anomaly checks and tracking loss.
Marketing data warehouseRead-only, and built outside my team. Joins the product funnel to ad spend and attribution. A standing rule says data questions get checked here and in Mixpanel.
SlackSearching decision history, reading release channels, and writing drafts for me to send. Custom bots handle design sign-off, ticket lookup and feature-request intake.
ClickUpNo managed connector, so the AI uses the API. Tickets, sprint health, prototype-to-backlog, a front-door intake flow and a space reorganization.
GitHubCommand line, read-only. What actually shipped, and pull request diffs read against the ticket's scope.
Gmail and CalendarAI meeting notes in my inbox are the source of truth for decisions. The calendar shows who was in the room before an action is routed.
Google DrivePRDs, SOPs, cost sheets and template decks.
Figma and FigJamPrototype references, roadmap Gantt charts, event diagrams and kickoff briefs.
PlaywrightThe QA-repo browser tests behind the tracking checks.
Built-in browserQA of per-pull-request preview environments, and setup steps that only exist in a web UI.
Claude Docs and ArtifactsDecision memos for the founders, compliance registers and shareable team pages.
Session analytics and VercelA 72-hour UX pulse (rage and dead clicks) inside the weekly update, and password-protected prototypes for demos.
Scheduled tasksThe routines. Most run locally in the desktop app; the morning brief runs in the cloud.

The weekly rhythm

My pattern is to do a job once, live, with the AI, and only then schedule it. The weekly product snapshot took about ten prompts to get right, and ended with me typing "love the tier system… let's schedule this weekly".

My recurring product routines
RoutineWhenWhat it produces
Morning briefWeekdays 7:00Meant to be one page: decisions needed, what is blocked on me, meeting prep. In practice it stalls on a permission prompt (see What broke)
Dev reviewDaily 7:00A review comment per ticket, checking the pull request against acceptance criteria and analytics events. Stale since mid-September (see What broke)
Anomaly monitorDaily 8:00Eight core metrics and two tracking-loss ratios vs the same weekday last week; a push only if something is off
Tracking parity checkDaily 9:13Tracking tests against production and QA, posted to a tracking channel
Release parity checkEvery 2 hours, 6:15 to 18:15The same test against production, only when a new release appears
Experiment monitorMon and ThuOne verdict per experiment; a push only when a decision is needed
Paid-channel funnelMondaysFunnel by acquisition channel, as a Slack draft for marketing
Product snapshotFridaysA three-tier weekly update, as a Slack draft

The weekly product snapshot

This replaced my status update, which meant reconciling six or more systems by hand. It reads closed tickets, the week's calendar, the week's AI meeting notes (1:1s excluded), merged pull requests and deploys, two Slack channels for blockers, the Mixpanel onboarding funnel and the last 72 hours of session analytics. Then it writes three tiers:

  1. A public draft. 20 to 28 lines, story first, no ticket IDs.
  2. An appendix. Decisions this week, actions by owner, and a meeting index. Actions owned by "the group" go last, flagged as a risk.
  3. Private PM notes. The sharp diagnostics, split into per-owner briefs. Checking that routing against calendar invites caught four misrouted actions in one week.

Claims from auto-generated notes are cross-checked against tickets and GitHub. One week the private notes caught a release note that the data did not back up, and an A/B split running well off its 50/50 target. I edited the draft and sent it myself.

The anomaly monitor

Its rules, summarized from the spec:

{
  "source": "server-side events (browser events only for the loss ratios)",
  "compare": "yesterday vs the same weekday last week",
  "metrics": "10: eight core metrics from leads to cancellations, plus two tracking-loss ratios",
  "flag_when": {
    "any_core_metric_moves": "more than 35% either way",
    "failures_or_cancels_rise": "more than 50%",
    "tracking_loss_exceeds": "15%",
    "core_event_is_zero": "urgent outage, listed first"
  },
  "otherwise": "one line: All normal for <date>",
  "if_connector_fails": "report it and push; never fail silently"
}

When it fires, it does the first pass of triage. One morning it flagged a sharp move across funnel and payment metrics. Unprompted, it pulled a 14-day trend, found the move had started days earlier, and confirmed browser and server events agreed: a real acquisition change, not a tracking fault. The catch: that push never reached me, because the terminal was active at the time. More on that under What broke.

Tracking parity checks

These exist because a rebuilt page once shipped with part of its tracking missing, and it took too long to notice. Data you never collected cannot be backfilled.

The AI helped write the blameless post-mortem, then a browser test (Playwright) that confirms our tracking setup survives every release. It runs daily against production and QA, and again within about two hours of a daytime production release, instead of waiting up to 19 hours for the next daily run. It tags people only when production fails, so the team does not learn to ignore the channel. Every recorded run since has been clean. The background on why browser events go missing is in building click-funnel events you can trust.

The experiment monitor

Twice a week, each running experiment gets one verdict: keep running, ready to call, guardrail regression, health check failing, stalled, or inconclusive. It never concludes or edits an experiment, and ignores variants under 30 exposures. Its most useful flag was a false alarm: a payment guardrail down 48%. Following up in the same session, the AI found the metric counted payment events, not people, so it mostly picked up recurring payments. Per person the variant was flat or ahead. The call was to keep the test running and fix the metric definition first.

Dated decision checkpoints

This is the pattern I would steal first. At launch, I write the verdict criteria into a one-time scheduled task, so the data is judged against rules set before anyone sees it:

{
  "question": "Did the portal relaunch cause the rebill gain?",
  "runs_on": "week 7 after launch",
  "baseline": "frozen pre-launch range",
  "checks": 8,
  "example_rule": "non-rebill below the bottom of the frozen baseline range supports a real gain; at or above its midpoint erases it",
  "output": "verdict, every check in a table, next steps"
}

A portal relaunch got three: week one, week seven and week ten. The week-seven verdict raised confidence that the relaunch caused the gain from about 50 to 60% to about 65 to 70%. The checkpoints also surfaced things nobody asked about, including an identity regression where, after launch, most dashboard views had lost their user ID. Ten weeks in, none of the three fixes it flagged had shipped and the identity gap had grown further. A follow-up read it proposed was never scheduled.

Working sessions

The rest is live work. I keep long-running chats by topic (repo questions, Mixpanel, weekly updates, the backlog, launches), each with roughly 190 to 270 prompts, so context builds up instead of restarting.

What my hands-on prompts are about (approximate, from keyword tagging)
Kind of workShare
Product analytics~24%
Prototyping and design review~17%
Stakeholder writing~13%
Codebase and release questions~12%
Backlog and operating system~11%
QA and tracking integrity~7%
Strategy documents~5%

Analytics I can defend

I also put server and browser events side by side on the same Mixpanel board on purpose (never in the same funnel), "because we get ad-blocked on client side". The gap became the tracking-loss metric the anomaly monitor watches.

The codebase as the source of truth

One chat, about 200 prompts deep, exists only to ask the code and release history how the product behaves. Doctors, patient services and marketing send me questions, such as whether cancelling one plan cancels the others or which states have different intake requirements, and I forward them word for word. Release questions go the same way. As I put it in one prompt about tracking for a new product: "never assume when it comes to onboarding".

Prototypes, writing and the backlog

How I brief it

Guardrails

Approval pop-ups are not where the safety lives. The guardrails that matter are written into prompts, routine specs, tools and repo instructions. Unattended routines run on a fixed allowlist: read tools, Slack drafts and one alert channel.

What broke

What it changed

I cannot give you an honest hours-saved number, so I will not invent one. The record shows decisions made on firmer ground, a tracking incident turned into permanent monitoring, and first passes that now run without anyone doing them by hand: the status update, the morning metrics check, experiment hygiene, the channel report and tracking checks after deploys. Some are young: the experiment monitor and channel report have only run a handful of times. It also clarified what should not be mine. A QA coverage audit ended with me concluding that test automation belonged with engineering.

If you are setting this up

  1. Connect the systems first. Without them you are asking a smart stranger to guess.
  2. Do the job once live, then schedule it. The first session is the spec.
  3. Write "why this exists" into every routine. Name the incident behind it.
  4. Start draft-only. Post directly only for machine status.
  5. Set decision rules before the data arrives.
  6. Turn every finding into a ticket with an owner. A flag nobody owns is just a log line.
  7. Distrust a quiet routine, and pre-allow what unattended runs need.
  8. Fix your tracking first. AI on broken events gives wrong answers faster.

The takeaway

The useful version of AI in product management is not a smarter chat window. It is small, well-specified routines connected to the real systems, draft-only by default, plus the habit of making the AI prove its numbers before you repeat them.

The routines do the gathering. I still make the calls, and anything in my voice still waits for me to press send.

Work with us

Want to run product like this?

I help product teams connect AI to the systems their work lives in, design routines that are safe to run unattended, and build the habits that keep the numbers honest.