Methodology
This document defines how every tool on aicoachingapps.com is classified and scored. Classification is facts. Scoring is judgement, bounded by a rubric and backed by cited public evidence. The two are kept separate on purpose: facts can be checked against the vendor's own site, scores can be argued with.
Version 2 (2026-10-13): the dimensions are about what a coach does for an athlete. Device support moved from a score to a fact (the partner matrix), and an explicit AI level separates conversational AI coaches from rule-based incumbents without pretending one is automatically better.
Rules that apply everywhere:
- Nothing is fabricated. A fact or score with no public evidence is
nulland is shown as "unverified" on the site. An empty cell is better than a guess. - Every fact and every score cites a URL. Classification facts cite the tool's own website (pricing page, features page, help centre, app store listing). Scores cite the page that justifies the score.
- Same rubric for everyone. YOUB is one entry and is scored exactly like the others, by the same rubric, from the same kinds of public evidence. No entry gets copy that the others do not get.
- Dates matter. Every evidence citation records the date it was checked, so stale facts can be found and re-verified.
Part 1 — Classification (facts)
Facts are what the tool is. They carry no score. Each field has a source URL on the tool's own site (or app store listing). If the vendor does not state it, the field is null.
1.1 Basics
| Field | Type | Allowed values / notes |
|---|---|---|
sports | list | running, cycling, triathlon, swimming, trail, hiking, strength, other. Every sport the tool builds plans for. |
platforms | list | ios, android, web. Only platforms with a first-party app or site. |
ai_level | enum | See 1.2. |
plan_type | enum | adaptive (the plan changes based on what you did), static (a fixed plan generated once), chat_only (advice via chat, no calendar of planned sessions), hybrid (adaptive plan plus chat coach). |
pricing.model | enum | subscription, one_time, free, freemium, bundled (included with hardware or another service). |
pricing.monthly / pricing.annual | number | Price as listed, in pricing.currency (ISO 4217). null if not sold that way. |
pricing.free_tier | enum | none, trial (time-limited), limited (permanent reduced tier), full (the product is free). |
pricing.trial_days | number | Length of the free trial in days. |
pricing.tiers | list | Optional per-tier name, price and what it includes, only when the vendor states them. |
languages | list | ISO 639-1 codes of UI languages. |
company, country, founded | string / string / number | Vendor name, HQ country (ISO 3166-1 alpha-2), year the product launched. |
1.2 AI level
How coaching decisions are made, from the vendor's own documentation. This is the field that separates a conversational AI coach from an incumbent adaptive plan. It is a fact, and it is not a score: a rule-based plan can out-score a conversational coach on plan quality or value.
| Value | Meaning | Evidence that qualifies |
|---|---|---|
static | A plan generated once from a template or questionnaire. It does not change from what the athlete does. | Vendor describes plans as fixed, or offers only "choose a plan". |
rule_based | The plan re-flows or scales by fixed rules: missed session, RPE, completed volume, a weekly fitness estimate. Typical of watch-maker coaches and template-based running apps. | Vendor describes adaptation in terms of rules and triggers, without a physiological or learned model. |
model_based | A physiological or statistical model (fitness-fatigue, critical power, machine learning on the athlete's history) decides load and session type. No free-text conversation. | Vendor documents the model or publishes its method. |
conversational | An LLM-based coach the athlete talks to in free text; it can explain, negotiate and change the plan, usually on top of a model- or rule-based planner. | Vendor documents a chat coach that can modify the plan, not just answer FAQs. |
When a product combines tiers, record the highest tier the vendor documents and let the capability matrix (1.3) carry the detail.
1.3 Coaching capabilities
What the coach does for the athlete. Three-level facts, not scores: a vendor page can support "it exists", not "how well". Values:
full— present as described in the "full means" column.partial— exists but generic, manual-trigger, add-on, or limited to one sport.none— the vendor states it is absent, or the feature list is explicit enough to exclude it.null— unverified.
| Key | Question | full means |
|---|---|---|
plan_building | Builds a periodized plan to a goal and date? | Plan generated from goal, date, availability and current fitness, with phases. |
workout_prescription | How precise is each session? | Interval-level targets (pace, power or HR) executable on a device. |
race_preparation | Race-specific work, taper, pacing? | Race blocks, taper and a pacing or race-day plan for the target event. |
life_adaptation | Re-plans around life? | Missed or moved sessions and availability changes re-flow the plan automatically. |
fatigue_adaptation | Reacts to fatigue and readiness? | Load changes from HRV, sleep, resting HR or subjective feel without the athlete asking. |
injury_illness | Handles sick or hurt athletes? | Pause and return-to-training progression, not just skip. |
post_workout_feedback | Explains what a completed session meant? | Per-session analysis referencing the athlete's data and the plan's intent. |
conversation | Can the athlete talk to the coach? | Free-text chat that can answer questions and change the plan. |
accountability | Does it come to you? | Scheduled check-ins, debriefs or reminders initiated by the app. |
human_coach | Is a human in the loop? | A human coach reviews or messages, included or as an add-on. |
strength_mobility | Strength in the plan? | Prescribed strength or mobility sessions scheduled alongside endurance work. |
nutrition_fueling | Fueling guidance? | Session- or day-specific fueling numbers. |
multisport | One plan across sports? | Balances load across sports in a single plan. |
Rendered in four groups: Plan (plan_building, workout_prescription, race_preparation), Adapt (life_adaptation, fatigue_adaptation, injury_illness), Relationship (post_workout_feedback, conversation, accountability, human_coach), Beyond endurance (strength_mobility, nutrition_fueling, multisport). "Communication style" is deliberately not a field: it cannot be verified from a vendor site.
1.4 Partners
Which devices, apps and platforms work with the tool, from a fixed vocabulary. A partner that is not listed is unverified, not "no".
| Kind | Partners |
|---|---|
| Watches | garmin, coros, polar, suunto, apple_watch |
| Bike computers | garmin_edge, wahoo_elemnt, hammerhead |
| Indoor trainer apps | zwift, rouvy, trainerroad, wahoo_systm |
| Training platforms | strava, trainingpeaks, intervals_icu |
| Wellness and health data | oura, whoop, apple_health, google_fit |
| Smart trainers | smart_trainer_erg (the app itself drives a trainer over BLE/ANT+) |
Each link has a direction: reads_activities, pushes_workouts, both, wellness_only, or none (the vendor explicitly states no support), an optional wellness: true on a watch link when sleep/HRV/RHR are read from it too, a required source URL and a checked date. Set garmin_edge only when the vendor names Edge or bike computers; "Garmin Connect" alone sets garmin.
Each fact object carries a source (URL) and checked (ISO date):
"ai_level": { "value": "conversational", "source": "https://example.com/how-it-works", "checked": "2026-10-13" }Part 2 — Scoring (judgement, 1–5 per dimension)
Each dimension is scored 1–5 by a reviewer from public evidence. Scores are integers. The rubric says what a 1, a 3 and a 5 look like; 2 and 4 are in-between. A dimension with no usable evidence is null, not 3.
Evidence tiers, strongest first:
- T1 Independent hands-on review (e.g. DC Rainmaker, a long-term user write-up with screenshots) or a documented methodology page that describes the mechanism, not just the claim.
- T2 Athlete discussion on the intervals.icu forum, reddit (r/running, r/cycling, r/triathlon, r/Velo, r/AdvancedRunning), Slowtwitch, the TrainerRoad forum, app-store reviews.
- T3 Vendor claims (feature pages, blog posts, changelogs).
Constraints:
- A 5 requires at least one T1 or T2 source. Vendor claims alone cap a score at 4.
- A 1 also needs evidence (a review or thread showing the shortcoming), not merely absence of a feature page.
- Athlete sentiment must cite at least two independent threads or reviews, or it stays
null. App-store star ratings alone do not qualify.
Each score object looks like:
"adaptation": {
"score": 4,
"evidence_url": "https://...",
"evidence_note": "One sentence on what the source shows.",
"checked": "2026-10-13"
}2.1 Plan quality
Zones, periodization, intensity distribution, load modelling.
| Score | Looks like |
|---|---|
| 1 | Generic templates. No individual zones; no stated periodization model; intensity distribution unknown. |
| 3 | Individual zones from a test or estimated threshold; a stated periodization model (base/build/peak/taper); sensible polarised or pyramidal distribution; race-specific blocks. |
| 5 | The above, plus explicit load/recovery modelling (fitness–fatigue, durability or VO2max estimates), published methodology or references, and independent reviewers who confirm the plans match the stated science. |
2.2 Adaptation
Does the plan react to what actually happened, to life, to fatigue and to illness?
| Score | Looks like |
|---|---|
| 1 | A fixed plan. Missed or extra sessions change nothing. Re-planning means starting over. |
| 3 | The plan re-flows after missed or moved sessions and adjusts volume or intensity from completed-workout data. Reaction is rule-based and per session. |
| 5 | Continuously re-plans from completed and missed sessions, subjective feedback and wellness signals (HRV, sleep, resting HR, illness, life events), with documented behaviour confirmed by independent users. |
2.3 Workout feedback
How precise is the prescription, and what does the athlete learn afterwards?
| Score | Looks like |
|---|---|
| 1 | Sessions are distance and effort only; nothing is said after a session. |
| 3 | Structured intervals with individual targets; a per-session summary states whether the target was hit. |
| 5 | Device-executable targets, and a post-session debrief that explains what the session did for the athlete relative to the plan, confirmed by independent reviewers. |
2.4 Coaching relationship
Can you talk to it, does it come to you, is a human there when needed?
| Score | Looks like |
|---|---|
| 1 | No conversation; at most an app-FAQ bot. |
| 3 | A coach chat that answers in context of the athlete's data and makes simple plan changes on request ("move my long run to Sunday"). |
| 5 | The coach initiates (check-ins, debriefs), negotiates trade-offs, and/or a human coach is in the loop; independent athletes compare it to a competent human coach. |
2.5 Transparency
Does it explain why?
| Score | Looks like |
|---|---|
| 1 | Black box. Workouts appear without reasons; the method is not described anywhere. |
| 3 | Each session has a purpose line and the vendor publishes an overview of its method. |
| 5 | Every plan change is explained in the app with its trigger ("moved because HRV was low"), the model is documented in detail, and limits and assumptions are stated. |
2.6 Value for money
What you get for what you pay, judged against the field.
| Score | Looks like |
|---|---|
| 1 | Priced at or above human-coaching level without human coaching, or core features paywalled behind add-ons. |
| 3 | Mid-market subscription (roughly the price of a premium wearable subscription) with the core adaptive plan included. |
| 5 | Free or low-cost with the full adaptive plan, or a premium price that independent reviewers agree is justified by a human-coach hybrid or exceptional depth. |
2.7 Athlete sentiment
What athletes say in public.
| Score | Looks like |
|---|---|
| 1 | Mostly negative threads: broken sync, plans that injured people, churn complaints, unanswered support. |
| 3 | Mixed. Praised for some things, criticised for others; a typical "it works if you accept X" thread. |
| 5 | Consistently positive across at least three independent communities, with long-term users reporting race results they attribute to the tool. |
Minimum evidence: two independent threads or reviews. The evidence_url holds the most representative one; the evidence_note may list others.
Part 3 — Overall score and ranking
- Overall = arithmetic mean of the non-null dimension scores, rounded to one decimal. All seven dimensions have equal weight.
- A tool is ranked only when at least 4 of 7 dimensions are scored. Tools below that threshold appear after the ranked ones, labelled "not enough evidence yet", alphabetical.
- Ties are broken by the number of scored dimensions, then alphabetically.
- Facts never change the score. More sports, more partners or a higher AI level do not add points; the filters and the AI-level badge exist for that.
Part 4 — Re-verification
- Facts are re-checked when a source URL changes or at least every 6 months.
- Scores are revisited when the weekly news crawl (
.claude/skills/news-crawl) surfaces new athlete threads or product changes. The crawl proposes edits; a human applies them. - Every change to
data/tools.jsongoes through git, so the history of a score is its commit log.
Changelog
- v2, 2026-10-13. Dimensions re-centred on coaching:
sport_science→plan_quality;plan_adaptivity→adaptation(absorbs "wellness feeds the plan");coaching_conversation→coaching_relationship(adds check-ins and human in loop); newworkout_feedback;data_integrationremoved as a score (now the partner matrix, a fact). Addedai_level, the 13 coaching capabilities and the partner matrix;human_coach_hybrid,integrations,workout_pushreplaced. - v1, 2026-10-13. Initial rubric.