How we rate

ÜBERMAD sums up research about getting things done. This page explains which studies we use, how we read them and what the stars mean. The same rules apply to every tool.

Posts and tools

Each post sums up one study or one review. It has one or more takeaways: one sentence each about what the study found.

Each takeaway is tagged with the tools it involves. There are two kinds:

• Actions are what you do, like “Track your progress”.
• Rules say when, where, how, how often, how much or with whom, like “Before going to bed”.

Tools are shared between posts, so a tool’s page lists every study that includes it. Only tools get stars. Right now the site has 153 published posts and 100 tools.

Tested and used

Every tool in a takeaway is marked one of two ways:

• Tested: it was the difference between the groups, on its own or together with other tested tools.
• Used: everyone in the study had it, or it was the thing people were trying to do rather than the method.

Only tested tools count toward stars. Both show up on the tool’s page. So a daily walk can appear in many studies without a rating, while the rules that helped people keep walking get the stars.

How a programme was delivered (with a coach, in a group, through an app, in weekly sessions) counts as tested only when the delivery itself was the difference. When a group course beats a waiting list, the course was tested, not the group.

Topics

Every takeaway gets exactly one topic. The topic depends on what the study measured, never on the method, who took part or the paper’s title. A tool can be rated in several topics, and the ratings can differ.

• ProcrastinationMeasures when you start or finish a task compared with when you meant to: questionnaires about putting things off, how long before starting, late hand-ins, going to bed later than you meant to.
• FocusMeasures attention on a task while you do it: time on task, mind-wandering, interruptions and phone checks during work.
• Time managementMeasures how you plan, prioritise and estimate time: time-management questionnaires, using a planner or calendar, how well you guess how long things take.
• HabitsMeasures how automatic something feels in its usual situation, measured with habit questionnaires.
• RoutinesMeasures how regular your daily activities are: regular bed and wake times, routine checklists, steps done on schedule.
• GoalsMeasures how much got done: goals met, tasks or homework finished, how often people exercised, medication taken as planned, grades and work output.
• MotivationMeasures interest, engagement or confidence as the main outcome. We keep these findings but don’t show stars until there is enough evidence.
• Other outcomes: wellbeing, stress, mood, ADHD symptom totals, overall self-control scores, and how much people used the programme itself. These are recorded but never rated.

If one takeaway could fit two topics, it goes to the outcome its claim is about. Methods such as if-then plans or working next to someone are never topics: they are tools, rated in each topic where they were tested.

The stars

Each tool gets two ratings in each topic, from 1 to 4 stars:

• Evidence: how sure we can be.
• Potential benefit: how much it could help.

Tool cards show the two added together, from 2 to 8, for the topic in view. Tool pages always show both separately.

Evidence

A well-controlled study picked its groups at random and also had at least one of these: a comparison with another real option or another version, 100 people or more, behaviour that was recorded rather than self-rated, or a plan registered before it started.

We take one star off, but never below one, for each of these:

• Every comparison was a waiting list or nothing, and everyone rated themselves.
• Every study behind the rating has a serious problem: only a short summary available, not yet checked by other scientists, whole groups assigned instead of people, unclear how groups were picked, only people who finished were counted, many people dropped out, numbers that don’t add up, groups that weren’t properly compared, a summary that claims more than the results show, or a very small study.
• The studies disagree.

Potential benefit

The size comes from the best fair test we have: first a well-controlled study that tested the tool on its own, then one that tested it with other tools, then any study. A fair test compares it with another real option, or records behaviour. A result against a waiting list or nothing, where people rated themselves, counts one star lower, because expecting to improve makes people rate themselves better.

• Benefit can’t be more than one star above Evidence: thin evidence doesn’t get to promise a large benefit.
• Four benefit stars need a study of at least 100 people, because small studies overstate effects.
• With one Evidence star, the benefit is only an estimate, marked est., and at most two stars.
• When a study helped but printed no size, it gives two benefit stars.
• Each tool page says what the benefit was measured against. That matters: something compared with another good option can look small even when it works.

The sentence under the stars

Each rating comes with one sentence whose words follow the stars. Four Evidence stars say it “helps”, three say it “probably helps”, two say it “may help”, and one says it’s “too early to say”. The benefit stars choose between “a lot”, plain, “a little” and “little or no difference”.

ADHD ratings and hand changes

When studies with people who have ADHD exist, the tool also gets a rating from those studies alone, shown in its own row. A person can move a rating by one star at most, and only with a written reason, which the tool page shows. Ratings are worked out again whenever a post changes.

Pooled analyses

A review that pools the results of many trials counts too. Each pooled result counts as one finding, and the better of the trial rules and these rules sets the stars:

• If the review’s authors rated their own certainty, high gives four Evidence stars, moderate three, low two and very low one. Programmes that combine several tools stop at three.
• Otherwise a review is sound only if it pooled at least five studies, judged their quality and checked for missing studies. A review that isn’t sound gives two stars at most.
• One star comes off for signs that small negative studies went unpublished, and the rating stops at two when most of the pooled trials had serious flaws.
• A trial that a review already pooled isn’t counted again on its own for the same tool.

Expert advice

Podcasts and articles have no findings of their own. Their takeaways record who said it and whether they recommend it or warn against it. Each person gets a level for each topic:

• High: has published peer-reviewed research on the topic, or speaks for a guideline or professional body.
• Medium: a relevant professional qualification, or a science writer who cites the studies.
• Low: everyone else.

Selling the thing they recommend lowers the level by one. Expert advice alone earns two Evidence stars at most, and its benefit is always an estimate. When studies exist, the studies set the stars. If the studies found little or no benefit and an expert recommends it, the page says so. There are no podcasts or articles on the site yet.

How studies are picked

Studies come from literature searches on each topic. Each one then has to pass three gates. Each gate is cheaper than the next, so most of the sorting happens early.

1. Before the full text: from the title, summary and details, is it on topic, does it report results, and has it been retracted? Studies without a legal full text wait; they are never rated from the summary alone.
2. The full text: an AI model answers fixed questions about the design, the comparison group, whether what people did is described well enough to repeat, what was measured, the group numbers and the size of the study. Code, not the model, turns the answers into a decision.
3. A full read: a second AI model reads the whole paper, tables included, and fills in a fixed form with a short quote for every answer: the parts of the programme, the comparison, how many people took part, the risk of bias, and red flags such as numbers that differ between sections. Code applies the rules again, and every exclusion for unreliable numbers is checked by hand against the quotes.

Each gate can pass a study, pass it with a warning, exclude it with a reason, or hold it while it waits for something. A study that reaches the site with a warning shows a ⚠ in its study line.

Lanes

Not every useful paper tests a tool, so the gates also sort papers by kind:

• Trials that compare a tool with something else count toward its stars.
• Insights: studies that watched people or followed one group. Their takeaways say what was found, such as how long habits take, but never move stars.
• Theories explain why something might work. They never move stars, however popular: popular theories have failed direct tests.
• Pooled analyses of trials can move stars under the rules above.

The kind of paper decides its lane, never its quality.

How the summaries are written

The summaries are written with AI help from the full text of each study, then rewritten in plain language so a 12- to 14-year-old could follow them, keeping every fact as accurate as the original. They are checked by automated checks and by model reviews: one AI model checked each result, how serious each concern was, whether groups were picked at random and what kind of outcome was measured, and a second model settled every disagreement and re-read every takeaway for claims stronger than the study shows.

The study line under each takeaway is built from the data, not written freehand: the study’s size and design, who took part, what they were compared with, whether people rated themselves or their behaviour was recorded, and any serious problems. We use one word for each meaning across the site, for example “put things off” and never “procrastinate” in a takeaway. The illustrations are made with AI image tools.

Limits

• Most studies in this field are small, and many have serious problems. No tool has four Evidence stars yet.
• Stars describe what studies found on average, not what will work for you.
• A benefit depends on what it was compared with. A tool can beat nothing and still lose to another tool.
• A rating in one topic says nothing about another topic.
• Results in one group, such as students or people with a health condition, may not carry over to others.
• The rules are fixed, so they can miss things a careful reader would catch.
• AI help can make mistakes, and not every summary has been checked by a person.
• We only rate studies whose full text we could read legally.

Not medical advice

ÜBERMAD sums up research. It is not medical advice, and it can’t tell you what is right for you. Pages about ADHD are not diagnosis or treatment advice: talk to a clinician, and don’t stop or change a treatment because of something you read here.

Spotted a mistake? See corrections.