Myths, tests and platform changes

Because recommendation systems are secret, complex and constantly changing, creators fill the gaps with theories. Some are well founded; many are myths spread by anecdote, misinterpreted data or people selling courses. Believing myths wastes effort and causes anxiety; testing ideas properly leads to better decisions. This lesson examines common myths and what the evidence suggests, shadowbanning and reduced distribution, how to evaluate claims, designing simple experiments, the statistics of noisy results, how platforms announce and roll out changes, and how to adapt without chasing every rumour.

Common myths

Several widely repeated beliefs deserve scrutiny:

• "There is a perfect time to post." Timing can affect how quickly early viewers see a post, especially followers, but recommendation feeds distribute content over hours or days, and platforms have said that timing is a minor factor. Posting when your audience is active is sensible; obsessing over exact minutes is not.
• "Hashtags are the key to reach." On most platforms today, hashtags play a small categorization role. Content, captions and engagement matter far more.
• "Deleting posts that underperform helps the account." Platforms have generally said this does not help, and on YouTube, deleting and reuploading can cause problems such as duplicate content.
• "Editing a post after publishing kills its reach." Minor edits to captions are generally fine on most platforms, though changes to videos usually require reposting.
• "Followers no longer matter." Recommendations reduce dependence on followers, but engaged followers still provide early signals, steady views and income.
• "Algorithms punish creators who take breaks." Audiences may need time to return, and the system needs fresh signals, but platforms have said there is no lasting penalty for breaks.
• "Using a certain app or feature boosts reach." Platforms sometimes promote new features, and early adopters may benefit briefly, but features do not override audience response.

Shadowbanning and reduced distribution

Shadowbanning, the belief that platforms secretly hide an account's content without telling the creator, is one of the most common creator fears. The reality is nuanced. Platforms do reduce the distribution of some content: posts that violate rules may be removed, and content that does not violate rules but falls into categories excluded from recommendations, such as borderline content, unoriginal reposts, or certain sensitive topics, may be shown to followers but not recommended to new audiences. Accounts with recent violations may have restricted features. In response to criticism and to regulation such as the EU's Digital Services Act, which requires platforms to explain restrictions to users, platforms have built tools that show account status, recommendation eligibility, and reasons for restrictions, with appeals. Most drops in reach, however, are explained by ordinary variability, changes in content, audience fatigue or platform changes, not by a hidden ban.

Evaluating claims

When you hear a claim about an algorithm, ask:

• What is the source? Official platform statements, help centres, engineering papers and creator liaison accounts are more reliable than anonymous posts or sellers of courses, though even official statements can be simplified or out of date.
• What evidence supports it? A single creator's experience is weak evidence; consistent results across many creators, or controlled experiments, are stronger.
• Could something else explain it? A post that performed well after using a new hashtag might have succeeded because of its content, timing or chance.
• Is it still current? Platforms change constantly; advice from two years ago may no longer apply.
• Who benefits? Sellers of tools and courses have incentives to promote dramatic "hacks".

This scepticism is not cynicism: some creator observations later prove correct and are confirmed by platforms. The goal is to weigh evidence rather than follow rumours.

Designing simple experiments

Creators can test ideas with simple experiments. The principles are those of any experiment (CRE-202 covers analytics in detail):

• Define a clear question and a measurable outcome, such as "Do videos with on-screen captions in the first second have higher three-second retention?"
• Change one thing at a time, keeping other factors as similar as possible.
• Repeat the test across several posts, because single results are noisy.
• Compare with a baseline, such as your recent median performance, rather than your best post.
• Record results systematically in a spreadsheet, including the date, format, topic, change tested and key metrics.
• Use platform testing tools where available, such as YouTube's Test and Compare for thumbnails or Instagram's trial reels, which run controlled comparisons.

The statistics of noisy results

Social media metrics are noisy: the same kind of video can receive very different results by chance. A difference between two posts is meaningful only if it is large relative to normal variation. A useful habit is to know your typical range: if your videos usually receive between 3,000 and 15,000 views, a video with 12,000 views is not evidence that a change worked. Comparing groups of posts reduces noise, and the more posts in each group, the more confident you can be. Platform A/B tools handle statistics for you by randomly assigning viewers to versions, which is far more reliable than comparing posts published at different times. Be wary of drawing conclusions from very small numbers, such as a difference of a few likes.

How platforms announce and roll out changes

Platforms change their recommendation systems continuously, with many small experiments running at any time and occasional larger shifts. Major changes are sometimes announced through official blogs, creator newsletters, help centre updates and posts by platform leaders or creator liaison accounts, such as YouTube's Creator Liaison and Instagram's head posting explanations. Many changes are rolled out gradually, by region or to a percentage of users, which is why creators may experience different effects at different times. Platforms also run experiments on features, such as new layouts or tabs, which can temporarily change reach. Following official sources is the best way to learn about changes, while recognizing that platforms do not reveal everything.

Adapting without chasing rumours

Constant change tempts creators to react to every rumour, leading to scattered strategies and stress. A steadier approach focuses on durable principles that have held across platform changes: make content your target audience genuinely values; package it clearly and honestly; hook viewers quickly and respect their time; build relationships and community; experiment thoughtfully; follow platform rules and originality expectations; and diversify across platforms and owned channels (Lesson 12). Within this framework, creators can adopt new features or formats that suit them, test promising ideas, and adjust when evidence accumulates, without abandoning what works because of a single anecdote.

Survivorship bias in creator advice

Much algorithm advice comes from successful creators describing what they did. This is useful but suffers from survivorship bias: we hear from those who succeeded, not from the many who followed the same practices without success. A creator who posted daily and grew may attribute success to daily posting, while many others posted daily without growing. Advice from successful creators is more reliable when it is consistent with platform statements, supported by many examples, and plausibly linked to how recommendation systems work, such as advice to improve hooks and retention.

Correlation and causation

Analytics show correlations, such as videos posted on Sundays performing better, but correlation does not prove causation. Sunday videos might perform better because the creator saves their best ideas for weekends, because the topics differ, or by chance. Testing a cause requires changing one factor while holding others constant, ideally with random assignment, which platform A/B tools provide. Thinking carefully about alternative explanations prevents creators from building strategies on coincidences.

Keeping an experiment log

An experiment log, a simple spreadsheet or document, records each test: the date, the hypothesis, what was changed, the posts involved, the metrics, and the conclusion. Over months, the log becomes a valuable record of what works for a particular creator and audience, more reliable than memory and more specific than general advice. It also reveals when conclusions change over time as platforms and audiences evolve, prompting retesting.

When platforms confirm creator observations

Sometimes creators notice patterns before platforms acknowledge them. Reports that reposted content was losing reach, that watermarked videos performed worse on Instagram, or that sends mattered for discovery were later confirmed or explained by platform representatives. Creator communities that share observations carefully, with data, can identify real changes early. The key is to treat early observations as hypotheses to test rather than as facts, and to update when better evidence arrives.

Avoiding anxiety and burnout

Obsessive monitoring of metrics and algorithm rumours contributes to creator stress and burnout (CRE-350). Healthy habits include checking analytics at set times rather than constantly, focusing on trends over weeks rather than hourly results, separating self-worth from view counts, and remembering that much variation is random. A calm, evidence-based approach to algorithms supports both better decisions and better wellbeing.

Testing with limited resources

Small creators may not have enough posts or views for strong statistical conclusions. They can still learn by testing larger changes that are likely to have noticeable effects, such as a completely different hook style, by running tests over longer periods, and by combining quantitative results with qualitative feedback from comments and messages. As their audience grows, more precise testing becomes possible.

A worked evaluation of a claim

Suppose a widely shared post claims that "videos longer than 61 seconds get more reach on TikTok". To evaluate it, check whether TikTok has said anything official about length; look for evidence beyond anecdotes, such as analyses across many accounts; consider alternative explanations, such as longer videos being made by more experienced creators; and check the date, since TikTok's length policies have changed. Then test it yourself by making comparable videos at different lengths over several weeks and comparing medians of reach and retention. The conclusion may be that length helps some content but not others, a more nuanced and useful result than the original claim.

Platform experiments that affect your results

Platforms constantly run experiments on users, testing new layouts, ranking adjustments and features with a percentage of their audience. A creator's content may be shown in a test version of the app to some viewers and the standard version to others, which can affect performance in ways creators cannot see. This is another reason to judge strategy over many posts and weeks rather than days. When several creators in the same niche notice a change at once, it may reflect a platform experiment or rollout; waiting to see whether the change persists, and checking official sources, is usually wiser than reacting immediately.

Testing is therefore less about finding a secret formula than about learning, steadily and honestly, what a particular audience values, and continuing to learn as both the audience and the platforms change over time.