The App Store Tax on Experimentation
Every mobile game developer knows the pain. You want to test a new pricing tier, a different offer placement, or a reward structure change. But on iOS, every change requires a TestFlight build, a review cycle, and an App Store submission. Android is slightly more forgiving with phased rollouts, but still requires APK updates. For a small team like CCFish, this kills velocity — a week of testing becomes a month of waiting.
Feature flags changed the game for us. By embedding a lightweight flag system into CCFish's Cocos Creator codebase and serving flag configurations from Cloudflare Workers, we can now A/B test any offer, pricing, or UI element in real time — without a single app store resubmission.
The Architecture: Flags as a Service
Server-Side Flag Definitions
Flag configurations live in a D1 database table:
```sql
CREATE TABLE feature_flags (
flag_key TEXT PRIMARY KEY,
description TEXT,
enabled INTEGER DEFAULT 0,
variants JSON, -- e.g., {“offer_a“: 50, “offer_b“: 30, “control“: 20}
targeting JSON, -- e.g., {“segments“: [“whale”, “engaged“], “platforms“: [“ios”, “android“]}
created_at TEXT,
updated_at TEXT
);
```
Each flag has:
- A **key** (e.g., `starter_pack_price_test`)
- An **enabled** toggle (kill switch)
- **Variants** with percentage allocation (50% see $2.99, 30% see $1.99, 20% control at $4.99)
- **Targeting rules** (only whales, only iOS, only US region)
Worker Endpoint
A Cloudflare Worker serves flags to the game client:
```javascript
export default {
async fetch(request, env) {
const url = new URL(request.url);
const playerId = url.searchParams.get('player_id');
const platform = url.searchParams.get('platform');
// Fetch all active flags
const { results } = await env.DB.prepare(
“SELECT flag_key, variants, targeting FROM feature_flags WHERE enabled = 1”
).all();
// Resolve flags for this player
const flags = {};
for (const flag of results) {
const targeting = JSON.parse(flag.targeting);
if (matchesTargeting(targeting, playerId, platform)) {
flags[flag.flag_key] = assignVariant(flag.flag_key, playerId, JSON.parse(flag.variants));
}
}
return new Response(JSON.stringify(flags));
}
}
function assignVariant(key, playerId, variants) {
// Deterministic assignment based on player ID hash
const hash = hashCode(key + playerId) % 100;
let cumulative = 0;
for (const [variant, weight] of Object.entries(variants)) {
cumulative += weight;
if (hash < cumulative) return variant;
}
return Object.keys(variants)[0];
}
```
The hash-based assignment ensures a player always sees the same variant of the same flag — no flickering between sessions.
Client-Side Integration (Cocos Creator)
In CCFish's game code, we integrated flag resolution at startup:
```javascript
cc.Class({
extends: cc.Component,
properties: {
flagUrl: “https://flags.ccfish.io/flags”
},
onLoad: function() {
var self = this;
var playerId = FireSDK.getPlayerId();
var platform = FireSDK.getPlatform();
FireSDK.httpRequest({
url: self.flagUrl + “?player_id=” + playerId + “&platform=” + platform,
method: “GET“,
onSuccess: function(response) {
self.flags = JSON.parse(response);
self.applyFlags();
}
});
},
applyFlags: function() {
// Apply starter pack pricing
if (this.flags.starter_pack_price_test === “price_low“) {
this.starterPackPrice = 1.99;
} else if (this.flags.starter_pack_price_test === “price_mid“) {
this.starterPackPrice = 2.99;
}
// Apply offer placement variant
if (this.flags.offer_placement === “post_level“) {
this.showOfferAfterLevel = true;
this.showOfferOnHome = false;
} else if (this.flags.offer_placement === “home_screen“) {
this.showOfferAfterLevel = false;
this.showOfferOnHome = true;
}
}
});
```
What We've Learned A/B Testing with Flags
Test 1: Starter Pack Pricing
We tested three price points for the first-purchase starter pack:
| Variant | Price | Conversion Rate | ARPU |
|---------|-------|----------------|------|
| Control | $4.99 | 2.1% | $0.10 |
| Mid | $2.99 | 4.8% | $0.14 |
| Low | $1.99 | 7.2% | $0.14 |
Surprising insight: $1.99 converted 3.4x better than $4.99, but ARPU was the same as $2.99. The “sweet spot” was $2.99 — same revenue per user as the cheapest option, higher perceived value. We rolled out $2.99 to 100% of players within 2 hours of seeing the data.
Test 2: Offer Placement
We tested showing the starter pack offer after level completion vs. on the home screen:
| Variant | Impressions | Clicks | Conversion |
|---------|------------|-------|------------|
| Post-level | 12,400 | 892 (7.2%) | 213 (1.7%) |
| Home screen | 18,200 | 637 (3.5%) | 89 (0.5%) |
Post-level placement won decisively — players are in a reward-seeking mindset after completing a level. Offers on the home screen feel interruptive.
Test 3: Reward Video Timing
We tested offering a rewarded video (watch ad → get extra life) at different points:
- **After death** (baseline): 12% accept rate
- **After death with a 3-second cooldown**: 18% accept rate
- **Before a hard level**: 24% accept rate
The pre-emptive variant — offering the reward before a known difficult level — had the highest engagement. We now serve this to 100% of players.
The Admin Interface
We built a simple admin panel in the CCFish Telegram Mini App to manage flags:
- Toggle flags on/off
- Adjust variant percentages with a slider
- View live experiment results (conversion rate, confidence interval, sample size)
- Target specific player segments
No app store, no review, no waiting. A pricing test that would take 2 weeks with an App Store submission now takes 5 minutes to configure and 48 hours to gather statistically significant data.
Pitfalls to Avoid
1. **Flag caching is dangerous.** We cache flags for 5 minutes in the game client. But for pricing experiments, a player could see variant A, close the app, and 5 minutes later see variant B. We solved this with the hash-based deterministic assignment — even if the cache clears, the same player gets the same variant.
2. **Too many flags hurt performance.** Each flag check is a function call. At one point we had 14 active flags in a single game scene, and we noticed frame drops. We now limit active flags to 5 per scene and batch flag resolution at startup.
3. **Don't forget the off switch.** Every flag must have a graceful fallback. If the Worker endpoint is down, the game should use default values, not crash or show blank UI.
Where We're Going Next
Feature flags were supposed to be just a tool for A/B testing offers. But they've evolved into our whole deployment strategy. We now use them for:
- **Gradual rollouts** — release new features to 1% of players, monitor crash rate, ramp up
- **Kill switches** — instantly disable a broken feature without hotfix
- **Personalized difficulty** — adjust game balance per player segment
The app store is still the distribution channel, but it's no longer the gatekeeper for our user experience. Feature flags on Cloudflare Workers gave CCFish back its iteration velocity.