Cold-Start Solutions

Example Onboarding Questions:
1. "Select your top 3 genres" (multi-select)
2. "Pick 5 items from this curated list" (image gallery)
3. "What's your primary use case?" (radio buttons)

Advantage: Direct, explicit signal
Drawback: Friction (users may skip, answers may be inaccurate)

UCB Score = μ (mean reward) + c * sqrt(log(t) / n)

Where:
  μ = observed mean reward (engagement rate)
  c = exploration constant (typically 1-2)
  t = total rounds played
  n = times this arm played

Intuition: Known good arms (high μ) are favored.
          Unplayed arms (low n) get bonus.
          Over time, as t grows, bonus decreases.

For each item (arm):
  α = successes + 1 (prior: pseudocount)
  β = failures + 1

At each step:
  1. Sample θ ~ Beta(α, β) for each arm
  2. Select arm with highest sampled θ
  3. Get reward (click = 1, no click = 0)
  4. Update: α += reward, β += (1 - reward)

Intuition: Uncertainty is modeled explicitly.
          Arms with high sampled values get more plays.
          Natural exploration as uncertainty shrinks.

if rand() < epsilon:
    recommendation = random_item()
else:
    recommendation = best_performing_item()

Challenge: ε is fixed; doesn't adapt as confidence grows.
Advantage: Simple to implement and understand.

LinUCB Score = (w_a^T * x_c) + α * sqrt(x_c^T * (A_a^-1) * x_c)

Where:
  w_a = learned weights for arm a
  x_c = context vector (user + item features)
  A_a = design matrix for arm a
  α = exploration parameter

Benefit: Personalized exploration based on user context
         Learns faster than non-contextual bandits

new_item_embedding = encode(title, category, tags, image)

# Find K nearest neighbors (warm items)
similarity = cosine_similarity(new_item_embedding,
                                warm_item_embeddings)
neighbors = top_k(similarity, k=10)

# Inherit audience: who interacted with neighbors?
inherited_users = union(neighbors_audiences)

# Score: boosted preference for inherited users
recommendation_score = base_score + 0.5 * inheritance_score

if rand() < 0.05:  # 5% traffic to cold items
    candidates = items(age < 7_days, interactions < 100)
    recommendation = sample_weighted(candidates, weight=quality)
else:
    recommendation = normal_ranking_algorithm()

item_age_days = (now - item_created_at).days

if item_age_days < 1:
    boost_multiplier = 10x  # First 24 hours: aggressive
elif item_age_days < 3:
    boost_multiplier = 5x   # Days 2-3: moderate
elif item_age_days < 7:
    boost_multiplier = 2x   # Week 1: gentle
elif item_age_days < 14:
    boost_multiplier = 1.2x # Week 2: minimal
else:
    boost_multiplier = 1.0  # Normal ranking

traffic_allocation = 0.05 * boost_multiplier

Experiment Design:
  Control: Normal ranking (no cold-start boost)
  Treatment: Cold-start boost (5-10% allocation)

Metrics:
  - New item CTR (primary)
  - User engagement (session time, multi-click)
  - Creator satisfaction (retention, repeat uploads)
  - System diversity (% budget to top 10 items)

Duration: Run for 1 week per cohort
Rollout: If CTR_treatment > CTR_control * 1.2, deploy

observed_ctr = clicks / impressions

# Position i has inherent CTR discount (e.g., 0.7x for rank 5)
position_discount = position_bias[i]

# Predicted CTR if item were at rank 1
predicted_true_ctr = observed_ctr / position_discount

# Use true CTR to evaluate item quality (not position-biased)

interaction_count = user.num_interactions

# Warm-up curve: as interactions grow, CF weight increases
cf_weight = tanh(interaction_count / 10)
content_weight = 1.0 - cf_weight

# Blend scores
final_score = (content_weight * content_score +
               cf_weight * cf_score)

Example:
  0 interactions: content_weight=1.0, cf_weight=0.0  (100% content)
  5 interactions: content_weight=0.38, cf_weight=0.62 (38% content, 62% CF)
  20 interactions: content_weight=0.10, cf_weight=0.90 (10% content, 90% CF)
  50+ interactions: content_weight≈0.0, cf_weight≈1.0 (100% CF)

For cold items:
  item_embedding = α * content_embedding + (1-α) * id_embedding

Where:
  content_embedding = learned from title, category, image
  id_embedding = trainable embedding (initialized near zero)
  α = 1.0 when item is brand new
  α = 0.0 when item is warm (many interactions)

Smooth transition as item_interactions grow

Stage 1 - Retrieval (Content-based):
  candidates = top_1000_by_content_similarity(user_profile)

Stage 2 - Ranking (Collaborative Filtering):
  candidates = sort_by_cf_score(candidates)
  candidates = take_top_100(candidates)

Stage 3 - Reranking (Business rules + exploration):
  final_ranking = apply_business_rules(candidates)
    - Boost new items (cold-start rule)
    - Suppress already-seen items
    - Diversify by category

  return final_ranking[:10]

if item.interactions < 10:
    # Find K warm items similar by content
    neighbors = find_content_neighbors(item, k=5)

    # Average their ID embeddings as proxy
    item.cf_embedding = mean([n.id_embedding for n in neighbors])

    # Use proxy for user-item scoring
    score = dot_product(user.embedding, item.cf_embedding)
else:
    # Once warm, use true ID embedding
    score = dot_product(user.embedding, item.id_embedding)

MAML Training Loop:

Outer loop (meta-update):
  For each task (user with K examples + evaluation set):
    Inner loop (adapt to user):
      1. Sample K interactions from user (support set)
      2. Take 1 gradient step: θ' = θ - α * ∇L(θ, K)
      3. Evaluate on evaluation set using θ'
      4. Compute meta-loss

    Update meta-parameters: θ = θ - β * ∇(meta-loss)

Inference on new user:
  1. Observe their first K interactions
  2. Compute adapted parameters: θ_user = θ - α * ∇L(θ, K)
  3. Use θ_user for recommendation

Benefit: After 1-5 gradient steps, model adapts to new user
         Better generalization than training from scratch

User Representation = mean(item_embeddings of clicked items)

For new user:
  1. They click on K items
  2. Represent user as avg of those K item embeddings
  3. Score candidates by distance to user representation

  score = -||candidate_embedding - user_representation||²

Strength: No gradient updates needed; very fast
Weakness: Limited to items with pre-trained embeddings

Offline (one-time):
  1. Cluster warm users by interaction patterns (K-means, EM)
  2. For each cluster, train a mini-model

Online (new user):
  1. Observe their first K interactions
  2. Compute distance to each cluster centroid
  3. Assign to nearest cluster
  4. Use that cluster's model for ranking

As user accumulates more data:
  5. Optionally re-cluster or transition to personalized model

If user.embedding is None:
    # Hash user ID to bucket
    bucket_idx = hash(user_id) % num_buckets
    user.embedding = global_bucket_embedding[bucket_idx]

    # Alternative: use demographic-based bucket
    bucket_idx = hash(user_age, user_location) % num_buckets
    user.embedding = demographic_bucket_embedding[bucket_idx]

Benefit: Every user/item gets an embedding
Drawback: Many users/items map to same bucket (collisions)

item_embedding[i] = (
    content_embedding[i] * 0.8 +  # Majority: content features
    id_embedding[i] * 0.2        # Small component: ID embedding (near zero for cold items)
)

As item warms up (gains interactions):
    - ID embedding component learns from interactions
    - Content component weight decays
    - Eventually: item_embedding ≈ learned id_embedding

if user.interactions < 3:
    recommendations = generic_recommendations()
elif user.interactions < 20:
    recommendations = hybrid_blend(content=0.6, cf=0.4)
else:
    recommendations = cf_recommendations()

# Monitor user experience (engagement, churn) per bucket
# Adjust thresholds based on cohort performance

1. Do you have user preferences?
   YES → Onboarding questionnaire, content-based filtering
   NO → Proceed to #2

2. Do you have item metadata (title, category, image)?
   YES → Content-based similarity for items
   NO → Proceed to #3

3. Can you afford to boost cold items?
   YES → Traffic allocation + quality gating
   NO → Proceed to #4

4. Do you have user context (age, location, device)?
   YES → Demographic clustering
   NO → Random exploration (last resort)

5. Are you accumulating enough interaction data?
   YES (10+ interactions/user) → Transition to hybrid blending
   NO → Keep content-based approach

import numpy as np
import pandas as pd
from sklearn.metrics.pairwise import cosine_similarity
from scipy.sparse import csr_matrix

# ============ 1. UCB BANDIT ALGORITHM ============

class UCBBandit:
    """Upper Confidence Bound multi-armed bandit for exploration."""

    def __init__(self, n_arms, c=1.0):
        self.n_arms = n_arms
        self.c = c  # exploration constant
        self.counts = np.zeros(n_arms)  # times each arm played
        self.successes = np.zeros(n_arms)  # clicks per arm
        self.total_plays = 0

    def select_arm(self):
        """Select arm with highest UCB score."""
        ucb_scores = []
        for arm in range(self.n_arms):
            if self.counts[arm] == 0:
                # Unplayed arm: infinite UCB
                ucb_scores.append(float('inf'))
            else:
                mean_reward = self.successes[arm] / self.counts[arm]
                confidence = self.c * np.sqrt(np.log(self.total_plays) / self.counts[arm])
                ucb_scores.append(mean_reward + confidence)

        return np.argmax(ucb_scores)

    def update(self, arm, reward):
        """Update counts after arm selection and reward observation."""
        self.counts[arm] += 1
        self.successes[arm] += reward
        self.total_plays += 1

# ============ 2. THOMPSON SAMPLING ============

class ThompsonSampling:
    """Thompson Sampling using Beta-Binomial model."""

    def __init__(self, n_arms):
        self.n_arms = n_arms
        self.alphas = np.ones(n_arms)  # successes + prior
        self.betas = np.ones(n_arms)   # failures + prior

    def select_arm(self):
        """Sample from Beta distribution; select arm with highest sample."""
        samples = np.random.beta(self.alphas, self.betas)
        return np.argmax(samples)

    def update(self, arm, reward):
        """Update Beta parameters."""
        self.alphas[arm] += reward
        self.betas[arm] += (1 - reward)

# ============ 3. CONTENT-BASED COLD-START ============

class ContentBasedCold:
    """Content-based recommendations for cold items."""

    def __init__(self, item_features):
        """
        item_features: (n_items, d) array of item embeddings
        """
        self.item_features = item_features
        self.n_items = item_features.shape[0]

    def recommend_for_cold_item(self, cold_item_idx, k=10):
        """
        Find k similar warm items using cosine similarity.
        """
        cold_embedding = self.item_features[cold_item_idx:cold_item_idx+1]
        similarities = cosine_similarity(cold_embedding, self.item_features)[0]

        # Exclude self
        similarities[cold_item_idx] = -1

        # Top-k similar items
        neighbors = np.argsort(similarities)[::-1][:k]
        return neighbors, similarities[neighbors]

# ============ 4. HYBRID COLD-WARM BLENDING ============

class HybridColdWarm:
    """Blend content-based and CF scores based on user age."""

    def __init__(self, content_scores, cf_scores):
        """
        content_scores: (n_users, n_items) content-based scores
        cf_scores: (n_users, n_items) collaborative filtering scores
        """
        self.content_scores = content_scores
        self.cf_scores = cf_scores
        self.n_users = content_scores.shape[0]

    def get_blended_scores(self, user_interactions):
        """
        user_interactions: array of interaction counts per user
        Returns blended scores based on interaction count.
        """
        blended = np.zeros_like(self.content_scores)

        for user_idx in range(self.n_users):
            interactions = user_interactions[user_idx]

            # Warm-up curve: tanh maps 0→1 range
            cf_weight = np.tanh(interactions / 10.0)
            content_weight = 1.0 - cf_weight

            blended[user_idx] = (
                content_weight * self.content_scores[user_idx] +
                cf_weight * self.cf_scores[user_idx]
            )

        return blended


# ============ EXAMPLE: SYNTHETIC DATASET & EVALUATION ============

def create_synthetic_dataset(n_users=1000, n_items=500, n_interactions=5000):
    """
    Create synthetic user-item interaction matrix with mix of cold/warm users.
    """
    # Random user-item interactions
    rows = np.random.randint(0, n_users, n_interactions)
    cols = np.random.randint(0, n_items, n_interactions)
    data = np.ones(n_interactions)

    interaction_matrix = csr_matrix((data, (rows, cols)), shape=(n_users, n_items))

    # Random item features (embeddings)
    item_features = np.random.randn(n_items, 64)
    item_features = item_features / np.linalg.norm(item_features, axis=1, keepdims=True)

    # User interaction counts
    user_interactions = np.array(interaction_matrix.sum(axis=1)).flatten()

    return {
        'interaction_matrix': interaction_matrix,
        'item_features': item_features,
        'user_interactions': user_interactions,
        'n_users': n_users,
        'n_items': n_items
    }

# ============ UCB EXPLORATION SIMULATION ============

def simulate_ucb_exploration(n_items=20, n_rounds=200, true_ctr=None):
    """Simulate UCB bandit over rounds; plot exploration curve."""

    if true_ctr is None:
        true_ctr = np.random.uniform(0.1, 0.5, n_items)

    bandit = UCBBandit(n_items, c=1.0)
    arm_selections = np.zeros(n_rounds)

    for round_idx in range(n_rounds):
        arm = bandit.select_arm()
        arm_selections[round_idx] = arm

        # Simulate reward
        reward = 1 if np.random.rand() < true_ctr[arm] else 0
        bandit.update(arm, reward)

    return arm_selections, bandit.counts, bandit.successes

# Test UCB
arm_selections, counts, successes = simulate_ucb_exploration(n_items=10, n_rounds=500)
exploration_rate = (len(np.unique(arm_selections)) / 10.0) * 100
print(f"UCB Exploration: {exploration_rate:.1f}% of arms explored")
print(f"Top arm selected {counts[np.argmax(successes)]:.0f} times")

# ============ CONTENT-BASED COLD ITEM RECOMMENDATION ============

data = create_synthetic_dataset()
content_model = ContentBasedCold(data['item_features'])

# Recommend for a cold item (item 0)
neighbors, similarities = content_model.recommend_for_cold_item(0, k=10)
print(f"\nContent-based neighbors for item 0:")
for idx, neighbor in enumerate(neighbors):
    print(f"  Rank {idx+1}: Item {neighbor} (similarity: {similarities[idx]:.3f})")

# ============ HYBRID BLENDING DEMO ============

n_users, n_items = 100, 50
content_scores = np.random.randn(n_users, n_items)
cf_scores = np.random.randn(n_users, n_items)
user_interactions = np.array([0, 2, 5, 10, 20, 50])  # Varying interaction counts

hybrid = HybridColdWarm(content_scores[:len(user_interactions)],
                        cf_scores[:len(user_interactions)])
blended = hybrid.get_blended_scores(user_interactions)

print(f"\nHybrid blending weights by interaction count:")
for interactions in [0, 2, 5, 10, 20, 50]:
    cf_w = np.tanh(interactions / 10.0)
    content_w = 1.0 - cf_w
    print(f"  {interactions} interactions: content={content_w:.2f}, CF={cf_w:.2f}")