Assignment goal. Analyze an existing recommender system; apply Scenario Design (three-question framework) for both the customer and the organization; reverse-engineer how it likely works from public signals; and propose concrete improvements.
NYT publishes an enormous volume of stories every day. The site/app surfaces modules such as “Recommended for You,” personalized rails on home/section pages, and end-of-article “next reads.” Personalization is essential to help readers find timely, relevant pieces while still discovering new topics.
Q1 — Who are the users?
Subscribers and registered readers on mobile/desktop. Common segments:
(a) headline checkers who want fast updates, (b)
deep divers who follow specific beats/authors, and (c)
browsers who skim lifestyle/culture. Accessibility
needs vary (audio, shorter formats, visual clarity).
Q2 — What are their goals?
- Stay informed fast without scanning hundreds of
links.
- Go deep on topics/authors/regions they care
about.
- Balance & serendipity: avoid echo chambers but
still see adjacent topics worth reading.
- Right format, right moment: short updates on phone,
long reads on desktop, audio on commute.
- Trust & control: understand why
something is recommended; follow/mute; save for later.
Q3 — How should the system support them?
- Cold-start relevance at publish time via text &
metadata understanding (topics, entities, section/desk, embeddings) so
brand-new pieces can be recommended immediately.
- Session-aware ranking that accounts for recency,
device, and time-of-day.
- Taste + diversity: blend known interests with
calibrated exploration; cap repetition in a session.
- Transparent controls: short “Because you read
…/follow …” reasons, Follow/Mute toggles, long-read indicators.
- Accessibility: surface audio/briefs on mobile; clear
headlines; readable layouts.
Q1 — Who are the users?
Editorial desks, personalization/ML, product/UX, growth/retention,
advertising, standards/trust & safety, and platform/SRE.
Q2 — What are the goals?
- Engagement & retention: loyal reading, depth,
saves, subscriber value.
- Editorial breadth: represent coverage across
desks/regions; avoid overly narrow feeds.
- Operational excellence: low-latency serving at scale;
resilient to breaking-news spikes.
- Brand integrity: uphold standards; reduce
sensationalism in ranking.
Q3 — How should the system support them?
- Hybrid modeling: combine content understanding for
cold start with collaborative signals as behavior accumulates.
- Explainable, steerable ranking: editorial constraints
(coverage breadth/region balance) and diversity knobs.
- Fast retrieval & caching: candidate pools by
section/topic; re-rank per user in milliseconds.
- Evaluation beyond CTR: optimize finish rate, dwell,
saves/follows, long-term return, and subscriber outcomes; monitor
fairness and diversity.
What the UI implies. Personalized slots mix fresh briefs with longer features; “Because you read …” hints appear at times; brand-new stories are recommended quickly. This suggests strong publish-time content features and hybrid ranking that updates as engagement data arrives.
Public background (high-level, paraphrased):
- NYT engineers have described a topic-based
representation for articles, adjusted with reader
co-consumption so items cluster near what similar readers
actually read.
- Per-reader topic preferences are maintained, with a
modest back-off so clicks aren’t treated as perfect endorsements.
- Serving aims for fast per-user computation;
article/topic updates happen offline/near-line; approaches are validated
via controlled experiments.
Contrast for context: Amazon’s classic item-to-item collaborative filtering precomputes similar-items tables and aggregates them in real time. That approach scales beautifully and informs “related items/reads,” but in news you also need publish-time content signals to handle constant cold start.
Likely features/signals (inferred): section/desk, tags and named entities, author, recency/decay, device & session context, engagement (click, dwell, completion, saves, follows), subscription state, soft diversity constraints.
A) Modeling & retrieval
- Add publish-time neural embeddings alongside
topics/keywords for better cold-start recall; keep topics for
explanation quality.
- Make ranking session-aware (time-since-last-visit,
device, reading streak) to balance recency vs. depth.
- Enforce intent-aware diversity: cap repeats of the
same topic/author per session; intentionally include 1–2 adjacent
topics.
B) Feedback & evaluation
- Prioritize finish rate, dwell, saves/follows over
pure CTR; discount short bounces.
- Use counterfactual evaluation (IPS/DR) to compare
rankers safely; track coverage breadth and
viewpoint diversity as first-class metrics.
- Treat Follow/Mute as durable signals with decay so
users can steer their feed.
C) Trust & standards
- Apply constrained optimization in ranking (limit
sensationalism; ensure regional/desk balance).
- Provide clear explanations (“Because you read X / follow Y”) and easy
“show me less of this.”
- Audit exposure to verify that similar-quality items from different
desks/viewpoints receive fair opportunity.
D) Delivery & ops
- Maintain edge-cached candidate sets by section/topic;
re-rank per user at request time.
- Build graceful fallbacks: editorial/most-recent when
personalization degrades; progressively enhance when signals
return.
- Run sequential A/B tests with holdouts focused on
long-term value, not just short-term clicks.
Building the Next New York Times Recommendation Engine
(NYT Open blog, accessed for conceptual background).
https://archive.nytimes.com/open.blogs.nytimes.com/2015/08/11/building-the-next-new-york-times-recommendation-engine/
Linden, G., Smith, B., & York, J. (2003). Amazon.com Recommendations: Item-to-Item Collaborative Filtering, IEEE Internet Computing. (Contextual contrast for scalable CF.) ““” path = “/mnt/data/Scenario_Design_Recommender_NYTimes_SachiKapoor.Rmd” with open(path, “w”, encoding=“utf-8”) as f: f.write(rmd)