Exploring Hierarchical Interest Representation For Meta

What happened

Hierarchical Interest Representation is an upstream representation layer designed to improve upon Meta’s deep funnel ranking optimization. It aims to connect businesses with the population of people on our platforms who carry the most genuine, latent interest in what they offer. The system is intended to function across Meta’s broader recommendation ecosystem, such as Meta’s Generative Ads Model (GEM), Andromeda, and the Adaptive Ranking Model, to advance deep funnel optimization.

Our latest research area is an upstream representation layer for learning universal, relational knowledge representations of users and ads entities. The representations capture users’ ads engagement patterns, absorb real-world world semantics, and cascade through multiple hierarchical granularities into latent-space projections at each level.

Based on the graph data structure, four design properties drive the system end to end: Hierarchical Interest Representation projects a raw graph into a configurable super-graph where each super-node is a learned latent interest primitive. User-ad edges that are sparse at the raw graph become meaningfully denser at the primitive interest graph.

Inherently, the primitive interest graph is more stationary and stable in vocabulary, even though the ads business is more dynamic. Hierarchical Interest Representation enriches heterogeneous entities, in particular advertiser and product types, with multimodal content features such as text, images, and video. These features are pulled from structured page metadata and advertiser catalog attributes and processed through vision or language models. This extra information complements existing engagement data.

Why it matters

People come to Meta’s apps and platforms to connect with people and content, expressing preference with every scroll. Engagement signals are used to understand both inferred and explicit interests and improve relevance of content across our platforms.

Utilizing frontier AI to map latent interests from sparse engagement signals and aligning them with the vast landscape of advertiser offerings is a transformative approach to addressing the challenges of signal scarcity and driving up deep funnel ad performance.

The mission is to strengthen reasoning relationships throughout the landscape of ads entities – spanning users, businesses, and products – utilizing multi-hierarchical granularities that allow our models to navigate between stable, high-level interest anchors and the specialized, sparse signals of deep funnel intent. Hierarchical Interest Representation pioneers a structural shift in representation modeling by navigating long-range graph topologies and distilling sparse engagement signals into unified interest clusters at various granularities.

By fusing real-world knowledge with a semantic grasp of advertised products, it effectively strengthens the connection between ads and user intents. Hierarchical Interest Representation encourages discovery-oriented ad experiences by extracting stable interest anchors from massive engagement datasets and grounding them in multi-modal world knowledge enrichment. This aims to enable the delivery of more relevant ad content to optimize deep funnel ads.

In addition, inferred interests based on engagement signals continue to play an important role for improving deep funnel ads. Every month, Meta’s ads network serves millions of ads, from millions of advertisers, to billions of people across our platforms. While this “vocabulary” is large, ad impression opportunities are limited and deep funnel user feedback is scarce.

Given the sparsity of the individual connections in the deep funnel, it is useful to observe common patterns from long range, graph connected entities and users and encode into representation. Capturing long-range relationships within large graph networks is computationally demanding. Even as hardware capabilities scale, the pursuit of modeling accuracy necessitates the design of memory-efficient attention kernels and high-performance learning algorithms.

What to watch

Instead of just knowing how users interact with something, it captures what that thing actually is. Even for entities it hasn’t seen before it understands the underlying businesses and products. Hierarchical Interest Representation learns thorough knowledge representation for users and entities, together with their latent interest primitives representation in a single metric space. It can infer entities’ mutual relationship and affinities across or within types.

Using embedding operations, Hierarchical Interest Representation can determine primitive-to-primitive and cluster-to-cluster relationships. It can also estimate user proximity to interest primitives; how closely an ad/advertiser serves certain interest primitives; and what are the similar users, ads, and products that are closest neighbors. Given the overall sparsity of deep funnel information, there is a trade-off on how to project into primitive interests.