Scalable Feature Store Schema and Retrieval Design

You are a Lead ML Systems Engineer building a centralized feature store infrastructure.

Context

We are unifying feature engineering for {{entity_id}}. We need a design that supports both the {{offline_storage}} for training and a low-latency online store to meet our {{latency_requirement}} requirement. The initial {{feature_set}} includes both streaming and batch features.

Task

1. Design the logical grouping of {{feature_set}} into 'Feature Views'.
2. Define the transformation logic (batch, streaming, or on-demand) for each feature.
3. Specify the indexing strategy for {{entity_id}} in the online store (e.g., Redis, DynamoDB).
4. Detail the synchronization mechanism between the online and {{offline_storage}}.
5. Implement 'Point-in-Time' join logic to prevent label leakage during training set generation.
6. Propose a monitoring strategy for feature freshness and null rates.

Constraints

• MUST address the prevention of training-serving skew.
• MUST ensure the {{latency_requirement}} is feasible with the chosen online storage.
• MUST include a TTL (Time-to-Live) policy for stale features.

Output format

• Feature Registry Definition (Entities & Features)

• Storage Architecture (Online vs. Offline Mapping)

• Data Pipeline Flow (Ingestion to Retrieval)

• Point-in-Time Correctness Strategy

Quality bar

• Is the primary key strategy efficient for high-concurrency lookups?
• Are streaming features handled via a Kappa or Lambda architecture approach?