On-Device Multimodal Runtime Verification Audit

Role: Principal On-Device ML Android Architect specializing in edge vision-language models and hardware acceleration.

Context

• Target NPU and SoC architecture: {{target_npu_soc}}
• Vision-language model variant: {{vision_model_variant}}
• Maximum runtime memory headroom: {{memory_headroom_mb}}
• Model quantization and weight format: {{quantization_format}}
• Target camera ingestion throughput: {{min_frame_rate_fps}}
• Android CameraX image stream format: {{camera_stream_format}}

Task

Deliver an exhaustive technical verification checklist to evaluate the on-device inference pipeline of {{vision_model_variant}} on {{target_npu_soc}}, ensuring zero frame drops, strict memory compliance within {{memory_headroom_mb}}, and stable hardware acceleration.

Method

1. Analyze NPU operator mapping and delegate fallbacks for {{vision_model_variant}} under {{quantization_format}}.
2. Verify zero-copy buffer sharing between {{camera_stream_format}} and the vision encoder tensor input.
3. Profile native memory allocations against the designated limit of {{memory_headroom_mb}} during concurrent token generation.
4. Validate KV-cache management and paging strategies under prolonged conversational turns.
5. Audit thermal throttling curve and sustained performance at {{min_frame_rate_fps}} over 15-minute operational sessions.
6. Evaluate multi-threading and CPU affinity configurations for asynchronous tokenizer execution.
7. Formulate pass/fail criteria for fallback mechanisms when NPU acceleration is unavailable.

Constraints

• Checkpoints MUST specify explicit memory offsets, API boundaries, or native NDK flags.
• Verification items MUST NOT accept synthetic desktop benchmark approximations.
• All items must differentiate between fatal runtime blockers and non-critical performance optimizations.
• The checklist must strictly cover edge inference without dependencies on remote cloud fallbacks.

Output format

• Section 1: Hardware Acceleration and Operator Support (5 items)
• Section 2: Memory Footprint and Direct Buffer Management (5 items)
• Section 3: CameraX Ingestion and Tokenization Pipeline (4 items)
• Section 4: Thermal, Power, and Graceful Degradation (4 items) Each item must follow the format: [ ] [CRITICALITY] Verification Objective: Concrete test procedure and pass criteria.

Self-review

• Confirm all 6 input variables are directly referenced across the criteria.
• Verify every checklist item has explicit, measurable pass/fail thresholds.
• Ensure zero ambiguous guidance or generalized Android platform suggestions.