MLP Compute-vs-Memory Bound Analysis

Question

Analyze `Y = GELU(X @ W + bias) + residual` on hardware with 1 TB/s HBM bandwidth, 25 MB SRAM, 100 TFLOP/s peak BF16 throughput, and model dimension 4096. Calculate compute time, memory time, and late...

Amazon interview candidates · Answer

Community-reported Amazon interview question with 1 related discussion. The full write-up and related discussions are available to members.

Comments