Luminous broad Hα emission and red rest-optical spectral energy distributions (SEDs) are the hallmark of compact little red dots (LRDs), implying highly attenuated dusty starbursts and/or obscured active galactic nuclei (AGN). However, the lack of observed far-infrared (FIR) emission has proved difficult to reconcile with the implied attenuated luminosity in these models. Here, we utilize deep new Atacama Large Millimeter/submillimeter Array imaging, new and existing JWST/MIRI imaging, and archival Spitzer/Herschel imaging of two of the rest-optically brightest LRDs (z = 3.1 and z = 4.47) to place the strongest constraints on the IR luminosity in LRDs to date. The detections at λrest = 1–4 μm imply flat slopes in the rest-IR, ruling out a contribution from hot (T ≳ 500 K) dust. Similarly, FIR nondetections rule out any appreciable cold (T ≲ 75 K) dust component. Assuming energy balance, these observations are inconsistent with the typical FIR dust emission of dusty starbursts and quasar tori, which usually show a mixture of cold and hot dust. Additionally, our [C II] nondetections rule out typical dusty starbursts. We compute empirical maximum IR SEDs and find that both LRDs must have log(LIR/L☉)≲12.2 at the 3σ level. These limits are in tension with the predictions of rest-optical spectrophotometric fits, be they galaxy-only, AGN-only, or composite. It is unlikely that LRDs are highly dust-reddened intrinsically blue sources with a dust temperature distribution that conspires to avoid current observing facilities. Rather, we favor an intrinsically redder LRD SED model that alleviates the need for strong dust attenuation.