http://localhost:1313/blog/ Recent content on Hugo -- gohugo.io en © 2026 Mon, 16 Feb 2026 00:00:00 +0000 http://localhost:1313/blog/posts/metagrads/ Mon, 16 Feb 2026 00:00:00 +0000 http://localhost:1313/blog/posts/metagrads/ <h2 class="relative group">Introduction <div id="introduction" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#introduction" aria-label="Anchor">#</a> </span> </h2> <p>One of the most critical components of training a foundation model (e.g., an LLM) is the choice of the training data. A key challenge in designing the training data mix is estimating the value of a given data source.<sup class="hovernote" tabindex="0" data-note="Example hover footnote: the value depends on how much the data improves downstream evaluation metrics.">1</sup> This challenge underlies questions such as:</p>