From df3c006010d153524fe5a8619e8b84db51576940 Mon Sep 17 00:00:00 2001 From: eric Date: Thu, 14 Aug 2025 06:51:16 +0000 Subject: [PATCH] deploy: 2a5ee6dd11e94b45c082f09173851eafcb8c7aea --- 404.html | 2 +- about/index.html | 2 +- categories/index.html | 2 +- index.html | 2 +- index.xml | 7 ++- .../index.html | 2 +- .../index.html | 2 +- .../index.html | 2 +- posts/index.html | 5 +- posts/index.xml | 7 ++- .../index.html | 2 +- .../index.html | 62 +++++++++++++++++++ posts/supabase-deep-dive/index.html | 2 +- .../index.html | 2 +- posts/useful/index.html | 2 +- sitemap.xml | 2 +- tags/index.html | 2 +- 17 files changed, 90 insertions(+), 17 deletions(-) create mode 100644 posts/secure-boot-dkms-and-mok-on-proxmox-debian/index.html diff --git a/404.html b/404.html index 142bdf9..c14d770 100644 --- a/404.html +++ b/404.html @@ -4,4 +4,4 @@ 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/about/index.html b/about/index.html index 740abc2..d9ec3db 100644 --- a/about/index.html +++ b/about/index.html @@ -4,4 +4,4 @@ 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/categories/index.html b/categories/index.html index 0a5af57..ffce625 100644 --- a/categories/index.html +++ b/categories/index.html @@ -4,4 +4,4 @@ 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/index.html b/index.html index d06cd49..e019e15 100644 --- a/index.html +++ b/index.html @@ -4,4 +4,4 @@ 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/index.xml b/index.xml index 5fd3f5e..ca5a0c9 100644 --- a/index.xml +++ b/index.xml @@ -1,4 +1,9 @@ -Eric X. Liu's Personal Page/Recent content on Eric X. Liu's Personal PageHugoenFri, 08 Aug 2025 17:36:52 +0000Beyond Words: How RVQ Teaches LLMs to See and Hear/posts/how-rvq-teaches-llms-to-see-and-hear/Thu, 07 Aug 2025 00:00:00 +0000/posts/how-rvq-teaches-llms-to-see-and-hear/<p>Large Language Models (LLMs) are masters of text, but the world is not made of text alone. It’s a symphony of sights, sounds, and experiences. The ultimate goal for AI is to understand this rich, multi-modal world as we do. But how do you teach a model that thinks in words to understand a picture of a sunset or the melody of a song?</p> +Eric X. Liu's Personal Page/Recent content on Eric X. Liu's Personal PageHugoenThu, 14 Aug 2025 06:50:22 +0000Fixing GPU Operator Pods Stuck in Init: Secure Boot, DKMS, and MOK on Proxmox + Debian/posts/secure-boot-dkms-and-mok-on-proxmox-debian/Sat, 09 Aug 2025 00:00:00 +0000/posts/secure-boot-dkms-and-mok-on-proxmox-debian/<p>I hit an issue where all GPU Operator pods on one node were stuck in Init after migrating from Legacy BIOS to UEFI. The common error was NVIDIA components waiting for “toolkit-ready,” while the toolkit init container looped with:</p> +<ul> +<li>nvidia-smi failed to communicate with the NVIDIA driver</li> +<li>modprobe nvidia → “Key was rejected by service”</li> +</ul> +<p>That message is the tell: Secure Boot is enabled and the kernel refuses to load modules not signed by a trusted key.</p>Beyond Words: How RVQ Teaches LLMs to See and Hear/posts/how-rvq-teaches-llms-to-see-and-hear/Thu, 07 Aug 2025 00:00:00 +0000/posts/how-rvq-teaches-llms-to-see-and-hear/<p>Large Language Models (LLMs) are masters of text, but the world is not made of text alone. It’s a symphony of sights, sounds, and experiences. The ultimate goal for AI is to understand this rich, multi-modal world as we do. But how do you teach a model that thinks in words to understand a picture of a sunset or the melody of a song?</p> <p>The answer lies in creating a universal language—a bridge between the continuous, messy world of pixels and audio waves and the discrete, structured world of language tokens. One of the most elegant and powerful tools for building this bridge is <strong>Residual Vector Quantization (RVQ)</strong>.</p>Supabase Deep Dive: It's Not Magic, It's Just Postgres/posts/supabase-deep-dive/Sun, 03 Aug 2025 00:00:00 +0000/posts/supabase-deep-dive/<p>In the world of Backend-as-a-Service (BaaS), platforms are often treated as magic boxes. You push data in, you get data out, and you hope the magic inside scales. While this simplicity is powerful, it can obscure the underlying mechanics, leaving developers wondering what&rsquo;s really going on.</p> <p>Supabase enters this space with a radically different philosophy: <strong>transparency</strong>. It provides the convenience of a BaaS, but it’s built on the world&rsquo;s most trusted relational database: PostgreSQL. The &ldquo;magic&rdquo; isn&rsquo;t a proprietary black box; it&rsquo;s a carefully assembled suite of open-source tools that enhance Postgres, not hide it.</p>A Deep Dive into PPO for Language Models/posts/a-deep-dive-into-ppo-for-language-models/Sat, 02 Aug 2025 00:00:00 +0000/posts/a-deep-dive-into-ppo-for-language-models/<p>Large Language Models (LLMs) have demonstrated astonishing capabilities, but out-of-the-box, they are simply powerful text predictors. They don&rsquo;t inherently understand what makes a response helpful, harmless, or aligned with human values. The technique that has proven most effective at bridging this gap is Reinforcement Learning from Human Feedback (RLHF), and at its heart lies a powerful algorithm: Proximal Policy Optimization (PPO).</p> <p>You may have seen diagrams like the one below, which outlines the RLHF training process. It can look intimidating, with a web of interconnected models, losses, and data flows.</p>Mixture-of-Experts (MoE) Models Challenges & Solutions in Practice/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/Wed, 02 Jul 2025 00:00:00 +0000/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/<p>Mixture-of-Experts (MoEs) are neural network architectures that allow different parts of the model (called &ldquo;experts&rdquo;) to specialize in different types of inputs. A &ldquo;gating network&rdquo; or &ldquo;router&rdquo; learns to dispatch each input (or &ldquo;token&rdquo;) to a subset of these experts. While powerful for scaling models, MoEs introduce several practical challenges.</p> diff --git a/posts/a-deep-dive-into-ppo-for-language-models/index.html b/posts/a-deep-dive-into-ppo-for-language-models/index.html index 22c1c0d..4b07160 100644 --- a/posts/a-deep-dive-into-ppo-for-language-models/index.html +++ b/posts/a-deep-dive-into-ppo-for-language-models/index.html @@ -23,4 +23,4 @@ where δ_t = r_t + γV(s_{t+1}) - V(s_t)

  • γ (gam 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/index.html b/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/index.html index b8af4f7..f4752ba 100644 --- a/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/index.html +++ b/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/index.html @@ -20,4 +20,4 @@ Our overarching philosophy is simple: isolate and change only one variable at a 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/how-rvq-teaches-llms-to-see-and-hear/index.html b/posts/how-rvq-teaches-llms-to-see-and-hear/index.html index 24353e2..390afee 100644 --- a/posts/how-rvq-teaches-llms-to-see-and-hear/index.html +++ b/posts/how-rvq-teaches-llms-to-see-and-hear/index.html @@ -18,4 +18,4 @@ The answer lies in creating a universal language—a bridge between the continuo 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/index.html b/posts/index.html index 27554ca..3b033bf 100644 --- a/posts/index.html +++ b/posts/index.html @@ -1,6 +1,7 @@ Posts · Eric X. Liu's Personal Page

    Posts

    Posts

    \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/index.xml b/posts/index.xml index 4f6b254..de918e7 100644 --- a/posts/index.xml +++ b/posts/index.xml @@ -1,4 +1,9 @@ -Posts on Eric X. Liu's Personal Page/posts/Recent content in Posts on Eric X. Liu's Personal PageHugoenFri, 08 Aug 2025 17:36:52 +0000Beyond Words: How RVQ Teaches LLMs to See and Hear/posts/how-rvq-teaches-llms-to-see-and-hear/Thu, 07 Aug 2025 00:00:00 +0000/posts/how-rvq-teaches-llms-to-see-and-hear/<p>Large Language Models (LLMs) are masters of text, but the world is not made of text alone. It’s a symphony of sights, sounds, and experiences. The ultimate goal for AI is to understand this rich, multi-modal world as we do. But how do you teach a model that thinks in words to understand a picture of a sunset or the melody of a song?</p> +Posts on Eric X. Liu's Personal Page/posts/Recent content in Posts on Eric X. Liu's Personal PageHugoenThu, 14 Aug 2025 06:50:22 +0000Fixing GPU Operator Pods Stuck in Init: Secure Boot, DKMS, and MOK on Proxmox + Debian/posts/secure-boot-dkms-and-mok-on-proxmox-debian/Sat, 09 Aug 2025 00:00:00 +0000/posts/secure-boot-dkms-and-mok-on-proxmox-debian/<p>I hit an issue where all GPU Operator pods on one node were stuck in Init after migrating from Legacy BIOS to UEFI. The common error was NVIDIA components waiting for “toolkit-ready,” while the toolkit init container looped with:</p> +<ul> +<li>nvidia-smi failed to communicate with the NVIDIA driver</li> +<li>modprobe nvidia → “Key was rejected by service”</li> +</ul> +<p>That message is the tell: Secure Boot is enabled and the kernel refuses to load modules not signed by a trusted key.</p>Beyond Words: How RVQ Teaches LLMs to See and Hear/posts/how-rvq-teaches-llms-to-see-and-hear/Thu, 07 Aug 2025 00:00:00 +0000/posts/how-rvq-teaches-llms-to-see-and-hear/<p>Large Language Models (LLMs) are masters of text, but the world is not made of text alone. It’s a symphony of sights, sounds, and experiences. The ultimate goal for AI is to understand this rich, multi-modal world as we do. But how do you teach a model that thinks in words to understand a picture of a sunset or the melody of a song?</p> <p>The answer lies in creating a universal language—a bridge between the continuous, messy world of pixels and audio waves and the discrete, structured world of language tokens. One of the most elegant and powerful tools for building this bridge is <strong>Residual Vector Quantization (RVQ)</strong>.</p>Supabase Deep Dive: It's Not Magic, It's Just Postgres/posts/supabase-deep-dive/Sun, 03 Aug 2025 00:00:00 +0000/posts/supabase-deep-dive/<p>In the world of Backend-as-a-Service (BaaS), platforms are often treated as magic boxes. You push data in, you get data out, and you hope the magic inside scales. While this simplicity is powerful, it can obscure the underlying mechanics, leaving developers wondering what&rsquo;s really going on.</p> <p>Supabase enters this space with a radically different philosophy: <strong>transparency</strong>. It provides the convenience of a BaaS, but it’s built on the world&rsquo;s most trusted relational database: PostgreSQL. The &ldquo;magic&rdquo; isn&rsquo;t a proprietary black box; it&rsquo;s a carefully assembled suite of open-source tools that enhance Postgres, not hide it.</p>A Deep Dive into PPO for Language Models/posts/a-deep-dive-into-ppo-for-language-models/Sat, 02 Aug 2025 00:00:00 +0000/posts/a-deep-dive-into-ppo-for-language-models/<p>Large Language Models (LLMs) have demonstrated astonishing capabilities, but out-of-the-box, they are simply powerful text predictors. They don&rsquo;t inherently understand what makes a response helpful, harmless, or aligned with human values. The technique that has proven most effective at bridging this gap is Reinforcement Learning from Human Feedback (RLHF), and at its heart lies a powerful algorithm: Proximal Policy Optimization (PPO).</p> <p>You may have seen diagrams like the one below, which outlines the RLHF training process. It can look intimidating, with a web of interconnected models, losses, and data flows.</p>Mixture-of-Experts (MoE) Models Challenges & Solutions in Practice/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/Wed, 02 Jul 2025 00:00:00 +0000/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/<p>Mixture-of-Experts (MoEs) are neural network architectures that allow different parts of the model (called &ldquo;experts&rdquo;) to specialize in different types of inputs. A &ldquo;gating network&rdquo; or &ldquo;router&rdquo; learns to dispatch each input (or &ldquo;token&rdquo;) to a subset of these experts. While powerful for scaling models, MoEs introduce several practical challenges.</p> diff --git a/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/index.html b/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/index.html index 1f5aeb7..151768c 100644 --- a/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/index.html +++ b/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/index.html @@ -44,4 +44,4 @@ The Top-K routing mechanism, as illustrated in the provided ima 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/secure-boot-dkms-and-mok-on-proxmox-debian/index.html b/posts/secure-boot-dkms-and-mok-on-proxmox-debian/index.html new file mode 100644 index 0000000..823d4af --- /dev/null +++ b/posts/secure-boot-dkms-and-mok-on-proxmox-debian/index.html @@ -0,0 +1,62 @@ +Fixing GPU Operator Pods Stuck in Init: Secure Boot, DKMS, and MOK on Proxmox + Debian · Eric X. Liu's Personal Page

    Fixing GPU Operator Pods Stuck in Init: Secure Boot, DKMS, and MOK on Proxmox + Debian

    + +3-minute read

    I hit an issue where all GPU Operator pods on one node were stuck in Init after migrating from Legacy BIOS to UEFI. The common error was NVIDIA components waiting for “toolkit-ready,” while the toolkit init container looped with:

    • nvidia-smi failed to communicate with the NVIDIA driver
    • modprobe nvidia → “Key was rejected by service”

    That message is the tell: Secure Boot is enabled and the kernel refuses to load modules not signed by a trusted key.

    Environment + +Link to heading

    • Proxmox VM (QEMU/KVM) 8.4.9
    • Debian 12 (bookworm), kernel 6.1
    • GPU: NVIDIA Tesla V100 (GV100GL)
    • NVIDIA driver installed via Debian packages (nvidia-driver, nvidia-kernel-dkms)

    Root Cause + +Link to heading

    • Secure Boot enabled (verified with mokutil --sb-state)
    • NVIDIA DKMS modules were built, but the signing key was not trusted by the UEFI shim/firmware
    • VM booted via the fallback “UEFI QEMU HARDDISK” path (not shim), so MOK requests didn’t run; no MOK screen

    Strategy + +Link to heading

    Keep Secure Boot on; get modules trusted. That requires:

    1. Ensure the VM boots via shim (so MOK can work)
    2. Make sure DKMS signs modules with a MOK key/cert
    3. Enroll that MOK into the firmware via shim’s MokManager

    Step 1 — Boot via shim and persist EFI variables + +Link to heading

    In Proxmox (VM stopped):

    • BIOS: OVMF (UEFI)
    • Add EFI Disk (stores OVMF VARS; required for MOK)
    • Machine: q35
    • Enable Secure Boot (option shows only with OVMF + EFI Disk)

    Inside Debian:

    • Ensure ESP is mounted at /boot/efi
    • Install signed boot stack:
      sudo apt install shim-signed grub-efi-amd64-signed efibootmgr mokutil
+sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=debian
+sudo update-grub
+
    • Create/verify a boot entry that points to shim:
      sudo efibootmgr -c -d /dev/sda -p 15 -L "debian" -l '\EFI\debian\shimx64.efi'
+sudo efibootmgr -o 0002,0001,0000     # make shim (0002) first
+sudo efibootmgr -n 0002               # BootNext shim for the next reboot
+

    Tip: If NVRAM resets or fallback path is used, copy as a fallback:

    sudo mkdir -p /boot/efi/EFI/BOOT
+sudo cp /boot/efi/EFI/debian/shimx64.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
+sudo cp /boot/efi/EFI/debian/{mmx64.efi,grubx64.efi} /boot/efi/EFI/BOOT/
+

    Step 2 — Make DKMS sign NVIDIA modules with a MOK + +Link to heading

    Debian already generated a DKMS key at /var/lib/dkms/mok.key. Create an X.509 cert in DER format:

    sudo openssl req -new -x509 \
+  -key /var/lib/dkms/mok.key \
+  -out /var/lib/dkms/mok.der \
+  -outform DER \
+  -subj "/CN=DKMS MOK/" \
+  -days 36500
+

    Enable DKMS signing:

    sudo sed -i 's|^mok_signing_key=.*|mok_signing_key=/var/lib/dkms/mok.key|' /etc/dkms/framework.conf
+sudo sed -i 's|^mok_certificate=.*|mok_certificate=/var/lib/dkms/mok.der|' /etc/dkms/framework.conf
+

    Rebuild/install modules (signs them now):

    sudo dkms build nvidia/$(modinfo -F version nvidia) -k $(uname -r) --force
+sudo dkms install nvidia/$(modinfo -F version nvidia) -k $(uname -r) --force
+

    Step 3 — Enroll the MOK via shim (MokManager) + +Link to heading

    Queue the cert and set a longer prompt timeout:

    sudo mokutil --revoke-import
+sudo mokutil --import /var/lib/dkms/mok.der
+sudo mokutil --timeout 30
+sudo efibootmgr -n 0002  # ensure next boot goes through shim
+

    Reboot to the VM console (not SSH). In the blue MOK UI:

    • Enroll MOK → Continue → Yes → enter password → reboot

    If arrow keys don’t work in Proxmox noVNC:

    • Use SPICE (virt-viewer), or
    • From the Proxmox host, send keys:
      • qm sendkey <VMID> down, qm sendkey <VMID> ret, qm sendkey <VMID> esc

    Verification + +Link to heading

    sudo mokutil --test-key /var/lib/dkms/mok.der   # “already enrolled”
+sudo modprobe nvidia
+nvidia-smi
+kubectl -n gpu-operator get pods -o wide
+

    Once the module loads, GPU Operator pods on that node leave Init and become Ready.

    Key Insights + +Link to heading

    • “Key was rejected by service” during modprobe nvidia means Secure Boot rejected an untrusted module.
    • Without shim in the boot path (or without a persistent EFI vars disk), mokutil --import won’t surface a MOK screen.
    • DKMS will not sign modules unless configured; set mok_signing_key and mok_certificate in /etc/dkms/framework.conf.
    • If you cannot or don’t want to use MOK, the pragmatic dev choice is to disable Secure Boot in OVMF. For production, prefer shim+MOK.

    References + +Link to heading

    • Proxmox Secure Boot setup (shim + MOK, EFI vars, DKMS): Proxmox docs
    \ No newline at end of file diff --git a/posts/supabase-deep-dive/index.html b/posts/supabase-deep-dive/index.html index 891faac..f4f5258 100644 --- a/posts/supabase-deep-dive/index.html +++ b/posts/supabase-deep-dive/index.html @@ -90,4 +90,4 @@ Supabase enters this space with a radically different philosophy: transparency. 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/index.html b/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/index.html index 414888f..96dfb71 100644 --- a/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/index.html +++ b/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/index.html @@ -30,4 +30,4 @@ But to truly understand the field, we must look at the pivotal models that explo 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/posts/useful/index.html b/posts/useful/index.html index aaf65f8..f918343 100644 --- a/posts/useful/index.html +++ b/posts/useful/index.html @@ -9,4 +9,4 @@ One-minute read
    • [c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml index 5c9c927..45bca2a 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -1 +1 @@ -/posts/how-rvq-teaches-llms-to-see-and-hear/2025-08-08T17:36:52+00:00weekly0.5/2025-08-08T17:36:52+00:00weekly0.5/posts/2025-08-08T17:36:52+00:00weekly0.5/posts/supabase-deep-dive/2025-08-04T03:59:37+00:00weekly0.5/posts/a-deep-dive-into-ppo-for-language-models/2025-08-03T03:28:39+00:00weekly0.5/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/2025-08-03T06:02:48+00:00weekly0.5/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/2025-08-03T03:41:10+00:00weekly0.5/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/2025-08-03T04:20:20+00:00weekly0.5/posts/useful/2025-08-03T08:37:28-07:00weekly0.5/about/2020-06-16T23:30:17-07:00weekly0.5/categories/weekly0.5/tags/weekly0.5 \ No newline at end of file +/2025-08-14T06:50:22+00:00weekly0.5/posts/secure-boot-dkms-and-mok-on-proxmox-debian/2025-08-14T06:50:22+00:00weekly0.5/posts/2025-08-14T06:50:22+00:00weekly0.5/posts/how-rvq-teaches-llms-to-see-and-hear/2025-08-08T17:36:52+00:00weekly0.5/posts/supabase-deep-dive/2025-08-04T03:59:37+00:00weekly0.5/posts/a-deep-dive-into-ppo-for-language-models/2025-08-03T03:28:39+00:00weekly0.5/posts/mixture-of-experts-moe-models-challenges-solutions-in-practice/2025-08-03T06:02:48+00:00weekly0.5/posts/t5-the-transformer-that-zigged-when-others-zagged-an-architectural-deep-dive/2025-08-03T03:41:10+00:00weekly0.5/posts/espresso-theory-application-a-guide-for-the-breville-barista-pro/2025-08-03T04:20:20+00:00weekly0.5/posts/useful/2025-08-03T08:37:28-07:00weekly0.5/about/2020-06-16T23:30:17-07:00weekly0.5/categories/weekly0.5/tags/weekly0.5 \ No newline at end of file diff --git a/tags/index.html b/tags/index.html index 00fb3a6..5ef3545 100644 --- a/tags/index.html +++ b/tags/index.html @@ -4,4 +4,4 @@ 2016 - 2025 Eric X. Liu -[c9ed800] \ No newline at end of file +[2a5ee6d] \ No newline at end of file