eric/ericxliu-me
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

deploy: 832aabc376

Browse Source
This commit is contained in:
eric
2025-10-02 08:34:26 +00:00
parent 4808a62cd0
commit 48268a2fc1
24 changed files with 343 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
posts/openwrt-mwan3-wireguard-endpoint-exclusion/index.html Normal file
View File

@@ -0,0 +1,101 @@
<!doctype html><html lang=en><head><title>OpenWrt: Fix WireGuard Connectivity with MWAN3 by Excluding the VPN Endpoint · Eric X. Liu's Personal Page</title><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=color-scheme content="light dark"><meta name=author content="Eric X. Liu"><meta name=description content="
Overview
Link to heading
When using WireGuard together with MWAN3 on OpenWrt, the tunnel can fail to establish or flap when the peer&rsquo;s IP is routed into the tunnel itself. This is a classic routing bootstrap problem: WireGuard wants to route 0.0.0.0/0 into the tunnel, but the UDP packets to the peer&rsquo;s public endpoint also get captured, so they never reach the Internet to bring the tunnel up."><meta name=keywords content="software engineer,performance engineering,Google engineer,tech blog,software development,performance optimization,Eric Liu,engineering blog,mountain biking,Jeep enthusiast,overlanding,camping,outdoor adventures"><meta name=twitter:card content="summary"><meta name=twitter:title content="OpenWrt: Fix WireGuard Connectivity with MWAN3 by Excluding the VPN Endpoint"><meta name=twitter:description content="Overview Link to heading When using WireGuard together with MWAN3 on OpenWrt, the tunnel can fail to establish or flap when the peer’s IP is routed into the tunnel itself. This is a classic routing bootstrap problem: WireGuard wants to route 0.0.0.0/0 into the tunnel, but the UDP packets to the peer’s public endpoint also get captured, so they never reach the Internet to bring the tunnel up."><meta property="og:url" content="/posts/openwrt-mwan3-wireguard-endpoint-exclusion/"><meta property="og:site_name" content="Eric X. Liu's Personal Page"><meta property="og:title" content="OpenWrt: Fix WireGuard Connectivity with MWAN3 by Excluding the VPN Endpoint"><meta property="og:description" content="Overview Link to heading When using WireGuard together with MWAN3 on OpenWrt, the tunnel can fail to establish or flap when the peer’s IP is routed into the tunnel itself. This is a classic routing bootstrap problem: WireGuard wants to route 0.0.0.0/0 into the tunnel, but the UDP packets to the peer’s public endpoint also get captured, so they never reach the Internet to bring the tunnel up."><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2025-09-28T00:00:00+00:00"><meta property="article:modified_time" content="2025-10-02T08:34:05+00:00"><link rel=canonical href=/posts/openwrt-mwan3-wireguard-endpoint-exclusion/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.c8e4eea149ae1dc7c61ba9b0781793711a4e657f7e07a4413f9abc46d52dffc4.css integrity="sha256-yOTuoUmuHcfGG6mweBeTcRpOZX9+B6RBP5q8RtUt/8Q=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=/>Eric X. Liu's Personal Page
</a><input type=checkbox id=menu-toggle>
<label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/posts/>Posts</a></li><li class=navigation-item><a class=navigation-link href=https://chat.ericxliu.me>Chat</a></li><li class=navigation-item><a class=navigation-link href=https://git.ericxliu.me/user/oauth2/Authenitk>Git</a></li><li class=navigation-item><a class=navigation-link href=https://coder.ericxliu.me/api/v2/users/oidc/callback>Coder</a></li><li class=navigation-item><a class=navigation-link href=/>|</a></li><li class=navigation-item><a class=navigation-link href=https://sso.ericxliu.me>Sign in</a></li></ul></section></nav><div class=content><section class="container post"><article><header><div class=post-title><h1 class=title><a class=title-link href=/posts/openwrt-mwan3-wireguard-endpoint-exclusion/>OpenWrt: Fix WireGuard Connectivity with MWAN3 by Excluding the VPN Endpoint</a></h1></div><div class=post-meta><div class=date><span class=posted-on><i class="fa-solid fa-calendar" aria-hidden=true></i>
<time datetime=2025-09-28T00:00:00Z>September 28, 2025
</time></span><span class=reading-time><i class="fa-solid fa-clock" aria-hidden=true></i>
5-minute read</span></div></div></header><div class=post-content><h3 id=overview>Overview
<a class=heading-link href=#overview><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p>When using WireGuard together with MWAN3 on OpenWrt, the tunnel can fail to establish or flap when the peer&rsquo;s IP is routed into the tunnel itself. This is a classic routing bootstrap problem: WireGuard wants to route 0.0.0.0/0 into the tunnel, but the UDP packets to the peer&rsquo;s public endpoint also get captured, so they never reach the Internet to bring the tunnel up.</p><p>This article explains the symptoms, root cause, and a minimal, robust fix: add an MWAN3 rule that forces traffic to the WireGuard peer endpoint IP to go out via a physical WAN policy (not the tunnel). Optionally, assign a metric to the WireGuard interface to keep tables predictable.</p><h3 id=environment>Environment
<a class=heading-link href=#environment><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><ul><li>OpenWrt 24.10.x</li><li>MWAN3 for multi-WAN policy routing</li><li>WireGuard interface configured with broad <code>allowed_ips</code> covering default route (<code>0.0.0.0/1</code> and <code>128.0.0.0/1</code> or <code>0.0.0.0/0</code>)</li></ul><h3 id=symptoms>Symptoms
<a class=heading-link href=#symptoms><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><ul><li><code>wg show</code> indicates the interface is listening, but <code>transfer: 0 B received</code> persists after bringing the tunnel up.</li><li>Intermittent reachability to public IPs until routing settles.</li><li><code>ip route</code> shows multiple defaults via WANs; a host route to the peer IP may exist but is still overridden by policy routing once MWAN3 applies rules.</li></ul><p>Example observations (sanitized):</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>wg show
</span></span><span style=display:flex><span>interface: wireguard
</span></span><span style=display:flex><span> public key: &lt;redacted&gt;
</span></span><span style=display:flex><span> listening port: <span style=color:#a5d6ff>39345</span>
</span></span><span style=display:flex><span>peer: &lt;peer-public-key-redacted&gt;
</span></span><span style=display:flex><span> endpoint: 203.0.113.55:51821
</span></span><span style=display:flex><span> allowed ips: 0.0.0.0/1, 128.0.0.0/1
</span></span><span style=display:flex><span> transfer: <span style=color:#a5d6ff>0</span> B received, 5.6 KiB sent
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>ip route
</span></span><span style=display:flex><span>default via 192.0.2.1 dev wan0 proto static src 192.0.2.10
</span></span><span style=display:flex><span>default via 198.51.100.1 dev wan1 proto static src 198.51.100.10 metric <span style=color:#a5d6ff>20</span>
</span></span><span style=display:flex><span>203.0.113.55 via 198.51.100.1 dev wan1 proto static metric <span style=color:#a5d6ff>20</span>
</span></span></code></pre></div><h3 id=root-cause>Root Cause
<a class=heading-link href=#root-cause><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p>With default-route <code>allowed_ips</code>, WireGuard installs routes so that all outbound traffic prefers the tunnel. MWAN3 then applies policy rules that also match “all traffic,” including the UDP packets to the WireGuard peer’s public IP. If those packets are selected to go via the <code>wireguard</code> interface (or a table whose default is the tunnel), the handshake cannot succeed. This creates a chicken-and-egg dependency.</p><h3 id=fix-exclude-the-wireguard-endpoint-from-mwan3-default-policy>Fix: Exclude the WireGuard Endpoint from MWAN3 Default Policy
<a class=heading-link href=#fix-exclude-the-wireguard-endpoint-from-mwan3-default-policy><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p>Force traffic to the WireGuard peer public endpoint to use a physical WAN policy. This guarantees the handshake packets always reach the Internet outside of the tunnel.</p><p>Steps:</p><ol><li>Resolve the peer endpoint IP (if you only have a hostname)</li></ol><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>nslookup vpn.example.com
</span></span><span style=display:flex><span><span style=color:#8b949e;font-style:italic># =&gt; use the returned A/AAAA address(es) in the rule below</span>
</span></span></code></pre></div><ol start=2><li>Add an MWAN3 rule targeting the endpoint IP</li></ol><p>Edit <code>/etc/config/mwan3</code> and place this rule before the default v4 rule so it takes precedence:</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-fallback data-lang=fallback><span style=display:flex><span>config rule &#39;wireguard_endpoint&#39;
</span></span><span style=display:flex><span> option dest_ip &#39;203.0.113.55&#39; # peer public IP
</span></span><span style=display:flex><span> option proto &#39;udp&#39;
</span></span><span style=display:flex><span> option use_policy &#39;wan_only&#39; # a policy that prefers a physical WAN
</span></span><span style=display:flex><span> option family &#39;ipv4&#39;
</span></span></code></pre></div><p>Notes:</p><ul><li>Use the actual public IP of your WireGuard server. MWAN3 rules match IPs, not hostnames.</li><li>If you have multiple WAN policies (e.g., <code>wan_only</code>, <code>wphone_only</code>), choose the one that must carry the VPN handshake.</li></ul><ol start=3><li>(Optional) Assign a metric on the WireGuard interface</li></ol><p>This is not strictly required for the fix but keeps routing behavior deterministic when multiple defaults exist.</p><p>Edit <code>/etc/config/network</code>:</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-fallback data-lang=fallback><span style=display:flex><span>config interface &#39;wireguard&#39;
</span></span><span style=display:flex><span> option proto &#39;wireguard&#39;
</span></span><span style=display:flex><span> option private_key &#39;&lt;redacted&gt;&#39;
</span></span><span style=display:flex><span> list addresses &#39;192.168.3.2/32&#39;
</span></span><span style=display:flex><span> option metric &#39;5&#39;
</span></span></code></pre></div><ol start=4><li>Apply changes</li></ol><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>/etc/init.d/network restart <span style=color:#ff7b72;font-weight:700>&amp;&amp;</span> /etc/init.d/mwan3 restart
</span></span></code></pre></div><h3 id=validation>Validation
<a class=heading-link href=#validation><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p>Confirm that the endpoint is routed via a physical WAN and that the tunnel is passing traffic.</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span><span style=color:#8b949e;font-style:italic># Verify policy routing for the endpoint</span>
</span></span><span style=display:flex><span>ip route get 203.0.113.55
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#8b949e;font-style:italic># MWAN3 status should show your WANs online</span>
</span></span><span style=display:flex><span>mwan3 status | sed -n <span style=color:#a5d6ff>&#39;1,120p&#39;</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#8b949e;font-style:italic># WireGuard should show RX increasing after a few seconds</span>
</span></span><span style=display:flex><span>wg show
</span></span></code></pre></div><p>Expected results:</p><ul><li><code>ip route get &lt;peer-ip></code> resolves to a physical WAN device/policy, not the <code>wireguard</code> interface.</li><li><code>wg show</code> shows non-zero bytes received and a recent handshake time.</li></ul><h3 id=operational-considerations>Operational Considerations
<a class=heading-link href=#operational-considerations><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><ul><li>Endpoint IP changes: If the server endpoint is behind DDNS, you must update the rule when its IP changes. Options include:<ul><li>Use a small script triggered by DDNS updates to modify the MWAN3 rule and reload.</li><li>Maintain an IP set and populate it from DDNS; match the set in firewall/PBR and keep MWAN3 in sync.</li></ul></li><li>IPv6: Repeat the approach with an IPv6 rule if your peer uses IPv6. Ensure <code>family 'ipv6'</code> and the correct policy are set.</li><li>Multiple peers: Create one rule per peer endpoint IP.</li><li>Ordering: Keep the endpoint rule above broad default rules so it always wins.</li></ul><h3 id=minimal-example-config-snippets-sanitized>Minimal Example Config Snippets (Sanitized)
<a class=heading-link href=#minimal-example-config-snippets-sanitized><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p><code>/etc/config/network</code> (relevant parts):</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-fallback data-lang=fallback><span style=display:flex><span>config interface &#39;wireguard&#39;
</span></span><span style=display:flex><span> option proto &#39;wireguard&#39;
</span></span><span style=display:flex><span> option private_key &#39;&lt;redacted&gt;&#39;
</span></span><span style=display:flex><span> list addresses &#39;192.168.3.2/32&#39;
</span></span><span style=display:flex><span> option metric &#39;5&#39;
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>config wireguard_wireguard
</span></span><span style=display:flex><span> option description &#39;peer-1&#39;
</span></span><span style=display:flex><span> option public_key &#39;&lt;peer-public-key-redacted&gt;&#39;
</span></span><span style=display:flex><span> option endpoint_host &#39;vpn.example.com&#39;
</span></span><span style=display:flex><span> option endpoint_port &#39;51821&#39;
</span></span><span style=display:flex><span> list allowed_ips &#39;0.0.0.0/1&#39;
</span></span><span style=display:flex><span> list allowed_ips &#39;128.0.0.0/1&#39;
</span></span><span style=display:flex><span> option route_allowed_ips &#39;1&#39;
</span></span></code></pre></div><p><code>/etc/config/mwan3</code> (relevant parts):</p><div class=highlight><pre tabindex=0 style=color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-fallback data-lang=fallback><span style=display:flex><span>config policy &#39;wan_only&#39;
</span></span><span style=display:flex><span> list use_member &#39;wan_m1_w3&#39;
</span></span><span style=display:flex><span> option last_resort &#39;unreachable&#39;
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>config rule &#39;wireguard_endpoint&#39;
</span></span><span style=display:flex><span> option dest_ip &#39;203.0.113.55&#39;
</span></span><span style=display:flex><span> option proto &#39;udp&#39;
</span></span><span style=display:flex><span> option use_policy &#39;wan_only&#39;
</span></span><span style=display:flex><span> option family &#39;ipv4&#39;
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>config rule &#39;default_rule_v4&#39;
</span></span><span style=display:flex><span> option dest_ip &#39;0.0.0.0/0&#39;
</span></span><span style=display:flex><span> option use_policy &#39;wan_only&#39;
</span></span><span style=display:flex><span> option family &#39;ipv4&#39;
</span></span><span style=display:flex><span> option proto &#39;all&#39;
</span></span><span style=display:flex><span> option sticky &#39;0&#39;
</span></span></code></pre></div><h3 id=why-this-works>Why This Works
<a class=heading-link href=#why-this-works><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><p>The explicit MWAN3 rule ensures that traffic to the peer’s public IP bypasses any routes that prefer the tunnel. This breaks the bootstrap loop and guarantees handshake packets traverse a real WAN uplink. Once the tunnel is established, the broad <code>allowed_ips</code> continue to route general traffic through WireGuard as intended.</p><h3 id=references>References
<a class=heading-link href=#references><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
<span class=sr-only>Link to heading</span></a></h3><ul><li>Session log and configs (internal): <code>~/Downloads/chat-MWAN3 WireGuard Routing Fix 🌐.txt</code></li><li>OpenWrt MWAN3 documentation: <code>https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3</code></li><li>WireGuard documentation: <code>https://www.wireguard.com/</code></li><li>OpenWrt WireGuard (UG): <code>https://openwrt.org/docs/guide-user/services/vpn/wireguard</code></li></ul></div><footer><div id=disqus_thread></div><script>window.disqus_config=function(){},function(){if(["localhost","127.0.0.1"].indexOf(window.location.hostname)!=-1){document.getElementById("disqus_thread").innerHTML="Disqus comments not available by default when the website is previewed locally.";return}var t=document,e=t.createElement("script");e.async=!0,e.src="//ericxliu-me.disqus.com/embed.js",e.setAttribute("data-timestamp",+new Date),(t.head||t.body).appendChild(e)}(),document.addEventListener("themeChanged",function(){document.readyState=="complete"&&DISQUS.reset({reload:!0,config:disqus_config})})</script></footer></article><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.css integrity=sha384-vKruj+a13U8yHIkAyGgK1J3ArTLzrFGBbBc0tDp4ad/EyewESeXE/Iv67Aj8gKZ0 crossorigin=anonymous><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.js integrity=sha384-PwRUT/YqbnEjkZO0zZxNqcxACrXe+j766U2amXcgMg5457rve2Y7I6ZJSm2A0mS4 crossorigin=anonymous></script><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/contrib/auto-render.min.js integrity=sha384-+VBxd3r6XgURycqtZ117nYw44OOcIax56Z4dCRWbxyPt0Koah1uHoK0o4+/RRE05 crossorigin=anonymous onload='renderMathInElement(document.body,{delimiters:[{left:"$$",right:"$$",display:!0},{left:"$",right:"$",display:!1},{left:"\\(",right:"\\)",display:!1},{left:"\\[",right:"\\]",display:!0}]})'></script></section></div><footer class=footer><section class=container>©
2016 -
2025
Eric X. Liu
<a href="https://git.ericxliu.me/eric/ericxliu-me/commit/832aabc">[832aabc]</a></section></footer></main><script src=/js/coder.min.6ae284be93d2d19dad1f02b0039508d9aab3180a12a06dcc71b0b0ef7825a317.js integrity="sha256-auKEvpPS0Z2tHwKwA5UI2aqzGAoSoG3McbCw73gloxc="></script><script defer src=https://static.cloudflareinsights.com/beacon.min.js data-cf-beacon='{"token": "987638e636ce4dbb932d038af74c17d1"}'></script></body></html>