11 thoughts on - Diagnosing IPv6 Routing

• I struggled with this under CentOS 7. I think there is a bug.

  You can run /usr/sbin/radvdump to print out RAs. Leave it running for some minutes.

  I had this in my /etc/sysctl.d/50-net6.conf (on C7):

  #
  # IPv6 Forwarding
  #
  net.ipv6.conf.all.forwarding = 1
  net.ipv6.conf.default.forwarding = 1
  net.ipv6.conf.all.accept_ra = 1
  net.ipv6.conf.default.accept_ra = 1
  #
  # Fix bug to received RAs from Router
  # Disable forwarding on enp4s0f1 interface so we still get RAs
  #
  net.ipv6.conf.enp4s0f1.forwarding = 0
  #net.ipv6.conf.enp4s0f1.accept_ra = 1

  where enp4s0f1 is the WAN interface. Note that ipv6 forwarding still works. I used my C7 as a firewall/gateway.

  I am running 8.1 now.

  Alan

• Once upon a time, Kenneth Porter said:

  I haven’t touched AT&T’s IPv6, but the typical way WAN IPv6 works is:

  – gateway sends a router solicitation and gets a router advertisement
  with “stateful config” set, which tells gateway to do DHCPv6 (but
  default route comes from RA)
  – gateway does DHCPv6 to get a WAN IP
  – after that completes, gateway does DHCPv6 for prefix delegation

  If you are running NetworkManager, then IIRC the accept_ra flag doesn’t matter, because NM manages everything rather than have the kernel handle autoconfiguration (because NM needs to know what’s going on with IPs).

  As for watching, “tcpdump -p -i -v ip6″ should show everything (and since you don’t have any routing yet, you don’t really need to filter out anything else!). You could filter “ip6 and multicast”, because RAs and DHCPv6 (and ND, neighbor discovery, the counterpart to ARP) are all multicast.

• I’m not seeing any outbound IPv6 traffic from my CentOS 7 box on the WAN
  interface. I do see RA’s emitting from the LAN interface, from radvd. Is there some setting in NM tells it to send solicitations? Is there some way to push one manually?

• Once upon a time, Kenneth Porter said:

  What’s in /etc/sysconfig/network-scripts/ifcfg-? I wonder if you have IPv6 disabled.

  I’m not using a “regular” (CentOS, Fedora, etc.) Linux as a gateway; I
  have OpenWRT on a dedicated box. I couldn’t find a way to handle the prefix delegation with the typical desktop/server tools (but it has been a while since I looked). OpenWRT has their own daemon for that.

  However, my local systems are all sending RA solicitations and getting DHCPv6-assigned addresses with NetworkManager (which matches the first steps of what you need on the WAN, just not the prefix delegation).

• Pasted below. V6 definitely works. I have a second server and gave it a WAN address and I can connect between them using their WAN addresses. That’s what told me that my ip6tables weren’t screwed up and that the problem lay elsewhere. That’s when I hit on looking at the route table and seeing no default IPv6 route.

  I’m using OpenWrt at home and it’s working mostly fine there. Except with my Android phone. I’m not getting a DNS setting for V6, but I do have the setting in the router’s config file. The Win10 clients work fine, though. Apparently Android has issues with DHCPv6, and I’m betting it’s interfering with my SLAAC config.

  LAN config, ifcfg-em1:

  TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=no IPV6_DEFROUTE=no IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=em1
  UUID

• Once upon a time, Kenneth Porter said:

  Yeah, Android refuses to support DHCPv6, so you either have to have IPv4
  DNS or SLAAC. I have IPv4 DNS on my home network, so don’t have an issue. I did just look, and OpenWRT is putting the DNS option for SLAAC
  in the RA, so that should work too (but I think that’s something relatively recent for OpenWRT).

  I didn’t get that you have a static assignment (presumably a business connection) – they may not do RAs on that (I don’t at my ISP job). Business connections (or at least, connections with static assignments)
  tend to operate differently. For that, they should have given you a static v6 address and gateway, just like they did for v4.

  So… there’s one thing you could try (but probably won’t work to a regular router interface) – see if there’s a MAC-derived fe80::/64
  link-local address on their end. Get the MAC of the gateway from the v4
  ARP entry and expand it to a LL v6 address as fe80::xxxx:xxff:fexx:xxxx
  (split the MAC, put ff:fe in the middle). Try ping6 that address with
  %em2 appended (have to append the interface when using link-local addresses). I doubt it’ll work, since I know Juniper (which IIRC AT&T
  likes) doesn’t assign those (I can’t remember for sure about Cisco and don’t have a handy test target).

  And frankly, giving you a /56 is pretty crappy, since ARIN rules say to give every site a /48. I’d only do a /56 for a home connection prefix delegation. But, that’s AT&T! :)

• –I didn’t think to ask when we were turning up the v4 and phones. I’m betting there’s a setting in their gateway box but I’m waiting for them to give me the credentials to log into it.

  Good idea but alas it’s not routing. I can ping6 their gateway but it won’t route after I add it to the route table for the WAN interface. While pinging a remote server, in a second shell I can tcpdump icmp6 packets and I see the packets going out but no replies coming back.

  I’d just read about that when researching this. Maybe they decided that since we only have about a dozen people at our site, we won’t have a lot of subnets. What do small offices DO with 256 public subnets, anyway? I
  suppose eventually we’ll have an IoT subnet on every person.

• Once upon a time, Kenneth Porter said:

  The idea with IPv6 is not to even necessarily think about it in terms of direct numbers, but in layers. It is not uncommon to have several layers of routers, firewalls, guest wifi networks, etc., and each layer should request a prefix delegation from its parent. So rather than 256
  subnets, think about it as 8 layers (at most… but if a layer has more than 2 children, you have fewer layers available).

  So for example, if your Internet gateway has a desktop firewall, a guest wifi, a public DMZ, and a development lab gateway connected, and you want to allow for more things at that layer, there’s 3 of your 8 bits in a /56. If the dev lab needs to fan out more, and maybe your public DMZ
  needs to break up for production and QA-testing networks, and you add a VPN concentrator to the desktop network… you can go through those bits fast.

  In IPv4, people would just NAT the crap out of everything, having to tunnel from one NATted network to another, making life really difficult. The plan is no NAT in IPv6, so allow for all potential allocations up front.

  Also, allocations should be larger than necessary and sparse, so that you never need another allocation (even if you grow to 1000 employees and multiple buildings on a campus). This is to hopefully prevent routing tables from exploding like IPv4 did (and also to avoid you having to renumber everything just to stay in a single block).

• I discovered that IPv6 is sort of working when I got an email rejection from Comcast for not having an IPv6 PTR record. I discovered I could telnet to port 25 on their MX server over IPv6! I then found I could tracroute6 to them, but I couldn’t to my Linode VPS in Fremont. It gets to the data center and stops. Going the other way, my Linode can traceroute6 almost to my AT&T-hosted server. Neither can reach the open port 25 on the other, but both can reach mx1.comcast.net via IPv6.

  I dropped the firewall (“systemctl stop firewalld”, confirmed with
  “ip6tables -L -v -n”) on my Fremont server and still can’t traceroute6 or ping6 it. Nor can I do so from my home system on Comcast.

  What else could be blocking ping6, traceroute6, and telnet to port 25 on IPv6?

  [root@saruman ~]# traceroute6 2600:3c01::f03c:91ff:febc:e22d traceroute to 2600:3c01::f03c:91ff:febc:e22d
  (2600:3c01::f03c:91ff:febc:e22d), 30 hops max, 80 byte packets
  1 2001:1890:1837:5b00::1 (2001:1890:1837:5b00::1) 3.230 ms 3.181 ms
  3.137 ms
  2 2001:1890:c05:2f3a::ee6e:61cd (2001:1890:c05:2f3a::ee6e:61cd) 6.906 ms
  6.901 ms 6.870 ms
  3 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:123:6:98) 8.305 ms
  8.254 ms 8.258 ms
  4 sffca21crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:65) 14.199 ms
  14.164 ms 14.153 ms
  5 sffca401igs.ipv6.att.net (2001:1890:ff:ffff:12:122:114:5) 9.861 ms
  10.163 ms *
  6 2001:1890:1fff:41e:192:205:32:222 (2001:1890:1fff:41e:192:205:32:222)
  9.789 ms 8.078 ms 7.365 ms
  7 linode-ic-342731-sjo-b21.c.telia.net (2001:2000:3080:1ac4::2) 6.788 ms
  6.327 ms 6.262 ms
  8 2600:3c01:3333:4::2 (2600:3c01:3333:4::2) 7.294 ms 7.728 ms 7.705 ms
  9 * * *

  [root@linode2 ~]# traceroute6 2001:1890:1837:5b01::100
  traceroute to 2001:1890:1837:5b01::100 (2001:1890:1837:5b01::100), 30 hops max, 80 byte packets
  1 2600:3c01::8678:acff:fe0d:79c1 (2600:3c01::8678:acff:fe0d:79c1) 0.783
  ms 2600:3c01::8678:acff:fe0d:a641 (2600:3c01::8678:acff:fe0d:a641) 0.834
  ms 0.972 ms
  2 2600:3c01:3333:4::1 (2600:3c01:3333:4::1) 0.645 ms 0.646 ms
  2600:3c01:3333:2::1 (2600:3c01:3333:2::1) 0.508 ms
  3 ix-ae-67-0.tcore1.sqn-san-jose.ipv6.as6453.net (2001:5a0:1000:500::a1)
  1.059 ms 2001:550:2:3::d9:1 (2001:550:2:3::d9:1) 1.967 ms ix-ae-67-0.tcore1.sqn-san-jose.ipv6.as6453.net (2001:5a0:100
  4 be2063.ccr21.sjc01.atlas.cogentco.com (2001:550:0:1000::9a36:1a1)
  2.340 ms 2001:1890:1fff:415:192:205:37:69
  (2001:1890:1fff:415:192:205:37:69) 3.209 ms 4.412 ms
  5 sffca21crs.ipv6.att.net (2001:1890:ff:ffff:12:122:114:6) 7.353 ms
  6.769 ms 5.379 ms
  6 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:66) 4.785 ms
  6.158 ms 6.073 ms
  7 2001:1890:ff:ffff:12:122:119:125 (2001:1890:ff:ffff:12:122:119:125)
  3.401 ms 3.406 ms 3.469 ms
  8 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:66) 8.444 ms
  8.415 ms 2001:1890:c05:2f3a::116e:61cd (2001:1890:c05:2f3a::116e:61cd)
  5.816 ms
  9 2001:1890:ff:ffff:12:122:119:125 (2001:1890:ff:ffff:12:122:119:125)
  4.688 ms 4.607 ms 4.565 ms
  10 * * 2001:1890:c05:2f3a::116e:61cd (2001:1890:c05:2f3a::116e:61cd)
  6.732 ms
  11 * * *

• Once upon a time, Kenneth Porter said:

  Yeah, unfortunately things like that can happen, v4 or v6 (like I
  couldn’t get to a local TV station’s website a little while ago from my home connection, but could from elsewhere).

  From your traceroutes, it kind of looks like it’s possible that it’s something on your gateway (but I’m not really sure). Do you have any IPv6 firewall running there?

  One other note about mail on v6 – not only do you need to have a valid reverse (with matching forward) DNS record, you probably need to do TLS
  with a valid cert (Let’s Encrypt is free and easy).

• –I dropped the firewall just to make sure it wasn’t the issue. My connections to Linode eventually started working. Comparison to the traceroute I posted earlier shows a different router just before my server, so I suspect they had a bad router. I still can’t connect the other way, towards AT&T. But routing must be working because replies from outbound connections are getting back. I’d file a ticket with AT&T but they still haven’t added my account to their ticket system after more than a week of service. So I’m stuck playing tag with the sales consultants.

  Thanks for reminding me. I added an IPv6-only name for testing and for my reverse and need to add that to my certificate.