Hi Chris,
The Tier-1 has two /64 subnets, one for "normal hosts" and one for the OPN/DTZ.
The hosts are all on the same Ethernet segment, so to avoid having the routers be a bottleneck we have simply placed direct routes for both subnets on all hosts.
For example, if 2001:db8:ab:1050::0/64 was our "normal" subnet and 2001:db8:ab:1060::0/64 was our OPN/DTZ subnet, a host with a primary interface called eth0 would get:
ip route add 2001:db8:ab:1050::0/64 dev eth0
ip route add 2001:db8:ab:1060::0/64 dev eth0
It works well for us and essentially mirrors what we have been doing with IPv4 for years.
- James
On Fri, 2018-02-02 at 16:48 +0000, Chris Brew wrote:
> Hi Tim,
>
> We were concerned about that, which is part of the reason for asking the question.
>
> However, the hosts would be on a \64 subnet, it's just that there would be two routes in/out of the subnet but that would be hidden from either end. So would that still cause issues.
>
> We want the worker nodes and the storage on the same subnet so that the traffic between them doesn't have to go through a router, otherwise we would just set up a DTZ subnet and a worker subnet and
> have simple routing.
>
> Yours,
> Chris.
>
> On 02/02/2018, 16:43, "Testbed Support for GridPP member institutes on behalf of Tim Chown" <[log in to unmask] on behalf of [log in to unmask]> wrote:
>
> Hi Chris,
>
> > On 2 Feb 2018, at 12:41, Chris Brew <[log in to unmask]> wrote:
> >
> > Hi,
> >
> > We’re just finalising our plans for rolling out IPv6 with networking here and was wondering how other sites are handling data transfer zones/science DMZs/Firewall bypasses.
> >
> > At the moment for IPv4 we’ve got a single /22 subnet for our whole cluster with all hosts in that subnet having the same router. The router then does policy-based routing and sends traffic
> from nodes in the top /25 of the /22 out along the bypass route to the site border router, traffic from hosts in the rest of the subnet gets send to the site core which then routes it out through
> the firewall as usual. Then the site border router does the opposite, traffic to the top /25 gets send of down the bypass and the rest goes through the firewall to the core.
> >
> > For IPv6 we’re essentially planning the same thing (‘cept with rather larger numbers), we’ve got a /64 for our cluster and we’ll uses one of the higher bits (possibly the highest) of the host
> section to denote the bypass zone and then let the router and border router handle the routing.
> >
> > Is there any reason why that might be a bad idea?
>
> It's really a good idea to keep to the /64 boundary for host subnets, and not play with semantics in the IIDs; see RFC 7421 for the rationale behind that - https://tools.ietf.org/html/rfc7421.
>
> Is it not possible to get a /56?
>
> Best wishes,
> Tim
>
|