DELL SONIC · CAPSTONE

Dell SONiC capstone: a whole fabric with public, internal and DMZ tenants

The same design as the ACI capstone, on Dell SONiC, so you can compare the two: an ACI fabric takes policy from one place, and here every line goes on a switch. PUBLIC reaches the internet router over BGP. INTERNAL reaches the campus router over OSPF, and its hosts sit behind a classic switch on a MCLAG pair. DMZ must go through ASA1 before the internet, so it has its own VRF, with a default route to the firewall, and the firewall’s outside is in a separate VRF. Only the outside routers and PHY1 are configured.

Dell SONiC track · Dell Enterprise SONiC-style, Cisco ASA-style, Cisco routers and switch · 8 objectives

Open in simulator More Dell SONiC labs

Objectives

The simulator ticks each one as soon as the network meets it.

Underlay: loopbacks and BGP on S1, S2, L1, L2, L3 and B1. The leaf pair shares AS 65001; L3 is AS 65002, and B1 is AS 65003.
MCLAG on L1+L2, with a port-channel 10 trunk carrying VLAN 20 to PHY1 on port 3 of each. PHY1 bundles both links.
Tenants in their own VRFs, with gateways at .254: Vrf-PUB (VLAN 10, L3 VNI 50011), Vrf-INT (VLAN 20, L3 VNI 50012) and Vrf-DMZ (VLAN 30, L3 VNI 50013), on the leaves with hosts, and all three VRFs on B1. PUB1 is on L3 port 3, and DMZ1 on L3 port 4.
Public: on B1 port 3, 10.9.1.1/30 in Vrf-PUB, with BGP in the VRF to 10.9.1.2 AS 65100. PUB1 reaches 203.0.113.1.
Internal: on B1 port 5, 10.9.3.1/30 in Vrf-INT, with OSPF area 0 in the VRF. INT1 reaches 172.16.1.1.
Firewall ASA1: inside 10.30.1.1/24 at level 100, outside 10.30.2.1/24 at level 0, routes to 192.168.30.0/24 via 10.30.1.254 and default via 10.30.2.254, NAT for 192.168.30.0/24 to its outside address, and inspect icmp in the global policy.
DMZ: on B1, port 6 is 10.30.1.254/24 in Vrf-DMZ, with a default route in the VRF to 10.30.1.1. Vrf-EDGE holds port 7 (10.30.2.254/24) and port 4 (10.9.2.1/30, BGP to 10.9.2.2). DMZ1 reaches 203.0.113.1, through ASA1.
The tenants stay apart: PUB1 cannot reach DMZ1 (192.168.30.30) or INT1 (192.168.20.20).

Start this lab

Before you start

Each VRF is its own routing table, and Vrf-DMZ’s only way out is the default route to the firewall, so nothing can skip it. Check layer by layer: BGP summary, show mclag brief, show vxlan remotevtep, show ip route vrf, then show bgp vrf and show ip ospf vrf for the outside sessions. Without inspect icmp, the firewall drops the echo replies, so the ping fails: try it before adding the line. The BGP and OSPF lines in a VRF are modeled from Dell documentation, so check them on your release.

Concept

A leaf-spine fabric on Dell switches

Every leaf connects to every spine, and no leaf connects to another leaf, so any two leaves are two hops apart over several equal paths. eBGP routes between loopbacks form the underlay, usually with one ASN per leaf pair. EVPN then runs on the same BGP sessions: VTEPs announce their VNIs and hosts, and tunnels form from those routes. SONiC can peer without link addresses: ipv6 enable and neighbor interface are enough.

Concept

VLT and MCLAG: two switches, one LACP partner

A leaf pair joined by a peer link looks like one switch to the device below it. That device bundles one link to each leaf with ordinary LACP, so spanning tree blocks nothing and both links forward. Dell OS10 calls this VLT, and SONiC calls it MCLAG. In a VXLAN fabric, the pair also shares one VTEP address, so the rest of the fabric sees a single tunnel endpoint, and either leaf can take the tunnel.

Concept

Tenant exits: BGP, OSPF and a firewall

A fabric VRF is one routing table, so each tenant can leave in its own way. A public tenant usually runs BGP with its provider, and an internal tenant runs OSPF with the campus. A DMZ must cross a firewall first, so its VRF gets only one route out, and that route points at the firewall. The firewall’s outside sits in a separate VRF.