MPLS Edge network configuration

Introduction
It has been a busy start to 2017 at work and I had some distractions as well on the family front.

But finally – here is part 2 of the mpls configuration article.

To recap, in the previous article we completed the basic mpls core configuration of enabling an igp in the core, enabling mpls interfaces on P1 and PE routers and completing the bgp peering between PE routers that will be sharing vpnv4 routes.

This article’s focus is on getting the customer edge (CE) routers to make use of the mpls backbone to talk other CE routers at remote sites.

Network topology
The configuration for the mpls edge fulfills the following requirements to establish a L3 vpn and route sharing over the mpls core.

Referring to the network diagram for a simple mpls set up.

Customer A and Customer B must have routing separation between each other.

Although in this example , I am using the term customer — this description could also be applied to different groups or business units within an internal mpls network for an organisation.

Both customers have their own independent addressing space and run their own routing protocols. There is no issue with overlapping addressing with CustA and CustB.

However there is also a SharedServices domain that both customers require access to. For example, in most corporate environment typically services such as user authentication, control of Internet access, Intranet web servers, printing, mail etc could be shared amongst different groups.

The Shared Services network must have unique addressing that does not overlap with the other networks.

MPLS recap
MPLS came out of Cisco tag switching and has since been rectified as an IETF standard RFC4364.

MPLS is an encapsulation technology where extra headers are introduced to the frame as depicted in the figure below.

The idea for mpls is to rely on switching labels rather than reference L3 route tables which take longer to process and consume memory as more route tables are added.

MPLS label switching depends on having:
— Label switch routers (aka P routers).

The function of a P router is to run label distribution protocol and efficiently process these labels as frames are received.
— Label edge routers (aka PE routers).

PE routers are the entry and exit point to an mpls network.

The function of a PE router is to
– connect to the customer (CE) router and exchange routes with the customer’s network
– manage label distribution with P router, insert labels for delivery over mpls core and to strip labels when returning traffic to the CE router

We assume the P and PE routers have been set up for ldp and has an igp running to provide communications between P-toP and P-to-PE routers.

Checking the mpls core
Although we are not repeating the mpls core configuration , you can run some show commands to verify your mpls core.

Here’re representative output for a P and a PE router.

P1#sh mpls ldp neigh

Peer LDP Ident: 2.2.2.2:0; Local LDP Ident 1.1.1.1:0
TCP connection: 2.2.2.2.29292 – 1.1.1.1.646
State: Oper; Msgs sent/rcvd: 41/42; Downstream
Up time: 00:24:35
LDP discovery sources:
FastEthernet0/0, Src IP addr: 11.0.12.2
Addresses bound to peer LDP Ident:
11.0.12.2 11.0.32.2 10.0.25.1 2.2.2.2
Peer LDP Ident: 4.4.4.4:0; Local LDP Ident 1.1.1.1:0
TCP connection: 4.4.4.4.27115 – 1.1.1.1.646
State: Oper; Msgs sent/rcvd: 14/14; Downstream
Up time: 00:00:30
LDP discovery sources:
FastEthernet1/0, Src IP addr: 10.0.14.2
Addresses bound to peer LDP Ident:
4.4.4.4 10.0.14.2

PE4#sh mpls ldp neigh

Peer LDP Ident: 1.1.1.1:0; Local LDP Ident 4.4.4.4:0
TCP connection: 1.1.1.1.646 – 4.4.4.4.27115
State: Oper; Msgs sent/rcvd: 15/15; Downstream
Up time: 00:01:32
LDP discovery sources:
FastEthernet1/0, Src IP addr: 10.0.14.1
Addresses bound to peer LDP Ident:
11.0.12.1 10.0.14.1 1.1.1.1

High level breakdown of how to configure an edge mpls
Configuring an mpls edge consists of setting up the vrfs, configuring mp-bgp and routing between PE and CE routers. A methododical approach will help to avoid mistakes and save effort in unnecessary troubleshooting.

Step 1: Configure vrfs
– 1a. On PEs set up the required vrfs and use Route Distinguishers to identify them uniquely.

– 1b. On PEs set up import and export Route Targets to control which vpnv4 routes received by the PEs can be passed into the vrfs.

Step 2: Configure mp-bgp between PE routers
– 2a. Set up ibgp between PE routers.

– 2b. Set up mp-bgp with address-family vpnv4 to allow vpnv4 route exchanges between the PEs.

Step 3: Routing between PE and CE routers
On the PE (within the appropriate vrf) distribute the routes learned from the CE router to the mpls (mp-bgp) vrf and vice versa

Understanding vrfs and Route Distinguisher (RD)
Before we start on our configuration, we must first mention the use of route distinguisher (RD) for a L3 vpn over an mpls network.

RFC4364 dictates the use of mp-bgp to support a “vpn-ipv4 address family”.

This vpn-ipv4 address is to allow bgp to distinguish between the same ipv4 address that can overlap for different vpns.

The notion of the vpn-ipv4 address is to add a unique RD in front of the ip address so that a 10.1.1.1 address on say vrf CustA and vrf CustB becomes:

RD#1:10.1.1.1 and RD#2:10.1.1.1 and are thus the vpnv4 routes are uniquely identified.

Since the ipv4 address and the vpn-ipv4 addresses are different , mp-bgp will treat each address family separately just as it treats ipv4 and ipv6 address-family as non-comparable and distinctly.

And RD is just a # but convention is to write it as AS:nn or nn:ipv4-address where
AS = bgp AS # and nn is any unique #.

Step 1a: Configuring vrfs and RDs
For each customer instance, create a vrf on the PE router. For our network therefore we will have:

On PE4 – vrf CustA, vrf CustB
On PE5 – vrf CustA, vrf CustB and vrf SharedServices

But before we go ahead to set up the vrfs we need to cover the requirement for an route distinguisher.

So now back to configuring the vrf but taking into account that we add the RD to the vrf.

On PE4:
ip vrf CustA
rd 100:41
!
ip vrf CustB
rd 100:42
By convention use the same RDs on PE5.
ip vrf CustA
rd 100:41
!
ip vrf CustB
rd 100:42
! ip vrf SharedServices
rd 100:43

Understanding Route targets (RT)
When a PE has a route to send it must be able to decide whether the route should be send to the remote peer.

Likewise when a remote PE receives a route it must determine which vrf to install the received route in.

RFC4364 defines a bgp community attribute which tags each vpn-ipv4 route with a route target (RT) to manage the export and import of a vpnv4 route from or into a vrf.

Every route sent to another PE is ‘exported’ with a RT and every route received from a remote PE via mp-bgp can only be installed into a vrf if the RT for that route matches the route-target import values in the vrf configuration.

A RT is written in the same format as the RD.

So to summarise a vrf configuration complete when we have:
– a unique RD
– a unique RT for ‘export’ (routes to send from the vrf)
– multiple RTs statements to match the RT for ‘import’ (routes to receive into the vrf).

Step 1b: Configuring RTs
Based on our understanding of RTs we can now complete the RT configuration.
The RT value does not have to match the RD value, although many engineers make the RD and RT values the same for ease of troubleshooting.

We will assign the following route targets to the vrfs:

vrf CustA – both route-target 1:1 for export and import
vrf CustB – both route-target 2:2 for export and import
vrf SharedServices – both route-target 3:3 for export and import

To maintain isolation between vrf CustA and CustB and to prevent each others routes from being accepted, the RT export and import statements are straightforward.

On PE4 and PE5:

ip vrf CustA
route-target export 1:1
route-target import 1:1
ip vrf CustB
route-target export 2:2
route-target import 2:2

Additionally export SharedServices routes on PE5 and import routes from CustA and CustB.

ip vrf SharedServices
route-target export 3:3
route-target import 1:1
route-target import 2:2

We also want CustA and CustB to learn SharedServices routes, therefore we import route-target 3:3 into their vrfs.

On PE4 and PE5:
ip vrf CustA
route-target import 3:3
ip vrf CustB
route-target import 3:3

Step 2: Configuring mp-bgp on PEs
Configuring an mp-bgp on the PEs is best completed in a methodical manner.

2a. enable ibgp on PE routers by including all PE routers (which need to exchange routes with each other) in the same bgp AS

2b. enable multi-protocol bgp address family for vpnv4

Step 2a Enable ibgp peering on PE routers
To complete the mpls core setup we will run bgp on all PE routers so that they can share routing information.
Again nothing fancy, just implement the standard bgp set up and ensure there is a full (bgp) mesh setup for all PE routers.

For larger networks it is more practical to use a route reflector (RR). For now I will not go into RR setup in this article.

To make it efficient to read bgp output, it makes sense to define your bgp router id.
I have used the loopback address to create the router-id. The router-id chosen for our examples will be PEx will have a router-id x.x.x.x

For example for PE6

PE6#sh run | s bgp
router bgp 100
bgp router-id 6.6.6.6
bgp log-neighbor-changes
neighbor 4.4.4.4 remote-as 100
neighbor 4.4.4.4 update-source Loopback0
neighbor 5.5.5.5 remote-as 100
neighbor 5.5.5.5 update-source Loopback0

Here are some output to verify bgp is working properly.

PE5(config-router)#do sh ip bgp summ

BGP router identifier 5.5.5.5, local AS number 100
BGP table version is 1, main routing table version 1
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
4.4.4.4 4 100 19 19 1 0 0 00:14:16 0
6.6.6.6 4 100 17 15 1 0 0 00:12:21 0

PE6#sh ip bgp summ

BGP router identifier 6.6.6.6, local AS number 100
BGP table version is 1, main routing table version 1
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
4.4.4.4 4 100 14 13 1 0 0 00:10:59 0
5.5.5.5 4 100 14 16 1 0 0 00:11:22 0

Step 2b: Enable mp-bgp with address-family for vpnv4
To distribute the RDs and RTs under bgp between the PEs we configure the address-family vpnv4. The vpnv4 address-family configuration sets up the exchange of extended communities for vpnv4 prefixes and route-targets.

On PE4:

router bgp 100
address-family vpnv4
neighbor 5.5.5.5 activate
neighbor 5.5.5.5 send-community extended — this line will be automatically added
exit-address-family

On PE5:

router bgp 100
address-family vpnv4
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended — this line will be automatically added
exit-address-family

Step 3: Enable routing between PE (vrf) and CE
Routing for each Cust instance can be different. Therefore the PE configuration is dependent and must fit with the routing protocol running on the customer’s network.

In our example PE4 and PE5 participates in route exchange with each CE router within each vrf.

mp-bgp uses the address-family ipv4 vrf configuration to isolate the routing in separate ip vpnv4 vrf tables.

The steps to setup for each vrf is as follows:

Step 3a. for vrf CustA we have to set the PEs for ospf and redistribution between ospf & bgp
Step 3b. for vrf CustB we set up the PEs for bgp
Step 3c. for Shared Services we set up the PEs for static routing

Step 3a. Enable ospf on address-family ipv4 for CustA vrf
To set up ospf on the PE routers for CustA we will use opsf area 0 and ospf 111.
I will assume CE21_CustA_Site1 and CE23_CustA_Site2 has the right ospf configuration.

On the PEs enable ospf 111 on vrf CustA, set up vrf address-family ipv4 4 for CustA and mutual route redistribution between ospf and bgp.

On PE4:
router ospf 111 vrf CustA
router-id 41.41.41.41
redistribute bgp 100 subnets
network 10.4.1.0 0.0.0.3 area 0

router bgp 100
address-family ipv4 vrf CustA
redistribute ospf 111
exit-address-family

On PE5:
router ospf 111 vrf CustA
router-id 51.51.51.51
redistribute bgp 100 subnets
network 10.5.1.0 0.0.0.3 area 0

router bgp 100
address-family ipv4 vrf CustA
redistribute ospf 111
exit-address-family

We can now check that we can talk to the remote site over the mpls network.

CE21_CustA_Site1#ping 23.23.23.23 source 21.21.21.21

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 23.23.23.23, timeout is 2 seconds:
Packet sent with a source address of 21.21.21.21
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 112/484/1604 ms

Step 3b. Enable bgp for address-family ipv4 for CustB vrf
CustB is running bgp on their internal network on AS2224.

Again I will assume the CE22_CustB_Site1 and CE24_CustB_Site2 has the right bgp configuration.
The PEs are configured to talk bgp to the CEs under the address-family ipv4 vrf command.

On PE4:
rouetr bgp 100
address-family ipv4 vrf CustB
neighbor 10.4.1.2 remote-as 2224
neighbor 10.4.1.2 activate
neighbor 10.4.1.2 as-override
exit-address-family

On PE5:
royter bgp 100
address-family ipv4 vrf CustB
neighbor 10.5.1.2 remote-as 2224
neighbor 10.5.1.2 activate
neighbor 10.5.1.2 as-override
exit-address-family

At both sites, Customer B’s BGP AS # is 2224 we should add an as-override statement to prevent received routes from being dropped by the remote CE.

We can now check that we can talk to the remote site over the mpls network.

CE22_CustB_Site1#ping 24.24.24.24 source 22.22.22.22

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 24.24.24.24, timeout is 2 seconds:
Packet sent with a source address of 22.22.22.22
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 96/286/872 ms

Step 3c. Enable static routing on address-family ipv4 for Shared Services vrf
To enable CustA and CustB to reach Shared Services we redistribute the static routes into bgp under the address-family configuration for SharedServices.

First let’s set up the static route to reach our ShardServices network (in our example I am using the loopback address on router CE_SharedServices_Site2 i.e. 25.25.25.25/32.

On PE5:
ip route vrf SharedServices 25.25.25.25 255.255.255.255 10.1.1.2

Since the SharedServices vrf is only found on PE5 that is where we apply the address-family configuration. Again on PE5,

router bgp 100
address-family ipv4 vrf SharedServices
redistribute static
exit-address-family

Finally add static routes on CE_SharedServices_Site2 to reach any ip address on CustA or CustB as desired and we are done !

We can now check that we can talk to the SharedServices ip address over the mpls network.

CE21_CustA_Site1#ping 25.25.25.25 source 21.21.21.21

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 25.25.25.25, timeout is 2 seconds:
Packet sent with a source address of 21.21.21.21
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 108/162/276 ms

CE22_CustB_Site1#ping 25.25.25.25 source 22.22.22.22

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 25.25.25.25, timeout is 2 seconds:
Packet sent with a source address of 22.22.22.22
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 112/120/128 ms