Configuring static lag on Brocade ICX switches to be used with Check Point bond

As we’re slowly (but surely) moving towards replacing our Cisco gear with Brocade I’m going to publish a set of articles related to ICX 6610 configuration. Bear with me since I’m still learning it. By the way, if you spot any mistakes please let me know.

In comparison with the Cisco 3745 the Brocade’s ICX 6610 wins by miles, both in terms of performance and price (big time!), so here we go.

The first article will be related to the configuration of LAG (Link Aggregation Group) on ICX 6610 to be used with Check Point’s bond interfaces. The idea is to aggregate two or more physical links into a virtual one, so in case there is an issue with one of the links (faulty cable or NIC) the connection is still operational. To summarize, on Brocade you configure LAGs and on Check Point you configure bonds. Initially, when I first started working on it, my main goal was all about redundancy and I didn’t really care about load distribution. At the end it came out that the traditional active/backup setup cannot be implemented with the bond where both legs are terminated on the same switch, so I ended up with the active/active implementation. It’s a mix of load balancing + redundancy so should be fine.

Here is my setup:

– two Brocade ICX 6610 switches running basic L2 software (version 08.0.10aT7f1). Each switch is 48 10/100/1000MB ports each + 8 dual mode (SFP/SFP+) ports + 10-port 160G module (used for stacking). Both units are in the stack. Mind you the software version: there is quite a difference in the configuration syntax between 07 and 08, especially with regards to link aggregation.

– Check Point R77.10 on Gaia installed on the IBM System x3655 server (with two quad-port 100/1000MB PCI-X cards).

I’m going to deploy three LAGs (two ports each) on the Brocade stack and three bonds (two ports each again) on the Check Point server. On the Brocade stack: one port of the LAG will be from Unit 1 and another port from Unit 2. Same for the Check Point server: slave interfaces of each bond are on different PCI-X cards. Nothing fancy in terms of VLANs — all ports are untagged. For the readers with Cisco background: untagged port is an equivalent of access port in Cisco while tagged port is a trunk (where you inject multiple VLANs).

Here is the configuration on the Brocade stack:

  1. !
  2. lag "INTERNET" static id 1
  3.  ports ethernet 1/1/1 ethernet 2/1/1
  4.  primary-port 1/1/1
  5.  deploy
  6. !
  7. lag "DMZ" static id 2
  8.  ports ethernet 1/1/2 ethernet 2/1/2
  9.  primary-port 1/1/2
  10.  deploy
  11. !
  12. lag "LAN" static id 3
  13.  ports ethernet 1/1/3 ethernet 2/1/3
  14.  primary-port 1/1/3
  15.  deploy
  16. !
  17. vlan 11 name INTERNET by port
  18.  untagged ethe 1/1/1 ethe 2/1/1
  19. !
  20. vlan 12 name DMZ by port
  21.  untagged ethe 1/1/2 ethe 2/1/2
  22. !
  23. vlan 13 name LAN by port
  24.  untagged ethe 1/1/3 ethe 2/1/3
  25. !

On the Check Point side, add Bond interfaces via Web GUI as per screen shots below:

2014041401

2014041402

Basically you create a new Bond interface, put a meaningful comment, set the Bond Group (should be unique number for each bond), choose slave interfaces and check Round Robin operation mode. Then you assign an IP address for the created bond (note that slave interfaces cannot have IP addressed configured, so if they do you have to clear it and assign it to the bond instead).

As for the Operation Mode this is where I got stuck for the first time and spent quite some time trying to troubleshoot. Initially I’ve chosen Active-Backup mode and I was not able to get it working with two cables attached to the lag at the same time. Apparently it came out that it cannot be achieved because I terminate the lag on the same switch and Active-Backup mode is used only when you have two physical switches (I guess it had something to do with the MAC address assigned to the bond interface). Anyway, I ended up with Round Robin mode — I didn’t really need load sharing but as long as it’s redundant it works for me.

Below is a list of commands to troubleshoot the lag on the Brocade switch:

  1. #show lag "DMZ"
  2. Total number of LAGs:          3
  3. Total number of deployed LAGs: 3
  4. Total number of trunks created:3 (117 available)
  5. LACP System Priority / ID:     1 / cc4e.2416.f7v8
  6. LACP Long timeout:             90, default: 90
  7. LACP Short timeout:            3, default: 3
  8.  
  9. === LAG "DMZ" ID 2 (static Deployed) ===
  10. LAG Configuration:
  11.    Ports:         e 1/1/2 e 2/1/2
  12.    Port Count:    2
  13.    Primary Port:  1/1/2
  14.    Trunk Type:    hash-based
  15. Deployment: HW Trunk ID 2
  16. Port    Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
  17. 1/1/2   Up      Forward Full 1G    2     No  2    0   cc4e.2416.f7v8
  18. 2/1/2   Up      Forward Full 1G    2     No  2    0   cc4e.2416.f7v8

The important part here is the State column — Forward means the port will pass the traffic.

You can also check the status of the interface itself (either primary or secondary one):

  1. #show interface ethernet 1/1/2
  2. GigabitEthernet1/1/2 is up, line protocol is up
  3.   …
  4.   Member of L2 VLAN ID 12, port is untagged, port state is FORWARDING
  5.   …
  6.   Member of active trunk ports 1/1/2,2/1/2, primary port is 1/1/2
  7.   Member of configured trunk ports 1/1/2,2/1/2, primary port is 1/1/2

Finally, you can check the output of ‘show logging’:

  1. #show logging
  2. 01 days 16h47m23s:I:STP: VLAN 12 Port 1/1/2 STP State –> FORWARDING (FwdDlyExpiry)
  3. 01 days 16h47m21s:I:STP: VLAN 12 Port 1/1/2 STP State –> LEARNING (FwdDlyExpiry)
  4. 01 days 16h47m19s:I:System: Interface ethernet 1/1/2, state up

There is a separate article explaining how to troubleshoot the bond interface on Check Point. For Gaia I used (in Bash):

  1. [Expert@CHKP01]# cat /proc/net/bonding/bondX

where bondX is the name of the relevant bond interface.

[20160110] : To make things a bit more intelligent you can create a dynamic lag instead of static (so you also don’t have to care about LAG ID) and change the Check Point bond operation mode to 802.3ad:

  1. !
  2. lag "INTERNET" dynamic id 7
  3.  ports ethernet 1/1/1 ethernet 2/1/1
  4.  primary-port 1/1/1
  5.  deploy
  6. !

2016011001

Tags: , , , ,

Leave a Reply