Load Balancing Across Links – EtherChannel, ECMP & Beyond

2. Per-Packet vs Per-Flow Load Balancing — The Core Choice

The fundamental design decision in any load balancing implementation is whether individual packets or entire flows are the unit of distribution.

3. EtherChannel and Link Aggregation Overview

EtherChannel (Cisco's implementation of link aggregation) bundles 2–8 physical Ethernet ports into a single logical Port-Channel interface. From the perspective of Spanning Tree Protocol, the higher-layer protocols, and the MAC address table, only one logical link exists — but all physical member links carry traffic simultaneously.

  Without EtherChannel:                With EtherChannel:
  Switch A ──── Switch B               Switch A ════ Switch B
           ──── (blocked by STP)                ════  (Port-Channel1)
  Only 1 Gbps active,                           ════  (all active)
  1 link wasted by STP                  3 Gbps bandwidth,
                                        3-link redundancy

EtherChannel Negotiation Protocols

4. EtherChannel Hashing — How Traffic Is Distributed

EtherChannel uses a hash algorithm to assign each frame to one of the member links. The switch computes a hash of selected header fields and uses the result to select the outgoing physical port. The same frame fields always produce the same hash result — guaranteeing that all packets in the same flow use the same physical link.

Cisco IOS Hash Method Options

Configuring and Verifying the Hash Method

! Configure hash method globally (affects all EtherChannels on switch)
Switch(config)# port-channel load-balance src-dst-ip

! Verify current method
Switch# show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-ip

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source XOR Destination IPv6 address

! Test which physical port a specific flow would use
Switch# test etherchannel load-balance interface port-channel 1 ip 192.168.1.10 10.0.0.5
Would select Gi0/1 of Po1

5. Hash Polarisation — The Hidden EtherChannel Problem

Hash polarisation is a serious but frequently overlooked problem that occurs in multi-tier switch networks where EtherChannel exists at multiple layers. When two EtherChannels in the same traffic path use the same hashing algorithm, all traffic from a given source may end up on the same physical links at every tier — instead of being spread across all available bandwidth.

  Without addressing polarisation:

  Access Switch → Distribution Switch → Core Switch
  (src-dst-ip)     (src-dst-ip)        (src-dst-ip)

  PC1 → Server traffic always hashes to same link at every tier.
  Result: 1 of 4 links carries 80% of traffic; others idle.

  Fixing polarisation — use different hash methods at each tier:

  Access Switch   → (src-mac)        — vary by source MAC
  Distribution    → (src-dst-ip)     — vary by IP pair
  Core Switch     → (src-dst-ip-l4port) — vary by port pair

  Result: Traffic spreads differently at each hop → balanced utilisation.

6. EtherChannel Configuration — Complete Example

! ── Switch A Configuration ─────────────────────────────────────────────
Switch-A(config)# interface range GigabitEthernet0/1 - 4

! Set identical parameters on all member ports BEFORE creating EtherChannel
Switch-A(config-if-range)# switchport mode trunk              ! Must match
Switch-A(config-if-range)# switchport trunk encapsulation dot1q
Switch-A(config-if-range)# channel-group 1 mode active        ! LACP active
Switch-A(config-if-range)# exit

! Configure the Port-Channel interface (logical interface)
Switch-A(config)# interface port-channel 1
Switch-A(config-if)# switchport mode trunk
Switch-A(config-if)# switchport trunk encapsulation dot1q

! Set the hash method
Switch-A(config)# port-channel load-balance src-dst-ip

! ── Switch B Configuration (must match Switch A) ─────────────────────
Switch-B(config)# interface range GigabitEthernet0/1 - 4
Switch-B(config-if-range)# switchport mode trunk
Switch-B(config-if-range)# switchport trunk encapsulation dot1q
Switch-B(config-if-range)# channel-group 1 mode active        ! LACP active

Switch-B(config)# interface port-channel 1
Switch-B(config-if)# switchport mode trunk
Switch-B(config-if)# switchport trunk encapsulation dot1q

Switch-B(config)# port-channel load-balance src-dst-ip

Verification Commands

! Summary of all EtherChannels — most useful first check
Switch# show etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator
        M - not in use, minimum links not met

Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi0/1(P) Gi0/2(P) Gi0/3(P) Gi0/4(P)

! (SU) = Layer 2, in use; (P) = bundled and active in port-channel

! Detailed port-channel status including LACP negotiation info
Switch# show etherchannel 1 detail

! Per-member traffic counters — useful for load distribution diagnosis
Switch# show interfaces port-channel 1 etherchannel

! Current hash method
Switch# show etherchannel load-balance

! EtherChannel port-level configuration
Switch# show interfaces Gi0/1 etherchannel

7. ECMP — Equal-Cost Multipath Routing

ECMP (Equal-Cost Multipath) operates at Layer 3: when a routing protocol discovers multiple paths to the same destination with identical metrics, it installs all of them in the routing table simultaneously. The router distributes outbound traffic across all equal-cost next-hops.

  ECMP Routing Table Example (OSPF):

  Router# show ip route 10.0.0.0/24

  O    10.0.0.0/24  [110/2] via 192.168.1.2, GigabitEthernet0/0
                    [110/2] via 192.168.2.2, GigabitEthernet0/1

  Both paths have identical metric [110/2]
  Traffic to 10.0.0.0/24 is distributed across Gi0/0 and Gi0/1
  Each flow consistently uses one path (per-flow hashing by default)

ECMP in Routing Protocols

ECMP Configuration Examples

! OSPF — increase max paths
Router(config)# router ospf 1
Router(config-router)# maximum-paths 8

! EIGRP — equal-cost max paths + unequal-cost via variance
Router(config)# router eigrp 100
Router(config-router)# maximum-paths 8
Router(config-router)# variance 2          ! Allow paths with FD ≤ 2 × best FD

! BGP — enable ECMP (disabled by default)
Router(config)# router bgp 65001
Router(config-router)# maximum-paths 4     ! eBGP ECMP
Router(config-router)# maximum-paths ibgp 4  ! iBGP ECMP

! Static ECMP — two equal-cost static routes
Router(config)# ip route 10.0.0.0 255.0.0.0 192.168.1.2
Router(config)# ip route 10.0.0.0 255.0.0.0 192.168.2.2

! Verify ECMP routes in routing table
Router# show ip route 10.0.0.0
! Look for multiple [metric] via entries for same prefix

8. ECMP Hashing at Layer 3

Like EtherChannel, ECMP routers use a hash algorithm to select which path a given flow uses. On Cisco IOS, ECMP hashing is performed by CEF (Cisco Express Forwarding), which maintains a load-sharing table derived from the hash of packet header fields.

! View CEF load sharing for ECMP paths
Router# show ip cef 10.0.0.0/24
10.0.0.0/24, epoch 0, 2 buckets/paths, flags 0x0
  next hop 192.168.1.2, GigabitEthernet0/0
  next hop 192.168.2.2, GigabitEthernet0/1

! View CEF load-sharing algorithm
Router# show ip cef exact-route 192.168.1.10 10.0.0.5
192.168.1.10 -> 10.0.0.5:
  GigabitEthernet0/0  (next hop: 192.168.1.2)

Router(config)# ip cef load-sharing algorithm universal
! or for specific interface:
Router(config-if)# ip load-sharing per-packet   ! enables per-packet (not recommended)

9. Limitations and Challenges

Hash Imbalance

Hash algorithms distribute traffic by computing a function of packet header fields. If the traffic profile is dominated by a small number of unique flow identifiers (e.g., a data centre where all traffic is between a few server IPs), many flows hash to the same value and end up on the same link — while other links carry almost nothing.

  Problem scenario — src-dst-ip hash with few unique IPs:

  All clients: 10.0.0.0/24 (same /24)
  All servers: 192.168.1.1 and 192.168.1.2

  Hash(10.0.0.x, 192.168.1.1) → always link 1
  Hash(10.0.0.x, 192.168.1.2) → always link 2
  Links 3 and 4: zero traffic!

  Fix: Add L4 port to hash (src-dst-ip-l4port)
  Hash(10.0.0.x, 192.168.1.1, src_port_54321, 443) → link 1
  Hash(10.0.0.x, 192.168.1.1, src_port_54322, 443) → link 3
  Better distribution across all 4 links

Asymmetric Routing

ECMP can cause asymmetric routing — where the forward path and reverse path for a connection travel through different routers. This is normally harmless for stateless packet forwarding, but causes problems with:

• Stateful firewalls: The firewall only sees one direction of the TCP session, cannot track state, and drops or resets the connection.
• NAT: Translation state exists on one device; reverse traffic hits a different device with no NAT table entry.
• Troubleshooting: Packet captures on one router miss half the conversation.

  Asymmetric routing with ECMP:

  Client → Router A → Server   (forward path uses R-A)
  Server → Router B → Client   (return path uses R-B)

  If R-A has a stateful firewall: only sees SYN, not SYN-ACK → drops connection
  Fix: Use symmetric hashing or stateful firewall clustering

Flow Stickiness / Elephant Flows

A single high-bandwidth "elephant flow" (e.g., a bulk backup or VM migration) hashes to one link and saturates it while other links remain underutilised. Per-flow hashing cannot split a single flow across multiple links — that would cause reordering. Solutions include:

• Use more granular hashing fields (add L4 ports) to separate sub-flows
• QoS to limit elephant flows and protect latency-sensitive traffic
• Some advanced data-centre fabrics use flowlet switching — detecting bursts within a flow and occasionally switching them to a different path when there is a gap in the flow

10. EtherChannel vs ECMP — Layer 2 vs Layer 3 Load Balancing

11. Troubleshooting Load Balancing

Key Command Reference

! ── EtherChannel Commands ──────────────────────────────────────────────
show etherchannel summary              ! Overview — all channels, protocol, port status
show etherchannel load-balance         ! Current hashing method
show etherchannel 1 detail             ! Detailed EtherChannel 1 status and port info
show interfaces port-channel 1         ! Port-Channel interface counters
show interfaces port-channel 1 etherchannel  ! Per-member traffic counters
test etherchannel load-balance interface port-channel 1 ip <src> <dst>

! ── ECMP / Routing Commands ─────────────────────────────────────────────
show ip route                          ! Full routing table — check for multiple paths
show ip route 10.0.0.0                 ! Specific prefix — shows all ECMP next-hops
show ip cef 10.0.0.0/24               ! CEF forwarding entry — load-sharing buckets
show ip cef exact-route <src> <dst>  ! Which path a specific flow uses

12. Common Misconceptions

• "EtherChannel doubles (or quadruples) bandwidth for all flows."
  EtherChannel increases aggregate bandwidth — multiple flows together can use the combined capacity. But any single individual flow is limited to one physical link's bandwidth. A 4-link 1G EtherChannel has 4 Gbps aggregate, but a single TCP session between two specific IPs still maxes out at 1 Gbps because per-flow hashing always puts it on the same link.
• "LACP timers must match on both sides."
  LACP has fast (1-second) and slow (30-second) PDU rates, but they do not need to match. The default is slow (30s) on both sides. What does matter is that at least one side must be in active mode — both sides in passive mode prevents the channel from forming.
• "Per-packet load balancing gives better throughput than per-flow."
  Theoretical aggregate throughput is similar for both methods. Per-packet causes TCP reordering which triggers duplicate ACKs and reduces actual goodput. Per-flow has better real-world application performance even though individual links may be less perfectly balanced.
• "Changing the EtherChannel hash method is traffic-affecting."
  Changing port-channel load-balance takes effect immediately and causes existing flows to be re-mapped to different physical links. This can cause a brief disruption to established TCP sessions. Schedule hash method changes during maintenance windows.

13. Key Points & Exam Tips

• EtherChannel bundles 2–8 physical ports into one logical Port-Channel. STP treats it as a single link — no blocking of member ports.
• EtherChannel protocols: LACP (IEEE 802.3ad, open standard — preferred), PAgP (Cisco only), Static/On (no negotiation).
• LACP modes: active (sends LACP PDUs) and passive (responds only). At least one side must be active — passive-passive does NOT form.
• PAgP modes: desirable (sends PAgP PDUs) and auto (responds only). Auto-Auto does NOT form.
• Hash method configured with: port-channel load-balance <method>. Verify with: show etherchannel load-balance.
• Best general-purpose hash: src-dst-ip or src-dst-ip-l4port — uses both endpoints for more unique values.
• Hash polarisation: Using the same hash method at multiple tiers can concentrate traffic on the same links. Fix by using different methods per tier.
• ECMP = Equal-Cost Multipath — multiple routing table entries for same prefix with identical metrics. All major routing protocols support it.
• OSPF ECMP: equal-cost only (maximum-paths). EIGRP: equal AND unequal-cost via variance. BGP: disabled by default, requires maximum-paths.
• Per-flow hashing (default) keeps packet order — required for TCP, VoIP. Per-packet causes reordering — avoid in production.
• Single large flows are limited to one link's bandwidth — EtherChannel cannot accelerate individual flows beyond one member link's speed.

Related pages: EtherChannel Configuration | OSPF Configuration | EIGRP Configuration | Spanning Tree Protocol | STP Overview | VLANs | show ip route | show interfaces | Trunk Port Configuration Lab

← Back to Home