Saturday, November 07, 2009

Mapping IXP access links

IXPs (Internet Exchange Points) are facilities where ISPs (Internet Service Providers) meet and can set up interconnection (peering) and exchange traffic between their mutual customers directly.

There was an interesting paper at IMC'09 called "IXPs: Mapped?", which fused data from various sources to get a more complete picture of peering at IXPs.

Here is one phenomenon that I think would be worth further research:

In the original model, the IXPs is at a colocation facility, where ISPs put their equipment (routers) and connect a router port to the exchange point switch via in-house cabling. But there's another option, where the ISP doesn't put a router up at the physical IXP facility, but instead connects an existing backbone router to the IXP by means of a leased line - could be a leased wavelength, or an Ethernet-over-{MPLS,SDH} service. Looking around me, I see this happening very frequently, e.g. quite a few medium-size Swiss ISPs lease bandwidth to Amsterdam, Frankfurt or London to connect to the large IXPs there (AMS-IX, DECIX, and LINX, respectively).

It is important to understand such non-local peering, because the impact of the access links can be quite large in terms of speed-of-light delay, and some transport media - e.g. Ethernet over MPLS - might be subject to unnoticed congestion. Also, long access links might be more difficult to upgrade in a timely manner when needed.

As an illustration, here's a traceroute between the ISP I work for (SWITCH, AS559) and an ISP in Gibraltar.

$ traceroute
traceroute to (, 64 hops max, 52 byte packets
1  swiws1-v4 (  2.652 ms  0.271 ms  0.259 ms
2  swics5-10ge-1-4 (  0.334 ms  0.308 ms  0.317 ms
3  swics3-10ge-1-3 (  0.412 ms  0.336 ms  0.340 ms
4  swiez2-10ge-5-2 (  0.358 ms  0.347 ms  0.356 ms
5  swils2-10ge-1-1 (  4.001 ms  3.916 ms  3.983 ms
6  swiel2-10ge-1-2 (  4.081 ms  4.036 ms  4.069 ms
7  swice3-10ge-1-3 (  4.826 ms  4.843 ms  4.879 ms
8 (  58.489 ms  58.552 ms  58.530 ms
9 (  75.874 ms (  78.614 ms  78.640 ms
10 (  76.205 ms  76.101 ms  76.203 ms
11  *  C-c C-c

Can you guess in which city we interconnect?