Compressing IP forwarding tables

Realizing information-theoretical space bounds and fast lookups simultaneously

Attila Korosi, J. Tapolcai, Bence Mihalka, Gabor Meszaros, Gabor Retvari

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

The Internet routing ecosystem is facing compelling scalability challenges, manifested primarily in the rapid growth of IP packet forwarding tables. The forwarding table, implemented at the data plane fast path of Internet routers to drive the packet forwarding process, currently contains about half a million entries and counting. Meanwhile, it needs to support millions of complex queries and updates per second. In this paper, we make the curious observation that the entropy of IP forwarding tables is very small and, what is more, seems to increase at a lower pace than the size of the network. This suggests that a sophisticated compression scheme may effectively and persistently reduce the memory footprint of IP forwarding tables, shielding operators from scalability matters at least temporarily. Our main contribution is such a compression scheme which, for the first time, admits both the required information-theoretical size bounds and attains fast lookups, thanks to aggressive level compression. Although we find the underlying optimization problem NP-complete, we can still give a lightweight heuristic algorithm with firm approximation guarantees. This allows us to squeeze real IP forwarding tables, comprising almost 500, 000 prefixes, to just about 140 - 200 KBytes of memory within a factor of 2 - 3 of the entropy bound, so that forwarding decisions take only 8 - 10 memory accesses on average and updates are supported efficiently. Our compression scheme may be of more general interest, as it is applicable to essentially any prefix tree.

Original languageEnglish
Title of host publicationProceedings - IEEE 22nd International
PublisherIEEE Computer Society
Pages332-343
Number of pages12
ISBN (Print)9781479962044
DOIs
Publication statusPublished - Dec 9 2014
Event22nd IEEE International Conference on Network Protocols, ICNP 2014 - Research Triangle, United States
Duration: Oct 21 2014Oct 24 2014

Other

Other22nd IEEE International Conference on Network Protocols, ICNP 2014
CountryUnited States
CityResearch Triangle
Period10/21/1410/24/14

Fingerprint

Data storage equipment
Scalability
Entropy
Internet
Heuristic algorithms
Routers
Shielding
Ecosystems
Computational complexity

Keywords

  • data compression
  • IP forwarding
  • prefix trees

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Korosi, A., Tapolcai, J., Mihalka, B., Meszaros, G., & Retvari, G. (2014). Compressing IP forwarding tables: Realizing information-theoretical space bounds and fast lookups simultaneously. In Proceedings - IEEE 22nd International (pp. 332-343). [6980394] IEEE Computer Society. https://doi.org/10.1109/ICNP.2014.55

Compressing IP forwarding tables : Realizing information-theoretical space bounds and fast lookups simultaneously. / Korosi, Attila; Tapolcai, J.; Mihalka, Bence; Meszaros, Gabor; Retvari, Gabor.

Proceedings - IEEE 22nd International. IEEE Computer Society, 2014. p. 332-343 6980394.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Korosi, A, Tapolcai, J, Mihalka, B, Meszaros, G & Retvari, G 2014, Compressing IP forwarding tables: Realizing information-theoretical space bounds and fast lookups simultaneously. in Proceedings - IEEE 22nd International., 6980394, IEEE Computer Society, pp. 332-343, 22nd IEEE International Conference on Network Protocols, ICNP 2014, Research Triangle, United States, 10/21/14. https://doi.org/10.1109/ICNP.2014.55
Korosi A, Tapolcai J, Mihalka B, Meszaros G, Retvari G. Compressing IP forwarding tables: Realizing information-theoretical space bounds and fast lookups simultaneously. In Proceedings - IEEE 22nd International. IEEE Computer Society. 2014. p. 332-343. 6980394 https://doi.org/10.1109/ICNP.2014.55
Korosi, Attila ; Tapolcai, J. ; Mihalka, Bence ; Meszaros, Gabor ; Retvari, Gabor. / Compressing IP forwarding tables : Realizing information-theoretical space bounds and fast lookups simultaneously. Proceedings - IEEE 22nd International. IEEE Computer Society, 2014. pp. 332-343
@inproceedings{16080ea8e1c94249b7a0c7d279b0eced,
title = "Compressing IP forwarding tables: Realizing information-theoretical space bounds and fast lookups simultaneously",
abstract = "The Internet routing ecosystem is facing compelling scalability challenges, manifested primarily in the rapid growth of IP packet forwarding tables. The forwarding table, implemented at the data plane fast path of Internet routers to drive the packet forwarding process, currently contains about half a million entries and counting. Meanwhile, it needs to support millions of complex queries and updates per second. In this paper, we make the curious observation that the entropy of IP forwarding tables is very small and, what is more, seems to increase at a lower pace than the size of the network. This suggests that a sophisticated compression scheme may effectively and persistently reduce the memory footprint of IP forwarding tables, shielding operators from scalability matters at least temporarily. Our main contribution is such a compression scheme which, for the first time, admits both the required information-theoretical size bounds and attains fast lookups, thanks to aggressive level compression. Although we find the underlying optimization problem NP-complete, we can still give a lightweight heuristic algorithm with firm approximation guarantees. This allows us to squeeze real IP forwarding tables, comprising almost 500, 000 prefixes, to just about 140 - 200 KBytes of memory within a factor of 2 - 3 of the entropy bound, so that forwarding decisions take only 8 - 10 memory accesses on average and updates are supported efficiently. Our compression scheme may be of more general interest, as it is applicable to essentially any prefix tree.",
keywords = "data compression, IP forwarding, prefix trees",
author = "Attila Korosi and J. Tapolcai and Bence Mihalka and Gabor Meszaros and Gabor Retvari",
year = "2014",
month = "12",
day = "9",
doi = "10.1109/ICNP.2014.55",
language = "English",
isbn = "9781479962044",
pages = "332--343",
booktitle = "Proceedings - IEEE 22nd International",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Compressing IP forwarding tables

T2 - Realizing information-theoretical space bounds and fast lookups simultaneously

AU - Korosi, Attila

AU - Tapolcai, J.

AU - Mihalka, Bence

AU - Meszaros, Gabor

AU - Retvari, Gabor

PY - 2014/12/9

Y1 - 2014/12/9

N2 - The Internet routing ecosystem is facing compelling scalability challenges, manifested primarily in the rapid growth of IP packet forwarding tables. The forwarding table, implemented at the data plane fast path of Internet routers to drive the packet forwarding process, currently contains about half a million entries and counting. Meanwhile, it needs to support millions of complex queries and updates per second. In this paper, we make the curious observation that the entropy of IP forwarding tables is very small and, what is more, seems to increase at a lower pace than the size of the network. This suggests that a sophisticated compression scheme may effectively and persistently reduce the memory footprint of IP forwarding tables, shielding operators from scalability matters at least temporarily. Our main contribution is such a compression scheme which, for the first time, admits both the required information-theoretical size bounds and attains fast lookups, thanks to aggressive level compression. Although we find the underlying optimization problem NP-complete, we can still give a lightweight heuristic algorithm with firm approximation guarantees. This allows us to squeeze real IP forwarding tables, comprising almost 500, 000 prefixes, to just about 140 - 200 KBytes of memory within a factor of 2 - 3 of the entropy bound, so that forwarding decisions take only 8 - 10 memory accesses on average and updates are supported efficiently. Our compression scheme may be of more general interest, as it is applicable to essentially any prefix tree.

AB - The Internet routing ecosystem is facing compelling scalability challenges, manifested primarily in the rapid growth of IP packet forwarding tables. The forwarding table, implemented at the data plane fast path of Internet routers to drive the packet forwarding process, currently contains about half a million entries and counting. Meanwhile, it needs to support millions of complex queries and updates per second. In this paper, we make the curious observation that the entropy of IP forwarding tables is very small and, what is more, seems to increase at a lower pace than the size of the network. This suggests that a sophisticated compression scheme may effectively and persistently reduce the memory footprint of IP forwarding tables, shielding operators from scalability matters at least temporarily. Our main contribution is such a compression scheme which, for the first time, admits both the required information-theoretical size bounds and attains fast lookups, thanks to aggressive level compression. Although we find the underlying optimization problem NP-complete, we can still give a lightweight heuristic algorithm with firm approximation guarantees. This allows us to squeeze real IP forwarding tables, comprising almost 500, 000 prefixes, to just about 140 - 200 KBytes of memory within a factor of 2 - 3 of the entropy bound, so that forwarding decisions take only 8 - 10 memory accesses on average and updates are supported efficiently. Our compression scheme may be of more general interest, as it is applicable to essentially any prefix tree.

KW - data compression

KW - IP forwarding

KW - prefix trees

UR - http://www.scopus.com/inward/record.url?scp=84919968570&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919968570&partnerID=8YFLogxK

U2 - 10.1109/ICNP.2014.55

DO - 10.1109/ICNP.2014.55

M3 - Conference contribution

SN - 9781479962044

SP - 332

EP - 343

BT - Proceedings - IEEE 22nd International

PB - IEEE Computer Society

ER -