1 Data Centers and the Enterprise Network Architecture

Network Architecture is about how the building blocks of the networks are connected. Whereas data flows are about the information that flows through the network.

A typical data network is built out of 3 parts:

  • Data center ~> Holds the organization’s servers and applications.

  • Core network ~> The part of the network that is used to connect all the parts of the network, including the user’s network, data centers, remote networks, and the internet.

  • User’s network ~> The part of network that is used for the user’s connectivity.

Here we have 2 data centers DC1 and DC2 with Users-1 and Users-2, respectively. And these 2 data centers are connected to the internet via 2 firewalls, which are located in the 2 data centers.

In the center of the diagram, we can see the ‘Wide Area Network’ (WAN) connectivity, which includes the routers that connect to the service provider’s (SP’s) network and SP network that establishes this connectivity.

In the lower part, we can see the remote sites that connect to the center via the SP network.

Now let’s focus on the protocols and technologies that are implemented on each part of the network architecture.

2 The Data Center, core and user networks

2.1 Data Center network

Let’s first look at an organization’s data network that holds the majority of the organization’s servers. Let’s take our above 2 data centers DC1 and DC2. The user network depend on the size, geographical distribution, and the number of users in the organization. The core network is the backbone that connects the users to the data center, remote offices, and the internet.

At the top we can switches that are connected with the server via 2 cables. This connectivity can be implemented as port redundancy or link aggregation (LAG). A typical connection is implemented with 2 wires, copper or fiber.

[If these are heavy-duty servers then there can be 2-4 wires or more]

  • Port Redundancy ~> Redundancy involves the duplication of connectivity, that mean if one server fails then the redundant duplicate can take over partially until the failed server recovers. To achieve port redundancy, we can define a redundancy relationship between any 2 ports in a switch. One port is defined as primary port and the other as the secondary port.

  • Link Aggregation ~> Redundancy + Load sharing

2.2 Core Network

In the center, we can see the core switches. They connect between the data center and user network, and also connect to the remote sites, the internet and other networks.

The connectivity between core switches and data center switches can be implemented in Layer 2 or Layer 3.

2.3 User Network and Access area

The access layer holds the switches that connect to the users, while the distribution layer aggregates access switches.

For example, in a University network, there will be a distribution switch for every building or group of buildings, while the access switches are connected to the nearest one.

3 Switching (L2) and Routing (L3) topologies

Layer 2 switches are devices that switch packets between ports, while Layer 3 switches or routers look at the Layer 3 header of the packer and make routing decisions.

3.1 L2 Switching

Here we have a single LAN switch. Whenever a frame arrives at the switch, the switch looks at the destination MAC address, makes a forwarding decision, and forwards the frame to the destination port (here port 3).

Next if we have another switch (on the right) that is connected with our left switch. Here port 3 is connected to port 1 on the right switch, which looks at its MAC address and forwards it to the right switch; that is port 4.

The decision on how to forward the frame is done ‘locally’.

3.2 L3 Routing

In routing, when a packet enters the router, the router looks at the Layer 3 destination address (destination IP address), checks if the packet’s destination is valid in the routing table, and then makes a routing decision and forwards the packet to the next hop.

In ethernet, the destination address comes before the source.

3.2.1 Router Vs. L3 Switch

Let’s say we have assigned an IP address to every physical port (Int1, Int2, Int3 and Int4) and each physical port is connected to a Layer 2 switch. An we connect a client PC to the external switch.

In a Layer 3 switch, it is all in the same box. The Layer 3 interfaces (called Interface VLAN in Cisco) are software interfaces configured on the switch. VLANs are configured and a L3 interface is assigned to each. Then, the external devices (like our client PCs) are connected to the physical ports on the switch.

3.2.2 Advantages of L3 routing

  1. It provide us with more flexibility in the design in that every department can get an IP subnet with access rights to specific servers.

  2. Routing protocols can be implemented.

  3. Broadcasts don’t cross routers so that only a small part of the network will be harmed.

4 L2 and L3 Architectures

On the left, we have L3 on the core and DC switches and on the right we have L3 on the DC.

4.1 On the Left side

  • Virtual LANs (VLANs) configured on the core switches ~> VLAN50 and VLAN60 are the user’s VLANs. Each user VLAN holds several physical ports and one logical L3 interface (also called Interface VLAN in Cisco terminology). Let’s say interface VLAN50’s IP address is 10.50.1.1/16, while interface VLAN60’s IP address is 10.60.1.1/16.

  • VLAN configured on the DC switches ~> Let’s say VLAN10 and VLAN20 are the server’s VLANs. Each server VLAN holds several physical ports and one logical L3 interface. Let’s say, interface VLAN10’s IP address is 10.10.1.1/16 and interface VLAN20’s IP address is 10.20.1.1/16.

  • The default gateways of the users in the network 10.50.0.0/16 and 10.60.0.0/16 are 10.50.1.1 and 10.60.1.1, respectively.

4.2 On the Right side

We can see a different topology, where all the interface VLANs are on the DC switches.

  • All the VLANs are configured on the DC switches.

  • The core switches are only used as Layer 2 devices.

  • The default gateways of both the user’s devices and servers are on the DC switches.

5 L2 and L3 Architecture Data Flow

5.1 Left Topology

  • When sending packets from the users to the servers, users on VLAN50 or VLAN60 send packets to the default gateway (let’s say L3 interface on the left core switch). From there, packets are routed to the L3 interface on the left DC switch and ultimately the server.

  • When sending the packet back, the servers on VLAN10 or VLAN20 send packets to the default gateway of 10.10.1.1 (which is on the left DC switch). The packets are routed to the L3 interface on the left core switch and then the user.

5.2 Right Topology

  • Here DC switches are default gateways for the users and the servers, so packets from both are sent to the DC switches and routed internally in them.

6 L2 and L3 Architecture Data Flow with Redundancy

In this topology, we taking user’s L3 interface are on the core switches.

In the diagram, a PC with an address of 10.60.10.10/16 is sending information to the server on 10.20.1.100/16. Let’s look at the main and redundant flows.

6.1 Under Regular Conditions

When all the network components are functional. Let’s look at the data flow.

  • When PC2 sends packet to a server, they go to its default gateway (1); that is, 10.60.1.1 on the lower left core switch.

  • From 10.60.1.1, the packets are forwarded to 10.20.1.1 on the top left DC switch (2).

  • From 10.20.1.1, packets are forwarded to the upper server; that is 10.20.1.100/16 (3).

6.2 When a Failure Occurs

Suppose, the left DC switch (DC-SW-1) fails. Then what will happen???

  • The MAC address of the server S1 is now learned on the DC switch on the right (DC-SW-2), and from there it will be learned on the core switch on the right (CORE-SW-2).

  • Packets that are sent from PC2 to the server S1 will be forwarded to the core switch on the right (a).

  • The core switch then forward the packet to the next hop (b), which is the DC switch on the right (DV-SW-2).

  • Lastly, the DC switch on the right forwards the packets to the server S1 (c).

7 L2 and L3 topologies with Firewalls

A common practice to secure an enterprise network is to add firewalls to 2 locations.

  • Data center firewalls ~> Protect the data center where main servers reside. This firewall usually have packet filtering, stateful inspection, intrusion detection, and application filtering.
  • Core firewalls ~> Protect different users and areas in the network.

7.1 Important Terminologies

  • Packet filtering ~> Filtering packets according to Layer 3 (IP) and Layer 4 (TCP/UDP) information.

  • Stateful inspection ~> Mechanism that watches the direction of traffic crossing the firewall and allows traffic to be forwarded in the direction where the session started.

  • Intrusion prevention ~> Mechanism that protects against intrusion attempts to the network.

  • Application filtering ~> Mechanism that works on Layer 7 and filter sessions based on the application and its content.

7.2 Data flow in a Firewall-protected network

  1. PC2 has an address of 10.60.10.10 sends data to its default gateway; that is the IP interface on its VLAN (1). From there, packets are routed to the DC firewall (FW1) at the top-left (2) and the required server (3).

  2. A second option is when PC4 sends packet to the server. First the packet go to the default gateway; that is the IP interface of the VLAN(a). From there, they are routed to the core firewall (FW4) (b), the DC firewall (FW2) (c), and finally the server (d).

  3. There are many other possible routes.

8 L2 and L3 Topologies with Overlays

When building a traditional enterprise network, the network structure ensures one thing: that packets are forwarded from the source to the destination as fast as possible.

8.1 Parameters

  • Bandwidth ~> Defined as the number of bits per second that the network can provide.

  • Delay ~> Round-Trip-Time (RTR) in seconds that will take a packet to get to the destination and the response to arrive back to the sender.

  • Jitter ~> Defined as variation in delay and measured in percent.

  • Packet Loss ~> Percent of packets that were lost in the transmission.

A network attack on a communication line can cause degradation in the performance of one or all these parameters.

Overlay technologies provide additional functionality to the network, in a way that we establish a virtual network(s) over physical ones.

  • Physical network ~> Underlay network
  • Virtual network ~> Overlay network

Here we can see our underlay standard network (physical network) is made up of routers with connectivity between them, while overlay network (virtual network) is made-up of end-to-end tunnels that create a virtual network over the real one.

There are various overlay technologies, such as VxLAN, EVPN, and others.

8.2 Principle

The packets from the external network that are forwarded through the overlaid tunnels are encapsulated in the underlying packets, forwarded to the destination, and de-encapsulated when exiting to the destination.

Since bits are eventually forwarded through the wires, attacks on both the underlay network and the overlay network connectivity can influence and cause downtimes to the network.

9 The Network Perimeter

The network perimeter is the boundary between the private locally managed enterprise network and public network such as the internet.

As shown in the figure, a network perimeter includes ~>

  • Firewalls
  • Intrusion Detection and Prevention Systems (IDPSes)
  • Application-aware software
  • Sandboxes

To prevent malware from being forwarded to the internal network.

Between organization’s private network and the internet, there are 3 zones that act as the boundaries.

9.1 Internal Zone

This is the area that is used for organizing users and servers. It also referred to as the trusted zone. This is the zone with the highest level of security. No access is allowed from the external zones to the internal zone and all access, if any, should be through the DMZ.

9.2 Demilitarized Zone (DMZ)

This is the area that users from the internet can access, under restrictions. For example, mail relays, which receive emails from external servers and forward them to the internal server on the SZ, as well as websites and proxies, which act as mediation devices for controlling access to important servers, and others.

9.3 External zone

This is the connection to external network, such as Internet Service Providers (ISPs) end other external connections.