4G WAN: Design Considerations

We generally recommend using a managed service (such as our Rapid Site Deployment solution) to deploy 4G WAN. That being said, it's always helpful to appreciate some of the design considerations that your service provider will be (or should be) thinking about. If you're interested in a slightly more technical overview to 4G WAN technology, this post is for you!

Introduction

The diagram below shows an example deployment based on a site router with all four UK mobile operators securely connecting via the internet to a pair of resilient hubs located in the customer data centre connected to the corporate applications.

As you will see in the diagram a bonded 4G service is based on internet VPN technology at a basic level, but it is using multiple connections and carriers to create a bonded service over the internet via cellular data connections.

You can see that there are 3 essential components to the selection.

• The first is a site router device.  There are many options for multi cellular routers across the popular vendors, such as Peplink and Viprinet, who we tend to favour.
• The second key component is a hub device. The easiest way to think of a hub is a VPN concentrator device, but instead of single tunnels from the site router, it terminates multiple tunnels from the site router (one per cellular modem). Each vendor has its own proprietary technology, so other standard technologies like IPSEC cannot be used to terminate bonded tunnels.
• The last component is the cellular connectivity.  You should aim to use SIMs from multiple carriers to ensure service just about anywhere there is cellular service, and avoid the risk of any specific carrier having poor availability at your location.

Bonding for reliable bandwidth

Bonding offers advantages over a load balanced connection, specifically, in that it is a two-way connection compared to a load balanced connection which is only really one way (eg internally initiated sessions going outbound to the internet where you have no way to control or influence how the traffic works coming back in).

With bonding what you actually have is multiple secure channels from the site router to an endpoint device, in this case a hub, as mentioned earlier the easiest way to think of the hub device is as a VPN concentrator.

Each cellular connection runs its own discrete VPN tunnel at AES 256bit level encryption back to the hub. Any packets sent to the tunnel are automatically split up across these channels to the hub where they get re-assembled and leave the LAN side of that hub in the same order, this ensures you don’t have out of sequence packets.

From a security point of view this makes data interception very difficult as not only would you have to decrypt the traffic but you have the data stream split across 3 different tunnels, so you would have to intercept and decrypt and re-assemble multiple streams to get to the data this is much more complex than a standard single VPN.

To the users and applications they run the bonding is transparent, they don’t know its happening as it just appears as if they have a more reliable internet VPN.

On top of that fact you have these multiple channels in which to send and receive data there are additional mechanisms within each vendors bonding technology to help make these connections more reliable and also give priority to specific applications.

For example, if you have a voice stream you need to make sure this latency sensitive traffic is sent as quickly as possible, in order to do this both Viprinet and Peplink have QoS mechanisms to ensure matched packets are sent down the connections which are lower latency or have a more reliable connection score, the devices are constantly monitoring the latency and packet loss of each channel so it helps the devices decide which channels are more reliable or have a lower average latency.

In order to match traffic to rules this follows the same methodology as any Cisco type installation, so the usual mechanisms such as DSCP tags, application port, source and destination can be used to identify specific application traffic and then apply some rules to it. This not only means you can prioritise traffic but you can also de-prioritise non critical traffic so it doesn’t eat all the available bandwidth. One example of this would be an FTP file transfer.  FTP is a very aggressive protocol and will take all the bandwidth it can. With the correct configuration you can restrict this application to use only a smaller percentage of the available bandwidth and/or lower its priority so when key applications which have been prioritized are running, the FTP traffic will be squashed down so that it doesn’t impact the critical apps.

For more information on bonding, see our post about Bonding versus Load Balancing.

Vendor considerations

We're keen users of Viprinet and Peplink devices because, over many hundreds of installations, we have found these to be particularly reliable, manageable and high performance.

Some key differences between the vendors is that Viprinet devices only bond.  I.e. they will only send traffic to a head end hub unit. You always have to have a hub, and local breakout on is not possible from the site routers in their range. They also don't have WiFi, except on the midrange 510 unit which is PSK only (this unit is designed for vehicle or portable use).

Peplink devices can bond but they can also breakout locally across the range of devices, and you can use them for load balanced direct internet access. Peplink devices have in-built WiFi across the range, which supports 802.1x as well as PSK security.

Viprinet devices have some additional licensed features which can be purchased, one of which is called stacking.  Stacking means you can combine multiple site routers to become one large router with more channels. For example, you can stack together two 310 units to create one router with 6 channels. This gives hardware redundancy, as it will failover to whichever unit is left running and drop channels.  It also gives an increase in bonding capacity.

The key thing to remember with stacking is that the model selection is key. The devices all have an upper speed recommendation. With the 310 for example it's 50Mbps, so you will never achieve more than 50Mbps, even if you keep adding more units to the stack. This is because one device runs as the stack master, and this will process most of the bonding load.

We have utilized stacking for extreme temporary installations such as stacking three 2620 models together to give 18 channels of 4G connectivity. This is usually only required when very high bandwidth is required or a large number of users are on a site.  For example, we recently connected a customer's corporate HQ to its data centre when their SHDS fibre circuit wasn't delivered in time for a site move.

Peplink Speedfusion

Peplink has its own proprietary bonding technology, called Speedfusion.

Speedfusion has the ability to split up data packets across multiple connections. As shown in the diagram below, we have an HD4 site router which has four cellular modems.  Each will be running its own tunnel back to the Peplink hub. The router and hub use all of these VPN’s together to split up packets to and from the site router.

There are a couple advantages to this:

• First, it means that if a packet is sent down one tunnel and that connection drops, it will automatically send it down another live tunnel. There is no delay, unlike traditional failover mechanisms which can take seconds to kick in. With traditional failover this is usually noticeable to applications and they can often completely drop the application session for the user.
• Another benefit is that the hub and router both monitor the quality of each tunnel. They check the latency and packet loss going across each tunnel. This means you are able to make more intelligent decisions about sending specific application traffic down the better connections.  Common QoS mechanisms enable you to identify and prioritise types of traffic but also enable you to use algorithms such as “lowest latency” to match against traffic types. This is possible because of the connection monitoring within the Speedfusion tunnel.

The importance of capturing application use

Capturing your network and application requirements is a key part of designing, installing and configuring a 4G WAN service such as Rapid Site Deployment. For example if you have latency or drop-sensitive applications then having knowledge of these and what they are is key to setting up a successful service.  

• An example of a latency sensitive application would be Citrix (high latency results in delay for screen refresh as well as keystroke and mouse movement delays, or worse: a complete drop of session).
• An example of a drop sensitive application would be a telnet or SSH session, such as point of sale green-screen type applications. These use very little bandwidth and aren’t especially sensitive to latency, but they are sensitive to packet loss or drops, as it can often result in the session being completely dropped.

We always ask for detailed information, so that we can tune the installation appropriately. To deliver a reliable, high performance connection it's critical to identify the applications that are to be used, and how they are to be delivered.   

For example you might have a latency-tolerant application, but it’s delivered to site inside a Citrix session.  Suddenly, the delivery mechanism of the application is latency sensitive. This kind of information is key to setting up the configuration on the equipment so the service runs as best as it can in each instance.

Many people will not know the bandwidth demands of their applications but it’s always getting as much detail as you can, to help steer you to the best solution for the requirement.

Considerations for the bonded hub

The first part of an installation is getting the hub itself installed. This often resides in your data centre or HQ site behind a firewall, so  the device will need a static public IP allocated to it from your existing connection.  Here are some points to consider:

• Do you have sufficient bandwidth spare on this connection to cope with sending bonded VPN traffic into it?
• Do you have physical rack space to host and run the unit?
• A common issue is that the hub will need to be installed into a data centre managed by a third party so information needs to be exchanged ahead of and install in terms of requirements (eg static IP requirement, which rules required on the firewall to allow the service to work etc).

In terms of scaling, the hub is the critical component here.  Hubs scale in two dimensions

• The aggregate bandwidth capacity
• The number of remote devices supported

Peplink limit both but Viprinet only limit on aggregate capacity - they set no hard limits on the number of remote peers. You need to consider how many branch routers you will be using at once and also roughly how much bandwidth. You need to be mindful of the bandwidth sum potential of all the site routers hitting the head-end hub.  

This, of course, impacts your internet connection. This may be a consideration early on in the project if you need to upgrade an existing internet connection or order a new one for the hub.

We find that many customers don't want the hassle of running a Hub (and potentially a redundant Hub), so we fairly commonly provide space on our shared Hubs.

Security

Security will always be a concern for anyone using a service which uses the internet.  Bonding does offer some additional security benefits over traditional internet VPN, as we have seen above.

To take advantage of these benefits you must allow certain protocols in through your firewall for the service to work. Both Peplink and Viprinet bonded services require specific ports opened on the firewall to allow the site routers to tunnel into the hub devices. 

In most instances, a NAT’d connection is the easiest and safest way to install.  This means the hub will be installed behind an existing firewall on an existing internet connection. You will need to allocate sufficient bandwidth to the service on your existing internet connection, and also at least one public IP address which is dedicated to each active hub device. For example if you ended up with a split data centre solution with an active hub in each, then one public IP would be required for each data centre internet connection.

Resilience

Resilience of a bonded 4G solution needs to be considered if you have multiple sites connecting through a hub.  The hub device then becomes a single point of failure across the multiple remote sites.  We have many customers with multiple sites on 4G, so by default we have dual active/passive hubs. This means a passive unit “shadows” the active and takes over all its functions, and addressing if the primary unit fails. The failed unit can then be replaced like for like as part of the service.

With site routers, resilience is built in, since  bonding uses multiple channels.  If a single cellular connection fails then the router will carry on working with the ones it has left.  There are options for site router redundancy as well.  

Installation location and aerials

A key consideration for the remote site routers is their installation location.  The usual place to find a router is within a rack in a comms room.  However, comms rooms are often internal, with no external windows or walls and are often in basements. Consequently, they often have a very poor cellular signal.

With a 4G WAN circuit, the router will need to be installed where it can get the best cellular reception. This is unlikely to be a comms room or rack. You often need to install the router in an office, near a window, perhaps on a higher floor to pick up a better signal.

Whereas a LAN cable can be run to 100m, the maximum cable distance for an external antenna is generally set at 5 meters to minimise signal loss.  Therefore, in these kinds of installations it’s easier to extend the LAN side of the unit by plugging into existing infrastructure cabling. 

Using an external antenna outside the building will often give great benefits to signal strength, and more importantly, signal quality.  It is signal quality that directly impacts the potential speed of each connection.  Always think about whether you can use an external antenna.  Here are some considerations:

• Which side of the building will give the best signal strength?
• What are the mounting options on the outside of the building?
• Where can you put holes through walls?
• Remember the 5 meter cable length!

One scenario where external antenna will always be a requirement is Portakabins on construction sites. Whilst the Portakabins of old were typically a plaster type unit, modern units are often sheet metal construction and effectively act a Faraday cage, shielding the inside against external RF signals. Even mobile phones can struggle to get a GSM signal, let alone any data connectivity. Hence external antenna are a must in these types of installation.  Magnetic mount aerials are often used on Portakabins.

Different options for external antenna are available. In most instances an omni directional aerial with gain will be appropriate. However, there can be circumstances such as very remote or weak signal sites which would benefit from high gain directional antenna. Directional antenna have to be pointed towards the tower providing service so the location of each carriers tower must be known before installation is possible.

Technical help

We have just scratched the surface in this post.  There are a host of other considerations, particularly around connecting to the corporate network, tuning for performance, and SIM management.  If you believe you may rely on a 4G WAN circuit, it would pay to speak to an expert who can give you confidence in a design for your situation.