Virgin Media has made some bold claims about its new Super Hub wireless router. Best in class, faster than the competition, dual band, WiFi n, all sounds good on paper but just how good is it?
After hearing about the independent testing methods Virgin Media used we decided to take a closer look ourselves.
We headed out to the University of Bristol and Farncombe testing centres with Virgin Media, where we got to see the new Super Hub put through its paces.
As well as this, we were guided through the unique four step process used by the University and Farncombe to test out WiFi routers.
Virgin Media are not the only clients who make use of the service provided by the University and Farncombe.
Many other manufacturers and UK ISPs have had their hardware put through the same testing procedure, one which Andrew Nix, professor of wireless communication systems, argues is the most rigorous in the industry.
As there’s been no standardised method of WiFi testing, it’s fallen to the University and Farncombe to work out their four stage method from scratch. Professor Nix talks with great enthusiasm and pride about the process they’ve developed:
“What we have done if far more rigorous than any other process that I’ve seen. I will defend what we’ve done since it’s all based on solid electromagnetics and information theory. One of the things we’re very proud of is the repeatability of this process. If you put the same unit in, we’ll get the same numbers out.
“We can guarantee the same results every time, unless we change something like the position of the antennas or the product on the board. Before, there really was no way for the industry to understand the intricacies of what would happen if they moved the position of the antennas inside a wireless hub.”
Graham Harvey, head of WiFi testing at Farncombe agrees, saying: “Devices that are WiFi Certified are really just functional, they simply comply with 802.11. There’s no sense of performance and how well they work. That’s where our job starts. We take WiFi Certified devices and say ‘how good are they’?”
Stage 1: Controlled environment tests in an anechoic chamber
Professor Nix adds that to get a complete picture of a device’s behaviour, it needs to be tested over a wide range of orientations. By the same reasoning, customers should experiment with their broadband routers in the home by turning and moving them to see if the WiFi reception can be improved.
“If you change the rotation by as little as ten degrees or move the router by a few centimetres, left, right, up or down you’ll change the way the power radiates and get a completely different set of data throughputs. It can be that sensitive.”
This testing takes place in an anechoic chamber, an echo-free environment that’s been designed specifically to absorb sound and electromagnetic waves. Though the surfaces resemble giant pencil heads, or something from the imagination of David Lynch, the foam pyramids stuck to the walls, ceiling and floor are excellent at soaking up external noise and minimising internal reflections, ensuring that the test is as pure as possible.
A 4×4 square of cones apparently costs £100 a throw and the entire chamber cost over £500,000 to set up. We were told to be careful when we poked our heads around to have a look.
The test router is positioned on a board that’s attached to a mechanical arm. The arm rotates the router ensuring that antenna data from every possible angle can be captured from an RF (radio frequency) receiver that sits at the other end of the room.
This part of the test can be repeated as often as is needed, depending on how thoroughly the client wants its product tested.
Ideally, a good router will have multiple antennas positioned at different angles in order to provide the best overall coverage. Orientation of antennas is important, because devices like phones aren’t always held in a way that suits the antenna in a router.
For this reason, part of the testing involves measuring signal strength when antennas are positioned along horizontal, vertical and where appropriate, diagonal axes.
The new Virgin Media Super Hub has five antennas in total, two of which transmit and receive data on the 2.4GHz band and three that which use the 5GHz band. The BT Home Hub 4 has a total of three antennas, two of which are active at 2.4GHz one which uses both the 2.4GHz and 5GHz bands.
In theory, the greater the number of antennas on a specific band, angled at a specific polarisation, the greater your overall coverage in the home will be.
In practice, Professor Nix and his team use data collected from this part of the trial to build up a 3D polar plot – a footprint of a router’s signal to identify where its strengths and weaknesses are. No matter how you set your router up, there are a number of variables which mean that there will be some weaknesses.
Part of the testing that the University and Farncombe do is about finding weaknesses and assessing how they can be changed, something that we’ll take a look at in detail at Stage 4.
At this point in the testing, individual plots for each antenna are gathered and then they’re compiled to give an average footprint. Warmer colours indicate a stronger signal.
While we were present we got to see the two devices compared side by side, the results of which you’ll be able to see in our video below.
Stage 2: Virtual testing in a simulated environment
Once all of the anechoic chamber data has been gathered, the team runs a computer simulation giving them an idea of how the device will perform in a real life environment.
“When a signal leaves the router it doesn’t just travel in a straight line to your phone. It bounces off the walls, bounces off the shelves, bounces off bodies and down staircases. It moves all around the home and eventually makes it to your device. In order to understand how these signals move, we use a piece of software which we’ve developed over the last ten years.
“The software visualises the wireless footprint of a device, based on the data we’ve given it. The reason we need to know how signals propagate is so we can work out how signals come out of the device and interact with signals from laptops and phones that are connected via WiFi.”
If it’s the forward antenna’s signal that dominates to a particular device under test, then the relevant power mainly comes from the front of the unit and connects directly. If it’s the rear antenna that dominates then the relevant power comes from the back of the unit, but bounces towards the test device, connecting indirectly via one or more reflections and transmissions.
This kind of data, combined with the tests from the anechoic chamber, is then used to perform more than 100 million theoretic throughput calculations. This takes into account access points and clients spread all around the test house. The end result is a far more accurate (and statistically relevant) estimate of how a WiFi device will behave in a real world environment.
Armed with this data, it’s then off to Farncombe’s testing house to gather some empirical data.
Stage 3: Empirical testing in a real-world environment
Our ride pulled in at the idyllic Grey Dawn Cottage in the Cotswolds, the two-storey house that Farncombe uses to put devices like the new Super Hub to the test.
This remote location has been chosen due to the fact that there’s no mobile reception in the area and the stone walls mean that outside interference is minimal. The Grey Dawn Cottage tests are mainly about seeing how routers perform when client devices like laptops connect to them in different rooms and floors of a house.
Farncombe uses various hardware and software for testing, including IxChariot, an industry recognised tool for performance benchmarking, and AirMagnet, to monitor how WiFi signals move about the house.
Farncombe has three ‘easy,’ ‘medium’ and ‘hard’ trial positions throughout its house, repeatedly testing routers throughout the day.
An easy test consists of connecting to the router when you’ve got line of sight to the device. Medium tests are done diagonally opposite the router in another room on the same floor and a hard test will be done diagonally opposite and one floor up.
The router is typically placed in one location throughout the day and then a client device like a laptop connects to the router via WiFi from the three locations.
The laptop then records data about signal strength and connection speeds in real time, relaying this to an IxChariot server that’s running on a Windows PC connected directly to the router. Typically, testing lasts a week, but can go on for longer depending on the level of testing that a client wants.
Stage 4: Tear it down and start again
The fourth stage entails a full physical teardown of the test product carried out by Paul Harman, head of hardware design at Farncombe.
Harvey explains that this part of the testing is used to find physical reasons for any spikes or dips in signal and to understand how a performance might be improved by changes in design:
“At Stage 4 we dismantle the router and look at the CPU, components and the overall physical layout: venting, thermals, all aspects. If it’s an access point and it’s incapable of transmitting upstairs into a bedroom, for example, there may be a physical explanation for that.”
Despite being called ‘Stage 4’ it’s not always the last thing to happen in the testing process. In fact, it can often happen earlier on, before the empirical testing in Stage 3 begins.
“Generally with the testing service, Stage 4 isn’t in isolation to the other components,” says Harvey. “It’s called Stage 4, but it can happen at any time during the process, it doesn’t necessarily happen after the third stage.
“For example, if we know that there’s a severe weakness with a product as early as Stage 1, we’d take a look at that earlier and try to find a physical reason for why that might be and then suggest changes to the overall design.”
If a big redesign is required, ideally you don’t want to hear that towards the end of a project. Increasingly, Farncombe are getting involved with things like hardware design from as early a stage as possible.
Talking about working on the new Super Hub, Harvey told us: “We like to get involved from a hardware design perspective as early as possible. On the new Super Hub we’ve worked with Virgin Media at earlier stages to help them with vendor selection and even earlier, when the first proof of concept units arrived.”
“We’ve been working with Virgin Media for three years now and we’ve seen them evolve. At Farncombe we’ve had the opportunity to look at the first Super Hub which, as I’m sure some people will say, wasn’t so super.”
Because of the longstanding relationship with Virgin Media, Farncombe were able to advise with the design of the new Super Hub.
“We got involved at various stages at the start of the new Super Hub’s life cycle. The new Super Hub was very much about optimising that WiFi performance and improving on some of the negative customer feedback Virgin had received with the last one.”
The future of wireless router design
With all the developments and improvements that led to the creation of the new Super Hub, what’s in store for the future?
As speeds outside British homes continue to increase, there’s been talk that broadband bottlenecks will move beyond the networks and into people’s homes.
The BT Home Hub 5, which is aimed at customers taking fibre-based broadband, will transmit data at speeds of up to 1Gbps about the home and we’ve already seen TalkTalk experimenting with signal shapes better suited to subscribers’ homes.
Harvey adds that there’s been a general evolution across the whole industry and routers supplied by ISPs will be designed more with customer’s needs in mind. Support of technologies like dual band will become the norm.
“If you look at some of the earlier kit offered by ISPs, it was single band and providers though they were doing their customers a world of good simply by providing wireless in the home at all.”
“What customers want now is WiFi all over the home and high performance WiFi. What we’ve seen here is a result of Virgin having listened to customers and delivering something that they want.”
Most new phones, tablets and laptops will make use of the 5GHz band and as customers make use of this second, less congested channel, this will allow for a smoother performance.
Leave a Reply