Measuring Broadband Canada

The information in this document is correct as of June 2020

2020

Measuring Broadband

Canada

Report, June 2020

The information in this document is correct as of June 2020

[THIS PAGE LEFT INTENTIONALLY BLANK]

The information in this document is correct as of June 2020

Table of Contents

Table of Contents .......................................................................................................................................................... 3

1. About the Project ................................................................................................................................................. 4

2. Executive Summary ............................................................................................................................................. 7

3. Measurement Results .......................................................................................................................................... 9

3.1 Download Speed ............................................................................................................................................. 9

3.2 Upload Speed ................................................................................................................................................ 14

3.3 Latency .......................................................................................................................................................... 21

3.4 Webpage Loading Time ................................................................................................................................. 23

3.5 Packet Loss .................................................................................................................................................... 27

4. Conclusion ......................................................................................................................................................... 28

5. Glossary ............................................................................................................................................................. 29

Measurements .............................................................................................................................................. 29

Statistical Terms ............................................................................................................................................ 29

6. Methodology ..................................................................................................................................................... 32

Whiteboxes ................................................................................................................................................... 32

Measurements .............................................................................................................................................. 32

Test Destinations........................................................................................................................................... 33

Sample Plan ................................................................................................................................................... 34

Data Processing ............................................................................................................................................. 35

Code of Conduct............................................................................................................................................ 37

The information in this document is correct as of June 2020

About the Project

The Canadian Radio-television and Telecommunications Commission (CRTC) has commissioned

SamKnows to conduct a study of the performance of broadband services sold to Canadian consumers.

SamKnows is a global leader in broadband measurement and has been working with governments, ISPs,

content service providers, application developers, consumer groups, and academics to accurately

measure Internet performance since 2008. In doing so, SamKnows has built a global Internet

measurement platform, which now spans five continents and conducts many millions of measurements

each day.

Data presented in this report was collected between the 1

of October 2019 and the 31

of October

2019.

3,266 Whiteboxes were deployed to Canadian volunteers as part of this study, out of which 3,024 were

validated

(i.e., the volunteer’s subscription package and other metadata were confirmed) by the

Internet Service Providers participating in the program. From this pool of validated Whiteboxes, a total

of 2,035 were on internet packages that were included in the sample plan, with a sufficiently large data

sample for inclusion in the report. Please note that no 1Gbps tiers are tested in the sample plan. While

some providers do offer advertised speeds of 1Gbps or faster, services above 940Mbps cannot be

measured with conventional speed tests. As such, speed tiers of 1Gbps or faster are excluded from the

2019 Measuring Broadband Canada project.

The ISPs participating in this project were Bell Canada, Bell Aliant, Bell MTS, Cogeco, Northwestel,

Rogers, Shaw, TELUS, and Videotron. The respective internet packages included in this report are those

packages with the highest subscriber counts, therefore representing a majority of Canadian fixed-line

broadband users. Other packages are offered by ISPs, but are not included in this report as they either

did not meet the sample plan conditions (see Section 6d), or, as mentioned above, the packages

offered speeds in excess of 940 Mbps which cannot be measured with conventional speed tests, or the

number of Whiteboxes did not meet the minimum required sample size. In addition, please note that

all references to “as a percentage of advertised speed” in the report refer to “percentage of maximum

advertised” [by the ISP]. The report does not compare results to advertised “minimum” or “most

customers get” as advertised by certain ISPs.

Internet packages included in this report represent three different access technologies, with each ISP’s

users testing across the following:



Bell Canada: DSL/FTTH (Fiber to the Home)



Bell MTS: DSL



Bell Aliant: FTTH



Cogeco: Cable/HFC (Hybrid fiber-coaxial)



Northwestel: Cable/HFC



Rogers: Cable/HFC



Shaw: Cable/HFC



TELUS: DSL; FTTH



Videotron: Cable/HFC

Whiteboxes conduct end-to-end tests 24 hours a day, 7 days a week to test servers located in major Canadian metropolitan areas and aims to

provide a representative picture on the status of internet performance across the majority of Canadian internet users.

The information in this document is correct as of June 2020

SamKnows recommends each internet package in the sample plan have a minimum number (40) of

Whiteboxes reporting and collecting data in order to provide statistical accuracy in the results. In

addition each Whitebox must have provided a minimum of 5 days of valid data during the reporting

period in order to be included.

Exceptions to the 40-Whitebox minimum have been made on a case-by-case basis. Northwestel’s

packages have been included in order to demonstrate the performance of a remote broadband

provider. Other internet packages have been included when the sample size is of at least 35

Whiteboxes, and the 95% confidence interval is within 5%, which indicate the vast majority of users

can be expected to have a very similar performance if further samples were collected.

Regional performance is also reviewed in the report. Canada is split into the West & North, Central and

Eastern regions. The Provinces and Territories which make up each region are listed here (Provinces

and Territories not listed did not have Whiteboxes measuring performance in that area). A map of

Canada showing the split between regions is included below:

 West & North: Yukon Territory, Northwest Territories, British Columbia, Alberta, Saskatchewan,

Manitoba.

 East: New Brunswick, Nova Scotia, Newfoundland, Prince Edward Island.

 Central: Ontario, Quebec.

The information in this document is correct as of June 2020

This report is designed to provide accurate data on the broadband performance experienced by the

majority of Canadian fixed-line broadband users, and should not be used to make comparative

marketing claims. The participating ISPs have agreed to a Code of Conduct that requires that they do not

use this Phase II Report to make or support performance claims in the marketplace, comparing its results

against those of any other participant, ISP or of different technologies.

As in the last Measuring Broadband Canada report the Whitebox will run tests regardless of whether

IPTV traffic is present or not. Many of the represented ISPs have service offerings that include the

potential for IPTV subscriptions to be combined and carried over the same medium as their internet

packages. For most of the tested speed tiers in this report, it is not anticipated to materially affect

results; however, some of the lower speed tiers, particularly carried over DSL, may see an impact

during peak periods where users are most likely to be watching TV while internet tests are being

carried out.

The information in this document is correct as of June 2020

Executive Summary

This report presents the findings of the measurement study conducted by SamKnows on behalf of the

CRTC during the month of October 2019, and investigates the following internet performance metrics:

download and upload speeds, latency, packet loss and webpage loading time.

The study was conducted using data collected from 2,035 Whiteboxes that were deployed to Canadian

homes. Volunteers were selected to take part in the study on the basis of their ISP and internet package.

Each Whitebox conducts end-to-end tests 24 hours a day, 7 days a week to test servers located in major

Canadian metropolitan areas and aims to provide a representative picture on the status of internet

performance across the majority of Canadian internet users. Results herein are presented by ISP,

internet package, and region.

Unless otherwise stated the results presented are taken from peak periods of user activity at the local

time, which is defined as 7pm to 11pm Monday to Friday. Off-peak periods are defined as any hour and

day exclusive of peak periods. A minimum sample size of 40 Whiteboxes has been used to determine the

inclusion of individual internet packages, except for few cases evaluated individually on the basis of

sample confidence intervals. Any case where the sample size is lower than 40 has been denoted by an

asterisk (*). In order to achieve a like-to-like comparison across internet packages featuring different

download and upload speeds, all test data pertaining download and upload in this report is expressed as

a percentage of the maximum speeds advertised by ISPs across both metrics.

The test data for this report finds that all major Canadian ISPs are delivering users with average

download speeds that exceed maximum advertised rates and that overprovisioning (providing users

with additional throughput) is common. The only internet packages included in this report that did not

meet the advertised maximum download speeds were Bell Canada’s DSL 100x10Mbps and Shaw’s

Cable/HFC 300x15Mbps at a respective 93% and 99%.

Upload speeds were similarly high across the board, with most ISPs delivering their advertised speeds.

The tiers tested for Bell Aliant and Bell Canada were lower than 100% with 94% and 87% during peak

hours. Bell Canada’s average upload was noticeably lower across the DSL 15x10, DSL 25x10, and DSL

50x10Mbps tiers, which tested at a respective 71%, 77%, and 91% of advertised maximum speeds.

Download and upload speeds were not found to decrease significantly during peak hours, which is

representative of a broadband network that is well provisioned to cope with periods of increased user

activity.

Latency data for this report was focused on Whiteboxes located within a 150km radius of the test

server locations in order to minimize the effect of distance on measurements. Average latency during

peak hours was of 4 milliseconds for users on fiber connections, and a respective 12 and 14

milliseconds for users on DSL and Cable/HFC connections. Packet loss was also low, averaging 0.06%

for users on fiber, 0.11% on DSL, and 0.13% for users on Cable/HFC connections. These are levels at

which most users would experience no discernible issues arising from packet loss.

The information in this document is correct as of June 2020

Average webpage loading times to a selection of websites popular amongst Canadian users was under

3 seconds, with the majority of users being able to fully load webpages between 1.3 and 2.4 seconds.

Load times are found to be approximately 0.7 seconds faster for internet packages with download

speeds of 25Mbps or higher, but see no discernible improvement as speeds increase to the 600Mbps

range.

The information in this document is correct as of June 2020

Measurement Results

3.1

Download Speed

Download speed measures the capacity of the user’s broadband connection for downloading content

from the internet. Higher speeds are more desirable, as it allows users to retrieve content, such as web

pages, videos, files, or music more quickly.

Internet packages are commonly advertised by service providers on the basis of download and upload

speeds, which are expressed in megabits per second (abbreviated as “Mbps” or, less commonly,

“Mbit/s”). For example, an ISP might advertise an internet package as “up to 50x10Mbps” or “up to

50/10Mbps”, signifying “the maximum speed expected is of 50 megabits per second for download and

10 megabits per second for upload”. In order for internet packages of different speeds to be

comparable, download speeds are therefore presented in terms of the percentage of their advertised

maximum speed. For example, if a user is subscribed to an internet package advertised as “up to

50Mbps download” and their download speeds averaged 45Mbps during the testing period, we would

say that the user’s average percentage of maximum download speed was (45Mbps/50Mbps)*100% =

90%.

Figure 1 below shows the average download speed as a percentage of advertised maximum during

peak and off- peak hours for each ISP and technology. The different columns represent the periods of

highest and lowest network usage in Canada. Peak hours are defined as 7-11pm from Monday to

Friday, whereas off-peak hours comprise all other hours of weekdays, as well as every hour of the

weekend. The 95% confidence interval

is also displayed as thin lines above and below the average

value.

The 95% confidence interval is a range in which the ‘true’ average value is estimated to lie and is a function of the sample size (i.e. number of

Whiteboxes) and standard deviation. If standard deviation is higher, then the confidence interval will be wider, reflecting greater variability in the

underlying data. If sample size is higher, then the confidence interval will be narrower, reflecting more certainty in the underlying data. For

example: we calculate the average download performance for a given ISP to be 91%, with a 95% confidence interval of ±2%. This means that if the

measurement were repeated, we would be 95% certain that the average of this repeated measurement would fall between 89% and 93%.

108%

109%

110%

112%

130%

107%

111%

103%

113%

115%

105%

106%

107%

109%

111%

129%

104%

110%

102%

113%

115%

20%

40%

60%

80%

100%

120%

140%

160%

Bell Canada

Bell MTS

Telus

Cogeco

Northwestel

Rogers

Shaw

Videotron

BellAliant

Bell Canada

Telus

DSL Cable/HFC FTTH

% Advertised Speed

Peak vs Off-Peak Download Speed as a percentage of Maximum Advertised Speed,

by Technology and ISP

Off-Peak

Peak

Figure 1 Peak vs Off-Peak Download Speed as a percentage of Maximum Advertised Speed, by Technology and

ISP

The information in this document is correct as of June 2020

Average download speeds for users in Canada exceeded those advertised across all ISPs included in this

report, ranging from 102% during peak hours for Bell Aliant FTTH to 129% for Rogers Cable/HFC. The

effect of network congestion during peak hours is minimal, with most ISPs seeing an average decrease

in download speed in the order of 1 to 2 percentage points. This decrease in performance is slightly

higher for Bell MTS DSL and Shaw Cable/HFC, at 3 and 4 percentage points respectively, but the impact

on performance would nevertheless be negligible to the vast majority of a user’s online activities.

Figures 2 to 4 below show the average peak-hour download speed as a percentage of maximum

advertised rates for each internet package included in this report. For ease of presentation, the charts

have been divided according to “speed bins”. Packages that did not meet the minimum recommended

sample size but had at least 35 Whiteboxes reporting and a small enough standard deviation have been

marked with an asterisk (*).

Figure 2 Download Speed as a percentage of Maximum Advertised Speed, by ISP and Package, 10-25Mbps download tiers

Average performance was good across internet packages whose download speed was advertised

between 10 and 25Mbps, as all services met their respective advertised speeds during peak hours. Most

internet packages in the chart above show download performances that exceed maximum advertised

rates by a significant margin, meaning that overprovisioning (ISPs providing users with higher speeds

than advertised) is commonplace.

Download Speed as a percentage of Maximum Advertised Speed,

by ISP and Package, 10-25Mbps download tiers

140%

120%

100%

80%

60%

40%

20%

10x2* 25x2 15x10 25x10 15x2 20x3* 15x1.5 15x1*

Bell MTS Bell Canada

Cogeco North- Shaw

westel

25x5 25x25 10x1.5 15x10

TELUS Videotron

% Advertised Speed

117%

100%

113%

108%

113%

112%

110%

109%

107%

117%

111%

The information in this document is correct as of June 2020

Figure 3 Download Speed as a percentage of Maximum Advertised Speed, by ISP and Package 30 –75Mbps download tiers

Overprovisioning also appears to be common amongst internet packages with maximum download speeds

between 30 and 75Mbps, where average speeds also exceeded advertised rates by good margins. Bell

Canada’s FTTH 50x50Mbps package was a particularly strong performer, delivering an average peak-hour

performance of 124% during the testing period.

Figure 4 Download Speed as a percentage of Maximum Advertised Speed, by ISP and Package 100 –600Mbps download tiers

Figure 4 above shows the highest-speed tiers included in this report, with speeds ranging from

100Mbps to 600Mbps. Most internet packages here were also overprovisioned to varying degrees, with

Rogers’ Cable/HFC 150x15Mbps service showing very high speeds compared to those advertised by the

ISP. Rogers’ Cable/HFC 500x20Mbps package was also a high performer that commonly exceeded

advertised rates, as was Bell Canada’s FTTH 150x150Mbps. Bell Canada’s DSL 100x10Mbps plan did not

meet, on average, the ISP’s advertised rates, delivering speeds of 93% during peak hours. Shaw’s

300x15Mbps package was also under 100%, at 99%.

Download Speed as a percentage of Maximum Advertised Speed,

by ISP and Package, 30-75Mbps download tiers

140%

120%

100%

80%

60%

40%

20%

50x10 50x50 40x10 60x10 50x4* 60x10* 75x10 75x7.5* 50x10 75x15 75x75 30x10 60x10

Bell Canada Cogeco

North-

westel

Rogers

Shaw

TELUS

Videotron

Download Speed as a percentage of Maximum Advertised Speed,

by ISP and Package, 100-600Mbps download tiers

180%

160%

140%

120%

100%

80%

60%

40%

20%

500x500 100x10 150x150 300x300 120x10 150x15 500x20 300x15 600x20 150x150 300x300

Bell Aliant Bell Canada Cogeco Rogers Shaw TELUS

% Advertised Speed

106%

102%

124%

93%

107%

120%

116%

106%

108%

101%

112%

147%

117%

108%

130%

113%

99%

101%

103%

119%

114%

109%

111%

The information in this document is correct as of June 2020

Figure 5 shows the average download performance across Canada by service technology and region.

Please note that no DSL products in the East region have been included in the final sample for this

report.

Canadian ISPs delivered users with average download speeds that exceeded the maximum advertised

rates across all technologies and regions. DSL services had comparable performances of 105% and

107% across the Central and West & North regions, with similar decreases of 3 percentage points

compared to off-peak hours. Cable/HFC services had their highest performance in the East at 128%,

followed by the Central region (116%) and then West & North (105%), with limited decreases

compared to off-peak hours. FTTH services instead saw their highest performance in the West & North

regions at 115%, followed by the Central region (112%), with performance in the East lower at 102%.

Service deterioration was lowest on FTTH services, with virtually no change (1 percentage point)

compared to off-peak hours.

Figure 5 Peak vs Off-Peak Download Speed as a percentage of Maximum Advertised Speed,

by Technology and region

108%

110%

117%

130%

108%

113%

103%

115%

105%

107%

116%

128%

105%

112%

102%

115%

20%

40%

60%

80%

100%

120%

140%

160%

Central West &

North

Central East West &

North

Central East West &

North

DSL Cable/HFC FTTH

% Advertised Speed

Peak vs Off-Peak Download Speed as a percentage of Maximum Advertised Speed, by

Technology and region

Off-Peak

Peak

The information in this document is correct as of June 2020

Table 1 below shows the advertised maximum download speed at peak hours for each ISP and internet

package included in this report, expressed both in terms of megabits per second as well as a percentage

of advertised, along with the sample count and standard deviation (in terms of percentage of advertised)

for each.

Table 1 Advertised download speed at peak hours for each ISP and internet package.

ISP

Technology

Package

Average Peak Hour

Speed (Mbps)

Average Speed

(% of Max Adv.)

Std. Dev.

Whitebox

Count

Bell MTS

DSL

10x2*

11.7

117.2%

2.0%

Videotron

Cable/HFC

10x1.5

11.1

110.8%

2.8%

Bell Canada

DSL

15x10

17.0

113.4%

13.4%

Cogeco

Cable/HFC

15x2

16.9

112.6%

2.5%

Shaw

Cable/HFC

15x1.5

16.6

110.4%

2.6%

TELUS

DSL

15x1*

16.4

109.1%

12.6%

Videotron

Cable/HFC

15x10

16.6

110.7%

8.6%

Northwestel

Cable/HFC

20x3*

22.4

112.1%

6.2%

Bell Canada

DSL

25x10

27.0

108.0%

18.3%

Bell MTS

DSL

25x2

25.0

99.8%

10.6%

TELUS

DSL

25x5

26.7

106.8%

13.5%

TELUS

FTTH

25x25

29.3

117.1%

2.0%

Videotron

Cable/HFC

30x10

32.7

109.1%

7.5%

Cogeco

Cable/HFC

40x10

42.8

107.1%

4.7%

Bell Canada

FTTH

50x50

61.9

123.7%

6.0%

Bell Canada

DSL

50x10

52.9

105.7%

12.2%

Northwestel

Cable/HFC

50x4*

54.0

108.1%

18.1%

TELUS

DSL

50x10

56.4

112.8%

12.6%

Cogeco

Cable/HFC

60x10

69.7

116.2%

3.9%

Rogers

Cable/HFC

60x10*

67.4

112.4%

6.2%

Videotron

Cable/HFC

60x10

66.9

111.5%

2.6%

Rogers

Cable/HFC

75x10

88.0

117.3%

12.3%

Shaw

Cable/HFC

75x7.5*

80.6

107.5%

13.9%

TELUS

FTTH

75x75

89.1

118.8%

2.6%

TELUS

DSL

75x15

75.8

101.1%

6.9%

Bell Canada

DSL

100x10

93.3

93.3%

6.6%

Cogeco

Cable/HFC

120x10

121.5

101.3%

10.8%

Bell Canada

FTTH

150x150

179.9

119.9%

5.6%

Rogers

Cable/HFC

150x15

220.2

146.8%

35.4%

TELUS

FTTH

150x150

170.4

113.6%

8.9%

Bell Canada

FTTH

300x300

319.5

106.5%

7.2%

131

Shaw

Cable/HFC

300x15

297.7

99.2%

15.8%

110

TELUS

FTTH

300x300

332.2

110.7%

0.7%

Bell Aliant

FTTH

500x500

507.5

101.5%

12.4%

Rogers

Cable/HFC

500x20

649.3

129.9%

18.3%

Shaw

Cable/HFC

600x20

618.8

103.1%

11.2%

The information in this document is correct as of June 2020

3.2

Upload Speed

Upload speed is the measure of how fast data can be transmitted from the home to the internet. Higher

speeds can allow for pictures, music, and documents to be uploaded and shared more quickly.

Results are presented as a ratio of the average measured speed to the advertised maximum by the ISP.

For example, if a user on an internet package advertised as “up to 50x10Mbps” receives, on average,

upload speeds of 12Mbps during tests, then we say that the user received (12Mbps / 10Mbps)*100% =

120% of maximum advertised speeds.

Figure 6 below shows the average upload speed as a percentage of maximum advertised during peak

and off-peak hours for each ISP and technology. The different columns represent the periods of highest

and lowest network usage in Canada. Peak hours are defined as 7-11pm from Monday to Friday,

whereas off-peak hours comprise all other hours of weekdays, as well as every hour of the weekend.

The 95% confidence interval is also displayed as thin lines above and below the average value.

Average upload speeds were largely above advertised rates during peak usage hours, with TELUS delivering

the highest at 164% on DSL (113% on FTTH). It is worth noting, however, the very wide confidence interval

which reflects that performance is likely to vary significantly from one user to the next. Northwestel

Cable/HFC and Bell MTS DSL also delivered high upload performance to their users at 119% and 115%.

These were followed in turn by Cogeco (109%), Videotron (109%), Shaw (105%) and Rogers (104%). On DSL

Bell Canada delivered upload speeds that were below their advertised maximum rates, at 87%. Similarly Bell

Aliant FTTH 500x500 fell under 100% of maximum advertised speed with 94% during peak hours.

As with

download speeds, upload speeds were not particularly affected by increased network congestion during

peak hours, thus showing a broadband network that is well-provisioned and able to cope with increased

user activity during the evenings.

Speeds for Bell Aliant FTTH 500x500 were in excess of advertised maximum speeds following correction of an issue with the software running

on their optical line terminals partway through the month. The issue resulted in customers measuring lower upload speeds during the initial

seconds of an upload test.

87%

115%

164%

110%

119%

106%

105%

109%

95%

109%

113%

87%

115%

164%

109%

119%

104%

105%

109%

94%

109%

113%

20%

40%

60%

80%

100%

120%

140%

160%

180%

200%

Bell Canada

Bell MTS

Telus

Cogeco

Northwestel

Rogers

Shaw

Videotron

Bell Aliant

Bell Canada

Telus

DSL Cable/HFC FTTH

% Advertised Speed

Peak vs Off-Peak Upload Speed as a percentage of Maximum Advertised Speed,

by Technology and ISP

Off-Peak

Peak

Figure 6 Peak vs Off-Peak Upload Speed as a percentage

Maximum Advertised Speed, by Technology and ISP

The information in this document is correct as of June 2020

Figures 7 to 10 below show the average peak-hour upload speed as a percentage of advertised rates for

each ISP’s package included in this report. As with download speeds, the different internet packages

have been categorized into “speed bins” for ease of presentation and comparison.

Internet packages with upload speeds between 1 and 7.5Mbps all delivered upload performance that

exceeded the maximum speeds advertised by ISPs during the busiest hours of the week. As with

download speeds, upload speeds are also subject to overprovisioning in order to ensure optimal service

during these busy hours, and the amount can vary from ISP to ISP and from tier to tier. For example,

TELUS’ DSL 25x5Mbps package exceeded maximum advertised speeds by 2 percentage points,

performing at 102%. By contrast, subscribers to the same ISP’s DSL 15x1Mbps received average upload

speeds that were in excess of 3 to 4Mbps. Northwestel and Bell MTS can also be seen to overprovision

certain tiers, such as the Cable/HFC 20x3Mbps and DSL 25x2Mbps respectively.

Upload Speed as a percentage of Advertised Speed, by ISP and Package,

1-7.5Mbps upload tiers

500%

450%

400%

350%

300%

250%

200%

150%

100%

50%

10x2*

25x2

15x2

Cogeco

20x3* 50x4* 15x1.5 75x7.5*

Bell MTS

Northwestel

Shaw

15x1* 25x5 10x1.5

TELUS Videotron

% Advertised Speed

114%

115%

111%

122%

114%

104%

105%

345%

102%

103%

Figure 7 Upload Speed as a percentage of Advertised Speed, by ISP and Package, 1-7.5Mbps upload tiers

The information in this document is correct as of June 2020

Subscribers to internet packages with maximum advertised upload speeds of 10Mbps generally received

speeds that either met or exceeded their sold rates, although results varied. For example, TELUS’ DSL

50x10Mbps plan delivered an average of 108% in upload speed during busy hours, but with a large

standard deviation (seen in the width of the confidence interval), meaning that the distribution was

uneven. Bell Canada’s DSL 25x10 and DSL 15x10Mbps packages saw especially low speeds with upload

performances of 77% and 71%. Both the 15x10Mbps and 25x10Mbps packages had fairly wide

confidence intervals, meaning that test results varied significantly across users. This is not surprising

however, given the effect of copper loop lengths (i.e. how far a customer is from the nearest central

office or node) on line performance.

Upload Speed as a percentage of Advertised Speed, by ISP and

Package, 10Mbps upload tier

140%

120%

100%

80%

60%

40%

20%

15x10 25x10 50x10 100x10 40x10 60x10 120x10 60x10* 75x10 50x10 15x10 30x10 60x10

Bell Canada Cogeco Rogers TELUS Videotron

Figure 8 Upload Speed as a percentage of Advertised Speed, by ISP and Package, 10Mbps upload tier

% Advertised Speed

71%

77%

91%

106%

107%

117%

104%

103%

105%

108%

110%

111%

The information in this document is correct as of June 2020

Internet packages with maximum advertised upload speeds between 15 and 75Mbps all delivered

performances that exceeded advertised rates. As with other tiers, overprovisioning occurs across all ISPs

to varying degrees. TELUS’ DSL 75x15Mbps and FTTH 75x75Mbps packages delivered upload speeds

much higher than the maximum advertised (at 137% and 120% respectively), as did Bell Canada’s FTTH

50x50 package at 116%.

Upload Speed as a percentage of Advertised Speed, by ISP and Package,

15-75Mbps upload tiers

160%

140%

120%

100%

80%

60%

40%

20%

50x50

Bell Canada

150x15

500x20

300x15

600x20

75x15

25x25

TELUS

75x75

Rogers

Shaw

Figure 9 Upload Speed as a percentage of Advertised Speed, by ISP and Package, 15-75Mbps upload tiers

% Advertised Speed

116%

103%

104%

105%

137%

116%

120%

The information in this document is correct as of June 2020

Maximum advertised upload speeds were also largely exceeded at the highest speed tiers, with Bell

Canada and TELUS’ FTTH 150x150Mbps packages overperforming. Bell Aliant’s FTTH 500x500Mbps

internet package averaged peak hour upload speeds just under the maximum advertised rates at 94%

during the measurement month. It should be noted that Bell Aliant identified an issue with the software

running on their optical line terminals, which resulted in their customers measuring lower upload

speeds during the initial seconds of an upload test. The issue was resolved on October 17

through a

configuration change applied to the entire base of 500x500Mbps customers, slightly more than

halfway through the testing month. In the latter half of the measurement month, after the

configuration change was applied, Bell Aliant’s FTTH 500x500Mbps package average upload speeds

were in excess of advertised speeds.

Upload Speed as a percentage of Advertised Speed, by ISP and Package,

150-500Mbps upload tiers

140%

120%

100%

80%

60%

40%

20%

500x500

Bell Aliant

150x150

300x300

150x150

300x300

Bell Canada

TELUS

Figure 10 Upload Speed as a percentage of Advertised Speed, by ISP and Package, 150-500Mbps upload tiers

% Advertised Speed

94%

114%

105%

111%

107%

The information in this document is correct as of June 2020

Figure 11 below shows average upload speed during peak and off-peak hours as a percentage of

advertised, by region. As with the regional download charts, please note that no DSL services in the

East have been included as part of this report.

DSL services had the highest upload performance in the West & North at 149% of advertised maximum

rates, whereas the Central region showed a much lower performance of 87%. Upload performance of

Cable/HFC services was instead nearly identical across the country, at 107% for the Central and West &

North regions, and 106% in the East. FTTH also had its strongest performance in the West & North at

113%, followed by the Central region at 109%, while maximum advertised speeds were not met in the

East, at 95%. Network congestion was minimal across all service technologies and regions.

Please note that Bell Aliant identified an issue with the software running on their optical line terminals,

which resulted in their customers measuring lower upload speeds during the initial seconds of an upload

test. The issue was resolved on October 17

and after the configuration change was applied, Bell Aliant’s

500x500mbps package average upload speeds were in excess of advertised speeds. This may impact on

regional performance, which was averaged over the entirety of the month.

87%

149%

108%

107%

109%

95%

113%

87%

149%

107%

106%

107%

109%

95%

113%

20%

40%

60%

80%

100%

120%

140%

160%

180%

Central West &

North

Central East West &

North

Central East West &

North

DSL Cable/HFC FTTH

% Advertised Speed

Peak vs Off-Peak Upload Speed as a percentage of Maximum Advertised Speed,

by Technology and region

Off-Peak

Peak

Figure 11 Peak vs Off-Peak Upload Speed as a percentage of

Maximum Advertised Speed, by Technology and region

The information in this document is correct as of June 2020

Table 2 below shows the advertised upload speed at peak hours for each ISP and internet package

included in this report, expressed both in terms of megabits per second as well as a percentage of

maximum advertised, along with the technology, sample count and standard deviation (in term of

percentage of advertised) for each.

Table 2 Advertised upload speed at peak hours for each ISP and internet package included in this report

ISP

Technology

Package

Average Peak

Hours Speed

(Mbps)

Average Speed (%

of Max. Adv.)

Std.

Dev.

Whitebox

Count

TELUS

DSL

15x1*

3.4

344.7%

267.3%

Shaw

Cable/HFC

15x1.5

1.6

103.9%

7.3%

Videotron

Cable/HFC

10x1.5

1.5

102.5%

10.7%

Bell MTS

DSL

10x2*

2.3

114.4%

2.2%

Bell MTS

DSL

25x2

2.3

115.3%

5.0%

Cogeco

Cable/HFC

15x2

2.2

111.4%

3.6%

Northwestel

Cable/HFC

20x3*

3.6

121.5%

5.6%

Northwestel

Cable/HFC

50x4*

4.6

114.4%

2.7%

TELUS

DSL

25x5

5.1

101.6%

43.0%

Shaw

Cable/HFC

75x7.5*

7.9

104.9%

5.0%

Bell Canada

DSL

100x10

10.6

105.9%

9.4%

Bell Canada

DSL

15x10

7.1

71.5%

36.8%

Bell Canada

DSL

25x10

7.7

76.8%

31.6%

Bell Canada

DSL

50x10

9.1

91.1%

22.5%

Cogeco

Cable/HFC

120x10

10.4

103.5%

3.1%

Cogeco

Cable/HFC

40x10

10.7

107.1%

6.7%

Cogeco

Cable/HFC

60x10

11.7

117.0%

4.0%

Rogers

Cable/HFC

60x10*

10.3

103.5%

6.9%

Rogers

Cable/HFC

75x10

10.5

104.7%

9.8%

TELUS

DSL

50x10

10.8

108.1%

42.7%

Videotron

Cable/HFC

15x10

11.0

110.4%

5.2%

Videotron

Cable/HFC

30x10

11.0

109.8%

4.4%

Videotron

Cable/HFC

60x10

11.1

110.6%

2.5%

Rogers

Cable/HFC

150x15

15.5

103.1%

10.6%

Shaw

Cable/HFC

300x15

15.8

105.0%

10.4%

110

TELUS

DSL

75x15

20.5

136.9%

8.1%

Rogers

Cable/HFC

500x20

20.7

103.7%

10.9%

Shaw

Cable/HFC

600x20

21.1

105.5%

4.1%

TELUS

FTTH

25x25

29.1

116.4%

13.2%

Bell Canada

FTTH

50x50

58.1

116.2%

4.6%

TELUS

FTTH

75x75

90.2

120.2%

6.0%

Bell Canada

FTTH

150x150

171.4

114.2%

24.0%

TELUS

FTTH

150x150

166.2

110.8%

15.9%

Bell Canada

FTTH

300x300

315.0

105.0%

14.0%

130

TELUS

FTTH

300x300

319.8

106.6%

10.9%

Bell Aliant

FTTH

500x500

472.4

94.5%

28.6%

The information in this document is correct as of June 2020

3.3

Latency

Latency is a measure of how long it takes a packet to travel between point A and point B. It is a

significant factor in internet performance, as latency is a fundamental property of the infrastructure

upon which everything else must build. If you have a high-latency link, then it does not matter how fast

your broadband connection is, as latency will always remain an impediment. Besides infrastructure

itself, latency is also dependent on geography: a user located physically closer to a content server or

exchange will have a lower latency than one located several hundred kilometers away.

While latency is unrelated to line speed itself, an increase in latency can have a detrimental effect on

how long it takes to transfer files and other internet content. Moreover, an increase in latency during

peak hours of usage is an early indicator of congestion in the network, as routers are taking longer to

receive data packets and pass them on.

The results presented below show “round-trip latency” (i.e., how long it takes for a packet of data to

travel from point A to point B, and back to point A). As latency is dependent on the distance between

the user and the target – in this case, a test server – results below have been divided into two sets to

account for test servers being located in the metropolitan areas of Halifax, Montreal, Toronto, and

Vancouver.

The chart below shows peak and off-peak latency split by access technology, including only test results

from volunteers located within a 150km radius of the cities listed above as, demographically, they

provide the most representative sample of the majority of Canadian internet users. Whiteboxes

included in the chart below represent a sample from homes connected to internet services by Bell

Canada, Bell Aliant, Cogeco, Rogers, Shaw, TELUS, and Videotron. As latency is (among other factors)

dependent on distance, users located farther away from test servers will experience higher latencies.

Figure 12 Peak vs Off-Peak Latency, by Technology

Fiber connections had the lowest average latency of 4 milliseconds, with no change shown between

peak and off-peak hours. DSL users averaged latencies of 12 milliseconds during both peak and off-peak

hours, with Cable/HFC connections showing very similar results of 13 milliseconds during off-peak hours,

which increased by a negligible 1 millisecond during peak hours. In real-world terms, all of these results

Peak vs Off-Peak Latency, by Technology

Off-Peak

Peak

Cable/HFC DSL FTTH

Latency (ms)

The information in this document is correct as of June 2020

are very good as most users would see no practical effect from a 10ms difference except in cases of

highly latency-sensitive applications such as in real-time multiplayer games.

As mentioned previously, latency results have been divided between those from users located in the

vicinity of test servers, and those outside of the 150km radius. The chart below shows latency results for

users of all ISPs in this report located outside of the metropolitan areas of Halifax, Montreal, Toronto,

and Vancouver. It should be noted once again that latency is highly dependent on distance between the

users and test servers, and therefore results may vary significantly across individual users.

A similar scenario was found in latency tests for users located over 150km away from test servers, with

fiber connections showing the lowest average latency of 12 milliseconds during peak times, followed by

DSL at 25ms and Cable/HFC closely behind at 27ms during peak hours. As with the previous chart’s

results, latency shows no real-world difference between peak and off-peak hours, and are indicative of

an infrastructure that is well provisioned to handle increased user activity in the evenings.

Peak vs Off-Peak Latency, by Technology

(Users located >150km from Test Server)

Off-Peak

Peak

Cable/HFC

DSL

FTTH

Figure 13 Peak vs Off-Peak Latency, by Technology (Users located >150km from Test Server)

Latency (ms)

The information in this document is correct as of June 2020

3.4

Webpage Loading Time

The webpage loading time test captures how long it takes for all of the elements of a webpage to be

received by the user. Unlike other tests in this report, which measure against dedicated test servers,

webpage loading time tests are conducted against actual websites commonly visited by Canadian users.

Webpage loading time is heavily influenced by many factors, including the user’s download speed and

latency (examined previously), along with the quality of the web server itself, distance between it and

the user, as well as congestion in other networks that contribute content to the website’s delivery.

Figure 14 below shows average webpage loading times for each ISP

during peak and off-peak hours.

Webpage loading times were well under 3 seconds across all ISPs, with negligible increases of 0.1 seconds

or less during peak hours of user activity. Users on Bell Aliant FTTH, Bell Canada DSL/ FTTH, Cogeco

Cable/HFC, Rogers Cable/HFC, and TELUS DSL/FTTH all recorded average loading times between 1.3 and

1.7 seconds with minor differences across service technologies, Shaw Cable/HFC and Videotron Cable/HFC

followed at 2 and 2.1 seconds. Average times for Bell MTS were marginally higher at 2.3-2.4 seconds, with

user distance to major content servers – which are typically located near major urban centers –

contributing to the small increment.

Northwestel have been excluded from the ISP breakdown for all web page loading time results as their remote location would have an

adverse impact on results compared to other ISPs.

1.6

2.3

1.6

1.5

2.0

1.5

1.3

1.4

1.6

2.4

1.6

1.7

1.6

2.0

2.1

1.6

1.3

1.4

0.0

0.5

1.0

1.5

2.0

2.5

Bell Canada

Bell MTS

Telus

Cogeco

Rogers

Shaw

Videotron

BellAliant

Bell Canada

Telus

DSL Cable/HFC FTTH

Webpage Load Time (seconds)

Peak vs Off-Peak Webpage Loading Time, by Technology and ISP

Off-Peak

Peak

Figure 14 Peak vs Off-Peak Webpage Loading Time, by

Technology and ISP

The information in this document is correct as of June 2020

Figures 15 to 17 below show the average webpage loading time at peak hours for users across separate

ISPs and internet packages. As with previous sections, these have been split and categorized according

to download speed.

Internet packages with maximum advertised download speeds in the 10 to 25Mbps range largely

recorded webpage loading times between 2 and 3 seconds. A small subset comprised of Bell Canada’s

DSL 25x10Mbps, and TELUS’ DSL 25x5 and FTTH 25x25Mbps tiers had lower webpage loading times in

the 1.4 to 1.6 second range, whilst Videotron’s Cable/HFC 10x1.5Mbps users saw a slightly higher

average instead at just over 3 seconds.

Webpage Loading Time, by ISP and Package

10-25Mbps download tiers

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0

3.1

2.8

2.1

2.3

2.2

2.3

1.6

1.4

10x2* 25x2 15x10 25x10 15x2

Cogeco

15x1.5

Shaw

15x1*

25x5

TELUS

25x25 10x1.5 15x10

Bell MTS

Bell Canada

Videotron

Webpage Load Time (seconds)

Figure 15 Webpage Loading Time, by ISP and Package, 10–25Mbps download tiers

The information in this document is correct as of June 2020

Figure 16 Webpage Loading Time, by ISP and Package, 30-75Mbps download tiers

Webpage loading times for packages in the 30 to 75Mbps download speed tiers were markedly faster,

with the vast majority averaging between 1.3 and 1.6 seconds, and only Shaw’s Cable/HFC

75x7.5Mbps package showing a higher average of 2.2 seconds.

Average webpage loading time for higher-speed internet packages in the 100 to 600Mbps maximum

advertised download range was similar to those in the 30-75Mbps range seen previously, with most

averages between 1.3 and 1.6 seconds, except for Shaw’s Cable/HFC 300x15Mbps service, which saw an

average of 2 seconds.

Webpage Loading Time, by ISP and Package

30-75Mbps download tiers

2.5

2.2

2.0

1.5

1.6

1.5

1.3

1.4

1.3

1.4

1.0

0.5

0.0

50x10 50x50 40x10 60x10 60x10* 75x10

Rogers

75x7.5*

Shaw

50x10

75x15

TELUS

75x75 30x10 60x10

Bell Canada

Cogeco

Videotron

Webpage Load Time (seconds)

Webpage Loading Time, by ISP and Product,

100-600Mbps download tiers

2.5

2.0

1.5

1.0

0.5

0.0

500x500 100x10 150x150 300x300 120x10 150x15 500x20 300x15 600x20 150x150 300x300

Bell Aliant Bell Canada Cogeco Rogers Shaw TELUS

Webpage Load Time (seconds)

1.5

1.3

1.6

1.5

1.6

2.0

1.6

1.4

Figure 17 Webpage Loading Time, by ISP and Package, 100 – 600Mbps download tiers

The information in this document is correct as of June 2020

Effect of download speed on webpage loading time

Internet users frequently wonder how much the various broadband performance metrics may affect –

positively or negatively – common tasks, such as browsing websites. Figure 20 below offers an additional

view, with peak-hour webpage loading times shown as a function of the package’s download speed (i.e.

“10” on the horizontal axis includes Bell MTS’ DSL 10x2Mbps and Videotron’s Cable/HFC 10x1.5Mbps

packages).

In the chart above, two distinct “groups” can be discerned: the leftmost side of the chart where average

loading times are 2.2 seconds and higher and comprises speed tiers up to 20Mbps download, and the

rightmost one comprising download speed tiers of 25Mbps and above, all of which average webpage

loading times of 1.7 seconds or faster.

The above shows that download speeds are indeed important when it comes to webpage loading times,

but only up to a certain point. A user on a 25Mbps internet package is unlikely to see noticeably better

loading times if they move to a 100, 300, or 500Mbps download package. There are, however, other

factors that can affect the speed of webpage loading, such as the location of hosting servers as well as

Content Delivery Networks, as well as the quality of their infrastructure.

Webpage Loading Time, by Download Speed

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0

3.0

2.5

2.2

1.7

1.5

1.6

1.5

1.6

1.4

1.6

10 15 20 25 30 40 50 60 75 100 120 150 300 500 600

Package Download Speed (Mbps)

Figure 18 Webpage Loading Time, by Download Speed

Webpage Load Time (seconds)

The information in this document is correct as of June 2020

3.5

Packet Loss

The rate of packet loss describes how likely it is that a packet of data sent from point A will not reach

point B. Packet loss is closely related to latency and is a fundamental metric in determining how

applications perform on a broadband connection. It is generally expressed as a percentage of packets

that failed to meet the target compared to total packets sent.

A high rate of packet loss will prevent many applications from working to a satisfactory level. As a rule of

thumb, a 1% packet loss average tends to indicate connection issues and may translate into user

experience problems such as webpages being unable to load, or the inability to successfully download or

upload files. A small increase in packet loss during peak hours is to be expected, as networks are busier

and congestion at even one point in a network path may lead to a packet being dropped.

Figure 19 below shows the average packet loss for each access technology. It should be noted that

packet loss tests occur concurrently with latency tests but, as packet loss is not directly correlated to

distance the way latency is, results are inclusive of all Whiteboxes regardless of distance from the test

server. Results from one ISP’s users have however been excluded, as tests may have been impacted by

a phenomenon known as “buffer bloat” that resulted in excessively high packet loss levels. Buffer bloat

can affect the results of latency/loss tests because these tests are carried out by the Whiteboxes

immediately after a speed test is concluded. If a modem has a large buffer, data from the previous

speed test could still be queued when the latency/loss test begins. If this “queueing" lasts more than

three seconds the latency/loss test times out, leading to reported packet loss.

Packet loss was low across all access technologies, averaging 0.06% for FTTH users during peak hours,

0.11% for DSL users, and 0.13% across Cable/HFC, with differences between peak and off-peak times

being negligible. These averages are well below the 1% packet loss threshold where users would notice a

degradation of services, such as slower webpage load times or interruptions of file transfers.

Peak vs Off-Peak Packet Loss, by Technology

0.16%

0.14%

0.12%

0.10%

0.08%

0.06%

0.04%

0.02%

0.00%

Off-Peak

Peak

Cable/HFC DSL FTTH

Figure 19 Peak vs Off-Peak Packet Loss, by Technology

Packet Loss (%)

0.14%

0.13%

0.10%

0.11%

0.07%

0.06%

The information in this document is correct as of June 2020

Conclusion

This report presents numerous findings about the current state of Canadian internet performance

across some of the most popular internet packages offered by ISPs. Overall, Canadian ISPs have mostly

met or exceeded maximum advertised download and upload speeds across tiers and regions. This

quality of service is consistent across Canada. Average download performance is in excess of 100% of

the maximum advertised speed across all regions and technologies.

Download speeds either met or exceeded their maximum advertised rates, with only Bell Canada’s DSL

100x10Mbps and Shaw’s Cable/HFC 300x15Mbps internet packages delivering speeds below their

advertised maximum (93% and 99% respectively) during peak hours.

The majority of ISPs delivered upload speeds that, on average, met or exceeded their maximum

advertised upload rates, with the exception of Bell Canada at 97%. Bell Canada’s average was

particularly affected by underperformance from their DSL 15x10Mbps, DSL 25x10Mbps, and DSL

50x10Mbps internet packages.

Across all regions upload speeds performed very well with two exceptions; the DSL tiers tested in Central

Canada fell just short of 100% maximum advertised upload speed as did the FTTH tiers in the East

region. It should be noted here that Bell Aliant’s 500x500 FTTH tier experienced a software issue which

was identified halfway through the reporting month, once this was addressed it tested in excess of

advertised maximum speed.

Importantly, neither download nor upload performance were found to be affected by peak-hour

network congestion to a degree that would be noticeable to users.

Other quality metrics such as latency and packet loss were similarly unaffected by peak-hour congestion

and present a picture of a broadband network that is well provisioned to cope with the demands placed

by increased user activity in the evenings.

The Measuring Broadband Canada project aims to continue providing Canadians with information on

their broadband services, to help inform and educate consumers on what the best provider and package

is for their needs.

As with any volunteer driven project, this would not be possible without the help of volunteers all over

the country hosting a Whitebox in their home. The project always needs more people to take part and if

you wish to do so then sign up here: www.measuringbroadbandcanada.com/signup.

The information in this document is correct as of June 2020

Glossary

Measurements

Download Speed: The speed at which data can be transferred from the SamKnows test server to the

user’s home, measured in megabits per second (Mbps). In this report, it is generally expressed in terms

of “measured download speed as a percentage of advertised download speed” in order to enable a like-

for-like comparison across different speed tiers.

Upload Speed: The speed at which information is transferred from the user’s home to the SamKnows

test server. Similarly to download speed, it is measured in megabits per second (Mbps) and also

expressed as a percentage of the upload speed advertised by the ISP.

Latency: the timespan it takes for a data packet to go from the user’s home to the SamKnows test

server, and back to the home. It is commonly expressed in milliseconds (ms). A shorter latency is better.

Packet Loss: is the number of data packets that are sent over the network, but fail to reach their

destination. It is measured as a percentage of packets lost to total packets sent.

Webpage Loading Time: also known as “webpage load time”. It is the time it takes for a website to fully

load all of its elements, such as text, images, sounds, and miscellaneous scripts, and is commonly

expressed in either seconds (s) or milliseconds (ms). It is determined by a myriad of factors, such as

download speed, latency, and the quality of the network of servers providing the website’s contents. A

shorter loading time is better.

Statistical Terms

Mean: the arithmetic mean, or average. It is calculated by summing all the samples in a set, and dividing

by the number of samples. This is the method used in calculations for packet loss.

Trimmed mean: also known as the truncated mean. It is calculated by removing the top and bottom

percentiles of a data set, and then evaluating the arithmetic mean of the remaining data. In download,

upload, and latency measurements for this report, the top and bottom 1% of results for an individual

Whitebox are “trimmed”, or excluded, prior to any other aggregation.

Standard Deviation: a measure of the variability of a data set. It is dependent on both the number of

samples, and the distribution of measurements. An internet package with few Whiteboxes, but whose

results are very similar will have a small standard deviation, whereas a package with a high number of

Whiteboxes but an uneven distribution of results (i.e., very high download speeds and very low ones)

may have a higher standard deviation.

Confidence Interval: when conducting a measurement experiment such as the one presented in this

report, one is using a finite number of samples to approximate the real average across an entire

population. The confidence interval is a range of values within which we are 95% certain that the “real”

average to be. It is a function of the standard deviation and the sample size.

Appendix

The information in this document is correct as of March 2020

Methodology

Whiteboxes

SamKnows deployed 3,266 Whiteboxes to Canadian consumers for the purposes of this project. The

Whitebox is a consumer-grade device that is installed in a user’s home network between their home

modem/router and their devices. The Whitebox’s core function is to measure the quality of the user’s

Internet connection. The Whitebox model sent out for this Phase of the project was the Whitebox 8.0.

The specs of this are included below:

The current generation of Whitebox (8.0) is capable of measuring 1Gbps downstream and upstream

over both TCP and UDP.

The specifications of the device are as follows:



Dual 2.4 GHz and 5GHz Wi-Fi radios, supporting 802.11a/b/g/n/ac



Dual-core 880MHz CPU



128MB RAM



16MB flash storage



4x 1Gbps LAN interfaces



1x 1Gbps WAN interfaces



USB 2.0 port



DC power (12V @ 2000mA)

The measurements are conducted autonomously by the Whitebox to a variety of destinations on the

Internet. No user interaction is required to conduct measurements; they are executed automatically

according to a test schedule.

End-user cross-traffic is monitored continuously by the Whitebox. If cross-traffic exceeds a certain

threshold then measurements are not conducted until the cross-traffic subsides. This ensures that the

Whitebox’s measurements are not distorted by end-user activity, and that the Whitebox’s measurement

traffic does not interfere with a user’s experience of the Internet.

A full description of the Whitebox and its features can be found at

https://samknows.com/technology/agents/samknows-whitebox

Measurements

The Whiteboxes run a suite of active performance measurements according to a pre-defined test

schedule. These include the following network measurements: download speed, upload speed, latency

and packet loss, as well as application measurements such as web browsing performance.

A full description of the methodology underpinning each test can be found at

https://samknows.com/technology/tests

The information in this document is correct as of March 2020

Test Destinations

The measurements carried out by the Whiteboxes are conducted against two different types of

destination servers.

Firstly, there are dedicated measurement servers. These are installed at major peering and Internet

exchanges at the following locations:

1. Montreal

2. Halifax

3. Vancouver

4. Toronto

Each server met minimum specifications set out by SamKnows. The dedicated measurement servers

are utilized for the download speed, upload speed, latency and packet loss measurements.

Secondly, there are real applications / content providers that measurements are carried out against. For

the purposes of the Measuring Broadband Canada project, this is limited to the web browsing

measurements only. The websites tested against were as follows:

1. ici.radio-canada.ca

2. cbc.ca

3. facebook.com/policies

4. google.ca

5. canada.ca

6. ebay.com

7. ca.yahoo.com

The information in this document is correct as of March 2020

Sample Plan

SamKnows constructed a sampling plan to govern the distribution of Whiteboxes amongst Canadian

Internet consumers. This sample plan was built using subscriber data provided by the ISPs, and took into

consideration which internet packages met the minimum sample size requirements outlined below:



The package had a minimum of 25,000 subscribers.



Legacy packages (those that are no longer in market and sold to customers) to be included if

they have over 10% of the ISP’s subscriber base on.



Packages with an advertised download speed of 10Mbps or less are not included.

While some providers do offer advertised speeds of 1Gbps or faster, services above 940Mbps cannot

be measured with a conventional speed test. As such, speed tiers of 1Gbps or faster are excluded

from the 2019 Measuring Broadband Canada project. Consumers should visit each of the

participating ISP’s websites for details about the availability of higher or lower speed tiers.

Table 3 Measuring Broadband Canada Sample Plan 2019

ISP

Package Download

Package

Upload

Bell Aliant**

100

Bell Aliant**

150

Bell Aliant

500

Bell Canada

Bell Canada**

Bell Canada

Bell Canada**

Bell Canada

100

Bell Canada

150

Bell Canada

300

Bell MTS*

Bell MTS

Cogeco

120

Northwestel*

Rogers**

Rogers*

Rogers

150

Rogers

500

The information in this document is correct as of March 2020

Shaw

1.5

Shaw*

7.5

Shaw

300

Shaw

600

TELUS*

TELUS**

TELUS

TELUS**

TELUS

150

TELUS

300

Videotron

1.5

Videotron

Videotron**

120

Internet packages marked with a single asterisk (*) did not meet the minimum required number of

Whiteboxes (40) but have nevertheless been included due to having at least 35 Whiteboxes and a

narrow (5%) confidence interval. Packages marked with two asterisks (**) were initially considered for

inclusion in the report, but have been subsequently excluded due to low sample size. The parties

acknowledge that a limited exception to the minimum number of subscribers per plan has been made to

address inequity between ISPs serving the same area.

Data Processing

All data included in this report was obtained via tests conducted between the 1

and 31

of October

2019. For inclusion in the final data set, a Whitebox must have:

1) Been successfully validated by the ISP (i.e., the subscribed download and upload tiers confirmed,

along with metadata such as province and postcode)

2) Have provided a minimum of 5 days of data during the reporting period. If the volunteer’s ISP or

internet package changed during the measurement period, considerations were made based on the

number of days of data obtained under each ISP/package, and sample plan requirements

3) For inclusion in peak and off-peak verticals, a minimum of 5 samples must have been provided by

each Whitebox.

Data for each Whitebox was then aggregated via a trimmed mean for download, upload, latency, and

webpage loading time metrics. This trimmed mean method excludes outlier results in the top and

bottom 1% and helps ensure a cleaner dataset that is better representative of the typical performance.

Packet loss data is instead aggregated via the arithmetic mean. Individual results from Whiteboxes are

The information in this document is correct as of March 2020

finally aggregated on the basis of ISP, internet package, and region for packages that meet the minimum

sample size requirements.

The information in this document is correct as of March 2020

Code of Conduct

CODE OF CONDUCT

MEASURING BROADBAND CANADA

2019

The Canadian Radio-television and Telecommunications Commission (the "CRTC") has established a

Canadian Broadband Measurement Project (the "Project") in collaboration with participating Canadian

ISPs ("ISPs") and SamKnows Limited ("SamKnows").

To ensure the effective and unbiased use of the data, and the integrity and validity of study results, this

Code of Conduct has been drawn up which each of the ISPs, SamKnows and the CRTC (each a

"Participant") agrees to sign.

The undersigned, as a Participant in the Project, agree with the following principles:

1. At all times during the Project, the Participants agree to act in good faith.

2. The Participants agree not to tamper with the testing infrastructure or methodology, or take any

other actions which influence the results of any test, through positive action or omission, for any

individual panelist or participating ISP.

3. The Participants acknowledge that it will not be a violation of the principle set out in paragraph

2 above for ISPs or SamKnows to:

i. Operate and manage their business, which includes modifying or improving

services delivered to any class of subscribers that may or may not include

panelists among them, provided that such actions are consistent with normal

business practices;

ii. Address service issues for individual panelists at the request of the panelists or

based on information not derived from the trial; and

iii. Advise their customer service representatives of the identity of subscribers who

are panelists in the trial so that those representatives may address service and

billing questions.

4. The Participants acknowledge that it will not be a violation of the principle set out in paragraph

2 above to monitor the tests and components of the testing architecture provided that no

impact to the CRTC data occurs. In particular, the Participants acknowledge that the ISPs may

advise SamKnows wherever a technical concern is observed in respect of an individual panelist,

so that SamKnows may contact said panelist to investigate and remedy the problem.

5. A Participant shall not use the Phase II Report to make or support performance claims in the

marketplace (including claims employing superlatives such as “best” or “fastest”) comparing its

results against those of any other Participant, ISP or of different technologies, except that a

The information in this document is correct as of March 2020

Participant may use the Phase II Report, however necessary, subject to compliance with Section

16 of the 2019 MOU and this Code of Conduct, to directly answer a question or correct any

perceived misinterpretation of the Phase II Report when responding to press or social media

inquiries or requests for comment.

6. For greater certainty, this Code of Conduct does not apply to any SamKnows measurement

panel, or any data generated from such panel, that any ISP or SamKnows operates

independently of the CRTC Broadband Canada Measurement Platform.

7. The Participants shall ensure that their employees, agents and representatives, as appropriate,

act in accordance with this Code of Conduct.

8. The ISPs shall utilize customer recruitment methods that ensure a random selection of panelists

for each speedtier and geographic area served pursuant to the Sample Plan.

9. Each press release and any other communication (including, without limitation, the Phase II

Report and any announcement published on the Measuring Broadband Canada website, the

CRTC’s, and SamKnows’ official website(s) and social media platform(s)) issued by or on behalf

of the CRTC or SamKnows that: (i) describes the Project or its objectives, or (ii) describes the

scope of Phase II testing or sampling methodologies, or (iii) references any Phase II test results,

must, in each case, prominently include the following statement: “While some providers do

offer advertised speeds of 1Gbps or faster, services above 940Mbps cannot be measured with a

conventional speed test. As such, speed tiers of 1Gbps or faster are excluded from the 2020

Measuring Broadband Canada project. Consumers should visit each of the participating ISP’s

websites for details about the availability of higher or lower speed tiers.”. Notwithstanding

anything to the contrary in the Project documentation, the initial press release announcing

Phase II of the Project and the press release announcing publication of the Phase II Report,

including any subsequent changes to the foregoing, issued by the CRTC or SamKnows are

subject to review and approval in accordance with Section 14 of this 2020 MOU.

10. Any breach of Section 9 of this Appendix A will be a material breach of the 2020 MOU and: (i)

the breaching Party shall cure any material breach within forty-eight (48) hours of becoming

aware thereof; and (ii) if SamKnows is the breaching Party, then the ISPs will be entitled to

withhold any and all payments in connection with the Project, whether under the 2020 MOU or

otherwise, until such material breach is cured.