22 June 2013

Education Technology Readiness - Preventing the Unexpected

It's the first day of a new blended learning program. You've figured out how to acquire computers for all of the students. You've chosen a really exciting online curriculum that includes adaptive learning. You've spent the summer learning how the system works, adding personal touches to the lessons and preparing to coach the students. You've gone to the classroom, made sure it has Wi-Fi coverage. Tested bandwidth and played videos. The students arrive, log into their laptops... and everything crashes.

Technology readiness is a new concern for state and district technology directors. With high-stakes assessments going on, concerns are even higher. The RTTA Assessment Consortia have collaborated on a Technology Readiness Tool that states and districts are using to survey and report on their preparedness to perform assessments. Smarter Balanced and PARCC have added Technology Readiness Calculators that help districts and schools perform capacity planning.

But there are a bunch of things that go wrong that are overlooked by conventional planning and testing. Some of them are even missed by experienced network technicians. It's not that the tools above are flawed. It's the nature of these kinds of problems that require them to be addressed in a different way. Today I hope to prevent them from biting you. The following list isn't comprehensive. But it's a good start.

Inadequate Bandwidth
This is why things work when you try them the night before, but on the first day of class, with 30 students all trying to stream video at the same time it all falls apart. It's a well-known issue and bandwidth planning is a key part of the planning tools I mentioned above. Still, this is important enough that it bears an additional mention.

EducationSuperhighway is doing a survey of actual in-classroom bandwidth and using that data to advocate for better connectivity. They use a bandwidth test and ask teachers and educators to run it periodically. While they acknowledge the weaknesses of this approach (listed below) getting a lot of samples will increase the accuracy of their reports. So, if you work in a school, please go to SchoolSpeedTest.org and run their test from time to time. Not only will it inform you but it will also contribute to nationwide advocacy for school bandwidth.

Mismeasurement of Bandwidth
Bandwidth tests like SchoolSpeedTest.org are useful tools but they can be misleading. Most internet service providers offer "burst speed" in excess of the guaranteed bandwidth. Consider a municipal ISP. Perhaps they purchase 10Gbps of bandwidth from their upstream provider and parcel it out to 100 customers at 100Mbps each. None of those customers will use all of their bandwidth all of the time. So the ISP lets them burst beyond 100Mbps, using some of their neighbors' unused bandwidth. So, if you do a bandwidth test at a favorable time, the result could be much higher than what's guaranteed by your ISP.

On the other hand, I've visited with schools who get much lower performance in their classrooms than what they pay for. In their cases, outdated networking equipment or problems with their wireless networking create a bottleneck that slows things below their purchased capacity.

Inadequate Access Point Capacity
Wireless networking bridges to wired networking through access points positioned around the building. Home networks combine the access point with the router. But commercial networks usually have one or two routers for the whole building with access points positioned strategically throughout.

Any access point has a limit to the number of computers it can serve. Consumer grade devices have a lower capacity but even commercial units can be overwhelmed if you get too many devices in the same room. Most people know that the access point's bandwidth (typically 54Mbps) is shared among all connected devices. But you can't just test the bandwidth in a room with a single computer and then divide by the expected number of computers to get available bandwidth. There's bandwidth overhead to each connection and there's a ceiling on the total number of computers that can be supported by a single access point. The max device count varies from model to model but it's always there.

Interference Between Access Points
One way to address access point capacity limits is to use more of them. But if you pack them too closely together they will interfere with each other -- thereby impairing your capacity rather than building it.

Interference From Other Devices
The 2.4Ghz band used by most Wi-Fi devices is also used by Bluetooth, some cordless phones, and many other devices. The 5Ghz band is also available for Wi-Fi but it isn't supported by as many devices and it has a shorter range. Microwave ovens also happen to be in the 2.4Ghz band. Poorly shielded units can jam all network traffic in their vicinity.

Bluetooth Keyboards
This particular case of device interference deserves special attention. Smarter Balanced supports iPads and other tablet devices as acceptable testing devices. However, we require a physical keyboard when taking assessments. Typically, people use wireless Bluetooth keyboards with iPads. As noted above, Bluetooth operates on the same frequency band as most Wi-Fi networks. It's not noticeable when three or four keyboards are in a room but when 30 get going, there can be significant interference with the WiFi network. The network won't go down, but it's bandwidth will be impaired.

Keyboards and other input devices can also interfere with each other. For example, Logitech's recommended density for their wireless keyboards and mice is far lower than a typical computer lab. Another danger is that students might mix up the keyboards among the devices. Finally, wireless devices have batteries to maintain.

Conveniently, Logitech and other manufacturers now offer wired keyboards for iPads.

Inadequate Router Capacity
In addition to bandwidth limits, routers have a number of other capacity limitations. Every open internet connection requires dedicated router capacity, even if it's idle. A lower-end router may not be able to handle more than 50 or 100 devices at a time.

Too Few Network Addresses
Routers also typically manage network address assignment. In a using the DHCP protocol, routers "lease" out addresses from their pool. Home routers typically have a pool of 100 or fewer addresses. Even commercial routers in their default configuration may not have a pool bigger than 250. Lease time is also important. A typical router configuration might have a pool of 200 addresses and give out week-long leases. In such a situation, if more than 200 devices come through the doors of the school in a week's period all of the addresses could be used up even if fewer than that number are present at any particular time.

Insufficient Power or Cooling in the Room
If you're setting up a temporary computer lab (e.g. for year-end testing) you may find that the room hasn't been wired with enough power for the number of systems you set up. Also, a desktop computer with monitor puts out about as much heat as a person. So adding 30 computers to a room meant for 30 people can double the cooling requirement. Laptops and tablets consume less power and generate less heat but the demand is still notable.

Averting Problems
Here are some ideas on how to prevent problems like the above before they happen:
  • Plan Ahead: Make sure the expected infrastructure is in place well in advance of key days (first day of school, first testing day) so that you have time to check everything out.
  • Wired is Better: Wired networking has much greater capacity and reliability than wireless. Wired input devices are naturally tethered to the corresponding device and don't require batteries.
  • Read the Specs.: Don't just test your bandwidth. Find out from your ISP how much is guaranteed. And compare your purchased capacity against your tested bandwidth. Likewise, don't just test the wireless network, look up the capacity specifications of your access points and routers and make sure they meet your needs.
  • Hire a Tech: Get a trained network technician on staff, or at least under contract, and have them do a site survey.
  • Do a Scale Test: Load a room with the expected number of devices and get that many people to exercise them all at once.
  • Identify Interference Points: Who shares your internet connection? What other rooms share an access point? What facilities share a router? What is between the access point and its intended devices? (walls, furniture, etc.) Do any of the barriers move?
  • Build In Redundancy: Install redundant devices at key places (e.g. routers and access points). If redundancy isn't affordable, have spare equipment available on-site.
  • Map Your Network and Document Your Configurations: List everything you would want to know when troubleshooting or replacing a defective item.
Despite the best planning, unexpected problems are still going to occur. In the first years of online learning and assessment they may be painfully frequent. So a final recommendation is to Handle Crises with Grace. Develop a contingency plan. Be ready with an alternative activity when the systems go down. For testing, build excess days into the testing window in case you have to cancel for a day. If we plan well, technology will be a blessing and not a burden for education.

Updated 24 June 2013 to add information about SchoolSpeedTest.org.

11 June 2013

Data Standards in Service of Learning

My friends at SETDA have published a new paper, "Transforming Data to Information in Service of Learning". It represents a movement that I favor. Historically, educational data has been used primarily for accountability purposes. But, properly reported, data can guide instruction and learning and personalize the experience. The result is significant improvement to student achievement.

The SETDA paper incorporates two models that I've used to categorize standards. The Four-Layer Framework for Data Standards divides data standards into four layers of work that build upon each other. The more recent Taxonomy of Education Standards looks at categories of data within the education sector. Shortly before I changed jobs, one of my Gates Foundation colleagues asked if I could make a chart placing existing standards efforts against these two models. I decided to do it all at once by merging the models into a matrix. Here's the result:

To understand the chart better, I recommend reading the descriptions of the two models. The Four-Layer Framework for Data Standards and the Taxonomy of Education Standards.

There's a lot of crowding in the area of student data. The standards in this area don't compete as much as it would appear. While there's some overlap, most fill complimentary roles. Details of all of these standards efforts and how they relate to each other are in the SETDA paper.

Here are links to the official websites and, in some cases, my writing related to each of the above standards.

CCSSCommon Core State StandardsBlog Post
CEDSCommon Education Data Standards
SEEDSoutheast Education Data Exchange (Digital Passport)
Ed-FiEd-Fi Alliance
EDIElectronic Data Interchange
ESBEnterprise Service Bus
GIM-CCSSGranular Identifiers and Metadata for CCSSBlog Post
IMSIMS Global Learning Consortium
IMS LTIIMS Learning Tools Interoperability
LRLearning RegistryBlog Post
LRMILearning Resource Metadata InitiativeBlog Post
OBIOpen Badge Infrastructure
PESCP20W Educational Standards Council
RESTRepresentational State Transfer
SIFSIF Association
TCAPITin Can API (AKA Experience API)

Like most everything on this blog, these models and this chart are free to reuse under a CC-BY license. I hope they're helpful to your efforts.