Data Science Course

Data Science Course

Educational systems are under increasing pressure to reduce costs while maintaining or
improving outcomes for students. To improve educational productivity,1
In the United States, online learning alternatives are proliferating rapidly. Recent estimates
suggest that 1.5 million elementary and secondary students participated in some form of
online learning in 2010 (Wicks 2010). The term online learning can be used to refer to a

Data Science Course

wide range of programs that use the Internet to provide instructional materials and facilitate
interactions between teachers and students and in some cases among students as well.
Online learning can be fully online, with all instruction taking place through the Internet, or
online elements can be combined with face-to-face interactions in what is known as blended
learning (Horn and Staker 2010).
many school

Data Science Course
districts and states are turning to online learning.
The purpose of this report is to support educational administrators and policymakers in
becoming informed consumers of information about online learning and its potential impact
on educational productivity. The report provides foundational knowledge needed to examine
and understand the potential contributions of online learning to educational productivity,
including a conceptual framework for understanding the necessary components of rigorous
productivity analyses, drawing in particular on cost-effectiveness analysis as an accessible
method in education. Five requirements for rigorous cost-effectiveness studies are described:
1) Important design components of an intervention are specified;
2) Both costs and outcomes are measured;

Data Science Course

1 As defined in this report, productivity is a ratio between costs and outcomes that can be improved in one of three ways: by
reducing costs while maintaining outcomes, improving outcomes while maintaining costs or transforming processes in a
way that both reduces costs and improves outcomes. Any improvements in productivity are likely to require initial
investments, but successful efforts reduce costs over the long term, even after these initial investments are taken into
account.
vi
3) At least two conditions are compared;
4) Costs and outcomes are related using a single ratio for each model under study;
5) Other factors not related to the conditions being studied are controlled or held
constant.
The report also includes a review of ways that online learning might offer productivity
benefits compared with traditional place-based schooling. Unfortunately, a review of the
available research that examined the impact of online learning on educational productivity
for secondary school students was found to be lacking. No analyses were found that
rigorously measured the productivity of an online learning system relative to place-based

Data Science Course

instruction in secondary schools.2
Given the limitations of the research regarding the costs and effects of online instruction for
secondary students, the review that follows also draws on examples and research about the
use of online learning for postsecondary instruction. While there are many differences
between higher education and elementary and secondary education (e.g., age and maturity of
students), postsecondary institutions have a broader and longer history with online learning
than elementary and secondary schools. The intention is to use the literature from higher
education to illustrate concepts that may apply to emerging practices in elementary and
secondary education. Findings from the studies of higher education should be applied with
caution to secondary education, as student populations, learning contexts and financial
models are quite different across these levels of schooling.
This lack of evidence supports the call of the National
Educational Technology Plan (U.S. Department of Education 2010a) for a national initiative
to develop an ongoing research agenda dedicated to improving productivity in the education

Data Science Course

sector. The evidence summarized in this report draws on literature that addressed either costs
or effectiveness. These studies typically were limited because they did not bring the two
together in a productivity ratio and compare results with other alternatives.
While rigorously researched models are lacking, the review of the available literature
suggested nine applications of online learning that are seen as possible pathways to
improved productivity:

2 Two research reports—an audit for the Wisconsin State Legislature (Stuiber et al. 2010) and a study of the Florida Virtual
School (Florida Tax Watch Center for Educational Performance and Accountability 2007)—include data about costs and
effects. These reports suggest that online learning environments may hold significant potential for increasing educational
productivity. Both found that online learning environments produced better outcomes than face-to-face schools and at a
lower per-pupil cost than the state average. However, these conclusions must be viewed cautiously because both reports
lacked statistical controls that could have ruled out other explanations of the findings.
vii

Data Science Course

1) Broadening access in ways that dramatically reduce the cost of providing access to
quality educational resources and experiences, particularly for students in remote
locations or other situations where challenges such as low student enrollments make
the traditional school model impractical;
2) Engaging students in active learning with instructional materials and access to a
wealth of resources that can facilitate the adoption of research-based principles and
best practices from the learning sciences, an application that might improve student
outcomes without substantially increasing costs;
3) Individualizing and differentiating instruction based on student performance on
diagnostic assessments and preferred pace of learning, thereby improving the
efficiency with which students move through a learning progression;
4) Personalizing learning by building on student interests, which can result in
increased student motivation, time on task and ultimately better learning outcomes;
5) Making better use of teacher and student time by automating routine tasks and
enabling teacher time to focus on high-value activities;
6) Increasing the rate of student learning by increasing motivation and helping
students grasp concepts and demonstrate competency more efficiently;
7) Reducing school-based facilities costs by leveraging home and community spaces
in addition to traditional school buildings;
8) Reducing salary costs by transferring some educational activities to computers, by
increasing teacher-student ratios or by otherwise redesigning processes that allow for
more effective use of teacher time; and
9) Realizing opportunities for economies of scale through reuse of materials and their
large-scale distribution.
It is important to note that these pathways are not mutually exclusive, and interventions
intended to increase productivity usually involve multiple strategies to impact both the
benefit side (pathways 1–4) and cost side (pathways 5–9).
Determining whether online learning is more or less cost-effective than other alternatives
does not lend itself to a simple yes or no answer. Each of the nine pathways suggests a
viii
plausible strategy for improving educational productivity, but there is insufficient evidence
to draw any conclusions about their viability in secondary schools. Educational stakeholders
at every level need information regarding effective instructional strategies and methods for
improving educational productivity. Studies designed to inform educational decisions should
follow rigorous methodologies that account for a full range of costs, describe key
implementation characteristics and use valid estimates of student learning.
Even less is known about the impact of online learning for students with disabilities.
Regarding potential benefits, the promise of individualized and personalized instruction
suggests an ability to tailor instruction to meet the needs of students with disabilities. For
example, rich multimedia can be found on the Internet that would seem to offer ready
inspiration for meeting the unique needs of the blind or the hearing impaired. In fact,
standards for universal design are available both for the Web and for printed documents. In
addition, tutorial models that rely on independent study are well suited to students with
medical or other disabilities that prevent them from attending brick-and-mortar schools.
However, while online learning offerings should be made accessible to students with
disabilities, doing so is not necessarily cheap or easy.
Any requirement to use a technology, including an online learning program, that is
inaccessible to individuals with disabilities is considered discrimination and is prohibited by
the Americans with Disabilities Act of 1990 and Section 504 of the Rehabilitation Act of
1973, unless those individuals are provided accommodations or modifications that permit
them to receive all the educational benefits provided by the technology in an equally
effective and equally integrated manner. The degree to which programs make such
accommodations is not yet known. To address this need, the U.S. Department of Education
recently funded the Center on Online Learning and Students With Disabilities, a five-year
research effort to identify new methods for using technology to improve learning. Similarly,
research regarding the degree to which current online learning environments meet the needs
of English language learners and how technology might provide a cost-effective alternative
to traditional strategies is just emerging.
The realization of productivity improvements in education will most likely require a
transformation of conventional processes to leverage new capabilities supported by
information and communications technologies. Basic assumptions about the need for seat
time and age-based cohorts may need to be reevaluated to sharpen focus on the needs and
interests of all students as individuals. And as a rigorous evidence accumulates around
effective practices that may require institutional change, systemic incentives may be needed
to spur the adoption of efficient, effective paths to learning.

Implications for Online Learning 1. Strategies should be used to allow learners to perceive and attend to the information so that it can be transferred to working memory. Learners use their sensory systems to register the information in the form of sensations. Strategies to facilitate maximum sensation should be used. Examples include the proper location of the information on the screen, the attributes of the screen (color, graphics, size of text, etc.), the pacing of the information, and the mode of delivery (audio, visuals, animations, video). Learners must receive the information in the form of sensations before perception and processing can occur; however, they must not be overloaded with sensations, which could be counterproductive to the learning process. Non-essential sensations should be avoided to allow learners to attend to the important information. Strategies to promote perception and attention for online learning include those listed below. 10 Theory and Practice of Online Learning • Important information should be placed in the center of the screen for reading, and learners must be able to read from left to right. • Information critical for learning should be highlighted to focus learners’ attention. For example, in an online lesson, headings should be used to organize the details, and formatted to allow learners to attend to and process the information they contain. • Learners should be told why they should take the lesson, so that they can attend to the information throughout the lesson. • The difficulty level of the material must match the cognitive level of the learner, so that the learner can both attend to and relate to the material. Links to both simpler and more complicated materials can be used to accommodate learners at different knowledge levels. 2. Strategies should be used to allow learners to retrieve existing information from long-term memory to help make sense of the new information. Learners must construct a memory link between the new information and some related information already stored in long-term memory. Strategies to facilitate the use of existing schema are listed below. • Use advance organizers to activate an existing cognitive structure or to provide the information to incorporate the details of the lesson (Ausubel, 1960). A comparative advance organizer can be used to recall prior knowledge to help in processing, and an expository advance organizer can be used to help incorporate the details of the lesson (Ally, 1980). Mayer (1979) conducted a meta-analysis of advance organizer studies, and found that these strategies are effective when students are learning from text that is presented in an unfamiliar form. Since most courses contain materials that are new to learners, advance organizers should be used to provide the framework for learning. • Provide conceptual models that learners can use to retrieve existing mental models or to store the structure they will need to use to learn the details of the lesson. • Use pre-instructional questions to set expectations and to activate the learners’ existing knowledge structure. Questions presented before the lesson facilitate the recall of existing