Remembering Prof. Howard Barrows: Notes on Problem-based Learning and the Schools of the Future

hbarrowsMe: “Howard, can I ask you a more general, philosophical question? Considering all your pioneering work in Problem-based Learning, how would you imagine the school of the future?”

Howard Barrows: “To start with, there would be no subjects. There would be no isolated classes for students such as geography, chemistry, accounting, history and so on.”

Me: “How can this work? What is the point of abandoning specialised subjects?”

Howard Barrows: “It is the designed learning outcomes that drive the learning process and in real-world problems, these outcomes are interconnected. As an example, you come from design. Supposed your students need to produce a TV program, then this is not only about holding a camera and putting it on air, but it is also about casting, accounting, clearing copyright, the ethics of journalism, buying airtime, mastering technology… a whole bunch of learning issues and they are all related. “

Me: “The school of the future would offer highly-integrated projects instead of traditional classes?”

Howard Barrows: “Yes. We would offer complex interdisciplinary projects that already include all the outcomes that traditional subjects intent to convey. Students develop knowledge and skills by achieving the designed learning outcomes. By presenting outcomes in the form of real-world problems, learning becomes more meaningful and relevant to students.”

Hallmarks of Constructivist Active Learning Pedagogy

These were not Howard Barrows exact words as this was not a recorded interview. But it is a truthful account of one of our last conversations during a teacher training workshop in Hamilton, Canada many years ago. Currently, almost two decades, later, Finland started abandoning traditional school subjects in favour of a curriculum reform under the title ‘Phenomenon-Based Learning’, which has much in common with Problem-based Learning (PBL) beyond sharing the same acronym.

(1) Problem-based Learning and Phenomenon-based Learning follow both a constructivist educational philosophy which refers to the idea that knowledge and the meaning of knowledge acquisition are actively created in the learner’s mind. The focus is on students’ critical evaluation of their learning, rather than passively internalising content. (2) Learning is contextual. A real-world problem or phenomenon such as climate change is contextual and requires considering different aspects and perspectives, such as e.g., the quality of people’s lives, mathematics, geography, meteorology, politics and policies, social psychology such as changing consumer behaviour and so on. This is different from most academic problems. Less complex and practical problems such as e.g., running a cafeteria would be another example. Each type of problem-solving or investigation into phenomena requires different kinds of competencies, each problem entails a distinct set of corresponding learning outcomes.

However, not all subjects can or should be replaced, such as e.g., languages, as they are by themselves meta-contextual in nature. Other subjects such as mathematics and music will e.g., still be taught in Finland which is adopting Phenomenon-based Learning. As Prof. Pasi Sahlberg of Harvard University comments “Finland’s National Curriculum Framework is a loose common standard that steers curriculum planning at the level of the municipalities and their schools. It leaves educators freedom to find the best ways to offer good teaching and learning to all children. Therefore, practices vary from school to school and are often customised to local needs and situations.”

(3) The combination of social immersion with autonomous motivation is the key to sustainable, self-directed learning within social contexts. Given the complex nature of most problems, problem-solving is ideally conducted within cooperating teams, rather than competing groups.

(4) The tutorial group structure of PBL avoids oversimplification that easily occurs on an individual level, but it also prevents ‘groupthink’ by encouraging open inquiry and critical, diverse thinking among group members. By discouraging individual power positions, social loafing but encouraging active participation, open inquiry, and consensus based on the better argument, tutorial groups resemble an approximation to what the social philosopher Jürgen Habermas called in his early philosophy ‘ideal speech situation’, which he later concluded as Discourse Ethics. Habermas’ doctrines almost read like a PBL tutorial guide, e.g., ‘Every subject with the competence to speak and act is allowed to take part in a discourse‘ (principle of social inclusion) or ‘Everyone is allowed to question any assertion at any time’ (principle of open inquiry).  In classroom practice, real world problems appear more meaningful to students as compared to decontextualized abstract tasks and thus support their intrinsic motivation.

pbl2Image above: Outline of the core PBL process for a tutorial group. It involves the stages of the group setting, problem identification, idea generation, the identification of learning issues, self-directed learning (research), research review, solutions development and the final self-assessment of learners of their individual and social roles. Educators such as Donald Woods have diversified the process structure for numerous faculties.

(5) In constructivist educational philosophy, reality is based on multiple representations allowing for multiple and multi-faceted solutions to emerge – unlike the single ‘model answer’ in traditional education. During the research and development phase, students become aware of different conceptual and practical approaches. Final solutions are the result of reiterative rounds of research and synthesis by the team, they are not spontaneous assumptions based on somebody’s pet ideas.

When we talked about grades, a hallmark of traditional education, Howard Barrows put forward a rhetorical question “If a pilot is taking 250 passengers from Frankfurt to New York, and you ask him how he feels, what would you think if he answers ‘Like a C minus’?” If an industry representative would ask me which of my students is able to do the job – wouldn’t it be ideal if I can honestly answer ‘All of them, all of my students can perform the job well’? Besides, what does a grade tell us? For example, if a student has obtained a ‘B’ in web-design, is this because she could handle the backend well, or the graphical user interface, or the integration into a database or any combination thereof? Does a grade tell us how a student was performing as a team member, a problem-solver or as a researcher? Traditional assessment is disappointingly un-informing when we take the requirements of a 21st-century workplace as a criterion. As compared to a mere summative assessment, assessment rubrics and formative assessment procedures have undoubtedly narrowed the gap to a fairer and more efficient evaluation, but social skills and research skills are rarely part of academic appraisals.

What is a ‘Problem’ and what is its function?

My colleagues and Howard also had intense discussions about the term of a ‘problem’. Doesn’t ‘problem’ sound too negative? We rarely perceive problems as something we would wish for. Problems can be of high and low complexity and they come in many forms, shapes, and sizes. Some problems might not be considered problems per se, but challenges. A product that works perfectly well without problems may just need an update to keep up with the times, or people might want to discuss openly competing models of desired social futures. More than often, we pose general questions even when we do not face an immediate functional problem, e.g., how do we conceptualise social fairness? What makes a happy childhood? What do we wish for the future of our communities? What is our political utopia?

At the end, the terminus ‘PBL’ was simply too well established in academic discourse so that alternate proposals, such as e.g., ‘Challenge-based Learning’ never gained popularity. For now, it needs to suffice that we talk about ‘problems’ in an extended sense meaning that issues can be of quantitative as well as qualitative nature. In a Kantian understanding, problems and system design issues can represent instrumental as well as non-instrumental (pure- intrinsic) types of rationality. In each case, it is the problem or issue at hand that drives the learning process, not the lecturer or teacher. From a research perspective, theoretical frameworks can complement PBL (by e.g., integrating Critical Theory, Social Conflict Theory, minority group perspectives, Social and Ecological Sustainability, Prospect Theory etc.), depending on the decided research approach.

In the light of terminology, ‘Phenomenon-based Learning’ as in the Finish example represents a limited signifier as well. Phenomenology typically refers to the particular structure of consciousness from a subjective, first-person view. It is a good term in a sense that it emphasises the active construction of new knowledge by a learner, but it neglects one of the key principles of constructivism which is the construction of knowledge with others in social context. It is in intersubjectivity, the reciprocal exchange of perspectives, where learning takes place. Related psychologists in support of constructivist learning are Albert Bandura (Social Cognitive Theory) and Lev Vygotsky. Vygotsky contributed an interesting perspective to learning by differentiating areas where a student can learn unaided, where a student can learn with assistance and a ‘Zone of Proximal Development’ (ZPD) where learning takes place in terms of a coherent social scaffolding for knowledge creation.

Unlike traditional education, PBL questions its own assertions by promoting metacognitive skills in students’ reasoning and epistemology. A probing tutor asks for the grounds and justifications of group members’ reasoning and asks how group members truly know what they assume to know. In academia worldwide, PBL has established itself as a sound pedagogy to address the deep social, economic, ecological and technological challenges of the 21st century.

Extraordinarily great pedagogy requires extraordinary effort: What is the price to pay?

Since much of what has been claimed about constructivist learning may sound too good to be true, the question arises if I had ever experienced any drawbacks. From my experience, there are critical institutional and individual obstacles to overcome. Constructivist pedagogy requires substantial training of staff and it demands a likewise substantial commitment by the education provider to promote student-centered pedagogy. Policy implementation should be conducted across the institution based on a voluntary buy-in. It requires support by consultants and needs a clear normative endorsement by management. In summary, the implementation of constructivist learning pedagogies is a huge collaborative effort.

On teacher level, resistance to new modes of curriculum delivery is rather common. Teachers that have been delivering classes by rote learning their entire careers often feel reluctant to let go of their cosy classroom authority. Many traditional teachers are not ready to give up their role as classroom authority and to embrace the more meaningful role as a facilitator of students’ learning. They feel that PBL is depriving them of their privileged status. To them, the classroom is their private kingdom. On the other hand, each school or college has some teachers that are motivated to try out learner-centered pedagogy. There are opportunities for supporting and recognising such teachers, to offer them the opportunity to become role-models for others, to organise’best practices’ sharing sessions and to publicise student feedback to persuade those that are resistant to change. On a university level, empirical studies might convince more scientifically oriented staff.

Another issue to resolve is the topic of prior knowledge. How much of prior (mostly conceptional) knowledge should be taught to students as a professional ‘body of knowledge’before they can embark on PBL? Especially students from disadvantaged socio-economic backgrounds deserve to start on the same level as their better-off peers. Related academic discussions contributed new ideas to the qualifying issue of prior knowledge such as e.g., initial lectures on common underlying concepts, ‘lectures on demand’ (by external experts, not necessarily lecturers) or ‘flipped classrooms‘, among many others. During policy implementation, it is critical to carefully balance the dissemination of prior knowledge with classroom learning expectations and to explain to all academic staff how both aspects complement each other fairly.

As a memorable experience, I remember my very first PBL class at Temasek Polytechnic in Singapore. Long after the class ended, I had to chase students literally out of the classroom (they ran overtime and the room was booked for other students) as they were still passionately immersed in discussions. I asked myself when was the last time that I saw students truly thrive this way. International studies have confirmed PBL as a superior and more efficient pedagogy as compared to traditional teaching, but practitioners and consultants are in rare supply.

Howard Barrows died in March 2011. To me, he was one of the most influential mentors, reformers, and innovators in contemporary educational philosophy. The list of groundbreaking innovations that PBL brought to the most diverse fields of education is incredible. It ranges from establishing students’ personal responsibility for their learning to learning in structured tutorial groups, employing simulated clients (today we also use computer simulations) and basing curricula on real-world multidisciplinary issues. Most importantly, Howard Barrows’ concept of PBL contributed to empowering learners of all ages by taking an active role in knowledge construction. PBL brought back meaning to students’ learning by introducing process-based solutions development for real-world applications. To conclude with a perspective of Albert Bandura, people are not merely onlookers of their behaviour, but proactive subjects capable of creating desired futures.

 

Bibliography

Barrows, H. S. (1971). Simulated patients (programmed patients): The development and use of a new technique in medical education. Springfield, Ill: Thomas.

Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical education. New York: Springer Pub. Co.

Barrows, H. S. (1992). The tutorial process. Springfield, Ill: Southern Illinois University School of Medicine.

Barrows, H. S. (1996), Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 1996: 3–12. doi:10.1002/tl.37219966804

Barrows, H. S., & Wee, K. N. L. (2007). Principles & practice of aPBL. Singapore: Pearson Prentice Hall.

 

The Advent of Online Education (Part I)

The following entry has been inspired by my participation in ‘Effective Online Tutoring’ with Oxford University. Part I explores broadly the current trends in online education and its pedagogical implications. A PDF-version of this post is available at  The Advent of Online Education – Part 1

1. Flipped Classrooms and Blended Learning: The Pragmatist’ Approach to Empower Learners

2. Graduate, Postgraduate & Professional Online Courses: Tutor-based  Environments for Elites

3. Massive Open Online Courses (MOOCs): Educating the World?

Digital Natives

Image (Getty Images): As numbers of ‘digital natives’ increase, do education systems catch up?

Introduction: The Digital Revolution and the Promise of Student-centered Education

In online education the facilitator or tutor has to define not only the scope of what should be learned, but also has to set the conditions under which learning is facilitated. This explicit definition of presented teaching materials, intended learning outcomes and course delivery is arguably the biggest conceptual difference as compared to traditional face-to-face learning. Online environments put the traditional employment of lectures, books, grades and exams under scrutiny. Epistemologically the ‘sage on the stage’ is no more the center of knowledge-creation and dissemination in a globalized world – to students with diminishing appeal. Online delivery entails the option of choosing between synchronous and asynchronous student-classroom interaction, enabling them to learn at individual pace and style. But does online education necessarily entail a more student-centered learning experience? Which conditions need to be fulfilled for e-learning to improve the quality of education? We shall have a look at some most prominent emerging models.

1. Flipped Classrooms and Blended Learning: The Pragmatist’ Approach to Empower Learners

In a nutshell, in a flipped (or inverted) classroom traditional class-work is done at home and homework is done in class with the help of networked media technology (Mangan, 2013). The rationale behind the flipped classroom is that valuable face-to-face time should be maximized to support the development of higher cognitive skills such as application, analysis, synthesis, evaluation and problem-solving. Students acquire conceptual knowledge or ‘How To’- skills via videos at home since one-to-one learning is more effective.

In short, the ‘sage on the stage’ moves into an online video that students can watch at their convenience at home while the lecturer turns into a ‘guide on the side’ when students apply and develop new knowledge in collaborative classroom sessions. The flipped classroom encourages self-directed learning at home and collaborative problem-solving in the classroom.

A practical example: Online videos that are reviewed at home may end with a final quiz and reflections that students write down after watching. The results are sent back as online feedback to the tutor who, when the class meets face-to-face, already has obtained a good overview how well the class has grasped the understanding of ‘prior knowledge’ (Mangan, 2012). By comparison, in a traditional classroom a lecturer has little overview about which student has understood how much. Flipped classrooms create more accessible data that tutors can use for formative assessment. Formative feedback and self-directed learning can effectively boost self-esteem and motivational beliefs  (Nicol & Macfarlane-Dick, 2006).

FLIPPED

Image (above): The Flipped Classroom has become a popular trend in undergraduate education

Flipped classrooms are becoming a success-model in many schools and colleges around the world as they increase between-persons interaction. Criticism often comes from students that still prefer a ‘sage on the stage’ and the commonly voiced-out argument to ‘why should I pay for my own learning?’ Traditional lecturers may also find it hard to let go of their power-position to switch from a ‘sage on the stage’ to the ‘guide on the side’. Good responses to typical critique on the flipped classroom (for example “How do I even know if kids are watching the videos?”) can be found in a script by November and Mull (2012). The flipped classroom was pioneered by Eric Mazur at Harvard University since the 1990s (Mazur, 2009) and had originally been called ‘peer instruction’ (PI). Today the Mazur Group at Harvard offers a wide range of papers on the topic.

The flipped classroom model is very adaptable to fit virtually any subject. An extended conceptual term is ‘blended learning’ (MacDonald, 2008) which goes beyond mere ‘flipping’, and includes online tutoring and the creation of ‘Blended Learning Communities’ (Kaplan, 2002). Early introduction to such models supports students’ self-efficacy to participate in virtual online communities (Vesely et al., 2007).

Besides online videos, prior knowledge can be disseminated via online reading materials or discussion forums where students summarize their initial understanding of a topic. Freely available Open Source software such as ‘Moodle’ allows schools and colleges to set up virtual classrooms without surrendering to more costly subscription models. Social networks such as ‘Facebook’ can be set up to facilitate initial student feedback and discussion. Lecturers can e.g., prepare podcasts which can be downloaded at home or on mobile devices. Technological options are plentiful.  In-class sessions logically start with a short review and discussion of material that has been presented online followed by rich, team-based learning activities.

Assisting staff can be professors who are content experts or invited experts. Student tutors can facilitate the group learning process, giving more freedom to faculty on how to provide optimal resources. Teachers are needed as experts in their field to assist students in practical activities rather than attempting to ‘teach’ a class in a low (or no)-feedback situation.

FC

Image above: In a ‘Flipped Classroom’ online preparation is done at home (left) and collaborative, assisted teamwork is moved into the classroom (right). This ‘flip’ supports self-directed learning at home while maximizing face-to-face time in class; engaging higher cognitive and social skills.

2. Graduate and Postgraduate Online Courses: Tutor-based Environments for Elites

Most online environments realized for Higher Education (and adult professional training) offer a wide range of State-of-the-Art technologies, both synchronous and asynchronous. They include, often within a single polished interface, options such as tutor-guided discussion forums, published weekly reading materials (both mandatory and optional), various specialized online rooms to post questions, access to essential electronic libraries to conduct peer-reviewed research, institutional email, online live chat and faculty contact including student supervisors, course managers, tutors, teaching assistants, e-librarians and the IT-Helpdesk. Students are introduced to new concepts via weekly publications and they are assessed personally with qualitative and quantitative feedback from their tutor. Learning outcomes and grading criteria for each study week are published beforehand to students.

state of the art BB

Image (above): Tutor-based virtual classroom. Credit: Liverpool University/ Laureate Education. (1) Discussion Forum: Posts can be collapsed for easier viewing, (2) Weekly Learning Resources  (available as PDF and in HTML), (3) Live Chat, Circles and Communities, RSS feeds, (4) Online Library with Advanced Search-Function and Automated Referencing Management. Blackboard is a polished commercial platform for teaching, collaborative learning and training.

Online participation is mandatory as social interaction plays a central part of the educational process. Further ‘luxuries’ include automated services to order study books in a timely manner, pay tuition and library fees online and review past grades. Many institutions use the commercial platform ‘Blackboard’ as their backbone which, in its latest version, also supports synchronous group collaboration. Teaching materials include literature as well as videos and multimedia presentations, e.g., for the presentation of case-studies or simulated clients. Students are educated to become successful problem-solvers, critical researchers and sociable team-players, the pedagogical assessment dimensions in Problem-based Learning (PBL).

Needless to say that such luxury comes at a price. The heavy price-tags for graduate and post-graduate studies disadvantage students from low-income backgrounds. Students from socio-centered cultures may face adaptation issues when dealing with ‘Western style’ critical discussions and argumentation, open critique, the facilitating nature of a tutor (instead of expecting instructional teachers) and responding to individual styles of colleagues from different countries.

Tutored online classes are not indefinitely scaleable which is why typical online class-size varies typically between 10-16 students.  Large tutorial groups create distinct disadvantages (Barrows, 1992) such as

1.)    The pleasure of working on a close, personal basis is lost to both students and tutor.

2.)    Students do not get equal opportunity to voice out their ideas and adding their contributions and individual ideas into the group’s ongoing deliberations.

3.)    The tutor is unable to monitor each student’s educational growth and may not detect emerging problems early, exactly when interventions are most effective.

One way of accommodating larger classes is to split them up into smaller groups, which demands some large- group management of its own. From an investment point of view, Open Source platforms such as ‘Moodle’ have leveled the playing field for institutions intending to host more complex and interactive courses as the software does not require the payment of licensing fees. In addition, social networks often complement (as a more informal forum for study-groups) the official learning platform.

Adobe Connect example

Image above: ‘Adobe Connect’ Synchronous Real-Time Learning Platform with video conferencing, live-chat (one-to-one, one-to-many), virtual meeting & discussion rooms, document sharing and online polls

Examples for Online Tutorial Education: Laureate Education, University of Californa-Berkeley, MIT, Oxford University, Stanford University, Harvard University, University of Liverpool, Walden University, Princeton University

3. Massive Open Online Courses (MOOCs): Educating the World?

MOOCs are realized as large-scale, tutor-less virtual classrooms. The largest learning environment ever set up and counting 160,000 subscribers was the course ‘Introduction to Artificial Intelligence’ conducted by Peter Norvig and Sebastian Thrun. Learning in MOOCs is facilitated by the presentation of videos, interactive quizzes, multiple-choice questionnaires and access to teaching materials, e.g. as PDFs. Many MOOCs offer peer-to-peer (p2p) forums where students can meet fellow learners to discuss their study issues.

MOOCs represent the extreme approach of a ‘single learner only’ environment where the student is entirely on his/ her own. Due to their large scale and high number of subscribers MOOCs cannot offer individualized guidance, support or qualitative feedback on assignments. MOOCs have been criticized for supporting predominantly passive learning, conceptually replacing the ‘sage on the stage’ simply by an anonymous automated system. Attrition rates are subsequently very high (Tyler-Smith, 2006) with a vast majority not completing courses, e.g., in one of Udacity’s latest course ‘Introduction to Computer Science’ only 1 out of 10 participants completed their studies. The record-breaking ‘Introduction to Artificial Intelligence’ mentioned earlier had only a 7% completion rate (Guzidal, 2012).

Advocates argue that such courses are not a zero-sum game and are not suitable for all learners, but the low completion rates still beg the question of the system’s efficacy and extended public benefit.

The advantage of MOOCs is the efficient dissemination of content knowledge and practical skills at undergraduate level, especially for highly motivated students that aim to acquire specialized skill-sets. Operational costs on a staff-student ratio are relatively small. In order to reduce the high drop-out rates MOOC-providers have started implementing selection processes to boost completion rates. Student profiles are adjusted towards the educational program, not vice versa, rendering MOOCs unsuitable for weaker students or students from different socio-cultural background (Parrish & Linder-VanBerschot, 2010). Justin Reich (2011) asks if MOOCs decrease digital equity and disadvantage less affluent students.

‘Single learner only’ environments come at the price of focusing on content while bypassing creative problem-solving, social skills, innovative thinking, higher cognitive and meta-cognitive skills, relevance for real-world application, taking over roles and responsibilities, active research and its critical review (Kim, 2012). MOOCs require the autonomy of strong, self-directed learners as a prerequisite for studies, but intrinsically do not foster autonomy and social relations themselves.

UDACITY-COURSERA

Image (above): Typical online quiz format and status of completed challenges at Udacity (left), Syllabus presentation with video lectures, quizzes and assignment listing at Coursera (right)

Dough Holton writes conclusively that ‘ ”Connecting” learners to one another or exposing them to content may often not be sufficient to magically cause learning to happen or to cause significant changes in beliefs and practice.’ (Holton, 2012). He criticizesthe presumption of merely descriptive connectivist’ theories which do not take into account human properties such as social processes, intentionality, meaning (semantics), mental agency or phenomenological-subjective and inter-subjective experience. This is most likely the reason why MOOCs are more geared towards convergent rather than divergent problems. Typical courses offer studies in computer science, programming and software, web-development, data-analysis, algebra, statistics or accounting which are based on solitary, internal monologue and analytical intelligence. MOOC courses render less appealing for students looking for a sociable learning experience, posing questions of implicit gender- and cultural bias.

Examples for MOOCs: Khanacademy, Udacity, Coursera, FutureLearn, edX, Open2Study, XuetangX

Conclusion

In terms of improving education qualitatively on a massive scale flipped classrooms and blended courses hold the biggest promise for most schools and colleges around the world. Implementation can be achieved with moderate additional effort and investment but requires support and commitment from the institution’s management. The approach is highly enjoyable while students’ enthusiasm translates fast into positive outcomes. Blended courses pave the way for a widely employed data-driven service design.

Besides Higher Education, tutor-based online learning is making its inroads into the commercial sector and professional training. This includes for example workforce one-the-job learning (Batalla-Busquets & Pacheco-Bernal, 2013), corporate training (Vignare et al., 2010), training of SMEs (Roy, 2009) as well as government staff (McKay & Izard, 2012).  Online tutorial courses require continuing development of new course materials, the maintenance and development of a complex online platform and employment of qualified and reliable staff. Costs for set-up and maintenance are substantial.

Finally, MOOCs have justification in their own right for smaller numbers of specializing students. They are limited to courses employing more solitary learning styles. The argument that online learning systems need to be scalable seems to rest on the wrong assumption that the human mind works as a form of computing. David Gelernter notes that ‘We don’t think with our brains only. We think with our brains and bodies together. (…) You cannot “run” another mind on yours, and a third mind on that, and a fourth atop the third.” (Gelernter, 2014). Information technology is recursive, the human brain isn’t. Online learning systems need to facilitate and enhance our mind’s potential, not sacrificing it for the sake of computational efficacy. Online education, after all, is about developing and celebrating human minds and lives. The exciting debate on how to create the best online learning experience for the greatest number of students has just begun.

References

Barrows, H. S. (1992). The Tutorial Process (2nd ed., pp. 1-5). Springfield, IL: Southern Illinois School of Medicine.

Batalla-Busquets, J., & Pacheco-Bernal, C. (2013). On-the-Job E-Learning: Workers’ Attitudes and Perceptions. International Review Of Research In Open And Distance Learning, 14(1), 40-64.

Geith, C., Vignare, K., Bourquin, L. D., & Thiagarajan, D. (2010). Designing Corporate Training in Developing Economies Using Open Educational Resources. Journal Of Asynchronous Learning Networks, 14(3), 3-12.

Gelernter, D. (2014). The Grand Analogy. In: THE EDGE ANNUAL QUESTION. Retrieved from: http://www.edge.org/responses/what-scientific-idea-is-ready-for-retirement

Guzidal, M. (2012). Udacity’s CS101: Who are you talking to? Computing Education Blog. Retrieved from: computinged.wordpress.com/2012/04/20/udacitys-cs101-who-are-you-talking-to/

Holton, D. (2012). What’s the “problem” with MOOCs? Retrieved from http://edtechdev.wordpress.com/2012/05/04/whats-the-problem-with-moocs/

Kaplan, S. (2002). ‘Strategies for collaborative learning: Building eLearning and Blended Learning Communities’. Retrieved from: http://www.icohere.com/collaborativeLearning.htm

Kim, J. (2012). Playing the Role of MOOC Skeptic: 7 Concerns. Inside Higher Ed. Retrieved from: http://www.insidehighered.com/blogs/technology-and-learning/playing-role-mooc-skeptic-7-concerns

MacDonald, J. (2008). Blended learning and online tutoring [electronic book]: A good practice guide / Janet MacDonald. Aldershot, England ; Gower, c2008.

Mangan, K. (2013). Inside the Flipped Classroom. Chronicle Of Higher Education, 60(5), B18-B21.

Mazur  E (2009). Farewell, Lecture? Science 323: 50-51.

McKay, E., & Izard, J. (2012). INVESTIGATING ONLINE TRAINING IN GOVERNMENT AGENCIES: DESIGNING ADAPTIVE WEB-BASED INSTRUCTIONAL PROGRAMMES TO RESKILL THE WORKFORCE. International Journal Of Business Research, 12(3), 69-82.

Nicol, D. J. & Macfarlane-Dick, D. (2006). ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice’, Studies in Higher Education, vol. 31, no. 2, pp. 199–218.

Parrish, P., & Linder-VanBerschot, J. A. (2010). Cultural Dimensions of Learning: Addressing the Challenges of Multicultural Instruction. International Review Of Research In Open And Distance Learning, 11(2), 1-19.

Reich, J. (2011). Thoughts on Digital Equity. Harvard Graduate School of Education. Retrieved from: http://www.gse.harvard.edu/news-impact/2011/11/thoughts-on-digital-equity-justin-reich/

Roy, A. (2009). The Training Process of SMEs: What Motivates SMEs to Use E-Learning. International Journal Of Advanced Corporate Learning, 2(3), 66-73. doi:10.3991/ijac.v2i3.991

Tyler-Smith, K., (2006). ‘Early Attrition among First Time eLearners: A Review of Factors that Contribute to Drop-out, Withdrawal and Non-completion Rates of Adult Learners undertaking eLearning Programmes’, Journal of Online Learning and Teaching, vol. 2, no. 2, June. Retrieved from: http://jolt.merlot.org/Vol2_No2_TylerSmith.htm

Vesely, P., Bloom, L., Sherlock, J. (2007). ‘Key Elements of Building Online Community: Comparing Faculty and Student Perceptions’, MERLOT Journal of Online Learning and Teaching, Vol. 3, No. 3, September 2007. Retrieved from: http://jolt.merlot.org/vol3no3/vesely.htm