Learning Machines and the Future of Academics

Institutions will try to preserve the problem to which they are the solution.

– Shirky Principle

Learning, How does it work?

There has been progress and evolution, but the roots of our academic institutions are essentially medieval. For all the progress that has been made, for a variety of technical and social reasons, the whole system is largely hierarchical and based on lineage. Expertise was always a scarce resource and the time and investment to transfer expertise required physical proximity. While we have passed the stage where participation is solely based on exposure to Latin and Greek as a filter to participation, on several levels there is still a strong bias that filters on context and circumstance. Subtle and sublimated as that bias might be, these filters may be least obvious to those who benefit most and have the power to change anything. Consequently, we have not yet fully leveraged available human potential. The present is not evenly distributed.

I did the advanced track of the Artificial Intelligence and Machine Learning classes from Stanford in the last 10 weeks and wanted to share a few thoughts. Technology and efforts like these have the potential to change everything about how people learn.

Information wants to be free. The marginal cost of broadcasting the highest quality lectures from the best teachers on the planet is trending to zero. That is changing everything. Stanford is changing it. MIT is changing it. Khan academy is changing it. Know It, Busuu, and probably a long list of education start ups I don’t even know about are going to be changing it. There is a good chance that this transition disrupts the university system as we now know it. In every sense of the word disrupt.

The two Stanford classes had a slight overlap in topic, but they were qualitatively very different. There are plenty of reviews about the classes already. What I’m interested in is slightly meta.

How do people learn? What is the incentive? What is a measure of progress? And what can they do with the things they learn?

In particular, what is the most effective path to someone being productive in a deeply technical skill?

What Possibility…

Now, back to the Stanford classes. The contrast between the two approaches provoked some thoughts.

Sebastian Thrun started out by stating the purpose of the AI class is 1) to teach you the basics of artificial intelligence and 2) to excite you. They definitely delivered on that purpose. Sebastian and Peter Norvig split time covering an introduction to AI. The format was video lectures with embedded questions at the end of most videos. The format was the same for the lectures, the homework, the mid-term and the final. Watch the video, answer the questions. Done.

The ML class used a different format. This system was also video lectures. Andrew Ng’s presentation in the video medium felt natural and flowing. This class didn’t cover as many topics but almost every topic came with a programming assignment. Questions in the lectures were not graded, but there were weekly review questions and the programming assignment. You were allowed to resubmit the review or the assignment multiple times with no penalty, so you were graded, but getting 100% was really a measure of persistence. (Andrew seemed excited to be teaching people. The thank you he gave in the concluding lecture was so heartfelt, I wanted to give him a hug. Andrew made me feel like it was a true honor for him to teach this. The honor was all mine.)

At the end of AI, you had learned some things from watching videos and got graded for submitting a bunch of forms, at the end of ML, you had learned some things from watching videos and had the opportunity to have working code to train neural networks, support vector machines, k-means clusters, collaborative filtering, etc. On the one hand you have people tweeting their scores on the other you have people BUILDING SELF DRIVING CARS!

By three methods we may learn wisdom: first, by reflection, which is noblest; second, by imitation, which is easiest; and third, by experience, which is the most bitter.

– Confucius

Take The Next Step

Which brings me to the point I really want to make. What is an education? What are academics? The pursuit of knowledge and understanding? These things people are doing and building to help people learn are amazing and inspiring, but that’s only one part of the equation… the dissemination of knowledge, understanding and skill. What about creation?

Scientific journals which at one point served as a filter of quality and point of aggregation, now act as a barrier to access. If the internet does anything, it disintermediates. This current system of publishing slows and prevents the access to information. The ‘publish or perish’ tenure and research grant funding process also creates disincentives to open collaboration. I imagine a future where collaboration in research is open and transparent. Experiments aren’t done in secret and partially explained in publications, but all the methods and results are shared and updated in real time. Like a Github for science. If I can’t replicate results, I open an issue. If I find an interesting pattern or insight, I open a pull request. Everyone can see everything, streams of open data. This has to scare the living hell out of some people. There is a lot of time, money and personal identity tied up in the current system, but its essential inefficiencies are not beneficial or necessary.

(Aside: Resistance to this is not unlike what we are witnessing with the entertainment/media/copyright lobby that resulted in the SOPA legislation, where entrenched institutions attempt to prolong the last gasp of disrupted models of creating and capturing value. That resistance won’t fix outmoded approaches to servicing markets that no longer exist, it can only stunt the growth of emerging models. Piracy is a distraction. People always made copies and traded media, just the medium has changed. People have also never had a problem trading for something they value. People love to buy stuff they love. Compete in the market. Embrace the opportunities.)

Finally, there would be a benefit to more permeability between academics and industry. There are literally billions, maybe even trillions of dollars worth of technology shelved in universities. Industry loses on the opportunity to greater utilize research and expertise while academics often lose touch with the reality of practice in the wild. We all lose on the prospect of more abundant prosperity. In most cases there is a risk and implied disincentives to transition between the two disjoint worlds, which in some sense don’t even respect each other’s reality. If the system facilitated a properly incentivized flow of people and information in both directions, I can’t help but believe both would be better off.

The open questions now are how quickly the transitions happen and to what extent to those personally attached to the status quo resist. Same story, different stage.

tl;dr We live in amazing times. You can either understand how to build self driving cars or you can’t. You will either help others do it, or you won’t. Get ready for the next level or better, help make it happen. Special thanks to Stanford, Andrew Ng, Sebastian Thrun and Peter Norvig for their contributions to the future.

6 responses to “Learning Machines and the Future of Academics

  • Roshan Choxi

    education startup here (www.trybloc.com)

    definitely agree that the cost of educational content online is approaching zero, but there’s so much more that can be done. i think online video lectures is a start, but the future of education involves constant feedback, learning at your own pace, and having a sandboxed virtual environment to play with new concepts. lot’s of great stuff on the horizon, we’re driving some of it at Bloc specifically for programming.

    • stochasticresonance

      I essentially agree with all of your points. I had written more comparison of the two classes and outlined some of this there, especially the pacing, but decided to leave it out for brevity and focus.

      I just did a lesson at trybloc. Now I’m curious about where you are going with the long term strategy.

  • Mike McGee

    I agree that the university system is a broken model, but I disagree that online is the best solution. Yes information wants to break from behind the ivy walls, however when it comes to programming, video lectures, tutorials, and books. This is not saying that online is not effective (yes people can learn), but from a beginner’s perspective, it is still difficult to learn by using online resources and doing it by yourself. My friend and I spent a year trying to learn by reading books, doing tutorials and 3-4 day workshops, but that was not enough (and it was not fun).

    That’s why we decided to start Code Academy (http://codeacademy.org) to create the best environment for people to learn how to build web applications. Being in an immersive environment learning with people who are just as (if not more) passionate than you to learn how to code is a great way to break through barriers faced by doing it yourself.

  • stochasticresonance

    People learn in different ways and ‘best’ is only useful in relative comparison. I have learned many things successfully when peers have struggle with the exposure to the same methods and material, and I have struggled with skills that other seemed to acquire more readily.

    I believe there is something to be said for immersion and proximity, and while I also believe that we are capable of providing more of that through technology online, I wouldn’t have it any way other than the whole group onsite for a project I’m going to get funded (http://angel.co/wu-tang-combinator it’s a little abstract there, but if you want info send me email.) From my perspective, the difference is about bandwidth and feedback cycles.

    In my experience with the AI and ML classes, there were times I thought it would have been nice to ask Andrew, Sebastian or Peter clarifying questions or explore some more theoretical aspect of the material. There’s no technical barrier to that happening, the limitation was really about their time, and for the price I paid, they had already given me plenty.

    There are other skills that I think will probably always require some proximate exposure to experts. Things like martial arts, dance and other physical skills come to mind, but here I think really about what it takes to get the proper feedback, like being physically put into position or punched in the face for example.

    • Mike McGee

      True people learn in different ways, but in the five months we have been doing Code Academy (and the year we spent learning by ourselves) the same patterns show up, at least for beginners like us. Having a physical environment, not only where you can ask questions to an instructor and your peers, but to just be immersed in learning is something you can’t do as well online.

      As for saying we are the best, you have a point on the relativity of that. However, we have had a lot of people who have tried all the major online resources and still want to do our program. Whether it is Codecademy, Treehouse, Lynda, OpenCoursewave, etc. We are an opinionated startup, and we haven’t seen anything yet that does what we do for who we are targeting. There are a few starting up in January, so we will see. I hope they do really well! Seriously, we need help in successfully training people, so the more people who decide to do it this way the better.

      And to your last point, I think programming goes up there with the likes of martial arts, dance and other physical skills. As much as programming takes place behind a computer screen, people underestimate how much of programming happens by communicating with other people, in-person. Pair programming is huge in Ruby on Rails, and having the soft skills of teamwork and communication is just as important as knowing how to program.

  • Brian Coyle

    I’ve got no bone to pick in the start-up arena. But stumbling on this post, I find it curiously naive. First, information does not want to be anything. These statements mask reality, and I think most people secretly know it. Information has value. A teacher who can impart information to a lot of people has value. The system that exposes them through the internet has value. The service Stanford and others provide is free, but it’s a public good, the spun-off benefit of institutional infrastracture, altruistic professors, and other sunk costs.

    The university benefits, without getting paid. Stanford, MIT, Berkeley, Harvard, and other institutions webcasting lectures do so selectively. The best, brightest, personable professors get displayed. Cutting-edge, stimulating subjects are offered. We love it, because quality is great. But Stanford benefits, because quality builds reputation.

    Its easy to wax idealistic about the model’s future. The best teachers offered to the whole world is a wonderful prospect. But your other examples, regarding the entertainment industry’s battle with consumers. as well as a previous decade’s internet business model failure, suggests the current university webcasts are a phase. Grab it while possible.

    True, hardly any entertainment firms offer truly free product. But neither do university webcasts; both early TV and web lectures often appear with ads. Still, entertainment co’s were contrarians from the digital era’s start, because many already had a business model with tiered pricing. Theater prices cost most, then hard media copies, then cable, then broadcast. The last offers viewers a pseudo-free experience (as does this blog, which has ads). Although perhaps not trendy, a tiered system helps maximize both potential audiences and the product maker’s revenue.

    Broadcast content quality deteriorated after its first decade or two. Broadcast customers were less specific, less young, less wealthy and/or less educated, compared to cable, media, and theater. Talent migrated upscale. Free broadcast tv has its moments, but it’s mostly bad. Yet this could be the default model of future universities.

    When online media piracy hit, it wasn’t like trading VHS tapes or recording someone else’s CD. The difference wasn’t incremental, it was exponential. Although techies and professors can view it as creative destruction (another term that conceals & reveals), the continuing impact on businesses, creativity, and careers is needlessly damaging. It’s anarchy, which does often occur as a new technology is massively exploited (think oil in the late 19th century). History rarely views such epochs kindly, because they waste human capital and benefit expropriators most.

    Content is most important, whether entertainment or a university lecture. Without quality content that people want, the medium is not the message. We’re beguiled by Stanford’s offerings, because of such high quality content. Media benefit the masses, as they open windows to content. But quality content does not want to be free.

    Why are the best university’s the ones offering the most online lectures? Why don’t mid-level colleges compete? This is a fascinating question, because first movers can have huge advantages. Further, disruptive innovation can offer small inefficient upstarts an advantage. I guess college administrators have anxiety, watching product stream into cyberspace. It could cannibalise adult learning markets some access. But that seems minor. Unlike entertainment companies, they don’t have an existing multi-tiered price model to defend.

    When biggest brands are the innovators, its due to necessary high cost technology, very dispersed component innovations that need aggregation, or a small market (like just NASA). Yet online lectures don’t seem a impossible cost, their components — professors — already aggregate on campus. Its the market that must matter.

    Instead of assuming no income gets generated by Stanford’s lecture, maybe its a two-step sale. Indian, Malay, South African, Polish, Russian, and American ghetto youth may know Stanford exists, but not its value. Universities face a future of diminishing profits if they don’t expand their undergrad customer base. Should inflation kick in, the relative cost of college could be more affordable to foreign students. How can a university reach this potential market? Webcasting the best professors it has to the world seems almost obvious.

    Universities will gradually learn what the benefits and costs really are. If this experiment is a marketing one, they will eventually target a narrower offering that hits who and where they want. There will be costs discovered as well, other colleges may poach lecture notes or even scripts, or online colleges rebroadcast as part of a certificated program.

    So grab it while you can.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: