Monthly Archives: August 2011

Continuous, Real-Time, Semi-Structured Feedback Instead of Annual Reviews

In a question on Quora about how a start-up should handle performance reviews, John Lilly writes:

After believing in annual reviews for most of my career, I don't really believe in them anymore. Not timely enough, demoralizing in general (everyone thinks they're above average), and just a hell of a lot of work for everyone. This negative view of annual & traditional reviews is quite strongly supported by university research — it's just counter-productive, even though we all think we should do it.

My own view is that you need tools like Rypple to do continuous, real-time, semi-structured feedback that adds up, over the course of months or years, to a real picture of how the person is performing, and gives both the employee and managers a way to get better.

Hear, hear. In particular, the "continuous" and "real-time" parts. This ensures that feedback is given soon after whatever event prompted the feedback. I've been involved in feedback sessions that reference events that occured months and months earlier–those never end well.

###

Other links:

Bryan Caplan offers tips on how to reduce your envy of others.

A University of Chicago project on defining and exploring the nature of wisdom.

Tyler Cowen riffs on which intellectuals have true influence. He has a very high standard for what "influence" means. The upside to such a high standard is it helps prevent word inflation. "Brilliant" is an adjective that has lost meaning due to overuse. The downside to such a high standard is that people who truly do wield influence but maybe not world-changing influence can more easily forget the responsibility that comes with it. When Tyler blogs, people listen. His influence is non-trivial. He should not taking his writing lightly…and I know he doesn't! He's influential.

Steve Silberman's sober, reflective look at what he learned from Allen Ginsburg:

One of my favorite Zen koans is, "Who was Buddha's teacher?" You don't need a spiritual parent to tell you that contentment is elusive and fleeting, that every thing and every place you treasure is in the process of being transformed into something unrecognizable, and that every love affair, killer startup idea and Facebook thread eventually ends with shrugs and a funeral.

Unbundling Education: Separate Out the Grading Process

The theory behind outsourcing is that it enables specialization: you do one thing really well and let others do the rest.

Some of the more frustrating customer service experiences happen with entities where there's limited outsourcing and specialization. As I've written before, airlines do way too much. They market their brand and flight routes, they handle reservations and bookings, they maintain aircraft, they deal with luggage. More airlines should do as they do with their regional jet business: focus on something and outsource the rest. In the regional jet example, the big airlines handle reservations and ticketing and outsource the actual flying of planes.

City governments are another example. They try to manage parks, sewers, potholes, utilites, and more. Yes, a government entity, as the sole provider of police, fire, roads, and a few other things, will always be more diverse in scope than any sane for-profit corporation. But many governments still do too much beyond the core essentials, and are not able to do any one of these things very well.

One way to think about improving complex, ill-performing products, services, or experiences is to see whether there's a way to unbundle it and allow greater specialization. Arnold Kling applies this approach to improving higher education. Specifically, he thinks schools should separate the task of evaluating students' work from the task of teaching the concepts. Here's the background:

In the legacy education model, teachers combine coaching, feedback, and content delivery. By coaching I mean advice, guidance, and encouragement. Feedback includes formal grading as well as informal praise and criticism. Content delivery includes lectures and reading assignments.

Perhaps the key to radically changing education is to break up those functions.

1. The coach should be someone who knows the student well, who can relate to and motivate the student, who can recommend a good educational path, who takes account of the student's strengths and weaknesses, and who stays on top of how well the student is doing relative to the student's ability.

2. The formal feedback can come from strangers. Students can solve problems or write essays and have these graded by a separate service.

3. The content delivery should be "pulled" by the student rather than pushed by a teacher. For example, a student and a coach could agree that the student should learn statistics. The student then selects a statistics curriculum and works through it. The Khan Academy lectures on statistics are particularly good, in my opinion. But Carnegie-Mellon has a good on-line stats course, also. My guess is that, overall, there is enough content on line to obtain a world class education.

Then, Arnold writes:

A few months ago, Ben Casnocha wrote,

"Maybe 5-10% of high school high achievers should pursue higher education without attending a four year traditional college. This is the "Real Life University" option for entrepreneurial spirits. This is for folks who can learn a lot on their own, can assemble mentors and advisors to guide the process, and most of all find their creativity smothered by drudgery of school — or otherwise are on a trajectory higher than what college can offer — and therefore need an alternative path." 

His estimate of the percentage may be high, particularly in the near term. But that is the group that I wanted to aim at in my post on schools without classrooms.

Anyway, one important issue with alternative education models is interfacing with the legacy credential system. If you take a course from an alternative college, how can you get the credits to transfer to a traditional college or translate into a credible degree?

Arnold's proposed solution: A Means A.

A Means A solves the problem of credibility and comparability of grades in courses taught at different institutions of higher education. The innovation is to separate the grading process from other aspects of higher education. For any college-level course, A Means A will devise an appropriate exam and use independent professionals to grade the exam, according to transparent, standard criteria

A Means A will extend the reliable, independent grading model of the AP exam to a broad spectrum of college-level courses. However, while the AP program compels instructors to "teach to the test," A Means A will "test to what you teach." That is, A Means A will take course objectives as given by instructors. It will design and grade tests that align with the objectives of the course.

It's a great thought. And it looks like one university is actually implementing part of it.

As a business opportunity, Arnold identifies the risks with A Means A, Inc. A company that promises to accomodate the idiosyncracies and variance of different schools' curriculua will have a hard time scaling the grading process in a cost-effective way. And making the credential have currency in marketplace in the early days will be tough. So while I am not so sure of the business opportunity, I think the high level prescription of unbundling is spot on. There are probably good business opportunties along these lines for education entrepreneurs–just need to brainstorm and iterate a bit more.

What Arnold has done with his A Means A post is bring to the table very specific ideas for improving the education system–not vague griping. And he aims his provocations directly at entrepreneurs–not policy wonks or politicans. A refreshing and useful approach.

Why Might Jobs Not Come Back This Time?

Jeff Jarvis says the jobs that technology has eliminated are not coming back, and Paul Graham rightfully asks, "Why not this time?" Historically, technology eliminates jobs but also creates new ones. Eliezer Yudkowsky, in the Hacker News thread, posits five possible reasons why "this time it's different" (i.e., this time the jobs won't come back):

1) The amount of regulation and regulatory burden has increased sufficiently between then and now, that new business relationships have a much harder time replacing old destroyed business relationships. I.e., it's now more expensive to hire a new employee.

2) It's a novel change in the financial side of the economy. Like, capital-holders now try to make killings by investing in the right HFT funds instead of trying to go out and invest in new businesses. Implausible? Maybe, but finance has changed a lot recently, so the difference, if there is one, might be there.

3) The amount of re-education required to get a new job has increased enough to be a killer barrier to re-employment, whether because of employers having imposed an arbitrary demand for bachelors' degrees or because the jobs are actually more complex.

4) Ricardo's Law does allow individual regions to lose as a result of tech improvements, because if someone else beats your comparative advantage, people will want to trade with them instead of you. There's nothing ahistorical about the Rust Belt having a persistent recession within the US. This is now happening to the whole US; people no longer want to trade with us, but China is growing plenty.

5) The Thiel hypothesis: Recent technological developments are intrinsically more boring than past technological developments; people are trying to build web apps that will sell to Google for $10 million instead of trying to invent the next "microchip" or "electricity". It's not that the 21st century poses a novel obstacle to forming new business relationships to replace old ones; rather, 21st-century technologies that obsolete jobs are missing a wow factor or wealth-generating factor that previously increased demand or increased total wealth at the same time as obsoleting jobs.

Here's a useful Paul Krugman post about how technology "hollows out" the middle of job market — low-skill (janitors) and high-skill (chemists) people are okay but middle-skill workers suffer. He notes the hollowing out is now creeping into higher and higher skilled professions.

Here's a Boston Globe piece on how manufacturing in America is alive and well — we make way, way more "stuff" than China — it's just that there are fewer manufacturing jobs because of huge productivity gains.

Here's Nick Schulz and Arnold Kling on creating jobs in the New Commanding Heights of the American economy: healthcare and education. They happen to be the two most regulated and government controlled sectors. "To revitalize these sectors and revive the American job market," Nick and Arnold say, "we must open up these industries to competition and entrepreneurial reform."