The Sell of a New Machine

SALAR FELT HEMMED in. It was early 2001 and he was not getting his way on consumer products. As Larry's first PM, he had helped launch Google news and been involved in the rebirth of Deja.com as Google groups. But there were always too many cooks stirring those pots.

"Everybody had strong opinions about everything," he remembers, "because the consumer product was what we all lived and breathed." I understood that perspective entirely. I kept bumping into concerned parties who wanted to rewrite my copy. Salar decided it would be nice to find some area that fewer people cared about. He settled on ads and began informally working with the ads engineers.

"Larry and Sergey had the strongest views about things on the consumer side," Salar realized. "We all knew that they were less interested in the details on the ads side." Omid, on the other hand, had a deep and abiding interest in ads. As head of sales and business development, he liked having Salar involved and suggested Salar make the arrangement official. Larry agreed and named Salar product manger for ads.

Salar did his best thinking late at night. He walked the bike-lined halls of the Googleplex after dark, thinking about how ads were sold, how they were displayed, and how they could be improved. In the cubicles around him, Matt Cutts and a handful of other engineers^* worked on maintaining the AdWords system.

Another team focused on ads optimization—a new system to predict which ads users were most likely to click. Predicting user behavior was an enormous technical challenge that required machines to learn in real time and then make educated guesses. Veterans Chad Lester, Ed Karrels, and Howard Gobioff were on that team,^† along with Noogler Eric Bauer, whose initial project had added a hundred thousand dollars a day to Google's revenue stream by replacing low-performing ads from the original system with the best-performing AdWords ads. Their leader was a redheaded Canadian by the name of Eric Veach, who had come to Google after making fur and smoke look more realistic in movies like Monsters, Inc. Eric enjoyed a challenge.

Eric and Salar bounced ideas back and forth from parallel tracks, though their common destination became clear soon enough. Google would need to build an entirely new ad system to replace AdWords. A new system based on cost per click (CPC) instead of cost per thousand impressions (CPM). Remember that with CPM pricing, an advertiser paid a set amount for each thousand times an ad was shown, regardless of how many people actually clicked on it. AdWords offered three CPMs, at ten, twelve, and fifteen dollars. The more you paid, the higher up on the page your ad appeared, which made your ad more likely to be seen and clicked by users. With CPC, an advertiser paid only when someone actually clicked on the ad, regardless of how many times it was displayed.

Advertisers loved CPC, but it scared the bejeezus out of Google executives. Netscape had offered CPC deals, guaranteeing the number of clicks their clients would get over a certain period of time. The numbers had been very large. When the clicks didn't materialize, Netscape had no choice but to keep running more and more ads. And the more ads they ran, the lower the clickthrough rate went, until every page was saturated with banners that were ignored by users. CPC could spawn a whirling, sucking spiral of death.

Salar, though, had seen a great future. He summed it up in one word: "syndication." Salar believed that to truly grow Google's revenue we needed to distribute ads on other websites. Advertisers would be unwilling to pay CPM rates to have their ads displayed across a network of sites over which they had little control. They would be much more comfortable if they paid only when people actually clicked on their ads. But if we were going to do CPC, we had to do it differently than GoTo. We had to do it better. The question was how.

The engineering director overseeing the ads team in 2001 was Ross Koningstein. Ross had a PhD in aerospace robotics and notions of his own about how the ads system should develop. He thought we could sell ads CPC, but rank them by CPM. By taking data from the logs files, we could calculate how much revenue each ad actually generated for Google, and display them accordingly. Ross told me that he presented his idea to Larry at a meeting in April 2001, but that Larry was not in the mood to hear it.

Larry only wanted incremental changes to AdWords, according to Ross, not a whole new system, even though the other engineers agreed a new system was needed and were already working to make it so. Besides, Ross said, Larry only wanted to communicate with engineers directly involved in writing the code—and with his handpicked PMs. At the meeting, Larry informed Ross that he was making Salar the lead for ads. Salar immersed himself in discussions and brainstorming with the engineering team. They shot down most of his ideas, such as selling keywords in bundles, until one night Salar experienced an epiphany. Google could assign a quality score to each ad. "Quality" would be our prediction of how likely a user was to click on an ad. If that score was factored with the amount the advertiser was willing to pay, we could rank ads by their potential for earning money for Google—their effective CPM. Everyone would benefit. The user would see more relevant ads. The advertiser would get more clicks. Google would make more revenue. It was a brilliant idea.

Ross believed he had already suggested something very similar, only to be ignored. According to Eric Veach, a number of people were arriving at the same place independently. "It's kind of an obvious idea," he claimed, though some of the things evident to Eric made my brain ache.^* AdWords already prioritized the ads by CPM. If you paid fifteen dollars, your ad appeared at the top. If you paid less, your ad was lower on the page.

"I know I proposed that at one point," Eric told me about effective CPM ranking. "I said, 'We can prove that it results in the highest expected income if you rank people by their effective CPM,' which essentially means the CPC you charge times the predictive clickthrough rate."

That one idea was worth billions and billions of dollars. So who came up with it? "I think you're going to find very little agreement on that," Eric went on. "It's true that each of us who was involved thinks we contributed. The sum of those numbers adds up to much more than one hundred percent."

But brilliant as the idea was, it wasn't perfect. Two things were still needed. The first was a method for determining the "quality" rating for each ad.

"When Salar was originally talking about multiplying by clickthrough rate," Eric explained, "what he meant was the historical clickthrough rate. If we had shown their ad a thousand times and somebody had clicked on it ten times, they would have had a one percent clickthrough rate." Eric thought that was the wrong approach. "The problem is, that kind of stuff is just, well, garbage really. You have to show a bad ad thousands and thousands and thousands of times to get any good information about how well it's doing."

"The clickthrough rate needed to be a predictive thing," Eric insisted. "It needed to be what we thought the chances were of somebody clicking on the ad given all the information we had about the query right then." That required enormous computing sophistication. And secrecy. Eric didn't want advertisers to know how Google was determining the quality of their ads, so that Google could keep refining and improving the algorithms without advertisers' trying to game the system. He insisted that Google retain complete control—that the ranking mechanism remain a black box. He knew that would frustrate and anger advertisers, but it would benefit users, who would see more relevant ads on every page.

While our competitors made trivial adjustments to their ad programs, Eric led an effort to build one of the biggest machine-learning systems in the world—just to improve ad targeting.

The second fix the new system needed was to correct a big problem that GoTo faced. GoTo displayed the price bid by each advertiser, and advertisers kept lowering their bids because they could see they were paying more than they needed to. If the high bid was twenty-five cents per click and the next bid was twenty cents per click, the top bidder was paying four cents too much. A smart advertiser would lower his bid to twenty-one cents.

Salar was intent on getting advertisers to tell us right at the start the maximum amount they were willing to pay for a click. He wanted to charge that full amount, just as Overture did, but he didn't want the advertiser to keep rebidding to lower it.

Eric showed him a solution. "The amount that you bid shouldn't be the amount that you pay," he said. He envisioned an eBay-type auction where the advertiser would pay the minimum amount necessary to win a position in the rankings. Eric had never heard of William Vickrey, the Nobel laureate who had created a "second-price auction" model; he worked out the idea himself. It just made sense to him that instead of charging as much as an advertiser was willing to pay, we should automatically lower the cost to the minimum amount required. Then advertisers would have no incentive to lower their bids, but they would have an incentive to raise them when the bids below theirs increased.

At first, Salar resisted the idea of a second-price auction, because it would confuse advertisers. They would have to trust us to lower their bids, and Salar wasn't sure they would be willing to do that.

Eric saw a fundamental difference between his approach and Salar's. "We both contributed to the design," he said, "but Salar was always looking at the product from the point of view of the advertisers, who he knew were the customers. They wanted transparency. They wanted control. I was looking out for the users."

Eric believed we should only show advertising when it was useful. He refused to put anything into the product that would weaken that principle. He and Salar went back and forth on the auction model and the need for secrecy about ad scoring. Salar was persuasive, but Eric brought an advantage to the debate about how the systems would be configured. "I was in charge of actually building them," he told me, "so as it turns out, I won most of the arguments."

It took both Eric and Salar to convince Larry and Sergey, who still hadn't signed off on offering CPC pricing, let alone on a method for implementing it. Salar explained the advantages for syndication. Eric showed how the auction model fixed GoTo's bid-lowering problem and suggested the changes might protect Google from GoTo's patent claims. Salar argued that different keywords had different values, so a fixed-price model made no sense.

Larry remained unswayed. It was only after Eric pointed out that ad quality and search quality were related—that with the changes we could control the relevance of the ads we displayed and improve the user experience—that he finally consented. As Eric put it, "Larry had always been the biggest champion of getting the best results for users. He was fine with making things tough for advertisers when we needed to."

In the end, Larry and Sergey weren't totally convinced, but they agreed to move ahead. That was not unusual for them. If you brought enough passion and logic to the fight, they'd take a chance that you might be right. Eric assumed they would rake him over the coals if it didn't turn out well, but he was glad to get the chance to move ahead and wasted no time putting together his team. He knew GoTo had a head start and was gaining momentum.