Cookies: A Godsend for Net Surfers, and Spies

[Ninotchka]

Cookies: A Godsend for Net Surfers, and Spies
John Schwartz New York Times Service
Wednesday, September 5, 2001

NEW YORK One day in June 1994, Lou Montulli sat down at his keyboard to fix one of the biggest problems facing the fledgling World Wide Web, and, as so often happens in the world of technology, he created another one.

At that moment in Web history, every visit to a site was like the first, with no automatic way to record that a visitor had dropped by before or viewed certain pages. Any commercial transaction would have to be handled from start to finish in one visit, and visitors would have to work their way through the same clicks again and again; it was like visiting a store where the shopkeeper had amnesia.

At 24, Mr. Montulli was the ninth employee hired by what would come to be known as Netscape Communications, and he was already a renowned "hacker" in the old sense of the word - a programmer of exceptional skill. He came up with an idea to address the problem and hammered out a five-page document describing the technology that he and co-workers would design to give the Web a memory.

The solution called for each Web site's computer to place a small file on each visitor's machine that would track what the visitor's computer did at that site. Mr. Montulli called his new technology a "persistent client state object," but he had a catchier name in mind, one from earlier days of computing.

When machines passed little bits of code back and forth for such purposes as identification, programmers called the exchanged data "magic cookies." Mr. Montulli would call his invention a "cookie."

It was a turning point in the history of computing: At a stroke, cookies changed the Web from a place of discontinuous visits into a rich environment in which to shop, to play - even, for some, to live. Cookies altered the nature of surfing the Web from a relatively anonymous activity like wandering the streets of a large city to the kind of environment where records of one's transactions, movements and even desires could be stored, sorted, mined and sold. Since then, cookies have become ubiquitous - and, for many, upsetting. A recent survey by Public Opinion Strategies, a Republican polling organization, found that 67 percent of Americans identified online privacy as their greatest concern - far more than those who identified fighting crime, at 55 percent, or constructing an anti-missile shield, the choice of 22 percent.

Whether willingly, begrudgingly or unknowingly, however, most Web users have already traded a slice of their privacy for the convenience that cookies bring to the Web. Most people accumulate cookies unknowingly; a search on the average Internet user's machine will turn up dozens, or even hundreds, of the small files.

All of the functions made possible by cookies can be performed without knowing the name of the computer user, because the anonymous, unique identifier included in the cookie is enough. But if a Web-site owner can combine that identifier with personal information, say from having visitors register with the site, then the cookie becomes a powerful mechanism for personal tracking.

Responsible Uses

Most business Web sites now use cookies, and most that do use them responsibly, privacy specialists say. But many in business fear that privacy concerns could put a further drag on the hobbled high-technology economy.

"The danger to the digital economy's longevity is not from the bursting of the dot-com bubble," said Richard Brown, chief executive of Electronic Data Systems, in a recent speech. "Those effects are minuscule compared with those inflicted by breaches of trust."

Still, cookies are not going away, said Koen Holtman, a Dutch computer scientist and privacy advocate who has fought to limit the expanding abilities of cookies. Web users "can't really live with cookies, because of user-tracking issues," he said, "but also can't live without them, because that would lose them some important functionality or reliability."

Mr. Montulli and the Netscape engineers built a few privacy precautions into cookies. They did not identify the user by name. Instead, each site issues a unique number to each visitor's computer.

Mr. Montulli said he also had considered but rejected an idea for creating a single number that a person's browser would use in all Web explorations; while convenient, it would be, he knew, a privacy nightmare. "We didn't want cookies to be used as a general tracking mechanism," he recalled.

But, he said, he also planned for cookies to be a flexible tool. "We wanted people to be able to use it for other uses" besides shopping carts, Mr. Montulli said, including "things we hadn't thought about."

By 1995, as Netscape's browser introduced millions of people to the wonders of the Web, another company had taken notice of its success and wanted in on the game. Microsoft took aim at the market for Internet browsers and servers with a concerted effort that became the focus of the federal antitrust suit against the software giant.

But when it came to keeping track of online shopping carts, Microsoft decided not to reinvent the wheel, said Michael Wallent, the head of the company's browser efforts. Microsoft's browser, Internet Explorer, largely incorporated Netscape's cookie system as a "no-brainer," Mr. Wallent said.

"I don't think anyone ever thought that cookies were anything that could be excluded in the browser and have that browser become a success in the marketplace," he said.

Like Netscape, Microsoft kept its cookies under the table. Cookies were designed to be exchanged silently, without alerting the user. Mr. Wallent said privacy was not, at the time, a central consideration, because the Web "was a very different place."

Although they were not obvious to the average computer user, cookies were quickly noticed within the technology community.

Internet Task Force

Members of the Internet Engineering Task Force, a group that evolved from the time of the Internet's predecessor, the Arpanet, to become the standards-setting body for the ever-evolving worldwide computer network, started in April 1995 to discuss cookies.

Discussions began among Internet experts about the kinds of things that Internet engineers fret over, such as ways to make the system more secure and reliable. Within the discussion, some were pressing for consideration of privacy issues.

And so in 1995 a group was formed to propose standards for cookies and their uses; it was led by David Kristol, a scientist at Bell Laboratories. He estimated that the job would take a few months.

He worked on it for nearly six years.

The work was public and was carried out largely through online postings and e-mail. The group increasingly became concerned about the ways that cookies might be used to violate consumer privacy. Mr. Holtman, the Dutch computer scientist, issued a warning to the group in December 1995 that would turn out to be prophetic.

Although cookies can only be read by the site that created them or a related site - another of Mr. Montulli's early privacy measures - Mr. Holtman realized that companies could, by agreement, place cookies across a network of related sites and that those cookies could be used to track users. "Someone is bound to try this trick," he wrote, "and it will, when discovered, generate a lot of bad publicity for the whole Web."

What Mr. Holtman did not know was that companies were already planning to exploit this wrinkle of the Web. Before long, large Internet advertising companies such as DoubleClick and Engage were displaying ads on thousands of sites, using a common cookie across the network that allowed the company to recognize a visitor. The innovation allowed these companies to rotate the ads that the user sees from site to site.

The concern of privacy advocates was that these "third-party cookies" could also be used to build a detailed profile of a Web user's habits.

If a Web surfer visited a large number of sites about AIDS treatment, for example, and if that data were tied to information that identified him - say, registration at one of the sites - an insurance company could, conceivably, collect the cookie data from an ad network and use it in a quiet decision to decline an application for a policy.

Third-party cookies were precisely the kind of tracking mechanism that Mr. Montulli had tried to prevent through his privacy measures. "That's the one 'gotcha' we had," he recalled.

By 1996, the existence of cookies and third-party cookies was becoming a hot topic in the news media and in online forums; Mr. Montulli and Netscape altered the company's browsers to distinguish third-party cookies from cookies coming directly from the site being viewed and to give consumers some control over them, allowing them to turn off all cookies or just the third-party variety.

Microsoft, too, implemented some cookie-control tools. But by default, browsers were, and still are, set to accept such cookies automatically unless the user told the software to reject them.

The Internet Engineering Task Force was pursuing a different tack, however, recommending in 1997 that browsers be set to block any cookie that did not come directly from the site being visited.

Mr. Kristol said the response from the advertising companies, which were by then well established, was: "This is terrible. This will destroy our business." Each argument caused further delay - time in which the advertising companies became more powerful and the market crystallized around the two leading browsers.

Raising Awareness

Mr. Kristol was not surprised, then, that neither Netscape nor Microsoft took to heart the recommendation that browsers block cookies unless instructed not to.

By then, Mr. Montulli said he had drifted away from the process, saying the working group had called for technical changes that companies would not comply with. "I was hoping we'd get some kind of incremental improvement" out of the working group, he said. "But what the new standard required," he said, "was that you start over."

If nothing else, the effort raised the visibility of the issues underlying cookies, Mr. Kristol said. Thanks in part to his group's work, he said, companies cannot violate consumer privacy, or even appear to, without attracting unwelcome attention.

When it comes to cookies, Mr. Montulli says that he is satisfied with the way things have worked out. Even though he does not favor the use of third-party cookies, he calls it "the best possible error," because "the only way it could be exploited is by someone who is extremely public, who is extremely large and who has a very long reach" - a company, in other words, that cannot afford a public relations fiasco, he said.

Over time, the views on cookies from privacy advocates have evolved. Richard Smith, the chief technology officer for the Privacy Foundation, a research organization in Denver, said he now believed that most cookies were benign.

"My first reaction," he said, "was 'Oh, they're terrible!' Over the last year and a half as I've looked at the Internet and how it works, it would be very difficult to have the Internet without them."