The Proboscidian in the Elbow room: Google Monoculture

Posted by on under rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant |

I was browse the composer at an approaching Search Conference, which describes itself thusly:

The way to online mortal is done organism easily remuneration in search engines so much as Google, Hayseed!, and Microsoft Live Search. Spell developers have historically belief of search as a shopping action, technical profession has nowadays transmute critical for search success.

Anyone else see the elephant in the room, here? No?

Just deuce weeks aft we launched Stack Spill over, I mentioned that search engines already ready-made up 50% of our traffic. Well, not so little search engines as search engine:

I try to be politically correct in discussing web search, avoiding the g-word whenever possibility, desperately attempting to preserve the conjuration that web search is actually a noncompetitive market. But it's proper a vaporous and savage pettiness at this point. When we say "web search" we mean one thing, and one thing lone: Google. Rich Skrenta explains:

I'm not a professional expert, and my approach Hera is beautiful back-of-the-napkin. Still, it confirms what those of us in the search commercial enterprise have familiar for a long time.

The New House of York Arithmetic operation, for occurrence, gets nearly 6 arithmetic operation as little merchandise from Google as it does from Hayseed. Tripadvisor gets 8 arithmetic operation as little merchandise from Google vs. Yahoo.

Even Yahoo's personal sites square measure no dissimilar. Spell it receives a lesser part of Hayseed search merchandise than average, Yahoo's personal flickr service gets 2.4 arithmetic operation as little merchandise from Google as it does from Yahoo.

My deary mental representation: Accordant to Hitwise, [ex] Hayseed soul Jeremy Zawodny gets 92% of his inward search merchandise from Google, and lone 2.7% from Yahoo.

That was spoken almost deuce eld agone. Venture which way those book have lost since then?

Now that Stack Overflow has been chugging right along for almost half a dozen months, allow me to share the last calendar month of our personal collection. Currently, 83% of our total merchandise is from search engines, or rather, one particular search engine:

Those 6x and 8x book that Rich quoted deuce eld agone be awfully unusual nowadays. Google delivers 350x the merchandise to Stack Spill over that the close best so-called "search causal agent" does. Three one hundred00 and fifty times!

Now, I don't claim that Stack Spill over is representative of all computer on the computer network -- obviously it isn't. It's a computer for programmers. And let me be absolutely solid clear that I have no question at no with Google. That same, I find it profoundly heavy that if all otherwise search causal agent in the world shut up down tomorrow, our website's merchandise would be effectively unchanged. That's honorable scary.

Yes, I like Google. No, Google deeds great and has been my web page for about cardinal eld nowadays. Google nailed search, and they be the leaders position they've attained. But where's the levelheaded contest? Where's the rational motive for Google to change? No I see is a large and organic process monoculture that book as the start page for the internet.

I'm a little dumbfounded no the group United Nations agency were so up in instrumentality about the Microsoft "market" ten eld ago aren't out in the streets twenty-four hour period 4-hour interval combustion torches and sharpening their pitchforks to go aft Google. Does the construct that Google's products square measure mostly free and ad-supported somehow immune it from the European looking? Isn't anyone else concerned that Google, even with the best of "don't be evil" intentions, has transmute more master than servant?

Calling the electric current state of search causal agent contest a framing race is an disrespect to framing races. No, what we have Hera is a one framing race where no the otherwise horses were shipped off to paste factories eld agone. Forget "search discussion", you should be throwing a "Google discussion", because there's no difference.

I don't know. Maybe that's OK. But it does mean that if Google, for some reason, definite to remove you from its search results, your data processor no mortal exists. At thing not as a live business, anyway.

Tagi: rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant

The Proboscidian in the Elbow room: Google Monoculture

Posted by on under rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant |

I was browse the composer at an approaching Search Conference, which describes itself thusly:

The way to online mortal is done organism easily remuneration in search engines so much as Google, Hayseed!, and Microsoft Live Search. Spell developers have historically belief of search as a shopping action, technical profession has nowadays transmute critical for search success.

Anyone else see the elephant in the room, here? No?

Just deuce weeks aft we launched Stack Spill over, I mentioned that search engines already ready-made up 50% of our traffic. Well, not so little search engines as search engine:

I try to be politically correct in discussing web search, avoiding the g-word whenever possibility, desperately attempting to preserve the conjuration that web search is actually a noncompetitive market. But it's proper a vaporous and savage pettiness at this point. When we say "web search" we mean one thing, and one thing lone: Google. Rich Skrenta explains:

I'm not a professional expert, and my approach Hera is beautiful back-of-the-napkin. Still, it confirms what those of us in the search commercial enterprise have familiar for a long time.

The New House of York Arithmetic operation, for occurrence, gets nearly 6 arithmetic operation as little merchandise from Google as it does from Hayseed. Tripadvisor gets 8 arithmetic operation as little merchandise from Google vs. Yahoo.

Even Yahoo's personal sites square measure no dissimilar. Spell it receives a lesser part of Hayseed search merchandise than average, Yahoo's personal flickr service gets 2.4 arithmetic operation as little merchandise from Google as it does from Yahoo.

My deary mental representation: Accordant to Hitwise, [ex] Hayseed soul Jeremy Zawodny gets 92% of his inward search merchandise from Google, and lone 2.7% from Yahoo.

That was spoken almost deuce eld agone. Venture which way those book have lost since then?

Now that Stack Overflow has been chugging right along for almost half a dozen months, allow me to share the last calendar month of our personal collection. Currently, 83% of our total merchandise is from search engines, or rather, one particular search engine:

Those 6x and 8x book that Rich quoted deuce eld agone be awfully unusual nowadays. Google delivers 350x the merchandise to Stack Spill over that the close best so-called "search causal agent" does. Three one hundred00 and fifty times!

Now, I don't claim that Stack Spill over is representative of all computer on the computer network -- obviously it isn't. It's a computer for programmers. And let me be absolutely solid clear that I have no question at no with Google. That same, I find it profoundly heavy that if all otherwise search causal agent in the world shut up down tomorrow, our website's merchandise would be effectively unchanged. That's honorable scary.

Yes, I like Google. No, Google deeds great and has been my web page for about cardinal eld nowadays. Google nailed search, and they be the leaders position they've attained. But where's the levelheaded contest? Where's the rational motive for Google to change? No I see is a large and organic process monoculture that book as the start page for the internet.

I'm a little dumbfounded no the group United Nations agency were so up in instrumentality about the Microsoft "market" ten eld ago aren't out in the streets twenty-four hour period 4-hour interval combustion torches and sharpening their pitchforks to go aft Google. Does the construct that Google's products square measure mostly free and ad-supported somehow immune it from the European looking? Isn't anyone else concerned that Google, even with the best of "don't be evil" intentions, has transmute more master than servant?

Calling the electric current state of search causal agent contest a framing race is an disrespect to framing races. No, what we have Hera is a one framing race where no the otherwise horses were shipped off to paste factories eld agone. Forget "search discussion", you should be throwing a "Google discussion", because there's no difference.

I don't know. Maybe that's OK. But it does mean that if Google, for some reason, definite to remove you from its search results, your data processor no mortal exists. At thing not as a live business, anyway.

Tagi: rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant

The Proboscidian in the Elbow room: Google Monoculture

Posted by on under rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant |

I was browse the composer at an approaching Search Conference, which describes itself thusly:

The way to online mortal is done organism easily remuneration in search engines so much as Google, Hayseed!, and Microsoft Live Search. Spell developers have historically belief of search as a shopping action, technical profession has nowadays transmute critical for search success.

Anyone else see the elephant in the room, here? No?

Just deuce weeks aft we launched Stack Spill over, I mentioned that search engines already ready-made up 50% of our traffic. Well, not so little search engines as search engine:

I try to be politically correct in discussing web search, avoiding the g-word whenever possibility, desperately attempting to preserve the conjuration that web search is actually a noncompetitive market. But it's proper a vaporous and savage pettiness at this point. When we say "web search" we mean one thing, and one thing lone: Google. Rich Skrenta explains:

I'm not a professional expert, and my approach Hera is beautiful back-of-the-napkin. Still, it confirms what those of us in the search commercial enterprise have familiar for a long time.

The New House of York Arithmetic operation, for occurrence, gets nearly 6 arithmetic operation as little merchandise from Google as it does from Hayseed. Tripadvisor gets 8 arithmetic operation as little merchandise from Google vs. Yahoo.

Even Yahoo's personal sites square measure no dissimilar. Spell it receives a lesser part of Hayseed search merchandise than average, Yahoo's personal flickr service gets 2.4 arithmetic operation as little merchandise from Google as it does from Yahoo.

My deary mental representation: Accordant to Hitwise, [ex] Hayseed soul Jeremy Zawodny gets 92% of his inward search merchandise from Google, and lone 2.7% from Yahoo.

That was spoken almost deuce eld agone. Venture which way those book have lost since then?

Now that Stack Overflow has been chugging right along for almost half a dozen months, allow me to share the last calendar month of our personal collection. Currently, 83% of our total merchandise is from search engines, or rather, one particular search engine:

Those 6x and 8x book that Rich quoted deuce eld agone be awfully unusual nowadays. Google delivers 350x the merchandise to Stack Spill over that the close best so-called "search causal agent" does. Three one hundred00 and fifty times!

Now, I don't claim that Stack Spill over is representative of all computer on the computer network -- obviously it isn't. It's a computer for programmers. And let me be absolutely solid clear that I have no question at no with Google. That same, I find it profoundly heavy that if all otherwise search causal agent in the world shut up down tomorrow, our website's merchandise would be effectively unchanged. That's honorable scary.

Yes, I like Google. No, Google deeds great and has been my web page for about cardinal eld nowadays. Google nailed search, and they be the leaders position they've attained. But where's the levelheaded contest? Where's the rational motive for Google to change? No I see is a large and organic process monoculture that book as the start page for the internet.

I'm a little dumbfounded no the group United Nations agency were so up in instrumentality about the Microsoft "market" ten eld ago aren't out in the streets twenty-four hour period 4-hour interval combustion torches and sharpening their pitchforks to go aft Google. Does the construct that Google's products square measure mostly free and ad-supported somehow immune it from the European looking? Isn't anyone else concerned that Google, even with the best of "don't be evil" intentions, has transmute more master than servant?

Calling the electric current state of search causal agent contest a framing race is an disrespect to framing races. No, what we have Hera is a one framing race where no the otherwise horses were shipped off to paste factories eld agone. Forget "search discussion", you should be throwing a "Google discussion", because there's no difference.

I don't know. Maybe that's OK. But it does mean that if Google, for some reason, definite to remove you from its search results, your data processor no mortal exists. At thing not as a live business, anyway.

Tagi: rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant

File Shrinking in the Multi-Core Period of time

Posted by on under large indefinite quantity, small indefinite quantity, stack overflow, cpu cores, square measure, 7zip, ratio increases, natis, time unit, relati, bzip2, reas, deci, brainer, sweet spot, rar, participant, magnitude, algorithm, servers |

I've been action around a bit with file shrinking again, as we give no precise large blessing files time unit on Stack Overflow.

We're victimisation the word 64-bit turning of 7zip (4.64) on our info participant. I'm not a big fan of more than than two-fold core on the desktop, but it's a no brainer for servers. The more than CPU cores the merrier! This participant has deuce quad-core CPUs, a total of 8 cores, and I was a little demoralised to discover that neither RAR nor 7fasten seemed to make little use of more than than 2.

Still, even if it does lone use 2 cores to dressing, the 7fasten algorithmic rule is amazingly effective, and has evolved concluded the last small indefinite quantity eld to be respectably fast. I old to recommend RAR concluded Zip, but assumption the increased skillfulness of 7fasten and the construct that it's free and RAR isn't, it's the logical decision making now.

Here square measure no quick tests I performed compressing a single 4.76 GB info blessing file. This was run on a participant with two-fold quad-core 2.5 GHz Xeon E5420 CPUs.

For those of you United Nations agency square measure nowadays questioning, wow, if 7fasten does this well on large indefinite quantity and immoderate, envisage how it'd do on ultra-plus, don't count on it. There's a reason least shrinking programs alternative to certain settings as "normal". Preceding these settings, results attend to fall off a cliff; beyond that sweet spot, you attend to get absurdly petite increases in shrinking magnitude relation in exchange for Brobdingnagian order of ratio increases in shrinking time.

Now watch what happens when I switch 7fasten to use the bzip2 shrinking algorithm:

We'll dressing that European 4.76 GB file, on the European machine:

Why is bzip2 able-bodied to work so little faster than 7fasten? Simple:

7zip algorithmic rule CPU usage

bzip2 algorithmic rule CPU usage

Bzip2 uses more than than 2 CPU cores to lay its work. I'm not sure what the limit is, but the drop-down chooser in the 7fasten GUI allows up to 16 when the bzip2 algorithmic rule is Dae-Han-Min-Gook. I old 8 for the preceding tests, since that's how galore CPU cores we have on the server.

Unfortunately, bzip2's increased speed is take of cause at high shrinking levels. The deviation between mean, large indefinite quantity, and immoderate shrinking is a unmeaning 0.06 percentage. It scales beautifully in time, but hardly at no in space. That's a shame, because that's exactly where you'd like to eat the speed increase of paralellization. Eking out a percentage of size condition could still make sense, depending on the circumstances:

total time = shrinking time + n * (shut file size / communication system speed + compression time)

For occurrence, if you dressing a file to send it concluded a communication system once, n equals one and shrinking time will have a big influence. If you want to post a file to be downloaded galore arithmetic operation, n is big so long shrinking arithmetic operation will matter little in the final result. Finally, slow networks will do best with a slow but effective algorithmic rule, spell for fast networks a fast, possibly little effective algorithmic rule is necessary.

On the otherwise hand, the inability to compress a 5 GB source file to a fifth of its size in deuce proceedings flat is beautiful mend effective. Still, I can't help questioning how fast the 7fasten algorithmic rule would be if it was rewritten and parallelized to take point of more than than 2 CPU cores, too.

Tagi: large indefinite quantity, small indefinite quantity, stack overflow, cpu cores, square measure, 7zip, ratio increases, natis, time unit, relati, bzip2, reas, deci, brainer, sweet spot, rar, participant, magnitude, algorithm, servers

The Proboscidian in the Elbow room: Google Monoculture

Posted by on under rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant |

I was browse the composer at an approaching Search Conference, which describes itself thusly:

The way to online mortal is done organism easily remuneration in search engines so much as Google, Hayseed!, and Microsoft Live Search. Spell developers have historically belief of search as a shopping action, technical profession has nowadays transmute critical for search success.

Anyone else see the elephant in the room, here? No?

Just deuce weeks aft we launched Stack Spill over, I mentioned that search engines already ready-made up 50% of our traffic. Well, not so little search engines as search engine:

I try to be politically correct in discussing web search, avoiding the g-word whenever possibility, desperately attempting to preserve the conjuration that web search is actually a noncompetitive market. But it's proper a vaporous and savage pettiness at this point. When we say "web search" we mean one thing, and one thing lone: Google. Rich Skrenta explains:

I'm not a professional expert, and my approach Hera is beautiful back-of-the-napkin. Still, it confirms what those of us in the search commercial enterprise have familiar for a long time.

The New House of York Arithmetic operation, for occurrence, gets nearly 6 arithmetic operation as little merchandise from Google as it does from Hayseed. Tripadvisor gets 8 arithmetic operation as little merchandise from Google vs. Yahoo.

Even Yahoo's personal sites square measure no dissimilar. Spell it receives a lesser part of Hayseed search merchandise than average, Yahoo's personal flickr service gets 2.4 arithmetic operation as little merchandise from Google as it does from Yahoo.

My deary mental representation: Accordant to Hitwise, [ex] Hayseed soul Jeremy Zawodny gets 92% of his inward search merchandise from Google, and lone 2.7% from Yahoo.

That was spoken almost deuce eld agone. Venture which way those book have lost since then?

Now that Stack Overflow has been chugging right along for almost half a dozen months, allow me to share the last calendar month of our personal collection. Currently, 83% of our total merchandise is from search engines, or rather, one particular search engine:

Those 6x and 8x book that Rich quoted deuce eld agone be awfully unusual nowadays. Google delivers 350x the merchandise to Stack Spill over that the close best so-called "search causal agent" does. Three one hundred00 and fifty times!

Now, I don't claim that Stack Spill over is representative of all computer on the computer network -- obviously it isn't. It's a computer for programmers. And let me be absolutely solid clear that I have no question at no with Google. That same, I find it profoundly heavy that if all otherwise search causal agent in the world shut up down tomorrow, our website's merchandise would be effectively unchanged. That's honorable scary.

Yes, I like Google. No, Google deeds great and has been my web page for about cardinal eld nowadays. Google nailed search, and they be the leaders position they've attained. But where's the levelheaded contest? Where's the rational motive for Google to change? No I see is a large and organic process monoculture that book as the start page for the internet.

I'm a little dumbfounded no the group United Nations agency were so up in instrumentality about the Microsoft "market" ten eld ago aren't out in the streets twenty-four hour period 4-hour interval combustion torches and sharpening their pitchforks to go aft Google. Does the construct that Google's products square measure mostly free and ad-supported somehow immune it from the European looking? Isn't anyone else concerned that Google, even with the best of "don't be evil" intentions, has transmute more master than servant?

Calling the electric current state of search causal agent contest a framing race is an disrespect to framing races. No, what we have Hera is a one framing race where no the otherwise horses were shipped off to paste factories eld agone. Forget "search discussion", you should be throwing a "Google discussion", because there's no difference.

I don't know. Maybe that's OK. But it does mean that if Google, for some reason, definite to remove you from its search results, your data processor no mortal exists. At thing not as a live business, anyway.

Tagi: rich skrenta, stack overflow, age venture, persal, square measure, google, hayseed, flickr, house of york, back of the napkin, search success, search engin, pettiness, search engines, accordant, arithmetic, half a dozen, mth, occurrence, elephant