Search Wars: The Empire Strikes Back

Search-Wars-Logo-e1421429928227We have all been bystanders as the tech giants duke it out in various arenas. Microsoft won the battle for the desktop, Google won search, Apple won the device war, and the phone war continues. The supremacy enjoyed by Google in search has been so strong for so long that we have all stopped talking about it, except as a possible cause of anti-trust law suits. That could be changing, and the cause might well be Windows 10.

Microsoft has had the Bing search platform for a good while now, but it’s only loosely integrated with their other products. It was once terrible; now it’s pretty good. In most cases, it’s just about indistinguishable from Google, and has clawed its way to about 20% of the overall U.S. search market. However, those clever guys over in the Evil Empire of Microsoft have plans.

In the new Windows 10 version (which is officially released today), Microsoft has wrapped search around all of its components. Irrespective of what you may be looking for, whether on your PC, in your Outlook or anywhere on the web, Windows 10 will be able to give you an answer without you ever leaving the Windows environment.  Since that environment is to be found on something like 93% of all desktops, that must be a little worrying to our good friends at Google. In addition, the new Windows browser is supposed to be lightning fast. The start bar is back, their digital assistant is impressive, and all in all, what I’ve seen looks both modern and very usable.

I’ve put in for the free upgrade, and I’ll know more once I get my eager little hands on it. However, if I’m rattling around on my Windows desktop using mostly Windows programs, I might not be bothered to go open a Chrome browser just to search on Google. Of course, there will be an adoption curve. There is still a measurable number of Windows 3.1 users, but if the Windows 10 search is as good as it looks, this is going to hurt Google’s search market share, and it will once again be game on in Search Wars. I’m getting popcorn; want some?

Stalking the Google Way

image

The recent data breach at Ashley Madison (a company whose boss was dumb enough to claim that he had the most secure site on earth) means the site may end up with private information about millions of people having affairs being released online. What is perhaps much more threatening to anyone trying to stay under the radar is the spy in your pocket. Google just announced the latest version of its timeline feature in Google Maps, and it’s kind of horrifying.

For as long as you have had location services turned on (the default is off, but many people turn it on to take advantage of other cool features), Google has been tracking your every movement. For example, last Christmas, we visited Las Vegas. On my timeline for Christmas day, it shows that we stayed at Caesars and visited an exhibit at the Luxor. It’s not perfect; it shows us at the Hilton (the Purple Rain tribute show), but has the time wrong. Nonetheless, it’s pretty amazing. The fact that I didn’t ask to be tracked and didn’t know it was happening is apparently neither here nor there.

In theory, all this rather creepy. Tracking is double opt in, but I bet most people have no idea what Google has been tracking for the past five plus years. Do you know where you were in April 2009? Google does. In some cases, it even shows me moving around inside my house. Again, weirdly creepy.

Obviously (as always), all this data collection comes down to commerce. If Google knows where we are, it can better target ads of all kinds at us. Since most of us don’t make much effort to control what we share (most don’t actually care), maybe it’s just another aspect of our “nothing to hide, nothing to fear” culture. However, since Google has no problems sharing with the government pretty much anything they ask for, you have to wonder what Google and our overlords are making of the places you go and the people you see.

Google Going Local?

unnamed

The word on the street for quite some time has been that Google wants in to the home services racket. The marketplace represented by all the work done by people like plumbers and lawyers is simply too huge to ignore. At the moment, most folks in the space (like Search Initiatives) focus on helping local businesses and national businesses with lots of locations drive business to those storefronts. Some sell leads, some build websites that do well in search, and some help those businesses spend their ad dollars to maximum effect. A great many of those ad dollars are going to Google, where the click prices for local search terms like “Plumber San Diego” are insanely high — over $40 a click in the case of that particular query.

The announcement last week that Google is buying HomeJoy and rolling their team into the Google-opolis reinforces the suspicion that something is afoot. The HomeJoy guys were essentially an Uber for house cleaning. They had raised a good chunk of funding, but were struggling with their latest round. They’re also getting into hot water in places like California, where the issue of their cleaners actually being employees rather than contractors was getting nasty.

Google now owns HomeJoy and a business reviews and profiles platform called Thumbtack. They represent different angles on the same space, and it’s likely that eventually, Google will pick a path and jump in.  The smart money says that Google will follow initiatives they have been announcing recently, where they become a vendor rather than only being the marketplace where all vendors compete. In local services, this means that a local plumber signs up and agrees to cut Google not how many clicks it costs to get a potential customer, but a piece of the actual value of the work involved. Amazon is trying something similar.

It’s likely that when they roll this out, the top results on search won’t be the people willing to pay the most for the click, but rather the people willing to cut the largest check out of the job value. The math runs something like this: it varies wildly, but many businesses will spend up to 10% of the value of a job on marketing. For example, an air conditioning repair guy will have to pay about $15 a click to have any chance of getting found on Google, actually closer to $30 to guarantee top spot, but $15 should get you some visibility. We know it takes around 5 to 10 clicks to get a solid lead, and most businesses close about 25% of all jobs they quote. That means a local business has to buy something between 20 and 40 clicks to get that billed job. That’s roughly $300-$600 of marketing cost per billed job in this case. As you can easily see, it’s tough to make money in the local search market where amateurs bidding on search terms are causing huge price spikes.

Given that Google is already earning a ton of money from this area, it will be interesting to see how they pivot into competing in the space. Convincing local businesses to buy search has been hard. I can’t imagine convincing them to sign over part of the contract value will be much easier. There will be questions like: how do they check on what got billed and paid? Is the job booked and collected by Google? What if the contract goes bad, and someone gets injured? Is Google essentially becoming the prime contractor and back office for local business?

It’s a massive opportunity, but also a huge potential pain in the neck. Local is hard, really hard. It’s the “Russian Winter” of the online space. It will be fascinating to see if Google has the stomach for the fight.

When AI Meets PC

image

Google has spent many years and billions of dollars making its algorithms really smart. They detect behavior and the kind of content we look at, and then Google targets ads to users which its algorithms “think” fit them best. The recent calamity where it categorized African Americans as “Gorillas,” and more recently the one where it categorized a Chinese man as a “horse,” may be just the tip of a problem that most of us probably didn’t see coming.

If you make your algorithms super smart, the problem is they will behave as trained and work really hard to send the right ads to the best match, irrespective of the societal norms. In recent studies, researchers at Carnegie Mellon created fake male and female users and browsed sites which have a strong gender bias — think caranddriver.com vs. cosmopolitan.com. It’s not an exact study, but it’s an interesting approach. They then set these fake identities to browse gender-neutral sites and tracked what ads Google presented to the various fake users.

The results were striking. Male users of news sites like CNN were more frequently shown job ads with higher salaries than females on the same sites. They also found that an image search for CEO only presented 11% female results. Frequent browsers about addiction were sent ads for rehab.

Some of these results can be explained empirically. The fact is there are many fewer female CEOs, so the number of images indexed is likely proportionately lower. Likewise, the addiction result. Google tracks your browsing history, so if you spend a lot of time looking at fly fishing sites, it’s likely that’s what you will get ads for on CNN.  The fact that it also works for addiction feels different to fly fishing, but it’s not really.

The jobs and gender question is much weirder. It suggests that Google is divining your gender from your behavior, then making a value judgment. If we can assume that job ads don’t/can’t in the vast majority of cases specify gender, then it’s odd and perhaps worrying that Google is doing the math for us. It seems unlikely that someone in Google sat down and came up with this as a neat strategy. But rather, it’s probably the Google ‘brain’ watching the kinds of jobs people of different gender apply for and preferring those kinds of jobs to that gender because they are more likely to earn that click. Google only gets paid when ads get clicked on, so it tries really hard to fit the best ad to what it knows about the user every time. That’s fine for fly fishing, but weird for jobs.

I suspect that this is simply the unintended consequence of the AI running the ad platform getting a little too good at its job. In society, we adopt behaviors and constructs to address what we perceive to be intrinsic societal problems. We make extra efforts to be inclusive; we go the extra mile. AIs don’t have the same PC conscience, unless it’s programmed into them. I imagine there will be a bunch of changes made in the near future to “level the playing field” on this topic. Ironically, it will probably reduce the overall effectiveness of Google’s ad product in some areas. But that might be a small price to pay to avoid the discrimination law suits.

Google Accused… Again

unnamed

 

There are some arguments that just won’t go away. The “Google is skewing their results” argument has been kicking around for several years, and a new version has just popped up. I’ve commented multiple times on this deathly dull topic, but let’s review for old time’s sake.

Google skews its results. It’s a simple fact. They have done it for years. For example, if you use SafeSearch, it will skew away from adult content. When you make a local search, it skews results to fit your location. Nobody complains about that kind of skewing; it’s the type where they favor their own results or partners in reference to a competition that causes concern.

The FTC investigated this a year or more back in the US and found no fault. It’s unlikely to revisit this challenge, in part because it’s tricky to prove and in part because Google has a monster lobbying engine and the tech jobs issue (much like guns) is a third rail issue. The equivalent of the FTC in Europe is looking at taking on Google. Those EU guys hate Google with a passion and are always on the lookout for a stick with which to beat them.

This latest stick comes from a study commissioned and paid for by Yelp. That in of itself probably makes it worth less than the paper it’s printed on, but it’s an interesting study nonetheless. Essentially, they showed Google search presentations with and without the Google “Focus on the User Listing” OneBox container. In their testing, they found that users clicked on the plain results rather than the optimized version 45% more frequently. Their conclusion was that Google is deliberately sabotaging the end user by making the results less easy to use, forcing users to search again or…what, give up in tears? For a complaint of this kind to prosper, you have to show harm, and their version of harm here is a lower click through rate.

They aren’t claiming that a Google OneBox business being preferred is actually benefitting Google financially. Rather, by featuring OneBox registered businesses, Google is harming the end user. The overall claim is at best paper thin.

There are several quite good reasons why Google often prefers OneBox results. To start with, OneBox businesses have taken the time to validate their business, confirm the info is correct and, in many cases, create a nice business profile page that the searcher may well click through to. Either way, what does seem to be clear is if this is annoying users, it’s doing so to the extent where they are changing their search engine. Most people have bigger fish to fry.

Search and how that search is presented (much like the weather) is a big, complicated topic. If you stare hard enough, you are bound to find things that might look a bit off. Keep staring and it will change. Complaining about it to an FTC which has been bought and sold by Google won’t get you very far. I imagine these researchers will be mailing their results to the EU commissioners right away.

Is Google Treading Water? Or Drowning?

4.si

I’m not really a stock market betting guy. It’s a game I’ve never been good at and frankly doesn’t interest me. I do keep an eye on several stocks, though, and one of them is Google. It’s been a tough year for them, and I’m sure their larger investors are letting them have it at their current shareholders meeting. The problem is that they have been more or less flat, showing a 1% decline in a period where the S&P 500 has hit nearly 10% growth.

I have extensively documented the factors driving these doldrums. The rapid move to mobile by users when advertisers have been slower to follow has hurt them. They have swapped desktop dollars for mobile pennies in many categories. The irony of that transition will not be lost on newspapers, who suffered a similar calamity a decade or so ago when print dollars became online pennies.

Having conquered search, they went on to miss out on social media. They missed Facebook, Twitter, Snapchat, and a bunch of other upstarts that have been pulling users away from Google properties. They also have a penchant for super expensive, “revenue-free” projects like Google Glass, Fiber, Nest, Driverless cars and Loon. That’s fine and dandy when you are king of the hill and leading the pack, but it’s less cool when you are just another online ad platform. Add to those woes the growth of markets over which they don’t have any sway, like Amazon and eBay. It’s tougher to be a Googler than it was a few years back.

At its core, Google is an advertising platform based around an auction system. The explosion of mobile inventory and the slower rate of adoption by advertisers has driven their click prices down month over month. That will likely improve as the ad world catches up, but it won’t be soon.

They need some game-changing, revenue-rich ideas. To that end, they are moving towards being the marketplace and selling goods and services direct, as opposed to being the forum where advertisers pay to reach the audience. It’s a good idea if they can make it work, but it’s also dangerous as they may end up in competition with their own advertisers.

There is talk of them moving into our local space. The idea is that Google becomes the platform which a local business uses to get jobs, then shares the profit on that job with Google (as opposed to merely buying ads to get customers). It’s huge and potentially game changing. It’s also fraught with friction and would require a fundamental change in how the local economy works.

It’s possible that in spite of having a massive war chest of cash and market leadership in something as fundamental as search, the glory days of Google growth are behind us. If they are, Google stands the risk of being discounted in the same way that newspapers were a decade or so ago. Maybe it is already far too far out, and not waving but drowning.

Facebook Looking to Keep Users from Google

FacebookFacebook has been testing its own in-app search engine that will allow users to post links in a status update without having to visit Google.

Some U.S. users of the Facebook app will see an “add a link” option next to buttons to add photos or a location to a status post. A user will type in a search term and then a drop down list of links will appear. The user will be able to preview what is on that website and then share the link on the social networking site.

Typically, a user would have to search on Google (or any other search engine) or go directly to a website and copy and paste the link into. Facebook has been working on cutting out that process and keeping people inside the Facebook app for as long as possible.

Facebook has indexed one trillion posts that have been shared on users’ feeds. This will allow the in-app search engine to suggest the most shared links. This data will allow Facebook to steal a march on Google. This, coupled with advertising opportunities could worry Google, given the stiff competition for mobile ad dollars. Over 70% of Facebook’s total advertising revenue comes from mobile and the company has been working hard to keep people in the app for longer.

Another attempt by Facebook to keep its users away from search engines is news. A report in the New York Times this year suggested Facebook was in talk with news publishers to host content on the social networking site rather than linking back to the publisher’s website. The aim would be to share ad revenues.

The right to search?

dt.common.streams.StreamServer.cls_-820x420
It’s a weird thing, how we have come to think of search as some kind of inalienable human right…like life liberty and the pursuit of happiness. It’s not.  There is no God given right for anything to be indexed or made search-able and there is no right or requirement to put any result anywhere. This issue has been a recurring theme throughout this year. Although the authorities in the US have consistently ruled on the side of Google against those calling foul over monopoly various European bodies have continued to attempt to fine Google or force them to return results differently. Perhaps the highest profile of these spats was the “right to be forgotten” and only this week Google unceremoniously booted all Spanish news content from its news index (although much of it can still be found in the main Spanish search).

Google is not touting a “buy it now” feature on its shopping search to retailers to allow end users the kind of one click ordering straight from the search results page which would be similar to what we have come to know and love from Amazon. Google has also been layering other kinds of offers around and above the inevitable Amazon result for almost any shipping search you care to carryout. No sooner has this feature come to light then the usual suspects are once again are yelling monopoly.

I have nothing against Amazon, I’m a constant Prime user, but I wouldn’t be offended if the Amazon result was pushed down in the search result…because I know I can always go straight to Amazon. Search is notoriously tough to do well and love them or hate them Google does it very well. Having said that it’s also voluntary…if various world governments want to force Google to publish their algorithms or force them to pay for news snippets I expect Google to respond as they did this week by simply shutting down that part of the service in that part of the world.

Google is too big to be stopped and on the whole does an amazing job. Are they an evil monopoly…maybe…but they are very good at being that. This year coming I expect to see either a bunch of potential trade disputes over search to gently fade away or a bunch of countries loosing access to the best search out there. It will be interesting to see who blinks first.

Image Recognition Getting Real

 

search-marketing

Humans are visual…very visual. at any one time something close to 65% of our brain run time activity is taken up with image processing. We also inherently look for patterns…it’s why we see castles in the clouds and Jesus on toast. Recognizing and responding to images is so second nature it’s easy to assume that huge powerful computers should be able to do the same thing…not so.

Although images have been a huge part of the web since day one the ability to recognize and label an image has been minimal at best. There are billions of images on line and many (most?) aren’t identified well or at all. I remember well back in the early days of image search at FAST we would get all kinds of complaints from users for delivering adult content in search results which wasn’t flagged as “adult.” What was happening in many cases was that adult sites would inadequately protect their content and our indexer would just breeze past and index all their images then make them available for free. The images were typically names something like TS1234, TS1235, TS1236 etc. Not only were there no captions to indicate their content our algorithms were rarely able to detect that they were adult programmaticly which meant we had to fall back on spot checks by humans. If you think you have a horrible job it’s nothing compared to that task.

That was over a decade ago and only now are we seeing real progress in this area. Google has announced an artificial intelligence program which can read an image and figure out what it’s of pretty much as well as a person could. It’s not perfect, but it is pretty impressive and can scan and categorize content at a massively faster rate than human. They achieve this pretty impressive feat through neural net technology which mimics the ways that brains think.  This kind of ability is particularity interesting for content areas like Flickr or Instagram where useful captions are the exception rather than the rule. Once it works on images it’s only a short hop to doing the same for video.

Of course the incentive here is if Google can index content and figure out what it is on its own it can make that content readily available to searchers, improving search results and increasing the volume of target-able data it can stack ads against…very cool.

The Vision Thing

google-office

Companies have mission statements…they used to be widely mocked as marketing nonsense but they have become pervasive. For as long as I can remember Google’s has been to “organize the world’s information and make it universally accessible and useful” their unofficial mission statement has always been “don’t be evil.”  Many would argue that they have done a pretty good job of delivering against their official version and at least recently a much less great job of delivering against their informal one. In any event Google has changed as it has grown and it now does a great deal more than just organize the world’s data…and that’s apparently leading to some concerns in the Google-Plex.  In a recent interview King Larry himself admitted that the company had outgrown its mission statement and was pondering what to do about that. He didn’t have any immediate suggestion as it’s clearly a tough question.

At the heart of the issue is the question “What does Google do?” The short answer is “lots” and probably “too much”.  Having organized the worlds data then found ways to turn it into the most powerful new advertising vehicle since the invention of the TV it has amassed a war chest so large that it has the resources to try almost anything…and they have.  They are in email, data infrastructure, social media and even medicine to name but a few areas. They power a significant majority of the world’s mobile devices with Android and have many projects with lofty (some argue crazy goals) like delivering Wi-Fi to the third world by high altitude balloon which cost billions with no measurable revenue attached as yet.

Despite the extraordinary range of projects they are involved with the vast majority of their revenue comes from their dominance in search.  Love them or hate them the majority of online users in the western world navigate the web through their products. It’s a great place to be in…but it’s not without problems which I noted in this blog a week or so back.  So it’s hardly surprising that the powers that be are struggling with a message which covers all the pies they currently have fingers in. Coming up with a mission that covers autism research, self driving cars and online advertising is a tall perhaps impossible order.