How do search engines work?

Logos of different search engines

“Let me Google it for a second” is a frequently made comment nowadays when something doesn’t come to mind right away. What we however don’t realize are all the considerations a search engine has to make to decide whether web pages are relevant to show. How can a search engine offer an appropriate selection of pages based on your search query? This is not just interesting stuff, it is also of great importance to have a good understanding of the process. Because then you’re able to respond properly! Did you know the whole process even begins before you have typed your search query?

How a search engine works: crawling and indexing

The internet is a gigantic library with a huge amount of web pages. Lots of books without a neat central system, so to speak. So called ‘crawlers’ of search engines systematically browse the web, checking out web pages and following links on those pages. Actually, just like you do yourself when surfing the web. But all the information found by the crawler gets delivered at the servers of the search engine and subsequently is organized. This organizing is what we call indexing.

Google versus Bing and Yahoo

Bing, Yahoo and Google. One by one renowned search engines, of which the latter is so well-known that the company name has been integrated into our languages as a verb. Google has a market share of 80% on average worldwide and leaves other search engines way behind.

Algorithms: smart, complicated codes

Search engines can determine the order of the search results with the help of smart algorithms – a kind of complicated mathematical formulas. As online marketers, we obviously want to know everything about these algorithms so we can optimize effectively for a full 100%.

However, the way these algorithms function always is a well-kept secret. Which is actually a good thing, because if websites could be optimized for a full 100% every random page could acquire a top position in the search results. Now that would seriously impair the organic search results and dissatisfy the search engine user. But what do we know about the algorithms and its updates?

Google keeps improving algorithm

Google is always improving its algorithm and there are several reasons for it. For instance, do the search results still meet expectations of the users? To be able to present the most relevant search results, Google needs to update its algorithm continuously. Thus, the search engine giant has to respond to technological developments, such as the rise of mobile devices. Also, online marketers have tried really everything in the past to score on keywords using spammy tricks. Just another reason to update the algorithm to punish these practices.

Penalties for ‘spammy’ websites

The implementation of these measures into the algorithm to combat these spammy practices had huge consequences for a lot of old school online marketers. Websites that had bought hundreds of links at link farms for link building purposes, were punished mercilessly with harsh penalties that resulted in huge declines in the ranking positions. It could even mean a complete removal out of the rankings!

To paint a complete picture of the development that Google has made as a search engine and the dilemmas that online marketers have dealt with, we will describe the most important updates of the algorithm. Keep in mind however, that Google barely discloses information and we have to rely on the findings of experience experts. The main algorithm updates:

• Panda
• Penguin
• Pirate
• Hummingbird
• Mobile friendly
• RankBrain
• Possum
• Fred



Google Panda update

In 2011 the first Google Panda update was carried out with the purpose of allocating higher rankings to high quality web pages. This update supposedly resulted in a change in the search results of a whopping 12%! Business that were solely engaged in setting up web pages with ads and spammy texts had to look for other ways of earning money, because they all received penalties resulting in low rankings. Then, what makes a web page of high quality? Google doesn’t tell. Guidelines do show the importance of preventing spelling mistakes, double content and over usage of keywords.

Google Penguin update

The next big step in rewarding high quality web pages and fighting ‘black hat web spam’ was the Google Penguin update in 2012. Mainly websites with artificial link profiles were identified and punished. Did you have back links from websites with irrelevant subjects, websites of poor quality and/or websites solely intended for link building? Then again you could count on receiving a penalty and forget a good search engine ranking!

Google Pirate update

With the Google Pirate update, Google took care of the rankings of websites that repeatedly violated copyrights and/or offered illegal content. Just think of torrent websites that illegally offer large amounts of movies, music and books. However, even mighty Google doesn’t always succeed (even nowadays) to drastically lower the rankings of new websites with forbidden content.

Google Hummingbird update

In 2013 Google replaced the motor of the search engine under the heading Google Hummingbird. Because of a better interpretation of the search query, henceforth better providing in the searching needs of the user. The most important addition was conversational search. When searching for “What is the nearest place to my home to buy a HD TV?” at first one would get presented pages based on the terms ‘buy’ and ‘HD TV’. Thanks to optimizing the use of synonyms, more theme related results could now be presented instead of merely looking at the exact keywords. This also was the moment when exact keyword targeting and the over usage of keywords by online marketers didn’t pay off anymore. Context already became more import here!

Google Pigeon update

The next algorithm was named the Google Pigeon update and this one primarily influenced local search results. Search users that filled in a local search term, like ‘bakery + town’ received more accurate and usable results.

Google Mobile Friendly update

With the Mobile Friendly update Google finally responded to the rise of the smartphone. From this moment Google valued mobile friendliness way more. This reflected in the mobile search results: mobile friendly pages scored better than pages that were not yet optimized fort the smaller screens of mobile devices thus were not responsive yet.

Google RankBrain update

RankBrain is the system that was launched in 2015 by Google and became a very important ranking factor. It is a self-learning system with artificial intelligence that helps Google better process keywords. Google even suggests it to be the 3rd most important cornerstone in determining the positions of websites in the search results. RankBrain is part of the Hummingbird update and is actually quite special since it was the first time the people of Google named an update their selves.

Google Possum update

2016 was the year of – amongst others – the Google Possum update. According to the search engine world this update mainly comprised optimizations in the field of local search results. This could especially be seen in the 3-pack (the 3 companies that were presented as local search results). It, for instance, resulted in the service area of companies receiving more value instead as opposed to the physical location of a business.

Google Fred update

March 2017: the Google Fred update came through. Experts suppose that this adjustment primarily influenced websites that were engaged in spam practices. Like low quality websites that were purely designed to attract click-traffic. Since you obviously (want to) run a high-quality website, you can definitely take advantage of this update and move up a few positions.

Our conclusion: optimize for Google and your visitor

All right, after reading this text you have gained insight into the way search engines work and the most important Google updates. It's really valuable to have some knowledge of and feeling with the developments the search engine has gone through. Clearly, they’ve had a significant influence on the search results. As an ambitious webmaster or online marketer your take away lesson is: don’t try to fool Google or any other search engine! Do make sure to not only optimize for Google, but also to keep your visitor in mind. The history of Google’s updates learns that people who try to fool Google, sooner or later get the worst of it. Do you keep your users’ interests in mind? Then you can automatically take advantage of new updates!

Learn how to effectively respond to search engines and their preferences by reading this article about designing a search engine campaign (SEO).

Can't wait to optimize your website for SEO? Then check out the all-in-one SEO tools we've used and reviewed ourselves.

Or how about SwissMadeMarketing's ranktracker to specifically keep track of your positions in Google and other search engines?

For more information about Ranktracker you can also check out the official website of SwissMadeMarketing.