Limitations of Google and the Open Web

It may feel like Google, with it’s millions of results for even the most trivial search, is serving us the entire universe of information. But Google, or any search engine, is only able to retrieve results from the surface web, the part of the web easily accessible through any search engine.

There are a variety of ways to keep a website inaccessible to Google and a variety of reasons someone might want to. Your email and online banking information are set behind login screens because that information is sensitive and private. Websites where illegal activity is taking place (the dark web, which is only a small part of the deep web) can require particular browsers or authorization in order to remain hidden from law enforcement. But the most relevant situation for the purposes of researching a wicked problem is when content is valuable enough that it is put behind a paywall and accessible only to those who are willing and able to pay. This includes streaming video sites like Netflix, some newspapers, magazines, and journals, and most books.

Think about the gatekeeping processes discussed in the Types of Sources chapter. A lot of these processes are time intensive and cost a good deal of money. Sources that have gone through a careful process of creation are likely to be both high quality and behind a paywall so that the costs of production can be recovered. If it’s good enough that some people would pay for it, the creator probably isn’t going to give it away for free on the internet.

Think about that: as much as Google can show you, it tends to be the best, most carefully created information that you cannot get for free on the web. (I say “tends to be” because you may also remember from the previous chapter that there are no hard and fast rules about determining the quality or usefulness of information. Some creators give away high quality information, and some charge for questionable content.)

Libraries & Access

If you want to participate in the conversation about a wicked problem, many important parts of that conversation may not be available on the surface web. Happily, one of the main functions of libraries is to provide access to information not freely available elsewhere. They do this both by buying print versions of sources and by buying subscriptions to portions of the deep web that are otherwise paywalled.

The way you can discover what materials the library provides access to is by searching the library databases. A database is an online collection of material – it can be a collection of anything really: ebooks, articles, videos, songs, images. Some databases are available on the surface web and can be used for free. Google itself is a database, one that contains links to websites.

Many databases are part of the deep web, meaning they are behind a paywall or require authentication to access. Each library chooses the databases that are most relevant to their communities and pays for access to those on behalf of their community. By using login credentials to limit who can access these databases, libraries make available deep web content that Google can’t show you. At PSU, our library makes available over one hundred databases. For example:

• Academic Search Premier: One of our largest databases, contains a mix of scholarly and popular articles on a wide variety of topics.
• PsycArticles: A smaller database that contains only scholarly articles about psychology-related topics.
• CQ Researcher: A database of hundreds of expert reports on current events, society, and world affairs.
• Gale Ebooks: (mentioned in previous chapter) Online reference books, including encyclopedias, covering most subject areas.
• Naxos Music Library: A collection of over a million classical music tracks.

As you can see, some databases focus on a particular subject area, and some focus on a particular format. It’s a lot to keep track of, and you don’t need to memorize what each of these databases is good for. You can always ask a librarian or use the Subjects menu on the database list page to see which databases are recommended for your subject.

The library catalog is also a database, but this database is a special case. A library catalog is a database that is built by each individual library and is unique to that library. It contains records of all of the materials physically located in a particular library and their locations and call numbers. The big search box you see when you come to the Lamson Library home page is how you can search our particular catalog. The material in the catalog is mostly books, and although it can also show you some of the online sources that are in our other databases, we recommend searching the other databases for the best search experience.

Search Terms

Take a look back at the example from the chapter Our Mental Shortcuts, where we found that different search terms related to the same concept can yield very different results. There are many tips and tricks for improving your search results and finding more of what you want, but the most important skill is choosing search terms.

Identifying Main Concepts

It is very useful to be able to state what information you are looking for in the form of a question. Many times we think we know what we want, but when we try to put it in the form of a question, we find out our ideas are a bit vague. Before you start typing anything into a search box, check that you can phrase what you’re looking for as a question.

Once you have articulated a question, look at it closely and try to find the main concepts that make up your question. These concepts are going to be the key to finding good search terms. For example, if you want to know,

What can employers do to prevent their employees from burning out?

There are really three concepts there: employers, prevention, and burnout. If you just searched for the concepts employers and burnout, you might get information about how employer practices that contribute to burnout. If you just used burnout and prevention, you might get information about what employees can do avoid their own burnout. If you just used employers and prevention, you would probably get information about preventing workplace accidents. You need all three concepts to get at your question.

Don’t Type the Whole Question

When using a search engine like Google, we can and often do just type in our entire question, and it works just fine. However, many other databases are much more literal. For example, in many of the databases libraries provide access to, if you a type in What can employers do to prevent their employees from burning out? the database will assume you only want articles that use all the words you typed in, including the unimportant words like what, can, and their. 

Really, try this search in this database and see how lousy the results are. It assumes you want articles that use the words burning out and so it overlooks articles that use the word burnout. Now instead of the whole question, just search for the concept words burnout employers prevention, and see how much better the results are.

Identify Synonyms for Your Concepts

Another way to get better results is to think of synonyms for your search terms. There are always a bunch of different ways to say the same thing. For example we could address the same concepts involved in burnout by using words like job stress or work life balance. Or we could combine the word employment with the phrases mental fatigue or emotional exhaustion. If the first set of search terms you try doesn’t work well, it doesn’t mean there is nothing on your topic, it just means it’s time to try some different words. Ideas for different ways to express your search concepts can come from the sources you read, from your discussions, or even from searching for synonyms in Google. Do yourself a favor and keep a running list of potential search terms as you’re reading, discussing and searching.

Using different synonyms for your concepts can help you find sources that approach your topic from a variety of different perspectives. For example the word choices of an expert who is writing a scholarly article may be very different from those of someone writing a casual blog post, even if they’re about the same topic. Trying multiple searches using a variety of search terms will help us retrieve both perspectives from different groups. To find instances of people talking casually about their work experiences, you might use search terms like work, job and gig, but to find works from a more academic perspective you might use search terms like labor, employment and occupations.

Narrow Your Scope

Coming up with search terms is closely related to finding the question that you want to focus on. Very often the first questions we ask turn out to be too broad, but they lead us to narrower, more focused questions. For example, the question What can employers do to prevent their employees from burning out? is actually enormous. After reading about all the many ways employers can help prevent burnout in their employees (adjust workplace culture, employee assistance programs, work from home options, etc.), you may choose to focus on just one, maybe the impact that increasing vacation time has on burnout. As your question changes so will your search terms. (Now try searching vacation burnout. There are far fewer results, but they are very relevant to the topic.)