What (and where) is the Deep Web?

By its most basic definition, the deep web represents the webcontent that is not indexed or searchable by standard search engines. When wesurf the internet for news, research, entertainment, and other common contentwe are only accessing just a small portion of the content that is out there. Infact, experts estimate that the Web we know and use every day makes up less than1% of the entire World Wide Web.

If we are just scratching the surface of the web with our searchesand surfing, what kind of content is available on the deep web? Let’s take acloser look at the concept of the deep web.

The major search engines we use to explore and navigate theinternet constantly index pages. Indexing, the process of adding eligible pagesto search results, is done by following links between sites (crawling) andgathering static pages.

Search engines are not able to crawl and index dynamic pages ordeeper content that lies beyond the search box. Pages that are kept behindprivate networks or those which are standalone and connect to nothing are alsonot indexed and considered part of the deep web. Personal accounts like email,online banking, social media, and gated pages and private databases are deepweb content because they are not found on search engines.

Here is an easy way to identify deep web content: if you mustlog in to an account with a username, password, or some other form ofauthentication, the information you access is on the deep web.

A layer of security protects the public from accessing deep webcontent – and for good reason. Deep web content should not be available toeveryone. Over 96%of online content is on the deep web and requires authorizationto access. If the public could access this data, private and proprietaryinformation would be visible and available to the world.

It is possible to access the content of the deep web with a directURL or IP address, but will typically require a password or some type ofsecurity access to see beyond public web pages. The most common content on thedeep web is believed to be databases, both public and those which users mustpay to access.

Intranet pages are also on the deep web because of their internaland proprietary nature. For example, a company might store personnel files orbusiness plans on a corporate intranet. If that information were public, itcould be disastrous for that company’s reputation and bottom line.

The concept of the deep web is commonly misunderstood. Manyconfuse it with the illegal and nefarious activities conducted on the dark webwhich is not accurate. For the most part, the deep web is a harmless portion ofthe internet and is quite important for protecting personal and sensitiveinformation from falling into the wrong hands.

The AJTC team are experts in data and network security and canhelp your company protect sensitive and valuable information. We will work withyou to create and implement solutions to best suit your business. Contact AJTC today to learn more about our managedservices, hosted services, and customizable technology solutions. You may also visit AJTC here or call708.942.8200.

Other blog posts