Most of you have probably heard about the deep web and the dark web

There are many myths surrounding the subject, along with the usual images that portray it as an iceberg, in which the majority of the content is hidden bellow the "water".
But how much of that is actual truth, and how much is pure sensationalism and clickbait is up for debate.

So, join me on my first steemit post, in which I will attempt to clear some of the misinformation using as few technical terms as possible

First of all, in order to understand what the deep/dark web is, let me explain what the surface web is.
The surface web is every part of a website that is indexed by google, bing, or any other web search engine. You might wonder why I did mention "part" of a website instead of just saying website. This will be explained soon.

In spite of popular belief , the deep web and the dark web are not the same thing.

THE DEEP WEB

The deep web consists of every part of a website (or the whole website) that google or other web search engines can not index.

This includes:

• Pages that hide behind passwords. For example, if your facebook profile isn't public ,it is part of the deepweb. That may come as a surprise to some people. But everything behind a password or a paywall is not accessible to google (or other engines). This probably includes your very own steemit account along with medical/scientific/legal documents, banking and government info and as you can imagine many many more things that are not readily available to the public.

• Sites that search engines dont have the link for. If for example you made a site and you didnt link its URL in an already indexed site, then google doesn't know about your site, therefore it is part of the deep web.

• Pages hidding behind the robot exclusion standard. The designers of a webpage can ask google to not index parts of the site through a file called robots.txt. This is more of a plead than an actual security measure. Meaning that certain web crawlers that disregard the robots.txt won't have any problem indexing that site.

• Dynamic content. There are many examples of dynamic content. Your facebook feed is dynamic content because it is formulated right then and there according to your friends and interests. Also the search results from any search bar in any site is dynamic content. This happens because google won't search every possible term and combination in every search bar, because that would overwhelm both google and the website itself. Many if not all instances of dynamic content are part of the deep web.
  

After reading all of the above,the speculation that the deep web is hundreds of times bigger than the surface web doesnt come as a surprise anymore, even though pictures like the one below may seemed outright impossible.

THE DARK WEB

The dark web is similar to the surface web but it uses some overlay networks called darknets.
A darknet is similar to a regular network with one key difference. You need special software to access it.
Each darknet has specific protocols, handshakes and encryption, meaning that your usual browser (chrome, firefox, opera etc) won't be able to receive and read darknet sites. Hence why you need specific software. That software will know how to ask and receive those special sites.

Here is a list of darknets:
l2p, RetroShare, Freenet, OneSwarm, Riffle and many more.

But one specific darknet has gained immense popularity and notoriety due to the anonymity it provides. And it is what people mean when they mention the dark web, the vast majority of the time.
Its called TOR, the onion router.

The sites made for this browser use the domain .onion and they are usually called onion sites. The Tor browser can open surface websites, but normal browsers can not open onion sites.

The Tor software is free and its main goal is anonymous communication.
It uses a technique called onion routing, hence the name TOR (The Onion Router).
Onion routing was developed by the US NAVAL Research Laboratory and later by DARPA.
It achieves anonymity because the transmitted information gets encapsulated in layers of encryption, similar to the layers of an onion. The encrypted data is transmitted through a series of network nodes called onion routers, each of which "peels" away a single layer of encryption, uncovering the data's next destination. When the final layer is decrypted, the message arrives at its destination. The sender remains anonymous because each intermediary knows only the location of the immediately preceding and following nodes.

At this point you may ask why would those US agencies make this software available to the public. The answer is pretty simple.
There is no point in making this system if it is known that every message within it originates from CIA agents. Flooding the system with thousands of people solves this problem. A public version of Tor was released around 2002-2004 in order for it to be useful for the American Inteligence agencies. At least up to 2013, the United States founded 90% of the money needed to keep TOR operational.

Now lets talk about the size of the deep web and the dark web

Even though the terms deep web and dark web are often confused. In reality the size of the dark web is a microscopic fragment compared to the vast deep web.

To put things into numbers:
(those numbers may not be entirely accurate, because one does not simply measure the internet)

In 2016, around:
1,100,000,000 unique sites existed in the surface web
15,000,000,000 different pages were indexed by google as part of the surface web.
3,500,000,000 unique surface web users were part of the web.

For perspective, only 60,000 unique onion sites existed in the dark web along with around 1,000,000 Tor users. (estimated in 2016). It is also estimated that from all those users, a very small <5%, visit onion sites, and the rest just use TOR to protect their privacy while browsing sites on the surface web.

To sum it all up

The deep web is hundreds of times bigger than the surface web. While the dark web is thousands of times smaller than the surface web.

Tor can help people communicate freely under totalitarian regimes, it can protect journalists and make many more positive things a reality, but...

The downsides

Naturally, the anonymity can also attract illegal activities.
A good example was the site called Silk road which operated from 2011 to 2013. Silk road was a marketplace for all kinds of drugs and such.The transactions that took place in the site reached a figure of over a billion dollars. The original Silk road site was closed due to the arrest of its owner by the FBI.

Silk road today

His arrest serves as an example that even though you may be anonymous in TOR. It is still sometimes possible to be found if experts put enough resources towards finding and arresting you.

Many similar sites have launched since, offering similar services (drugs,porn,counterfeit money, fake id's).
All those sites use anonymous currencies such as Bitcoin and Monero in order to protect their users privacy.

Hundreds upon hundreds of forums and image boards that host content such as illegal porn, stolen personal info, leaked documents etc exist. But one kind of site mentioned in videos or articles about the dark web always seems to capture the attention of the viewers.

I persuaded my self to not put a cliche image of a dark ominous room here

The so called "red rooms".
Red rooms are supposedly darknet web sites in which people are being tortured in a live stream and the viewers that pay the most money to the host choose the fate of the victim. Even though there are many kind of stories about red rooms on the dark web. Most if not all of them are fictitious. First argument supporting this theory is that TOR or other darknets such as I2P aren't actually suitable for live streaming. And second is the fact that there is virtually no proof of such things being broadcasted live, nor have such cases been reported to authorities such as the FBI.
The inspiration for all those "red room" stories seems to be the fact that violent snuff videos are indeed circulating the dark web (and the surface web). But the whole point of these "red room" stories is the fact that the atrocities are happening live, and that the fate of the victim depends on the richest viewer.
Adding to the myth, are the many "honeypot" sites that are masquerading as red rooms. Those sites require an up-front payment in order to access the "live feed" but once someone pays, nothing happens. And since the host of the site is anonymous and the payment transaction is irreversible, making such sites is a tactic that many people have adopted in order to make some money (bitcoin, monero etc).

The same tactic can be used with a different front. For example instead of a "red room", such a site could have hitmen for hire, illegal porn or drug trade as a front. The people behind those sites could either be people that do this for economic purposes or agencies such as the FBI that are trying to lure in criminals.

But even though the majority of the dark web stories are fake.
There is certainly stuff in the dark web that could cause severe damage to the mental health of a person.

A simple google search of "dark web forum screenshot" will give you enough proof of that.

Harmful, illegal and malicious content is hosted in almost half (estimated) of the existing darknet sites.

Using Tor is legal in most countries (needless to say, this is not legal advice), but browsing requires care and responsibility.
Do the risks outweigh the dangers ? Probably not, but that is mostly up to the one browsing.
In my opinion there is enough info on the surface web to paint the complete picture and create this article. (documentaries, youtube series of people browsing the dark web, info in programming sites, surface web forums, Wikipedia etc)

To close off, I would like to say that you should not take anything mentioned in this article as an absolute fact. Because in the ever changing internet, no statistic or security measure should be taken as something absolute.
Nevertheless, I hope I helped some of you quench your thirst for this topic. This is my first post on steemit, and I am not sure about how big a post here should be, so I tried to explain things quite briefly.

Thanks for taking the time to read my post, any feedback is greatly appreciated.

See you next time!