PDA

View Full Version : How the internet works



botmaster
05-04-2007, 07:55 PM
THE INTERNET

You use it daily to check your e-mail and to access SRL forums, to gather information, to authenticate your scripts, to hack people, and to torrent files. Simple enough, you call it "the internet". This tutorial will basically teach you a little bit about how HTTP/1.1, Protocols, and TCP/IP. Probably all techie stuff you've heard about before, but never really figured out what it REALLY means.

If you have a short attention span or can't stand text longer than a few lines, please exit this tutorial NOW (thanks to Boreas for that line :D ).

Feel free to skip certain parts when you feel you already know about stuff, but if you don't get something later because you didn't read earlier stuff, don't complain. Feel free to ask questions.

1. THE BASICS

So, what exactly is the internet? Is it the web you use to access SRL and google.com? I betcha 90% of the people on this world believe this to be the case - and they're WRONG! The internet is just that - a network of computers connected together by wires. Anything that transfers data in this network is the internet. This means bittorrent, e-mail, and MSN are part of the internet!

A computer in the internet can do two things: send and receive. Usually, it does both. Think of it as a telephone: You can send (speak into the mike) and receive (listen at the receiver). The receiving end is called a server because it constanly "Listens" to the connection, and the sender is usually called a client.

So, how is stuff sent through the internet? Answer: there's two ways to send data. One is called UDP, but I won't go into detail for that one, because it is complicated and used less often (like, almost never :p ). Basically, it sends data as a single stream through the wire (or fiber optics cable).

The more commonly used method for transferring data is to break down the data you want to send into small things called "packets", which are then individually sent into the internet. Packets are just that: like a packet you bring to the postoffice, except that this one contains data.

Like a packet at the postoffice, the packet has a receiving and a destination adress. Also, like at a mail office, the packets doesn't get sent directly to the destination computer but are instead passed over various servers that serve as "post offices". You can try tracing the servers your packets go through before arriving by using the followging command in ms-dos (or linux bash shell):

tracert hostname

where you replace hostname with the web address/IP you want to trace to. Speaking of IP's - I think I haven't talked about them yet.

2. ADDRESSING COMPUTERS HOOKED ONTO THE INTERNET

Every packet is adressed, as already said. Now, there are two types of adresses: IP's and Web adresses. There's a third kind called a mac adress, but it's rarely used and kind of complex so I won't explain it.

An IP is four number code, ranging from 0.0.0.0 to 255.255.255.255 . Basically, it specifies the adress of a computer. You can simply type in http://72.14.207.99 into your browser window, and you will land at... google.com!

Now, since four-code numbers are very hard to remember, a second type of adress has been developed: a web-address. This is the familiar address you type into the browser. But how does the internet know that if you type in www.google.com you actually mean 72.14.207.99? The answer is kind of complex: DNS servers. DNS stands for "Domain name server". Basically, it is a kind of server that serves as a sort of telephone book, where your computer looks up which IP corresponds to the domain name you typed in. Maintaining these servers costs money, so you usually have to pay to register a domain name in these servers.

3. PORTS AND SOCKETS

You occasionally hear techies talk about "ports" and "sockets". So here I'm going to explain to you what they are.

Imagine your computer as a call center, with thousands of incoming and outgoing connections to the internet. If that were the case, each connection would be a port. A port is simply said, a gateway into your computer. Ports are usually assigned a number between 0 and 65535.

If you continue with the call-center analogy, each person would be a socket. A socket is something that either listens for, or sends data. This data can be anything (usually stored in packets).

4. WTF ARE PROTOCOLS?

Okay, so now that we've covered the basic stuff, ports, IP's sockets and well, DNS, you know how computers are able to communicate to each other physically. However, there is still the matter of organizing the data into packets and telling the server/client what you're sending. You could be sending a textfile, and for all you know the server you're uploading it to could interpret it as a picture. In order to prevent this stuff from happening, computers use protocols.

Protocols are simply a way of organizing the data that gets sent. For example, to send an instant message, you need to tell your buddy that the data your sending is an instant message, not a file transfer or an invitation into direct-connection.

This is usually done by giving each packet a "header" which tells the client what the packet contains. A header could say "I'm part of a textfile called sometextfile.txt, part #3" and the client gathers up all parts separated into packets and puts them back together into textfile sometextfile.txt

5. HTTP/1.1 PROTOCOL: EXAMPLE

So, how does the www work? How does your computer manage to access SRL-forums.com? The web works by using a protocol called http. This is the stuff you don't see, the "behind the scenes". Even when looking at the html code of your page, you just see half of the story. The "behind the scenes" exchange of data from the server is also called "header data", and contains information about the version the server is running, the local time/UTC of the server, type of data transferred, and lots more.

The transaction starts at the client. Let's say, the user wanted to access google.com. You type it in, and press the enter button. Here's the sequence of events that happen when you wait for the page to load:


User sends GET request to google.com (get me the resource!), including information on the browser you use (user agent), the operating system, and loads more. You wouldn't believe how much info you reveal just based on your header data! :eek:
Server sends status report (usually 200, if it's ok, 404 if it doesn't find the page)
Server sends it's own header data (server version, date, time, cookies to be set, other stuff that can be quite revealing)
Server sends the resource
Client repeats this until all resources associated with the site are loaded (pictures, .swf's, scripts, etc.)


So, let's say you wanted to make a post on SRL forums. How do you send data to the server with http? Well, you use the POST request :) . Then there's also the HEADER request and some other more complex and less used ones, but I won't discuss these. Basically, the POST and GET request are the ones most commonly used.

You can find out more about the http/1.1 specification at http://www.w3.org/Protocols/rfc2616/rfc2616.html. WARNING: this site may give you a heart attack if you're easily overwhelmed by lots of technical details. Not for the faint of heart!


6. WHY SHOULD I CARE ABOUT THIS BORING STUFF?

Well... basically, you can use it to build auths using SCAR, to make programs that automatically retrieve information from websites, and lots of other useful stuff like protecting your privacy by controlling what your browser sends as header data. You can also craft custom packets and send them to other people or change the return address of the packet (i.e. your IP on the packet). Basically, lots of fun stuff.

I'm not to be held responsible for any incidents that result because of you reading this tutorial. Use your brain and don't do anything stupid - you'll regret it and you WILL get caught.

Have fun programming the web :f: !!!

Smartzkid
05-04-2007, 08:03 PM
:D Nice tut



An IP is four number code, ranging from 0.0.0.0 to 255.255.255.255 . Basically, it specifies the adress of a computer. You can simply type in http://72.14.207.99 into your browser window, and you will land at... google.com!

You can also type http://1089054563/ :P

botmaster
05-04-2007, 08:10 PM
:D Nice tut




You can also type http://1089054563/ :P

That's their mac adress or their adress in decimal? :p I used to know a trick on how to spoof a URL using mac addresses, but I forgot about it.

Smartzkid
05-04-2007, 11:53 PM
Actually, it's just the google url converted to binary

:D I use it to get past almost any content filter....works for ~1/6 of the sites that are blocked (depends how the site was made, I think)

botmaster
05-05-2007, 07:44 AM
I might try that at my skools content filter. Sounds a lot easier than proxying, just convert the adress to binary :D.

EDIT: wait a minute - binary is 1's and 0's, yours is numbers. Did you convert it back to numbers? Looks like it...

mastaraymond
07-10-2007, 02:41 PM
Good tutorial helped me a lot! Thanks!
Can anyone tell me how to edit Packets? Or read them?

~Raymond

Here
07-10-2007, 05:16 PM
Good tutorial helped me a lot! Thanks!
Can anyone tell me how to edit Packets? Or read them?

~Raymond

Ethereal or Wireshark for you.

kaustav_002
07-16-2007, 02:57 PM
How do u find the binary address of a site?And how can i change the header data?Someone plz tell me

Pyro
07-23-2007, 01:08 AM
Slightly old but you have the thing about udp and packets wrong.... It isnt udp and packets. Its udp and tcp.. Those are the two main ones. The difference being udp is a connectionless protocol. connectionless being that i will just send it randomly. Like we dont have to connect and authenticate everythign first before i start sending crap.

Also your protocol bit didnt really do much for the mind :p What about ipx/spx, netbios, and all that other ones. Or were we just focusing on internet? and not networking.

Its ok some good information if you didnt know about it

Smartzkid
07-23-2007, 01:14 AM
I might try that at my skools content filter. Sounds a lot easier than proxying, just convert the adress to binary :D.

EDIT: wait a minute - binary is 1's and 0's, yours is numbers. Did you convert it back to numbers? Looks like it...


The numbers are referred to as ‘octets’. An octet is an eight bit (read as: eight digit) binary number. Eight bits can represent any value from 0 (00000000) to 255 (11111111). All IP addresses are 32 bits long. Four octets (4 x 8) represent these 32 bits. Users and administrators read and write IP addresses in octets because using a stream of ones and zeroes is impractical—and could give your retina serious screen burn!

Content filters are expecting IP addresses in the standard decimal notation. Instead, we can express the same 32-bit number as one big number, instead of four smaller ones.

Convert the example IP address of 64.233.167.99:

Verify that the calculator is in Decimal (‘dec’ should be selected)
Type in the first octet of the IP address (64)
Convert the number to binary by clicking the ‘bin’ radial button.
Write this number down. The calculator displays ‘1000000’. Octets represent EIGHT digits. The result from the calculator shows only seven digits. In order for this technique to work correctly enter each result in eight digits. Pad the beginning of the number with zeroes until the octet has eight digits. This means you should write down ‘01000000’
Switch the calculator back to Decimal.
Clear the calculator display.
Repeat steps 1 through 6 for the remaining octets. Your results should be: 233 (11101001), 167 (10100111) and99 (01100011)
Switch the calculator to binary.
Combine the results of your conversion into a single 32-bit number (01000000111010011010011101100011) Notice, if you failed to pad the last number with a zero, the result would be only 31 bits, and the technique would fail.
Type this number into the calculator and convert it to decimal. This should give you a decimal result of 1208930147.
In your browser, type http://1089054563 and hit enter.
Notice that the Google search engine appears.

Not mine.

Note: Make sure that you pad all >8 digit numbers with 0's (at the front; thus 10010 needs to become 00010010)

SRL Forums: 72.249.29.114 / 1224285554

Bourdou!
07-27-2007, 01:39 AM
Not mine.

Note: Make sure that you pad all >8 digit numbers with 0's (at the front; thus 10010 needs to become 00010010)

SRL Forums: 72.249.29.114 / 1224285554
Is there an online calc i could use for that?

palmpilot71
04-03-2008, 05:13 AM
Finnally I get it. My dad was like" I just push send!"

danhill
08-06-2008, 11:00 PM
here is a great 3d animation which puts it in a quite a good context:

http://www.warriorsofthe.net/movie.html

dweg
11-01-2008, 09:08 PM
nice explanation, thx i used it for writing report on the internet for my computer teacher.

enovy9
11-06-2008, 02:20 AM
thank you u answered some questions i had. nice tut

newber_dan
11-23-2008, 02:45 PM
wow, now i know a bit more bout the internet, thanks for the tut

Diego_Alonso
02-24-2009, 01:57 PM
Good job there are really good detailed information

lysolman00
02-25-2009, 04:55 PM
Interesting! I want to learn more about the Fiberoptics internet since we are getting it soon. I was wondering what it had to do with SCAR, but you said at the bottom of your guide.

achokshi
04-01-2009, 10:46 PM
This should go on like howstuffworks.com or something haha. Its very noob-friendly.