A binary number is a number expressed in the base-2 numeral system, which uses only two symbols: 0 and 1. Each digit is refered to as a bit. Due to it's simple implementation with logic gates, the binary system is used by almost all modern computers and computer-based devices.

Conversion

100 4    1000  8
 10 2     100  4
  1 1       1  1
----------------
111 7    1101 13

Break down the binary value in chunks of 4, multiply each 1 by it's equivalent value (8, 4, 2, or 1). Add the resulting numbers together to get the result. For example 1100 would be (8*1 + 4*1) or 12.

1   1  0  1  0 1 0 1 Binary
128 64 32 16 8 4 2 1 Values
128 64    16   4   1 = 213

Arithmetic

Addition

Binary addition is essentially the same as you learn in school for decimal. First line both numbers up (one under the other), then, starting at the far right, add each column, recording the result and possibly "carrying" the remainder:

Here are the possibilities:

0 + 0 = 0
1 + 0 = 1
1 + 1 = 2 which is 10 in binary which is 0 with a carry of 1
1 + 1 + 1 (carry) = 3 which is 11 in binary which is 1 with a carry of 1

Multiplication

Multiplication is about as easy as addition. It's similar to decimal multiplication, but easier since you're only dealing with 0 and 1.

We line the two numbers up (similar to addition). Then we multiply the entire top number by each individual digit of the bottom number. As we move across each digit we pad out the result with 0's to line it up. Finally we add all the results together.

 21    10101
  6      110*
     -------
       00000
      101010+
     1010100+
     -------
126  1111110

Power of 2

Binary numbers can be multiplied and divided by multiplies of 2, by rotating one bit left to multiply by 2, or one bit right to divide by 2. Similar to how to can multiply by 10 in base 10 by "adding a 0 to the end".

22     00010110
44 ROL 00101100
11 ROR 00001011

Alternative method

In the first column, divide the first number by 2 by removing the last bit, until 1 is reached. In the second column, multiply by 2 by adding an extra bit of 0. The answer is found by adding the numbers in the second column with odd numbers in the first column. A binary number ending with 1 is odd.

This example multiplies 35 by 19, to arrive at a result of 665.

100011 |      10011
 10001 |     100110
  1000 |    1001100
   100 |   10011000
    10 |  100110000
     1 | 1001100000
       | 1010011001

The result can be deconstructed at:
10 1001 1001 = 1 + 8 + 16 + 128 + 512 = 665

Subtraction

Similar to binary addition, you can work through, column by column, starting on the right. Instead of carrying forward however, we will borrow backwards (when necessary).

Here are the possibilities:

0 - 0 = 0
1 - 0 = 1
1 - 1 = 0
0 - 1 We can't, so we borrow 1 from the next column. 
      This makes it 10 - 1 which is 1.

I find it a bit confusing and prefer the two's complement method.

Two's Complement method

This is the way that computers subtract binary digits.

Let's say we want to compute 1000 (8) - 11 (3).

Step 1: Write the equation out, padding the bottom number with 0's

1000
0011 -

Step 2: Invert the digits of the lower number

1000
1100 -

Step 3: Add 1 to the lower number

1000
1101 -

Step 4: Add those two numbers to get 10101
Step 5: Remove the leading 1 (and any 0's after it). You're left with 101 (5)