They are called Favicon's and now i'll show you how to make one..
Make an image in your favorite dtp, and resize it to 16 x 16 pixels
Now Save it as bmp and name it favicon (File>Save As name it favicon and choose bmp as file format).
We're done with the graphical part of your favicon.
Now open your ftp client and upload it to your server. Once uploaded rename it from favicon.bmp to favicon.ico
To have it load into your website you just have to add this html line to your pages: (assuming you stored it into the images folder on your server)
Code:
<link rel="shortcut icon" href="images/favicon.ico" />
Thats it it really is that simple
Enjoy