There are 2 ways to do this under Windows...

(1) From the Start Menu, take the Run option and type in cmd or command (under some versions of Windows cmd works, command shouldwork under all versions). Once you get a command window, type hostname, the output is the computer name.

An alternative method

(2) From the Start Menu, take the Control Panel option. Click the System icon and then click the Computer Name tab. You should see the name of your computer displayed. You can also change your computer name from this dialog - don't do that unless you are sure you want to. Once you have made a not of the computer name, click the Cancel button

As an additonal check, you can download, unzip and run this simple test program that will report your Windows computer name: Computer Name Test Utility
 

Under Linux...