Audacity is impressive, having both an intuitive interface and a host of features.
Audio can be imported in any of the following formats: WAV, AIFF, AU, IRCAM, MP3, OGG and MIDI.
Once imported the audio can be manipulated in all the ways you would expect, including cutting, copying, adding/removing silence, altering gain, and mixing tracks.
Various effects are available to spice up your recordings, ranging from simple amplification to wah-wah effects.
Multiple undo levels make a good base for experimentation, and the built-in amplitude envelope editor and spectrogram mode may prove useful.
It only has limited MIDI editing tools, but it does allow recording and editing audio in 16, 24 and 32-bit samples.
A selection of plugins are available, including LAME MP3 Encoder (allows Audacity to export to MP3 format), and VST Enabler (for loading VST plugins).
There are numerous changes and fixes in Audacity 2.0. For more information, view the release notes.