WPaudio MP3 Player Plugin is a great plugin which adds an inline audio player to links which you have given the wpaudio class to, or alternatively all MP3 links. Check out some Pegasus Christmas gems as an example.
FYI I had to edit the file
wp-includes/js/tinymce/themes/advanced/skins/wp_theme/content.css
adding a .wpaudio with a margin: 0px placeholder to get the theme to show in the class selector in the Insert/Update link box in WordPress.
For video (and for audio, but not as nicely as the plugin) you can use the embedded media tool in the kitchen sink editing toolbar.
