How do I recover lost/inaccessible data from my storage device?

In case of mechanical failures.

Pray, it will help you and calm you down. :-)

Introduction

If you have a mechanical failure (e.g. random crashes, just stops working one day, weird "screeching"/"beeping" type noises), EVERY time you plug it in and turn on, you could be making it much worse. If it is very important data, I would recommend taking it to a lab / professional data recovery service.

However, if you want to do it yourself, you can summarise mechanical failure into two categories:

Spindle/inside problems or outside/controller problems.

First, Spindle/inside problems. This is the worst thing that can happen to a physical hard drive, If it is this, it really depends how bad. My favourite tool for this (not free) is R-Studio, it allows you to create an image from the drive and performs many passes*, then perform the recovery from the image.

* (I have actually had drives fail a read from a sector, but just from trying over and over, it has worked - even unplugging and plugging it back in)

Depending on how important the data is, and if you are not able to read, I would try the freezer trick. This sounds like a joke but it really is not. Put the drive in an air tight bag and stick it in the freezer for a good few hours (I usually leave it in for 6 hours or overnight), then, when you plug it in, you can get a good 15-30 minutes before it crashes again.

If it is a controller board problem then, the only way to fix it is to re-flash the drive (check manufacturers website), or most commonly, switch the controller board (Carefully) from one that is an identical model.

For flash drives, again, if important, go to a lab. If you want to do it yourself, there isn't really a lot to say.

Is the error with the controller or the memory?

Typically, if it is the controller, when you plug in the drive, nothing will happen. If it is the flash memory itself, it sort of acts like a floppy/CD drive without the media in - you can see a drive letter, but just can't access it (sometimes get the insert media warning).

If it is the memory itself, I do not know a fix.

If however it is the controller, I have only ever had luck about 40% of the time doing it myself(and it depends on the architecture of the stick). Many of the cheaper sticks you see have two boards - one is the controller and the second (not sure the technical term), is a snap on daughter board. You can usually just unplug the memory and plug it in to another board.

Usually the board doesn't even have to be from a similar drive, just try to get the chip provider correct (e.g. branding on one of the ICs), the most common one I see is from Winbond and they typically work with any memory chip.

In case of corruption or bad sectors.

Pray, it will help as it calms you down. :-)

Get direct access to the data.

When recovering files from an external drive it's important to have the shortest connection possible.

This means that you want to get rid of any extra USB cables, USB hubs or equipment you don't need.

If you are recovering from an external hard drive, try to get it out and connect it using a SATA cable...
If you are recovering from an USB stick, try to connect it to the back of your computer, try different ports.

Downloading and burn an Ultimate Boot CD needed for further steps.

Most tools used in this post are all available on the Ultimate Boot CD.

Download the Ultimate Boot CD at the bottom of this page: Click on the icon next to a mirror.
Optionally, to ensure quality, run a checksum with this program against the checksum listed here.
Burn the ISO to a CD using ImgBurn on Windows, LiquidCD on Mac OSX or Brasero on Linux.
Optionally, to ensure quality, make sure it verifies the CD.

Take a backup (EASEUS Disk Copy).

As we'll try to recover the file system and/or recover the data we are going to tamper with the disk, for this reason you might want to take a preliminary back-up to ensure that if things go wrong you still have a back-up available. If you suspect disk failure you might even want to consider to exercise the back-up instead so you can still send your hard drive to forensics companies if you really need the data...

Start the Ultimate Boot CD.
Go to HDD --> Cloning Tools --> EASEUS Disk Copy.
Do a disk copy to another device that has enough space free.

This will copy the data exactly at a sector-by-sector level.

Check if a hard drive is still in a fine state (SMARTUDM).

Before we tamper with the drive we want to be sure we aren't making its state worse, so let's first check the state:

Start the Ultimate Boot CD.
Go to HDD --> Device Management Tools --> SMARTUDM.
Check if any of the S.M.A.R.T. attributes has a ***** that is in yellow or red, this denotes a bad state.

If the state isn't fine, try to recover in case of mechanical issues.

If the state is fine, then we'll do an error scan to be aware and get rid of issues:

Start the Ultimate Boot CD.
Go to HDD --> Diagnostic Tools --> ViVARD.
Let it perform an error scan, note how much errors are found and how many remaps are done.

Identify the file system.

Covered by How do I identify the file system used on a partition?.

Try to repair (TestDisk).

Prior to doing the actual recovery, you might sometimes have the need to repair the partition(s) and file system(s) first. This is where TestDisk comes into play, I would recommend to take a look at what it does.

This is how to get to it:

Start the Ultimate Boot CD.
Go to HDD --> Data Recovery Tools --> TestDisk.
Read the documentation at the bottom of this page and try to repair your data.

Use recovery software (PhotoRec).

Now that the preliminary stuff has been done, this is how you can start recovering:

Start the Ultimate Boot CD.
Go to HDD --> Data Recovery Tools --> PhotoRec.
Read the documentation at the bottom of this page (example: step by step) and recover your data.

TestDisk is a free open source partition scanner and data recovery tool. It is very useful in recovering lost partitions. PhotoRec is another free commonly used data recovery tool. TestDisk and PhotoRec in addition to being included in the Ultimate Boot CD, as Tom Wijsman mentioned in his answer, are also included in the software repositories of many Linux distributions and on the System Rescue CD. System Rescue CD is similar to Ultimate Boot CD, but it is more lightweight, which is an advantage because it is normally run from a CD or a USB flash drive where performance is important.

TestDisk is a lot more efficient than PhotoRec. The problem with Testdisk is that it doesn't always recover all deleted files. If you accidentally reformat a partition, TestDisk can recover thousands of files without missing a single file, but if you deleted a file by sending it to the Trash and then emptying the Trash, TestDisk can't always recover it.

So use TestDisk first, and if you recovered all of the deleted files with TestDisk, then you're done. If you recovered most of the deleted files with TestDisk, you can decide whether you're done or not. If you're not done after running TestDisk, you can try recovering the deleted files using PhotoRec.

PhotoRec can't recover deleted files that have been completely overwritten (for example, with the dd program). In some cases, the filename is stored in the file itself. PhotoRec tries to recover the filename in this case, but most of the time PhotoRec can't recover the filenames.

Recover files based on filetype using PhotoRec

It is preferable to boot from a Linux live DVD/USB before following these steps, in order to avoid using the operating system in which the deleted file is located.

Install TestDisk if it is not already installed in your OS. In Linux distributions, installing TestDisk will also install PhotoRec along with it.
Open a terminal and launch PhotoRec (launch from a terminal in a live CD/USB or launch as root).
Select hard disk.
Select partition type.

If your hard disk has Linux partitions, then select [Intel].
Select filetype option.

Move to [File Opt] and press Enter. Here you can disable all file types by pressing s. Use space to toggle the check button. Select filetype(s) to recover.
Select options.

PhotoRec also has a list of different options. Under normal circumstances you don't need to modify them.
Select partition.

Move the selector to the partition from which you have removed the file. Then press Enter on [Search].
Select filesystem type.

If you are using Linux, it's going to be ext2/ext3/ext4, so the default selection is ext2/ext3. Otherwise if you are recovering files from a partition formatted as FAT or NTFS select Other.
Select space for analysis.

Select Free if you didn't write to that partition after removing the particular file, otherwise select Whole.
Select a directory to recover files.

Now select the path where the recovered files will be stored. Then press Y.

PhotoRec will show how many files it has recovered.

_{Source: revised from How To Recover Deleted Files in Linux Using PhotoRec}

Another tool that could be very helpful is Foremost. The name of the tool is a dictionary word that might contribute to its relative obscurity as it can never get the first position in web search for its own name.

Foremost

Foremost is a Linux program to recover files based on their headers and footers. The headers and footers are specified by a simple configuration file, so you can choose which headers you want to look for.

The tool has a prominent origin in the work of the United States Air Force Office of Special Investigations and The Center for Information Systems Security Studies and Research.

It has recently saved me a day when PhotoRec was unable to find any traces of sound files (ordinary RIFF WAVE format) on an FAT-formatted SD card with corrupt root directory entry.

Usage

The usage is very simple:

install foremost from your distribution repository if available or compile foremost from source. Compiling works like a charm as the tool has very few external dependencies
run foremost on the device or device image, e.g.

foremost -i /dev/hda -t wav -o /recovery/foremost
examine the /recovery/foremost directory for useful files.

Remarks

Foremost does not modify the original device or device image, so it is usually safe to run it on the original device unless the device is failing. To be on the safe side, always have a backup dump.

Foremost is available for Ubuntu and is in fact the first choice for extracting individual files from a corrupt image.