Is there a way to configure the keyboard on an iPad

I just got an iPad 2 and I find the keyboard difficult to use, because the keys always show as caps.

Is there a way to modify the keyboard to display lower or upper depending on what the what the case of the next typed character will be?

Or better, is there a way to change the keyboard to a better one, perhaps with numeric keys accessible from the main keyboard?

Unfortunately what you're looking for is unavailable as of iOS 5.1.

The letter keys you type are lower case unless the shift key is blue. This does match a physical keyboard so hopefully you can get past that lack of displaying exactly what you get.

From a design standpoint, each app can add to the keyboard or replace the keyboard entirely, but you can't load a different keyboard if you don't like the ones that are delivered from iOS.

One example of an augmented keyboard is the Prompt ssh application by Panic.

One modification you can do to the keyboard is to "pull the keyboard apart". Using two fingers start in the center of the keyboard and slide your fingers towards the side of the iPad as if you were stretching the keyboard apart.

Also, if you're interested in editing in a language other then your device's default you can go to Settings -> General -> International then select one of several dozen languages available including Hebrew, per your comment. Once you've selected an alternate language you can tap the international icon to cycle through your selected languages as noted in the image below.

There are various apps (search for "keyboard") which provide alternative keyboards that may be useful for certain purposes. Killibee Type is one. The apps Unicode Map will let you create a sort of custom keyboard.